Breach Management: How to Respond to Vendor Data Breach

Unfortunately, vendor management alone will not solve the problem. In fact, I am hard-pressed to recall a case where vendor management would have prevented the inevitable. That is not to say that good vendor management is not necessary: it is. However, ensuring good defense-in-depth security through vendor relationships is a lot more than just vendor management.

 

Vendor Breaches

Not all vendors are created equal. Some vendors have direct access to your network as some form of a managed service provider. Others provide various cloud services that store or process information for you. The following is a list of recent vendor data breaches:

• Hafnium (2021). Microsoft and other security firms have been shouting from the mountain tops about targeted attacks against on-premises, hosted, or hybrid (combination Exchange and Office 365) Exchange servers. These attacks take advantage of four security vulnerabilities in Exchange, leading to full email and server compromise, as well as potentially taking additional malicious actions, including ransomware. The primary threat actor involved is a new Chinese state-sponsored hacking group known as Hafnium. However, at least five additional threat actors are actively exploiting these same vulnerabilities. Cybersecurity journalist Brian Krebs initially reported the compromise as affecting “at least 30,000” US-based organizations. That number has since grown to at least 60,000 US-based organizations compromised as of this writing.
• SolarWinds (2020). Who can forget SolarWinds? Prior to the above-mentioned Hafnium, this breach was all the rage. The actual breach occurred way back in 2019. Attackers accessed the code of SolarWinds’s Orion software and injected malicious code. Updates including this malicious code were then sent out to customers using Orion, providing a backdoor into roughly 18,000 customer networks and giving the hackers access to Orion customer networks and data exfiltration. Notably affected agencies include the Department of Homeland Security, the Treasury Department, and elite cybersecurity firm FireEye as well as several Fortune 500 companies. SolarWinds remains one of the most effective cyber espionage campaigns of all time.
• American Bank Systems (2020). This breach involved Avaddon ransomware and network compromise. American Banking Systems provides document management and compliance services for financial institutions. Avaddon leaked over 50GB of data, including loan documents, contracts, private emails, invoices, credentials for network shares, and confidential company files.
• Health Share of Oregon (2020). Health Share of Oregon is the largest Medicaid coordinated care organization (CCO) in Oregon. The company’s medical transportation vendor had a laptop stolen that contained personally identifiable information of 650,000 members, including names, addresses, phone numbers, birth dates, and social security numbers.

 

Components of a Modern Vendor Management Program

Although vendor management isn’t a silver bullet to preventing vendor data breaches, it’s a necessary component to a healthy overall information security program. We’re going to continue utilizing vendor relationships, so truly managing our vendors remains extremely important. A good vendor management program contains the following components: risk assessment, due diligence, contract review, and the watch list.

Vendor Risk Assessment

Everything that’s good in information security starts with a risk assessment. If you cannot measure it, you cannot manage it. In fact, results of the final two components (due diligence and contract review) must also provide input into the vendor risk assessment for it to be meaningful, but we’ll get to that shortly. All of your risk assessments, including your vendor risk assessment, should help you make better decisions. Ultimately, you’re seeking answers to two questions: Who do I want to do business with? Do I want to continue doing business with this vendor? You should seek to quantify the answers to these questions.

Vendor risk assessment starts with a Protection Profile. Put quite simply, a Protection Profile provides an initial assessment of importance. A Protection Profile can also identify vendor criticality classifications such as critical, important, and incidental (level 1, level 2, level 3). The criticality of the vendor is based on its Protection Profile and should identify the depth and frequency of the review performed on each vendor. This will allow you to focus on what’s really important, as noted in Figure 1 below.

Vendor risk assessments must also identify threats. The FFIEC identifies five threat categories: operational, compliance, strategic, reputational, and credit. Combine values assigned to each of these categories with your Protection Profile, and you’ll have an inherent risk value. Residual risk is then quantified by taking away from that inherent risk through the answering of review criteria questions for both due diligence and contract review.

 

Figure 1: Vendor Criticality

 

Vendor Management Requirements

Now that you’ve identified criticality, what are your review requirements? You’ll want to spend 80% of your time on roughly the top 20% of your vendors. Therefore, review depth and frequency are higher the more critical the vendor relationship. So are the documents you’ll look to collect to validate the goodness of those reviews. Examples of good vendor management requirements are noted in Figure 2 below.

 

Figure 2: Vendor Management Requirements

Due Diligence and Contract Review

This is where we’ll get the majority of our data to ensure the goodness of a vendor relationship. It’s also the most time-consuming and potentially daunting task of vendor management that frightens people and stops them from making meaningful progress. Let’s be honest, due diligence and contract review is tedious work, it takes a lot of time, and it doesn’t always feel like it’s providing adequate value. Nevertheless, it is necessary, and it need not be that daunting. If you have a good risk assessment as discussed above, you’ll know where to focus most of your energy. If you’ve identified what you want to include in a review for each specific level, you’re well on your way to having an effective vendor management program. The next step is to identify review criteria.

Luckily your primary federal regulator provides you with good starting points. The FDIC, OCC, FRB, and NCUA all provide their own general criteria for due diligence and contract review. We would encourage you to go further though, as much of their guidance is generally dated by today’s standards. We would encourage looking into developing your own question sets for things like SOC reports, cloud providers, and foreign-based service providers, to name a few. Remember, the more critical the vendor, the deeper dive into the review you should do. Also, remember to ensure you quantify your answers to the review questions in a way that allows you to demonstrate the difference between inherent and residual risk.

The Watch List

Now that you’ve performed your vendor reviews and have quantified inherent and residual risk, what do you do with those values? What if a particular vendor relationship doesn’t live up to our expectations? After all, we’re seeking to answer the question: Do I want to continue doing business with this vendor? Occasionally a vendor review doesn’t live up to our expectations or risk appetites. A vendor not meeting expectations can be due to the vendor providing outdated or insufficient documentation based on our review requirements. It could be that upon review of those documents, we found troubling items that resulted in less risk reduction than we would have liked. If that’s the case, we are now presented with the choice of whether or not to continue doing business with the vendor. Assuming the decision is made to continue the relationship, the vendor in question should be placed on a “watch list.” Your vendor watch list should mimic your loan watch list. It identified problematic vendors that require additional oversight. If a vendor is on the watch list, increase their review frequency identified by the risk assessment until such a time that you’ve decided to either:

1. Accept the risk = Do nothing, but make sure you document it as a known risk exception!
2. Resolve the risk = Work with the vendor to address issues until they’re resolved.
3. Change the risk = Find a new vendor or bring the service in-house.
4. Transfer the risk = Insure against a loss.

Vendor Management + Incident Response

If you’ve stuck around this long, you’ll recall I mentioned in the introduction that good vendor management alone wouldn’t fix the problem of a vendor data breach, and it won’t. That’s not to say a good vendor management program isn’t an integral component of your information security program, it is. But we must plan to fail well. Assume the worst is inevitable. How can we prepare for the worst in order to ensure the best outcome?

Include Vendor Compromise as a Threat

Your incident response plan should already identify the most severe threats your institution faces. Vendor compromise ought to be one of those. For a threat assessment example, see Figure 3 below. Hint: Your incident response plan, in order to be comprehensive, should have a detailed response scenario for each item listed in the threat assessment. This can be based on what your vendors have access to. What IT assets do they provide? Where do those IT assets reside? Are they on your network directly or hosted by the vendor themselves?
 

Figure 3: Threat Assessment

Perform Tests

You should perform tabletop testing of your incident response plan. Develop scenarios based on your threat assessment and walk through those scenarios with the incident response team. When performing a tabletop test for a vendor compromise scenario, reach out to the vendor prior to the test and encourage their direct participation. If the vendor chooses not to participate (note that in your vendor management program), put together a list of questions or requested information resulting from the tabletop test. Don’t forget to ensure your tabletop test is well documented: Who attended? What was the scenario? What steps were determined to be taken? What did we do well? What can we improve? What additional questions do we need to answer?

Digital Forensics and Incident Response

Assuming the worst does happen, there are two questions you need to ask:

1. If an unauthorized party was in my network, how would I know?
2. If an unauthorized party was sending information out the back door, would I be able to tell?

If you aren’t certain you can answer those questions or even find the answer after an incident has occurred, get started now. The best time to start logging is yesterday, the next best time is now. In the event something does happen, you’ll need to do digital forensics in order to determine the depth and severity of the breach. Digital forensics seeks to determine the: who, what, when, where, why, and how of a data breach. Without good logs, that process becomes next to impossible.

 

Controls to Reduce Vendor Breach Risk

Modern vendor management requires a modern approach to controlling risk. The following controls, when implemented properly, will reduce a significant amount of risk:

1. Multi-Factor Authentication (MFA). I cannot stress this one enough. Use it wherever possible but must be on all internet-facing apps. It is true that had MFA been in place it would have likely prevented 100% of data breaches that our incident response team has investigated.
2. Strong Password Requirements. Even with MFA in place, a strong password is still a must. Why give a potential attacker one of the pieces of the puzzle for free? Also, MFA isn’t always feasible on all applications.
3. Religious Patch Management. This one mostly speaks for itself.
4. Weapons-Grade Data Backups. Minimally utilized the 3-2-1 data backup methodology. Keeping three (3) copies of your data, on two (2) different forms of media, and one (1) of those copies being off-site.
5. Network Segmentation. The greater the segmentation, the harder it is for an attacker (or malware) to move throughout your network.
6. Egress Firewall Filtering. Firewalls, by default, block everything coming in and permit everything to go out. You gain great control over what resources your internal systems can access when egress filtering is enabled.

Additional tips to help control risk include:

• Make sure you’re logging the right activity and that you’ve established a baseline on your network. Anything outside of the baseline could be an indicator of compromise. Ensure you have some type of central logging capability. Central logging capability is not SIEM. This is a place for you to store your collected logs. Make sure this system is a bastion host. Its data may be key in an investigation.
• Have separate user accounts. The ultimate rule is that one user means one account and allows for accountability. All users should be restricted users, especially vendors. If a user is also an administrator, ensure they have a separate, privileged account to perform those administrative tasks. Ensure no one uses service accounts. Service accounts are often administrative in nature. Ensure each service that needs a service account has its own service account, similar to how individual users have their own user accounts. Remember, it’s about accountability.
• Develop yourself or familiarize yourself with an incident response preparedness checklist. The list should highlight what organizations must have in place ahead of time to ensure both the ability to respond to an incident quickly and perform a digital forensics investigation should the need arise. Use the 50+ Incident Response Preparedness Checklist Items blog as a primer to get your list started.
• Get cybersecurity insurance. If you haven’t already gotten it, please look into it. Unfortunately, it is a tricky subject since there is no standard. If you haven’t gone down the path of obtaining cybersecurity insurance, ensure you understand the following: What is really covered? What do insurance companies expect from your cybersecurity controls being paying the claim? Does your coverage include incident response and digital forensics costs? Ask the questions, look into the options. The insurance companies are more than willing to help.
• Familiarize yourself with legal, and not just your own legal counsel, but law enforcement. Understand what their capabilities are and what they can provide in an event. Engaging your legal team can help protect your organization, especially if an investigation is needed, and it normally is. Ensure you run all communications through your legal team. Engage with law enforcement. Legal counsel and law enforcement will work with you to determine if and when to notify customers.

 

Bottom Line, It’s Your Data

We’re more reliant on vendors than ever before. Vendors are storing, processing, or transmitting data on behalf of you. However, it’s your data, so it’s your responsibility to protect your customer information, your employees, and your institution no matter where the data resides. Your vendor is not going to notify your customers about a breach for you or take the blame. Understand your vendor’s security practices through your vendor management program. Align those vendor relationships with your cybersecurity goals and standards. Ensure you’re tying vendor management and vendor relationships into your incident response considerations. Remember, it’s not IF something is going to happen, it’s WHEN. If we plan to fail well and build that capability into our vendor management process, we set ourselves up better to come out the other side as healthy as possible.