The Breach – Part 4

And the Story picks up where we left off here.

Date of Discovery

All along the way, third-party companies were lining up with bags opened, hoping to get them filled with “incident response” money.  Most of them were trying to create some real sense of urgency to engage and take action.   I never fell for the high-pressure tactics; I wanted to get some options, evaluate the risks, and make an informed decision.  However, in the days that followed, the attorneys started educating us in the generalities of cyber statutes.  Most of them had a requirement to respond with their timelines after the “date of discovery.”  This was the only timeline that mattered.  So, we had time, but much, to rationally engage third-party forensics to a specific scope of the data breach so that we could formulate a responsible response plan.

250 Records Lost – What A Relief

After sending the forensic team copies of the hard drives from machines in scope, we had many calls with the team. After their investigation, they thought we could have lost about 250 records of personal information.  At first, it was a relief, as it could have been much worse.  But shortly after, I started to realize that their analysis was speaking in hypotheticals.  The words beings used were “we think,” “they might have,” “it possible,” rather than more explicit language.  So, I started to understand that they didn’t know, but the forensics team advised me on the potential risk of what was accessible to the cyber attackers.  After this call, we had a pretty good idea about the size and scope of the attack.  At this point, I was finally able to provide some tangible details to the stakeholders of the company.  Until now, the information I had was all hypothetical; now, we had some excellent news to act on. 

The First Invoice — Yikes

A few weeks into this saga, the Company received its first invoice from the Data Breach Coach (attorneys) and the forensics company.  Let’s back up. We engaged this firm because we had access to them through our insurance coverage.  And, we received about 60% more financial relief from all the expenses if we used the insurance companies’ providers rather than just find our resources.  So, naturally, we engaged the providers recommended by our insurance company.  But, there was a limit to what the coverage would allow for. At first, I thought the coverage was way more than enough until the first invoice arrived.  The hourly billable rates were so high, and I felt they misplaced a decimal point.  Our first invoice ate up around 35% of our allowed coverage.  I needed to make sure the Company was using its coverage limit wisely.  The Data Breach Coach and forensics team afforded plenty of opportunities to do work and eat up our remaining coverage. 

The Response

After the scope of the event had been determined, it was time to work on the response.  The legal team briefed us on the types of actions that we needed to take.  First, we needed to determine what states the impacted parties were residing in.  Each state has its reporting requirements when a data breach involves personally identifiable information.  Some states even require credit monitoring and other services to protect individuals from identity theft.* I quickly realized that the attorneys understood these requirements and would be an extremely valuable resource.  They took on crafting notification letters to 15+ states on our behalf and having guidance that his juncture of the story was very comforting.  I understood our risk and felt comfortable we were doing the right things.

*note — These requirements are now commonplace in most states, but during this time, these requirements were not the norm

Our MSP Should Know How To Help Us Remediate… Not So Fast

At the same time our talks with the attorney and forensics, the Company was actively trying to remediate the root cause of the cyber attack.  We were confident we had stopped the attack.  But, verifying we had no lingering effects of our attack proved harder than we thought.  We reach out to our MSP for advice and assistance.  We planned a weekend to come in, install advanced cleaning tools, and clean every machine in the building (over 100).  We only completed the task to find the infected device (that was cleaned) with some unrelated malware. We scheduled another weekend to re-clean all the machines again.  After the 2nd round of cleansing, the malware was found again.  It was then I realized our MSP was not equipped to handle our situation.  To date, they have served us well, but the problem overmatches us.  In summary, they were not security experts, so we needed additional support.   

The moral of this story, Managed Service Providers, are good at traditional things, procurement of new hardware, architecting new infrastructure, end-user support.. etc.  But, cyber security experts were not.  Through lots of activity and a few 3rd parties, we could get our environment clean and remove any remnants of the attackers.

Final Part 5 is coming soon.

About the Author:

This blog was written by John Hensberger, Managing Partner of Black Bottle IT.  Earlier in his career, John was also part of a company that experienced a cybersecurity breach. That experience fueled his passion for assisting other companies with their cybersecurity needs to mitigate their risk. As Technology Executive and Cybersecurity Advisor, John was recognized as the Pittsburgh CIO of the Year, 2014.  Connect with John here.