The Breach: Final Chapter

by John Hensberger | Oct 25, 2021 | Breach Response, Cybersecurity

The Breach: Final Chapter

Why Does The Attorney General of North Carolina Care About Our Data Breach?

Our Data Breach Coaches had filed the necessary communications with several state Attorney Generals. We were advised that most states would acknowledge, and it would end there. However, the Attorney General’s Office in North Carolina reached out and requested more information. This was surprising to me, as I didn’t know how to respond or why they would be asking for more details. Fortunately, our Data Breach Coach fielded the request, and we had a call to discuss the strategy of how to respond. Our response was most reassuring that we had taken the necessary measures to prevent an issue like this from happening again. This was just another point, where had we not engaged the right people, we would have had no idea the reporting requirement existed and/or how to respond to requests for more information. As I learned, each state has different reporting requirements, and North Carolina was not the last state to request more information.

Credit Monitoring, Makes Sense

After two months, we were rounding the home stretch of our incident response. The final piece was to figure out how to protect the impacted individuals from any potential damages. Our Data Breach Coach asked us how we wanted to proceed. (Let’s pause for a second, this is a common theme throughout the process. At various points, third parties who had expertise in a subject matter would ask us our thoughts on how we’d like to proceed. We wanted guidance from the experts in areas we needed; I thought it strange we were asked this, rather than just presented options, back to the story) Obviously, we didn’t know how to respond, so we asked for options.

As a sign of “goodwill,” they suggested offering everyone credit monitoring for two years. That seemed reasonable to us. So they referred us to a national brand that provides these services. During our first call, they offered credit monitoring, explained the process. All of which was acceptable, but then they started talking about setting up a call center, handling questions by the impacted individuals, printing notification letters, and lots of other services. All of which sounded like a great idea had the scope of our incident warranted. However, our situation was so small that it seemed like overkill and yet another way to spend the Company’s money needlessly. So, we set up the necessary processes with internal staff, contracted to offer the credit monitored, and sent out the notification letters. Materially, the incident response was effectively over.

You Want A Detailed Recap Of The Breach Response. Are You Sure?

Finally, after all notification requirements were handled, credit monitoring handed out, and all inquiries were answered, we had a chance to exhale a bit. The incident could have been much, much worse. But we’d made it through. After thinking about it, I realized I didn’t have a formal recap of our response from our Data Breach Coach. I thought, should any inquiries be going forward about how we responded, I should have something to prove that we satisfied all the legal requirements and acted ethically. So, I launched a call to our Data Breach Coach. Out of everything we experienced, this final chapter is something that surprised me the most. After a few days, the Data Breach called me back and acknowledged my request.

Here is our dialog:

Data Breach Coach: What do you want a formal recap of all the activity that occurred on your behalf in responding to this incident?

Me: “Yes, I do; I want to be able to produce some formal documentation of how we responded to our incident in the event it is needed in the future.”

Data Breach Coach: “Are you sure you want this?”

Me: “Yes, I’m sure.”

Data Breach Coach: “Are you really sure?”

Me: “Rather than ask me if I really want it, can you just tell me why you are asking if I want it?”

Data Breach Coach: “If we produced it, it becomes a legally discoverable document, which might not be in your best interest, should future legal action be taken against you and your Company.”

Me: “So, let me get this straight, we spent three months and 250K responding to this incident. We dotted every ‘i’ and crossed every ‘t.’, But it’s not in our best interest to have a formal recap of the actions taken, to prove that we handled the incident in accordance with all the requirements?”

Data Breach Coach: “We can add a high-level summary to your file, should you need it, you can call our office, and we’ll produce it, but having a detailed document isn’t in your best interest.”

Me: “Ok, if that’s what you think is best, then I guess I don’t want it.”

This last interaction left me a little dumbfounded. But in hindsight, I should have expected it. The whole experience was foreign to the Company and me. In the end, we spent our entire amount afforded to use by our insurance coverage to respond to this incident for the potential loss of 250 records. During the initial days, there was lots of confusion, unknowns, and decisions that needed to be made. We didn’t always make the right ones, but we did have enough presence of mind to slow the situation down, get as data as we could, and make informed decisions.

I’m hopeful that our story can help, inform or at least mildly entertain anyone reading.

About the Author:

This blog was written by John Hensberger, Managing Partner of Black Bottle IT. Earlier in his career, John was also part of a company that experienced a cybersecurity breach. That experience fueled his passion for assisting other companies with their cybersecurity needs to mitigate their risk. As Technology Executive and Cybersecurity Advisor, John was recognized as the Pittsburgh CIO of the Year, 2014. Connect with John here.

The Breach: Part 3

by John Hensberger | Oct 18, 2021 | Breach Response, Cybersecurity

The Breach – Part 3

In “The Breach – Part I” and “The Breach – Part 2 ”, Black Bottle IT shares a true story of cybercrime that took place with a manufacturing client and how it unfolded.

“The Breach” continues…

Engaging the Data Breach Coach

It turns out; Data Breach Coaches are law firms that specialize in cyber statutes. It finally felt good to be talking to someone who I thought would guide us through our options. Their role, as it was explained to me, was to quarterback the breach response. They would:

Take the lead and run point
Advise our team on what to do next
Bring in third-party expertise, if necessary, to help us mitigate any legal risk.

This information all sounded great. My first question was, “Where do we start?”

A formal engagement letter was emailed to me shortly after our call ended. I read it and found some language I didn’t like, as with most legal documents. Most things I could live with, except the clause that says, “Data Breach Coach, acting in the best interest of the client, can engage and agree to terms with third party services, with all financial responsibility passed on the client, without client consent.” It says the law firm can spend company money without anyone consenting or agreeing. The second call we would have was to review the engagement letter. When I questioned this language, “only to be told,” the response was, “It’s standard language, for your own good so that we can move quickly.” I had this line removed from the agreement and officially engaged the Data Breach Coach.

Forensic Swat Team

One of the first things the Data Breach Coach advised was to engage a third-party computer forensic company to determine what data was lost when it was lost, etc. We had a call, and the person leading the conversation had a super high sense of urgency. He wanted to send 2-3 forensics technicians on-site the next day. I paused and said, “How much is that going to cost?”

Our insurance coverage was up to 250K of expenses. Anything over was coming from the company coffers. We were a small company; having thousands of dollars in fees would put the company in a vulnerable financial state. So, I was always trying to walk the line of protecting the company, doing our duty to investigate, and being very mindful of the financial situation.

Back to the story – The forensics company said we didn’t have time to wait, which was the best option. I countered with, can’t we gather information with our staff and send you what you need? Let’s start there, make a list, and we’ll begin compiling.

Date of Discovery

All along the way, third-party companies were lining up with their bags wide open, hoping to get them filled with “incident response” money. Most of them were trying to create some real sense of urgency to engage and take action. I never fell for the high-pressure tactics; I wanted to get some options, evaluate the risks, and make an informed decision. However, in the days that followed, the attorneys started educating us in cyber statutes’ generalities. Most of them had a requirement to respond with their timelines after the “date of discovery.” This timeframe was the only timeline that mattered. We had time to rationally engage third-party forensics to identify the scope of the data breach to formulate a responsible response plan.

Continue to Part 4

About the Author:

This blog was written by John Hensberger, Managing Partner of Black Bottle IT. Earlier in his career, John was also part of a company that experienced a cybersecurity breach. That experience fueled his passion for assisting other companies with their cybersecurity needs to mitigate their risk. As Technology Executive and Cybersecurity Advisor, John was recognized as the Pittsburgh CIO of the Year, 2014. Connect with John here.

The Breach: Part 2

by John Hensberger | Oct 4, 2021 | Breach Response, Cyber Insurance, Cybersecurity

In “The Breach – Part I,” Black Bottle IT shares a true story of cybercrime that took place with a manufacturing client and how it unfolded.

“The Breach” continues…

What Do We Do Know?

Immediately following, we start to take action to contain the incident. We immediately block the traffic in/out from the bad actor IP addresses. We look at some simple log data and verify that the Company has been transmitting data to these IP addresses — for as long as we have logged data over 30 days.

In the meantime, I’m getting SOS calls from the CEO, wanting an update from the FBI meeting and wanting to know, “How bad is this?” Since I don’t know the extent of the incident, I don’t have much, other than we think we’ve contained it. He asked good questions: “Don’t we have a firewall, AV, and a 3rd party that is supposed to help us with these things?” All of which the answer is yes. This then follows by the inevitable, “How could this happen?” Someone once told me that when you have bad news, the sooner it’s delivered, the better. So my message to the CEO was, we think we’ve contained it; now we need to figure out what the damage is, which is something that I need to figure out now.

The Response

So, let’s recap:

the FBI informs us that we have been “cyber-attacked” but offers no more information.
We think we’ve stopped it.
Our CEO is asking us questions about whether the company can survive this. And we have no good answers.

It was a pretty bad 24 hours. So, the real challenge for us was, what do we do next? First, we mobilized our third-party MSP to help. They scheduled time over that weekend to visit our location, install some advanced security tools, and “clean” every machine in the building. We didn’t know what had caused the breach, but this seemed like a good place to start.

Next, I got on the phone and started calling companies for advice/help. Most of these calls went something like this: “Yes, we’d love to help you; you need to determine what data was lost. So, you can buy blocks of time for 50K each. When do you want us to start?” Our Company was small, so spending money in chunks of 50K would also very quickly lead to the company’s financial demise. So, after calling and listening to multiple solutions, I had some options. In the background, Company stakeholders are asking me, “How bad is it?” “Are we out of business?”

Ah-Ha. We Have Insurance for That

At some point during the first 48 hours, the CFO had inquired to our insurance agent about having cyber coverage. It turns out we had coverage. She launched a call to our service representative, who then told us, we would receive a call within 72 hours with instructions. I was in the mindset that the company might not survive 72 hours. I didn’t wait around and continued searching to find some resource to help; that made sense. Then, I received a call from the insurance customer service rep. She told me that if I checked the policy, I would find instructions for utilizing the “Data Breach” services that our company was entitled to as part of our coverage. I thanked her, started reading, and found that we indeed had access to a “Data Breach Coach” or a list of them. I picked the first name and called them.

Continue to Part 3

About the Author:

This blog was written by John Hensberger, Managing Partner of Black Bottle IT. Earlier in his career, John was also part of a company that experienced a cybersecurity breach. That experience fueled his passion for assisting other companies with their cybersecurity needs to mitigate their risk. As Technology Executive and Cybersecurity Advisor, John was recognized as the Pittsburgh CIO of the Year, 2014. Connect with John here.

Employees Are The Largest Attack Vector

by blackbottledev | Aug 30, 2021 | Cybersecurity, Risk Management

By now, our inboxes, LinkedIn feeds, and websites of cybersecurity companies have all tried to tell you that Company employees are the most significant attack vector and pose the most considerable cybersecurity risk to all businesses. Those same companies advise firms to subscribe to some online training for “all your problems will be solved.” This advice, unfortunately, is not valid.

Since the early days of the westward expansion, fast-talking elixir salespeople have been peddling the magic potion that cures what ales you. As with most things, complicated problems demand a complex solution. This could not be more true when evaluating cybersecurity risks and putting together a strategy to lower those risks.

It’s true; cybersecurity awareness training does affect reducing employee-related cyber attacks. However, it’s only a piece of a larger strategy to improving a companies security posture.

Black Bottle IT advises clients to address six critical areas to tangibly lower cybersecurity risk.

Security Awareness Training – Online training, monthly newsletters, in-person training. These are all great ways to educate employees on the day-to-day threats. Education material needs to be delivered with more regularity, we recommend monthly.
Email Security – Email is the most common way employees get duped into giving credentials or cutting a check to the wrong payee. Email security alone just isn’t enough. A phishing AI engine that learns employee email habits can effectively flag and stop the excellent attackers from posing as an executive and social engineering an incident.
Security Operation Center — Having suspicious activity analyzed in almost real-time to detect unauthorized network access is critical to stopping/limiting a cyberattack before any real sensitive data is stolen. Some companies may have cyber tools to alert, but having the expertise to analyze alerts, determine if the threat is credible, and quickly determine the next steps is crucial to respond to an actual attack.
Ransomware Protection — Stopping a ransomware attack before it encrypts meaningful amounts of data is the best peace of mind a company could ask for. Bad actors will attack, employees will click on threatening emails, and ransomware will try to encrypt critical data.
Solid Back-Up Strategy – In the unfortunate event of ransomware attacks, having off-site, isolated back-ups is the only way to restore business operations and prevent a costly crypto payment from resuming operations.
Incident Response Planning — Knowing the who, what, where, when a cyber-attack is suspected saves valuable time when a cyber threat is supposed. Performing annual “fire drills” to simulate actions taken during a cyber attack will ensure a quick response and could potentially limit the damage during an actual incident.

Ok, so there are seven recommendations, but this one is outside our expertise. We’ve seen enough offer this advice:

7. Cyber Insurance — having a good cyber insurance policy can further reduce the financial risk of a cyber attack. Most companies with some kind of cyber insurance have no idea if the coverage is correct for their level of risk. Look to FifthWall Solutions for more information about access to the right insurance policy for your size of business and industry.

About the Author:

This blog was written by John Hensberger, Managing Partner of Black Bottle IT. Earlier in his career, John was also part of a company that experienced a cybersecurity breach. That experience fueled his passion for assisting other companies with their cybersecurity needs to mitigate their risk. As Technology Executive and Cybersecurity Advisor, John was recognized as the Pittsburgh CIO of the Year, 2014. Connect with John here.

Investing in Cyber Security is Risk Management

by blackbottledev | Aug 23, 2021 | Cybersecurity, Risk Management

When talking to IT leadership, one of the most common things we hear is how difficult it can be to justify the investment in cyber security to the stakeholders of the Company.

Let’s face it, Companies invest in technology and expect to see an ROI from operational efficiencies, increase market share, or launching of new products or services. Most experienced IT leaders understand this; hence, it’s hard to justify the effort to vet potential vendors and/or new technology, craft a proposal to justify the need, only to have the Company’s stakeholders look up and say — “Where’s the ROI on this investment?”.

Is there a Tangible ROI in Cybersecurity?

It’s true; there isn’t a tangible ROI that can be easily calculated. How to put a return on recovering from a fictitious – “it can’t happen to us” data breach scenario. Besides, having anti-virus/malware and a solid backup strategy is enough to protect us from a nasty virus or ransomware attack, right? And in the event there is an incident, the Company probably has done its fiduciary responsibility and secured a cyber insurance policy, in the unlikely event an employee would click on a bad email and allow bad actors access to the network and sensitive data. So, where’s the risk? Insurance will cover the financial burden.

The hard truth is, all IT Leaders know their Companies need better cyber security tools. But most of the time, it’s never part of their budgets, or it’s the first line item that gets trimmed during financial planning. And that is leaving lots and lots of companies exposed to lots and lots of risk.

Here are some statistics to show just some of the exposure Companies are facing:

43% of data breaches involved small businesses.
24% of data breaches are caused by human error
29.6% of companies will experience a data breach in the next two years.
The United States has the average cost of a data breach at $8.19 million
The average size of a data breach is 25,575 records
The average time to identify a security breach is 279 days
780,000 records are lost to hacking each day

So really, the conversation around cybersecurity is a risk management issue.

Can your company accept the risk of no liability or workers comp insurance?

A cyber insurance professional put it in these terms to me — “Just because you have Workers Comp insurance doesn’t mean you stop putting down salt on icy steps and sidewalks.” The same could be said for investing in cybersecurity solutions. Just because you have a cyber policy or some cyber security tools in place doesn’t mean you stop investing in new tools to lower your risk of a cyber attack.

What’s Next?

Our mission at Black Bottle IT is to help companies prepare, respond and remediate cyber incidents. We advise clients to invest in cybersecurity tools and services, and there is NO ROI. Still, your risk exposure will significantly be reduced, allowing the Company and its employees to continue serving their customers to the best of their ability.

About the Author:

This blog was written by John Hensberger, Managing Partner of Black Bottle IT. Earlier in his career, John was also part of a company that experienced a cybersecurity breach. That experience fueled his passion for assisting other companies with their cybersecurity needs to mitigate their risk. As a Technology Executive and Cybersecurity Advisor, John was recognized as the Pittsburgh CIO of the Year, 2014. Connect with John here.

5 Tips to Keep Technology at Schools Secure

by blackbottledev | Jul 26, 2021 | Cybersecurity

Did you know that the average business manages 162.9 TB of data? Because there’s so much sensitive information on an organization’s server, it’s essential to safeguard your server and applications.

Managed security services are the perfect way to ensure that malware stays out of your system. Here, we’re going to talk about what these solutions do and why they’re essential. Read on to begin protecting your business!

What Are Managed Security Services?

Managed security services are IT solutions that protect businesses from security threats. You subscribe to a package from a managed IT security services provider. In turn, this provider develops a unique plan based on your business’s individual situation to secure your digital data.

This process generally means implementing specialized software that integrates seamlessly with your data storage system. It also means building a security expert network that can respond to potential breaches in real-time. Experts, not just tools, make the difference in response time, reporting, and moving your business forward risk-free.

Because these security services are typically fully managed, you don’t need to worry about maintenance and upkeep. Managed security providers handle modifications and upgrades for you. They also implement the latest virus/spam blocking, intrusion detection, firewall, and VPN systems to keep you safe in an ever-changing digital world.

Why Are They Important?

First and foremost, you need high-quality security solutions to protect your business financially. A security breach costs a company an average of almost $9 million. If you’re an SMB, it’s unlikely that you have that kind of money.

Sensitive data theft also can cause untold problems for your employees and clients. You likely have employee SSNs stored somewhere within your system. Customer financial information is also often saved to a server.

When a company experience a data breach, they lose the trust of employees and clients. Not only that, but they will likely have severe concerns about identity theft and future financial problems. This problem isn’t something you want to be responsible for because it leaves you open to lawsuits, shutdown time, and customer loss.

Real-Time Network Security Monitoring: The Basics

Now that you understand the importance of data monitoring, you must choose the appropriate solutions. Real-time network monitoring is ideal for companies of all sizes.

Traditional monitoring uses analytics tools that only look into at-rest data. Since digital information is changing by the second, this becomes quickly outdated. Users won’t be able to notice new threats until it’s too late.

However, real-time network monitoring solutions like Black Bottle IT offers are different. They showcase only current data so that you can see what’s going on in your data system by the second. You, therefore, will get more relevant information and discover what needs to be done to safeguard your data in real-time.

Streamline Your Security Operations Today

Now that you know why managed services for security are the right choice for your organization, it’s time to get started. Schedule a free consultation to learn more about our managed security services.

Our experts are happy to discuss the best ways to keep your specific digital information secure by meeting your unique needs. We also will help you come up with a payment plan that works for you. Since we’re committed to helping your business succeed with secure and safe information, we look forward to hearing from you soon.