Taking a Deep Dive into Measuring the ROI on Cybersecurity

How do you measure the absence of a thing?

One of the perennial questions with regard to any IT investment you might make is: “what will the ROI or payback on this investment of funds be?”  It’s a devilish question, especially when you’re investing in something that will reduce the incidence of a thing occurring – in this case, reducing the risk of a cybersecurity issue. 

It turns out, you can do it in seven steps.

Step 1: You start by defining Objectives

Defining your cybersecurity objectives is the foundational step in measuring a Return On Investment (ROI). Setting clear, desired outcomes will help to later define a method for measuring key performance indicators that are related to cybersecurity.

Cybersecurity objectives should align with the broader business goals and digital security needs of the organisation. These objectives can vary widely, from reducing the number of breach incidents, improving detection times for cybersecurity threats, to enhancing the efficiency of response strategies following an incident. It’s crucial that these objectives are Specific, Measurable, Achievable, Relevant, and Time-bound (SMART), to ensure they provide a clear direction for the cybersecurity initiatives that will follow and allow for accurate measurement of their effectiveness.

For example, an objective could be to enhance system resilience against ransomware attacks by 30% over the next year, through implementing advanced threat detection systems and conducting regular staff training sessions. This goal is specific (enhancing resilience against ransomware), measurable (by 30%), achievable (with targeted security improvements and education), relevant (addressing a significant cybersecurity threat), and time-bound (over the next year).

Another important aspect of this step, if you can accommodate it, is understanding the current security posture of the organisation. This involves conducting a comprehensive risk assessment to identify vulnerabilities within your systems and processes. Such an assessment (typically using several different tools) will help in identifying and prioritising the cybersecurity objectives that are most critical to your organisation’s security.

Incorporating statistics and case studies relevant to the Australian cybersecurity landscape can provide additional context.  You may choose to set an objective that references external statistics and set an objective to outperform them.  For example, referencing the OAIC reports into notifiable breaches may provide you with an objective.  Or, you may choose to take a published average of the cost of a breach and use it as a key part of your ROI calculation.

Step 2: Break down objectives into KPIs

Identifying Key Performance Indicators (KPIs) is the second step in measuring the Return On Investment (ROI) for cybersecurity. KPIs serve as quantifiable metrics that directly reflect the effectiveness of your cybersecurity efforts against the objective(s) set in Step 1. Choosing the right KPIs enables your organisation to track progress, identify areas for improvement, and make informed decisions about future cybersecurity investments.

When it comes to cybersecurity, your KPIs should focus on aspects that offer tangible evidence of how security measures are performing.  Here are several examples:

1. Number of Detected Threats: This KPI tracks the volume of potential security threats identified by your cybersecurity systems. An increase in detected threats could indicate better detection capabilities, while a decrease might suggest either improved security or a need to update detection methods.

• Response Times to Incidents: Measures how quickly your team can respond to a detected cybersecurity incident. Faster response times can significantly reduce the impact of breaches, making this a critical KPI for assessing the agility and efficiency of your response strategy.

• System Uptime: Reflects the availability and reliability of critical systems, with higher uptime percentages indicating robust protection against cyber attacks and other business interruptions.

• Phishing Simulation Click Rates: If your organisation conducts phishing simulation training, the click rate (percentage of employees clicking on simulated phishing emails) can be a vital KPI. Lower click rates over time suggest improved employee awareness and training effectiveness.

• Compliance with Security Policies: Tracking compliance rates with internal cybersecurity policies helps assess the organisation’s adherence to established security protocols and the effectiveness of employee training programmes.

Step 3: Fully calculate your investment

Quantifying the Cost of Cybersecurity Measures is the third step in calculating your cybersecurity ROI.  It involves calculating the total investment in cybersecurity, encompassing technology, personnel, training, and any other related expenses. By accurately quantifying these costs, your organisation can better assess the efficiency and effectiveness of your cybersecurity spending in relation to the benefits gained, such as reduced incidents and improved security posture.

Direct Costs

1. Technology and Tools: This category includes the costs associated with purchasing and maintaining cybersecurity software and hardware, such as firewalls, antivirus programs, intrusion detection systems, and encryption tools. Given the rapid evolution of cyber threats, Australian businesses often need to invest in state-of-the-art technologies to stay protected.

• Personnel (either inhouse or outsourced): Cybersecurity personnel costs cover salaries, benefits, and training for in-house cybersecurity teams. Considering the shortage of skilled cybersecurity professionals in Australia, organisations may also incur higher costs in recruiting and retaining talent.

• Training and Awareness Programs: Regular training for staff on cybersecurity best practices and emerging threats is crucial. These programs can vary widely in cost depending on their scope and delivery method.

Indirect Costs

1. Compliance and Certification: Many Australian businesses operate under regulations that require compliance with specific cybersecurity standards. Costs associated with achieving and maintaining compliance, including audits and certifications, fall into this category.

• Cyber Insurance: Cyber insurance premiums are another significant cost for businesses seeking to protect themselves against the financial fallout of data breaches and other cyber attacks.

• Incident Response Readiness: Investments in developing and maintaining an incident response plan, including regular drills and the engagement of external consultants, are critical yet often overlooked costs.

By thoroughly quantifying these costs, you can establish a clear baseline against which to measure the effectiveness and ROI of your cybersecurity effort. It’s essential to adopt a comprehensive calculation and consider both your direct and indirect costs.

Step 4: Calculate the cost of a Cyber Incident

It’s getting real now.  Calculating the Cost of a Cyber Incident is the fourth essential step in measuring the Return On Investment (ROI) of your cybersecurity efforts. This step involves estimating the financial impact of an incident on your organisation, encompassing both direct and indirect costs, to provide a comprehensive understanding of the potential losses that your cybersecurity investment aims to prevent. Accurately assessing the costs is crucial for calculating your ROI.

Here are some example cost categories:

Direct Costs

1. Recovery and Remediation: These are the immediate costs incurred to address and rectify the aftermath of a cyber incident. It includes expenses related to technical investigations, restoring data and systems, procuring new hardware for bricked devices and implementing security measures to prevent future incidents.
2. Notification Costs: Laws in Australia, such as the Notifiable Data Breaches (NDB) scheme, require businesses to notify affected individuals and the Australian Information Commissioner in the event of a data breach. The costs associated with these notifications, including communication and potential credit monitoring services for affected customers, must be accounted for.
3. Legal Fees and Fines: In the event of a data breach, organisations may face legal actions and regulatory fines, especially if the breach involves sensitive customer data or violates privacy laws. These costs can be significant, depending on the nature of the data involved and the extent of the breach.

Indirect Costs

1. Reputation Damage: Although the public’s understanding of the frequency and pervasiveness of breaches has softened their reaction to news of a breach, a cyber incident can still significantly damage an organisation’s reputation and trust within its client base, which is particularly critical in the Australian market where consumer confidence is paramount. Witness the fallout from the HWL Ebsworth law firm breach in 2023 that continues to the date of publication.  This can translate into reduced revenue and market share over the following months.
2. Operational Disruption: Cyber incidents often result in operational downtime, affecting productivity and leading to lost revenue. The cost associated with operational disruptions can vary widely depending on the nature of the business and the duration of the downtime.  Don’t just think about files, which may well be backed-up safely.  Think about the simple cost of disruption.  How much would it cost if your systems were offline for several days while you recovered or replaced all your network equipment?
3. Increased Insurance Premiums: Following a cyber incident, organisations can expect increased premiums for cyber insurance, reflecting the higher risk profile associated with the incident.

Calculating the cost of an incidents, especially ahead of time, is tricky.  You’re likely to arriber at a “best guess” as it requires a nuanced approach that considers both the immediate financial impacts and the long-term consequences that may be hard to quantify. For Australian businesses, incorporating local case studies and statistics can provide more context and help in estimating these costs more accurately.

Step 5: Evaluate your Effectiveness and Make an Estimate of Likelihood

OK, you’ve got a cybersecurity objective and you have broken it down into measurable KPI’s.  You have established the cost of your planned cybersecurity measures, and you understand the cost of a cybersecurity incident if it occurs. 

What’s missing is a co-efficient of risk.

A cyber incident is neither 100% likely nor 100% unlikely to occur and to cause a serious issue.  There’s a certain likelihood that one of the many automated or human attempts to infiltrate your network will succeed and bring about the costs that you estimated in the previous step.  But balanced against that likelihood is the effectiveness of the systems you have in place.  Thus, to calculate your ROI you have to arrive at a number between 0 and 1 that reflects your current risk profile, or co-efficient of risk.  Your number is more likely to be a range between X and Y, indicating your level of confidence in your security countermeasures to prevent a breach.

How can you accurately calculate or estimate your risk co-efficient?  That’s the million-dollar question.  Quantifying risk is a discipline unto itself, requiring a custom approach for every company.  In reality, even businesses that employ hundreds of staff members are usually too small to see value in defining and executing a consistent risk appraisal methodology.  That’s because the issues detected in your company’s security profile can be prioritised from most likely to be exploited to least likely, and highest impact to lowest.  This double rating is an effective proxy for the quantification of risk – in other words, you are very likely to make the same decisions about which risks to address from this assessment as you are from a much more complex quantification project.

Note: If you would like to read more into the quantification of risk, consider this text: https://www.amazon.com/How-Measure-Anything-Cybersecurity-Risk-dp-1119892309/dp/1119892309/ref=dp_ob_title_bk

So, you could apply some real-world intelligence to your risk co-efficient calculation by engaging in one or more of these actions:

1. Conducting a Cybersecurity Audit. One of the primary methods for evaluating the effectiveness of cybersecurity measures is through regular cybersecurity audits. These audits should examine all aspects of the cybersecurity framework, from technological solutions and processes to employee compliance with security policies. An audit can identify vulnerabilities, assess the implementation of security controls, and recommend improvements. Engaging external auditors (like Computer One) can provide an objective assessment and benchmark your organisation’s cybersecurity posture against industry standards like ISO 27001’s Annex A controls.  If you pass with flying colours, with no serious issues detected, your risk coefficient will be lower.

• Scenario Analysis and Penetration Testing. Scenario analysis, including tabletop exercises and penetration testing, is another valuable method for evaluating cybersecurity effectiveness. These exercises simulate cyber attacks or breach scenarios to test the organisation’s response capabilities, including detection, mitigation, and recovery processes. Penetration testing, conducted by ethical consultants, can uncover vulnerabilities in your cybersecurity defences that need to be addressed.

• Monitoring and Analysing KPIs. The Key Performance Indicators (KPIs) identified in Step 2 are essential tools for ongoing evaluation. By continuously monitoring these KPIs, your organisation can track its performance over time, identify trends, and make data-driven decisions to adjust its cybersecurity strategies.

• Leveraging Cybersecurity Frameworks. Frameworks such as ISO27001 or the Australian Government’s Essential Eight Maturity Model provide structured guidelines for evaluating cybersecurity practices. Aligning with such frameworks can help you assess your cybersecurity measures’ maturity and effectiveness, ensuring that you meet or exceed industry standards and best practices. If you are accredited to ISO27001 or have achieved Level 2 or 3 Maturity against the Essential Eight, you will have reduced your co-efficient of risk by a substantial amount.  Heuristically, your co-efficient will be between 0 and .5, indicating that you have at least halved your risk factors.

Using the information you have gathered, and comparing your organisation’s posture against one or more security frameworks will help you arrive at an estimate of the original (untreated) and residual co-efficient of risk that you’ll use in the calculation of your ROI.

Step 6: Calculate your ROI

This calculation provides a clear, numerical indication of the value derived from investing in cybersecurity measures, comparing the likely costs saved by preventing cyber incidents against the total costs of implementing those measures.

The Basic Formula for Cybersecurity ROI

The ROI for cybersecurity can be calculated using a basic formula:

Cybersecurity ROI

=

[(Original Risk Coefficient x Original Worst Case Incident Estimate) minus (Residual Risk Coefficient x Revised Worst Case Incident Estimate)]

——————————————–
Cost of Implementation

In this formula, you are calculating the potential savings thanks to a reduction in your risk profile and then comparing them to the cost of implementing the risk reduction actions.

Here’s an example

The Worst Case Incident Estimate in the next 12 months for a manufacturer of frozen bakery items with 150 staff might be a ransomware attack that not only deletes all common file types on the company’s on-premises and cloud infrastructure, but also ransoms them back with a promise of publication if ransom demands are not met, and also bricks the company’s networking equipment, causing delays of a week’s productivity while replacements are sought, configured and installed and offline backups restored.  Client and supplier confidence may be shaken and have a knock-on effect as alternative suppliers are preferred in the following few months.  The cost of such a wholesale cybersecurity incident may be estimated to be $280,000.

The original co-efficient of risk may be estimated at .55, reflecting the fact the company is largely unprotected, with poorly secured internet-facing services, low protection over email, no vulnerability detection protocols in place and inconsistent server patching among other shortcomings. 

After a security review and the application of numerous security countermeasures, the company’s residual co-efficient of risk may drop to just .2, reflecting the fact there is more work to do over the coming months to make the network and information even more secure.  Nevertheless, the company will have come a long way.  The reduced estimated worst case incident is now $50,000.

The cost of implementing those countermeasures may be $60,000.

The ROI calculation would look like this:

Original risk co-efficient x Original estimated worst-case cost = .55 x $280,000 = $154,000

Residual risk co-efficient x Reduced estimated worst case cost = $.2 x $50,000 = $10,000

Original minus Reduced (in other words, the gain in reduced exposure from your cybersecurity initiatives) = $154,000 – $10,000 = $144,000

$144,000 reduced exposure
——————————————————
$60,000 cost of interventions/mitigations

=2.4 or 240%

The calculation of your worst case cost and your risk co-efficient are a matter for speculation.  It’s unlikely two people (even certified risk appraisal professionals) would arrive at exactly the same numbers.  But the basic premise of comparing the potential costs of a cybersecurity incident before and after intervention and dividing the difference by the cost of intervention to produce a multiple/percentage return on investment, is sound. And if you intend to produce a regular recalculation of your organisation’s ROI, then as long as you follow the same set of assumptions and calculation steps each time, you’ll be benchmarking against your own data, which will give your calculated ROI more credibility.

Want more cybersecurity?

Check out the the Ultimate Cyber Security Checklist from Charles Square.