I have spoken with board members and it is clear that “C” levels are not reporting cyber risk to them in a meaningful way. Modern boards are tired of seeing traffic light reports. The reporting can no longer say – this risk is red but don’t worry – we have a risk mitigation plan.

“I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind” William Thomson (aka Lord Kelvin)

So I was a little enthused when I came across this article on LinkedIn:

https://www.csoonline.com/article/3658118/cybersecurity-metrics-corporate-boards-want-to-see.html

I started to read it with good hope but that feeling quickly vanished.

I doubt very much that the metrics proposed in the article would “help map security investments with strategic goals and risks”. If anything, they would confuse the board even more.

Let me quickly respond to each of the metrics that this article recommends.

“Cyber risk: the percentage of inappropriate usage activities out of all usage activities”

This is not risk – this is failure rate.

To be a risk you need to consider consequences.

Remember that all things are not created equal. I would be more concerned about inappropriate usage of the finance system compare with the inappropriate usage of some random SaaS product on the web that the staff are using for their footy tipping competition, lumping them in the same bucket is not helpful.

“Cybersecurity efficacy: percentage reduction in cyber risk provided by the real-time cybersecurity controls“

OK.. what is your current risk? (Inherent risk – risk mitigation = current risk). Assuming you have a method for converting risk into a number. As each asset has a different starting point of inherent risk, hence each cyber control will have a different percentage reduction for different assets.

But this is about cybersecurity efficacy. “Efficacy” implies some sense of value or cost. There are methods for converting risks to dollar amounts (eg FAIR). However, we need to remember that risk mitigation is not a linear curve. Hence, efficacy is not linear. Cost is exponential, the closer we try to get risks to a hypothetical zero the greater the costs.

Note – I also hate the idea of percentages in this context. It implies that you can achieve a 100% reduction of cyber risk.

“Cyber exposure: average number of usage activities per IT asset“

That will be radical news to the board, our corporate website has more cyber exposure than our internal payroll system. This is some kind of efficiency measurement, something for the CFO instead of the CSO

“Cyber resilience: average number of real-time controls applied for each usage activity“

Real-time controls are not created equal when it comes to resilience. For example, stop doing backups and then try and tell me you have a resilient solution…

Like the measure before this is a pseudo efficiency measurement. Should this number be tracking up or down?

“Risk aversion ratio: the willingness to accept productivity impairment (e.g., password failures, false positives) compared to the malicious activity allowed or denied (true positives plus false negatives)“

I had to read this one several times. I think it is conflating a couple of concepts thereby making it confusing.  I think, on reflection, this talking about something like facial recognition where we have to balance trying to get the app to recognise your face multiple times (higher true negatives)  vs  easier recognition (higher false positives). Here this is really about what you are trying to achieve – increase ease of use or improving security.

Ease of use vs improved security is often a necessary decision. I think boards understand when it come to talking about productivity impairment (ease of use) compared to malicious activity denied (improved security). An excellent example is multi factor authentication – it is a productivity impairment with a significant security benefit.

Again, this is another metric proposed that is VERY hard to measure, let alone display as some kind of ratio.  You could quantify the productivity loss for the 3 minutes extra every employee takes entering their MFA code . You could then quantify the financial costs of a major cyber security incident. Compare one to the other and present this as the “risk aversion ratio”.

Is there another way?

We need to be saying – this risk is red. There is a 1 in 10 probability the risk will happen this year. If it happens it will cost us $1 million dollars. This means there is a 2 in 3 chance (0.66 probability) it will happen at least once in the next 10 years. So averaged out over the 10 years it is a $66,000 loss.

There is software we can buy that will reduce the probability of the risk to 1 in 100 ($6,600/year)

The software costs $20,000 per year

Dear board – There is a reason the CIO/CSIO is spending more money this year. The question is: do you want to spend $66k or $26,6k per year on this risk?

We need to incorporate methodologies like FAIR to bring real dollar costs to risks.

For now, I will go away and put together a response on what I think we should report cyber security status to a board.  

In the meantime, I would love to hear other people thoughts.

What Cyber Security metrics do you think a board wants to see?