MSSP Metrics: Key Risk Indicators (KRIs)

Key risk indicators (KRIs) are used to measure future adverse impacts of events and activities. They are widely used in areas such as healthcare, operations, and disaster risk management. KRIs use existing system and security sensor data to calculate residual risk due to IT operations.

The inputs are similar to a combination of SIEM, GRC, and threat intelligence systems; the output is continuous, objective, actionable metrics. With easy-to-understand and security-posture relevant metrics, technology leaders can design measurable goals and communicate the status and health of security operations to business leaders for decision making purposes.

A platform that supplies KRIs approaches risk measurement differently from traditional systems:

  • Observed Behaviors Across the Enterprise. The platform collects information on actual events observed in the enterprise – not theoretical “possibilities” based on predictive analytics. The concept of false positives does not exist because only real, live events reported by sensors are observed and go into the calculation of a risk indicator.
  • Residual Risk. Even with all cyber defenses in optimal configuration, risk factors ebb and flow throughout enterprise systems. The platform collects evidence and calculates risk factors, tracking and reporting on residual risk. This is the risk that really matters, and not, for example, theoretical risk due to “intelligence” of activity on the internet.
  • Normalization, Quantification, and Context. The platform applies machine learning and advanced statistical analysis to determine what is normal, what is important, and provides context around both of those in the form of calculations or reports.
  • Continuous, Objective. Audit results tell a story of compliance around a point in time. Manually compiled reports are unreliable. KRIs measure risk and activities in real-time directly from sensor outputs, offering a continuous and consistent view of activities independent of interpretations, audit schedules or quarterly reports.

KRIs can “roll up” the stream – which is more like a fire-hose – of sensor data into easy-to-understand metrics:

Vulnerabilities

What vulnerabilities are you susceptible to, and what steps do you need to take to resolve those issues?

The key here is not to bludgeon or overwhelm your customers with problems. There might be dozens of new vulnerabilities being discovered on a daily basis, but only one or two of them are actually relevant to the customer’s critical IT environment. (On the other hand, if all of them really are noteworthy, then you have bigger issues to deal with.)

Threat Intelligence

What threats in the wild are you susceptible to, and what steps do you need to take to resolve those issues?

Your approach to threat intelligence should be very similar to how you approach vulnerabilities. Customers should be able to easily answer questions such as:

  • Is there any evidence that I have been breached by one of the well-known threats?
  • Does my MSSP regularly conduct threat-hunting missions in my environment? How many of these missions have occurred in the past week or month?
  • Is my MSSP finding evidence of data breaches or attacks against my industry peers?

New Threats

What are the most recent threats to appear in the cyber security landscape?

One extremely important IT security metric is the number of new threats that the customer has faced recently. If this figure is steadily increasing or has seen a rapid spike from normal levels, it is a strong indication that the customer is on the receiving end of a targeted attack, or will be in the near future.

New threats are often more dangerous because clients and MSSPs likely do not have a prescription ready to handle them.

Defense Effectiveness

Are attempts to shut down a threat or eliminate a vulnerability effective?

When you try to patch a security flaw, you need to know that you have resolved the problem and that it will not continue to resurface every few days or weeks. Just like treating an illness, successfully handling cyber security issues means that you identify the root cause of the matter, instead of just addressing the symptoms, then remove it.

Severity and Velocity

Is your environment getting better or worse in terms of the pace and intensity of threats?

Of course, any company above a certain size will be the target of probes and attempted attacks from malicious actors, and many of them will already have fallen victim to a breach. However, an increase in the leading indicators of attack activity, such as the pace or severity of events, is a clear sign of a targeted attack or client industry-wide focused campaign.

Metrics that show severity and velocity will allow you to easily pinpoint the concentration of this kind of attack, allowing you to react quickly.

Surface Area

Is there a dramatic increase or decrease in the number of hosts showing activity on your network? Are there any major critical or high events against my critical monitored assets?

A significant spike in the number of hosts showing defense activity, such as a tenfold increase from 100 to 1,000, could indicate that attackers are broadening their scope. Of course, it could also be due to a benign event, such as the acquisition of a new company and the merger of the two networks. Whatever the reason, such an anomaly needs to be identified and examined by security experts.

Another consideration for surface area is the location of your critical assets. You would expect that parts of the network such as the DMZ are more susceptible to would-be attackers trying to expose issues and create holes. These events are definitely important, but far more important would be something like a sudden increase in the number of high and critical events against your customer’s financial data. In this case, you should look into the problem immediately and find out what’s going on.

Architectural Maturity

Are your tools working to identify, detect, protect, respond and recover as you need them to?

MSSPs might fill out an OWASP Cyber Defense Matrix on their clients, to keep track of how well each customer’s security architecture is performing and compliance framework is being covered. This will give you a clear picture of where you are, what you are missing and how you will resolve it along the way.