Thinking Strategically about Information Security Metrics

How many patches did you apply last quarter? It is an important question, but it isn't a strategic one. In order to think about risk and security, we need to elevate our discussion from the operational, such as patch levels and coverage, to the strategic.

One way to do this elevation is take a step back and consider where the risk comes from. It isn't uncommon to get caught up in our ACLs and tokens and signatures and simply work within the domain of security controls. So we associated our risk with the failure of controls. But that is only part of the story, and in a lot ways not even the most important part. Even regulations tend to focus more on control weaknesses than they do the underlying assets. And so we lose focus.

If we are thinking strategically, we must consider risk from the business technology perspective (you could even think more broadly if you like, but I find that a bit ambiguous from a metrics perspective and consider it out-of-scope). Thinking this way, it becomes obvious that the future "unwanted outcomes" that constitute our risk, and which we are trying to prevent, are attacks, exploits, compromises, and losses. Further, it is clear to see that these unwanted outcomes occur within the set of all systems activity and therefore this activity is a key aspect of the risk we are trying to minimize. For our purposes, consider these generic system activities that involve an input source and a target transactions.

We also know that risk has two components - a likelihood component that we can derive from the transactions discussed above, and a value component that identifies the amount of money at risk. We don't specifically need this latter one if we all qualitatively recognize the value of, say, a system compromise. However, understanding and quantifying value is an inherent part of what we do, and is crucial to our decisionmaking process. So the second piece of the model is value.

Now, we can start discussing the security controls in place and how they effect the transactions occurring in the IT environment. These controls are applied to transactions that may be good or bad, and result (simplistically) in a decision to allow or deny a transaction. We end up with four possible outcomes from any control test - good | allow; bad | allow; good | deny; and bad | deny. Controls are the third aspect to the strategic metrics model.

Finally, we have incidents. These are somewhat self-explanatory at the level we are operating on. Incidents are simply the manifestation of the unwanted outcomes. The frequency of these incidents within the context of the full volume of transactions provides historical information that can help estimate the risk associated with the population.

Comments

[PhilA]

Pete,

As always, a good post. Can you bring this to more practical terms?

What would your Infosec approach for metrics be at a technology manufacturing company or an Online business?

Thanks,
PhilA

[Brad Andrews]

I am not sure metrics are this simple. Incidents are totally outside your control, unless you limit those to "successful incidents." While you can do some things to secure your border, for example, you cannot control when a new Worm is released or when someone decides to DoS your servers.

Applying these to other areas, like manufacturing or software development, has a lot more of the metrics under the control of the individual. Many areas even know the industry values for many things. Auto mechanics have a book that tells how long a given repair should take.

I don't see how we are ever going to get to that state since we don't have a repeatability to many things here, especially the incidents as I discussed above.

That makes me wonder if the whole focus on metrics is ultimately a waste of time....

Brad

[Pete]

@Brad -

Incidents by definition or successful attacks and are within our control to the extent that our control infrastructure works to minimize them. If they are entirely out of our control, we wouldn't have a profession (or a job).

It is okay to be skeptical, but I recommend not giving up without looking a bit more closely, and with fewer analogies.

Thanks for the comment,

Pete