Better metrics needed for security, says expert

BOSTON — The security industry has done a poor job of finding ways for companies to measure their security, but that does not mean that collecting data is not valuable, the former head of the U.S. Department of Homeland Security's cyber group told attendees at the SOURCE Boston conference on Thursday. Amit Yoran, CEO of …

[Tom Paine]

The problem with risk management

Fundamentally, the whole metrics racket is a hiding to nothing when it comes to security. Suppose you have a webserver, and for the ease of the example, let's say your company does all it's business through this system. It has a bug. Should you patch it? Well, you can put a number on amount of business lost through downtime when patching (yes yes, no clustering here, we're in Gedankensville, OK?) You might even people to put a wet finger in the air and come up with some sort of guesstimate about how much it would cost you in lost business and reputational damage should you arrive at work one day to find your front page replaced by Fleshbot (although I'd argue that value is entirely theoretical, and anyway is probably a hell of a lot less than you might like to think it is - especially if you're the person trying to diddle the numbers so you get a bigger budget next year.) One thing you absolutely /cannot/ do, though, is put any sort of probability value on the chances of getting pwned /through that specific vulnerability/, per day. That's the sort of numbers the insurance business like to crunch to work out your car insurance premium, and why middle-aged me pays less to insure my 250 BHP turbo-nutter-bastard mobile than a 22 yo with a hot hatch, set of alloy wheels, Haynes manual and a bodykit :) )

Given that the final number at the bottom of the page that's supposed to allow you to rank your systems, the stuff you could do to secure them, and how much you should spend to do so are based on garbage - and we've all seen what lousy risk management based on garbage input can do to the world economy - it follows that one should beware of snake-oil salesmen bearing metrics.

Mine's the one with the "kick me" note stuck on the back by the Sales Director...

[Gary]

Quote

"He who would defend everything, defends nothing.!"

Gneisenau (I THINK!)

"He who would collect all data copllects nowet!

Gary (with appologies to Gneisenau!)

Gary

[Daniele Futtorovic]

Gosh

"[...] Former head of the U.S. Department of Homeland Security's cyber group [...]: 'we need to measure everything'"

Why am I not surprised?

[Anonymous Coward]

Re: need to measure everything

But isn't the real problem that what you are actually trying to measure is what you are NOT doing, so that you can identify it and do something about it. Of course if you could measure it, then you would know about it, and knowing about it you would have done something about it.

Or not, depending on your local implementation of the PHB.