Lack of metrics hobbles security, says Enisa

Businesses are not producing coherent sets of data about security incidents and IT system failures due to a lack of common standards, the security agency has said
Written by Tom Espiner, Contributor

A lack of coherent resilience metrics is hampering public- and private-sector organisations' efforts to improve information security, according to a European computer security agency.

A number of systems have been developed to measure the resilience of networks, but metrics refer to different measures and are not harmonised, the European Network and Information Security Agency (Enisa) said on Monday. This creates confusion about the overall effects of incidents that test network resilience, the agency said.

"There aren't any consistent metrics," Enisa spokesman Ulf Bergstrom told ZDNet UK on Monday. "There is a lack of structure and coherence in what you really measure. There are so many different objectives when measuring resilience."

Effects on computer systems caused by cyberattacks, software and hardware failure, natural disasters, human error and obsolescence do not have sets of global, or even Europe-wide metrics, Enisa said in reports issued on Monday. This makes it difficult to measure resilience of networks.

Some security standards, including ISO/IEC 27004:2009, NISTIR 7564, and NIST SP 800-55 Rev 1, deal with metrics in different ways, Enisa said. Bergstrom said Enisa did not want to "name and shame" any particular system or standard, but he said no metrics standards are harmonised.

There aren't any consistent metrics. There is a lack of structure and coherence in what you really measure.
– Ulf Bergstrom, Enisa

"We need a common understanding, rather than various existing non-compatible standards," Bergstrom said.

Data on incidents that test network resilience need to be collected, aggregated and presented for study to enable a consistent response, Bergstrom added. Enisa launched two reports on Monday to start the discussion about metrics — one aimed at European policy makers and the other at technologists.

Inconsistent metrics

Andy Buss, access and infrastructure service director for analyst firm Freeform Dynamics, also told ZDNet UK on Monday that metrics standards were inconsistently applied across businesses, depending on regulation.

"In most cases there is a complete fragmentation," said Buss. "Many organisations will be completely unaware [of metrics], with no processes in place to audit their systems."

Buss said that, when businesses decide IT budgets, management processes such as collecting metrics are "low down on the investment criteria". Reporting cyber-incidents may also not be in the company's interests as this could lead to reputational damage, he said, suggesting that data-breach legislation may be required to incentivise businesses to report security incidents.

In addition, some standards frameworks can be too cumbersome to make it worthwhile for businesses to use them, Buss said. He recommended that a set of standards should be flexible enough for businesses to implement.

Get the latest technology news and analysis, blogs and reviews delivered directly to your inbox with ZDNet UK's newsletters.
Editorial standards