Information technology metrics are only part of the story

Metrics and measurement are important in the information technology world, and offer tangible proof that a system or application is or is not working up to expectations. It's certainly the stuff of presentations and case studies. However, a new study questions how much metrics really matter in the end.

IT leaders seem to have mixed feelings about the relevance of metrics as they relate to the big picture. A survey recently released by Kaseya, based on input from more than 900 IT leaders, finds somewhat less-than-robust adoption of the key metric of metrics, service level agreements (SLAs).

Overall, most organizations, believe it or not, do not have active functioning SLAs associated with their systems. The organizations in Kaseva's sample that are most advanced ("aligned and strategic") in their IT approach are most prone to embrace SLAs, but still, fewer than half (46%) do it. Only 21% of all respondents have SLAs in place, as do fewer than 10% of so-called "reactive" companies (defined in the report as those with IT departments that do no more than simply respond to user challenges and requests in their day-to-day existence).

Do metrics make a difference? SLAs are important, as they are the contractual form of assurance between service providers -- be they internal IT teams or cloud vendors -- and consumers. SLAs are what can be used to hold providers' feet to the fire, and seek recourse when performance drops off.

Still, the survey report's authors question whether intense metrics really make a difference in the end. They looked at mean time to recover (MTTR), finding close to 60% of the aligned strategic companies actually measure it, versus only 22% of reactive companies. So, it's definitely a best practice among the IT leaders. However, at the same time, overall performance seemed to be the same across both leaders and laggards, suggesting that intense measurement and reporting of all the gauges and dials did not necessarily translate into stellar results.

This is an interesting observation, and the survey researchers should have taken things to the next level. That is, while metrics such as uptime and MTTR are important, even more important are the things that are harder to measure -- what value is being delivered to the customer, and how are these systems contributing to business agility? This is the squishy stuff, and while there are metrics that touch upon it -- key performance indicators such as sales metrics, customer satisfaction levels, product revenue -- ultimately, it's a combination of factors that deliver growth. I always see the squishy stiff in action when I ask CIOs what their implementations are delivering to the business -- usually the responses deal with enhanced end-user satisfaction and more positive customer experiences. It's squishy, but it matters.

Accordingly, the Kaseya survey also shows that the CIO's role today has gone well beyond simply ensuring that systems and applications are up and running. Among the more inspiring places to work (the "aligned and strategic" companies) in the survey, 81% say their CIOs has a prominent place in the C-level executive suite.

There is some movement, though far from the majority, toward the concept of IT being more of an advisory service versus coding, maintenance and firefighting. One in five of the forward-looking companies (20%) are positioning their IT departments as information technology brokers for their enterprises, compared to 10% of the less-advanced companies.

The survey also looked at artificial intelligence and cloud-adoption plans. Almost a quarter of the leading organizations use predictive analytics, compared to only four percent of reactive companies. In addition, more than half of the leaders expect to have deployed cloud services, PaaS, IaaS or SaaS. By comparison, only percent of reactive organizations use cloud services. Again, this makes SLAs more important as ever, as it contractually commits cloud providers to deliver expected performance. But metrics are just part of this unfolding story.