Why IT can't seem to deliver measurable productivity

Are your investments in IT bearing measurable results? Or are the benefits more "feels-right" types of results? Perhaps IT can't deliver measurable productivity because the measurements are wrong.

Perhaps IT can't deliver measurable productivity because the measurements are wrong

Every time I've spoken to a CIO and IT manager over the past decade, one question I always ask is if he or she has been able to measure the results of programs, be it service oriented architecture, CRM, Web-to-host, what have you.  And, I have to admit, I rarely hear measurable numbers -- it's usually anecdotal evidence, such as speedier processing, or positive end-user or customer feedback.

Don't blame the CIOs, though -- it's just that the benefits of IT are inherently difficult to quantify at any high level. In this vein, Janne Korhonen just published an interesting piece over at ebizQ, explaining why it's so hard to measure the productivity impact of information technology.

Janne quotes MIT's Erik Brynjolfsson, who in 1993 published a landmark paper on why the productivity impacts of IT is so hard to measure: 1) measurement error due to use of conventional productivity-measurement approaches; 2) time lags in IT payoffs; 3) localized optimization; and 4) lack of explicit measures of the value of information.

We don't seem to have come too far in the 16 years since Brynjolfsson published that analysis -- at least in Janne's opinion. He delves into Brynjolfsson's four challenges in some detail. For example, he notes that when it comes to measurement errors -- caused by outmoded ideas about constitutes productivity -- he calls for "rigorous new means to measure IT productivity and output are needed to account for IT's role in innovation and new value creation, commensurable with IT-enabled efficiency." IT's contribution to "operational improvements, new capabilities, new products and new markets" -- long underestimated -- is a good place to start.

As discussed here at this blogsite, there are benefits and gains that are either tough to capture, or tough to tie directly back to a particular IT initiative. "Business agility" is a classic example -- how do you measure business agility?  This is the benefit touted for service oriented architecture -- how much agility is being delivered by an SOA initiative, versus other systems? The businesses at the forefront of SOA tend to be at the forefront of other advanced management practices as well.

Then there's the whole matter of oversell, companies pouring money into IT products and services that may be, on the whole, unnecessary or overkill. Or worse yet, end up as shelfware. You don't need to be a productivity expert to see the waste there. So you have the combined storm of hundred of thousands of dollars being spent on something for results that, if measured, will be measured against archaic productivity standards. Will cloud breath clarity into this confusion?  Probably not.

It should be noted that Brynjolfsson hasn't been entirely pessimistic on the ability of IT to deliver business success since that 1993 paper. More recently, he and MIT's Andrew McAfee published data that shows IT making a big difference at a macro level. They observed that industries that made the greatest investments in IT during the 1990s have become the most competitive. "On average," they said, "the whole U.S. economy has become more 'Schumpeterian' since the mid-1990s. [Joseph Schumpeter coined the term "creative destruction" in 1942] What's more, these changes have been greatest in the industries that buy the most software and computer hardware."

Again, it's more than pouring money into products. McAfee and Brynjolfsson state that even with a lot of IT, "both innovation and replication require a combination of leadership and insight from executives. Take innovation: Many companies use IT to capture huge amounts of data from their operations, but relatively few have been able to use this data creatively."

And it takes perceptive management to identify the technology solutions that will make a difference, and be able to effectively measure the impact of those solutions.  As Janne put it:

"Not all IT projects are productive. They may even be detrimental to the business, but misaligned incentive schemes and other structures sustain non-optimal IT decision-making with predominantly short-term planning horizon and focus on operational and cost efficiency. IT investments should be judged by their overall bottom line impact, including not only cost reductions and efficiency gains but also the indirect impact that IT has on increasing business effectiveness."

The bottom line is that there has never been an expectation that IT would be solely responsible for a company’s rise or fall. Adroit management, supported by the right IT tools, makes the difference. A company that smartly and innovatively leverages its IT in new and creative ways will move to the head of the pack. And, thanks to IT, you don’t need a workforce of thousands to do so. And we need to measure these changes in more holistic ways.