CoIT: How an accidental future is becoming reality

Today's highly mobile, social cloud has set everyone's expectations for how easy, powerful, and simple IT can be. The genie will never be put back into the bottle.It's a story as old as the IT department: New technology arrives in the market, it makes some type of work easier to accomplish, the business asks for it, and IT reacts and delivers it. Not always however, and usually somewhat slowly. It was this way with PCs, it was this way with the Internet, and now IT is faced with what is turning out to be a veritable perfect storm of technology and social change.

Over the years this now-classic but laggy and imperfect process has built up a steady tension between the control that many feel must be imposed to prevent chaos and the progress that must be made for an organization to be efficient and competitive. While IT departments certainly do introduce new technology and ideas all the time, they do so on their own schedule and with their own perspective. It's not been an ideal way to apply technology to business but lacking a viable alternative it's worked well enough up until now. So this is how most organizations run today, with a clear divide between IT and the lines of business and IT competence largely centralized in a bureaucracy that is perceived as overhead to be minimized whenever possible.

Enter today's hyper-evolving technology landscape that includes major, generational changes in the way that most people adopt, consume, live with, and otherwise relate to technology. For the most part, the focal point of technology advancement today has been largely encompassed by all things related to the Web, whether this is open source software, UX innovation, workable large scale integration, mobile devices, social computing, cloud, and the list goes on and on. The disruptive innovations from this behemoth incubator of new ideas been washing over the bulwarks of IT with increasing frequency over the years.

All of this has resulted in a situation where most individuals can now wield just as much IT capability as most enterprises, and for a fraction of the cost, (see: Are we ready to declare the "time of death" for the IT data center.) This is a recipe that is ripe for causing change and potential disruption.

It's been the greatly improved maturity of SaaS, cloud computing, and uber-powerful smartphones that has given this consumer-driven wave of change much of its critical mass. Coming in the next wave, the rise of consumer app stores and what I've called enterprise app stores will put another nail in the traditional IT coffin. Finally, there is the whole social business revolution that most organizations are still grappling with, another consumer phenomenon that is breaking across the bow of most businesses today.

Shadow IT: The canary in the coal mine?

What I increasingly see as I talk to businesses these days is a desire to solve problems and meet challenges on their own (usually much more immediate) schedule with solutions that fit them best. IT is increasingly being marginalized by such business priorities and needs on one end and on the other end by workers on the ground who feel they have a better grasp of their own local requirements and are therefore happy to use the consumer IT they are so familiar with to address them. Often called shadow IT or at least "gray IT" when it comes from an entire business unit, numbers are hard to come by but the latest research shows that the trend is up considerably from low double digits five years ago to an estimated one third of all IT spending today according to recent estimates.

This seems to herald a future where the business takes much greater control over the strategic use over information technology, while IT departments continue focused on what they are so good at: the tasks of meeting security, audit, Sox, archiving, performance, and governance requirements. Part of this is evolution is possible because of the much greater tech savvy (and thus expectations) of modern workers including managers. It's also because the IT bottleneck (backlog) is getting so tight that needed solutions can't get through to the business in time. In other words, the pace of change in the technology world is accelerating almost exponentially while the rate of reaction by the traditional business/IT duality has only increased a small amount. The growing gap is becoming unsustainable and it's this combination of forces that's leading to a new conception of IT.

This is something that Computerworld Editor-in-Chief Scot Finnie tackled last month in a piece that was notable for the introduction of a term that I think captures the spirit of a new way of connecting IT to the business. That term is 'CoIT.' Scot meant it to refer to the consumerization of IT but it's also a good short-hand for "cooperative IT." We've long talked about the growing dislocation of IT with the main technology sector and with the virtual invasion in some cases of so much recent innovation. We need a new organizing principle that describes how we'll start to close the gap or at least stop the patchwork beasts that many organizations are at risk of becoming.

Related: 'Success and failure in the time of social'

Like last year's discussion of emergent enterprise architecture, we are at a moment where we can still seize the reins and create operational capabilities that set business users free to solve problems at a local level with a set of workable lightweight constraints around security, data sharing, and other non-functional requirements. Because frankly, the challenges that the greater network are about to impose are an order of magnitude larger than they are today. The pressure created by disruptive new distribution channels like SaaS and app stores has barely begun for example and it will become a siren call to users who readily know that for most of their problems, "there's an app for that."

A vision for CoIT

Here's what CoIT could look like if we extrapolating forward current trends such as 1) businesses taking more local control for IT, 2) workers using their own preferred computing devices and apps, and a 3) manageable processes for rapid uptake of enterprise apps, mashups, and devices matched with IT support processes that scale to match.

• Decentralized (or at least distributed) governance. Not by committee but by fact. Let internal competition for finding the best IT-powered business solutions and proving them out decide. Understand and leverage the reality of fiefdoms and only hold them accountable for meeting non-functional requirements and begin transparent, only supporting them to the extent they are good corporate citizens.
• IT support that scales up to the new app/device proliferation. This is one of the most difficult discussions I've had with IT managers, namely the question of "who is going to support all of this?" There is a good reason that there is an IT bottleneck, the reasoning goes. It ensures that bad things don't happen and that the most important solutions get attention while poor ones get vetted and put down. But the high rate of IT failures and low utilization of many big budget ERP and CRM solutions belies this story. We often get it wrong when we do big IT to quickly, in isolation, and don't let internal competition and shared experience drive the right answer. So how do we scale IT? We use models that have been proven to do so externally (on the Web and elsewhere). App stores which allow policy-enforced requirements yet enable choice are offering serious potential to scale out access and policy. One-off SaaS is more problematic and we'll have to see what the market comes up with, but I suspect it also look a lot like today's emerging enterprise app stores. Systematizing the data integration between all these new solutions in a much more productized fashion will be essential. The backbone of the future enterprise will have to look a lot more like the Web and it's open APIs. Connections and data flow between these apps and devices will increasingly take pages from the success of the consumer world. In other words, run your next-generation enterprise integration (aka SOA) like a startup I say.
• Business led IT solutions with an enabling infrastructure. I want to tell the business world that it's not only OK for them to meet their challenges with solutions that involve IT, but that it's necessary. There is an important supporting role for IT but history has shown that only in a small number of companies does IT lead business innovation. A larger group of non-IT workers than ever before in history can now see what's clearly possible with technology, we are surrounded by it every day on the Web and in our persona lives.

CoIT is a new partnership powered directly by a business vision for the future and supported by a newly scalable IT infrastructure. A close look at today's successful IT projects often shows that this model already works best (business support and commitment, the #1 factor is project success -- is so high because they are driving it). Many in IT will say that businesses will never understand how to apply new technology to transform what they are doing, and I would have agreed five years ago. However there is now a critical mass of world class examples in everyone's hands; today's highly mobile, social cloud has set everyone's expectations for how easy, powerful, and simple IT can be. The genie will never be put back into the bottle.

With CoIT itself become a widely used term? Perhaps, but it's not a buzzword that has to be adopted for organizations to get the message. The important lesson here is that major changes must happen and that they will either be imposed or they'll be embraced. The latter poses the least amount of risk and disruption and offers the greatest amount of promise and opportunity. I say let's cooperate.

What are you seeing as the biggest challenges in evolving IT in the era of pervasive technological, Web-driven change? Please put your comments in Talkback below.