2007: The year enterprises open their SOAs to the Internet?

Those that follow the trends on the Internet and the trends within the enterprise have long noticed a very similar direction in both spaces for a while now; a push to move their software to a real services model.  The reasons for this push seem straightforward: easier integration between systems, a better foundation for building new applications, dynamic business process automation that crosses organizational boundaries, and better management and monitoring of IT systems. 

By having a fully-functioning services infrastructure there's even the potential of such esoteric sounding things as BAM, or business activity monitoring, which reasons that if you expose most of your software as services in a standard way, you can finally see what your business is actually doing, while it's doing it.  Thus, services are currently the hot topic as new IT systems come out-of-the-box with open services and older ones are retrofitted with them.  Consequently, as we shall see shortly, 2007 will likely be the tipping point for most organizations to make service-oriented architectures one of their top priorities.

But when it comes to Internet services and enterprise services, the big difference has always been the disparate drivers in these two very different environments:

1) Control - Individual parts of enterprises typically feel unrewarded for opening up and sharing their services and data.  Contrast this with the Web, where opening up and sharing with others is usually rewarded by new customers in terms of users, increased business, advertisers, etc.  Simply put: The culture of most enterprises, with their hierarchical reporting structures, often just does not encourage the freewheeling distribution of information and access to functionality the way that the Web does.

2) Cost structure - In the enterprise, IT systems are typically an overhead cost, or worse, belong to a particular business unit, with either owner driving up overhead costs or unfairly taxing a business unit when a system opens up within the enterprise.  On the Internet however, services via open APIs already have a natural built-in cost recovery mechanism; nominally a fee for usage or other positive motivation such as industry adoption or ecosystem establishment.

3) Mandate vs. necessity - Many enterprises have been trying to make the move to SOAs for several years now, often with SOA projects that attempt to bring consistent standards, technologies, products, and schemas across the organization.  These are typically top-down initiatives (and indeed, almost need to be) that because of their necessary infrastructure focus then all too often fail to connect directly with the operational reality of running a business.  This results in the typical query from the CEO: "How is my SOA specifically helping me run my business better?".  On the Web however, having an open API has nearly become a matter of survival.  If a SaaS product or Web 2.0 site fails to offer an API, it's practically a deal killer since data can't easily be migrated in and out of the service, integration with other existing systems isn't an option, and 3rd party add-ons will not be available.

These drivers give us the state of affairs we seem to be witnessing on the Internet and in the enterprise currently; longer than expected ramp-up and adoption for SOAs for most organizations, while open APIs, syndication, and mashups seem to be flourishing in the laboratory of the Web and some of the bigger players such as Amazon are even building multi-hundred million dollar businesses from their API sales.

What's likely to happen in 2007 with SOAs?

If the current drivers are keeping enterprises in low-gear as far as wide-spread SOA adoption despite clear advantages, how will they break out this year?  As fellow ZDNet blogger Phil Wainewright recently noted (and as I've been been clear about throughout 2006), it's very likely the convergence of SOA and Web 2.0 that will begin to guide many organizations:

Once your software becomes a service in the cloud, it opens up the potential to link it up with other services that are out there. For many vendors and users this is still a barely dawning realization, but it's of fundamental importance. In many ways, the Internet cloud is one great global SOA — still very rudimentary in many ways, but flexible enough to accommodate different levels of sophistication, and evolving fast. Leading-edge SaaS providers use the Internet not only as a delivery mechanism to deliver their services to customers but also as an aggregation platform to enhance and extend their own capabilities by linking up with third-party services.

Certainly, large organizations such as American Express, whose presentation at the Web 2.0 Summit I covered recently, helps paint part of this picture.  As one of the larger enterprises clearly looking at how to leverage the ease-of-use of the consumer Web to make integration easier, they note "Web 2.0 (mashups and RIAs) can bring the data in a SOA to life" including not only things like REST but even the new ad hoc Web components that are emerging: badges and widgets.

But one could argue that this is just well-informed speculation with a couple of good data points.  What are CIOs really concerned about in 2007?  As it turns out, it's opening up their SOAs not just internally, but to their trading partners.   A new bulletin from the respected McKinsey and Company research firm found that in a survey of CIO agendas for 2007, SOA is one of the big two issues on CIOs radars this year (the other being lean manufacturing processes to data center operations.)  In fact, they found that 64% of the respondents to the survey were planning to implement service-oriented architectures in 2007.  Even more interesting from an trend perspective was this:

Just as interesting to us, 48 percent of the CIOs we surveyed in 2006 said that they plan to implement service-oriented architectures for integration with external trading partners in 2007. Traditionally, companies pilot any new integration technology within the firewall, and broader adoption for integration with trading partners follows a few years later. The fact that so many IT departments are already moving beyond the internal pilot stage means that enthusiasm for this trend is high. What’s more, the wide-spread adoption of software as a service promises to encourage the spread of service-oriented architectures, because they make it easier to integrate enterprise systems with applications from third-party vendors.

So, nearly half of CIOs are planning to open their SOAs to the cloud in 2007 (the cloud being where their current and potential trading partners are).  While some companies might not actually do it over the Internet, opening an SOA online (with security and controls of course) is the most powerful when you provide accessible, low-barrier, easy-to-consume services.  And that takes us full circle back to Web 2.0 techniques for opening up applications into platforms. 

The last major difference between enterprises and Internet companies: Service models

There is an almost infinite variety of ways to provide services over a network.  Each one has strengths and weaknesses, varying tool support, differing levels of support in developer tools, are easier/harder to secure, and so on.  But in general these days, opening up software with services is being done with Service-Oriented Architecture -- primarily based on the SOAP protocol -- while on the Web it's been done with something sometimes called web-oriented architecture (WOA), which is primarily based on something called REST.  While the whole SOAP vs. REST debate is too much to go into here, there is an increasing number of options for SOA developers to use that seem to be working much better on the Web that the traditional Web services models do.

Furthermore, current Web models for sharing information that barely resemble Web services, such as RSS and ATOM, but allow systems to be connected together extremely easily (primary example: the blogosphere's massive syndication ecosystem with millions of RSS services connected together to hundreds of syndication platforms) are now another option as well.  And now, though there has been some debate, additional service models such as JSON are not only becoming quite popular but are already actively supported by many of the large Web properties including Yahoo!  This is bringing even more options to SOA architects wishing to provide services that customers and parters actually want to use.  Finally, as another example of how companies are experimenting and innovating to make their global SOA services attractive to users, Amazon has begun making some of its services available via the open source protocol BitTorrent to further drive down the costs of usage to API customers.

This latter trend highlights one last big difference between services on the Web and in the enterprise: There's a lot more competition amongst services on the Internet than there ever will be in the enterprise, resulting in better, more innovative products and many more options for service consumers.  More and more, this will mean that service consumers in the enterprise will have to choose between their internal services and global SOA services.  This trend will further drive both the consumerization of the enterprise as well as the outsourcing of services.

The net result if all this is that the service models that are getting uptake on the Web are generally not the same as the ones being promulgated by enterprise SOA standards.  The Web is the largest petri dish of software experimentation that exists and from here we often get the best examples of what works.  So in pragmatic fashion, most Web sites today do offer SOAP services (though none of any of the Big SOA-style WS-* services that I'm aware of), most developers will tell you that it's simple API models such as REST and JSON which are the easiest to use.  And remember, the browser doesn't speak SOAP at all without help, meaning that REST and JSON are driving the mashup phenomenon and vice versa, again leaving our traditional SOA models out in the cold.

What does all this mean?  Enterprises are going to be opening up their SOAs quite in a bit in 2007, both to trading partners and increasingly to anyone that wants to pay for access to them.  The ones that will be the most successful? The ones that use the proven models that work.  Note: I'm not recommending throwing out existing SOA standards at all, just augmenting them with the Web 2.0 models that work and using the right solutions for the right situations.

And yes, security -- never an afterthought in services -- becomes a critical topic with opening up an SOA.  This will likely be the biggest challenge to these efforts in 2007.

Are you planning to open up your SOA to the Internet in 2007?  How will you do it?