One of the more striking differences between IT and the online world these days is the contrast between traditional enterprise service-oriented architecture and its equivalent on the Web, open APIs. More and more lessons are coming from the online space, providing key insights into how we might invigorate the way we open up our IT systems for maximum value.
SOA does not have the same business urgency and lacks critical focus in this regard in most organizations. So while some new data shows that 75% of all large enterprises will be using SOA by the end of this year (and 60% will even be expanding it), the most obvious successes with service-oriented approaches aren't classical organizations at all. They are Web companies that offer APIs out of a basic need: To build a network of partnerships quickly and cheaply as well as tap into external innovation and inexpensive 3rd party investment.
A quick examination of Google News shows several useful new public-facing Web services (aka open APIs) that were announced this week, including one for Microsoft's Bing as well as from smaller companies like School Loop, which just launched an API that "lets gradebook and assessment systems pull data--such as rosters and assignments--from School Loop and write scores into the School Loop gradebook for display to parents, teachers, students, and other stakeholders." Both of these APIs let anyone, anywhere build applications that interact with and incorporate their respective capabilities.
These are just two typical examples of more than 40 new APIs that were released to the world over the last 30 days alone, according to Programmable Web's API dashboard, currently the most reliable source for such information. This pace of release is fairly steady: A "global SOA" is growing up around us on the Web.
Joe McKendrick recently asked here on ZDNet if we needed an iTunes model for Web services. The reality is, it already exists -- albeit in Web-friendly, simple form -- and not in the failed visions of UDDI directories of yore, but in the pragmatic release of hundreds and hundreds of new APIs every year.
Now, it's also true that SOA initiatives in large companies generally don't publicly announce their internal developments, so it's much harder to get a sense of what is being created and used in most organizations. However it's fairly clear that there are some significant differences and outcomes between these two approaches for open services, even as they ostensibly have the same goals on the face of it: To encourage interoperability between different business systems and enable opportunities that would otherwise be too difficult, expensive, or time-consuming to capture.
What's especially intriguing about these two sides of the same coin are the innate assumptions that they make: SOA is usually an overhead effort (thought it can also be done on the ground) between IT and the business which ultimately allows businesses to achieve improved results and even serendipitous outcomes when it comes to the integration and leverage of existing investments in systems and data. The ROI is very often hard to measure and rapid improvements to the business are usually not the norm. SOAs also tend to be more inward facing and designed for internal consumption.
Contrast this with open APIs, in which the API is considered of primary strategic advantage to the business. The view is the investment in the development of an API is warranted because of immediate benefits that can be gained: increased reach to new customers on the network, tapping into external innovation, increased 3rd party investment, and a scalable model for 3rd party relationships. Interestingly, the bigger the organization, the more value an API has to offers to existing and potential partners, primarily because of the data tends to be richer and more valuable and/or the functionality it exposes is world-class through the success of the enclosing business. This is a vision where a service-oriented business channel (open APIs, not Web pages) often becomes the dominant channel for interaction with their customers as it arguably has for market leaders such as Amazon, Twitter, and others. Unlike most SOA efforts, APIs also tend to be designed for consumption by the broader world, though they are certainly used internally as well.
In would be a gross oversimplification to say that SOA is a technical approach to solving a outstanding set of business problems and open APIs are a business solution that uses a technical approach, but increasingly that seems to be the case. A couple of years ago I asked if it was the timing was right for businesses to open up to the cloud particularly since a near majority of CIOs were clamoring for it. For more enterprises, that just hasn't happened, leaving strategic gaps in execution that has helped lead to the recent discussions about the possibility of the quiet death of SOA.
These points highlight a key difference between the goals and strategies of SOA and open APIs. Open APIs are about business, first and foremost, by attracting developers to build useful new things that are mutually beneficial. In creating a highly consumable and valuable set of services, Web companies hope to attract over time the cumulative efforts of an ecosystem of developers and their respective businesses. It's the classic "turning an application into a platform" strategy that has been successful for top Web 2.0 companies: SOA does the same thing with a different focus, which I've discussed here on this blog for several years now. The success of the API model is clear: Back only a year ago there were only 700 APIs in the global SOA. Today there are almost double that, nearing 1400. Collectively, they provide a backbone of services in which almost any kind of service can be found and which companies can use to create the richest and most competitive products and services for their customers.
So in the end, if all this openness, partnership, and interoperability on the network is so good for us, where are we falling short in terms of building service-oriented businesses? This question is particularly germane to the enterprise where SOA has frequently been perceived as providing lackluster results recently. To get a sense of the issues, I asked two well-known figures in their respective fields to give us some additional perspective.
First off, I contacted Burton Group's Anne Manes, a respected authority on enterprise SOA, a central figure in the debates on the future of SOA, and in my opinion a level-headed voice of reason about technology and business, asking her to briefly summarize what it takes to make SOA successful for a business. She provided this reply:
IT organizations must realize that SOA is not an end goal. SOA is a means to an end: delivering systems that are more manageable, flexible, and agile. But the primary goal must be to support the ever-changing needs of the business. IT groups should spend less time focusing on technology and infrastructure, and instead focus on delivering systems (aka services) that deliver measurable business value.
I then went to Oren Michels, CEO of Mashery.com, a leading firm that helps large companies like Netflix, MTV, Best Buy, and the World Bank deliver open APIs to their internal and external customers, and asked him the same thing. Here's what he said:
Providers of Web services benefit from the creation of a distribution channel for their core services. Few companies have as their core competency the creation and running of a distribution channel, and rarely can a company grow their channel nearly as quickly as they like. Taking advantage of the reach of the partners who incorporate your Web services allows you to focus on making those services better - in other words, focusing on what differentiates you.
So whether you are a provider or consumer of Web services, it comes down to focus - you build, run and manage the things that define and differentiate you and make those Web services powerful, reliable, scalable, secure, and bulletproof. Providing that value will get you paid. Building and running things that are available elsewhere will cause you to lose focus, which is a direct path to failure.
Both of these views underscore the same basic point: Business value is the driver for the success of services in an organization, period. Both Anne and Oren also independently used the word 'focus' as a key discriminator for the success of a service-oriented channel (either SOA or open API.) Oren's view was even more extreme however; in an online world where lack of engagement doesn't just mean a failed services endeavor, it often means the end of your business, survival is increasingly hinged upon being able to provide an attractive set of services for your most important customers; your business partners.
It's very telling that Twitter still gets 10 times more usage through its API from partners than through its Web interface and this ratio is not unusual. It's also very uncommon for a new Web product to launch without an API; services are the route to success in today's Web where if you have anything of value, you need to make sure it has leverage. SOA does not have the same business urgency and lacks critical focus in this regard in most organizations.
In short, traditional enterprise SOA has a lot to learn from the open API world. Not all results are spectacular, but there are plenty of excellent examples and it's part of an increasingly urgent call for open business models and without a capable and healthy services ecosystem that functions well on the open network, organizations are increasingly at a disadvantage in a Web-oriented world.
How can SOA's take the lessons of open APIs and apply them, fortunately, the means are straightforward and are slightly more business-oriented than they are technical. Finally, some SOAs are indeed doing these things, but too many are missing the opportunities that face them.
There are other important aspects of open APIs such as governance (rate limiting, security audits, version management) and service-level agreements, but these are more readily understood by today's enterprises. But for those that are seeking to make their services more critical to the business, even vital to the bottom line (see the great revenue graph from Amazon), the points above enumerate many of the differences between services that drive value and services that don't.
The emergent business environment of the Web has provided hard lessons for online companies that must survive in the rough and tumble world of the Internet, through any techniques possible. Along the way, this increasingly seems to have proven out a model for services that contains many hard-nosed lessons for SOA, namely: 1) Our business vision and partner outreach for SOA needs significant improvement and, 2) opening up to the cloud will be a key next step for many businesses looking for the full value from their investments. These lessons are continuing to come in and we'll look at some stories, including cloud computing/SOA, the new Semantic Web, and others in an upcoming post soon.
How are you reaping business value from your SOA? Please add comments in Talkback below and I'll do my best to respond to every comment.