Web 2.0 success stories driving WOA and informing SOA

The striking contrast between the stories that we've been hearing lately about the slow going of SOA initiatives in the enterprise and the vibrant and rapidly growing ecosystems similar to them on the consumer Web has been generating a lot of debate and discussion in the enterprise IT community recently. This discussion was brought into sharp relief when ZDNet colleague Joe McKendrick recently reported on Burton Group's Anne Manes stating that it "has become clear to me that SOA is not working in most organizations", based on a wide ranging study they performed.

It's become clear that the SOA world will have to change some basic assumptions.This is just one data point of many recently showing the continued shortfalls we've experienced in trying to get our enterprise systems to work together in the ways that we would like. Organizations clearly want to leverage high levels of interoperability to seize new business opportunities, innovate on top of existing assets, and properly leverage the extensive landscape of software, data, and infrastructure that most organizations have accumulated in large quantities over the years. But we are still having a great deal of difficulty doing so and SOA investments are just not reaping the types of return on investments that most businesses would like to have.

Looking for answers on how to improve SOA

This has driven a search for new models since there's little question that the core ideas behind SOA seem to be the right ones. Rather, it's been how we've gone about designing and implementing SOAs that appears to be at the crux of the issue. As we look at the most successful examples of SOA actually working, we keep being drawn back to the Web itself, with companies such as Amazon and their highly successful Web Services Division (with hundreds of thousands of business consumers of their global SOA), Google and its numerous and varied open Web APIs from Google Maps to Google Data, eBay and billions of dollars in listings it generates through its public SOA, or the rise of applications like Twitter (which gets 10 times the use through its APIs than from its user inteface) and applications that are primarily used via their SOA presence. Then there is the increasingly widespread adoption by millions of users of a sort of "visual SOA" with Web widgets and gadgets as well the rapidly growing story of software mashups, aka composite applications in the SOA world. There are many more SOA-ish success stories like this on the Web, but few in the enterprise.

So if so-called Web 2.0 companies -- which value participation almost above all else, both from consumers and organizations that want to integrate them into their offerings -- are seeing highly desirable levels of adoption and significant ROI, how can this help understand how to improve our efforts in the enterprise? Most new Web 2.0 applications start out life with an API since getting connected to partners that will help you grow and innovate is a well-known essential for success online today. Despite years of SOA, we still don't focus on consumption and openness as fundamentally essential characteristics to building an internal partner ecosystem that have beat a path to your door to use the services you are offering to them to build upon.

One big issue, as I've written about in the past, is that enterprises are often very much unlike the Web. Many of the aspects that make the Web successful (simple flat link structure, billions of deep interconnections, a handful of simple and easy to process data formats, etc) just don't exist in the enterprise with it's wilderness of relational databases, proprietary applications, and silos of every description, despite some success in adding a traditional SOA layer on them.

Turning to WOA as an exemplar of good SOA

Dana Gardner recently wrote about here on ZDNet and respected SOA expert David Linthicum covered on Infoworld, the integration and ecosystem models that are working on the Web are increasingly being discussed in the SOA community as potent examples of how to implement SOA with better results. As WOA examples get better and better, it's emerging solutions to many of the issues we have in getting robust SOAs built with high levels of adoption and measurably better business outcomes.

As we extract the positive outcomes of thousands of startups on the Web that have tried virtually every integration and composition scenario imaginable, we've begin to see the broad outlines of what works. And these new models intrinsically take advantage of the important properties of the Web that have made it the most successful network in history. This extraction of what works best has begun to be called Web-Oriented Architecture (WOA), which I've explored here in the past several times and was coined a few years ago by Nick Gall. The concept of WOA has evolved as we've learned more lessons about the new approaches working on the Web.

Read a more complete and technical description of Web-Oriented Architecture (WOA).

One of the challenges of WOA is that it's an emergent set of best practices that we can independently observe and verify on the open Web. It's not a formal set of standards but an set of approaches and delivery techniques that seem to be work extremely well and achieving most of the major goals that we originally set out for ourselves in the SOA world. But there is work to adapt WOA to the enterprise and while we're seeing early successes with RIA models like Ajax, which are natural fits for a SOA and consume Web services naturally, there is more work to be done to figure out how governance, security, orchestration, and other issues will look like. Because like many things in the Web 2.0 world that are then adapted to business, there is usually an "enterprise context" that has to be added to make them workable inside our organizations.

What sort of things might we see in an organization that is trying to adopt WOA? While WOA adoption is actually happening at a grassroots level in many organizations already, we can use those early stories and what's happening on the consumer Web to give us an idea:

What WOA looks like in the enterprise:

• A rich web of REST resources. Instead of a few point SOA services, enterprise data will be exposed through millions of granular REST resources (like the Web itself), which almost any application than can consume HTTP and XML can use. Much higher levels of syndication using RSS and ATOM will also be a hallmark of WOA adoption as it has been so successful in unleashing the Web of data on the Internet.
• Simple tools to weave the Web of resources into new applications. There are is a rapidly growing set of tools becoming available to build mashups quickly out of WOA (and SOA if you must) resources. Increasingly, these tools can be used by end-users but developers can very easily compose WOA resource into new solutions. IT developers will also use these tools as well as their own to create interoperability and integration where needed, like they do with SOA now.
• Highly consumable and reusable WOA "parts" including widgets, gadgets, and embedded social apps. There are now tens of thousands of Web components that can project data and functioanlity throughout the Web (and the enterprise) with a simple point and click. In combination with tools like blogs, wikis, and anything else which lets users add markup, this lets users build basic applications out of commonly available parts. Enterprise adopting WOA will take their own WOA resources and package them using popular "widget models" like Google's OpenSocial and many others. The very presence of these Web parts drives viral adoption of a WOA as other users see them and use them, spreading them to the corners of the organization and beyond.
• Open Web APIs exposed on the Internet to ad hoc partners. Many organizations will start usintheir WOA to work with existing partners and start going into the Web services business, ala Amazon, themselves. We are learning that many organizations are highly underleveraging their vast assets in data and functionality and open them up to the Web in a dynamic way allow innovation and growth to be pushed out in the cloud and exciting new scenarios emerge. And as we've seen in many cases on the Web now, it's also a direct revenue driver (see middle here).

The future is bright for SOA but it's become clear that the SOA world will have to change some basic assumptions about the approaches that are used for design and implementation. Because the consensus is growing that the techniques riding under the aegis of WOA will be the ones to bet on.

Get a full overview of the intersection between Web 2.0 and the enterprise.

Is your organization looking at or using any of the ideas in WOA? Share your story in Talkback below.