When the worlds of SOA and Web 2.0 collide

Noted business and IT forward-thinker John Hagel wrote a detailed piece yesterday about what he calls the "highly dysfunctional gap" between SOA and Web 2.0.  And it's true, there are few worlds in the IT industry that seem more opposite from each other, yet are more strangely intertwined, than SOA and Web 2.0.

Yet these two cultures are generally failing to cross-pollinate like they should, despite potentially extraordinary opportunities.As most of you know, SOA or Service-Oriented Architecture, is a corporate means of normalizing the aspects of IT systems to make them more shareable, rewirable, dynamic, and integrated.  Web 2.0 is a similar, but more populist and social concept, that also involves in its own way the turning of applications into platforms that can be reused, shared, and aggregated.  For its own part, SOA has stodgy-sounding composite applications, while Web 2.0 has a virtually identical concept with the much hipper moniker: mashups, a reference to the musical phenomenon it so much resembles.

Hagel notes that these two worlds seem quite far apart, despite being practically related in their technology genes, though certainly far from being identical twins.  For one, SOA is technically more complex and has higher-order concepts, like orchestration, while Web 2.0 has social, presentation, ad hoc organization,and participatory aspects that SOA generally doesn't address at all.   As a result, as Hagel kindly observes, I've noted in the past that SOA and Web 2.0 practically complete each other.  Yet these two cultures are generally failing to cross-pollinate like they should, despite potentially "extraordinary opportunities."  Why and how to fix it?  Hagel has some thoughts about this:

What is required to break this SOA logjam?  Two things.  First, Web 2.0 technologists need to work on connecting directly with line executives of large enterprises without trying to go through the IT departments. Second, they should avoid the temptation to present grand visions of new architectures and concentrate instead on starting points where these technologies can deliver near-term business impact. (This should not be too hard since by nature Web 2.0 technologists are bootstrappers and hackers.)

Of course, the invariably insightful opinion-maven Nicholas Carr noted Hagel's post too, and had a thing or two to say as well, taking a position that I'm increasingly coming around to as well, namely that SaaS might be the way this all happens in the enterprise, more than through the social tools.  Though that's not entirely clear yet either.  Anyway, Carr says in response to Hagel:

That makes sense - but only if you assume that Web 2.0 collaboration tools, like wikis and tagging, will actually pay off within businesses in a broad and substantial way. There are, as I wrote previously, reasons for caution here. If Web 2.0 technologies fail to fulfill the promises being made for them, they could end up slowing rather than accelerating the transition to the next generation of business software. My own sense is that it may be software-as-a-service (SaaS) providers, more than the Web 2.0 crowd, that will end up breaking the logjam, not only through their discrete application services but through integration platforms like Salesforce.com's AppExchange.

ZDNet's very own Joe McKendrick was recently spotted wondering if SOA has jumped the proverbial shark. And I just read about venture investor Peter Rip wondering something very similar about Web 2.0.  The reality is that both notions represent singular and hugely valuable concepts that have a large significance to those that build and use them.  While the terms might be tired, evolving, being reinvented or whatever, the fact is that the innovation on the Web is pouring over the firewall of the enterprise and remaking it.  Slowly in some cases, but much faster in others.  Web 2.0 seems to be overtaking SOA in subtle, yet telling ways.

The funny thing is, it's the intangibles like trust and not technology that matter the most in both worlds, and that often is the biggest holdup for moving things forward.  And I'm talking lack of trust as barrier.  As in not trusting the data in a socially constructed database, not trusting that your Web service supplier will be there when you need them and will stay in business, or even not trusting SaaS given that it's no more reliable than the network cable snaking into your office.  These are some of the bigger issues that Web 2.0 software could actually exacerbate before it helps, and likely converges with, the world of SOA.

But if the intangible concerns with highly federated data and software get surmounted, expect to see the proliferation of just-in-time enterprise mashups, and the tools to create them, for solving situational business problems and supporting dynamic business processes.  This is one of the biggest potential benefits of the merging of Web 2.0 and SOA.  But only if software architects and designers find the right ways of adding critical enterprise context and social utility to the resulting mix.  As he notes, this will unleash the enormous opportunity that Hagel observes.

Is quickly delivering incremental near-term solutions from the bottom up the answer?  Yes, I think Hagel probably has it right there.  What do you think?