SOA for the masses: Widgets, pipes and teqlets

For many years SOA (Service-Oriented Architecture--building composite applications by assembling components from multiple sources within and beyond an enterprise) and its antecedents have been the province of the developer priesthood. Mere mortals, lacking programming skills, have not been privy to arcane, powerful secrets of SOA codes.

Now the power of SOA is starting to be unleashed to the masses, or at least the masses with a modicum of technical aptitude and logical reasoning ability. Call it SOA 2.0, lightweight SOA, DIY (Do It Yourself) SOA, Recombinant Web or Mashup 2.0 or Widget 2.0, a new class of tools is evolving that hides much of complexity involved in creating applets and threading together composite Web applications. 

Widgets--bite-sized Web or desktop applications--are taking off in all directions. Yahoo, Google, Netvibes, Pageflakes, Windows Live, SpringWidgets, Yourminis, Apple, and Serence Klipfolio are among the players trying to build widgetariums (check John Musser's Programmable Web for more info on the mashup universe). Building widgets is not for the non-programmer, but many widgets are configurable by users. 

Netvibes and Pageflakes are now focused on unleashing their widgets, making them exportable to to any Web page. In an interview with Erick Schonfeld, Netvibes CEO Tariq Kim that widgets are taking over the Web world.  "Widgets are killing the Web page. It is time to go to something else. We are entering the widget economy. We are going there no matter what," he said. Kim also said that Netvibes is developing a Universal Widget API that will allow widgets to communication with one another and synchronize among themselves.  

The bit about "killing the Web page" is over the top, but the notion of repositories with thousands of widgets and Web services that have awareness of each other is appealing. Issues of security, performance, standards and reliability in the widget world still need to be addressed fully.

As part of the widget ecosystem, Widgetbox has developed a Web widget directory, syndication and analytics platform, and Clearspring is also developing a widget syndication platform with analytics. 

Deeper mashing up of widgets and content by mere mortals is where startup Teqlo is heading. The company launched an open beta of its mashup platform for non-techies--no knowledge of scripting or APIs required. I wrote about Teqlo during its more formative stage here. Applications are assembled out of Web services, called Teqlets (XML wrappers that semantically normalize Web data inputs and outputs use a routing algorithm to determine the proper sequencing of the services for the application to run ) on a drag and drop canvas board. 

More on Teqlo from Dennis Howlett and Mark Crofton.

In addition, Coghead has an browser-based application delivery environment with a visual drag-and-drop interface for "tech savvy" users, with no coding required. Denver-based Orchestr8, still in stealth mode, is beginning to shed light on its AlchemyPoint Server, which the company said is designed to "build a huge variety of services that pull data together from the Web, RSS/ATOM feeds, file servers, enterprise data sources, etc."

Yahoo Pipes, introduced earlier this month, lets users remix feeds and create data mashups in a visual editing environment. It requires some programming knowledge, and currently uses RSS and Atom feeds, but the plan is to add other data types.

Jeremy Zawodny of Yahoo introduces Yahoo Pipes and has links to coverage of the service, and Techmeme captures the full blast of people writing about Yahoo Pipes after it was unveiled.

ZDNet blogger Phil Wainewright notes that the Yahoo Pipes demonstration mashups use "deceptively well-structured data, but in the real world data structures are a semantic minefield." Using Yahoo Pipes to mash up business critical data is much more challenging.  

On that front, Serendipity Technologies recently introduced WorkLight, a lightweight, server-based integration service that allows users to define corporate data feeds and securely connect them to end user applications, such as RSS aggregators, personal Web pages and AJAX widgets. Enterprise data, such as sales leads, can be put in the del.ici.ous social bookmark manager.

Dion Hinchcliffe has written extensively on the how enterprises are adopting Web technologies and leveraging SOA for more rapid browser- and server-based application development. Dion writes:

The technique of using the browser itself as the location for rapid, on-the-fly integration of functionality (widgets) and services showed how easy integration could be done on at the point of consumption with simple Web technologies like XML, Ajax, and Javascript snippets.  From an enterprise perspective, it gave a lot of people pause to see how easy it could be done (Paul Rademacher's HousingMaps.com being the original example), compared to the methods used by formal and costly enterprise application integration and service-oriented architecture projects.

In the diagram below, Dion illustrates the enterprise mashup concept. In his post, he highlights IBM"s QEDWiki,  a browser-based enterprise mashup tool that allows less technically inclined users to visually assemble online Web applications out of widgets. IBM announced that it is integrating Yahoo Pipes and QEDWiki.

SAP recently launched a widget toolkit for creating loosely coupled applets that can provide dynamic data, such as alerts for KPIs, via Web services. Microsoft has a Connected Services Sandbox, aimed at developers of "managed network mash-ups," combining Web and telecom services. Companies like salesforce.com have support for basic mashups and widgets, such as with Google Maps and Yahoo Widgets.

BEA has been showing off early versions of Runner, a rapid application services engine for creating composite applications. “Runner lets you take the compositing notion in portals and apply it to any application in an enterprise, regardless of the language without having to change its code,” David Meyer, vice president of product management at BEA, told me last September.  “It’s an application proxy, a broker or gateway between the browser and back end applications and business logic.” Supposedly, Runner can take a record from Siebel or SAP and embed it in a wiki page without writing any code.

In writing about Teqlo, Dennis Howlett said: "There is little doubt in my mind that self-assembled services is the way to set users free from the tyranny of having to make do and mend."

Just as more user friendly writing and video editing tools, as well as increasing bandwidth, have given regular people more of a voice and levers to push, making the Web more programmable--harnessing and mashing up any kind of data without heavy duty coding--will be another quantum leap for mere mortals.