REST Battles SOAP for the Future of Information Services
There has been some subterranean discussion in the content management standards arena about what is the best way to support the interoperability of content services with applications. Should vendors support content services through the myriad of web services support layers that have been developed over the last decade? Or should we take a leap into the future with the simple REST architecture used by Amazon, Yahoo, Google and dozens of other web properties? The answer can affect how applications, developers and even consumers view information services. I don’t think it is going to be as simple as let’s do both. Over the last two years, I have been involved in two standards bodies and informal discussions with other vendors that are trying to wrestle with the problem of how to handle interoperability of content services. The problem is quite simple to describe. Ever since the database industry standardized on SQL and programming bindings of ODBC and JDBC, the industry has taken off massively for not just the database vendors, but for application developers. Standardization of SQL allowed not just enterprise applications to grow without fear of lock in, but enabled much of the web itself. If the content management industry could accomplish the same thing, the potential would be huge. To achieve this goal the content management standard would need to access remote repositories and be neutral of any language or platform.
Even two years ago, the logical basis for this type of standard seemed to be so obviously web services based that AIIM formed the iECM standard group to address the specification of web services for content management systems. Applications would be free to work against any content repository that complied with this standard. It only took one meeting before we started to talk about what web services meant with several participants starting to ask, “Surely you don’t mean SOAP?” SOAP is the Service Oriented Architecture Protocol, which has evolved as a complicated mix of standards trying to handle the vagaries of security, addressing, and transactions. To most people, web services mean SOAP, although there were several niche alternatives protocols. And besides, IBM and Microsoft had poured billions into development and marketing of web services for SOAP that it seemed impossible for it not to succeed.
REST (Representational State Transfer) was first described by Roy Fielding in 2000 in his doctoral thesis, where he described the architectural model that is the web itself. I know Roy from some of the content repository standards efforts. The interesting thing about REST is that it is neither a standard nor a technology. REST is an architectural style and set of architectural principles and technically speaking, REST is a web service architecture, even though most people think of SOAP as web services. Unlike SOAP, REST is about resources, such as documents in a repository, a universal syntax for addressing the resource, such as a URL, a set of well-defined operations, such as Get and Set, and content types for transfer of information, such as HTML. It was an exercise originally to describe the architecture of the web, but has become a way of describing how you can develop any web application. This can be described as a triangle of Nouns for the resources and limited set of Verbs and Content Types that interact with those resources.
SOAP, on the other hand, looks very similar to standard RPC mechanisms, but with lots of controls to deal with the eccentricities of the web. SOAP is usually the protocol that comes up in the context of standards, although there are other SOA protocols. SOAP with the WS-* standards can support security and transactions more clearly. SOAP has also been mandated in the development of many Fortune 1000 companies and government agencies. However, the SOAP protocol is rather verbose and does not handle large amounts of content easily. SOAP is really focused on the services or the end points rather than resources, which means that although REST and SOAP are theoretically not mutually exclusive, they are in practice.
Three things are making me believe that REST is likely to become the de facto way that people will manage and access content and perhaps information in general. First, REST fits the way applications handle content better than SOAP does. Second, web companies are taking a strong leadership role in developing remote APIs and patterns using REST and especially in enabling mashups, which are becoming the most popular way of accessing information on the web. Third, Fortune 1000 companies are starting to question their current investments in web services and wondering if REST might be a better way of architecting their systems.
The REST architecture for accessing systems across the web really started to take off in the early part of the decade. Amazon supports both REST and SOAP types of protocols with a purported 10 times more applications choosing the REST protocol. Much of the difference has to do with the fact that most web sites interacting with the Amazon are developed with PHP, which can work with REST-style interfaces more easily. Google has also been putting much more effort into the REST side, particularly with their GData initiative. They are now very influential in how REST develops, even though there are no real standard bodies driving this. By early 2006, it was clear that REST was catching up with SOAP as a viable way of integrating systems together.
Nothing has been driving lightweight REST-style interfaces the way mashups and AJAX have. The early Google maps integrations in particular have been very influential in creating the “I’ve got to have that!” stampede toward mashing up visual web pages with all sorts data sources. Hover-over preview and summary views have also removing the need for users to click through to other pages. REST-style APIs are much easier to execute against inside JavaScript in a web browser compared to SOAP. RSS and ATOM are actually examples of REST-style APIs that have actually been integrating with other systems for a while. These interfaces still have issues in terms of security and transaction control. However, any application provider either inside or outside the enterprise wants to create an interactive and more usable interface. Mashups are high up on most new web sites’ requirements list. Mashups are not high yet on enterprise web applications, but employees are likely to put pressure on IT to deliver the same experience they are getting on the web.
Many Fortune 1000 CTOs and enterprise architects recognize the ease of use issues. I was somewhat taken aback when the enterprise architect of a major pharmaceutical company pleaded impassionedly for us to consider REST in one of the iECM meetings. The same is true of major banks and manufacturing companies I have visited. Government is probably a bit further behind. Interestingly, I think IBM is coming to this conclusion. As big as IBM is, supports all of the web services standards and REST standards, especially through ATOM architect Sam Ruby. After billions of dollars and nearly 10 years of trying to make SOAP work, the simple integration patterns of REST are starting to look attractive. This is especially true of the mashup pattern of integrating in the browser, where no elaborate plumbing is required. I think it is also interesting that many technology providers wouldn’t even consider anything other than REST.
REST compared to SOAP is also inherently more content-oriented and scalable. After all it is using the same protocols and mechanisms that are driving the content-rich web. The web manages to capture and serve up gazillions of pages daily. REST can scale as big as the biggest web sites, because it uses the same infrastructure as any web site. However, SOAP requires custom infrastructures to scale to any large size. REST and the web also handle streaming of content efficiently and effectively. SOAP is only now just establishing new interfaces to more effectively handle streaming of content. The fact that a REST interface is stateless is also very important for scalability. REST is also about resources and resources mean content. SOAP is not as naturally distributed or “federated” across multiple servers.
The content management community must have an interoperable standard for building applications. The AIIM iECM standard effort has disbanded, but that doesn’t remove the need for a language-neutral, remotely accessible standard for content interoperability. Once vendors cooperate on such a standard, the ECM industry is likely to take off the same way that database did. Momentum for web services and SOAP may push a standard in that direction. However, a REST-style standard would help launch content management into new applications that have proven to be appealing to users and have been much easier to construct. Anyone who is looking at integrating any sort of information service needs to at least consider REST.