Five key tests for any SOA

Chris Benedetto of Solstice Software keeps beating the drum for end-to-end SOA testing. In a new commentary which appears at Line56, he calls end-to-end testing the "missing link" in SOA.

You may recall a couple of weeks back, he provided some thoughts to this very Weblog on HP's acquisition of Mercury Interactive, observing that little was mentioned about the testing capabilities Mercury brings to the table.

Testing may become the most important feature of all, as SOAs are built on many moving parts that need to be kept in synch:

"With so many moving parts within SOAs, there's only one way businesses can ensure all crucial parts are working properly. Traditional testing tools and methods, which look at some, but not all, aspects of an application, are not the answer. SOA's missing link is 'end-to-end' testing of all services, components, and messages along a business process path."

There are five key areas where SOA needs to be tested, Benedetto says:

1 - Defect detection: "Defects usually remain unseen until the full system can be tested at a project's tail end... and have often multiplied throughout the system, and are among the most costly to fix."

2 - Error permutations: Though XML messages are communicated via standardized SOAP envelopes, they contain many customized formats and fields. "An error can occur anywhere in the fields that contain both logic and message data."

3 - Explicit programming: "Web services and SOA demand rigorously defined inputs and outputs that behave predictably" -- such as monetary figures that must accept dollar signs. Unless this is explicitly expressed for services in SOA, communication errors can abound, Benedetto says.

4 - Process confirmation: SOAs involve dozens of systems, and "most provide a confirmation message that a process, such as an update, has occurred--and that's it." There's no guarantee that the data pulled from one system and acted on by another system "was accurate in the first place, modified correctly, or stored in the right place."

5 - Evolving standards: "Standards for SOA validation, addressing, and governance are not readily available, but these efforts are in practice in a variety of companies today."

To address these five areas, Benedetto advises an end-to-end approach to testing. This requires interaction between various teams involved in the SOA process, such as developers, architects, QA, and operations. Ideally, such testing should be conducted with automated tools, and invoked often throughout the development cycle. "In this way, the architecture or composite application cost-effectively evolves with quality baked in, as opposed to being built and later thrown over the wall to a QA team (which is the most common and costly way to test)."