SOA's 'blind spot' -- where's the timely and trusted data?

My colleague Ash Parikh reports he has done a measure of "soul searching" -- which included a lot of discussions with experts and practitioners -- about service oriented architecture, and has come to the conclusion that much of the world is operating under a couple of misconceptions about SOA.

Incomplete, inconsistent, inaccurate data torpedoes SOA

SOA “guarantees” business agility: Not necessarily, Ash says. "Applications and business processes require timely and accurate information, but SOA often overlooks or worse yet, simply assumes this requirement."

SOA does “not” need to include a data integration layer: Traditional technology underpinnings of SOA such as EAI and ESB attempt to address data integration, "but in many cases cannot efficiently deal with data volumes and latencies."

Actually, Ash has been talking for some time about these missing links in SOA efforts. Companies are missing the effective establishment of data services that assure the information flowing through SOA-enabled applications is timely and accurate. (Loraine Lawson also channels Ash's insights on the need for better data integration in her latest post.)

Ash discussed the role of data services in the success of SOA in a post last year, observing that data services "can provide a model and standards-based re-usable data abstraction layer that can make holistic, accurate and timely information available to an enterprise integration infrastructure, without all the typical complexity and maintenance costs. More specifically, a data service is a modular, reusable, business-relevant service that enables the access, integration, and delivery of complex enterprise data throughout the enterprise and across corporate firewalls in what’s known as "right time"—in batch, near real-time and real-time modes, including federation."

This disconnect is becoming even more apparent as companies move into the next level of SOA maturity, in which services start reaching across enterprise boundaries. Ash relates what was described to him by an enterprise architect at a leading financial services company, which seemed to have all the right ingredients in place for enterprise SOA:

"Until recently, they prided themselves in having what they regarded as a complete SOA stack. Their SOA implementation included a Web-based self-service portal which was supported by a set of business services, an EAI and ESB product, a business process orchestration technology and a service-oriented registry. On paper, this seemed like a complete SOA stack that could ensure that the IT organization would meet the business’ needs and guarantee agility... What followed was an IT fiasco as they did not think through their data integration situation in detail. Their back-end data sources processed customer information at varying latencies such as batch, trickle-feed and real-time. Their customer information across these heterogeneous data sources was incomplete, inconsistent, inaccurate and not in the desired format. Further, there was no seamless way to prove who touched the information and when the information was updated to support their requirements for regulatory compliance."

Sounds like they had a lot of fun, doesn't it? And this is a financial services company, so they were probably having a crummy year anyway, so they didn't need the extra aggravation of trying to get consistent, accurate and fresh customer information to the portal while going across multiple, distributed and varied data sources. Without an SOA approach at the data layer, SOA may not be ready for enterprise prime time.