SOA's new place in the world: data service cloud

Behind the massive wave of APIs now shaping the cloud and data analytics space is service oriented architecture. SOA sensibilities are essential if enterprises and vendors want to scale their API-based processes or offerings into something of business value.

That's the view of John Crupi, CTO of Software AG's JackBe unit (acquired in August by Software AG) and  former CTO of Sun Microsystems' enterprise web services practice. I had the chance to catch up with Crupi, who reflected on the continuing evolution of service orientation in enterprises, at Software AG's recent Innovation World event.

JM: Back in the  day, you made SOA real and tangible through the JackBe tools, such as enterprise mashups.

Crupi: We wanted to put a face on SOA. We wanted to build a platform for SOA, just as BI did for data warehouses. But it didn’t catch on like we wanted, because enterprises move a lot slower. You have to jump through a lot of hoops.

JM: Any SOA success stories you can share?

Crupi: We have a customer that uses us as Platform as a Service. They have built a whole SOA infrastructure, a beautiful SOA that really is truly at the right granularity. Everything goes through it, so everything they do is service oriented. They started out building these locked-down applications, but using SOA, are opening up data and services via dashboards and all those things. They're opening it up so their business units can start building applicatiions on it a lot faster.

That’s the real value of SOA -- it provides agility and flexibility so you don’t just get locked down into siloed applications.

Q: And the ideal SOA is like weightlessness in space -- frictionless.

Crupi: It's really like a data service cloud if you think about it. Everybody is opening up their APIs, and behind the scenes, its SOA. Salesforce started as WSDL and SOAP and they moved to REST. You have to build SOA to open up APIs to that scale. There's a maker of athletic monitoring devices who opened up an APIs to developers, to share analytics. So they’re building SOA from the start. This is really where we wanted SOA to go.

Maybe SOA's best value is the cloud; that’s maybe where it has its biggest impact. It makes analytics as a service possible. You may be getting raw data, but as the data becomes larger, you want to have nice APIs. But you want to have analytics with it. Customers want more than just data, they want it moved to a service and put into a cloud.

JM: As someone who has worked to make data and services widely accessible to the enterprise, do you see any risks or dangers in its proliferation?

Crupi: There was a lot of pushback in enterprises years ago, and there still is some pushback. If you talk to data warehouse people who do data cleansing and master data management, they say 'there’s no way I'm giving you access to the data warehouse directly, because I'm not going to be responsible for you screwing up the data.' But then they would dump the data into Excel and then say 'go at it.'

JM: Has this been a challenge with SOA all along as well?

Crupi: SOA is so loosely coupled, and the problem with that is that it is loosely coupled. There’s no context with security, because it's somewhat stateless. What happens when these completely loosely coupled disparate systems starts to get coupled? One plus one, 'if anybody sees this, we're all in trouble.' Correlation is very powerful.

Because of that, we have to do all the security in the middle. And then ensure that there's nothing happening between that. I think that’s one of the reasons that SOA took a little while to start moving, because we had to figure out how to do security across multiple systems, and you’re left on your own, maybe with WS-Security for WSDL and SOAP, but not a lot.

JM: Because you're looking at a system looking at another system looking at another system.

Crupi: With mashups its even harder because were pulling data from multiple systems, but we can also lock you down, put high-level governance, and require more security on data that’s combined versus individual data. But I find a lot of people don’t like you to combine data.

That's the number-one reason why companies push out data on PDFs. It's so that you can't take it and connect it to something else.

But there’s a lot of goodness in all of this. I think that we're going to see the value of SOA really driving value, especially now that we can provide end-users with visualizations on top of their data.

There’s another thing about SOA that doesn't get promoted -- it's made for real time.  It has a real-time nature to it -- it's designed about getting information that is changing.

JM: So if something changes on the back end, it doesn't affect the business view or process?

Crupi: That was an 'aha' moment that we had when we moved into the BI space. We have to call this real time, because by definition, the way that we see data flowing from systems that are always changing. If it was static or stale, then I would have a database. So why would you put a service on something that’s not changing? I wish we would have figured it out seven years ago, but it takes a while to figure these things out.