Applying metrics to enterprise services, or simply getting people to work together

Can we effectively measure the benefits we're getting from enterprise service deployments? The problem is getting various groups across the enterprise to agree on common goals and metrics that will provide the big picture. How can organizations see this big picture when everyone has their own take on what's important to their domain?

The ever-elusive enterprise view of project success

One of the things we always point out here that is for SOA to be successful, you need to be able to measure results. The same rule applies to new paradigms such as Enterprise 2.0 (social networking, mashups) and cloud as well. However, in any organization, everyone has their own ideas of what should be measured; everyone has their own take on project "success."

This was the topic of discussion in Dana Gardner's latest BriefingsDirect analyst panel, which I had the opportunity to join, that explored return on investment (ROI) aspects of services. (Audio available here, transcript available here.)

Joining the panel were Michael Krigsman, president and CEO of Asuret, as well as Dion Hinchcliffe, founder and chief technology officer at Hinchcliffe & Co. to explain how their methodology to measuring the ROI and business value of enterprise services, Pragmatic Enterprise 2.0, works. Michael and Dion are also bloggers here at the ZDNet community.

Panelists also included Miko Matsumura, vice president and chief strategist at Software AG; Ron Schmelzer, managing partner at ZapThink; Tony Baer, senior analyst at Ovum; Sandy Rogers, independent industry analyst, and Jim Kobielus, senior analyst at Forrester Research.

In the session, we talked about the requirement for collaborative project management to move new technologies and methodologies on the road to business value. Enterprise 2.0 itself, which emphasizes collaboration, may help in this process. But by not being able to effectively measure a technology's benefits in a collaborative way, we're missing out on many of the potential benefits, Dion said. "This is the classic technology problem. Technology improves and gets better exponentially, but we, as organizations and as people, improve incrementally. So, there is a growing gap between what’s possible and what the technology can do, and what we are ready to do as organizations."

Mismatched expectations are another challenge. People tend to blame software and IT for project failures, but the real failure may be in communications, Michael added. "If we look a little bit deeper, we often find the underlying drivers of the project that is not achieving its results," he explained. "The underlying drivers tend to be things like mismatched expectations between different groups or organizations. For example, the IT organization has a particular set of goals, objectives, restrictions, and so forth, through which they view the project. The line of business, on the other hand, has its own set of business objectives. Very often, even the language between these two groups is simply not the same."

How do you measure this human element? That's the challenge. "The difficulty is how you measure, examine, and pull out of a project these expectations around the table," Michael said. "Different groups have different key performance indicators (KPIs), different measures of success, and so forth, which create these various expectations." The challenge is to surface this information so that it can be shared between decision makers in various parts of the enterprise.

Getting at the big picture is an issue with Enterprise 2.0, and it's certainly has been a long-running issue with SOA as well. Where are the dependencies? How will one system impact another? As Sandy Rogers pointed out, "one of the fundamental needs in running any type of business project -- an SOA project or an Enterprise 2.0 IT project -- is the ability to share information and expose that visibility to all parties at levels."

Success measurements need to get past system-level elements, Sandy added. "If you design your services around what business elements are there and what matters to the business, then you can get past that IT-oriented view in bringing business stakeholders in aligned management and business goals to what transpires in the project. Any way that you can get out -- Web-based, easy-access dashboard with information -- and measure that regularly, you can allow that to proliferate through the organization. Having that awareness can help build trust, and that’s critical for these projects."

Trust has been one of the key requirements for success in service oriented architecture, especially for the sharing of services between parties across the enterprise. And trust is just as essential in Enterprise 2.0 and cloud scenarios as well.