How badly are your business applications performing?

Due for formal release later this week, a study conducted by Dimensional Research on behalf of Precise at SAP's May 2009 SAPPHIRE reveals some remarkable stats about systems performance and the response times for problem resolution. 695 persons completed the survey. Some key metrics:

• 30% reporting it taking days or weeks to resolve typical performance issues.
• 35% reporting 3-5 incidents per month, 33% reporting 1-2 with 8% reporting almost daily issues.
• 39% of those reporting monthly issues also reporting unresolved issues.
• 80% reporting that performance issues impact the business with 46% expressing concern about deterioration in customer satisfaction.

Issues exist across the board from ERP through BI and on to CRM and supply chain management. As the image at the top shows, BI leads the field with 63% reporting issues but ERP comes a close second with 59%. Ed Colonna, VP marketing at Precise had this to say: "We're not setting out to take a pop at SAP but it has been extremely difficult to get hard data reflecting performance. It is worrying that 53% of respondents are using manual methods to fix performance issues."

While the data is relatively 'raw' we can draw some preliminary conclusions:

• Significant numbers of customers have yet to upgrade to the newer versions which provide access to Solutions Manager.
• While SM can be used for fix purposes, it is more a deployment tool that despite its age has yet to mature into a tool that existing users can satisfactorily use for this purpose. 36% of respondents were SM users.
• There are many reasons why a particular issue can arise. Although SAP's role was not part of the survey, SAP has a habit of trying to figure first whether it is a customization for which the stock answer is 'not our problem, that's a consulting engagement.'
• Configuration issues introduced during implementation - again, not our problem.
• Customers not understanding their configurations - 56% of respondents did not know how much time a transaction spends in the system of record.
• Lack of documentation that thoroughly explains the data flows - documentation is the last thing on any implementer's to-do list. If not done correctly then customers are disadvantaged in their understanding of the system.
• Lack of benchmark data from which to assess performance.
• Under par support from SAP/implementers/third parties can all play a part in causing frustration.

Transaction performance issues are not uncommon in large complex systems so while this study refers to SAP customers, it is likely that Oracle, IBM and to a lesser extent Microsoft customers will yield similar results. Precise has for instance been solving similar problems for PeopleSoft and Siebel customers. However, that's not an excuse for doing nothing.

Coincidentally, I was on an SAP Mentor call today where the topic of certification and education cropped up - again. I was appallled to learn that the new Level 1 Associate certification is being proposed as a multiple choice examination. This is something that can be gamed and for that reason is viewed as less than optimal. Also, SAP is now emphasizing end to end business processes which requires a fundamenatally different understanding of how a system 'fits' together than being say, a finance or CRM expert. As Jon Reed, SAP education specialist  reports:

At Sapphire, I Tweeted a comment overheard from a consulting manager: “The silo-functional geek is screwed.” A manager at a large SAP consulting partner also told me: “I got a call from a client who said, ‘I don’t want an SD consultant, I want an order-to-cash consultant.” Of course, that’s easier said than done. It’s not that hard to find a good SD consultant, but finding an outstanding order-to-cash consultant is another matter.

Duh?

In a follow up email, Jon Reed said: "I went to Sapphire pretty troubled by all this [the multiple choice issue] and was intending to challenge Leo on it because he had insisted at our last session in NYC that certification and SAP consultant quality went hand in hand."

If quality problems remain, it is hardly surprising that companies experience issues they don't know how to resolve. Education, not just about what works but what DOESN'T work, plus the solutions to fix them should surely be high on any responsible consultant's agenda? Why are the vendors then not taking a strong lead on this instead of leaving it up to third parties which struggle to get attention?