2011 ERP survey: New IT failure research and statistics

Panorama Consulting's new 2011 ERP report (registration required), covering the year 2010, offers worthwhile reading to anyone interested in understanding small business ERP projects.

The survey covers a variety of ERP-related issues; however, we will focus on several points with a direct bearing on the successful outcome of projects. In summary, the survey shows a substantial increase in over-budget and late projects, relative to the previous year.

Project size. The average project size reported by the survey is about $5.5 million. (See notes below for discussion of project size.)

Success and failure metrics. In general, the simplest definition of failure consists of projects that are late, over-budget, or do not deliver planned benefits.

The following diagram tells this part of the story for 2009 and 2010:

We see a dramatic increase in late and over-budget projects in 2010, as compared with 2009. Significantly, survey respondents reported improved achieving a higher level of expected benefit than in the prior year.

One interpretation of this data suggests that companies prioritized project results over maintaining the budget or schedule plan. While that's a positive sign, the survey commentary points out these numbers also indicate that executives did not plan their ERP implementations in a sufficiently realistic manner. These results definitely show good news and bad mixed together.

Implementation rollout approach. From a project failures perspective, implementation approach is a significant condition to examine. As the diagram below shows, more than half of the implementations surveyed used a phased rollout, in which the system go-live occurs in stages. Surprisingly, 35% of the projects went live in a "big-bang" (all modules going live simultaneously), which increases risk.

Although the report does not supply sufficient data to draw any conclusions on this point, it would be useful to correlate implementation rollout approach with budget or schedule overruns.

The diagram also shows that some implementations use a hybrid rollout, which incorporates both phased and big-bang for different parts of the system.

Customizations. ERP customizations can add complexity to implementations, increasing project cost and risk. In addition, customized code can make downstream upgrades more difficult and expensive. Despite these risks, only 15% of survey respondents installed "plain vanilla" software without any customizations.

Vendor selections. The following table shows the vendors that survey participants chose for their implementations; Oracle, SAP, and Microsoft top the list:

STRATEGIC ANALYSIS

The Panorama study is a useful indicator for understanding success and failure on ERP projects. For that reason, we must commend Panorama for its research contribution.

Related: 2010 ERP survey: New research and statistics

That said, the survey reporting contains gaps. Although the report describes average project size, it does not characterize variation around that average. Without this additional data, we must make assumptions about the results.

Since Panorama is a consulting company serving the small- and mid-sized market, we can assume a preponderance of survey respondents are small organizations. These smaller companies are probably most likely to arrive at Panorama's website, which is where the survey is located. If this assumption is accurate, then most projects in the survey may be smaller than the reported average; a few very large projects could skew the average substantially.

Given this uncertainty, we should consider the Panorama survey to be useful only in the context of small ERP projects, valued at a few million dollars; the survey does not provide useful data about mid-sized or larger projects. Panorama was unwilling to go on the record and disclose additional data to clarify this point.

My take. Small business ERP projects continue to face implementation challenges and difficulties; misplaced expectations around time and budget drive dramatic overruns on many projects. At the same time, more companies experienced benefit from the ERP implementations, suggesting increased priority on this critical point, which is ultimately the most important of all.