Gartner Magic Quadrant lawsuit: Sour grapes or real gripes?

Industry analyst firm, Gartner, is the target of a lawsuit from software vendor, ZL Technologies, challenging the "legitimacy" of Gartner’s Magic Quadrant rating system. The suit has brought forth an array of divergent opinions.

Background. As one of the top analyst firms, with revenue in excess of a billion dollars, Gartner's opinions and recommendations carry substantial weight with technology buyers and influencers.

On a special website page devoted to the lawsuit, ZL Technologies claims that Gartner's Magic Quadrant does not present a fair and accurate portrayal of the software market. The company says:

Gartner’s use of their proprietary “Magic Quadrant” is misleading and favors large vendors with large sales and marketing budgets over smaller innovators such as ZL that have developed higher performing products.

Here is the original legal filing:

ZL v Gartner Complaint

Gartner's legal response is highly technical, but states that Magic Quadrant reports are "clearly opinion" and therefore protected by the first amendment right to free speech. See page ten of Gartner's Motion to Dismiss the ZL complaint: Gartner Motion to Dismiss

The four legal documents made available to date are available by clicking here.

Divergent opinions. Lawsuits often involve strong emotions and polemic, and observers in this case have divided opinions.

In the anti-Gartner camp, ZDNet blogger, Dennis Howlett, quotes a third party questioning Gartner's objectivity and integrity:

‘And you know what, it doesn’t stop at tin cupping. They double dip. They say they’ll help you with strategy, charge you for the pleasure and then go on to sell analysis based on that to say exactly the opposite.’

Howlett adds:

[T]here is no denying that Gartner opinion stated as fact has become a powerful tool for vendors to wave in front of customers.

Taking a different view, Mark Logic's CEO, Dave Kellogg, blogs:

“Sour grapes” spring to mind as an immediate reaction. In fact, ZL concedes that they’ve been ranked in the “niche” segment of every email archiving quadrant since 2005. (Ouch!) But they nevertheless argue that bigger stakes are in play and that this is not only about ZL, but Gartner itself, technological innovation, and very nearly preservation of the American way of life.

THE PROJECT FAILURES ANALYSIS

The lawsuit raises important issues about the role of industry analysts in the enterprise software ecosystem.

Analysts help technology buyers and sellers understand the market environment in which both groups operate. This intelligence helps technology vendors determine the strategies and features that are important to buyers. Likewise, analysts guide enterprise buyers through the challenging process of procuring large, expensive systems. When this system works properly, analysts provide a valuable service.

Carter Lusher, strategist at analyst relations consulting firm SageCircle, believes reputable analyst firms strive for neutrality, but don't always communicate limitations of their research. He told me:

Savvy research consumers use a variety of sources for information and advice. Unfortunately, consumers of analyst research don’t always know how to interpret reports such as Gartner’s Magic Quadrant. Analyst firms should do a better job educating readers.

Other observers, such as longtime industry analyst Jonathan Yarmis, question the relationship between commercial contracts and favorable analyst coverage. When Gartner analyst, Thomas Bittman, wrote a blog post staunchly defending the analyst business model, Jonathan responded with a strongly worded comment:

If you think all that money vendors are spending is purely because of your insights, again you're being naive. There's an influence component and an insight component. Why do you think they spend so much more with Gartner? Are your insights really that much keener? Or are the vendors more concerned with your market reach and therefore the need to more strongly influence you compared with everyone else?

I spoke with Jonathan and asked him to elaborate:

Most analysts have integrity and I do not believe that a commercial relationship with vendors overtly influences analyst opinions. However, it is also true that money buys access, which, deftly used, can lead to improved favorability.

My take. Analyst research and reporting is not an exact science, which does lead to real or perceived conflicts of interest. The analyst industry can reduce potential conflicts by improving transparency around how it forms opinions and makes recommendations.

Analyst clients should ask firms for spreadsheets and other supporting documentation that explain how vendor and product rankings are determined. Although differences of opinion will always exist, transparency helps ensure impartiality.

To increase transparency, analyst firms should also disclose their revenue relationships with vendors. How much do they get from whom, and what is the break down between retainer and consulting revenue? Consulting is much more influential than subscription research or retainer arrangements, because it represents one-on-one relationships with an individual analyst.

Regardless of who wins or loses the lawsuit, it is beneficial to shine the light of day into this important segment of the technology industry.