OS scorecard: Usage stats don't tell the whole story

My ZDnet colleague, Ed Bott, just published The 2010 OS scorecard: Microsoft loses a little, Apple wins a lot and I'd like to comment on his post. I've posted something about market share many times (see Microsoft: market share, growth and market value andMarket share is like a beach ball for a more complete explaination of my point of view.)

Does internet usage tracking really tell the whole story?

Internet tracking really doesn't indicate the actual market share of an operating system. At best such market statistics show what has been observed on the net. This figure, of course, is only a subset of all of the copies of system software that have been shipped in a given year and even a smaller subset of all copies of that same system software in use around the world. So, no, looking at internet usage does not tell the whole story.

Here's the text from the post Market Share is Like a Beach Ball:

Market share is something bandied about by suppliers of operating systems, Web browsers, as well as all sorts of systems.  Often these suppliers speak as if everyone agrees on what “market share” is, how it is counted and will put forward with their views on the market. Is there a unified view of what is market share? No, of course not.

Share of what?

There are many different things that can be counted, segmented and leaders declared. Here are a few that are of interest suppliers and consumers of technology. Please note that all of these are of interest to someone at some time.

Share of revenues

• Worldwide
• Specific geographical area
• Specific vertical market
• Specific size of organization
• By operating system
• By application
• On-premise or in the cloud

Share of shipments

• Worldwide
• Specific geographical area
• Specific vertical market
• Specific size of organization
• By operating system
• By application
• On-premise or in the cloud

Suppliers will publicize any market research that shows they’re winning somewhere, somehow and in some market.

Doing this is easy isn’t it?

Conducting supply side research appears to be easy - just count up how much revenue or how many products have been shipped and then segment that total by the category in question. Is it easy? No, of course not.

Analysts face some rather difficult challenges in just getting the raw data.

Public companies must report part of this data, but there are no standards on this.  One supplier publishes only 2 top line numbers even though it sells tens of thousands of different products and services. Segmenting their revenue into categories can be very challenging.

Privately held companies only report what their marketeers think will be useful. There’s no direct way to cross check their statements.

Let the reader beware

In the end, it is important to remember that market share is a very complex concept pretending to be very simple.  Analysts, after having worked very hard to come up with a supportable market segmentation, will see journalists report their work incorrectly. Research on revenues will be reported as shipments. Research on shipments will be reported as revenues. A study focused on one geographical area will be reported as representing worldwide figures.

Most research firms will allow their subscribers to use a limited part of their research in their press releases. This in itself can create confusion.

While I was a research manager at IDC, requests came in from five different suppliers. Each of them wanted to publicize the they were the market share leader. All of them, of course, were granted rights to publish a small segment of the research.

So, back in the late 1980s, press releases went out from five companies, each declaring that they were number one.  All of them were right! Here were the categories that were publicized: worldwide revenues, worldwide shipments, U.S. revenues, U.S. shipments and revenues for technical computing workloads.

In the end, it was a little like looking at a beach ball.  One side is red, one side is blue, one side is green and one is white. What color is the beach ball? Well, that depends upon what side of the ball you’re viewing doesn’t it?

Snapshot Analysis

While statistics of web usage are very important to those running websites or those hoping to sell something to uses of that software, a better indicator of how an operating system is doing is tracking the actual shipments of the software during the the life of that software and/or the revenues the supplier received through the sales of the software that was shipped.

So, Ed's comments are important to those interested in web usage, but not to those interested in sales of system software, sales of systems, potential for services that could be offered to individuals or companies who purchased or are using that software. Other statics would be helpful to them.