Windows 10 versus Windows 7: Whose numbers do you trust?
Long-term shifts in the installed base for Windows PCs are nothing new. For decades, businesses have been dealing with the logistics and the costs of migrating from one Windows version to another.
As my colleague Steve Ranger noted earlier this week, the shift to Windows 10 is following that familiar pattern, with the current corporate standard OS, Windows 7, hanging on tenaciously.
I've been following the same transition, and my view of how it's playing out differs a bit from Steve's. Part of the difference of opinion is just a matter of interpretation, of course, but a larger part comes from the data itself.
His analysis was based on numbers from Net Applications (aka NetMarketShare). I find that data source extremely problematic.
I first wrote about the problems with this data three years ago (see Net Market Share vs. StatCounter: Whose online measurements can you trust?). A fresh look at current data reveals that those problems still exist.
For this post, I've assembled the latest usage figures from NetMarketShare and from two other highly regarded sources that release similar data. The first is StatCounter Global Stats. The second is the US government's Digital Analytics program, which I've written about previously (November 2015, February 2016, and June 2016).
The following series of tables offer a summary of Windows usage worldwide over the second half of 2016. (Note that for the sake of apples-to-apples comparisons, I have normalized the StatCounter numbers so that they represent the same population of Windows PCs as the other two data sources.)
Three data sources, three very different views of the Windows installed base.
A few obvious conclusions leap off the chart.
First, Windows 7 and Windows 10 completely dominate PC usage, accounting collectively for 72.7 percent to 88.9 percent of visits from the installed base. That's a pretty wide range, though, which I'll get into in a moment.
Second, about half of the installed base continues to use Windows 7, with all three data sources pegging the number within a couple points of 50 percent.
Windows 10
Windows 8.x usage is steadily shrinking, and all three sources agree that only the most determined of dead-enders (roughly 1 percent) still use Windows Vista, whose end-of-support date is less than 90 days away.
Finally, Windows 10 usage has increased since the one-year free upgrade offer ended in July 2016. Converted to an annualized rate, Windows 10 usage grew by somewhere between 8 percent and 12.5 percent per year. Again, that's a pretty big spread.
Where the three sources diverge most dramatically is in their measurement of how many people are still using Windows XP, which has been unsupported for nearly three years. NetMarket Share says a staggering 9.1 percent of its visitors use XP, while DAP shows XP usage down near Vista levels, under 2 percent.
So, who do you believe? Start by looking carefully at where the data comes from.
DAP is a measurement of actual visits to US government websites. Roughly 95 percent of the traffic comes from outside the government, and 20 percent or so of that is from outside the US. The sites themselves are a broad mix of consumer-focused information (NASA pictures, National Weather Service forecasts, tax forms, and so on) and sites for business users.
StatCounter and NetMarketShare, by contrast, report only aggregate numbers and not actual numbers of visits. The analytics are targeted to commercial, ad-supported websites.
As I noted in my earlier article, there's a crucial difference in how the two companies measure traffic. NetMarketShare attempts to measure daily unique users, while StatCounter measures total traffic. If you visit a single page in the NetMarketShare network, you're counted, and then your visits to any other page on any other site in the network are (in theory) ignored for the rest of the day.
In addition, NetMarketShare weights the data by country, whereas StatCounter doesn't.
All three sources count billions of visits per year, so sample size isn't a problem.
Those methodological details offer one highly plausible explanation for why XP usage appears to be so much higher on NetMarketShare (and to a lesser extent on StatCounter) than on the DAP numbers.
In a word: botnets.
Ed Bott's 10 most popular Windows 10 tips
When I've spoken to representatives of both analytics firms in the past, they've acknowledged that, like all companies in the online advertising business, they're engaged in a constant battle with scammers and crooks trying to game the system for ad dollars. Because the sites in the DAP network are all government-funded and not ad-supported, there's no incentive for botnets to try to rig the traffic stats.
For NetMarketShare in particular, I'm highly skeptical of the wild gyrations in each category from month to month. Is it really believable that usage of Windows 7 worldwide plunged by a full percentage point from October to November, representing 15 million PCs, and then climbed by an equal amount the following month?
That result strains credibility to the breaking point, especially when compared to what you would expect as Windows 7 machines slowly retire from the installed base and are replaced by newer machines running Windows 10. And if you look at the actual traffic from DAP and StatCounter, that's exactly what you see, a steady but small decline in Windows 7 usage month over month.
StatCounter has an additional problem with its data -- one that isn't visible in the charts here. Over the last six months of 2016, a staggeringly high number of visitors were counted as Unknown. For the first four months of that period, the Unknown category was 5.9 percent of all visitors. StatCounter acknowledged that that was a problem and the number has since dropped, but it's still at a couple percentage points and makes me suspicious of who those visitors are and what OS they're running.
The DAP data is the most transparent of all but has its own problem when one tries to compare it to worldwide measurements. It tends to over count US visitors relative to those in the rest of the world, and it skews in favor of a wealthier demographic of business owners and people who can afford to travel and do business internationally.
Microsoft, of course, could help resolve some of this confusion by releasing an updated count of monthly active users of Windows 10. The last report, more than four months ago, pegged that number at 400 million, which would have been between 25 and 30 percent of the installed base of 1.5 billion Windows PCs.
Today, I suspect that the Windows 10 installed base has grown to roughly a third of PCs in use worldwide, which would put the base of monthly active users at 500 million. (I'm hoping Microsoft will update that number next month when it announces availability dates for the Windows 10 Creators Update.)
And then there's the question of those stubborn Windows 7 users. I suspect most of them are holding on to this aging but still serviceable OS because they understand how to manage it and don't have to deal with the update headaches of Windows as a Service.
This year will be especially crucial when measuring whether Microsoft can successfully avoid another XP-style support crisis. If the population of Windows 7 users is still hovering in the 40-percent range at the end of 2017, that's a sign of a significant problem on the horizon.