Mac versus Windows vulnerability stats questioned
For his first post on the Zero Day blog after the departure of Ryan Naraine, George Ou has stirred up a hornets nest by suggesting that Macs have far more security holes than Windows PCs. No stranger to controversy, George compiled a bunch of security advisory figures from Secunia and reached this conclusion:
So this shows that Apple had more than 5 times the number of flaws per month than Windows XP and Vista in 2007, and most of these flaws are serious. Clearly this goes against conventional wisdom because the numbers show just the opposite and it isn’t even close.
I'm sure this will surprise no one but a lot of people disagreed with George's findings. As I write this there are over 300 comments, most of which are negative. Ignoring the knee-jerk "That can't be true" reactions however, a number of posters have raised what seem to be legitimate concerns with the analysis. In the interest of balance I wanted to highlight a few of them.
buddhistMonkey pointed out that George seemed to be ignoring this warning on Secunia's web site:
"PLEASE NOTE: The statistics provided should NOT be used to compare the overall security of products against one another. It is IMPORTANT to understand what the below comments mean when using the statistics, especially when using the statistics to compare the vulnerability aspects of different products."
RestonTechAlec calls the comparisons misleading, giving two examples of bulletins that are treated as equals but are far from it:
Two examples from December's list illustrate this. First, for OS/X:
"Integer overflow in print-bgp.c in the BGP dissector in tcpdump 3.9.6 and earlier allows remote attackers to execute arbitrary code via crafted TLVs in a BGP packet, related to an unchecked return value."
What type of user uses tcpdump? Is this a concern? Yes, it is, but ask yourself-- for who?
Now, a Vista detail:
"Buffer overflow in Microsoft DirectShow in Microsoft DirectX 7.0 through 10.0 allows remote attackers to execute arbitrary code via a crafted (1) WAV or (2) AVI file."
You can catch a WAV or AVI file surfing with IE. So this is also a concern, but for who? Probably everybody.
whooda (don't you just love these aliases?) said the search criteria was flawed:
From your very link in the article, you are ONLY reporting vulnerabilities for Microsoft Windows XP Professional.
The problem being that you are only reporting CVEs for Windows for the XP Professional and Vista products (leaving out the Home Edition and Server products). However, you are reporting all OS X CVEs, including any for 10.0, 10.1, 10.2, 10.3, and their respective SERVER products because Secunia doesn't provide a finer-grained OS X search.
On top of that, Apple also posts security updates for third-parties that can effect OS X or other OSes.
Francois (f.r) looked at the reports in more detail and noticed several discrepancies:
The OS X columns contains 7 duplicates...
The following 20 reports in the OSX column have a CVE that says "reserved" with no mention of the affected OS or product. How do you know those are OS X flaws ?...
There are 16 reports in the OS X column for the Sun JRE/JDK. However, Sun does not provide a JVM for OS X. Indeed, the corresponding CVE reports don't list OS X as an affected OS. Why are those reports in the OS X column ? ...
CVE-2007-3504 is described as Windows-only. However, it appears in the OS X column. Why ?
CVE-2007-3756, CVE-2007-3758 also affect Safari on Windows (and iPhone) but apparears only in the OS X column. Why ?
I am curious to know why you listed the following 7 SquirelMail vulnerabilities in the OS X column. This product is not bundled with OS X. And since it's pure PHP code, those are surely present on Windows as well. ...
Same question for the 7 MySQL vulns ... There are also 8 PHP vulns ...
The OS X column also contains Ruby on Rails vulns. And Safari 3 vulns (which Apples lists under OS X AND Windows but not you). And Adobe Flash player.
It looks like to me that you did not consider the same type of usage. One one hand, a Windows desktop, with no third-party software. On the other, a Mac Server loaded with PHP, SquirelMail, Ruby on Rail and MySQL. Obviously, you will find more security holes in the second case.
In all fairness, there were a few posters that supported George's claims. My favorite was from tomhoffman, who wrote:
You probably won't see this on a Mac commercial!