Mac versus Windows vulnerability stats questioned

Mac versus Windows vulnerability stats questioned

Summary: George Ou writes: "Apple had more than 5 times the number of flaws per month than Windows XP and Vista in 2007, and most of these flaws are serious." Think Apple fans are going to take this laying down? Not a chance.

SHARE:

For his first post on the Zero Day blog after the departure of Ryan Naraine, George Ou has stirred up a hornets nest by suggesting that Macs have far more security holes than Windows PCs. No stranger to controversy, George compiled a bunch of security advisory figures from Secunia and reached this conclusion:

So this shows that Apple had more than 5 times the number of flaws per month than Windows XP and Vista in 2007, and most of these flaws are serious. Clearly this goes against conventional wisdom because the numbers show just the opposite and it isn’t even close.

I'm sure this will surprise no one but a lot of people disagreed with George's findings. As I write this there are over 300 comments, most of which are negative. Ignoring the knee-jerk "That can't be true" reactions however, a number of posters have raised what seem to be legitimate concerns with the analysis. In the interest of balance I wanted to highlight a few of them.

buddhistMonkey pointed out that George seemed to be ignoring this warning on Secunia's web site:

"PLEASE NOTE: The statistics provided should NOT be used to compare the overall security of products against one another. It is IMPORTANT to understand what the below comments mean when using the statistics, especially when using the statistics to compare the vulnerability aspects of different products."

RestonTechAlec calls the comparisons misleading, giving two examples of bulletins that are treated as equals but are far from it:

Two examples from December's list illustrate this. First, for OS/X:

"Integer overflow in print-bgp.c in the BGP dissector in tcpdump 3.9.6 and earlier allows remote attackers to execute arbitrary code via crafted TLVs in a BGP packet, related to an unchecked return value."

What type of user uses tcpdump? Is this a concern? Yes, it is, but ask yourself-- for who?

Now, a Vista detail:

"Buffer overflow in Microsoft DirectShow in Microsoft DirectX 7.0 through 10.0 allows remote attackers to execute arbitrary code via a crafted (1) WAV or (2) AVI file."

You can catch a WAV or AVI file surfing with IE. So this is also a concern, but for who? Probably everybody.

whooda (don't you just love these aliases?) said the search criteria was flawed:

From your very link in the article, you are ONLY reporting vulnerabilities for Microsoft Windows XP Professional.

The problem being that you are only reporting CVEs for Windows for the XP Professional and Vista products (leaving out the Home Edition and Server products). However, you are reporting all OS X CVEs, including any for 10.0, 10.1, 10.2, 10.3, and their respective SERVER products because Secunia doesn't provide a finer-grained OS X search.

On top of that, Apple also posts security updates for third-parties that can effect OS X or other OSes.

Francois (f.r) looked at the reports in more detail and noticed several discrepancies:

The OS X columns contains 7 duplicates...

The following 20 reports in the OSX column have a CVE that says "reserved" with no mention of the affected OS or product. How do you know those are OS X flaws ?...

There are 16 reports in the OS X column for the Sun JRE/JDK. However, Sun does not provide a JVM for OS X. Indeed, the corresponding CVE reports don't list OS X as an affected OS. Why are those reports in the OS X column ? ...

CVE-2007-3504 is described as Windows-only. However, it appears in the OS X column. Why ?

CVE-2007-3756, CVE-2007-3758 also affect Safari on Windows (and iPhone) but apparears only in the OS X column. Why ?

I am curious to know why you listed the following 7 SquirelMail vulnerabilities in the OS X column. This product is not bundled with OS X. And since it's pure PHP code, those are surely present on Windows as well. ...

Same question for the 7 MySQL vulns ... There are also 8 PHP vulns ...

The OS X column also contains Ruby on Rails vulns. And Safari 3 vulns (which Apples lists under OS X AND Windows but not you). And Adobe Flash player.

It looks like to me that you did not consider the same type of usage. One one hand, a Windows desktop, with no third-party software. On the other, a Mac Server loaded with PHP, SquirelMail, Ruby on Rail and MySQL. Obviously, you will find more security holes in the second case.

In all fairness, there were a few posters that supported George's claims. My favorite was from tomhoffman, who wrote:

You probably won't see this on a Mac commercial!

Topics: Operating Systems, Apple, Hardware, Software, Windows

Ed Burnette

About Ed Burnette

Ed Burnette is a software industry veteran with more than 25 years of experience as a programmer, author, and speaker. He has written numerous technical articles and books, most recently "Hello, Android: Introducing Google's Mobile Development Platform" from the Pragmatic Programmers.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

65 comments
Log in or register to join the discussion
  • Exactly, it was a poor comparison...

    ... and no conclusion can be reached either way.
    I realize that the authoring experts are writing in a Blog, yet I assume there will be at least a level of professional research, wanting to delve a little deeper than the first peek at the list and run over to the keyboard to write 'Look Everyone, there's more of this than that.'
    If the readers can take a minute to check it out, why can't the 'experts' ?
    el1jones
    • There is nothing meaningful that can be drawn from these comparisons.

      This has been discussed numerous times within the ZDnet forums and the end result is the conclusions from such comparisons don't mean much.
      ye
      • agree + 1 NT

        NT
        tombalablomba
      • But it is very useful anyway

        You see the clear bias of the person doing the reporting. They don't even bother to read the security advisories, but they will spout off on the inadequacies of the 'looser' OS in their hatchet pieces they call 'blogs'. This is quite useful since you learn about the author's clear biases. I started reading the Mac list in the referenced article. The FIRST entry was a third party (Adobe) vulnerability that affected Mac, Windows and any Unix.
        shis-ka-bob
      • agree +2 NT

        ....
        TechExec2
  • That "warning" is talking about the comparison of Advisories, not CVEs

    That "warning" is talking about the comparison of Advisories in the charts that Secunia made, not CVEs. Apple likes to bundle up to 40 CVEs per advisory while Microsoft at most will bundle half a dozen or so CVEs. So when Secunia compares advisories, it's not a comparison of who has more bugs so they warn you not to do that.
    georgeou
    • Inveigle

      Inveigle.
      Win3.1
    • GeorgeOu you are no Ryan Naraine.

      Give it up will you, you only make yourself look bad.
      The_Nutty_Zealot
      • agree +1 NT

        NT
        tombalablomba
      • agree +2 NT

        .
        TechExec2
    • And yet, George,

      you've yet to answer Francois' point. By his count (and isn't it nice that someone
      did the research for you?), you have [b]at least 70[/b] errors in your list. Over 70
      errors in a list of 287. I'll do the math for you: 25% of your list is wrong.

      To keep that fact in the background, where you originally wrote "I used
      vulnerability statistics from an impartial third party vendor Secunia," you're now
      pretending otherwise.

      The fact is, George, you've once again written an article without doing any fact
      checking, and, when found out, continue to try to divert from that fact, with no
      corrections or apology.

      Keeps that hit count up, though, doesn't it?
      msalzberg
      • Message has been deleted.

        ego.sum.stig
    • And yet you can't even accurately compare CVE's

      You counted CVE's for OSX that were for third party software while ignoring CVE's for software that is part of Windows.

      You counted at least one CVE for OSX that was clearly a Windows-only CVE.

      You counted multiple CVE's for OSX that were clearly marked as not being valid.

      Your entire "analysis" was simply nothing more than a [u]very[/u] thinly veiled hit piece against OSX.
      Letophoro
  • Hastily thrown together???

    as pointed out in your column many things were ignored or not well reseached.
    dsw0@...
  • Of course

    Anything that in any way questions anything related to Apple is automatically wrong.

    Even if you remove all questionable items on the list Apple still comes up looking pathetic.

    What else did you expect the Kool-Aid-drenched Apple fanboys to do, including you, Ed Burnette.

    Sticking their heads in the sand and yelling la-la-la-la has always been the Apple Way. How else would you keep the Apple-is-superior facade alive other than putting blinders on?
    Qbt
    • Bingo! (nt)

      .
      ye
    • but then again

      It would have been so much better if these questionable items wouldn't have been in the list....

      Then you could have a meaningful discussion. But I wonder why an expert like Rian never made such comparisons. Maybe because he knew they're basically meaningless....

      I'm more interested in stuff that gets exploited than possibilities for exploits, especially when most are with regards to services that are never exposed to the net.
      tombalablomba
    • Translation

      25% of the list is wrong, but we don't care because we can pretend Windows doesn't
      suck if we can shout and point loud enough at Apple.
      frgough
    • But if at least 25% of the list is wrong..

      you really have to question the point of the list. You really have to start over, and see
      where that takes you. It may turn out the same, but it may not. Personally, I've had
      no security problems with Windows (3.1 through Vista) nor OS X.

      Unfortunately, George has an agenda, and fits the data to suit the conclusion he
      wants. This is not the first time he's done that.
      msalzberg
    • I totally agree. Nice post.

      .
      NonZealot