Windows vs Linux security report card redux

Windows vs Linux security report card redux

Summary: Jeff Jones has expanded his project to count security flaws (publicly reported and fixed) in the major workstation operating systems and his latest numbers show Windows Vista has by far the best security profile when compared to the major Linux distributions.

SHARE:

Orlando, Florida -- Jeff Jones has expanded his project to count security flaws (publicly reported and fixed) in the major workstation operating systems and his latest numbers show Windows Vista has by far the best security profile when compared to the major Linux distributions.

Jeff Jones, security strategy director in Microsoft's Trustworthy Computing group, led a TechEd 2007 discussion on the metrics and techniques used to keep track of vulnerabilities and offered a glimpse at his upcoming report card that compares flaws found/fixed during Vista's first six months on the market against Windows XP, Red Hat Enterprise Linux 4 WS (full), Ubuntu 6.06 LTS (full), Novell SUSE Linux Enteprise Desktop 10 (full) and Mac OS X 10.4 (Tiger).

Here's a chart from Jones with the results, which will be revealed in full in a few weeks:

Jones uses data from several public databases and vendor security bulletins to track "days of risk" and actual flaws being reported and patched to determine which workstation OS could be considered safer.

[ SEE: 90-day report card: Windows Vista fared better than competitors ]

He explained the difficulties -- and dangers -- associated with trying to get an accurate picture of the flaw landscape because of the different ways that vendors release flaw information in advisories and suggested that the NIST's NVD (National Vulnerability Database) does the best job of aggregating flaw information across the board. Still, he warned against using the NVD as a foolproof database because it's "only accurate for certain things."

Jones also discussed some problems with rating the severity of reported flaws since all vendors use different rating systems. Some vendors, like Apple, offer no rating whatsoever, putting the counting/rating game into a bit of a subjective twist.

During a Q&A session, Jones provided a clue as to why Microsoft does not use the CVSS (Common Vulnerability Scoring System) to rate flaws in its bulletins, describing the methodology as confusing.

He made it clear he was expressing his personal opinion (not Microsoft's official take on CVSS) before picking apart what he perceives as weaknesses in the system currently being used by Cisco, Oracle and several big-name vulnerability research firms.

"I don't agree with how CVSS works," Jones said. "I believe a rating system should provide practical usefulness for making decisions and CVSS doesn't do that in all cases," he added.

Specifically, Jones pointed out that the middle-range scores offered by CVSS can be interpreted differently. "I think a CVSS 10.0 is probably a 10.0 and a 2.0 or 3.0 is probably a low-risk issue. But, everywhere in the middle, it becomes much less definitive and confusing," he added.

Topics: Linux, Open Source, Security, Windows

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

210 comments
Log in or register to join the discussion
  • A Microsoft report skewed Microsoft's way.

    [i]Jeff Jones, security strategy director in Microsoft?s Trustworthy Computing group[/i]....

    Come on.....Does anyone think that any report created by a Microsoft employee, would say otherwise?

    Bogus article. I'll wait for an unbiased third party report, and we all know how THAT will turn out.
    linux for me
    • Would someone with a vested intrest in Linux

      give an unbiased report? No, but I will guess the "Third Party" you speak of will be that of the Linuc community.

      And we do know how THAT will turn out. ;)
      GuidingLight
      • Wow That 3.2 Billion M$ Spends... ;)

        on viral marketing sure buys a lot of F.U.D.sicles for the "Monkey Man" to suck on! I guess you help by spreading all the Joy of F.U.D. around with the guy in this Blog with 8 eyes? hehe

        So GuidingLight are you from their "WatchTower"? :)
        i2fun@...
    • yeah...

      This may be slanted towards MS as part of their anti-linux compaign, but I dont think its possible to get a completely unbias'ed report, unless you had a rep from each branch of the OS world working together on it. But the chances of that are nill.
      Stuka
      • Why are the chances of that nil?

        Is it because the people who have the most to lose don't want to cooperate? Or is it the OSS crowd that refuses to work with the main two commercial vendors? I suspect it is the former, rather than the latter.

        What would be useful would be a common set of analytical guidelines used by representatives from each of the major organizations concerned to produce several reports that might be compared side by side so that consumers could make a rational, reasoned choice.

        Obviously that will never happen, because almost none of the parties concerned are about serving the customer. True CS is dead, passed away in the 90s' or earlier. The bottom line is the concern, and taking as many dollars as quickly as possible in exchange for as little reciprocal value as one can get away with seems to be the rule of the day.

        A shame, imo.

        Come the revolution, we will all use Linux and love it, as we walk uphill both ways to school and work, in the snow, with no shoes. We'll love that, too.
        fuzzy2k
    • You mean the third party from Linux advocates.

      ;)

      But if those parties are MS advocates, they are shills. Everyone talks about their favorite BS subject, and scorns the opposition. Whatever guy.
      osreinstall
      • You mean the third party?

        Objectivity is so old-school dontchaknow!
        LBean
        • Objective referees are hard to find.

          It may be old, but it is more honest and civilized.
          osreinstall
    • Shouldn't you...

      ...at least investigate his methodology before coming to such a conclusion?

      Carl Rapson
      rapson
      • Absolutely

        It's difficult to create a large bias in a dataset if that data is simply an aggregation of bug count data from third parties. Or are all of the third parties biased as well?

        Before condemning the numbers at least wait until the methodology is understood.
        nmh
        • It's easy to skew the results...

          ... when you don't disclose 90% of the bugs in the OS. How long did the .ANI bug go undisclosed to the general public before it was patched with an "emergency update"?
          JJQ1000
    • A Microsoft report skewed Microsoft's way.

      Shocking isn't it?

      I've got to ask since so many "articles" seem skewed to favor Microsoft. Did a Redmond Washington corporation buy ZDNet?

      Just curious.
      LBean
      • No! ZDNet is NOT connected to M$!

        They just the messenger of the Beast from the Dark Side!

        Not that they wouldn't take any of that 3.2 Billion M$ spends on Viral Marketing against their competitors. But hey, we all have to make money some way! Right? They are just helping to spread M$ lies. After all that's not illegal! Is it? ;)
        i2fun@...
  • How exactly

    Would one measure unpatched and in the wild? I'm a little confused about how we'd be able to measure undisclosed issues. I'd like to see it to, please pardon my ignorance in this regard.

    ~A!
    anthony@...
    • It would only apply

      to known vulnerabilities that have known exploits in the wild. Such as the MS office holes that have shown up this year. Although many are patched now, for a while there people were being exploited.
      Stuka
  • Maybe you are color blind?

    Vista has the second least amount of red and XP has the very least. The ratio has nothing to do with anything. It's comparing the same time frame for each system. <br>
    Obviously all other systems had 10 to 100 as many flaws total, but from this graph you can't read at what point in time the unpatched flaws are from. Vista has so few overall flaws, they could have come from the last week of the time range.
    <br>
    This chart clearly shows that Vista was far more secure out of the gate than any OS in history.
    xuniL_z
    • Good One

      9.3 on the Coxometer.

      [B]and vendor security bulletins to track ?days of risk? and actual flaws being reported[/B]

      and we all know MS is beyond diligent at disclosing every security problem they know about internally. They have never sat on problems they know exist or are circulating in the wild.

      See my and Jimbo's posts below on how to make the comparison valid, notwithstanding the vulnerabilities MS knows about but are sitting on.

      TripleII
      TripleII-21189418044173169409978279405827
      • Do you even know, for certain

        why microsoft has a monthly patch cycle? Or why they may sit on a given reported flaw?
        xuniL_z
        • xuniL_z you only would if you worked @ MS

          xuniL_z you are a plant.

          Not a very good one at that.
          ihatelinux
    • They're only counting flaws that they've been forced to ADMIT to

      We all know how that routine goes:

      1) Flaw is discovered in the wild by third parties
      1a) If the third party is a security outfit, the pass the word
      1b) If the third party is a hacker, they exploit it
      2) MS issues a release saying that there is no known flaw
      3) MS issues a release saying that the flaw only affects "a handful of systems, which are not properly set up or don't have the latest updates installed"
      4) Security outfits warn about the increasing number of exploits, and provide fixes to protect the weak point in the MS code
      5) 6 months later, MS releases a patch, marked CRITICAL and advising every user to install the update
      6) The day after the patch is issued, the next exploit for the flaw is released, which evades the sloppy MS patch and goes straight into the OS
      7) GOTO Step 3
      critic-at-arms