Orlando, Florida -- Jeff Jones has expanded his project to count security flaws (publicly reported and fixed) in the major workstation operating systems and his latest numbers show Windows Vista has by far the best security profile when compared to the major Linux distributions.
Jeff Jones, security strategy director in Microsoft's Trustworthy Computing group, led a TechEd 2007 discussion on the metrics and techniques used to keep track of vulnerabilities and offered a glimpse at his upcoming report card that compares flaws found/fixed during Vista's first six months on the market against Windows XP, Red Hat Enterprise Linux 4 WS (full), Ubuntu 6.06 LTS (full), Novell SUSE Linux Enteprise Desktop 10 (full) and Mac OS X 10.4 (Tiger).
Here's a chart from Jones with the results, which will be revealed in full in a few weeks:
Jones uses data from several public databases and vendor security bulletins to track "days of risk" and actual flaws being reported and patched to determine which workstation OS could be considered safer.
He explained the difficulties -- and dangers -- associated with trying to get an accurate picture of the flaw landscape because of the different ways that vendors release flaw information in advisories and suggested that the NIST's NVD (National Vulnerability Database) does the best job of aggregating flaw information across the board. Still, he warned against using the NVD as a foolproof database because it's "only accurate for certain things."
Jones also discussed some problems with rating the severity of reported flaws since all vendors use different rating systems. Some vendors, like Apple, offer no rating whatsoever, putting the counting/rating game into a bit of a subjective twist.
During a Q&A session, Jones provided a clue as to why Microsoft does not use the CVSS (Common Vulnerability Scoring System) to rate flaws in its bulletins, describing the methodology as confusing.
He made it clear he was expressing his personal opinion (not Microsoft's official take on CVSS) before picking apart what he perceives as weaknesses in the system currently being used by Cisco, Oracle and several big-name vulnerability research firms.
"I don't agree with how CVSS works," Jones said. "I believe a rating system should provide practical usefulness for making decisions and CVSS doesn't do that in all cases," he added.
Specifically, Jones pointed out that the middle-range scores offered by CVSS can be interpreted differently. "I think a CVSS 10.0 is probably a 10.0 and a 2.0 or 3.0 is probably a low-risk issue. But, everywhere in the middle, it becomes much less definitive and confusing," he added.