For NSS Labs president Rick Moy the questions about his company's study of Internet Explorer security must get old. It's a bit like the movie Groundhog Day. Microsoft finds IE is more secure than other browsers according to a study by NSS and then Moy gets questions about the data and methodology because the software giant funded the research.
Every time there's a NSS study, Microsoft markets the results. However, the NSS relationship started as an engineering relationship. Microsoft found NSS' backed up its own findings and it's pitch city.
Today's headline: IE 9 was able to stop 99 percent of socially engineered malware. IE 8 checks in with 90 percent. The rest of the browsers don't come close. I'm so skeptical about vendor funded studies that I usually don't bother with the briefings. For some reason---slow news day and perhaps curiosity---I took this one. The headlines are all fairly predictable: Microsoft funded study claims IE most secure.
Here are the results based on NSS' research.
Those findings were based on 636 validated malicious URLs. Specifically speaking:
From an initial list of 8,000 new suspicious sites, 1,209 potentially-malicious URLs were pre-screened for inclusion in the test and were available at the time of entry into the test. These were successfully accessed by the browsers in at least one run. We removed samples that did not pass our validation criteria, including those containing adware or that were not valid malware. Ultimately 636 URLs passed our post-validation process and are included in the final results, providing a margin of error of 3.88% with a confidence interval of 95%.
Moy says the NSS methodology and findings are solid. NSS validated the URLs by sandboxing them and watching the malware work. The methodology is the same one NSS does for anti-virus testing and enterprise studies.
However, there's still skepticism. I asked---and should receive---the spreadsheet of all the malware URLs. Moy's point is that this study is picked over more than the rest because Microsoft paid for it. From the NSS perspective, it has to monetize the research somehow. NSS started in 1991 doing testing for tech magazines. From there it branched out with enterprise consulting.
In my interview with Moy, it was clear that the methodology questions were getting old. Microsoft was doing briefings and Moy was getting pelted with questions. His key points: