Study: IE8's SmartScreen leads in malware protection

A recently released NSS Labs study, claims that Internet Explorer 8 greatly outperforms competing browsers in terms of protecting users against web based malware.

According to the study based upon a modest sample of 492 URLs, not only is IE8's SmartScreen Filter achieving a leading position against the rest of the popular browsers, but also, it also outperforms them in terms of the average time it takes to block known and already tested malicious sites. Among the key conclusions is that Opera 9.64 and Internet Explorer 7 provide "practically no protection against malware".

Here's how the study ranks the browsers:

• Microsoft Internet Explorer v8 (RC1) achieved 69% block rate
• Mozilla Firefox v3.07 achieved just over 30% block rate
• Apple Safari v3 achieved 24% block rate
• Google Chrome 1.0.154 achieved 16% block rate
• Opera 9.64 achieved 5% block rate
• Microsoft Internet Explorer v7 achieved 4% block rate

The study's methodology is however, greatly flawed at several key points, making its conclusions open to interpretation which should be the case when making such comparative tests.

• Go through related posts detailing the growth of client-side vulnerabilities: Secunia: popular security suites failing to block exploits; Google introducing Safe Browsing diagnostic to help owners of compromised sites; Report: 92% of critical Microsoft vulnerabilities mitigated by Least Privilege accounts

For starters, NSS Labs undertook a rather minimalistic approach towards the definition of web malware. In this study, the malware URLs they're using are basically "links that directly lead to a download that delivers a malicious payload", a decision that directly undermines the statement of "block rate" in times when client-side vulnerabilities are massively abused courtesy of web malware exploitation kits. And since no live exploit URLs were taken into consideration, the DEP/NX Memory Protection feature within IE8 was naturally not benchmarked against known exploits-serving sites, or at least wasn't mentioned in the report.

Moreover, the competing browsers' use of SafeBrowsing's API, a combination of automatic (honey clients) and community-driven efforts to analyze a web site in a much broader "malicious" sense has a higher potential to maintain a more comprehensive database of known badware sites. It also comes as a surprise that Firefox, Safari and Chrome have such a varying block rates given that the browsers take advantage of the SafeBrowsing project's database. Basically, having a set of ten malicious URLs and running it against the browsers is supposed to return identical results due to the centralized database of known badware sites.

Interestingly, the study used Apple Safari v3 in order to come up with the 24% block rate, which excludes the built-in anti-phishing and anti-malware features introduced in Safari v4. The report is released prior ot IE8's debut, but even if NSS's study is in fact relevant in a real-life attack scenario, does it really matter that IE8's outperforms the rest of the browsers in times when IE8 users are downgrading to IE7? That very same IE7 which according to the study is offering "practically no protection against malware"?

Anyway, consider going through the report, with a salt shaker in hand.