Study: IE8's SmartScreen leads in malware protection

Study: IE8's SmartScreen leads in malware protection

Summary: A recently released NSS Labs study, claims that Internet Explorer 8 greatly outperforms competing browsers in terms of protecting users against web based malware.According to the study based upon a modest sample of 492 URLs, not only is IE8's SmartScreen Filter achieving a leading position against the rest of the popular browsers, but also, it also outperforms them in terms of the average time it takes to block known and already tested malicious sites.


A recently released NSS Labs study, claims that Internet Explorer 8 greatly outperforms competing browsers in terms of protecting users against web based malware.

According to the study based upon a modest sample of 492 URLs, not only is IE8's SmartScreen Filter achieving a leading position against the rest of the popular browsers, but also, it also outperforms them in terms of the average time it takes to block known and already tested malicious sites. Among the key conclusions is that Opera 9.64 and Internet Explorer 7 provide "practically no protection against malware".

Here's how the study ranks the browsers:

  • Microsoft Internet Explorer v8 (RC1) achieved 69% block rate
  • Mozilla Firefox v3.07 achieved just over 30% block rate
  • Apple Safari v3 achieved 24% block rate
  • Google Chrome 1.0.154 achieved 16% block rate
  • Opera 9.64 achieved 5% block rate
  • Microsoft Internet Explorer v7 achieved 4% block rate

The study's methodology is however, greatly flawed at several key points, making its conclusions open to interpretation which should be the case when making such comparative tests.

For starters, NSS Labs undertook a rather minimalistic approach towards the definition of web malware. In this study, the malware URLs they're using are basically "links that directly lead to a download that delivers a malicious payload", a decision that directly undermines the statement of "block rate" in times when client-side vulnerabilities are massively abused courtesy of web malware exploitation kits. And since no live exploit URLs were taken into consideration, the DEP/NX Memory Protection feature within IE8 was naturally not benchmarked against known exploits-serving sites, or at least wasn't mentioned in the report.

Moreover, the competing browsers' use of SafeBrowsing's API, a combination of automatic (honey clients) and community-driven efforts to analyze a web site in a much broader "malicious" sense has a higher potential to maintain a more comprehensive database of known badware sites. It also comes as a surprise that Firefox, Safari and Chrome have such a varying block rates given that the browsers take advantage of the SafeBrowsing project's database. Basically, having a set of ten malicious URLs and running it against the browsers is supposed to return identical results due to the centralized database of known badware sites.

Interestingly, the study used Apple Safari v3 in order to come up with the 24% block rate, which excludes the built-in anti-phishing and anti-malware features introduced in Safari v4. The report is released prior ot IE8's debut, but even if NSS's study is in fact relevant in a real-life attack scenario, does it really matter that IE8's outperforms the rest of the browsers in times when IE8 users are downgrading to IE7? That very same IE7 which according to the study is offering "practically no protection against malware"?

Anyway, consider going through the report, with a salt shaker in hand.

Topics: Security, Social Enterprise

Dancho Danchev

About Dancho Danchev

Dancho Danchev is an independent security consultant and cyber threats analyst, with extensive experience in open source intelligence gathering, malware and cybercrime incident response.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Whaaaaat?

    <i>"does it really matter that IE8?s
    outperforms the rest of the browsers in times
    when IE8 users are downgrading to IE7?"</i>

    So, IE8 quickly picked up adoption throughout
    the week-end and suddently on monday it drops.

    I would expect that from Informationweek (who
    are actually behind this stupidity).

    But I'm really disappointed that you just
    repeat such obvious stupidity as gospel.

    What happens on monday mornings? Bingo! People
    go back to work. Wanna bet how many workplaces
    have upgraded to IE8 by now?

    Wanna bet what happens come friday aftenoon?

  • RE: Study: IE8's SmartScreen leads in malware protection

    hahahaha this is so funny.... I think NSS
    labs is a MS Partner
    • Right....

      Hahahaha.... wouldn't that be great? To be able to say Microsoft somehow skewed the results... ah well you could only hope. Too bad
  • RE: Study: IE8's SmartScreen leads in malware protection

    Again, Microsoft achieves greatness! I upgraded to IE8 on all my computers the day it came out and I couldn't be happier. I thought the accelerators and web slices were great, but knowing that IE8 tops the malware protection charts makes it that much better.
    Loverock Davidson
    • Um....

      69% isn't "greatness". It's abject failure, albeit less so than the others. But hey, keep that cheerleading handy when the first browser is able to hit in the high 90 percentile. Then it's time to pop a cork or 2.
      • So you are saying

        So you are saying that he should [b]not[/b] use IE8 because it [i]only[/i] got 69%?

        Since malware protection is important to him, what are the alternatives then?

        Firefox that scored less than half?
        That security disaster that is Safari?

      • I'll take a 31% failure rate

        ove a 95% failure rate any day of the week.
        • Defender

          feeds an awful lot of information back on malware attacks I would suspect. I'd say that gives Microsoft an edge in building a safe surfing database, although ideally they'd all work together and create one, open database. But that probably won't happen.

          I agree with honey though. IE8 would only logically be upgraded faster at home than at the office. So a drop from the weekend into a Monday shouldn't be surprising at all, and the fact that anyone would act like that is some sort of evidence of people uninstalling the browser is amusing. If the study is bogus then the article that was linked about IE8 dropping is down right perposterous.

          - meant to reply to story, oh well.
  • How ideology can't stand facts

    I mean how twisted do you need to get to spin an IE convincing win into a loss?

    I also have another reason as to why there was a short blip in IE8 takeup. That was all the ABMers downloading the new IE8, realising it blew the other browsers away and then uninstalling it as it disagrees with their ideology.

    Don't like that explanation? - then make up your own. It has as much validity as the author's fantasizing.

    Put IE8 on all my business's computers yesterday - no complaints, quite a few compliments.

    How pathetic. The safest way you can browse the web is to just use Firefox on Linux. Then you dont need malware protection because the OS is secure by itself. Microsoft's constant disregard for true security has lead to entire industries such as this one to pick up the slack.
    xunil skcor
    • Safer maybe

      But Linux being basicly useless doesn't make that possible for most people. Don't even bother responding how great it is noone uses it.
    • What a load of bollox!

      Who the hell wants Linux? what the friggin hell has that to do with the discussion.Get real...less that point two of one percent use the bloody operating system world wide..when we talk business OS we talk MS or MAC. Just for once stick your Linux where the monkey stuffs his nuts....politley that is.
      Richard Turpin
      • You're wrong.

        Linux is a force in the server market, and is a legitimate replacement for Windows on a desktop PC for home/casual use, in addition to powerful development and some business use.
        The market share of linux is probably closer to 8% considering the figures people use for linux are paid deployments. Even Microsoft agrees Linux has a larger home PC OS market share than Apple:
        (take a look at that graph, tehe.)

        I have rarely if ever heard of someone deploying Macintosh for business. My school uses linux and Microsoft OSes on their machines, in addition to Solaris. I see unix and linux used lots of places. I have never seen Apple machines when we talk business.

        tl;dr: You're simply wrong.
  • RE: Study: IE8's SmartScreen leads in malware protection

    I treid ie8,I do not know much about malware,but I had a few problems with IE8 so I uninstalled IE8,problem solved.
    • Problems

      Would you care to share what those problems are? Maybe somebody here can shed light on your issues.
  • I read the report

    The study only tested the browsers against sites that use social engineering rather than clickjacking or drive-by downloading. In fact, they went out of their way to exclude the latter two. One of the main reasons why i stopped using IE was because of the drive-by downloading that it was allowing. Since socially engineered malware distribution doesn't work on me, this report has no value for me.

    • IE8's smartscreen blocks drive-by downloading

      if you visit a known dangerous site, IE8 will block that page and any download from that site, so drive-by downloading is blocked.
      • Right, and it does this with Amazing reliability. The other browsers

        don't even come close. <br> Check out the results.
  • RE: Study: IE8's SmartScreen leads in malware protection

    We at NSS Labs clearly stated this is a study of socially engineered malware, and not exploits. It was the title of the report, and defined therein; perhaps it got overlooked accidentally by some readers. See:
    The points about DEP and memory randomization etc are good and important to protect against client exploits; but they don't apply to the scope of this test. That is the subject of a different test. The complexities of doing exploit, malware and phishing testing all in one harness together are simply problematic from an automation standpoint.

    I understand the expectation of SafeBrowsing. As proponents of opensource products, we use them every day, and had high expectations. But the data is correct. We went through extensive validation. This is why testing is important: should <> is. Now that there is an independent benchmark against live malware sites, developers across the board have more information which they can use to improve. We expect and hope that all of them will. And we'd be happy to assist.

    Safari 4 beta came out after we had locked in the test harness, validated the stability etc. And we excluded beta software from this test across the board. This is a completely fair practice as it?s beta for a reason. We're happy to evaluate products at any stage, but each test project has its parameters and cannot be changed in mid-flight.

    The sample size is the largest sample of live, validated web-based malware tested ever AFAIK. Our focus was freshness, so sites were fed into the system in < 2hr after detection. Actually the total sample was 4x that, but we were conservative on the validation. Thus it was not ?minimalistic.?

    More questions: no problem. We're giving an in-depth view of how we did the testing on this webinar.

    Rick Moy, NSS Labs
  • The headline was wrong

    This piece should have been titled "Study: IE8's SmartScreen leads in phishing protection", not "malware protection". It is a good thing, though; since IE is the browser most likely to be used by the clueless, it's very important that IE have good phishing protection.

    It's not a reason I would use it, however.