Microsoft vs. McAfee: How free antivirus outperformed paid

How effective is free antivirus software? I had a chance to see a real, in-the-wild example just this month, and the results were, to put it mildly, unexpected. The bottom line? Microsoft’s free antivirus solution found and removed a threat that two well-known paid products missed. Here are the details. [Update: After I publlished this post, a second example appeared, courtesy of a rogue commenter in the Talkback section. See the results at the end of this post.]

I’ve had Microsoft Security Essentials (MSE) installed on my main working PC for most of the past year. Mostly, I use it for real-time protection. I typically disable the scheduled virus scans on my PCs and instead occasionally do a manual scan just to confirm that nothing out of the ordinary has snuck through. Last month I decided to perform a scan using the Full option. Because I have 2.5 terabytes of hard disk space, with roughly 40% of it in use, I knew the scan would take a long time. So I scheduled it to run while I was out running errands.

When I came back, here’s a snippet of what I found:

MSE had detected several files files that it considered malicious. One was a rigged PDF file (not shown here). The other was a single file in the Java cache folder on this system that contained three separate exploits. Using the information in the MSE history pane, I found the file and uploaded it to Virustotal.com, which is a free service that allows you to scan a suspicious file using 43 separate antivirus engines. The file, identified by a unique hash, had already been analyzed, so I got the results immediately:

Only 17 of 43 antivirus products detected this as a threat. The full results page showed the identification, if any, for each product on the list. Microsoft, Symantec, Avast, and F-Secure were among the engines that flagged the file. But the majority didn’t. That means one of two things. Either the file was a false positive, and I was about to delete something harmless and perhaps even necessary. Or it was real, and most AV programs were missing it.

To get to the bottom of the issue, I sent e-mail messages to contacts at three companies. I asked Microsoft to reanalyze the file and confirm that it was indeed malicious. I also asked McAfee and Sunbelt to look at the file; both of them had reported the file as clean, according to VirusTotal.

Microsoft had two analysts review the file. Here’s a portion of their response:

We have confirmed that the threat detection you received from Microsoft Security Essentials is indeed valid. There were more than 3.5 million reported CVE-2008-5353 attacks in Q3 2010, and Java vulnerability exploitations like these, while once a rare occurrence, have spiked this year. … [T]his exact file is something we have seen in the wild more than 40,000 times in the past six months.

This October 18 post by Holly Stewart on the Microsoft Malware Protection Center blog provides useful additional detail on why these types of attacks can be challenging for IDS/IPS vendors, as well as the steps customers should take to ensure that they are protected.

According to the scan results, this threat was first identified in definition 1.85.1774.0, which was released by Microsoft on July 9, 2010.

McAfee responded quickly to my e-mail as well. A spokesperson sent this reply:

Our Labs team took a look at the file you referenced and it is malicious. We are in the process of developing new heuristics to combat the effects from a stream of recent malicious JAR files more proactively, the file corresponding with the hash you mentioned is in the queue.

Sunbelt’s Malware Response Manager, Dodi Glenn, reported that this file was in the company’s repository and submitted it for detailed analysis. Here are the results:

This file contains a malicious java.class … that exploits the CVE-2008-5353 vulnerability. … We are currently testing our updated detection for this exploit and expect to release it shortly.

The good news is that my system wasn’t compromised in any way. The exploit in question was blocked by a Java update that I had installed last year. Likewise, the booby-trapped PDF file (which all of the antivirus programs detected) relied on the user having a very outdated version of Adobe Reader installed, and mine was fully up-to-date.

Last week, when I wrote about Microsoft’s decision to expand its distribution of Microsoft Security Essentials via Microsoft Update, McAfee complained that free software simply isn’t as good as its paid protection. Here’s what a spokesperson told me:

McAfee wants consumers to be safe online. Options that provide an elementary level of security are free products including Microsoft Security Essentials, however these mostly rely on traditional protection mechanisms.  McAfee products offer not only more features but most importantly, McAfee products offer real-time protection using cloud-based Global Threat Intelligence to combat even the most sophisticated threats thus ensuring complete protection and peace of mind.

In this case, at least, that protection wasn’t as complete as the free Microsoft product it was comparing itself to.

As an aside, it’s worth noting that criticizing Microsoft Security Essentials because it’s free misses an important point. MSE uses the same scanning engine and definitions as its enterprise-grade Forefront product, which is most assuredly not free.

One certainly shouldn’t draw definitive conclusions from a single anecdotal example, but as this case shows, the gap between antivirus products isn’t as simple as free versus paid, and even the best and brightest researchers can miss a threat.

Update 15-Nov 7:00AM PST: Another real world example just dropped into my lap. A commenter in the Talkback section of this thread posted a link to a news website claiming to offer a video of the full Sunbelt report. (The malicious comment and link were deleted almost immediately.) Visiting that page (which is hosted on a legitimate website that has clearly been compromised) displayed a video window with the message “Sorry, this video cannot be played. Problem: plugin is not found.” It then helpfully included a “Download plugin” link. Here’s what the browser displayed:

Of course, this is one of the oldest tricks in the malware book. The link leads to an executable file, which I downloaded (but did not execute) on a system that was not running any antivirus software and submitted to Virustotal.com. The result? 15/43 scanning engines detected it as malware. Microsoft Security Essentials was one of them. It identified the file as TrojanDownloader:Win32/Waledac.C, which was originally included in definition file 1.63.2017.0, released on August 27, 2009. The McAfee Gateway edition identified it as a suspicious file (and thus would have blocked it). McAfee’s consumer product line did not detect the threat at all.

To its credit, Sunbelt successfully identified this threat. On the list of companies that missed it? Symantec, Avast, and TrendMicro.