Comparing corporate AV solutions

The folks at AV-Comparativeshave done an interesting test. They took four of the AV products that they rate Advanced+ for effectiveness and compared the number of signature updates per week. Kaspersky Labs came out way on top with 150 updates per week on average over  June and July 2006.  ESET had 16, Symantec 7, McAfee 5.  That jibes well with Kasperky’s reputation for being the first out the door with a new signature. But is that the best measure of a good AV product? Most enterprises would certainly prefer fewer updates so they can schedule testing in a reasonable timeframe. The report from AV-Comparatives makes the argument that more frequent updates can lead to more false positives and corrected signatures in later releases.

 

There is also an interesting graph that puts ESET in a very favorable light.

From previous “comparatives” ESET has been shown to have the most effective behavior based detection and blocking of malware, something ESET prides itself on.  As you can see there is a correlation between the number of signatures released each week and the effectiveness of a product’s behavior based detection at least for three of the products tested. ESET’s Nod32 is the outlier with much more effective behavior blocking and yet a signature release schedule on par with the others. 

 

Despite the loud prognostications from industry pundits and analysts that signature based systems are doomed to die they are still with us.  There are lots of great reasons for that.  First of all, the output from a corporate AV solution is one of the most useful security metrics.  Second of all, few people feel comfortable with the idea of harboring malware on all of their computers that is prevented from being effective by a pure behavior based system but not identified and removed.   There is something comforting about deterministic signature based systems. 

 

At the same time there is one class of malware that I have written about that cannot be detected with pure signature based systems. (And I am not talking about so-called 0day threats.) Custom Trojans and viruses can use very old techniques and attack well known vulnerabilities. Many times they do not even require a vulnerability, just the user's culpability, to execute.  A custom Trojan such as the one Michael Haephrati wrote can be sent in to a target organization and used to steal intellectual property.  It may never be seen by the research labs of the AV vendors.  Countering custom Trojans requires a behavior based technique. ESET is on the right path: combine best of breed behavior based detection with a signature engine with comprehensive updates.