Windows Defender Beta 2 vs. spyware

As promised a few days ago, I finally got a virtual machine upgraded to Service Pack 2 for testing Windows Defender Beta 2. For the sake of convenience, I'll refer to it as WD for most of this post. When I wrote about WD previously, I mentioned the review at PCMag.com where WD was tested against 6 keyloggers, which is not a particularly valuable test in my opinion.

The tests were done on a virtual machine with Windows XP with SP2, fully patched, running in VMware Workstation 5.5.1. Testing consisted of two parts. For the first test, I had WD running with all components of real-time protection turned on. I surfed to Claria's website and downloaded two Claria apps, GotSmiley and a screensaver. When I downloaded the apps, Windows Defender presented an alert and asked whether or not to remove, get more information or ignore.  I chose ignore and allowed the installation. After installation, I did the full scan and WD detected both apps correctly and asked me to select an action.

In the second test, I went to a website known to spyware researchers as a consistently reliable source of spyware. Immediately prior to going to the site, I ran InCtrl5 in order to track changes to the system. I turned off WD's real-time protection for this test so I could test scan and removal capabilities. I had to restart the test twice because the vm quickly became so infested it froze. On the third try, after about 5 minutes on the site, I disconnected NAT, killing the internet connection for the vm, so I didn't lose control of the machine. Before running any scans I ran InCtrl5 again. In less than 6 minutes, the spyware had added 230 registry keys, deleted 32 keys, added 386 values, deleted 82 values, changed 46 values, added 16 folders, and added 389 files. I ended up with the following:

SpySheriff
QuickLinks
CmdServices, also known as Command
ZToolbar
AzeSearch
NetMon aka Network Monitor
Paytime.exe, related to CoolWebSearch
AvenueMedia/Internet Optimizer also known as DyFuCa
Targetsavers
SurfSideKick
Smitfraud-C
CAS-Client (ConsumerAlertSystem)
AproposMedia
Trojan.VB.TG
Trojan.Downloader.VB.TW
Trojan.Tofger.CD
TagASaurus, aka enbrowser
Trojan.StartPage.GEN
ADSlime
W32.Spybot.Worm
Look2Me
drsmartload1.exe  aka Troj/Drsmartl-N
MoneyTree Dialer
Service: Windows Overlay Components - file name C:\WINDOWS\tihotdj.exe, aka Trojan.Adclicker
My homepage was changed to c:\secure32.html

Besides checking the InCtrl5 log, I ran several anti-spyware apps and used Google search to identify the spyware programs and files. I ran the full scan rather than the quick scan on each app. Ad-aware SE (free version) identified 141 critical objects. Spybot Search & Destroy identified 13 unique spyware programs which included the files and registry keys but I didn't get a total count of the traces. SpywareDoctor identified 484 traces and SpySweeper identified 501 traces. No removals were performed. Differences in scan reporting and the way some traces are labeled account for some of the differences in the scan results, but obviously SpywareDoctor and SpySweeper out performed Ad-Aware and Spybot Search and Destroy. These numbers do not include cookies.

Then I ran Windows Defender's full scan and allowed it to perform the default action for each of the 24 threats it detected. At that point I saved a snapshot of the infected vm. The next day I started the vm again, keeping NAT disconnected except for a few minutes while I uploaded  some files to scan at jotti. During that time the remaining spyware managed to download another rogue app, AdwareSheriff, which is very similar to SpySheriff. I scanned again, this time with just SpywareDoctor and SpySweeper. I subtracted the files and registry keys related to AdwareSheriff from the numbers to determine how many traces were not removed by Windows Defender in comparison to SpywareDoctor and SpySweeper. On the follow up scan, SpywareDoctor reported 170 traces (after I subtracted the traces related to AdwareSheriff), and SpySweeper reported 127 traces. 

Conclusions? Windows Defender detected and removed approximately 65% to 75% of the spyware compared to SpywareDoctor and SpySweeper. Windows Defender left behind quite a few registry keys.  It did better with file removal than with registry clean up. WD failed to remove some spyware that initiated the download of AdwareSheriff when I reconnected the vm to the internet. 

I plan to do another test with Windows Defender by going to the same website with real-time protection enabled. We'll see how well Windows Defender Beta 2 protects from real spyware in the wild.

Update 6:45 PM: In the talkbacksI said I would post the spyware applications that Windows Defender did not remove. Here's the list:

QuickLinks
toolbar.exe (I couldn't definitely identify what spyware program this belongs to. The search results bring up several different apps with files by that name.)
ADSlime
files named tool1.exe, tool2.exe. tool3.exe, tool4.exe, which are labeled as different apps depending on which vendor's description you read
drsmartload1.exe  aka Troj/Drsmartl-N
Look2Me executable

One problem with some of these files, or traces, is that different vendors give the apps different names. I might scan the same file, or set of files and registry keys, with 3 different anti-spyware or antivirus programs get 3 different names for that spyware app.  Also, sometimes the same traces are used in more than one spyware app.  For example, the rogue anti-spyware apps share many of the same files and registry keys and even look the same.  See the screenshots here of the SpySheriff family of rogue anti-spyware apps. All the apps in that group share many of the same files and registry keys.  There has been talk in the industry of sharing samples and using naming conventions, but that may be a long way in the future if it ever would happen. 

Designing tests of anti-spyware programs against real spyware is a challenge. Perhaps the most comprehensive anti-spyware testing that's been done was nearly 1 1/2 years ago, by Eric L. Howes. His test results are outdated now because spyware and anti-spyware have changed, but his methodology was very sound and worth reviewing. I'm not aware of any other comparable tests having been done since that time.  You can see his work here.