Windows Defender Beta 2 vs. spyware

Windows Defender Beta 2 vs. spyware

Summary: As promised a few days ago, I finally got a virtual machine upgraded to Service Pack 2 for testing Windows Defender Beta 2. For the sake of convenience, I'll refer to it as WD for most of this post.

SHARE:
TOPICS: Malware
41

As promised a few days ago, I finally got a virtual machine upgraded to Service Pack 2 for testing Windows Defender Beta 2. For the sake of convenience, I'll refer to it as WD for most of this post. When I wrote about WD previously, I mentioned the review at PCMag.com where WD was tested against 6 keyloggers, which is not a particularly valuable test in my opinion.

The tests were done on a virtual machine with Windows XP with SP2, fully patched, running in VMware Workstation 5.5.1. Testing consisted of two parts. For the first test, I had WD running with all components of real-time protection turned on. I surfed to Claria's website and downloaded two Claria apps, GotSmiley and a screensaver. When I downloaded the apps, Windows Defender presented an alert and asked whether or not to remove, get more information or ignore.  I chose ignore and allowed the installation. After installation, I did the full scan and WD detected both apps correctly and asked me to select an action.

In the second test, I went to a website known to spyware researchers as a consistently reliable source of spyware. Immediately prior to going to the site, I ran InCtrl5 in order to track changes to the system. I turned off WD's real-time protection for this test so I could test scan and removal capabilities. I had to restart the test twice because the vm quickly became so infested it froze. On the third try, after about 5 minutes on the site, I disconnected NAT, killing the internet connection for the vm, so I didn't lose control of the machine. Before running any scans I ran InCtrl5 again. In less than 6 minutes, the spyware had added 230 registry keys, deleted 32 keys, added 386 values, deleted 82 values, changed 46 values, added 16 folders, and added 389 files. I ended up with the following:

SpySheriff
QuickLinks
CmdServices, also known as Command
ZToolbar
AzeSearch
NetMon aka Network Monitor
Paytime.exe, related to CoolWebSearch
AvenueMedia/Internet Optimizer also known as DyFuCa
Targetsavers
SurfSideKick
Smitfraud-C
CAS-Client (ConsumerAlertSystem)
AproposMedia
Trojan.VB.TG
Trojan.Downloader.VB.TW
Trojan.Tofger.CD
TagASaurus, aka enbrowser
Trojan.StartPage.GEN
ADSlime
W32.Spybot.Worm
Look2Me
drsmartload1.exe  aka Troj/Drsmartl-N
MoneyTree Dialer
Service: Windows Overlay Components - file name C:\WINDOWS\tihotdj.exe, aka Trojan.Adclicker
My homepage was changed to c:\secure32.html

Besides checking the InCtrl5 log, I ran several anti-spyware apps and used Google search to identify the spyware programs and files. I ran the full scan rather than the quick scan on each app. Ad-aware SE (free version) identified 141 critical objects. Spybot Search & Destroy identified 13 unique spyware programs which included the files and registry keys but I didn't get a total count of the traces. SpywareDoctor identified 484 traces and SpySweeper identified 501 traces. No removals were performed. Differences in scan reporting and the way some traces are labeled account for some of the differences in the scan results, but obviously SpywareDoctor and SpySweeper out performed Ad-Aware and Spybot Search and Destroy. These numbers do not include cookies.

Then I ran Windows Defender's full scan and allowed it to perform the default action for each of the 24 threats it detected. At that point I saved a snapshot of the infected vm. The next day I started the vm again, keeping NAT disconnected except for a few minutes while I uploaded  some files to scan at jotti. During that time the remaining spyware managed to download another rogue app, AdwareSheriff, which is very similar to SpySheriff. I scanned again, this time with just SpywareDoctor and SpySweeper. I subtracted the files and registry keys related to AdwareSheriff from the numbers to determine how many traces were not removed by Windows Defender in comparison to SpywareDoctor and SpySweeper. On the follow up scan, SpywareDoctor reported 170 traces (after I subtracted the traces related to AdwareSheriff), and SpySweeper reported 127 traces. 

Conclusions? Windows Defender detected and removed approximately 65% to 75% of the spyware compared to SpywareDoctor and SpySweeper. Windows Defender left behind quite a few registry keys.  It did better with file removal than with registry clean up. WD failed to remove some spyware that initiated the download of AdwareSheriff when I reconnected the vm to the internet. 

I plan to do another test with Windows Defender by going to the same website with real-time protection enabled. We'll see how well Windows Defender Beta 2 protects from real spyware in the wild.

Update 6:45 PM: In the talkbacksI said I would post the spyware applications that Windows Defender did not remove. Here's the list:

QuickLinks
toolbar.exe (I couldn't definitely identify what spyware program this belongs to. The search results bring up several different apps with files by that name.)
ADSlime
files named tool1.exe, tool2.exe. tool3.exe, tool4.exe, which are labeled as different apps depending on which vendor's description you read
drsmartload1.exe  aka Troj/Drsmartl-N
Look2Me executable

One problem with some of these files, or traces, is that different vendors give the apps different names. I might scan the same file, or set of files and registry keys, with 3 different anti-spyware or antivirus programs get 3 different names for that spyware app.  Also, sometimes the same traces are used in more than one spyware app.  For example, the rogue anti-spyware apps share many of the same files and registry keys and even look the same.  See the screenshots here of the SpySheriff family of rogue anti-spyware apps. All the apps in that group share many of the same files and registry keys.  There has been talk in the industry of sharing samples and using naming conventions, but that may be a long way in the future if it ever would happen. 

Designing tests of anti-spyware programs against real spyware is a challenge. Perhaps the most comprehensive anti-spyware testing that's been done was nearly 1 1/2 years ago, by Eric L. Howes. His test results are outdated now because spyware and anti-spyware have changed, but his methodology was very sound and worth reviewing. I'm not aware of any other comparable tests having been done since that time.  You can see his work here.

Topic: Malware

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

41 comments
Log in or register to join the discussion
  • Traces or Apps ?

    Interesting. Is that "65% to 75%" number the traces or actual Spyware apps that were prevented from running again? I tend to subscribe to the theory that knocking out the spyware app is what counts, not necessarily all of the traces. Trace removal is "nice to have" but not critical.
    klawrence
    • executables

      The 65% to 75% represents traces, the majority of which were registry keys. There were executables left behind for several apps, which allowed them to continue to run. I'll update the post later today and run down what was left behind. I agree with you, knocking out the apps, or executables, is more critical.
      Suzi_z
      • but don't count out traces

        While I agree that killing the executables is much more important with spyware, make sure you don't go soft on traces. Some traces will resurrect a program (although, technically you could consider a trace with this function part of the executables of the program). I'm looking forward to hearing more of your findings. One of these days I'll find some time to mess around with WD.
        RPMayhem
    • I have to agree on the traces

      I have to agree on the traces. She should have really put them into two categories, actual spyware apps removed, and traces of spyware apps removed.

      Those are two different things completely.

      The traces, every once in a blue moon, will make your computer download more spyware/adware, but those traces are few and far between.
      Leria
    • Identify before install?

      Also be interesting to know if Defender identifies and prevents the installation of anything the program can't remove after it is installed.

      Removal and identification are different.
      Anton Philidor
      • next test

        I agree, they are two different things. I said on the blog I plan to test the real-time protection soon.
        Suzi_z
        • True.

          Be interesting to see efficiency of removal when the program knows it's there.

          Also, having just done battle for a friend, I think Sun's Java is the most vulnerable part of a pc. Evaded everything except an online scan.
          And I'm still looking for a setting that will empty Java when the browser closes.
          Hope your test includes something that sneaks into Java.

          Thanks for considering this.
          Anton Philidor
          • Preventing Java infections

            Just go into the Java control panel and set the size of the Java cache to 0.
            Greenknight_z
  • Good Test.

    I am suspecting that Microsoft could make a killer antispyware app if they actualy put serious effort into it. WHo can collect the requiered data easier then Microsoft, and who can put the manpower on the job easier then Microsoft? Question is, will they bother to do it. Im useing Windows defendor right now, and if it dosnt improve Im moveing on, Microsoft could do the requiered work if they want to, articles like this prove they need to get on it, so Billy G., do something right now you have a chance.
    Cayble
  • Nice job!

    Good information, even if Defender is just a beta.
    No_Ax_to_Grind
    • I'm impressed...

      Windows Defender seriously kicks ass...
      A_Pickle
  • Killing the Beasts

    I am a professional spyware fighter in NC. Suzi is one of my favorite resources.

    I have been working on this Win-XP PC for several days during which I ran all the updated antispyware programs:

    Ad-aware, Ewido, A-Squared, Counterspy, SpySweeper, TrendMicro, HiJackThis, AVG, plus all my registry cleaners many times in safe mode and normal mode. When it finally came up clean, I was ready to deliver this PC.

    Then I decided to download and run Microsoft Defender and leave it installed. Below are the results.

    I am surprized that our best software did not catch these two components. I am also impressed with how much progress that MSFT has made in improving its detection program.

    Where did I go wrong?

    Zonny
    =========================================

    Category: Browser Modifier Description: This program has potentially unwanted behavior.
    Advice: Remove this software immediately.
    Resources:
    regkey: HKCU@S-1-5-21-2704639424-135449575-2542918944-500\SOFTWARE\MICROSOFT\INTERNET
    EXPLORER\MENUEXT\&Define

    regkey: HKCU@S-1-5-21-2704639424-135449575-2542918944-1006\SOFTWARE\MICROSOFT\INTERNET
    EXPLORER\MENUEXT\&Define

    regkey: HKCU@S-1-5-21-2704639424-135449575-2542918944-1003\SOFTWARE\MICROSOFT\INTERNET
    EXPLORER\MENUEXT\&Define

    iemenuext: HKCU@S-1-5-21-2704639424-135449575-2542918944-500\SOFTWARE\MICROSOFT\INTERNET
    EXPLORER\MENUEXT\&Define

    iemenuext: HKCU@S-1-5-21-2704639424-135449575-2542918944-1006\SOFTWARE\MICROSOFT\INTERNET
    EXPLORER\MENUEXT\&Define

    iemenuext: HKCU@S-1-5-21-2704639424-135449575-2542918944-1003\SOFTWARE\MICROSOFT\INTERNET
    EXPLORER\MENUEXT\&Define

    ===============================================
    Category: Trojan Description: This program has potentially unwanted behavior.
    Advice: Remove this software immediately.

    Resources: regkey: HKCU@S-1-5-21-2704639424-135449575-2542918944-500\software\hiwire

    regkey: HKCU@S-1-5-21-2704639424-135449575-2542918944-1006\software\hiwire

    regkey: HKCU@S-1-5-21-2704639424-135449575-2542918944-1003\software\hiwire
    Zonny
    • Killing beasts is fine....

      But we really need to kill the sites that they are being installed from. Maybe have a special group of industry people that are empowered to shut down sites with any means necessary that install spyware and adware without any notice.

      I also wonder if she was using IE7. Since switching to it, I have not had any problems with adware/spyware programs getting installed on my machine.
      Leria
      • not IE7

        Leria, I was not using IE7, but that would be a good test.

        Killing the sites is hard because a lot of them are located outside of US juristiction, in places like Russia and the Ukraine. It would take an international effort.
        Suzi_z
      • ActiveX

        IE7 just about turns activeX off, a prime vulnerablility of IE and a red carpet for the bad guys. I hope Microsoft reconsiders and makes IE7 available to Windows 2000 Professional users as well as Vista and XP-SP2 users.
        WiredGuy
    • Thanks

      Zonny, thanks for the kind words. I've always said that users need more than one anti-spyware program. No one anti-spyware app will ever detect and remove all spyware. I recommend people have 2 or 3.
      Suzi_z
      • I've found that knowing what you're running help

        If you know that your install should have certain number or processes running then you can determine when new ones appear and should be removed.

        Of course this gets difficult if you install a lot of new software and change your hardware frequently. Still spyware is easily detected for most types.

        Personally I like to see laws that say software vendors must tell me exactly what is being installed and what processes will run. At least that way you can see what should be killed and removed if it doesn't match up.
        voska
  • defender vs. orig MS AntiSpyware

    I think the original microsoft antispyware was better in that it showed warnings when questionable activity took place: home page changes, startup program changes, etc... maybe I'm not using Defender correctly, but it hasn't done anything similar and when I've thrown some
    basic security tests at it, it did nothing. So,
    I've removed Defender from my PC and reloaded the original... It 'expires' on July 31,2006, whatever that means. I guess I'll pay Webroot for SpySweeper on August 1st!

    MS Antispyware.....
    hopperdave2000@...
    • You might also try CounterSpy

      I have had better luck with CounterSpy. It's based on the same technology as the original MS-Antispyware (MS licensed the tech from them), but it has been signifcantly improved over the version that Microsoft shiped and runs on all versions of Windows instead of just XP/sp2.
      WiredGuy
      • CounterSpy

        WiredGuy, I do consulting for Sunbelt Software, so I can't comment on CounterSpy. It would be a conflict of interest to do so.
        Suzi_z