Beta of Microsoft's antispyware tech falls short in tests

Beta of Microsoft's antispyware tech falls short in tests

Summary: As Microsoft continues its own foray in to the security software business, critics (mainly supporters of the existing cottage industries) have argued that Microsoft will never to be able to build antivirus, antispyware, and personal firewall tools that are as good as those that come from the third party providers that are far more focused (as a percentage of the companies' overall efforts) on malware -- companies like Symantec, McAfee, and Zone Labs (a subsidiary of Checkpoint).

SHARE:
TOPICS: Security
52

As Microsoft continues its own foray in to the security software business, critics (mainly supporters of the existing cottage industries) have argued that Microsoft will never to be able to build antivirus, antispyware, and personal firewall tools that are as good as those that come from the third party providers that are far more focused (as a percentage of the companies' overall efforts) on malware -- companies like Symantec, McAfee, and Zone Labs (a subsidiary of Checkpoint).  Meanwhile, other industry observers see Microsoft's entries as being the death knell for third party products.  When I last asked long time Zone Labs executive Fred Felman for his assessment (Felman has exited the security business for now and is pursuing other opportunities),  the only thing he would say on the record is that he thinks the security business "is beat" right now (as in "out of gas").  That doesn't mean it can't find some successful niches (for example, products that focus on the needs of enterprises).  For those waiting to see how the rubber actually meets the road, Suzi Turner -- ZDNet's Spyware Confidential blogger -- has been conducting a series of exhaustive tests to see how well Microsoft's Windows Defender (currently in beta) holds up to other products that are designed to keep our systems spyware free.  While her tests are not finished yet, the results could be proving the critics of Microsoft's strategy correct.  Writes Suzi in her blog:

Windows Defender detected and removed approximately 65% to 75% of the spyware compared to SpywareDoctor and SpySweeper. Windows Defender left behind quite a few registry keys.  It did better with file removal than with registry clean up.

Windows Defender is the name of Microsoft's antispyware product.  It will be included for free in Windows Vista and a free download will be made available to users of Windows  XP SP2.  The two caveats to Suzi's conclusions so far are that Windows Defender is still in beta and that she's not done with her testing.  With a product that's in beta, anything can change. In her first round of tests, Suzi basically checked to see how good Windows Defender was at removing spyware after the fact (in other words, after it was already put onto the system).  Windows Defender also includes some realtime protection capabilities designed to catch spyware before it sneaks onto your system.  Between WD's removal capabilities and it's real-time protection capabilities, it may very well prove to be worth it's free price.  So stay tuned to Suzi's blog for her findings. 

On a related note, Suzi is conducting her tests using the virtual machine technology found in VMware's VMware Workstation.  In addition to the many reasons I've proposed that everyone should be using virtual machine technologies like VMware or Microsoft's Virtual PC, testing new software and Web sites is another one.  If the software doesn't work or that Web site turns out to be malicious, if you run your tests in a virtual machine, then those tests cannot negatively impact the rest of your system .   And speaking of malicious Web sites, Suzi found a new one today -- a Web site that poses as the provider of an antispyware tool called Spy-Shield, but that installs adware on your system.  Keep away (and where are the authorities... this is fraudulent!).

Topic: Security

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

52 comments
Log in or register to join the discussion
  • AdAware and Spybot were among the also-rans.

    The comparison was to the results of two other programs. Those programs found more problems, but I'm uncertain so far about the significance.

    The reason for my patience is the chance for false positives and the difference between traces and parts of the spyware. That's not doubt about the methodology, which is well explained. It's a caution on the need to wait for more thorough tests before drawing definite conclusions.

    An issue which can be considered now is whether the program could be considered a failure if it were to do only as well as AdAware or Spybot. To say that Defender is unacceptable in that case would be to say that AdAware and Spybot were also unacceptable.

    I think the basis for evaluation should be the ability to find and protect against standard spyware in normal use. AdAware and Spybot do that, and Defender may prove able to do as well.

    There should be software for special situations, and then the evaluation should be based on thoroughness.
    Anton Philidor
    • Correct premisses, odd conclusion

      You are right when you say "To say that Defender is unacceptable in that case would be to say that AdAware and Spybot were also unacceptable."

      And the two you have mentioned consistently under-perform on any test published since 2003. My favorite is one done by zdnet.au last year, when AdAware failed to remove most of the stuff it detected, it also failed to detected most of the stuff installed, but the scan was very fast!
      yuppie - it is fast, hence you should have it ;)

      Users should not be religious about what they use. If it works, great. If it does not work do not use it and get over it.
      duck01
  • Hmmm, the beta isn't up to par with mature products.

    Was this a big DUH moment for you?
    No_Ax_to_Grind
    • Hmmm...

      Didn't they initially get this product by buying GIANT, smacking the Microsoft name & logos on it and promptly proclaiming it a beta? Then change the name again? Not an indictment of MS...plenty of other folks like EMC, Cisco, Symantec, etc have done/continue to do this. Slapping a 'beta' tag on it does not dismiss the fact that at least in it's previous incarnation it was someone else's mature code that quite frankly didn't work very well for removal. I've been using it pretty extensively and give it a thumbs up for real-time protection of a clean system, but it simply doesn't cut it for cleaning up an already infested system. I give those a one-two shot of AdAware & SpyBot and a final exorcism with Ewido, then install MS before I hand the PC back to the owner. Just did my first Defender install last night...don't know what changes were made under the hood, but the install wizard and GUI have been changed...hope that isn't what qualifies it as a beta ;)
      Gasman_z
  • Defender Beta caused no problems for me so far.

    I've only ran one quick scan on my system (before the %$#@! updates paralyzed my PC again), and while it found nothing it didn't crash, abend, or kill my hard drive. This could actually work, as long as I can keep the signatures up to date.

    Then again, I also use ZoneAlarm Pro, Avast!, FireFox, a HOSTS file, and some good surfing sense. So Defender may be looking for something to do while everything else keeps the crapwarez at bay.
    Mr. Roboto
  • Beta Falls Short...

    To give credit where credit is due (from the second to last paragraph of the articl): "The two caveats to Suzi's conclusions so far are that Windows Defender is still in beta and that she's not done with her testing. With a product that's in beta, anything can change."

    Zealots on all sides ought to get a dopeslap if a flamewar starts in this thread.
    Tert
  • And the beat goes on

    So much for "Trustworthy Computing" :P

    Aw, but I'm being hard on Redmond. They'll gt it right, just like
    they said they would with Win 2k and XP.
    dropzone@...
    • RE: And the beat goes on

      >>...Aw, but I'm being hard on Redmond. They'll gt it right, just like they said they would with Win 2k and XP....<<

      Yeah, right! Just like outlook express and IE. I don't use Windows that much, but I think that when I do, I will stick with 3rd party browsers, AV, Anti Spyware/Malware, and Firewall.
      richdave
  • anti-spyware

    So long as there is a population of people who think that they will get something for nothing, then companies will continue to decieve. To think for a moment that Microsoft, or any other company is offering us "something for nothing" is absurd. They are in the business to pry money from our hands... period. We will pay, and continue to pay one way or another.
    stenodoctor
    • While I agree with you

      That there is always going to be a sucker. I just don't want to see the adds flooding in and filling up my disk sectors. It's just like what happens to the junk mail that comes in it goes in the recycle bin which is to say it wastes resources, time and energy. As for the spyware people they are just theives in one way or the other. And should be in jail.
      maldain
  • Conumdrum

    Microsoft has built into its products backdoors, user uninteruptible
    reporting, automated code modification, permissions disabling, on
    and on. Until Microsoft changes its' business model it is, by
    definition, going to be an insecure computing environment.
    Techscan
  • Beta MS Antispyware

    I have had the beta in my system for a couple of months now, and it has never caught a thing. I run Adaware right behind it and catch at least 3 or more things. Vaporware as far as I am concerned.
    tejasmed
    • You are correct: It never catches anything

      I have an office PC here, behind our own firewall and spam filters. I still get junk e-mail despite a spam fileter, and still get annoying popups and spyware deposited on my hard drive without fail.
      I have been using the MS anti-spware product for approximately 3 months - and it has never caught one thing. It runs every night at 2 AM. Never, I repeat NEVER found anything. I run Ad-Aware every weekend, it always catches something (not as bas as the home PC, but that's another tale).
      I will wait until the MS product goes GA, and if no changes, it gets Dee-leeted . . .
      apapaleo@...
      • It alerts because of Cookies

        You might want to also realize, that the reason that Ad-Aware always catches something, is because they lump in cookies as being a large security threat (they are NOT!).

        That's why I finally deleted Ad-Aware and Spybot, and switched over to Microsoft Anti-Spyware and now, Windows Defender.
        I got so fricking tired of the cookie warnings, that I felt like ripping the hard drive out of my computer every time it alerted for one of them.
        Leria
  • Please don't put Zonelabs on that list...

    They quit supporting a functional ZoneAlarm Pro product about a year and a half worth of releases ago (I use to pay for 3 yearly subsctiptions just for my home pcs * 3 years). It used to be the best, but now it must be some checkpoint executive's red-headed step-child.
    Brakiss
    • Huh??

      Not to hijack the thread, but what do you mean by "They quit supporting a functional ZoneAlarm Pro product about a year and a half worth of releases ago". I still pay for the releases... I get updates regularly. Is there something that I missed about ZA? Thanks for any info. Regards, John
      heli@...
      • Huh??

        Brakiss post reads like a patent troll. Use Zone Alarm Pro at work and Zone Alarm Free at home. I have not noticed a drop off in quality or support. Still the best software firewall available in both the free and pro versions. Bar none.
        richdave
        • Huh??

          richdave... tyvm. I thought that I missed something. Much appreciated!
          heli@...
    • free

      use the free version. there are others out there.
      windowsovermac
  • Microsoft is expert in INSECURITY

    Their programmers are totally incapable to develop antispyware. The solution is that they buy some company that already has developed the product.
    jolumoar