DEP - A missed opportunity to protect millions of Windows users

DEP - A missed opportunity to protect millions of Windows users

Summary: Imagine coming across someone who had both an antivirus package and firewall software installed on their PC and yet both were switched off. You'd think that they were pretty dumb, way too brave or a little bit crazy (or they are antivirus researchers!). But the fact is that there are literally millions of Windows XP SP2 users who have a defense mechanism in place that would protect them against many of the vulnerabilities that threaten them, but that protection is, by default, partly disabled.

SHARE:
TOPICS: Security
8

Imagine coming across someone who had both an antivirus package and firewall software installed on their PC and yet both were switched off.  You'd think that they were pretty dumb, way too brave or a little bit crazy (or they are antivirus researchers!).  But the fact is that there are literally millions of Windows XP SP2 users who have a defense mechanism in place that would protect them against many of the vulnerabilities that threaten them, but that protection is, by default, partly disabled.

I'm talking about a technology called Data Execution Prevention.  DEP is a feature that is built into Windows (you must have SP2 installed to take advantage of this) that prevents an application or service from executing any code that resides in a non-executable memory region.  The idea is that this technology will halt buffer overflows in their tracks.

There are two kinds of DEP:

  • Hardware-enforced
  • Software-enforced

By far the most effective form of DEP is hardware-enforced DEP.  This relies on having a CPU that supports the NX or XD bit.  Modern AMD processors support NX (which stands for No eXecute) while modern Intel CPUs support XD (which stands for eXecute Disable).  Both features carry out the same function and differ only in name.  If you don't have a CPU that understands NX/XD then you are limited to the inferior software-enforced DEP and you'd have to upgrade the CPU or buy a new PC if you wanted to use hardware-enforced DEP. 

It's important that I point out that DEP will not secure you from any malicious applications that you yourself choose to run, it only offers a defense against buffer overflows found used by hackers to run malicious code.

OK, so what's the problem with DEP?  Why is it a missed opportunity? Well, by default it isn't set up to offer you the best protection.  Instead it only monitors and protects you from malicious applications trying to leverage essential Windows programs and services.  In the default DEP configuration doesn't offer a great deal of protection.  Will things get better when Windows Vista is out?  Nope.  The defaults are set up in exactly the same way.

Why is DEP set up this way by default?  Because there are a lot of badly written applications out there that routinely execute data as code.  This triggers false alarms, which can be annoying.  Fortunately you can add exceptions for any misbehaving applications you come across.

So how do you fully enable DEP? You can do it with just a few clicks on Windows XP:

  • Click Start > Control Panel
  • Click on Performance and Maintenance (if you are in Classic View, skip this step)
  • Click on System
  • Click on the Advanced tab
  • In the Performance group, click on Settings
  • On the Performance Options dialog, click on the Data Execution Prevention tab
  • Click on Turn on DEP for all programs and services except those I select
    Switching on DEP
  • Click OK
  • Click OK to confirm that the system will need to be restarted
  • Finally, reboot the system

Here's what you'll see if your CPU doesn't support hardware-enforced DEP:

DEP

The process for fully activating DEP on Windows Vista is a little different:

  • Click Start > Control Panel
  • Click System and Maintenance
  • Click on System
  • Click on Advanced system settings
  • Click Continue on the User Account Control dialog that will be generated
  • In the Performance group, click on Settings
  • On the Performance Options dialog, click on the Data Execution Prevention tab
  • Click on Turn on DEP for all programs and services except those I select
  • Click OK to confirm that the system will need to be restarted
  • Click OK
  • Finally, reboot the system

Once you’ve rebooted, you can test that DEP is working by downloading and running a small utility called NXTEST by Robert Schlabbach.

NXTEST 1

NXTEST 2

So, what can DEP protect you against?  Well, there have been three big security scares this year that have been stopped in their tracks by hardware-enforced DEP.  These include the WMF vulnerability from the beginning of the year and the latest VML vulnerability affecting Internet Explorer.  You should never rely solely on hardware-enforced DEP to protect you against malicious code, but given that the detect rate for the VML vulnerability is still pretty awful, it's a handy safety net to be running.

Topic: Security

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

8 comments
Log in or register to join the discussion
  • How do you completely

    disable it?
    bjbrock
    • How do I get DEP to allow HP Printer software to work?

      I just had a call from a client that could not get her HP printer to
      print. It would shoot out blank sheets. I went in and turned off "DEP
      all" and the printer now works fine. I have not been able to figure
      out where the offending .exe is.
      Zoraster
      • Any warnings?

        Or just crashes? The DEP dialog box should list applications that have triggered DEP to step in and give you an option to define an excemption..
        Adrian Kingsley-Hughes
        • No warnings or crashes.

          The printer just pushes out blank pages. I found numerous others
          having the same problem doing a search on the web. But HP's web
          site makes no mention of this glaring defect in their software. So
          far I have been able to see 3 or 4 processes start when I initiate a
          print job. I have not yet had time to track them all down and create
          a list. For now this client has to leave DEP set to Windows processes
          only.
          Zoraster
          • No DEP dialog either

            DEP does not give any dialog either. The printer acts like it got the
            job but no head movement just blank paper. The HP Assistant that
            shows ink levels opens when DEP is off. But with DEP on the dialog
            window does not open. The print manager sees the job sent,
            processed and finished and the job appears in the completed list.
            Zoraster
    • Tons of info here

      http://support.microsoft.com/kb/875352
      Adrian Kingsley-Hughes
      • Been there, I need to call HP.

        I already know how to fix it using dep and setting it to ignore
        the specific .exe that causes the problem. The printer is an HP
        Deskjet 5500 series and with DEP on The printer activates and
        trys to print but it spits out blank pages (No inkjet head
        movement). With DEP set to Windows files only everything is
        fine. I need to hunt down all the applications that get triggered
        by the print command and one by one give each an exception to
        DEP until I hit the one offending .exe and then remove the
        others. Someone really needs to compile a list of processes that
        will not run with DEP on. And HP really needs to do some work
        on it's software. The Printer companies are really bloating up
        what needs to be installed. Do I really need a low ink indicator
        that launches my browser to sell me ink at an inflated price?
        Zoraster
  • How do you tell the difference between...

    ... a benign DEP exception from a badly-written program, and someone exploiting a dangerous buffer-overflow bug which can completely compromise your system?

    I enabled full DEP on both of my sisters' PCs, but last time I looked, I remember seeing one exception enabled on one machine. I didn't have time to investigate precisely what the exception had been, but it seems to me that the only way that DEP can be [b]truly[/b] useful is thru "tough love". In other words, anything that triggers a DEP exception should be forbidden from running in anything other than a sandbox. (Or something similar.)

    Someone somewhere needs to give application vendors a strong incentive to fix buggy code!
    Zogg