IDS: Keeping watch

For most of its short history, the network and information security industry has aimed to create a static defensive perimeter--the electronic equivalent of a fortified wall. But that wall is far from impenetrable; the size and complexity of modern networks can make it difficult for an administrator to even know where the perimeter is, much less secure it. Moreover, most successful security breaches are perpetrated by the company's own employees, partners, or clients--attackers who start out inside the defensive perimeter.

The development of intrusion detection systems (IDSs) in the late 1990's brought real-time detection and response within the grasp of most mid-to-large sized businesses. Operating on the assumption that at some level an attack looks different from legitimate activity, IDSs automatically collect and analyze different types of data from various sources throughout the network. By monitoring activity as it happens, the IDS can identify suspicious behavioral patterns and either notify network administrators, initiate an automated response to the perceived attack, or both. Administrators can then act to counter a specific attack and/or tailor defenses to defeat similar attacks in the future.

Network-based IDSs (NIDS)--such as NFR's NID, Internet Security System's RealSecure Network Sensor, Intrusion.com's SecureNet Pro, and the open-source application Snort--are the most commonly deployed type of IDS. These systems examine individual data packets as they move throughout the network, and compare them against a database of known attack patterns (or "signatures"), much like anti-viral software. Commercial NIDS packages usually rely on dedicated hardware sensor appliances installed on specific network segments to examine traffic as it passes, but most can also collect traffic data from different firewalls, routers, and hosts.

NIDS are extremely fast, and can automatically block suspicious traffic or adjust network configuration in response to a perceived attack in progress. Because they operate in real time, however, NIDS can act as a traffic bottleneck and adversely affect network performance. The size of a performance impact--if any--is difficult to predict, and will vary widely from moment to moment based on available hardware and software, type and amount of network traffic, and network topology.

Host-based IDSs (HIDS) also look for attack signatures, but monitor operating system activity on specific machines, rather than network traffic. Repeated attempts to guess a log-on password might set off a HIDS alert, for example, as might attempts to access restricted local files. Some host-based tools can also monitor specific applications for strange behavior. eEye's SecureIIS Application Firewall, for example, monitors Microsoft's Internet Information Services (IIS) application. HIDS can operate in real time, use automated responses, and typically share most of NIDS' strengths and weaknesses, but HIDS are best suited to detecting different kinds of attacks.

Most commercial vendors bundle NIDS, HIDS, and other tools such as file integrity checkers and log analyzers into a single package. "These are complementary rather than competing technologies," says Marcus Ranum, CTO at NFR Security. "Each is optimized to find different kinds of problems, and together provide overlapping 'fields of fire.' They can also share data, letting them catch issues that anyone alone would miss."

When they were first introduced, IDSs were heralded as the ultimate weapon against online intruders. Finally, network administrators would have the ability to see the attackers in action, and would therefore be able to stop them in their tracks. Years later, however, IDS packages are only beginning to gain mainstream acceptance, and many view them as too expensive and resource-intensive for most companies. (Purchase price is only a part of the equation. Needs assessment, planning, installation, and configuration will usually add up to far more than list price. Total install cost is going to depend on the specific arrangements negotiated with the party doing the install, and will vary widely according to the size of the project and the relationship between the client and installer.) Moreover, many specialists have begun to question whether IDSs represent the best use of limited security resources.

According to Andrew van der Stock, senior architect at security consultancy e-Secure, "IDS is worse than useless in most environments--in most cases, it only gives a false sense of security. IDS is really only suitable once you have a top-notch security environment and are looking for an additional layer of defense."

Perhaps the most serious difficulty with IDS is what is commonly known as the "tuning problem." Most successful online attacks are specifically designed to closely resemble legitimate activity, and a variety of issues can cause harmless or accidental activity to resemble an attack. Every network, moreover, has different norms for acceptable activity. As a result, IDS packages must be carefully "tuned" to minimize the number of false alarms, while still catching actual attacks. In practice, most IDS packages will produce a substantial number of "false positives" no matter how well tuned; over time, overworked administrators tend to tune out or turn off their IDS.

Moreover, the security community has only begun to develop effective responses to attacks in progress. Though most IDS packages are capable of automated responses, most experts warn against their use on a regular basis. Given the frequency of "false positives," automated responses can easily end up interfering with legitimate activity. For example, a savvy attacker can intentionally trigger automated responses simply to cause interference. On the other hand, manual responses tend to be both slow and non-specific. While administrators can take steps to counter or minimize the damage from a specific attack, they are often left with the choice of isolating their network from the Internet (losing much or most of its functionality) or simply allowing an attack to continue.

Though IDS packages are far from a cure-all, they can be a valuable addition to the security professional's toolbox. They are complex and difficult tools to use, however. Perhaps the most important step is recognizing that an IDS is not a replacement for more traditional security tools; it should be seen as "icing on the cake." Well-developed security policies and procedures, solid network architecture, properly configured firewalls, and strong authentication are all prerequisites for an effective IDS deployment.
Don't underestimate the amount of time and resources necessary to properly plan and prepare for initial IDS installation. Properly placing IDS sensors requires a thorough understanding as to which data and assets you're trying to defend, as well as the types of threats of primary concern. Tuning alerts to minimize false positives requires an intricate understanding of your standard network activity, security policies, and enforcement standards.

Substantial as they are, initial deployment costs are only a small fraction of the total investment needed to make an IDS effective. The most common mistake in deploying an IDS is thinking of it as a "set and forget" tool. Be prepared for an ongoing commitment of staffing, training, and financial resources. If you can't afford a dedicated security staff, consider outsourcing your IDS management to a specialist "managed security" firm such as Counterpane Internet Security, Guardent, or Riptech.

An IDS can be a powerful defensive weapon. If you're looking to improve your security, take a look. But be aware of what you may be getting yourself into.

San Francisco-based security consultant and columnist David Raikow holds a law degree from U.C. Berkeley's Boalt Hall School of Law.