Which intrusion detection/protection system?

Need extra network protection to shore up your defences? Read our review of six intrusion detection (IDS) and prevention systems (IPS).
Written by Matt Tett, Contributor and  Staff , Contributor

Despite a rocky beginning, intrusion detection and prevention systems are an important part of any security arsenal. We road-test six hardware and software-based systems.
Intrusion detection systems (IDS) are yet another tool offered to Security Administrators to augment their network security arsenal. IDSes these days fall into two distinct categories: the first are those that are passive they purely watch the data traffic that flows through them, and then capture and log any suspicious databased on policies and rule sets. This data can be examined by the security team and written off as false positives or escalated for further attention.
The second category of intrusion detection systems are those that are active -- they not only detect and log, but also make some attempt to prevent potential threats and attacks from these intruders. These systems are now commonly becoming known as either IPS (intrusion prevention systems) or IDPs (intrusion detection and prevention).
Both IDS and IDP systems apply similar basic methodologies when trying to pick up likely intruders or mischief on the network. The basis of this in most systems is a signature database, which can be regularly updated as new threats are identified.
Security administrators deploy software or hardware remote sensors or agents at key locations within their network, generally on the network perimeter or at gateways with other networks -- basically those that an audit has identified as being good scanning/pickup points where network traffic converges. Behind the firewalls is always a good idea. The remote sensors then report back to a central machine that manages the global policies for the system and stores the data in one location for easy logging, alerting, and reporting.
The IDS/IDP sensors deployed on the network tap into the data streams that are passing by their point and they then analyse the traffic and try and match it against the signatures in their databases. Depending on the threshold set, when a match is made the system then activates and performs whatever task the administrator has set for it, be it drop the TCP connection, alert the security team, or simply log the details for later analysis.
Naturally the performance of the network needs to be assessed prior to deploying a sensor to ensure that the sensor chosen can match the maximum traffic expected through that particular tap location. If a sensor can't handle the throughput, it will result in lost packets (therefore not checking all the data passing through). Even worse, it can impact on the overall performance of the network by creating a bottleneck. It is definitely better to overestimate rather than underestimate the potential network traffic at the point the sensor has been deployed. This approach to IT security has attracted its fair share of critics since the first systems came into play, mainly due to the large number of false positive triggers going off. Each type of system IDS/IPS has its pros and cons, and the decision to deploy one or the other is up to the security team given its own resources, unique network environment, and the threats posed to it.
Alternatively the option exists to deploy more than one type of system to give the network multiple levels of security. For example, you could couple a perimeter hardware solution monitoring the ingress/egress points of the network with host-based software covering critical machines in the network infrastructure.
The biggest threat to IDS/IPS deployments is that of the security team becoming desensitised to the data being logged over time. This is something that needs to be taken into account when creating security policies. Even if there is a high rate of false positives when a system is first deployed, it needs to be constantly tweaked to reduce the number over time, and to build a practical, robust system that may one day save the company data and the security administrator's job.
Where to put your sentry?
The most common place for an enterprise to deploy an IDS/IPS is behind the firewall. This is because IDS/IPS systems are basically data traffic analysers, which involves a large amount of capture and logging of that traffic. Depending on the company's required level of logging, these devices can generate overwhelming volumes of log files which need to be sifted through. This can be a mind-numbing task for security operators, yet it is crucial that it is performed accurately so any potential threats to the business can be discovered and warded off in the future.
Anyone who has actually gone through firewall logs -- and seen the sheer number of packets turned away over the course of a very short period -- will realise there is absolutely no sense in deploying the IDS/IPS in front of the firewall. Think of IDS/IPS as the second line of defence or interception within your company's security cocoon. The firewall filters the most obvious unwanted data while still allowing some potentially questionable packets through, that masquerade as legitimate network traffic. The IDS/IPS system then receives all those transmissions and handles them. For this reason, while some firewalls have quite advanced logging features, they generally log too much unnecessary information and can be cumbersome to run reports from the data they generate.

Quick link to the reviews:
IPSes take the logging role one step further by allowing the security team to create and compile complete lists of various attacks and attempted attacks against the network. These can then be used for management/threat risk analysis or even presented as evidence in court cases should the suspected perpetrator be apprehended. They can also be used to study and define patterns which may point to previously undetected network weaknesses, and enable the team to build their own ruleset or signatures to apply to their unique situation.
IPSes take their task another step forward by being able to not only monitor and log traffic according to their user set rules and policies, but they can also actively block, drop, or handle the traffic in specific ways.
IDS/IPS technologies can also can help mitigate risk for the security administrator. They not only create prodigious amounts of log data, they can also work with system administrators' reports to create policies which will launch certain procedures when triggered. In some cases, these systems can be used to track users behaving suspiciously or out of character on a network, even on trusted accounts. In some cases, when used in a "honeypot" environment, the intruders can be routed to a virtual network and all their information tracked and captured in the hope of providing the necessary forensic reports of their activities and in turn allowing law enforcement agencies to capture them physically. All that is another story, but by now you should have an idea the IDS' role in an organisation.
In short, IDSes are sophisticated virtual alarm systems for networks designed to detect and alert security staff of a possible intruder.
Computer Associates eTrust Intrusion Detection 3.0
Installation of the eTrust application was very straightforward. Initial configuration was equally easy, then the operator can get onto applying their required policies and rule sets according to their company's security needs and policies. If the necessary included foundation policies do not fit the task, then the administrator can modify them or even create their own from scratch.
The product can be deployed in several ways: either standalone for smaller networks or using several remote data probes all reporting back to a central database server. The central server is where the administrator can connect to control the remote probes as well as view the collected data.
One of the most impressive features of this application -- and one that some may take for granted -- is its user interface. What could potentially be an overwhelming array of tools and information -- both captured and real-time -- is handled and displayed with an amazing level of clarity.
CA has built in the ability to scan data that may be coming in from the outside but to also monitor traffic on the inside. This may be deployed as a management prerogative to ensure employees are not breaching their contracts or workplace rules by using the Internet and network inappropriately. This provides potential snoops with the ability to record and playback individual sessions such as HTTP and telnet; while some of the images may not be stored by the system, they can easily be loaded by linking to the live pages. All this data is linked back to the IP address or network mac address and even the system name etc for easy reference.
The system can also be configured to block unwanted network traffic, which can be of benefit to organisations with strict security policies. Online regular updates of standard rule sets and policies can be applied at the administrator's discretion to keep the system up to date. The rules and policies that are included or can be downloaded are very thorough. They include very detailed descriptions in plain, easy-to-understand English, even to the point of providing Web links to appropriate bug-traqs and sites that contain further information on potential threats and how to deal with them. There is even the option to run the included antivirus (AV) engine, complete with automatic updates, if your company needs further levels of antivirus protection. As far as we're concerned, the more protection implemented in the network the better.
Overall this is a very refined solution providing high levels of transparent intrustion detection, URL blocking, and session monitoring/logging combined with an integrated AV engine and automatic updates. If a software solution is in your sights then the Computer Associates eTrust Intrusion Detection product is worthy of evaluation.
Juniper IDP 10, 100, 500, and 1000
Juniper IDP 10, 100, 500, and 1000 Juniper's IDP range of 1RU chassis-mounted intrusion detection and prevention systems are very well constructed and based along relatively standard hardware designs. As most people are now aware, Juniper Networks earlier this year purchased Netscreen, a large IT security system vendor. Juniper networks traditionally was a networking equipment vendor -- routers, DSLAMs, and so on, concentrating on large carrier-grade networks. The acquisition of Netscreen has now augmented Juniper's range of network security products. The IDP-10 can run at a 10Mbps speed, the IDP-100 at 100Mbps, and so on, through to the IDP-1000 at 1Gbps.
When deploying a Juniper IPS solution on the network, first define the perimeter of the network and decide at which points to deploy one or more (depending on the level of fail-over redundancy required) IDP machines, referred to as "sensors". Each of the sensors report back to a centralised management server, which collects all the logs, maintains, and stores all the policies and access information. The security team can then remotely access this centralised server to check the logs, run reports, and manage the configurations of the sensors. The management interface of the IDP series is very impressive, with obvious thought given to usability.
Overall, Juniper provides a very scalable and robust solution with it range of IDP machines. This solution should definitely be on the shortlist for any enterprise with a large distributed network that may need different speeds or classes of IPS solutions in different locations. It has the added benefit and reduced administration overhead of having a centralised management and administration console.
McAfee Security IntruShield I-1200, I-2600 and I-4000
McAfee Security IntruShield I-1200, I-2600 and I-4000 McAfee is covering both ends of the intrusion detection and prevention scale by offering both hardware-based systems and software-based ones. Let's start with the hardware.
The 1200 and 2600 series of Intrushield IPSes are in a 1RU form factor, while the 4000 is a 2RU chassis. All the chassis appear very robust and well constructed. Each of these devices is designed to offer complete transparency when monitoring network traffic.
An interesting design feature is the pass-through ports for the network connections; they are hardwired through, so even if the machine fails, the network traffic can still be routed through without being cut off.
The McAfee units are deployed in much the same way as the Juniper systems -- at various critical points throughout the company network -- and act as sensors relaying information to and receiving updates from the main management server. There are two versions of the management software, the IntruShield Global Manager which is suitable for IPS deployments up to several hundred sensors, and the Intrushield Manager which is suitable for deployments of up to six sensors. The I-1200 unit runs up to 100Mbps, the I-2600 up to 600Mbps, and the I-4000 runs up to 2Gbps.
McAfee likes to talk about the concept of virtual intrusion detection and prevention systems, similar to network switches that support virtual LANs (VLANS). Basically this means each sensor can be segmented into a number of virtual sensors; each can then be customised with different rules and policies, from focusing on a single IP address on the network to a group of machines. These units or sensors also incorporate an internal firewall, which can also be virtualised. This internal firewall is not designed to replace the existing firewall at the network perimeter. However, it allows the option to implement stronger security policies and procedures to enable further protection for critical resources on the network by the intelligent placement of sensors.
Overall, McAfee's appliances are a very scalable solution, again one to definitely shortlist on any evaluation, particularly for large organisations' security needs. Particular note should be placed on the internal firewall and IPS virtualisation features.
McAfee Entercept 5.0
McAfee Entercept 5.0 The key differentiation when comparing McAfee security software IPS with the other software applications such as the Computer Associates or Snort applications is that McAfee software is designed as a individual distributed host-based system, really a last line of defence. The other two applications -- whilst configurable to run as standalone, single-port host-based systems -- are really designed to be network-wide monitoring systems.
This last-line host-based defence is a similar concept to the firewall-on-a-card systems the Test Lab reviewed in the June 2003 issue of T&B. These are basically firewall systems integrated onto a PCI card that are designed to replace the network interface card (NIC) in the host PC and provide a last-line firewall defence against intruders intent on targeting that specific machine. Likewise, the McAfee Entercept application is designed to put the IPS agent directly on that specific machine and then report back to a centralised management server.
The more walls or obstacles a security team can place in a potential hacker's path when targeting the network, the better chance the team has of either rebuffing their attacks or creating a notification system that works well. The main point to keep in mind when applying the onion or layered approach to security is to make sure that management of the system is not too much of a burden for the security team as far as their time and resources are concerned. [If you're interested in a further discussion of the layered approach to security, there will be an article in next month's issue of T&B -- Ed.] Installation, configuration, and administration of the Entercept 5.0 package was simple. Initial installation gives the operator the option to install either the management server, a console, or an agent. We installed all three on the one machine, although obviously the administrator could choose to run a centralised management server with SQL Server for the database, a separate admin console, and several agents deployed on the hosts to be protected.
The installation routine gives the user the option to install Microsoft SQL Server Desktop Engine (MSDE) or run into a full SQL server. The installation also installed Crystal Reports 9.
This product is an excellent last line of defence, or even a front-line defence if there are specific machines on the network that require IDS/IPS monitoring. This is particularly applicable in very open networks with undefined boundaries or perimeters where the security team must treat every node on the network as being potentially hostile. Don't forget the amount of data logged by IDS/IPS systems can be overwhelming, and if a security team is not large enough to monitor all network activity, or the security budget simply does not stretch to a total network monitoring, at least the top primary hosts deserving security can be covered.
Snort 2.1.3
Snort 2.1.3 Snort started out as an open source IDS for Linux (and similarly flavoured systems), and is now even available as a Win32 binary. We downloaded this version briefly just to see how it ran on a Window 2000 machine. The application also requires WinPcap v2.3 (the Windows packet capture architecture library) to be installed. This Win32 version of Snort runs in a very similar command-line mode to the Linux version. Personally we prefer to stick with the Linux environment for Snort.
We installed Snort on a Slackware 9.1 environment on the test machine. The Linux installation takes slightly longer than the Win32 package, mainly because it needs to be compiled and installed from the source code. Libpcap 0.8.3 is also required, and must be installed on the system prior to installing Snort.
Once Snort is installed, running it in basic mode is very straightforward, enabling the administrator to specify what data to collect and where to store it. However, this is just the basics. Snort is a totally rules-based IDS and with 2427 pre-defined rules available at the time we reviewed the product. Undoubtedly there is something in there for everyone. The supporting documentation for most of the rules is also very well presented and documented for administrators to get to the root of the attacks that they may be experiencing.
Snort is not necessarily a standalone application. While Snort applies the rules and logs the data, there is a range of add-on applications such as Barnyard, LogHog, SnortSentry, and Acid as well as a plethora of other tools and user interfaces or front ends. There's even a plugin for the webmin management console. These tools assist with tasks such as configuration, offloading data handling, viewing logs, generating reports, and and analysing the collected data.
While it's not necessarily everyone's cup of tea, Snort would definitely provide an entry into the IDS field, and something is definitely better than nothing. Snort could ideally be deployed to monitor specific ports on the network for traffic. It could even be used as a portable network monitoring tool running on a notebook or older system for diagnostics or intermittent traffic monitoring and analysis. According to the Snort Web site, the package will soon be upgraded to include IPS functionality.
SonicWALL IPS service for SonicWALL appliances
SonicWALL IPS service for SonicWALL appliances Along similar lines to the McAfee and Juniper devices is the SonicWALL range of security appliances. For those familiar with SonicWall's family of internet and network security appliances -- or indeed for those who already have SonicWall equipment (running at least SonicOS 2.2) -- deploying the IPS can be as simple as buying the service subscription and following the upgrade manual.
The SonicWALL IPS software will run on a variety of SonicWALL's existing security appliances from the entry-level TZ170 to the PRO5060; there is also support for the PRO2040, PRO3060, and PRO4060 appliances. The SonicWALL appliances are designed to be jacks of all trades and very easy to deploy and maintain. With the right model and subscription keys, users can enable features in the extended SonicOS 2.5 such as firewall, antivirus, content filtering, IPS, and even a multipoint wireless security gateway (when deployed with SonicWall's access points). The main differentiation in the PRO series is the number of available network ports and network throughput performance ranging up to Gigabit over fibre or opper in the 5060. Applying the IPS system will incur approximately a 15 to 20 percent throughput penalty, so if your SonicWALL appliance is already running out of steam, consider upgrading to the next model before deploying the IPS subset. Naturally if you are starting from scratch and will be deploying a new IPS, you can match the correct SonicWALL IPS performance to your environment.
The main management and administration interface is via a Web console that can be enabled (or disabled) on either the LAN or WAN port for local or remote access. The interface uses a simple management style with drop-down menus on the left hand side. These provide access to the enable options on the appliance as well as allowing the operator to access the licencing section to add further licence keys and subscription updates to the system.
IPS logs are stored intenally on generous amounts of flash memory and can easily be scheduled to be sent off the machine. There are quite a lot of included IPS rules and definitions, 1801 in total, however at this stage the users can't create their own.
This is a very neat device and a great concept to provide a scalable security solution in a single box. Unfortunately, we did not get very long to play around with this equipment, however what we saw was quite impressive.
Product CA eTRUST Intrusion Detection 3.0 Juniper IDP McAfee Intrushield I-series
Vendor Computer Associates Juniper McAfee
Phone 02 9937 0500 02 8913 9800 1800 644 646
RRP (as tested inc GST) AU$3639 for 125 sessions IDP100 AU$29,990 I-1200 AU$19,633.93
Warranty Maintenance Licences include free support for 12 months 1 year hardware/software, can be extended to 3 years One year, renewable annual as part of support agreement
Real-time traffic analysis Yes Yes Yes
Virus/worm/trojan detection Yes Yes Yes
External attack detection Yes Yes Yes
Internal attack detection Yes Yes Yes
Attack blocking capability Yes Yes Yes
External probe detection Yes Yes Yes
Internal probe detection Yes Yes Yes
Probe blocking capability Yes Yes Yes
Blocking definitions Yes Stateful signatures, protocol anomaly, backdoor detection, traffic anomaly, layer 2 protection, syn flood, enterprise security profiler Updates, user-defined block lists and customisable rules
Real time alerting E-mail, pager, run application, SNMP, console Email, syslog, snmp, logfile, SMS-external Console, e-mail, pagers, SMS via e-mail
Data packet logging Workspace (proprietary), ODBC database Syslog, internal database Oracle, MySQL
Content searching Yes Yes N/A
Content matching Yes Yes N/A
Content filtering Yes Yes N/A
Filtering methods URL database Admin definied N/A
Reporting tools Yes Yes Yes
Operating system support Windows 2000 (standalone) Windows 2000/2003/XP for Remote Engine Management console Windows, Linux; Management server Linux, Solaris Management console Windows 2000

Product McAfee Entercept 5.0 Snort 2.1.3 SonicWALL IPS service
Vendor McAfee Snort ACA Pacific
Phone 1800 644 646 +410 423 1901 03 9674 8188
RRP (as tested inc GST) Management server AU$8920, Windows Server agent AU$1730, desktop agent AU$37 Free under the GNU General Public Licence TZ170 US$595 to PRO5060 US$14,950
Warranty One year, renewable annual as part of support agreement No warranty provided 1 year included, extended warranty/available
Real-time traffic analysis Yes Yes Yes
Virus/worm/trojan detection Yes Yes Yes
External attack detection Yes Yes Yes
Internal attack detection Yes Yes Yes
Attack blocking capability Yes Yes Yes
External probe detection Yes Yes Yes
Internal probe detection Yes Yes Yes
Probe blocking capability Yes Yes Yes
Blocking definitions Updates, user-defined block lists and customisable rules Updates, third-party integration, user customisable Updates
Real time alerting Console, e-mail, pagers, SNMP, process spawning Logs, e-mail, console, third-party applications Logs, e-mail, syslog, SGMS
Data packet logging Microsoft SQL Server N/A N/A
Content searching N/A Yes Yes
Content matching N/A Yes Yes
Content filtering N/A Yes Yes
Filtering methods N/A Admin defined Blacklist, third party, admin defined
Reporting tools Yes N/A (sold separately) N/A (sold separately)
Operating system support Management system Windows 2000; console Windows NT, 2000, XP; agents Windows, Solaris, HP/UX Linux, Windows Any IP environment
How we tested
For software and management tools, what platforms are suppored? For hardware, what ports are provided?
Is the system scaleable to grow with your needs, and is there a reasonable upgrade path?
Do the features, usability, and performance justify the price?
What service and maintenance contracts are available? For hardware, what is the warranty?
Testing was relatively tricky as some products were software -- Linux or Windows-based -- and some were hardware appliances. Some incorporated intrusion prevention and some didn't. Computer Associates and SonicWALL sent engineers to the Labs to run us through the installation, configuration and administration of their respective systems before leaving them with us to break on our own. Snort was downloaded, compiled, installed, and run on a Slackware 9.1 machine in the Lab. We visited Juniper Networks and McAfee to look at their hardware IPSes and McAfee provided a copy of Entercept 5.0 for us to install in the Lab on a test server.
Sample Scenario Company: JHL publicity This company has become concerned about external attacks and wants to implement a network intrusion detection/prevention system to trace and manage attacks on its 150-node network.
Approximate budget: AU$600 per monitor.
Requires: One network intrusion detection/prevention system, preferably as an appliance rather than running on a server.
Concerns: The ability to recognise and block external attacks is the key issue, but the network manager wants to be sure the device can intelligently handle the data to reduce management effort. The ability to integrate with existing network/enterprise management software will also be taken into consideration.
Best solution: The best choice here is Juniper with a straightforward range of hardware-based IPS solutions that perfectly meet this scenario. Both McAfee and SonicWALL were potential candidates and very close second, however Juniper adds other options and levels of capabilities that exceeded what we asked for in this scenario.
Things to look out for...
  • Off-system storage. You should be able to log and archive data on to external or other systems independent of the IDS itself, ideally a centralised management server. This provides for a more robust infrastructure and allows backups to be created more in line with the company backup policy instead of adding a new system just for the IDS. Remember, IDS information may be one of the most crucial forms of data your enterprise collects and may be needed for referral many months or even years down the track. Also, if you have a large network with several IDS/IPS sensors deployed, having all the data in one location also reduces data management tasks. One thing that IDS/IPS systems are good at is creating massive volumes of data.
  • Perfomance matched to your businesses size and requirements. The sheer volume of data that some IDSes need to process from the networks that they watch over can be overwhelming. As we know, the network is only as fast as the slowest link in the chain, so don’t let your IDS create that bottleneck. If need be, deploy several IDS sensors on different network segments; this may create more administration overheads, but may not impact so much on performance. Throughput is critical here. Don't put a 100Mbps IDS/IPS machine on a Gigabit backbone.
  • Scalability. Along similar lines to the point made with performance, ensure that when you are performing IDS evaluations that the equipment can scale with your security needs as the organisation grows, particularly in respect to data that it processes.
  • Standardisation of captured data. Even though most IDSes have their own inbuilt report generators, you never know what or even when reports may be needed to be generated from the data gathered. It may be years after the IDS itself has been replaced, so it is wise to ensure that the data is in a standardised form that can be run through independent systems.
  • Final Words
    The Computer Associates eTrust Intrusion Detection software provides an excellent IDS platform to log potential threats and intrusions as well as to look at potential internal anomalies that may be occouring on the network. McAfee Entercept 5.0 provides host-based IPS protection and firewall on a host-by-host basis, truly a last point of prevention application which could just save the day. Snort, while a relatively rudimentary IDS, is nonetheless effective; definitely a no-frills system.
    The Juniper Networks IDP10 to IDP1000 series provides a range of robust hardware-based IPS options. The McAfee Intrushield I-1200 to I-4000 series of equipment provides IPS functionality, comes with an integrated internal firewall system, and has the ability to virtualise both IPS and internal firewall systems. The SonicWALL range also adds the option of firewall capabilities but can also handle wireless security gateway and management tasks when combined with SonicPoint wireless access points.
    Overall these complementary systems provide a very impressive array of equipment for security teams to consider, each with its own nuances. In the right combination, these provide security administrators with a plethora of options and possibilities when trying to track and discover potential vulnerabilities in their network system, before they become gaping holes. The very nature of IT security these days ensures these devices are going to be gaining in popularity over the coming months, so it's worthwhile to spend some time shortlisting and evaluating products to ensure that they slot in to your existing network and security procedures with the minimum of fuss. With so much riding on them, you would have to be crazy not to. And remember, if you're in IT security, never ignore the logs.
    This article was first published in Technology & Business magazine, a ZDNet Australia publication.
    About RMIT IT Test Labs

    RMIT IT Test Labs is an independent testing institution based in Melbourne, Victoria, performing IT product testing for clients such as IBM, Coles-Myer, and a wide variety of government bodies. In the Labs' testing for T&B, they are in direct contact with the clients supplying products and the magazine is responsible for the full cost of the testing.
    The findings are the Labs' own--only the specifications of the products to be tested are provided by the magazine. For more information on RMIT, please contact the Lab Manager, Steven Turvey.

Editorial standards