Keep application and system logs

If your company is lax about archival backups, consider my inspiration for writing this week's column: I recently assisted with a case that's been going on for two years. The case involves an intrusion into a remote system from a local company that occurred in late 2000.

I was unable to help with the forensics of the case because the logs were on a UNIX system that crashed long ago. Also, there are no archival backups of the crashed system to either confirm or deny the intrusion. The case itself rests partially on evidence that cannot be corroborated.

How does this relate to your network's security? In the event of a compromise, system and application logs provide forensic information that can determine when, where, and how an attacker broke into your system. This also raises the question of how long you should keep archival backups. If you ask three developers how far back they keep archival backups of application data or systems, chances are, you'll get three different answers.

Logistically, you can't keep backups indefinitely, and not all companies can afford contracts that handle disaster recovery situations. Not to mention the fact that some of the data might not be something you want to leave around on a tape or CD-ROM. However, don't wait until someone walks out the door with a stack of tapes or a couple of CD-ROMs--go ahead and use a secure safe or storage facility now.

In large enterprises, it's typical to have a firewall between the corporate network and the Internet. This is normally the first place that records both legitimate and nefarious activity. Firewall logs are useful for tracking this activity as well as for other purposes, such as diagnosing problems with the firewall software itself.

Firewall logs typically get large quickly, so it's not always practical to keep months and months of this type of data. Many firewall systems are implemented as "appliances" and don't have the ability to archive or offload old log files without some additional effort--but it's worth it to keep them. Most systems simply rotate firewall logs on a regular interval (e.g., weekly or monthly) and only store a few months worth of information. If it's possible, I suggest companies archive firewall logs for at least a year, perhaps even two or three.

On systems without firewalls, the ability to keep backup copies of system logs is important; yet it might not be a factor when trying to determine the origin of a compromise. For example, when an attacker gains root access to a UNIX system, they often destroy log files first to cover their tracks. Remote logging to a central "syslog" server is a way to circumvent this and can even consolidate system logs for multiple UNIX systems.

There are additional tools that can identify and perhaps head off a compromise on UNIX systems, such as the Swatch system-log watching tool or the Tripwire security tool. For Windows NT and Windows 2000, it's probably wise to turn on security logging (Windows calls this auditing). By default, Windows NT and Windows 2000 disables security auditing, so you'll need to enable it for the items you wish to track. Also, Windows 2000 adds two additional auditing categories. (Note: I'm unfamiliar with Windows XP's auditing capabilities.)

Let the impetus of my column be a lesson to all developers: Keep application and system logs for at least one year. That will make researching a compromise on your network much more productive. Chances are, you'll never know how important this type of information is until you need it.

Jonathan Yarden is the senior UNIX system administrator, network security manager, and senior software architect for a regional ISP.