Can greylisting help stem the tide of junk e-mail?

If you asked 10 people about their most frustrating problem with the Internet, I'm willing to bet that nine would mention e-mail. While there are certainly other security problems with the Internet, e-mail is where several Internet security problems converge.

If you asked 10 people about their most frustrating problem with the Internet, I'm willing to bet that nine would mention e-mail. While there are certainly other security problems with the Internet, e-mail is where several Internet security problems converge.

It's no accident that many of the viruses and Trojans commonly found on compromised Windows systems have features that enable them to act as e-mail proxies. Junk e-mail is a big business, and it's unfortunately part of the Internet that will likely not change.

Like many Internet application protocols, the original design behind Simple Mail Transfer Protocol (SMTP) didn't have security in mind. Instead, developers have simply extended SMTP with security features over the years.

However, securing e-mail systems themselves and stopping junk e-mail are dissimilar problems. Keep in mind that the majority of e-mail traffic on the Internet is junk e-mail from compromised Windows PCs on broadband networks—not compromised e-mail servers at corporations.

While the spam gangs make millions from unscrupulous advertisements, junk e-mail is wasting time, bandwidth, and therefore money for anyone using the Internet. For proof, look no further than the market for commercial junk e-mail filtering services, which clearly highlights how much users consider junk e-mail to be a primary problem with the Internet.

But stopping junk e-mail is a complicated matter, probably much more so than most users understand. And even pattern matching and statistical-based e-mail content filtering have their limits. In addition, as I've experienced many times with relay block lists (RBLs) and filters, one person's junk is another's legitimate e-mail. What is basically comes down to is that there's really no single automatic solution to stop junk e-mail.

Of course, this doesn't mean current methods aren't working; they're just not working perfectly, nor should we necessarily expect them to. Like any other area of Internet security, e-mail security—or, in this case, junk e-mail prevention—works best when applied in layers. The more layers of security, the more effective overall security becomes.

Every approach has its limits; for example, "blacklisting" and "whitelisting" are tedious to implement. Each requires a great deal of user intervention, but they can prove to be effective provided your lists are accurate.

Despite our best efforts, junk e-mail always seems to find a way to get into inboxes, sometimes even when valid e-mail doesn't. But it isn't time to abandon hope: A new approach to combating junk e-mail, dubbed "greylisting," is proving to be an efficient junk e-mail prevention method that stops e-mail before it ever reaches the destination SMTP server.

In addition to helping prevent junk e-mail, greylisting also specifically addresses another SMTP deficiency: The protocol doesn't distinguish between an e-mail server and an e-mail client.

Greylisting attempts to create this specific distinction between SMTP servers and e-mail clients. By doing so, it can effectively stop junk e-mail proxies. While both e-mail servers and e-mail clients utilize SMTP, legitimate SMTP-speaking systems understand specific return codes.

SMTP is typically is a "store-and-forward" system that queues e-mail until it can deliver it. A greylist-enabled SMTP server records information about connections it receives, and it returns a status code that tells a connecting computer that it's temporarily unable to accept e-mail.

A legitimate e-mail system that "speaks" SMTP—for example, Microsoft Outlook or an e-mail server running Microsoft Exchange, Sendmail, or another SMTP server software—will attempt to deliver the e-mail at another time. At this point, the greylist-enabled SMTP server will pass the message along.

When the valid SMTP connection tries to e-mail again, the server won't block valid messages because the greylist-enabled server matches the IP address, as well as information in the MAIL FROM and RCPT TO SMTP headers, from the previous attempt. If they match, the server accepts the e-mail. Greylisting typically rejects junk e-mail proxies immediately because the proxies don't store e-mail.

However, spam gangs are adapting to greylisting, so this will not always be the case. But because greylisting also records the IP address as well as the sending and receiving e-mail addresses, it's possible to further extend greylisting to monitor this type of information and act on it. For example, an SMTP server implementing greylisting could monitor SMTP connections for "seeded" users that don't exist and then use this to identify junk e-mail proxies and source IP addresses.

So where do you turn to for your greylisting needs? While the Postfix SMTP server includes a sample greylist implementation, a production-quality system called Postgrey would be a better choice for e-mail systems. However, since greylisting is a feature implemented at the SMTP server level, and considering that it's such a new method, chances are you won't find greylisting built-in to your SMTP server.

In such cases, it's possible to "front-end" your existing e-mail server with Postfix and relay incoming e-mail to the real SMTP server. This lets you use greylisting without making any changes to your existing e-mail server.

In fact, I've implemented Postgrey for a number of Microsoft Exchange and Lotus Domino servers running in corporate environments where companies were drowning in junk e-mail. After installing Fedora Linux on commodity PC hardware using spare equipment, it took less than 15 minutes to configure Postfix and Postgrey to work with an existing SMTP server—providing all the features of greylisting with a minimum of changes to existing infrastructures.

Jonathan Yarden is the senior UNIX system administrator, network security manager, and senior software architect for a regional ISP.