One giant step towards ending spam

The fight against spam is one of the biggest tasks facing the industry today, and a great deal of time and money has been invested in trying to thwart unwanted email -- but with little real success. But a new anti-spam measure currently under development, Sender Policy Framework (SPF), is attracting a lot of attention. Not only is it quick and inexpensive to implement, a recent trial by AOL indicates that it could generate the kind of widespread support fundamental for any anti-spam system to succeed

Some estimates put the level of spam at around 50 percent of all email sent over the Internet. In addition to messages advertising dubious goods and services, there are emails carrying worms that pose an additional hazard. While anti-spam and antivirus filters can help prevent these rogue messages making your inbox unusable, they can be defeated by changing the format of junk emails -- and the messages themselves are still using valuable bandwidth.

The problem is that simple mail transfer protocol (SMTP) has so few security features. Originally, any SMTP server would accept mail from anyone, for anyone -- known as an open relay. This wasn't a problem in the early days of the Internet, where there were fewer users, and virtually no commercial ones. However, open relay meant that a spammer could connect to a random server and use it to send thousands of messages. Open relay abuse can be dealt with by only permitting mail from or addressed to that server's registered users. But while open relay is no longer an issue for the vast majority of companies, mail that's correctly addressed to a valid mail address, but comes from a dubious source, is the big, big problem.

Closing the loophole
This is where SPF comes in. Also known as "Sender Permitted From", its purpose is to prevent forged email being sent by checking the sender is authorised to send email from the domain they're claiming to be from. That way, if a spammer attempts to send email from a faked address, the message will be rejected. Similarly, recent email-borne worms have pulled fake sender details from address books or Web page caches; they wouldn't be able to use this trick.

SPF uses the domain name service (DNS) to provide information about which servers will send mail on behalf of a particular domain -- in the same way as receiving mail servers are currently specified using Mail Exchange (MX) records. When a mail server gets an incoming message, it looks up the sender's domain to get the SPF record, and checks whether the sending server is in the permitted list. If not, the mail is rejected as a forgery. The use of DNS as the underlying mechanism means that no additional infrastructure is needed to make SPF work worldwide.

SPF's creators are careful not to over-hype the system's capabilities: It won't solve spam completely, overnight or unilaterally. It can't prevent spam with genuine addresses (although this is easier to trace) or from spam-friendly domains that choose not to participate in the scheme. But it will stop a large amount of the spam sent currently and will help make spammers easier to trace and prosecute.

SPF isn't yet an official standard, but is being submitted to the Internet Engineering Task Force (IETF) for consideration as a standard. There's every reason to believe it will be adopted, since it's vendor-neutral, patent-free and requires very little in the way of extra software to implement. It also doesn't require the sender to do anything special, such as visit a validation Web page or include a special code in the email. Crucially, it also doesn't rely on having a third party verify mail or provide any sort of certificate -- everyone is master of their own domain.

The downsides
SPF isn't without other issues, but these don't affect the majority of users, and they're relatively easily overcome. The most immediate problem is that genuine users won't be able to send mail directly from remote locations. For instance, a teleworker may wish to send work email from home, but through their ISP's SMTP server. If the company's SPF record stated that this wasn't acceptable, these genuine messages would then be rejected. However, this can be worked around quite simply. The user could connect to the office network via a virtual private network (VPN) and use the company's SMTP server to send mail, thus ensuring that it came from a machine permitted by the SPF record. If connecting to a VPN isn't possible, you could make a single SMTP server available for sending from outside your network. This would need to be secured using SSL or SASL to prevent it being abused, with genuine users given credentials to log onto the server.

SPF also breaks mail forwarding, as used by some mail providers or by users who create a '.forward' file on Unix systems to send mail from one account onto another. The problem is that the forwarded mail isn't coming from the originating domain -- that of the mail's sender -- so is quite legitimately rejected by an SPF check. A solution to this is the Sender Rewriting Scheme (SRS). This is a method of rewriting the headers on a message so that it passes SPF checking, but still carries the address of the original sender so that replies to the message can be sent. SRS also includes some extra security features to prevent it being abused. Most organisations don't need to worry about implementing SRS -- it's really only an issue for mail forwarding providers.

What else is needed?
SPF won't stop a spammer buying a domain, creating an SPF record for it, using it to send messages and then discarding it once it's blacklisted. To counter this, reputation management schemes can be set up. These can work by mail administrators reporting domains used to send spam manually, or by recording the number of messages sent by a particular domain, and how many of these were rejected by the recipient. How long a domain has been registered can also be used to judge it -- a domain that's only a few hours old is unlikely to want to send large numbers of messages if it's genuine, so messages from it can be throttled. But most importantly, if spammers are having to buy domains to use, it's affecting their business model, and it's also giving a paper trail back to their real identity via the registry used to create the domains.

As of March 2004, over 8,000 domains had published an SPF record. Included in there is AOL, a popular domain to fake among spammers. For SPF to be truly successful, it needs to be the rule rather than the exception, so widespread adoption is key. The advantages of SPF over other similar schemes include its minimal implementation cost. There's also nothing to be lost in running a trial implementation, and it can work in conjunction with existing anti-spam measures.

Click here for Part Two of this special, "How does Sender Permitted From work?"

For more information, see

http://spf.pobox.com/,http://spftools.infinitepenguins.com/ and www.libspf.org/.