Sender Policy Framework: Part 2
A new anti-spam measure currently under development, Sender Policy Framework (SPF), is attracting a lot of attention. Not only is it quick and inexpensive to implement, a recent trial by AOL indicates that it could generate the kind of widespread support fundamental for any anti-spam system to succeed.
SPF is designed to prevent forged email being sent by checking the sender is authorised to send email from the domain they're claiming to be from. That way, if a spammer attempts to send email from a faked address, the message will be rejected. Similarly, recent email-borne worms have pulled fake sender details from address books or Web page caches, and they wouldn't be able to use this trick.
There are two sides to SPF; checking incoming mail, and allowing others to check the mail you send. You do the former by using an SPF-enabled message transfer agent (MTA), and the latter by registering an SPF record in DNS. Publishing your own record is the easier half to do, so we'll deal with that first.
An SPF record is a single TXT entry in the DNS for each domain. This details which of the SPF mechanisms should be used to verify the authenticity of a message, and what should happen if there's no match. There are a number of mechanisms defined by the draft SPF standard, but the system is extensible, so that new mechanisms can be supported without having to rewrite the standard.
The most widely used mechanism will probably be 'MX', which states that any server designated a mail exchanger for your domain is allowed to send mail on its behalf. If you only have a single outward facing mail server this is the ideal mechanism, but it's probably still appropriate for most systems.
The 'PTR' mechanism instructs the receiver to check the domain name of the sending host through DNS, and see if this corresponds with the domain name being checked. If they match, the message is fine, but if they don't it's a forgery.
Next, the 'A' mechanism allows you to specify that any machine with an address entry in the DNS which matches your domain is allowed to send mail. This is slightly different to the PTR mechanism in that it doesn't require special PTR DNS records, but instead performs an exhaustive search. This also allows for the case where an ISP is using a single mail server as the mail sender -- but not receiver -- for several domains, which the PTR mechanism can't be used with.
The 'include' mechanism tells the receiver that a different domain is allowed to send mail on behalf of the one in question, and the other domain's SPF record should be checked to see if the message is coming from a valid address. For instance, zdnet.co.uk may issue an include directive for cnet.com (the domain of our parent company) which would then point someone checking our domain to our parent, which may be sending mail on our behalf. They can then check that the sender is permitted for cnet.com by retrieving their SPF record.There are other mechanisms whereby you can specify the IPv4 or IPv6 address range that's permitted to send mail on behalf of a domain. These are less flexible but marginally faster than the previous mechanisms, since they don't require another DNS lookup. There's also a rather sophisticated DNS-based method called exists, that allows you to check for the existence of a particular host by manipulating the sending email address using macros.
SPF also allows further mechanisms to be used without having to add them to the specification itself. This could include mechanisms that state that, for instance, all valid mail from a given domain will be digitally signed. The danger here is that these more esoteric mechanisms won't be supported by many MTAs, making them far less useful than the address-based mechanisms defined in the standard.
SPF also allows you to designate mechanisms to soft fail, meaning that if a message doesn't pass scrutiny, it's still accepted, but flagged as dubious. This can be used while you're in the process of evaluating SPF, and aren't sure you've got your record composed properly. Longer term, it can also be used where you need to specify that messages being sent from certain places isn't standard practice, but isn't unknown.
While this may seem like quite a complicated arrangement, the reality is that the MX and PTR mechanisms will cover the vast majority of cases, and only large, complex networks are likely to need to resort to the other mechanisms involved. You don't even have to work through all this by hand: There is an SPF record composer wisard available at the SPF Web site, which will create the text of your SPF record based on the options you choose. There are also more SPF related tools available, such as a record checker to test your entry once it's live.
The beauty of using DNS to distribute SPF records is that popular records will be cached by ISPs, meaning that there won't be a huge amount of bandwidth taken up by SPF queries. Popular domains will be requested more often, but are more likely to be cached closer to the requester. It also means that the infrastructure required to distribute records already exists, is known to be stable and won't require any more expenditure over and above that needed to operate a domain.
After publishing your own SPF record, the next step is to check all incoming mail. Despite SPF being a relatively new proposal, quite a few MTAs have been SPF-enabled. Unsurprisingly, these are mostly open-source efforts: plug-ins are available for Sendmail, Exim and Postfix. SpamAssassin v2.70 includes support for SPF checking. If your mail server can call external scripts, libraries for Perl and Python are available. A C library for SPF checking has also been created, although this will require more skill to integrate with a mail system.
If your mail system doesn't support SPF -- as will probably be the case if you use a messaging system like Notes or Exchange -- you can still use the system by putting an SPF-enabled MTA in front of your system to act as a filter: Failed messages are refused, messages with good or unknown status can be passed on, but marked with an extra header to alert any later filtering systems of its ambiguous status. The "Received-SPF:" header is used to do this, and the format of the header is defined in the SPF specifications. Messages which pass SPF validation should stiff be marked with a Received-SPF header to show that they're genuine.
While it's technically possible to SPF-enable a user mail client, this is less productive than doing the checking at server level, since the mail will already have been transmitted across the Internet. However, if you don't have your own incoming mail server, but use your ISP's server, this will be your only option until your ISP implements SPF checking themselves. Mail clients should look for headers generated by SPF checkers, and use them when doing automatic spam filtering. An ambiguous SPF status, in conjunction with other spam detection methods, can be used to identify unwanted messages. Similarly, an unambiguous SPF pass can be used in a points scoring system to increase the chances of genuine mail getting through spam filters.
The worry when first implementing SPF checking on incoming mail is that you'll be rejecting genuine mail: This shouldn't happen. Mail will only ever hard-fail SPF checking if the purported sending domain has published an SPF record, and the mail is being forged. While it's possible that the originating domain's SPF record is incorrect, this would quickly become apparent to the mail administrators of that domain when large numbers of messages fail SPF checking. If you're really concerned about rejecting genuine mail, one option you have is to accept messages that fail checking, but rewrite the subject line to show that this message is possibly faked.