Earlier, we reported on a new spam defense called "="" rel="follow">'Sender Policy Framework', or SPF. Here's part 2.
A new anti-spam measure currently under development, Sender Policy Framework (SPF),
is attracting a lot of attention. Not only is it quick and inexpensive
to implement, a recent trial by AOL indicates that it could generate
the kind of widespread support fundamental for any anti-spam system to
SPF is designed to prevent forged email being sent by checking the
sender is authorised to send email from the domain they're claiming to
be from. That way, if a spammer attempts to send email from a faked
address, the message will be rejected. Similarly, recent email-borne
worms have pulled fake sender details from address books or Web page
caches, and they wouldn't be able to use this trick.
There are two sides to SPF; checking incoming mail, and allowing others
to check the mail you send. You do the former by using an SPF-enabled
message transfer agent (MTA), and the latter by registering an SPF
record in DNS. Publishing your own record is the easier half to do, so
we'll deal with that first.
An SPF record is a single TXT entry in the DNS for each domain. This details which of the SPF mechanisms
should be used to verify the authenticity of a message, and what should
happen if there's no match. There are a number of mechanisms defined by
the draft SPF standard, but the system is extensible, so that new
mechanisms can be supported without having to rewrite the standard.
The most widely used mechanism will probably be 'MX', which states that
any server designated a mail exchanger for your domain is allowed to
send mail on its behalf. If you only have a single outward facing mail
server this is the ideal mechanism, but it's probably still appropriate
for most systems.
The 'PTR' mechanism instructs the receiver to check the domain name of
the sending host through DNS, and see if this corresponds with the
domain name being checked. If they match, the message is fine, but if
they don't it's a forgery.
Next, the 'A' mechanism allows you to specify that any machine with an
address entry in the DNS which matches your domain is allowed to send
mail. This is slightly different to the PTR mechanism in that it
doesn't require special PTR DNS records, but instead performs an
exhaustive search. This also allows for the case where an ISP is using
a single mail server as the mail sender -- but not receiver -- for
several domains, which the PTR mechanism can't be used with.
The 'include' mechanism tells the receiver that a
different domain is allowed to send mail on behalf of the one in
question, and the other domain's SPF record should be checked to see if
the message is coming from a valid address. For instance, zdnet.co.uk
may issue an include directive for cnet.com (the domain of our parent
company) which would then point someone checking our domain to our
parent, which may be sending mail on our behalf. They can then check
that the sender is permitted for cnet.com by retrieving their SPF
There are other mechanisms whereby you can specify the IPv4 or IPv6
address range that's permitted to send mail on behalf of a domain.
These are less flexible but marginally faster than the previous
mechanisms, since they don't require another DNS lookup. There's also a
rather sophisticated DNS-based method called exists, that allows you to
check for the existence of a particular host by manipulating the
sending email address using macros.
SPF also allows further mechanisms to be used without having to add
them to the specification itself. This could include mechanisms that
state that, for instance, all valid mail from a given domain will be
digitally signed. The danger here is that these more esoteric
mechanisms won't be supported by many MTAs, making them far less useful
than the address-based mechanisms defined in the standard.
SPF also allows you to designate mechanisms to soft fail,
meaning that if a message doesn't pass scrutiny, it's still accepted,
but flagged as dubious. This can be used while you're in the process of
evaluating SPF, and aren't sure you've got your record composed
properly. Longer term, it can also be used where you need to specify
that messages being sent from certain places isn't standard practice,
but isn't unknown.
While this may seem like quite a complicated arrangement, the reality
is that the MX and PTR mechanisms will cover the vast
majority of cases, and only large, complex networks are likely to need
to resort to the other mechanisms involved. You don't even have to work
through all this by hand: There is an SPF record composer wisard
available at the SPF Web site, which will create the text of your SPF
record based on the options you choose. There are also more SPF related tools available, such as a record checker to test your entry once it's live.
The beauty of using DNS to distribute SPF records is that popular
records will be cached by ISPs, meaning that there won't be a huge
amount of bandwidth taken up by SPF queries. Popular domains will be
requested more often, but are more likely to be cached closer to the
requester. It also means that the infrastructure required to distribute
records already exists, is known to be stable and won't require any
more expenditure over and above that needed to operate a domain.
After publishing your own SPF record, the next step is to check all
incoming mail. Despite SPF being a relatively new proposal, quite a few
MTAs have been SPF-enabled. Unsurprisingly, these are mostly
open-source efforts: plug-ins are available for Sendmail, Exim and
Postfix. SpamAssassin v2.70 includes support for SPF checking. If your
mail server can call external scripts, libraries for Perl and Python
are available. A C library for SPF checking has also been created,
although this will require more skill to integrate with a mail system.
If your mail system doesn't support SPF -- as will probably be the case
if you use a messaging system like Notes or Exchange -- you can still
use the system by putting an SPF-enabled MTA in front of your system to
act as a filter: Failed messages are refused, messages with good or
unknown status can be passed on, but marked with an extra header to
alert any later filtering systems of its ambiguous status. The
"Received-SPF:" header is used to do this, and the format of the header
is defined in the SPF specifications. Messages which pass SPF
validation should stiff be marked with a Received-SPF header to show
that they're genuine.
While it's technically possible to SPF-enable a user mail client, this
is less productive than doing the checking at server level, since the
mail will already have been transmitted across the Internet. However,
if you don't have your own incoming mail server, but use your ISP's
server, this will be your only option until your ISP implements SPF
checking themselves. Mail clients should look for headers generated by
SPF checkers, and use them when doing automatic spam filtering. An
ambiguous SPF status, in conjunction with other spam detection methods,
can be used to identify unwanted messages. Similarly, an unambiguous
SPF pass can be used in a points scoring system to increase the chances
of genuine mail getting through spam filters.
The worry when first implementing SPF checking on incoming mail is that
you'll be rejecting genuine mail: This shouldn't happen. Mail will only
ever hard-fail SPF checking if the purported sending domain has
published an SPF record, and the mail is being forged. While it's
possible that the originating domain's SPF record is incorrect, this
would quickly become apparent to the mail administrators of that domain
when large numbers of messages fail SPF checking. If you're really
concerned about rejecting genuine mail, one option you have is to
accept messages that fail checking, but rewrite the subject line to
show that this message is possibly faked.