Can e-mail survive?
E-mail has taken a battering over the last year or so with mountains of spam and viruses delivered to our mailboxes daily. Can the problem be fixed, and can e-mail still be free?
You walk in to the office with one more reason to hate Monday: spam. You can sort of deal with it during the week, but Mondays? The spammers have spent the entire weekend filling your inbox with junk that you'll have to sort.
If you haven't seen any spam in your lifetime then you've probably never held an e-mail account. Some anti-spam vendors, who admittedly have a vested interest in overstating the problem, estimate that spam now accounts for more than 50 percent of the world's e-mail. Even if the figure is half that, the problem is still a big one.
Some of those companies claim to offer the solution to all your spam woes, but the silver bullet needed to slay the hairy beast seems, at this stage, elusive. There's filtering, white- and blacklisting, trust-based models and extensions to the Simple Mail Transfer Protocol, SMTP.
The original idea of e-mail was to be able to send anyone on the planet an e-mail and have them receive it and read it. That original idea now seems to be an outmoded, misty-eyed romantic notion -- that the Internet could exist without miscreants seeking to abuse its open nature.
So can e-mail survive in the world of v1agr4 p1ll5, 5p3ci4l pr0n XXX 0ff3r5, and bogus l.0t3rry win5?
Filtering
One true believer in filtering solutions is SpamÃ,ÂÂAssassin creator Justin Mason. Back in 2001 Mason was mildly frustrated by spam and set out to create a simple piece of software that could deal with the problem. That software later became SpamAssassin, the open-source spam filter.
"When I started it, spam wasn't half as bad as it was now," he says.
SpamAssassin filters messages at the gateway based on a series of criteria ranging from the source IP of the message to its structure or even basic keywords.
"We mostly rely on pattern-based signatures and the structure of the message," Mason says. "Abuse of open proxies and that sort of thing, we can have signatures for that too."
Does spam represent a threat to e-mail? No, says Mason.
He's not just the creator of SpamAssassin, he's its biggest fan, too: filtering technology will save the day, he says. As anti-spam measures mature, he predicts they will become much easier to use. Automatic "signature" downloads will become common, much the same as anti-virus solutions today.
His company, DeerSoft, was acquired by Network Associates (recently renamed McAfee) at the end of 2002. While SpamAssassin was an open-source project, DeerSoft specialised in creating proprietary extensions to the software to make it easier to manage. Network Associates snapped up the company, Mason and his fellow engineers along with it.
He says the vendors are actually coming up with products that are useful, but concedes you can only go so far. "There are limitations as to how accurate you can get," he admits.
Perhaps of most concern is the number messages tagged as spam that are in fact legitimate. The current rates of what is cutely referred to as "collateral damage" or "false positives" are running at around 0.5 percent in the present generation of Network Associates products, Mason claims.
That's one in 200 messages, which may be too high for some people to accept. Say you don't get the e-mail from your sister in London: big deal, you can live without it. If it's from someone trying to establish contact with your company, then you may have lost yourself some valuable business.
Blacklists
| "They're a really mixed bunch. You've got some seriously dodgy, vigilante type situations, where someone has decided this ISP has too many spammers... and they just blacklist the lot." |
The problem is, as some see it, the people who run them. Many have been accused of over-zealousness; their operators black-listing entire netblocks of thousands of users because one person had spammed from the range.
"They're a really mixed bunch. You've got some seriously dodgy, vigilante type situations, where someone has decided this ISP has too many spammers... and they just blacklist the lot," Mason says.
However, Mason insists blacklists are a valuable tool.
"You've got the other ones like the Spamhaus guy. They're really careful to avoid false positives," he says. "They do a really good job of tracking real spammers... we definitely use those guys."
It isn't just irate network operators who have a problem with blacklist maintainers. Spammers, not surprisingly, have a beef with any technology that messes with their turf. The services are frequently bombarded with distributed denial of service (DDoS) attacks aimed at rendering them completely ineffective. Some of these attacks have been so serious the services have shut down altogether.
Osirusoft, a blacklist operator, closed its virtual doors in August last year after being bombarded with DDoS traffic. Not only does this make an online service very difficult to operate, but can result in site maintainers being hit with massive bandwidth bills to cover the flood of onerous traffic.
One blacklist that puts fire into the belly of many a network operator is the Spam Prevention Early Warning System (SPEWS). Often described as the cowboys of the anti-spam scene, a basic Google search reveals many, many gripes.
Tired of being "zonked by the spineless anonymous cowards at SPEWS," some have even resorted to supporting an anti-spews movement designed to counter the group's radical tactics. SPEWS has been known to black-list entire providers, such as Telstra, due to a relatively small amount of spam originating from such networks.
| |
A radical approach?
Filtering and blacklists have been around for some time, and while some headway has been made, vendor solutions may not be the answer for all users. So what can be done to change e-mail to wipe-out spam? It depends on who you talk to.
Yahoo launched its initiative, DomainKeys, last year. Working in conjunction with the Sendmail Corporation, the plan is to add a layer of authentication to e-mail. The approach will eliminate sender address spoofing, and make it very difficult for spammers to pretend to be someone they are not.
| "I think that e-mail is such a huge driver for the Internet. We used to talk about content being king, but communication is the key." |
However, the idea is to create a "framework within which comprehensive authorisation systems, reputations systems and their ilk can be developed".
Rachel Watt, senior producer at Yahoo's Sydney office, says changing the fundamentals of e-mail isn't workable, but tweaking it is.
"Some people believe that e-mail usage will decrease and instant messenger usage will increase because of spam," she says. "I think that e-mail is such a huge driver for the Internet. We used to talk about content being king, but communication is the key."
Microsoft plans to get in on the action as well. The company's exchange product manager in Australia, Andrew Cunningham, is optimistic about future SMTP standards stamping out spam.
| "E-mail being e-mail, it would be like making a mobile phone that won't talk to any other carrier besides your own. Anything we do has to converge and work with standards other than our own." |
The software giant will engineer its products to be compatible with whatever other authentication measures crop up, such as DomainKeys, he says. It's also planning to contribute its own authentication technique, known as e-mail caller-ID.
While some may doubt the company's insistence that it's eager to ensure full interoperability with its competitors' products, Cunningham is adamant that Microsoft's Exchange line of products will not try to lock out other efforts.
"E-mail being e-mail, it would be like making a mobile phone that won't talk to any other carrier besides your own. Anything we do has to converge and work with standards other than our own," he says. Let's not forget this is the company that recently announced a vision of charging for e-mail to eliminate spam, a plan that was never likely to garner much popular support. What does Cunningham have to say about it?
"It's one concept or one vision that may address the problem of spam," he says. "Charging for e-mail might be one way to do it, the way you pay for a phone-call. There are a lot of factors that need to be considered to make that a reality." Let's just hope one of the factors that winds up being considered is that it's a really lame idea.
Another company laud the benefits of some measured tweaking of SMTP is US-based IronPort. Its Australian chief Michael Bosch says its products will support Microsoft "caller-ID" for e-mail, the Pobox.com-developed Sender Policy Framework, or SPF, and Yahoo's DomainKeys. The company has also proposed its own changes. "We've proposed industry standards that will incorporate sender reputation into the e-mail," he says.
IronPort also maintains a "reputation index" of senders, collecting data from its mail analysers and probes across the globe. In a deal signed with Microsoft, the company is able to analyse Hotmail e-mails in an effort to gather intelligence.
If the buzz is anything to go by, this data could turn out to be very useful in the fight against spammers. Mason praises the company's efforts. "They've got some very useful information in their system [and] their Senderbase system is very interesting," he says.
| |
Legislation
Worldwide efforts to put in place legislation aimed at cracking down on spam are yet to bear fruit, but the new laws may have a gradual effect.
| "Big governments are dying to prosecute these guys because they're a pain in the neck... they're really causing trouble." |
"Everything I have read suggests that a very small number of people are responsible for a very large chunk of spam," he says. "Law enforcement should crack down on them. A couple months of work to nail the top 10 would send a huge message."
Australia's Spam Act became law earlier this year amid a storm of criticism. The naysayers called the legislation weak -- it would never have an affect on the booming business of spam. However, a few serious victories could see spammers getting scared off, according to Mason.
"The laws that have been brought in will make a huge difference. There are a few bottom feeders who send out little bits... but there's a surprisingly small number of guys sending out huge volumes of spam," he says. "Big governments are dying to prosecute these guys because they're a pain in the neck... they're really causing trouble."
A couple of convictions could stifle spamming efforts and push spamming to the outer fringes of the Internet; the practice would be pushed well underground, limiting its appeal. "I think if they start managing to prosecute some of the big guys it will go down... but I don't think it will ever disappear."
Martin agrees. "They would have to bust them, put them out of business and drag them through the court system. Make it unprofitable and make it very public," he says.
A lesson from Instant Messenger?
Instant messenger software proved that new Internet applications are on the way. Many companies use IM for communicating both internally and sometimes externally and between sites. It's a fast, efficient mode of communication. With one key difference to e-mail: sending unsolicited messages is much more difficult.
Even consumer instant messenger applications -- such as ICQ, AIM, Yahoo, and MSN -- rely heavily on contact lists. No one can send you an instant message unless you've allowed them to do so. Could such a model work with e-mail? Yahoo's Watt -- who claims spam is the number one complaint from Yahoo users -- says it's not likely. People will use instant messenger, but perhaps for different applications to e-mail, she says.
"As we see better infrastructure, videoconferencing will probably be a no brainer," Watt says. "People want to have that kind of communication ability."
Microsoft's Cunningham also says an IM model is unlikely to take hold in the e-mail space. Its anyone-to-anyone nature is what makes it attractive.
"The mentality in the two is very different. One of the benefits of e-mail is you can send to anyone you want to," he says.
Instead of engineering Exchange products to operate like IM clients, Microsoft is planning to tightly mesh its Messenger product with Outlook.
| |
Money talks
Chy Chuawiwat, the former general manager of mail-filtering concern Clearswift's Australian office, says the amount of money being thrown around is likely to yield results. Describing antispam solutions as easier to sell than "tax cuts", Chuawiwat believes the return on investment for spam solutions is clear.
"There's been an increase in general interest in mail filtering due to spam," he says. "As opposed to viruses, porn, IP protection. Spam, spam, spam is all it's about."
Spam wastes bandwidth and spam wastes time, so its no surprise that every man and dog is claiming to offer a solution, Chuawiwat says. There's not much truly innovative technology bubbling to the surface just yet, he argues, just a lot off vendor fluff.
"You use a blacklist, you add a few text words, which you know are spam, and if you wanted to you could get a Bayesian algorithm and use that. Then you can claim to have a spam filtering product," he laughs. "But good spam software still takes quality engineers to create."
Don't forget there was a time when viruses were a more serious problem than they are now, he says. It will take some time, but eventually the whole situation will boil down into some sort of status-quo. "There is no single key, and the game of new spam technology versus new antispam technology will continue, just like with viruses, in the broad sense."
Another key difference which makes wiping out IM spam much easier is the lack of a distributed and open architecture. Every single instant message must first pass through the provider's servers, thus giving the provider control over all messages. If it detects an anomaly, such as 50 messages a second coming from an account, it could immediately stop them.
E-mail's openness is e-mail's peril. Like having children, you don't need a licence to run a mail server. There's no control, no licensing body, and no grand oversight. The Internet was founded on the basis of total technical freedom; anyone can contribute to the mesh that makes it up, an attribute which has undoubtedly contributed to its explosive growth, at least in the early days. Could the founding ideals of the Internet lead to the abandonment of its most useful applications? The scene seems set for a lengthy war of attrition.
Sender Policy Framework
The original idea behind Sender Policy Framework, SPF, was very simple. While Domain Name Systems hold the records for where to send mail to for a given domain name, they have never held information pertaining to where mail from that domain should come from.
So, an e-mail purporting to come from Telstra.com, for example, would have to actually come from Telstra's servers to get accepted by an SPF-enabled server. As the telco has shown in the past, this isn't beyond the realms of possibility, but spammers who log on to major ISPs to do their worst aren't the real scum, they're just the low-hanging fruit.
Microsoft's caller-ID incorporates SPF -- they merged in May 2004 -- and a few nifty header inspection tricks, while Yahoo's DomainKeys uses domain name servers to spit out cryptographic keys used to verify messages are coming from where they say they are.
The owner of a DomainKeys server generates a cryptographic key pair, one public and one private. The private key is used by the outgoing mail server to sign all outbound messages, while the public key, which is published to the domain name's DNS record, is used by recipients to verify that mail was signed by the private key.
While these tricks all sound cool, time will tell if it actually makes a difference. Yahoo's system seems great, but could it just encourage spammers to get more desperate? Junk mailers may start breaching DomainKeys -- enabled servers to spread their bad grammar, and with all these systems what's to stop spammers registering legitimate domain names with valid name server records, just as they do today?
These systems may not kill off spam entirely, but they will make it easier to identify bad domains. Perhaps the solution is simpler than we ever imagined. All that's needed may be a list of the 20 worst spammers, 12 elite SAS soldiers, 20kg of plastique, a box of assault rifles, and a helicopter.
| |
Can you avoid spam?
The simple answer is: maybe.
The key to avoiding spam is to understand how spammers get a hold of addresses. If they can't get your online address, they can't spam you.
Sometimes spammers will use brute-force tactics to "guess" an e-mail address on a popular domain, such as SomeHugeISP.com. They'll try aaron@SomeHugeISP.com, adam@SomeHugeISP.com, john, johns, johnsmith, smithjohn etc. Some even use random combinations of letters, fishing for valid addresses.
Other times they'll use bots -- agents designed to scour the Web and newsgroup postings, looking for e-mail addresses, much in the same way search engine bots scour the net for fresh content.
You may even give your e-mail address to spammers if you're one of those poor souls who still thinks plugging their address into a porn-site's registration page is kosher.
So how to avoid being consumed with e-junk? It's quite simple. Don't post messages to newsgroups or mailing lists. Don't put your primary e-mail address into a Web form. Ever. And get yourself an e-mail address on a really obscure domain name which is unlikely to attract an attack.
For US$10 (~AU$14) you can register your own .com, .org or .net domain name, and you can probably have it hosted by your ISP fairly cheaply. Don't even set up a Web site on that domain -- keep it mum; your e-mail address and online presence is best kept low-key.
Set up a Hotmail account to use when you're prompted for your address. This is handy when you need to provide an address to obtain information required to, say, download some software.
There's still a chance you could find yourself on a spammer's list, but these rules can help you to avoid spam.
This article was first published in Technology & Business magazine.
Click here for subscription information.
| |