JamSpam: It just might save the Internet

You asked me to organize an industry-wide anti-spam consortium, and now it looks like you're going to get it.

Last week, I had the pleasure of attending and speaking at a spam conference held at MIT in Cambridge, Mass. To the best of my knowledge, this was the first time that some of the world's brightest and most respected people (including open source guru Eric Raymond) gathered to discuss this scourge. With a little luck (and your show of approval in ZDNet's TalkBack forums), it won't be the last time such a group convenes.

Discussion at the MIT conference was largely academic yet absolutely fascinating. In a series of captivating 20-minute presentations, anti-spam experts described the level of success and failure that their anti-spam algorithms, lodged at the edge of the network, were experiencing so far. If successful (and, even by the presenters' own standards, none were 100 percent successful), these algorithms might one day keep our inboxes unclogged, removing significant interference to our productivity and minimizing the drag on valuable resources such as storage and bandwidth.

The majority of presentations were deeply technical, discussing the science of hashes, polynomials, corpuses, how anti-spam algorithms can mirror electrical engineering principles, and other hard-core science principles. Much of it boggled my mind, but I found the candor and knowledge-sharing--even among potential competitors--to be a refreshing change from other industry events.

As the next-to-last scheduled speaker of the day, I knew I would be raising an issue that remained largely unaddressed by the plethora of technical wizards and anti-spam magicians.

Only theWorld.com president Barry Shein raised this other spam issue, which lives at the heart of the Internet where the ISPs are attempting to do their part: the practice of blacklisting.

As I've detailed in a previous column, false-positives are an unfortunate consequence of the blacklisting practice used by ISPs. A false-positive happens when a legitimate e-mail is falsely classified as spam and thus denied safe passage through the Internet. Anecdotal research suggests that false-positives number in the millions per year, if not the tens of millions. The e-mail that I've received from Internet users at large and small businesses indicates that the false-positive phenomenon isn't reserved for the lunatic fringe. Individuals and organizations from all walks of life report that their e-mail is disappearing into the ether.

More often than not, end users are left with limited, if any, recourse to fix the broken link in their e-mail chain-- if they can even find it.

At the conference, theWorld.com's Shein chronicled the mayhem that often confounds the ISP's recovery-focused technical staff after a spammer unleashes a torrent of SMTP traffic laced with falsified headers in such a way that the resulting rejection notices and "bounces" are almost as bad as the spam itself. Shein said the Internet will break if something isn't done soon. I agree-and I've made similar arguments in previous columns.

At the very least, as the false-positive problem continues to grow (paralleling the Internet's growth), the SMTP network (the virtual part of the Internet devoted to Simple Mail Transfer Protocol, aka "Internet e-mail") could easily become gridlocked. Ironically, should we get to an irretrievable breakdown of the SMTP network, the spammers will be in the same boat as the rest of us.

But, even if the SMTP network isn't 100 percent gridlocked, we still have a big problem with the false-positives. The best anti-spam solution in the world for your e-mail client or server doesn't amount to a hill of beans for the legitimate e-mail that never made it in the first place. That e-mail could be one that I sent you or the one that you sent me. It could be the one with the time-sensitive financial data for your business or the addendum to your graduate school application. Get the picture?

I think edge solutions are great. But what are we going to do about the other part of the problem--the blacklists at the heart of the Internet and the potential breakdown of the SMTP network?

I'll tell you what we're going to do. In one of my previous columns, I asked if an industry-wide anti-spam consortium could be formed to come up with a universal solution to the problem. One possible solution would be an open, royalty-free, interoperable anti-spam protocol that can be built into all e-mail clients, servers, and services. Many of you said yes, and even asked where to send money to support such an effort.

Over the last several weeks, I've been asking the primary members of the e-mail ecosystem to commit to starting such a consortium with the goal of producing a standard solution that those organizations build into their products and services.

I approached the companies that make the most popular e-mail technologies and those who run the biggest e-mail services. I explained how such an industry-wide effort isn't only for the betterment of the entire Internet community, but also for their own commercial benefit. After all, if the SMTP network breaks down, the value of their products and services is greatly diminished. (It's that very principle that caused bulk e-mail technology provider IronPort Systems to come up with the Bonded Sender program. Legitimate bulk e-mailers found that blacklists were stripping IronPort's solution of its effectiveness.)

The response from the e-mail ecosystem has been overwhelmingly positive. Already, a handful of well-known companies have officially accepted my invitation. More companies are expected to respond by the end of this month and, with any luck, the consortium (codenamed "Project JamSpam") will be official by March.

Now is your chance to let all of the members of the e-mail ecosystem know that you'll stand for nothing less than an industry-wide consortium committed to an open, royalty-free, interoperable, anti-spam protocol. Show your vote of approval where it can be seen -- vote in our QuickPoll or leaves a message in ZDNet's TalkBack forum. If you want get involved, write to david.berlind@cnet.com.