U.S. blunders with keyword blacklist

COMMENTARY--The U.S. government concocted a brilliant plan a few years ago: Why not give Internet surfers in China and Iran the ability to bypass their nations' notoriously restrictive blocks on Web sites?

Soon afterward, the U.S. International Broadcasting Bureau (IBB) invented a way to let people in China and Iran easily route around censorship by using a U.S.-based service to view banned sites such as BBC News, MIT and Amnesty International.

But an independent report released Monday reveals that the U.S. government also censors what Chinese and Iranian citizens can see online. Technology used by the IBB, which puts out the Voice of America broadcasts, prevents them from visiting Web addresses that include a peculiar list of verboten keywords. The list includes "ass" (which inadvertently bans usembassy.state.gov), "breast" (breastcancer.com), "hot" (hotmail.com and hotels.com), "pic" (epic.noaa.gov) and "teen" (teens.drugabuse.gov).

"The minute you try to temper assistance with evading censorship with judgments about how that power should be used by citizens, you start down a path from which there's no clear endpoint," said Jonathan Zittrain, a Harvard University law professor and co-author of the report prepared by the OpenNet Initiative. The report was financed in part by the MacArthur Foundation and George Soros' Open Society Institute.

That's the sad irony in the OpenNet Initiative's findings: A government agency charged with fighting Internet censorship is quietly censoring the Web itself.

The list unintentionally reveals its author's views of what's appropriate and inappropriate.

The IBB has justified a filtered Internet connection by arguing that it's inappropriate for U.S. funds to help residents of China and Iran--both of which receive dismal ratings from human rights group Freedom House--from viewing pornography.

In the abstract, the argument is a reasonable one. If the IBB's service had blocked only hard-core pornographic Web sites, few people would object.

Instead, the list unintentionally reveals its author's views of what's appropriate and inappropriate. The official naughty-keyword list displays a conservative bias that labels any Web address with "gay" in them as verboten--a decision that affects thousands of Web sites that deal with gay and lesbian issues, as well as DioceseOfGaylord.org, a Roman Catholic site.

More to the point, the U.S. government could have set a positive example to the world regarding acceptance of gays and lesbians--especially in Iran, which punishes homosexuality with death.

In order to reach the IBB censorship-evading service, people in China or Iran connect to contractor Anonymizer's Web site. Then they can use Anonymizer.com as a kind of jumping-off point, also called a proxy server, to visit Web sites banned by their governments.

Ken Berman, who oversees the China and Iran Internet projects at IBB, said Anonymizer came up with the list of dirty words. "We did not," Berman said. "Basically, we said, 'Implement a porn filter.' We were looking for serious, hard-core nasty stuff to block...I couldn't come up with a list (of off-limits words) if my life depended on it."

In an e-mail to the OpenNet Initiative on Monday morning, Berman defended the concept of filtering as a way to preserve bandwidth. "Since the U.S. taxpayers are financing this program...there are legitimate limits that may be imposed," his message said. "These limits are hardly restrictive in finding any and all human rights, pro-democracy, dissident and other sites, as well as intellectual, religious, governmental and commercial sites. The porn filtering is a trade-off we feel is a proper balance and that, as noted in your Web release, frees up bandwidth for other uses and users."

OpenNet Initiative did its research by connecting to the Anonymizer service from computers in Iran and evaluating which Google Web searches were blocked that theoretically should not be.

The report concludes: "For example, usembassy.state.gov is unavailable due to the presence of the letters 'ass' within the server's host name, and sussex.police.uk is unavailable for the same reason. In addition, the words 'my' and 'tv,' which are also domain suffixes, are filtered by IBB Anonymizer. As a consequence, all Web hosts registered within the domain name systems of Malaysia and Tuvalu are unavailable."

"For example, usembassy.state.gov is unavailable due to the presence of the letters 'ass' within the server's host name." --OpenNet Initiative's report

Harvard University's Berkman Center worked on the project, as did the University of Toronto's Nart Villeneuve and Michelle Levesque. They tested only connections from Iran, but Anonymizer said the same list of keywords was used for China.

The U.S. government "asked us to filter broadly based on keywords to generally restrict" Web sites, says Lance Cottrell, founder and president of San Diego-based Anonymizer. "What they didn't want to get into was something complex, fine-grained filtering which is going to try to remove all the porn. What they wanted was something that would generally remove most of the adult content while not blocking most of the information that these people need."

Cottrell said Anonymizer would manually unblock non-pornographic Web sites if requested by Chinese or Iranian Net surfers. "Literally, we have never been contacted with a complaint about overbroad blocking," he said.

Monday's report also takes a swipe at IBB and Anonymizer for not using the SSL encryption method to scramble the Web browsing behavior of Iranian citizens. "I would think that if the U.S. government is going to go through the trouble of funding and offering the service, they might offer the more secure one," Harvard's Zittrain said.

Anonymizer's Cottrell said he discontinued that feature because "it seemed to cause trouble for a lot of people. The utilization of the service went way down." Iran currently doesn't monitor the contents of Web pages downloaded. But if that changed, encryption would be turned back on, Cottrell said. (Because China does do that kind of monitoring, SSL is already enabled for Chinese users.)

This episode represents a temporary black eye for IBB, but it should also serve as a permanent lesson to the agency. When American taxpayers are paying the bill, any "anticensorship" scheme needs to be beyond reproach.

biography
Declan McCullagh is CNET News.com's Washington, D.C., correspondent. He chronicles the busy intersection between technology and politics. Before that, he worked for several years as Washington bureau chief for Wired News. He has also worked as a reporter for The Netly News, Time magazine and HotWired.