While the Illinois Institute of Technology Research Institute (Iitri) review team confirmed that the software program can snoop on email communications in a manner limited by a court order, it voiced concern over the lack of any method of assuring that FBI agents don't abuse the system.
"[In its analysis], Iitri did not find adequate provisions -- [for example], audit trails -- for establishing individual accountability for actions taken during the use of Carnivore," stated the report.
That lack of accountability means that FBI agents could pry into the communications of suspected criminals, public figures and the average citizen with equal ease and without a court order -- and all without leaving any evidence that they had done so.
Overall, however, the team found that the FBI had produced a program that works as advertised.
The FBI started the Carnivore project -- originally under the name "Omnivore" -- in February 1997 to design a system that could perform the equivalent of a telephone wiretap on Internet communications. Information referring to a prior system dating back to early 1996 has remained classified but hinted at in documents released under the Freedom of Information Act.
When news of the system leaked to the public in July, privacy advocates immediately cried foul and sued for details of the software. Responding to congressional concerns, the DoJ decided to fund a study of the system by an independent team of researchers.
Despite several top-notch universities' public decisions not to submit proposals due to the restrictive guidelines, the DoJ selected Iitri in mid-September to analyze the system.
The DoJ received its analysis last Friday and edited out sensitive material over the weekend.
"We found that the system does not over-collect, and that it basically does what it's represented to do," said Henry Perritt Jr, head of the review panel and dean of the Chicago-Kent College of Law, in a Wall Street Journal report. "Some of the larger concerns were way, way overblown."
One of those concerns -- the alleged ability of the system to monitor all traffic -- has no factual basis, stated the report. "Carnivore does not have nearly enough power 'to spy on almost everyone with an email account'," said the seven member team in the report. In addition, the software cannot change or introduce new data into the network.
However, the system is a tool without safeguards to prevent misuse, according to the report. The system does not keep track of who accessed it and when. Every operator has the same user name -- "administrator" -- and there is no feature for confirming that the wiretap has been ordered by a court. In addition, the report warned that misuse of the system could result in significant privacy violations, confirming the accusations of many privacy advocates.
"While the system was designed to, and can, perform fine-tuned searches [of personal email], it is also capable of broad sweeps," said the review team in the report. "Incorrectly configured, Carnivore can record any traffic it monitors."
As if that wasn't enough, several bugs were found by the Iitri team and reported back to the FBI.
The balanced criticism of the review team has apparently taken many in the privacy community by surprise.
Reacting to Tuesday press reports indicating that the review team had released a more favorable analysis, the American Civil Liberties Union criticised the report before the DoJ published it on the Web.
The Electronic Privacy Information Centre reiterated criticisms of the review soon after its release, saying that the analysis raises more questions than it answers, but then quoted the review team's own observations regarding lack of accountability.
With the increasing use of wiretaps by law enforcement, the Privacy Foundation called for a revamping of surveillance legislation to give citizens the same protections on the Internet as they have on the telephone.
"The trend in law enforcement... is to use technology to the utmost," said Stephen Keating, executive director of the Denver-based organisation. "Given the rising use of email and the surveillance potential of Carnivore, federal lawmakers should put Carnivore on a very tight leash."
The report also gave more details of two components that -- with Carnivore -- form the DragonWare Suite.
Known as Packeteer and CoolMiner, the two components aid agents in analyzing the raw data collected by Carnivore.
Packeteer processes the raw output of Carnivore to turn Internet data into more meaningful information about the targeted communication. CoolMiner uses statistical analysis to summarise the Internet traffic, display TO and FROM information, or show the full content of the message.
According to the report, a major improvement to the system -- Carnivore 2.0 -- is currently entering preliminary tests.
As part of the report, the Iitri team recommended that the FBI create separate versions of the software: one to satisfy court orders to trace where messages are going, and another to actually tap into the message content itself.
In addition, the review board asked that new versions of the software identify operators and provide a log of what each operator did, so that agents can be held accountable for their use of the system. It also recommended that the configuration of the system -- including what users are targeted and how much information is collected -- be tied indelibly to the actual data.
Finally, though the team agrees that Carnivore's current source code should not be released for security reasons, future versions should be.
The draft report is available for comment until Dec. 1, at which time the review team plans to finalise the analysis of the Internet surveillance system. Iitri plans to provide the final report to the DoJ on 8 December.
Have your say instantly, and see what others have said. Click on the TalkBack button and go to the ZDNet News forum.