Bugs at Internet speed?

So say critics of two plans by anti-virus vendors to automatically deliver updates to desktop PCs.

The systems for delivering anti-virus updates are IBM Corp.'s Digital Immune System, to be deployed by Symantec Corp. (symc) and Network Associates Inc.'s Rumor. They will respond to virus attacks by pumping the latest digital cures down to hundreds of thousands of PCs. In a few years, that number could be in the millions.

But are these automated anti-virus offerings a panacea or the makings of a disaster?

"If they did everything perfectly, there wouldn't be a problem," said Fred Cohen, a security consultant, researcher, and author of the first paper on computer viruses in 1984. A single error, however, could transform the system from helpful to harmful, he said. "If the update gets corrupted, you are essentially sending (malicious code) out to everyone as legitimate updates."

That's not so unlikely as it sounds. On Wednesday, an Oklahoma coal and natural-resource venture reported that an update from Network Associates Inc.'s (neta) McAfee business unit had disabled 250 of the firm's machines for most of the day. In early October, updated definitions for Symantec's Norton Anti-Virus conflicted with Network ICE Corp.'s BlackICE firewall, shutting it down and leaving affected systems open to exploit.

McAfee and Symantec are not alone. Almost every anti-virus maker has erred with its updates in the past. One reportedly missed detecting the QAZ Trojan in early October. Another dropped several old definitions from its lists, resulting in viruses thought to be a decade dead popping back up on computers worldwide.

With the new technologies, isolated incidents could become worldwide problems.

Today, program and virus-definition updates are posted to servers on the Internet, and wait there until clients download the data. In some cases, the process is automated, but good system administrators will test the updates, said Rob Rosenberger, anti-virus industry watcher and editor of Virus Myths.

"In a large organization, you need to know that the updates work with the software that's on the PCs," he said. "If you automatically update your executables, it may one day break your system."

Yet companies are the ones clamoring for a solution that delivers fixes as fast as possible. "In their mind, faster is better," said Rosenberger.

That's understandable. During the Melissa outbreak, anti-virus software makers responded in many cases within 12 hours to the new macro virus.

For panicked customers, however, that wasn't fast enough. The number of unique visitors on McAfee's Web site spiked from an average of 350,000 a day to more than 8 million, said Victor Kouznetsov, vice president of engineering and chief technology officer for Network Associates' MyCIO.com business unit. "Virus writers are always one step ahead," he said. "We need a system to help ... them."

Well, fast is here. The Digital Immune System and Rumor are the anti-virus industry's answer to outbreaks of digital diseases such as Melissa and LoveLetter. The Digital Immune System -- based loosely on an analogy of the body's reaction to disease -- pushes solutions down to individual customers as they become available.

Rumor, the technology announced last week by MyCIO.com, uses a more sociological analogy -- the passing of a juicy rumor between friends -- to deliver definitions by the currently vogue method of peer-to-peer networking.

Both systems will result in the latest definitions being distributed quickly to customers. That means less time for PCs to be vulnerable.

Yet, if an error creeps into either the definitions or update code -- or if the software conflicts with other applications -- it also means less time for companies and anti-virus firms to react to the mistake.

And errors do creep in. Just ask Richard Armstrong.

On Wednesday morning, calls started trickling into Armstrong, a system administrator for a Tulsa, Okla.-based coal and natural resource venture. Each caller complained that his or her computer would not boot up. Soon after, the trickle turned into a torrent.

By noon, almost 250 computers at the office -- more than 80 percent of the company -- were unusable, said Armstrong, who asked that his company's name not be used.

Armstrong and other system administrators puzzled through the problem and by early evening found the cause: The latest updates from anti-virus software maker McAfee had reacted badly to the Oklahoma company's own software, fouling up the works.

"We had people here late last night," he said. "We basically had to manually remove the anti-virus software."

That's not all that unusual, said Virus Myths' Rosenberger, who found conflicts between installed software and anti-virus scanners on a weekly basis when he consulted with a large firm. He kept a 25-user license of a competitor's virus scanner to install on troublesome machines in the hopes that if one didn't work, the other would.

"It gets to the issue of 'safe hex,' " he said. "Are you checking your updates, turning off macros, and testing software? You should be."

Add the lack of testing on the side of clients to an automated system, and you have a recipe for a major meltdown.

Such problems are not flights of fancy, either. In the late 1970s, two researchers at Xerox PARC, John Shoch and Jon Hupp, wrote a paper on their experiences using the first computer worms to do network maintenance and distributed computing. "Instead of viewing (our network) as 100 independent machines," they wrote in a 1982 article in Communications of the ACM, "we thought of it as a 100-element multiprocessor, in search of a program to run."

The programs they ran were fully automated and distributed themselves throughout the system as needed -- one of the attributes of worms and, in many ways, similar to the increasingly automated update systems of today.

In the case of Shoch and Hupp, the complete lack of oversight led to problems.

One night, a small worm was left running to test its ability to control itself. When the duo arrived the next morning, dozens of the Alto computers used in their experiments were dead. Each time Shoch and Hupp restarted an Alto, it would freeze up.

"The worm would quickly load its program into (the computer); the program would start to run and promptly crash, leaving the worm incomplete -- and still hungrily looking for new (computers)," they wrote.

"The embarrassing results were left for all to see: 100 dead machines scattered about the building."

For Armstrong, a failure meant 250 computers on the blink; for Shoch and Hupp, it was 100. For a system connecting an anti-virus firm directly to all its clients over the Internet, the result could easily be 100,000 machines.

For that reason, such firms have continued to test the security and slowed the transition to the new systems. Symantec will only deploy the Digital Immune System in medium to large companies, where professionals can oversee the process. For the time being, no consumers will be using the system, said Vincent Weafer, director of Symantec's Anti-virus Research Center.

"When we look forward, we have to make sure that the infrastructure is secure," he said. Symantec also intends to make its system forgiving as well. The company will soon add an automatic rollback feature -- letting customers return to the last set of definitions -- to its newest scanner.

MyCIO's Kouznetsov agrees that security needs to be done right.

"The most important thing to get right in these systems is security," he said. "With these systems, you are starting to control the (client's) computer. Every step along the way, we thought about how to make sure that people don't abuse the system."

And so it boils down to a matter of trust, said Kouznetsov.

"There is a leap of faith here," he said. "Do you trust the people who make the solution? You trust the people who have control of nuclear weapons. We are asking you to trust us."