Yahoo! How did it happen?

Experts are still picking through the rubble of the massive denial of service attack that downed Yahoo.com for three hours.
Written by Bob Sullivan, Contributor
There was an eerie quiet among the hacking community on Monday night, in the wake of the Yahoo! service outage. That was unusual in a community where intruders are normally more than willing to take credit for computer mischief.

There are still few clues about who's responsible for Yahoo!'s (Nasdaq: YHOO) outage yesterday -- but there are some definitive hints about how it was done.

There's nothing new about denial of service attacks, where a Web site is overwhelmed with so many requests that legitimate users get the cyber equivalent of a busy signal. But they have come into fashion of late, and updates to this old-fashioned cyber ploy have made "DoS" attacks freshly sinister.

Yahoo! COO Jeff Mallett left no doubt in statements yesterday that his site was crippled for three hours by a denial of service attack.

"It appears that unfortunately it was a planned attack that looked like it was directed at Yahoo! specifically," Mallett said on CNBC. "We had an excess amount of automated mock traffic that was generated to one of our server clusters here in California ... where there was such demand in a short period of time that we were unable to serve all the pages that were requested."

Site is now safe
Mallet said no data was lost, and the company's site is now safe from the attack. But with computer vandals having demonstrated that even the Web's biggest site isn't safe, the rest of the Internet is left wondering how it happened.

"There is something fishy going on here," said Elias Levy, executive at consulting firm SecurityFocus.com. "This is not just your regular DOS."

Sources close to Yahoo!'s internal investigation of the incident say the company is still sorting out the possibilities.

This much is known: Mallett made clear in statements yesterday that Yahoo!'s routers were overwhelmed during the attack. Routers act like mechanical air traffic controllers for traffic coming in and out of computer networks. Mallett's repeated use of the term was a signal to computer experts.

There's a new attack which can compromise computers and routers, released just last month, according to Russ Cooper, who runs the popular security mailing list NTBugTraq. It's known by several names - stream.c, happy.c, spank.c among them.

"There have been numerous copies of the thing floating around," Cooper said. In addition to sending routers into a spiral that eventually causes complete service disruption, the attack exposed new vulnerabilities in the FreeBSD operating system.

That's the software Yahoo! uses to run its site.

"I would find it hard to believe that they wouldn't have updated, but that's corporate life," said security expert Tim Yardley, who has done extensive research on stream.c "The thing is to update, you have to reboot your systems. They might not have gotten around to it yet. It's relatively new."

Spiral effect
The attack works like this - every time information is sent across a network, the receiving computer sends an acknowledgement that the information arrived successfully. In the stream attack, such acknowledgements are sent without any preceding data, which confuses the router, and sends it into a spiral. The attack is particularly tricky because the acknowledgements are generally harmless traffic generated by friendly computers from inside a network.

But Levy argues that stream.c's effect on servers has so far proved to be erratic, and said in its current form could not have caused the massive Yahoo! outage. He suggested another possibility known as DDOS - Distributed Denial of Service attack.

The National Infrastructure Protection Center, an arm of the FBI dealing with computer security, issued a warning in December about DDOS, also called "Tribal Flood." Using this technique, computer vandals first hide rogue programs on several large computers around the Internet, usually university computers. Then later, the programs are activated simultaneously with a focused attack on one system. NIPC said in December it had found such rogue programs planted on computers around the Internet. According to the NIPC warning, "Many of the victims have high bandwidth Internet connections, representing a possibly significant threat to Internet traffic." Were all that bandwidth focused at Yahoo!, it could cause a shutdown of the service.

One source told MSNBC that 3,500 computers were aimed at Yahoo! yesterday during the height of the attack using the DDOS method.

But if that were true, Yahoo!'s Web services provider, Global Center, should have experienced other problems.

"Global Center claims it did not affect any of their other customers," Levy said. "That does not make sense. If it was a bandwidth starvation attack the packets going to Yahoo! must have crossed Global Center's network and at least degraded the performance of some of their other customers."

Spillover effect
But it's possible Yahoo!'s account is so large that it's entirely separate from any other Global Center accounts, Levy said, and thus would have been protected from such spillover effects.

Still another possibility is the so called "syn" flood attack, which has been used since at least 1997.

A "syn" is a request to synchronize, essentially a request for one computer to connect with another. Vandals discovered years ago that they can confuse Web sites by altering the address of the computer used to dial up a Web site - the one sending the initial "syn" message. When the Web site tries to complete the conversation, it's trying to connect to a computer which doesn't exist - and wastes valuable processor time. If enough faked "syn" packets are sent, the computer eventually collapses.

Whatever the theory, security experts are anxious to know what happened and how Yahoo! solved the problem. For example, there is currently no known cure-all to protect sites from a well-organized DDOS attack, so if Yahoo! found one, the information needs to be shared, according to Levy.

"The truth is right now we really don't know what happened. We need more details," he added.

Editorial standards