Microsoft outage down to 'human error'

A simple mistake by a systems administrator led to the Windows Genuine Advantage problem that affected thousands of users

A problem at the weekend with Microsoft's Windows Genuine Advantage was down to "human error" during an update, the company admitted on Tuesday.

The wrong tape was loaded by accident, resulting in WGA treating users' legal software as illegal or counterfeit. Although the immediate problem was fixed in half an hour, the effects ran on longer.

"It was human error when the wrong software was loaded during an update," Mike Haigh, Microsoft's Windows client marketing manager, told on Wednesday. "Code was sent out that should not have been. It shouldn't have happened."

According to Microsoft's Alex Kochis, senior product manager for WGA, two things happened that should not have. "First, activations and validations were both affected when pre-production code was accidentally sent to production servers," he wrote in the WGA blog.

"Second, while the issue affecting activations was fixed in less than 30 minutes (by rolling back the changes) the effect of the pre-production code on our validation service continued after the rollback took place," he wrote.

On WGA, product activation and validation are two separate processes for greater security. If either fails then WGA will assume that the product is illegal or in some way not satisfactory and should then go into what is called a "Reduced Functionality Cycle", which means that the user will gradually lose the different capabilities of the software.

Some users first noticed the problem when they lost the Aero user interface from Vista.

According to Microsoft's Haigh, there was little chance of wider damage as a result of the incident. "We have built enough safeguards so that in the event of an outage everybody would be OK," he said.

He also stressed that incidents like this, in which users may lose functionality on their systems, are rare. "This is the first issue of this type I have heard about with WGA," he said.

Haigh said the problem was a difficult balancing act. "How do you balance what our customers need and the need to protect our intellectual property?", he said. "I believe we do a good job on this."

Haigh said he did not know if any UK users were affected by the issue with WGA.