Are Microsoft's update servers broken?

Are Microsoft's update servers broken?

Summary: Millions of people rely on Microsoft's Automatic Updates and Windows Update to deliver critical security patches. But four days after this month's Patch Tuesday, those updates are not being delivered for many Windows users. Windows Update log files point to "heavy download traffic" as the culprit. Are Microsoft's servers collapsing under the load?

SHARE:
TOPICS: Windows
117

Update 14-Aug: I've posted a follow-up here with a few more details.

On Tuesday, August 8, Microsoft released a total of 12 security updates for different versions of Windows, Office, and Internet Explorer. One, MS06–040, is a vulnerability in the Windows Server service and affects all versions of Windows XP. The latter is so serious, in fact, that the United States Department of Homeland Security issued an urgent news release recommending that all Windows users apply this security patch as soon as possible. At least seven other updates on this month’s list are rated Critical for users of Windows XP with Service Pack 2 and involve the potential for “an attacker to compromise your Windows-based system.”

All those patches are supposed to be released via Automatic Updates and automatically delivered This week, Microsoft's update servers have apparently collapsed under the load to computers that have the Automatic Updates service turned on. They’re also available via Windows Update. Millions of people rely on these servers to keep their Windows computers secure.

But this week, Microsoft’s update servers have apparently collapsed under the load. In my office, I have four computers running various flavors of Windows XP. All have Automatic Updates enabled. Four full days after the release of these Critical updates, only one of those machines is fully patched. It turns out I’m not the only person experiencing problems. A search of public newsgroups turns up dozens of complaints about Windows Update and Automatic Updates not working properly:

August 9: When running Window update it hangs up during it's search for updates every time I try it. The only way out is to power down. Task manager shows the cpu usage at 100%.

August 9: I've never had any issues w/Windows or Microsoft Update until today's patch release. Some installed, but KB920214, KB920683, KB921398, and KB922616 will not install either via Automatic Updates, or by me downloading the file from MS Downloads and running it myself. Each of them fails w/the same sequence of errors…

August 9: I have my updates set to notify me when there is an update. I tried to downloand them yesterday, 8/8/06, and they would not install. When I tried to access them via the windows update site...it just kept seaching for updates...then froze... I spent 2 hrs..on the phone with microsoft...clearly there is a problem they are not yet aware of.

August 10: I'm having the same problem with the August updates. This never happened before. It happens both with automatic updates, and by selecting Tools -> Windows Update from the IE menu. My resources get so taxed I can't even pull up task manager or restart, I literally have to unplug the computer! I'm running Windows xp sp2 and have made no software or setting changes since the July updates. I finally disabled automatic updates til MS acknowledges and fixes the problem.

August 11: Each time I've turned on the computer for the past few days a small yellow shield pops up in the system tray saying the updates are being downloaded. But the shield disappears after about 30 seconds and nothing is downloaded.

August 11: We have setup our computers for automatic Windows Updates. It seems as though none of these updates are actually running. There is an error in the event viewer … “Unable to Connect: Windows is unable to connect to the automatic updates service and therefore cannot download and install updates according to the set schedule. Windows will continue to try to establish a connection.”

I can confirm all of these experiences. On three of four XP-based computers here, the MS06–040 patch was properly installed, but no other updates were delivered. A check of the Windows Update History shows a gray Cancelled button next to one or more of the missing updates. The Windows Update log file (%windir%\Windowsupdate.log) on two computers shows dozens of attempts to download the other patches, followed by this error message: Update is not allowed to download due to regulation. Microsoft Knowledge Base article 910340 explains what this message means:

During periods of heavy download traffic, the Automatic Updates service can reschedule download requests on a day-to-day basis. This rescheduling can occur over several days.

I’ve looked in vain to find what Microsoft’s guarantee of service is on Automatic Updates. In the past, I’ve never had to wait more than 48 hours to receive updates. A four-day wait is simply inexcusable.

Update 12-August 4:30PM PDT: No, it's not my imagination. The performance of Automatic Updates has definitely slipped recently. I have one machine that has been in continuous operation and has been configured for Automatic Updates since late 2004 (that's when Microsoft inaugurated the "Patch Tuesday" program, in which Critical security updates are released on the second Tuesday of each month). By checking the Windows Update history on that machine, I was able to plot when the updates were actually delivered and compare the delivery dates to Patch Tuesday. In 11 of 13 months between December 2004 and January 2006, this machine received updates on the Wednesday or Thursday after Patch Tuesday. On average, it took 2.0 days for Automatic Updates to arrive in 2005. Since February 2006, those updates have routinely arrived 3-5 days after they were released, an average of 3.4 days later. To put it another way, only once in the 14 months between December 2004 and January 2006 did I have to wait four days for updates to arrive. Since the beginning of 2006, Automatic Updates have been delayed for at least four days in four of seven months for which I have data.

Ironically, the problem has also affected the computer I’ve been using to test Windows Genuine Advantage. On Thursday, I reinstalled a volume license version of Windows XP using a blocked product ID supplied to me by Microsoft; since then, I’ve been unable to download any updates, including the WGA validation and notification components. When accessing Windows Update, the system either hangs completely or displays an unhelpful error page (0x8DDD0009). At irregular intervals, Automatic Updates displays a yellow shield icon in the taskbar, which notes that downloads are updating and displays a progress meter reading 0%. It goes away after a few minutes without succssfully retrieving any updates. So this computer is blissfully free of WGA nag messages; unfortunately, it’s also free of crucial security updates.

On two “certified Genuine” computers, one running Windows XP Media Center Edition 2005 and another running Windows XP Home Edition, I was able to manually install all of the missing Critical updates by visiting either the Microsoft Update or Windows Update site. But having Automatic Updates turned on means I shouldn’t be required to do that.

A few weeks ago, Microsoft announced it was going to push IE7 as an automatic update when it’s finally released this fall. At the time, I thought that was a pretty good idea. Now I’m not so sure. If the infrastructure for delivering Critical security updates is failing under the current load, what impact will a big update like IE7 have?

Microsoft needs to explain exactly what’s going on here. What’s the problem with Windows Update, and when will it be fixed?

Topic: Windows

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

117 comments
Log in or register to join the discussion
  • Windows Update is working fine here!

    Windows Update successfully downloaded and installed the latest critical updates on three different computers I use on 08/08/06. No issues at all. Yesterday (08/11/06) I download and installed all the recommended critical updates for a computer running Windows Server 2003. Again, I did not have any issues.
    This was my experience and I cannot vouch for issues other users may have had for one reason or another.
    cnfrisch
  • More FUD...

    My rep hand delivers me all of the patches on "Gold DVD Media". We need to maintain 2 Windows XP workstations here for a legacy application. I have threatened my MCSDs to upgrade this app to Vista or else. Once the app is upgraded, we will be 100% Vista and we will no longer have to patch.
    Mike Cox
    • LOL

      Better watch out boy. You are gonna threaten the wrong person one day
      X41
    • LOL again (NT)

      :-)
      Mikael_z
    • Message has been deleted.

      Jay E Court
    • :-D (nt)

      .
      LowerCal
  • Automatic updates no, manual yes

    For me, automatic updates are refusing to be very useful. I have it set to notify me of new updates, but all that's been there is the Windows Genuine Advantage tool, no others are listed as available.

    In the end, I just went to install them manually on windowsupdate.microsoft.com, where it found them all, and I was able to install them all without problem.

    Tip of the day: Don't always trust Automatic Updates to keep you up to date auomatically.
    Kaleld
  • I think this is an abomination.

    Enough said.
    jsaltz
  • Backwards

    Someday Microsoft may develop a "swarm" technology capable of using the otherwise unused upload bandwidth of the hosts of machines downloading patches. They could "seed" a "tracker" at microsoft.com, then direct potential downloaders to clients which already had the pieces required.

    Oh, well, it's only a dream. Maybe Microsoft Research will someday overcome the obstacles to implementing something like it. Perhaps they can ship it with Cairo.
    Yagotta B. Kidding
  • The servers aren't broken, MS has decided all copies of Windows are fake

    LOL!.....Could it be?

    Would they really do that?

    That's the best excuse I can come up with for them to deny critical updates to everyone.

    You see what we powerless public get for complaining about WGA?

    Hey, have you checked to see if at least the WGA update was still available?.....Maybe you have to download that first....LOL!
    Yodaddy
    • No, the WGA update is not available

      As I noted in the post, my pirated copy of Windows XP is not displaying any WGA error messages, because it can't contact the update servers to download those components!
      Ed Bott
      • I understood, I was making a joke about it.

        Sorry for my choice in humor. Some things ain't funny.

        I had autoupdate turned off but I managed to capture all the most recent critical updates late yesterday evening.

        I think it's ironic that Microsoft insist we get this most critical update, but their servers are only sputtering them out.
        Yodaddy
    • MS Server exceeded bandwidth??

      Could be that the updates are slow because MS uses all their bandwidth trying to find, warn and badger XP users????

      Try to dismantle REMOTE Access on your XP and see what happens.

      I have XP legit. I have called 2x because I cannot activate. MS gives me a code and it works ok.

      Today, I attempted to install components from XP original disk. ERROR-426 or 448..Windows Catalog not found???verification problem..cannot continue
      install.

      I have performed a Clean Install 5x in 45 days..this is getting old.....
      I wonder if the Pres had to download the tool? Does it call the Whitehouse everyday??

      Who needs the FBI when we have MS???
      sarahbeth0404
  • updates

    Microsoft sent me an email saying they were upgrading their update computers to 64 bit units that could simultaneously upload 32 and 64 bit updates. My 64 bit stuff all works, but my 32 bit updates took many tries to reach an update computer that worked.
    jdyagi
  • Windows Update

    I HATE WINDOWS UPDATE!

    But...I suppose ONE PC with WinXP isn't TOO bad.

    I got the updates twice - once on Tuesday and again on Thursday.

    This isn't a critical PC, so I haven't bothered to check to see if they actually downloaded and installed properly, although it stated it had done so - both times.

    I'm wondering how many updates the PC will take before it crashes or the drive gets full.
    outinarizona-21313778528081432291957710209928
  • Notify Only

    I have AU set to notify me of critical updates. I used to click "download" for the updates that I wanted. Starting a few months ago, when I clicked "download" the gold shield would disappear a few seconds later and the updates would not be downloaded. I would have to go to Microsoft Update to get those updates. Starting this month, I simply read the notification and then clicked "cancel" (didn't tell it to hide the updates). Then I went to Microsoft Update to download and install the updates I wanted. The downloading and installing went off without a hitch. This is what I will do from now on.
    SMP0328
  • How Open Source solves that problem.

    How come non-profit linux and BSD can do this with the OS and a ton of apps?

    1) You can download the fixpack once and share it with multiple computers... on linux. Used to be that you could do the same with Windows 95/98, etc. Now, no more. Each home computer must check in with the mothership and get its updates one at a time.

    2) Open Source OS's use multiple mirror sites, so that all the load doesn't fall on the mothership. Think of "Akamaization long before Akamai existed". Because WGA requires each home PC to contact the mothership, independant mirroring is not an option for Windows; too bad.

    3) When an Open Source app or kernel is patched/upgraded, the fix goes out *NOW*. It doesn't wait for next month's "Patch Tuesday". The result is a managable low constant load on the delivery servers, rather than a panic once a month, followed by 4 weeks of idling.
    Knorthern Knight
    • The topic of discussion was "Are Microsoft's update servers broken?"

      The topic of discussion was "Are Microsoft's update servers broken?" I fail to see how your post and response is even remotely relevent to the topic.
      cnfrisch
      • Seems relevant to me....

        As it highlights one of the many issues with MS, and why it so convoluted & difficult to maintain.
        (especially for a casual user)

        If you use Windows, and are concerned with security, I think it far better to do it manually. As most important & critical updates (performance & security) sometimes only trickle through AutoUpdate. Some have taken several weeks if not a month to get out through AU, yet they push out something like WGA rapidly.

        Extra Revenue..... Just like WGA, they creating the impetus for other income... OneCare & Managed Solutions, etc.

        One every couple of months I will do a complete system update. This not Only will update the OS & services but all the apps I use. Last night I did this, and updating over 500 apps/services/packages took about 15 minutes to download another 10 minutes to apply without a reboot, all while still surfing.

        if necessary...
        After testing I can now push it to hundreds if not thousands of machines.

        For the casual user with one machine or those who manage/administer many. Linux/BSD is far easier, less costly & time consuming to maintain than Windows. (I manage some of those as well.)

        So yes it seems very relevant to me.......
        LazLong