Microsoft: WGA meltdown due to human error

Microsoft: WGA meltdown due to human error

Summary: The glitch in Microsoft's Windows Genuine Advantage (WGA) system that affected an estimated 12,000 users over the August 24-25 weekend was due to "human error," according to a new August 28 blog posting by WGA Senior Product Manager Alex Kochis.

SHARE:
TOPICS: Microsoft, Outage
148

Microsoft has published yet more details about the glitch in its Windows Genuine Advantage (WGA) system that affected an estimated 12,000 users over the August 24-25 weekend.

Microsoft is attributing the WGA problems to "human error," according to a new August 28 blog posting by WGA Senior Product Manager Alex Kochis. Kochis explains on the WGA blog:

"Nothing more than human error started it all. Pre-production code was sent to production servers. The production servers had not yet been upgraded with a recent change to enable stronger encryption/decryption of product keys during the activation and validation processes. The result of this is that the production servers declined activation and validation requests that should have passed."

Windows XP and Windows Vista activation and validation were both impacted as a result. Kochis said Microsoft actually fixed the activation problem in less than 30 minutes, but "the effect of the preproduction code on our validation service continued after the rollback took place."

Kochis noted this past weekend's WGA problems have resulted in Microsoft putting some new systems in place, including improved monitoring capabilities "to alert us much sooner hould anything like this happen again." He said Microsoft also is working on increasing the speed of escalations and "adding checkpoints before changes can be made to production servers."

While it's admirable that Microsoft hasn't attempted to sweep this incident under the rug and has continued to follow through on the WGA blog with updates, there are still a number of things about this past weekend's meltdown that are troubling. Kochis' insistence that the WGA mess was not "an outage" is one such sticking point. On August 28, Kochis said:

"It's important to clarify that this event was not an outage. Our system is designed to default to genuine if the service is disrupted or unavailable. In other words, we designed WGA to give the benefit of the doubt to our customers. If our servers are down, your system will pass validation every time. This event was not the same as an outage because in this case the trusted source of validations itself responded incorrectly."

Outage or no outage, users reported that they were incorrectly identified as running "non-Genuine" versions of XP and Vista. In the case of Vista users, their Aero interfaces were disabled. They were considered guilty, not innocent. This presumption goes to the very heart of why WGA, as it is currently designed, is unpopular not just with Microsoft critics, but many customers, too.

Topics: Microsoft, Outage

About

Mary Jo has covered the tech industry for 30 years for a variety of publications and Web sites, and is a frequent guest on radio, TV and podcasts, speaking about all things Microsoft-related. She is the author of Microsoft 2.0: How Microsoft plans to stay relevant in the post-Gates era (John Wiley & Sons, 2008).

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

148 comments
Log in or register to join the discussion
  • This has GOTTA be a joke, right?

    "Nothing more than human error started it all."

    Reminds me of the old cartoon which shows a janitor unplugging the power cord for the hospital patient's heart and lung machine so he can plug in his vacuum cleaner.
    OButterball
    • That Sucks!

      ]:)
      osreinstall
  • So, it's OKAY for just any old programmer ...

    ... in the Redmond offices to swap code on the Windows update servers? 'Cause THAT is what it looks like yer playin' apologist for MS about.
    OButterball
  • Incorrect

    The day Apple or Linux gets a 20% market share, you will see WGA disappear in a heartbeat.
    frgough
    • Ditto

      I agree.

      WGA was intended to thwart pirates, yet it was pirates that actually assisted MS with its OS dominance. Microsoft knows this. However, once MS starts to see the scales tip, WGA will silently disappear. If there is one thing that MS does well, it is to respond to threats on the horizon.
      DarienHawk67
      • OK, I'll make it a point...

        ... to purchase a distro system this next year. I'll take a chance that it may work. I'm tired of M$ bullying tactics from DO$1.1. They destroyed my work as a programmer in 1993, so now I only use computers for hobby. I already bought two iMacs - but cannot use them because of the way their mouse hurts my wrists. No more Macs; but I'll go with something, probably not Suse, since M$ now "owns" it. Perhaps Ubuntu or Linspire (since they first irked M$, ... and have signed some useless "agreement" to keep them at bay).
        Media-Ted9
        • Purchase a new mouse

          Hi Media-Ted,
          If the Mac mouse is hurts your wrist purchase a new one. Giving up a Mac or PC
          because of a mouse sounds ridiculous. But you'll go to Ubuntu perhaps, you're
          smart enough to know the OS doesn't cause your wrist to hurt.

          Ted, just purchase a USB mouse more ergonomic for you. Any USB mouse you
          desire to use will perform properly on the Mac or PC. On computers where I didn't
          like the feel of the mouse I'd purchase one that better fit me. The cost was less
          than $20.00.
          BubbaJones_
        • Would that worked, but alas ...

          ... no matter what mouse I have used, at home or at work, including the really cool supermouse, the slingshot effect is still present.
          The fault/cause lies within the sensing program in Apple OS's since the early all-in-won-der. The system senses, and responds to movement, direction, and speed, but also thrust. It amplifies the thrust and speed changes as well as directional changes, so the user(s) end up overshooting and having to backtrack, or try to figure how to overcome the amplification of speed changes. Physicians, andChiropractors (at least those who do not use Macs) have held, in my hearing, that the Mac mouse is the largest single contributor to carpal tunnel syndrome. They have written/issued orders to patients (like my wife) to be placed in empoyee files stating that this person is not to be working on Apple products. I can run a normal PC for 10+ hours a day with no ill-effects, but one hour on any Mac product has me hurting to my shoulder. I try switching hands, but, again to no relief.

          The other reason for injury from a mouse (Logitech, Micro, Mac, any kind) is due to the manner of selection on Apples. On a PC, you can select a drop down menu which stays until you roll over a secondary selection and click. Apple requires the user to click, hold tension, and move - under tension - to the secondary selection and then release; otherwise, the selection process is dropped and must be begun ab initio. In my workplace, you can see nearly all the Mac users in wrist braces; the PC side, ... very few.

          Thirdly, a PC (power)user knows they can more easily type their way through most applications, both Windows and 3rd Party. Most of the time my fingers are in the touch-typist's "home" position. I seldom leave the keyboard to manipulate a mouse. Can't do that with any Mac, new or old. Mac was made for kids and artist/graphics people who do not type for a living, but, oddly enough, my wife, who is an artist, ... cannot use a Mac!!! Two carpal tunnel operations; that's all you get.

          I would like to see that running something like Windows on a Mac would eliminate the mouse problem, but that resurrects the problem(s) with Windows, doesn't it?
          Media-Ted9
          • Mouse settings.

            Windows and Mac OS both allow the mouse settings to be controlled (Control Panel in Windows, System Preferences in Mac OS X). Windows, too, will perform mouse acceleration (the “thrust” response you were talking about) if set to do so (in Windows XP, this was renamed to “Enhanced pointer precision” — despite what the name may imply, checking the box turns [i]on[/i] acceleration [at only one level available], and [i]un[/i]checking it turns [i]off[/i] acceleration — earlier versions of Windows called it acceleration and had at least three levels plus off).

            I just tried my boss’s Mighty Mouse on his Mac OS X, and could not discern any acceleration effect, nor was there a setting to enable such a thing.

            Finally, there is some evidence that pyridoxine hydrobromide (vitamin B6) may aid in the prevention and treatment of CTS. Some even suggest that CTS is actually a B6 deficiency disease, with the repetitive stroke injury being only a trigger, and not the actual cause, of CTS.

            The hypothesis goes like this: the salient characteristic of CTS, and what causes the symptoms, is a swelling of the myelin sheath of the great median nerve that runs through the carpal tunnel (hole formed by the seven bones of the wrist). Many other things must pass through that same tunnel, including blood vessels and the tendons that control the fingers. That leaves little, if any, room for the median nerve’s myelin sheath to swell, and the resulting pressure on the nerve sheath and the underlying median nerve itself is what causes the tingling and weakness of the three fingers furthest away from the thumb, and other symptoms.

            B6 inhibits myelin sheath swelling, so the hypothesis goes. Injury can only cause sheath swelling in the absence of sufficient levels of B6 in the body.

            I have heard both pros and cons on this, but I can only say that the one time I had really bad CTS in my life (when I was a tech support person typing dozens of letters per day), taking 75mg of natural-source timed-release B6 daily made the symptoms go away entirely in only three days, without any change in my computing habits.
            Joel R
          • Respectfully disagree

            Ted,
            It isn't the mouse, it isn't the PC and it isn't the Mac. It's the end user they way
            they hold their wrist. The same as you slouch in a chair whether using a PC, Mac
            or Unix box, read, paint, etc. This isn't rocket science you know.

            Quote:
            Physicians, and Chiropractors (at least those who do not use Macs) have held, in
            my hearing, that the Mac mouse is the largest single contributor to carpal tunnel
            syndrome.
            Unquote.

            The largest single contributor...and you believe that statement? You mean they
            went around the world talking with all the folks that had carpal tunney syndrome,
            and, came to the conclusion it was caused by the Mac mouse? You're smart
            enough to see through that statement; I hope. If that was a true statement OSHA
            would ban the Mac mouse from this country. Ted please...

            Whether you like the Mac mouse or not you don't need to use two fingers hovering
            lightly over the buttons as you do on a two button mouse; less strain on the
            fingers which are fed through the carpal tunnel.

            My wife is a nurse, years (decades ago) had surgery for capal tunnel well before
            computers as has many of her nurse, and doctor co-workers. Your statement is
            completely unfounded. If you believe the carpal tunnel statement above I have a
            bridge in Brooklyn for sale.

            In all due respects Ted you're way off base, it looks as though you're looking for
            excuses rather than looking at the user. In this instance it isn't the device, truly it
            is the end user with how they use the device. Are they constantly using proper
            wrist angle? Though we may want to we don't.

            I'm not a doctor nor to I play one on these talk-backs: take some vitamin B6 it has
            shown and proven to reduce swelling of the carpal tunnel.
            BubbaJones_
          • To reply to the above 2...

            I humbly accept the idea of the B-6. Thanks.

            As to the "professional" "Medical" response to my claims. There is in my wife's employment folder where we work, a doctor's statement that she is not to be put on a Mac at any time. I have heard it from that doctor, as well as other health care professionals, that Mac users have far more problems with CTS than any other source; and they specify the problems with Apple products. I have no reason to doubt them or their statements. Perhaps you consider it brash for me to refer to what they said as though I believe it.

            I also did not lie when I have said that I have had several years experience with the Apple mouse - many more with Apple systems dating back to around 1982. Of course, working with them 7 days a week during heavy loads we had off and on - that's 56 hours a week for a few months. Not that I was on Macs only during that time, but when the company was in need, PC operators who were familiar with both machines were pulled over until the work load was balanced. In addition,I own 2 iMacs, and can tell the difference in the way a mouse responds.

            I am fairly aware of the means of adjusting mouse attributes in PC's and Macs; in fact, I have even used the International settings and set myself up as handicapped in order to reduce any other problems with the acceleration. This is only available in OS X. I also cannot deny all the wrist braces worn by the workers, nor their numbers in the Mac and PC areas.

            I have puzzled over why so many do not claim to notice the differences, and still have no answer. I can't claim, or resent the actions of those who do not understand or admit the problems; as though they are living in "denial"; nor can I lie and fake admitting that I was wrong. I know what I have suffered, and I know that my wife cannot work on a Mac. I fully realize that there is no total agreement within any profession, especially the Health Care Growth Industry, but to discredit my statements because you may know a Health Care Professional who personally disagrees with me doesn't prove your position any more than it disproves mine.

            I have not tried to overstate the facts, especially regarding the mouse situation. I have tried to describe it in the simplest terms possible. Usually I can be expected to stretch the truth to add dimension or humor/satire in my writings, but I assure you; the problem I have described was as carefully covered as I can present it.
            Media-Ted9
          • Reduce the "Tracking Speed"

            You can turn off mouse acceleration in OS X by setting the tracking speed to Slow in Preferences. I find your story hard to believe because Windows also has mouse acceleration turned on by default, and allows it to be turned off in a similar way. Acceleration is what allows you to move the pointer very precisely when moving the mouse slowly, without having to move your mouse 8ft to move the pointer across the screen.

            Furthermore, OS X does NOT require you to hold down the mouse button when selecting menu items. Menus work by clicking, or dragging, just like in Windows.

            I would guess that any suspicions/rumors about Macs causing carpal-tunnel dates back to the 1980s when Macs were the only mainstream computer to use a mouse, and Microsoft users were running MS-DOS.

            I can't deny your personal experience, but in my experience, I switched from Windows to a Mac three months ago and noticed no change in wrist fatigue.
            pointzerotwo1
          • I blame training and (my) stupidity for some of this...

            I had a visit from a Mac (real) expert last week, and he showed me many things I had been taught wrong. Of course, he uses a "pad" and "pointy-thing" (Kate & Leopold) to do his work and seldom uses a mouse.

            What really gripes me is that in two decades, I could not find an Apple user who could/would teach me this. They hobbled around showing me other "neat stuff", but his guy made some things resoundly stupid, ... the way I thought they had to be done.

            I owned an Osborne OS-1 when the Mac first came out, so I'm no stranger to them, but, ... come on? I'm an old dog, but I can still think.

            I still suffer some wrist fatigue with the iMac's I own, but they are very old. When I get back to work (we get 3-4 months off), I'm going to mess around with the Macs there until I can see if this can be overcome.

            As to the matter of Doctors' statements. Those have been around since 1998 and on for us. The only doctors who don't say what I have quoted are either Mac users, or simply unfamiliar(???) with computer related problems. Can you believe it these days???

            I still have problems with Mac products, but I'm still trying to get along with them - especially since I cannot consider Vi$ta a wise choice in the near future.

            BTW, folder actions are cool!!!
            Media-Ted9
          • Try a different type of mouse

            I use a Thumb Ball mouse from Logitech on both Apple and M$ machines. These aren't for everybody and not great for art or some games but for regular everyday stuff they work great!
            snixon1
          • Been there, tried them...

            ... had a keyboard with a tracker system where a sensor pad would be on a laptop.

            Also have a larger, older model with a ball the size of a small potato(e?). I just can't get the reverse action of rolling the ball while holding a switch down, versus holding and moving the mouse (dragon drop - don't you love that term???).

            I'll repeat what I said in the other latest post: I somehow never discovered that I could move the mouse without the tension. I play with parameters at home and at work all the time, but can't figure out how I missed that one over the decades!

            It's probably a conspiracy against me - when I complained, the Mac $hill$ $neaked into my office and changed everything, ... just to make me feel stupid.

            Well, ... it worked! I do!!!
            Media-Ted9
  • Let's be realistic...

    [i]Since there are no numbers available from MS at this time as to the number of users ACTUALLY impacted...[/i]

    MS [b]does[/b] know how many were affected, as activation transactions are recorded. All they have to do is pull up the number of transactions that failed during the outage(How do you think they catch piracy operations?). They won't release a number though, as any quantification would be an embarrassment to their activation scheme.
    Tony Agudo
    • That seems a bit naive...

      ... History and experience with M$ has taught me that they don't give a rip about "numbers"; their only concern is "presence" and "power", and intimidation. Believe that they care about numbers if you want, but if they did, WGA would not be the raging bull it has been all this year. The only way to stop the bull is to butcher it. Everything it red to them. They even live in the mound of Red.

      As they say in real estate: "Location, location, location". They want ubiquity, and they have it. Why else would they consider every "activation" as coming from a "Pirate" unless they view all their users as enemies? What's the point of nearly giving away their product to China, worry of loss of profits??? Why not cut the cost worldwide and then the "Pirates" would have no competition with "high prices" to drive the business in their direction???

      Cool idea, though; what if China causes all those activation problems from their millions of WGA attacks? Is M$ going to relax the "standard" for them??? Now there, brothers and sisters, is a can of worms...
      Media-Ted9
  • It's necessary? Why? (nt)

    .
    Henrik Moller
  • The more the merrier

    WGA meltdown is GOOD! The more the better. The sooner more people will learn what an unscrupulous corporation Microsoft is.
    Ole Man
    • Unscrupulous?

      You probably would also call a homeowner who shoots a thief breaking into their house at night unscrupulous. After all, the thief was only stealing, he didn't deserve to be shot, right?

      MS has every right to defend their products from stealing. And you have every right to not purchase their products.

      That should be the end of the discussion.
      rkuhn0401729