Skype: Microsoft patches didn't spark our outage

Skype: Microsoft patches didn't spark our outage

Summary: Skype offered up some more details about its two-day outage last week and noted that Microsoft patches were not to blame.This point was fairly clear in its initial blog posting on the incident.

SHARE:

Skype offered up some more details about its two-day outage last week and noted that Microsoft patches were not to blame.

This point was fairly clear in its initial blog posting on the incident. Microsoft's Windows update was a catalyst not the reason for the outage.

Skype said Tuesday (see Techmeme for more):

We don’t blame anyone but ourselves. The Microsoft Update patches were merely a catalyst — a trigger — for a series of events that led to the disruption of Skype, not the root cause of it. And Microsoft has been very helpful and supportive throughout.

The high number of post-update reboots affected Skype’s network resources. This caused a flood of log-in requests, which, combined with the lack of peer-to-peer network resources at the time, prompted a chain reaction that had a critical impact. The self-healing mechanisms of the P2P network upon which Skype’s software runs have worked well in the past. Simply put, every single time Skype has needed to recover from reboots that naturally accompany a routine Windows Update, there hasn’t been a problem.

Unfortunately, this time, for the first time, Skype was unable to rise to the challenge and the reasons for this were exceptional.

So what was different this time around with the patches.

Skype said nothing was different.

In short – there was nothing different about this set of Microsoft patches. During a joint call soon after problems were detected, Skype and Microsoft engineers went through the list of patches that had been pushed out. We ruled each one out as a possible cause for Skype’s problems. We also walked through the standard Windows Update process to understand it better and to ensure that nothing in the process had changed from the past (and nothing had).

More importantly Skype said it has squashed the bug that led to the outage. As for the lessons learned they still apply for Skype.

Topics: Microsoft, Collaboration, Outage, Social Enterprise

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

23 comments
Log in or register to join the discussion
  • Skype must switch to Linux to avoid headaches

    windoze is a plague that damaged their business.
    Linux Geek
    • Linux Geek is right

      Everyone that lost service should team up for a class action suit against M$ for damages form the service outage.

      Only M$ could possibly have caused this, this could never happen on Linux. M$ shold have known this and urged Skype to move theirserver to Linux BEFORE this happened, so there would NEVER EVER be an outage for Skype cutomers.
      KTLA
      • Never happen in Linux

        Well looks like the article says that MS was not the problem, it was skype. Did it ever occur to anyone that there are certain apps, databases, etc. that run on their systems. I doubt they are using only MS products for everything? But to say this could never happen in Linux just makes your statement worthless. So what are the recent event at Ubuntu? People need to get their heads out of their a**. Windows has flaws, Linux has flaws, and so does Apple. First hand experience speaks alot louder than your opinions based on emotion.
        OhTheHumanity
      • are you guys really dumb

        some people are truely dumb aren't they?

        They said the problem was too many customers installed windows update therefore restarted.

        What they mean is too many customers went down in the same time and too many of them came back up in teh same time. This was not an issue of what kind of server they were using.

        This was the issue that their software was not stress tested for situation that alot of customers can disapear and alot of customers could logoni simantously.

        Principle of server application design remains the same no matter what freakign platform you develop it for.


        You guys truly are a bad reputation for linux fans.
        ericsami
        • Well...

          ...I guess you could claim that if no one were using Windows, all those millions of PCs wouldn't have been rebooting at the same time. So, in a sense it could be said to be Microsoft's fault. That is, of course, ignoring that it was really due to a flaw at Skype's end.

          However, the OP of this thread is a known Mike Cox wannabe (albeit unsuccessful), and I suspect the 2nd poster was being sarcastic (unfortunately with much the same success I have when trying to be funny). :)

          Carl Rapson
          rapson
        • Pot to Kettle: You're Black...

          Try reading the article again. It had nothing to do with CUSTOMERS rebooting their machines. It had EVERYTHING to do with Skype rebooting too many of their servers at once:

          [i]The high number of post-update reboots affected Skype?s network resources. This caused a flood of log-in requests, which, combined with the lack of peer-to-peer network resources at the time, prompted a chain reaction that had a critical impact. The self-healing mechanisms of the P2P network upon which Skype?s software runs have worked well in the past. Simply put, every single time Skype has needed to recover from reboots that naturally accompany a routine Windows Update, there hasn?t been a problem.[/i]

          In other words, when their servers rebooted and got up and running again, all of the log in requests from clients who were disconnected were like a massive DDOS attack.

          It sounds to me like Skype's doing rather well for themselves. Too many clients trying to log into the system at the same time. I should have this kind of headache...
          Wolfie2K3
    • Yeah, and sell to, ummm, no one.

      DUH!!!


      Wait for it,


      Shut up, fool!
      No_Ax_to_Grind
    • Shut up, fool. (nt)

      No, seriously.
      Hallowed are the Ori
    • Holy smokes...

      are you dumber than a rock or what?

      Are you suggesting that anyone that wants to use Skype must install linux to use the service?

      HAA, that's rich. You sure do provide your linux cause a great deal of credit with comments like those :)
      BFD
      • Rocks have filed an offical protest...

        But seriously.. He's not dumb. He's just on a personal jihad against anything Microsoft. He makes Osama Bin Loser sound like a sane person and that's NO easy feat.

        Actually... LG was suggesting that Skype fdisk all of their Windows servers and install Linux. Like as in Linux boxes NEVER have to be rebooted. EVER.

        It's still a daft idea. Never mind how many millions it would cost Skype to completely redo their current infrastructure and software just to run on Linux. With all the customers they've got now, they're rich enough...
        Wolfie2K3
        • UBUNTU isn't Linux?

          Gee, the guys at UBUNTU would be surprised to know that all their compromized servers weren't Linux. They didn't need to be rebooted either -- unless, of course you have a problem with their servers attacking other servers...

          (And YES, I do know that the vector of attack was the FTP service. However, the UBUNTU administrators CHOSE to run that particular FTP service and CHOSE to run it insecurely -- so much for the brilliance of Linux admins who SHOULD know better.)

          Hate to break it to you, but there is NO SUCH THING as a REAL server that will NEVER need to be rebooted at some point in time. And no, running a Linux server in Mom's basement that nobody else ever accesses doesn't count.
          Marty R. Milette
    • hahaha that's the dumbest thing I've heard

      A single OS issue didn't cause all the problems they encountered. Suggesting they switch to a different OS shows significant lack of understanding of computer systems as a collection of interdependent components of a much larger system.

      Stick to troubleshooting your home PC.
      GeiselS
  • "The Microsoft Update patches were merely a catalyst ? a trigger" . . .

    Saying "The Microsoft Update patches were merely a catalyst ? a trigger ? for a series of events that led to the disruption of Skype, not the root cause of it." is like . . .

    Saying after a homicide "The murderer pulling the trigger was merely a catalyst ? a trigger ? for a series of events that led to the child's death, not the root cause of it."

    The kid should have been wearing a bullet proof vest. It's his fault.

    .
    Basic Logic
  • Message has been deleted.

    ballmerrules@...
    • Shut yer spamhole. (nt)

      nt
      Hallowed are the Ori
  • high traffic

    "So what was different this time around with the patches."

    Higher than usual traffic, apparently.

    From the blog:
    "The high number of post-update reboots affected Skype?s network resources. This caused a flood of log-in requests, which, combined with the lack of peer-to-peer network resources at the time, prompted a chain reaction that had a critical impact."

    They have some sort of algorithm in their software designed to deal with the excess traffic caused by all of the reboots that happen when Windows updates. Apparently there was a bug in the algorithm, and that bug prevented their network from handling the large volume of traffic generated by the reboots.

    Sounds to me like it's mostly sheer volume - isn't Skype a growing business?
    CobraA1
    • EXACTLY! n/t

      ...
      Wolfie2K3
  • Growing business?

    Not any longer....
    justanitguy
    • I disagree - the setback is temporary

      I disagree. Sure, there's going to be a setback now, but it's going to be a temporary one. They'll be back on track before you know it.
      CobraA1
  • I don't believe Skype's excuse

    I don't buy the massive patch excuse. Even if every PC around the globe was scheduled to download updates and reboot at, say, 9:00 PM local time, the existence of time zones would distribute the number of simultaneous reboots -- 9 PM in the UK is 4 PM on the eastern coast of the US and 1 PM on the western coast. IMHO, the more likely event was that something major (and as yet undisclosed) happened at Skype's end, which suddenly triggered every system (that was already logged in) to re-authenticate.
    jimsj