VMware blames stray code for 'time bomb' hiccup

VMware blames stray code for 'time bomb' hiccup

Summary: VMware boss Paul Maritz is blaming "a piece of code" mistakenly left in the final release of ESX for the time-bomb hiccup that crippled virtual machines around the world.In a message posted online, Maritz said the glitch caused a license expiration hiccup that caused virtual machines to be powered off, suspended fail or disrupted migration.

SHARE:
7

VMware blames stray code for ‘time bomb’ hiccupVMware boss Paul Maritz is blaming "a piece of code" mistakenly left in the final release of ESX for the time-bomb hiccup that crippled virtual machines around the world.

In a message posted online, Maritz said the glitch caused a license expiration hiccup that caused virtual machines to be powered off, suspended fail or disrupted migration.

[ SEE: VMware bug causes worldwide disruption ]

"The issue was caused by a piece of code that was mistakenly left enabled for the final release of Update 2.  This piece of code was left over from the pre-release versions of Update 2 and was designed to ensure that customers are running on the supported generally available version of [ESX 3.5 and ESXi 3.5] Update 2," Maritz said.

  • In remedying the situation, we've already released an express patch for those customers that have installed/upgraded to ESX or ESXi 3.5 Update 2.  Within the next 24 hours, we also expect to issue a full replacement for Update 2, which should be used by customers who want to perform fresh installs of ESX or ESXi.

[ SEE: Techmeme discussion ]

Maritz said VMware failed in two areas:

  • Not disabling the code in the final release of Update 2; and
  • Not catching it in our quality assurance process.

"We are doing everything in our power to make sure this doesn't happen again," he said.

* Image source: cote's Flickr photostream (Creative Commons 2.0)

Topics: VMware, Virtualization

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

7 comments
Log in or register to join the discussion
  • Well..

    I wonder how much money this cost them? I was almost about to buy hosting using that, but decided against it. Thank God, I hate legal battles.
    ZachE84
  • RE: VMware blames stray code for 'time bomb' hiccup

    Serious mistake! Fire your entire QA team.
    dan.dumaraos
    • That is not an intelligent reaction

      Unless the QA employees willfully sabotaged the product, there is no reason to fire them. It is just a bellicose and unjust reaction. Are you telling me you have never made a serious mistake?

      If an employee makes a big mistake once or twice they are more likely to understand likely problems in the future and work to prevent them. Provide them additional training and tools to reduce the chance of error (like another poster suggested build switches).

      But if employees are paranoid that one mistake will cost their job, they will behave needlessly conservative and not at top performance.
      colinnwn
  • RE: VMware blames stray code for 'time bomb' hiccup

    That really represents poor execution and sloppiness of the whole development and QA teams. Either get better at managing tasks to turn this stuff on or off or have some sort of build switches to automate turning this on and off. Maybe a better design is needed?
    richz470
  • RE: VMware blames stray code for 'time bomb' hiccup

    I guess it's difficult to write unit tests for features such as "expires 90 days from now" unless your test framework lets you alter the system clock. Ironically, VMs make this more feaasible ;-)
    anionic
  • To test for 'time bomb' hiccups...

    Try having a time lab setup in the QA that runs every version on fictitious future time. Might be a good lesson learned for all of us.

    -RB
    Red_Beard
  • Time bombs should explode gracefully... not like this.

    Time bombs should explode gracefully... not like this. Some warning messages first... then several days later it should stop new VMs to boot... then several days later should shutdown the running VMs.

    Just shutting down without previous warning is just... mmmh... bad design.

    Regards,

    MV
    MV_z