Virtual machines cause Ninefold headache
Australian cloud computing provider Ninefold is currently having issues with its virtual machines that have brought the operations of some customers to a grinding halt.
The trouble started last Friday morning when Ninefold, via its support blog, told its customers that they might experience issues provisioning virtual machines. The company advised that it was looking into a fix and reported a resolution 12 hours later.
"Virtual machines can now start and stop. However, this may take up to three hours to complete. The root cause is being investigated," Ninefold told its customers.
The company followed that posting with another 14 service updates over the weekend, reporting ongoing issues with virtual machines while apologising for the problems.
Ninefold identified 42 instances experiencing issues across two server hosts at 11:30am AEST on Saturday, before reporting at 3:00pm that the number had dropped to 23.
Other reported problems included issues with the Xen hypervisor and a corrupt Ubuntu template, which was affecting the creation of new virtual machines.
The issues continued today, with the company's latest update coming at 2:30pm.
"While users can stop and start instances, the cloud management platform is not recognising some of these starting and is attempting to restart them, causing errors. We have been working with Citrix (Xen) throughout the weekend and this issue has been further escalated with them," the company said most recently.
Ninefold's issues took down countless sites over the weekend, including popular Australian-based Apple forum, MacTalk.
Ninefold has been working with its customers via its support centre and Twitter over the weekend to tackle the issues.
One customer even praised Ninefold's efforts despite the ongoing drama.
"Big ups to the @Ninefold crew. I know for a fact they've been working their butts [off] all day," said one user.
Ninefold managing director Peter James told ZDNet Australia that it was too early to speculate on the cause of the outage, but that it would be considering compensating its affected customers once it has rectified the issue.
"Our focus is obviously on ... the immediate needs of our customers. When we put out our post-incident report, we'll consider [compensation]. But at the moment, our focus is always on the good of the customer," James said today, adding that customers needed to keep the issue in perspective.
"It's important to keep it in context with the size of the cloud we've built and continue to build," he said.
James added that the company had reduced the number of affected instances to "a handful".