Redbus boss begs customers to stay

Redbus customers who suffered hours of downtime last Tuesday should stick with the data centre company, whose upgrade is now "almost complete", Mike Tobin, the company's chief executive, told ZDNet UK on Monday.

Other hosters are facing similar problems but hushing them up, alleged Mike Tobin.

Users remain to be convinced, and Redbus is likely to receive claims for compensation for damage and downtime from users whose kit was destroyed by a power spike during the problem, and some of which took up to 27 hours to repair. There is evidence of the trouble on the Web site of service provider Andrews & Arnold.

"A power surge caused widespread damage to client equipment," said Redbus customer Mike Tims of mtn-i, who is organising a Redbus user group. "There is no surge protection built into cabinet sockets or the floor supply."

Tobin welcomed the Redbus user group: "If they are going to the trouble of forming a user group, it suggests they don't want to move. We value that loyalty and want to respond to it."

In the next few days, Tobin will receive an inventory of damage, and a list of demands from the group: "We are not a protest campaign, just a user group who want to ensure that our servers receive uninterrupted power at the correct voltage," said Tims.

Redbus' London One location in Harbour Exchange -- where the failure occurred -- is a "first-generation" site built during the "land grabbing" phase of Internet hosting, explained Tobin. "We were the last kid on the block" he said. "There were a lot of things we learnt in that phase."

London One has had three failures in the last year (the first two were in June and November), while its two "second-generation" sites have had none, said Tobin: "Two years ago, things began getting strained. London One is at the edge of capacity. We stopped putting new customers in London One and set up a programme to upgrade it."

Redbus has already spent nearly £2m on the facility, he said but, ironically, it appears to be the work process itself that has caused the problems. "London One is one of the most connected places on earth, with 70 ISPs," he said. "No one's upgraded a live building with that much connectivity and that many customers."

Some customers are not convinced: "Apart from the power surge which got through to the data floor, the UPS behaved exactly as it should and consequently their £2m upgrades to the UPS systems wouldn't have saved the day," said one user. "This could happen again even after customers are migrated to the new UPS systems!"

"We, along with others I'm sure, are looking to move," he continued. "In the mean time we have to take the absolutely ludicrous step of putting our own rack-mount UPS in because we've lost faith in Redbus completely (and our customers are demanding to see some action which Redbus certainly aren't providing)."

"Don't move out, stay with this centre," pleaded Tobin. "All the infrastructure is in place, and over the next six months or so we will migrate people over to the new infrastructure in orderly shutdowns." He expects to have all the London One customers on the new infrastructure by summer.

Customers of other hosters may be facing similar problems, and have worse to come if their providers don't upgrade, Tobin alleged: "In the same week we had an outage, two of our competition also had outages," he said, claiming that these competitors (who he would not name) made customers sign non-disclosure agreements (NDAs) to keep a lid on the occurrences.

It's amazing that our competitors who have older infrastructure than ours are not doing something about it," he said. "One day, the others will fall over."