Level 3 London datacentre outage takes more than 50 businesses offline

A severe power failure in a Level 3 Communications central London datacentre has knocked at least 50 businesses offline and hit companies that use the location for connectivity and hosting.
The incident, which began at around 3:30am BST in Level 3's Braham Street London datacentre, was resolved by 10am. The problems affected all customers at the site, including colocation provider Adapt, which saw more than 50 customers affected by the outage. It also hit companies that use it for connectivity, such as ISP LCHost.
"A/C Power to the content delivery network equipment and customer colocation are down," Adapt wrote in a report circulated to customers around 8am on Tuesday.
Because the Braham Street facility is a major connectivity point for Level 3, companies that use services that plug into the transit provider were also severely affected.
ISP LCHost told customers via its Facebook page on Tuesday that "there is an outage for all ADSL customers at present due to the power failure in Level 3 Braham Street where our onward connection to BT's network passes through".

Though LCHost is not a Level 3 customer, its connectivity was being routed through Level 3 and went down with the datacentre. Level 3 operates one of the largest IP transit networks in North America and Europe. It is one of only six Tier 1 internet providers in the world and aims to be "the core of the networked world". 
"Something like this shouldn't happen," Phillip Baker, the technical director of LCHosts, told ZDNet. "We will be asking questions of our wholesale capacity supplier as to why they appear to be dependent on a single power feed."
The Braham Street facility has two diesel generators that are tested monthly and meant to kick in in the case of a power outage, according to Level3 (PDF). It also has four hybrid rotary uninterruptible power supply (UPS) systems to provide back-up power in the case of a generator outage. In addition, it has a variety of power distribution units.

It was the UPS system that had problems, according to Adapt's operations director Justin Lewis. This implies a generator failure.

"When I saw this I was very surprised — this is not a normal event by any means," Lewis told ZDNet UK. "You would not expect to have a total failure of this nature in a datacentre." 

"Earlier today one of Level 3's data centres at London's Braham Street  experienced a brief power outage," Level 3 said in a statement. "This has since been resolved by our team of on-site power technicians. We are currently working closely with customers to ascertain any effects to service provision."