Lightning strikes Amazon's European cloud

Lightning strikes Amazon's European cloud

Summary: The lightning strike damaged a power company's transformer, causing disruption to Amazon Web Services's European cloud, and may have affected Microsoft's BPOS as well

SHARE:
TOPICS: Cloud
5

Amazon Web Services's European cloud suffered an outage over the weekend after lightning struck a transformer at one of its power suppliers.

Read this

AWS disrupted by US east coast failure

Amazon Web Services' cloud has taken a hit from problems in its US East Coast region, downing multiple sites that depend upon the service.

Read more+

The outage, which Amazon Web Services (AWS) acknowledged on Sunday evening, affected its Dublin-based Elastic Compute Cloud (EC2) and Relational Database Service (RDS) cloud services, among others. The damage to the electricity infrastructure may have affected Microsoft's Business Productivity Online Services (BPOS) cloud as well, Microsoft said in a separate statement.

"A lightning strike hit a transformer from a utility provider to one of our availability zones in Dublin, sparking an explosion and fire," AWS wrote in an update to its status page at 11pm on Sunday. "The transient electric deviation caused by the explosion was large enough that it propagated to a portion of the phase control system that synchronises the backup generator plant, disabling some of [the generators]."

"Power sources must be phase synchronised before they can be brought online to load. Bringing these generators online required manual synchronisation," it added.

Amazon acquired 240,000 square feet of Dublin datacentre space in February to help it expand its AWS cloud infrastructure. The Dublin datacentres provide services to customers in Europe who want to access low-latency cloud services.

Service impaired

The AWS disruption affected a single "availability zone" within the European EU-WEST-1 region, the company said. An availability zone is a set of hardware that supports cloud services and is independent from other zones.

A lightning strike hit a transformer from a utility provider to one of our availability zones in Dublin, sparking an explosion and fire.

– AWS

As a consequence of the transformer problem, services were "impaired" from 7pm on Sunday, according to AWS. At the time of writing, the Amazon unit said it expected problems to continue for at least the next 24 hours.

The power shutdown took out a swathe of AWS's servers, which led to the outage in cloud services such as EC2, RDS and Elastic Block Storage (EBS). Because the cloud services are made up of complex pieces of software, the company needed to allocate further hardware to resurrect services when power was restored. This drew the recovery period out further.

"Restoring [EBS] volumes requires that we make an extra copy of all data, which has consumed most spare [hardware] capacity and slowed our recovery process," AWS wrote at 7am on Monday.

BPOS disruption

Microsoft also reported disruption in its Dublin-based BPOS cloud service on Sunday. The problems arose from "a widespread power outage in Dublin," the company said in a statement. It did not respond to request for comment as to whether the lightning-hit transformer was at the root of it.

The disruption in BPOS provision ran from around 7pm on Sunday through to 1:30am on Monday, according to Microsoft. "Throughout the incident, we updated our customers regularly on the issue via our normal communication channels," it said.


Get the latest technology news and analysis, blogs and reviews delivered directly to your inbox with ZDNet UK's newsletters.

Topic: Cloud

Jack Clark

About Jack Clark

Currently a reporter for ZDNet UK, I previously worked as a technology researcher and reporter for a London-based news agency.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

5 comments
Log in or register to join the discussion
  • I'm pretty sure this all started on Sunday evening (7th August), not Saturday... And it's still ongoing for some unfortunate souls...
    anonymous
  • Clive, thanks for that, we have updated the story. Yes, we're aware it's still ongoing, as per AWS's statement.
    Jack Clark
  • It's unfortunate that a Amazon's offering is having this many problems lately. It offers an unrivaled platform for computing that offers power and flexibility, but I think it's recent stability issues are overshadowing it's disruptive edge in the market. Not to say it won't do well, but these issues are hindering it's wide spread adoption.
    anonymous
  • have they said which availability zone was affected, EU-West-1a, EU-West-1b or EU-West-1c?
    jules345@...
  • Jules345,
    As far as I can tell they haven't publicly disclosed the availability zone, just the region. (http://status.aws.amazon.com/)
    Jack Clark