Weathering the storm: What public utilities can learn from cloud computing

David Gewirtz examines the startling similarities between public utilities and cloud computing providers in the context of Hurricane Matthew. Public utilities have become complacent and are subject to disruption from Internet-age companies.

matthew2016-10-010225z.png
Image: earthdata.nasa.gov

We think of cloud services as a creation of the modern digital world, but one of the first cloud services was installed almost 140 years ago, on Pearl Street in Manhattan, just south of Fulton Street.

On September 4, 1882, the Pearl Street Station began providing light for 400 lamps and 82 customers. Like many cloud services, Pearl Street Station grew. Within two years, it was providing power for 500 customers, powering more than 10,000 lamps.

Rather than each individual customer building power-generating infrastructure, they all relied on this one centralized service. This service was the Edison Illuminating Company, which would eventually become Con Ed (Consolidated Edison), the $12B public utility that today provides power to most of New York City and Westchester County.

Shared characteristics

Where I live, in Central Florida, we rely on another regulated public utility, Florida Power and Light (FPL). Both of these utilities share a lot of the characteristics we're used to in IT cloud services.

We've obviously devoted a lot of virtual ink to cloud computing, so I'm not going to rehash all the elements here. But it's important to realize that both cloud computing providers and public utility providers are the keepers of the physical infrastructure. In the case of cloud computing, that's servers, storage, and network. In the case of public utility providers, that's power generation, power storage, and power distribution.

On one hand, that's great for consumers of these centralized resources. If you want to start your own online application business, you no longer need to build out a physical infrastructure. Back in the late 1990s (before we had real cloud access), I did that. Each time I reached a server's max capabilities, scaling involved a major investment to jump to the next level. But with cloud, you just scale smoothly, with only incremental expense.

Likewise, with power, I don't have to build and maintain my own onsite generator, and figure out a way to manage and safeguard fuel deliveries. If I use a bit more, or a bit less, power each month, it's simply reflected in my bill.

These services generally work reliably and consistently. I can check my Gmail, for example, without worrying about servers and infrastructure. I can brew a pot of coffee without worrying about whether or not the generator has been topped off with fuel.

We, as a society, have come to rely on cloud and public utility services to such an extent that they actually define our civilization. When these services fail, our lifestyle stutters. For example, when access to Facebook or Gmail goes down, we suddenly feel disconnected from our friends and colleagues. Sometimes, we're unable to complete work on time, or stay in touch for critical communications.

When the power goes out, everything comes to a halt. There's no air conditioning, no lights, no food preservation. Nothing.

Blasted back to the Stone Age

In most cases, failures are brief. They last a few hours, at most. But this weekend, Hurricane Matthew blasted Central Florida back to the Stone Age. It wasn't pretty.

Those of us who live on the southeast coast of the United States knew Matthew (which was a Category 4 storm, based on the Saffir-Simpson Hurricane Wind Scale) was coming for almost a week. It was due to hit the Space Coast (where I live) on Friday morning.

My wife and I spent Tuesday, Wednesday, and Thursday preparing the house. We disassembled the workshop, and turned it back into a reinforced garage to make room for one car. We moved our second car to the garage at my parents' old house. We battened down all the hurricane shutters. We filled tubs and gallon jugs with water. We did our best to prepare.

By midnight on Thursday, we'd mostly finished our preparations. Our nerves were on edge. To distract ourselves as the storm approached, we decided to binge-watch the latest season of Game of Thrones. Power dropped out for a few minutes during episode one, and another few minutes during episode two. We kept glancing at an app on my phone to watch the track of the hurricane's eye.

Read this

Video: IT heroes of Hurricane Sandy

ZDNet interviewed a panel of IT heroes who kept their organizations running During Hurricane Sandy with successful disaster recovery plans. Watch the full 40-minute panel discussion as these IT leaders share the lessons they learned.

At 4:43am, about 20 minutes into episode three of Game of Thrones, the power went off for the final time. The storm had arrived in full force.

We were terrified, because all the track maps essentially showed the worst of the storm, including 140 mph winds, making landfall pretty much on top of us. Fortunately, that didn't happen. The storm mostly missed us, with the eye about 20 miles offshore. Even so, we were hit by winds in excess of 70-90 miles per hour. By about 7am, the worst of the storm had passed.

It was still unsafe to go outside, or even open up the internal windows behind the storm shutters. So we had no air flow in the house until about 3pm Friday. We tried to sleep. When the wind finally died down, and the limited LTE I still had on my iPhone showed that the storm had tracked northward, we removed the first of our fortifications (the shutters over our front door). We stepped outside into the fresh air.

We were very fortunate. Our house sustained no damage. A neighbor's fence had blown down. A home down the street had some roof damage. No one was hurt.

But we had no power. On Friday, with the winds from the tail end of the storm still surging, at least we had a breeze. We thanked goodness that the supernatural heat of the summer was behind us. We opened our windows, and we lit some candles. The cross-breeze helped a little.

Verizon's LTE was barely functional, so getting information was nearly impossible. We had no idea when power would be restored. My iPhone was down to 50 percent. Normally, I'm very happy with the battery on the iPhone 6s Plus. But with no idea when I'd be able to recharge, I started to get nervous. We had some D-cell batteries for the fans, but, again, we didn't know how long we'd be without power.

All we could think about was how long we'd have to go without power, and how the hell we'd make it for however long it would be. When we finally regained some level of internet connectivity on the phone, the FPL status update site merely said that they were working hard to restore power. No time estimate was possible.

On Saturday afternoon, power did come back on... for 20 seconds. The lights came on, and I almost teared up with relief. It was short-lived relief. All of a sudden there was a boom. The lights went back off. A nearby transformer had exploded, probably from debris across the connectors. It would be another day before we got power back again.

All told, we were without power for three days. I'm ashamed to say that I didn't take my forceable removal from civilization well. I was miserable, uncomfortable, desperate, and a little crazy. I couldn't sleep. I didn't eat much. There was nothing to do, nothing to work on, and -- as a person who usually enjoys the illusion that I'm very much in control of my own destiny -- nothing I could do to improve our situation.

We simply had to wait

This is the problem with centralized services like cloud services and public utilities. The convenience, scalability, cost-savings, reduced maintenance, and general reliability come at the cost of self-determination. If those services fail, they take you down with them.

This is why, with cloud computing services, we often talk about redundancy, and keeping local backups. We can also employ a similar strategy with public utilities, although the implementation is much more complex, much more costly, and much less reliable.

I do not own a generator. I regretted that a lot over this very long weekend. However, while there are relatively inexpensive generators available, one that can power A/C for a longer duration is very large and incredibly expensive. Worse, there's the question of how to safely store the fuel during the storm, and whether the actual generator will survive the pounding of the storm.

A similar concern exists for solar power. It would be great to put up solar cells and not have to pay the monthly power bill at all. But in a hurricane-prone area, solar cells are likely to be torn off the roof before they can provide the emergency power they're intended for. It's kind of a Catch-22.

The real answer is that the public utilities, the power companies, need to implement more robust power distribution mechanisms.

Okay, let me stop here for a moment. Before I criticize FPL and its ilk, I want to give a huge shout-out to all the very hardworking repair teams who restored our power over the weekend. I spoke to some of the guys working the lines, and they told me they'd been flown in from out of state before the storm. They worked their way up the state, restoring power county-by-county, city-by-city. They had had almost no sleep for days, while having to work with live power lines in 90-degree heat. They're champions and heroes.

Demand a better solution

That said, this is not how it should work. All our power lines (and broadband lines, for that matter) are exposed and hanging. This is unconscionable. The power services know that we're prone to hurricanes, yet they allow these lines to remain open and exposed.

Power Pole
Image: David Gewirtz

Worse, they're often poorly maintained during non-emergency times. The picture you see to the right is the transformer behind my house. Notice all the overgrowth? If a branch crosses over the connectors, that transformer will either spark or explode. It's already exploded once. And yet, that's how FPL distributes power in an area prone to wind storms.

Can you imagine such irresponsibility among cloud computing providers? It's as if, knowing what they do about the prevalence of hackers and infiltrators, Google just didn't bother using firewalls, intrusion prevention, or even password security. It's as if Google's entire cybersecurity strategy was "eh, call us when you're hacked, and we'll fix it when we get to it."

No one would tolerate such a thing. But that's because Google has competition, which keeps it agile and competitive. We're stuck with our single power provider, FPL, who has no competition. As such, they can choose to prioritize repairs using a system that's essentially waiting to see what breaks, rather than building in any preventive infrastructure.

There is no way, yet, to prevent these terrible storms. But the damage due to the storm is often the result of a failure in infrastructure planning, maintenance, or investment, not due to acts of Mother Nature. Katrina was a bad storm, to be sure. But it was the failure to maintain the levees protecting New Orleans that was the cause of most of the damage.

Here in Brevard County, we have a little over 300,000 power customers. Friday night, more than 200,000 of them were without power. By Saturday night, 100,000 were still without power. And even today, after Friday, Saturday, Sunday, and now Monday, some of our friends are still waiting to have their power restored.

It's not that FPL didn't have repair escalation plans in place, or dedicated workers. They did. Those folks are fantastic. They emergency response was well executed. But infrastructure as poorly maintained as the transformer in my back yard does not show an ongoing dedication to emergency prevention and disaster mitigation.

We allow our public utilities to be monopolies because of the enormous investments required to deliver service to all customers. We regulate them because they're monopolies. But we don't do enough to demand that they harden and protect their infrastructure -- and that's because they don't have any competition.

If FPL had competition, the way Google has to compete against Microsoft and AWS, you can be sure we'd not only have spent this weekend in cool comfort, we'd probably spend a lot less each month on the services we do get.

Perhaps as companies like Tesla (and even Apple) develop more robust battery technology, we can replace generators and solar cells with highly efficient in-structure batteries. Then, maybe we'll be able to withstand five days without power from the grid, simply by tapping into our own private pool of battery power.

Or, perhaps, similar to the way the internet has disrupted other forms of infrastructure, we'll start to see new and innovative ways we can produce our own energy. Perhaps we will be able to replace the service we get from the grid or, at the very least, have an alternate source available for when storms like Matthew hit an entire region.

Final thoughts

Before I wrap this up, I want to say a few more things. To everyone who lived through Matthew this weekend, I hope you weathered the storm well. To you amazing folks I spoke to here in Brevard County who reported to work and did your jobs, even though you had no power or sleep at home, bravo and thank you. To our fellow citizens, neighbors, and residents who approached what could have been an extinction event with courage and class, you're awesome. To the power line workers who gave us back our civilization, all of us are incredibly grateful for your dedication.

Finally, to our two bottom-of-the-barrel presidential candidates who had a debate on Sunday night, I ask this: why didn't you at least acknowledge the pain, fear, and challenge the entire United States southeastern seaboard went through? Are you so caught up in your petty scandals that you can't bother with being presidential?

To Mr. Trump and Mrs. Clinton, the fact that you didn't even mention Hurricane Matthew or the millions of your fellow citizens who struggled to retain their sense of civilization, while you couldn't even remain civil, is both disturbing and disappointing.

This is America. Our citizens are heroes. Our leaders are zeroes.

We need a better plan from our public servants, who benefit greatly from our patronage, on how to move forward with hardened infrastructure into a future that is bound to be fraught with issues from "the clouds," whether those clouds are weather and climate related, or digital in nature.

You can follow my day-to-day project updates on social media. Be sure to follow me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.

Newsletters

You have been successfully signed up. To sign up for more newsletters or to manage your account, visit the Newsletter Subscription Center.
See All
See All