IT failures roundup: Emergency services around the world

Today's roundup focuses on public emergency services. Although these systems should never fail, here are several that did.

The impact of most IT failures is limited to inconvenience, delays, and higher costs. However, computer problems affecting police, fire, and medical response can directly cause loss of life and property.

Maryland 911. A boy almost drowned when the fire department didn't respond following a faulty Verizon system upgrade. From the Washington Post:

Peter Lucht, a Verizon spokesman, said the disturbances in Prince William "stemmed from an unusual combination of factors" and were "not something that we usually see." The company has worked closely with the county "to stabilize the system and ensure that [problems] won't be repeated," he said.

The county purchased its 911 system from Verizon in 2002, and it was installed in 2003, and fire officials said there were no problems until the May 28 upgrade. There have been no problems since July 12, they said.

Maine 911. Seven "system shutdowns" prevented callers from reaching emergency services. From the Kennebec Journal:

Between April and June, emergency communications systems at the Cumberland County regional communications center, the state dispatch center in Gray and the Penobscot County regional center had problems receiving 911 calls. The most severe problems were at Cumberland County's system in Windham where a series of seven system shutdowns in April and May left callers getting no answer, in one case for as much as an hour.

In response, FairPoint installed a switch allowing dispatch centers to immediately transfer calls to a backup center when there is a problem, and a designated a separate telephone line to alert staff when there is a problem. The equipment was installed in the six dispatching centers in the state that had the type of equipment affected by the malfunctions.

London ambulance services. Failed computers caused London ambulance workers to fall back on pencil and paper call tracking. From the Evening Standard:

Ambulance staff were forced to record emergency calls with pen and paper and find addresses using A to Z after their computerised system crashed early yesterday morning and a computer failure at the Royal Free Hampstead NHS Trust means patients face delays and records have been lost.

The Royal Free introduced the system in June to reduce paperwork but since then it has crashed, leaving those waiting for operations and blood tests badly affected. When the same system was introduced at Bart's and The London NHS Trust, cancer patients missed critical appointments because their records were lost.

A spokesman for the Royal Free said: "With change on this scale, it is inevitable that it will take time for staff to familiarise themselves fully with all the functions that they need to use but overall we are pleased with the progress being made.

"It is recognised that while staff are getting used to the system a small number of our patients may have to wait longer than expected in clinic."

Australian ambulance services. System failure forced Queensland, Australia emergency services to record and track calls using whiteboards. From IT Failures blog:

Thorough investigations have been conducted as to the cause of each, with the following outcomes:

• Two of the outages have been attributed directly to human error. The first was an SQL update which had unexpected results, the second was a server which was inadvertently shut down.
• The third outage has been traced back to a fault with the replication software ‘Replistor’, which resulted in the primary database server rebooting.

[No humorous picture today, because there's absolutely nothing funny about these failures.]