According to a story in the Wall Street Journal, Americans have little to worry about from the new NSA datacenter, for one simple reason; it doesn’t work.
In an exercise in how not to rush the completion of a major datacenter project, the new facility is plagued with as yet unsolved electrical problems, which have resulted in the destruction of hundreds of thousands of dollars’ worth of equipment.
According to an article in Forbes, an insider reported that the facility was laid out with equipment density that was beyond the capabilities of the wiring plant, resulting in the creation of "kill zones" that resulted in electrical arc fires that would destroy the equipment in a specific area.
Documents reviewed by the Wall street Journal showed that this had happened at least 10 times since the facility was commissioned, most recently the last week in September.
It is being reported that the contractors responsible for the design and construction of the datacenter were unable to pinpoint the cause of the problem so the government sent in a US Army Corps of Engineers Tiger Team to save the day. Unfortunately, the problem apparently remains unsolved.
The Journal article also covers a report that was issued on the problem and states that it blames the decision to fast track the construction of the datacenter and bypass regular quality control in design and construction. Full operation of the facility is expected to be delayed at least a year.
I find it hard to believe that issues of equipment density and power weren’t on the top of someone’s list of concerns. I don’t recall ever talking to a datacenter infrastructure person or people building new facilities who couldn’t tell me, down to the watt, the amount of power that they could deliver to each rack in the facility and how, exactly, that power was provided.
Delivering power to the IT load equipment is one of the fundamental design and construction issues when you build a datacenter, and to have electrical faults on this scale is unconscionable, not only in a government project, where they suffer from the "other people's money" issue, but in any datacenter design.