Complexity and the greening of the datacenter
The majority of datacenters were built 15 years ago when power requirements and densities were much lower. Power consumption has risen from 2 kilowatts to 40 kilowatts over the past decade. Consumption per square foot tripled from from 40 watts to 100 watts from 2004 to 2005, and it is estimated that energy costs are approaching 30 to 40 percent of overall IT costs. The worldwide cost for powering datacenters is estimated at $7.2 billion.
Those statistics came from Dave Edwards, a research analyst with Morgan Stanley, who led a panel of experts at the AlwaysOn Stanford Summit O7 in a discussion of some of the complex issues involved in the greening of datacenters.
The panelists (below) included Steve Sams, vice president of global site and facilities; Mike Rigodanzo, senior vice president of technology services at HP; Stephen Trainor, vice president strategy, Level-3; and Subodh Bapat, vice president and distinguished engineer at Sun.
The problem is universal, but each datacenter and geography involves different issues."In Europe, companies are concerned just about getting energy. They are worried about the oil tap being turned off. In many cases, customers don't really know what they are paying for power," Sams said.
He added that many companies are being charged on an average per square foot across office buildings and datacenter facilities, which can be a good deal but doesn't address the carbon footprint and overall cost problem.
"You have to get to facts of cost structure, how efficient or inefficient the data centers are. For some clients, for every 10 kilowatts, you add 5 kilowatts to support it. Some are spending an extra 40 percent to support installations. It's a broad range so there is huge room for improvement in can understanding the facts and where problems really exist," Sams said.
"When you talk about 100 watt to 150 watt per square foot datacenters, they are not homogeneous. You have all kinds of technology with different heating and cooling requirements," he added. "We are abysmal at predicting what technology will do 10 to 15 years out, so designing for what can be predicted is a critical." For example, the datacenters built 15 years ago had no concept of blades and the higher densities deployed today.
Rigodanzo said the average datacenter cooling is overprovisioned by 2.5 times. Basic changes in layout of datacenter, routing of airflows and optimizing cooling with thermodynamics and computation models based on fluid dynamics can reduce the problem.
For datacenters with densities that require above 100 watts per square foot, more engineering is required to gain efficiencies. HP is applying sensor technology to identify what is going on in different zones and to target energy use where it needs to be, Rigodanzo said.
Virtualization and server consolidation are leading to lower hardware costs but to denser datacenters. "We have to keep in mind it is goodness to get more efficient utilization of servers, but it introduces other problems, such as managing demand for power in a one location," Rigodanzo said. Internally, HP is consolidating from 85 to six datacenters, increasing the density and energy requirements per location.
Level-3's Trainor added that his company's business includes about 7 million square feet of datacenter facilities in use, including colocation space for clients. "We have an environment that is subject to change, so it's more difficult to model with computational fluid dynamics, so you have to take care of basics."
Sun's Bapat talked about innovations inside the server box and enclosures, including more intelligence in the components such as smarter firmware to calibrate power draw to optimize utilization. He noted that server memory can burn a lot of watts, so turning off unused memory is useful.
Smarter power supplies can also improve power consumption. On average they run at only 50 to 70 percent efficiency. "We want to get to the 80 to 90 percent range and get insturmentation on power supplies to know how many watts they are drawing," Bapat said. "We haven't paid attention to large scale power systems that monitor and calibrate workloads. We can take advantage of dynamic migration capability of new virtualizaiton to rebalance hotspots in a datacenter. We will see a lot more innovation on software side in next several years."
The panelists also shared a few new technologies that could have an impact on reducing the cost of powering modern datacenters. Moving from electons to photons could have an impact, but that is on the far horizon. A move from AC to DC power could also reduce the power draw. Building ruggedized servers that can withstand high temperatures over 100 degrees Farhenheit is also a possibility.