CFD Thermal Modeling: Who’s Using It, and How?

One of the cool technologies in use in some data centers is thermal modeling using Computational Fluid Dynamics (CFD),  a tool for analyzing the effectiveness of cooling within the racks and aisles. The 3D visualizations of a data center are visually striking, but can also save data center operators large sums of money by identifying areas where cold air is not reaching equipment or is mixing with hot air. 

Since cooling can represent 40 percent of energy usage in some data centers, the savings from a CFD analysis can be significant. But so can the up-front cost, which is a challenge in technology adoption.

Customers of CFD modeling fall into three primary categories, according to Sherman Ikemoto, General Manager, North America for Future Facilities,  a UK-based company that specializes in CFD tools:

• Hardware manufacturers such as Dell, IBM, Verari Systems and Cisco, who use Future Facilities’ 6Sigma software to understand the thermal profile of their equipment.
• Building services firms such as EYP Mission Critical Facilities (now part of HP) and BrunsPak, who use CFD modeling to perfect air flow in their data center designs before starting construction.
• Data center operators who use 6Sigma and other CFD to trouble-shoot existing facilities to track airflow changes as more racks and cabinets are added to a data center.

Among companies operating data centers, use of CFD is concentrated in banks and brokerages. “We are mainly strong in the financial sector,” said Ikemoto. “They seem to have the top mission-critical facilities, and they’re constantly upgrading their facilities. They see their data center as a necessity, not as overhead.”  

Intel: Servers Do Fine With Outside Air

Do servers really need a cool, sterile environment to be reliable? New research from Intel suggests that in favorable climates, servers may perform well with almost no management of the environment, creating huge savings in power and cooling with negligible equipment failure.

Intel’s findings are detailed in a new white paper reviewing a proof-of-concept using outside air to cool servers in the data center - a technique known as air-side economization. Intel conducted a 10-month test to evaluate the impact of using only outside air to cool a high-density data center, even as temperatures ranged between 64 and 92 degrees and the servers were covered with dust.

Intel’s result: “We observed no consistent increase in server failure rates as a result of the greater variation in temperature and humidity, and the decrease in air quality,” Intel’s Don Atwood and John Miner write in their white paper. “This suggests that existing assumptions about the need to closely regulate these factors bear further scrutiny.”

Intel set up a proof-of-concept using 900 production servers in a 1,000 square foot trailer in New Mexico, which it divided into two equal sections using low-cost direct-expansion (DX) air conditioning equipment. Recirculated air was used to cool servers in one half of the facility, while the other used air-side economization, expelling all hot waste air outside the data center, and drawing in exterior air to cool the servers. It ran the experiment over a 10-month period, from October 2007 to August 2008.

The temperature of the outside air ranged between 64 and 92 degrees, and Intel made no attempt to control humidity, and applied only minimal filtering for particulates, using “a standard household air filter that removed only large particles from the incoming air but permitted fine dust to pass through.” As a result, humidity in the data center ranged from 4 percent to more than 90 percent, and the servers became covered with a fine layer of dust.

A Third of Data Centers in the Dark on Power

Thirty six percent of data center managers say they don’t know whether their electric bills increased between 2007 and 2008, suggesting that a large chunk of the industry continues to operate their facilities without effective communication aboutpower costs. That’s one of the findings from a survey of 600 facilities managers by Data Center Decisions, which is summarized at Tech Target.

The survey shows strong adoption of energy effiency strategies, including server virtualization, improving air conditioning efficiency, and powering down idle servers. The numbers suggest that many data center operators are implementing  technologies to lower their power bill, yet may not know how much they are saving.

Green Grid, ASHRAE to Collaborate

The Green Grid and ASHRAE will work together on publications that provide guidance on data center design, a development that could improve the effectiveness of data center cooling. The two groups said today that they will share technical information on energy efficiency.

ASHRAE develops best practices for 50,000 professionals in HVAC (heating, ventilation and air conditioning), and has issued recommendations for data center efficiency through its Technical Committee 9.9. The Green Grid is an industry consortium of data center operators and vendors.

‘Roll Your Own’ Thermal Monitoring

Blade servers and high-density racks continue to create “hot spots” inside data centers. Detecting these hot spots before they cause servers to overheat is a critical challenge. There are a growing number of vendor offerings that can provide data center managers with sophisticated thermal mapping and monitoring of their facilities. But in recent weeks we’ve seen several providers develop their own monitoring systems. Here’s a look at two innovative approaches to hot spot detection:

• Austin colocation provider Core NAP has built a system of low voltage thermal sensors tied together over Cat5 cable, which is summarized at IT Knowledge Exchange: “The monitors report back to a database that can map data center temperatures in real-time. (Core NAP) plans to be able to put multiple monitors in cabinets, under floors, and in the cable runs above of the racks. The sensors from Maxim IC report to USB readers plugged into Linux hosts. The hosts log data to a local web server, and Core NAP plans to combine that info with Visio maps of the data center.” Jeremy Porter, the Senior Internet Data Center Architect at Core NAP, says this approach will save money compared to off-the-shelf solutions.

IBM Cool Blue Shines in Vendor ‘Chill Off’

IBM’s Cool Blue Rear Door Heat eXchanger liquid cooling unit earned bragging rights in a head-to-head vendor “chill off” that tested the energy efficiency of vendor cooling products in the same data center environment. The unusual competition between leading cooling technologies from IBM, APC, Emerson Network Power and Rittal Corp. was sponsored by the Silicon Valley Leadership Group (SVLG), which announced the results yesterday during the group’s Data Center Energy Summit in Santa Clara, Calif.

Cool Blue, developed by IBM and licensed by Vette Corp., is a rear door heat exchanger that doesn’t use fans, which gave it an energy advantage over competing products in the analysis, according to Dean Nelson, Senior Director of Datacenter Design Services at Sun Microsystems, who presented the findings. There was also keen interest in the results for APC and Emerson Network Power’s Liebert unit, which wound up virtually deadlocked, with APC products faring slightly better at low power loads while Liebert’s gear was more efficient as loads neared 10 kilowatts per rack, the upper limit for the testbed.

Rittal’s LCP rack-based liquid cooling product trailed the pack. The full results and testing criteria for the chill-off are available on the Accenture web site (PDF) along with all of the 11 case studies presented during the SVLG energy summit.

The unusual contest was conducted in Sun Microsystems’ data center in Santa Clara, with officials from Lawrence Berkeley National Labs overseeing the testing, and Modius providing the metrics with its OpenData Data Center Infrastructure Manager software. More than 40 million individual measurements were taken during the study.

1,500 Watts A Square Foot? A Look at TSCIF

Switch Communications says it is successfully cooling a section of its Las Vegas data center running at nearly 1,500 watts per square foot using air cooling. How are they accomplishing this?

The key to Switch’s high-density cooling is a design known as Thermal Separate Compartment in Facility (TSCIF), according to company co-founder Rob Roy. The ingredients in this approach include high-capacity AC units placed outside the data center area, and a tightly integrated hot aisle containment system for the racks. Here’s an overview:

• The cabinets are set on a slab, with no raised floor.
• Chilled air is delivered into the cold aisle near the ceiling rather than through the floor, and enters the cabinets through the front.
• Each cabinet fits into a slot in the TSCIF unit, which encapsulates the rear and sides of each cabinet, while the open front extends beyond the enclosure.
• The hot aisle containment system delivers waste heat back into the ceiling plenum, where it can be returned to the chiller.

Some photos of the TSCIF system can be seen here, and more images and diagrams are available on the Switch web site. A number of data center providers forego a raised floor for overhead cooling, most notably Equinix (EQIX). Heat containment systems are also becoming more widely used.

Switch says the combination of those techniques, along with custom cooling equipment, enables it to handle unusually high power and heat loads. Roy says the data center cold aisle is maintained at 68 degrees, while the temperature in the hot aisle reaches well above 100 degrees, creating a heat differential of nearly 40 degrees.

The Vegas SuperNAP: A Data Center Revolution?

Switch Communications’ SuperNAP, a 400,000 square foot data center under construction in Las Vegas, is a conversation starter. The facility’s operators say the $350 million facility will be the most advanced data center yet, supporting power loads exceeding 1,500 watts per square foot using only air cooling.

Rob Roy, co-founder of Switch Communications, says his company is the best-kept secret in the data center industry. After operating for eight years in stealth mode, serving a client base of military government and military and government customers and large Internet companies, the SuperNAP represents a coming out party for Switch and the data center technologies it has developed.

After years of media silence, Roy has begun discussing Switch’s operations and ambitions for the SuperNAP. The first mention of the SuperNAP leaked out May 5 on the blog for Silverback Migration Solutions, a Switch customer. This past weekend Roy was profiled in The Register, which also received a tour of one of Switch Communications’ five existing Las Vegas data centers. There’s also now a company website, providing information about Switch’s operations and a video preview of the SuperNAP.

We’ve spoken recently with Roy, who was enthusiastic about the data center technologies developed by Switch, and dismissive of those in the industry who might view his claims as improbable. “This is an industry of naysayers,” Roy said.

Roy says Switch operates a room in one of its data centers with cabinets for a name-brand customer running at 1,462 watts per square foot. Roy predicts that the SuperNAP will also be able to support customer power loads of 1,500 watts per square foot.