Austin, Texas-based Green Revolution Cooling submerges servers into horizontally placed tubs filled with dielectric fluid. (Photo: Green Revolution Cooling)

Austin, Texas-based Green Revolution Cooling submerges servers into horizontally placed tubs filled with dielectric fluid. (Photo: Green Revolution Cooling)

Is Direct Liquid Cooling Making a Comeback?

1 comment

First developed by IBM in the 1960s to cool mainframes, the use of direct liquid cooling in data centers was going strong until the 80s, when complementary metal oxide semiconductors (CMOS) were invented. These chips could crunch through a lot more numbers per watt, and liquid cooling took a back seat to lower-cost cooling systems that used air for heat transfer.

Today, however, the idea of bringing liquid coolant directly to the heat source in the data center is enjoying somewhat of a renaissance, according to a recent report by 451 Research, titled Liquid-Cooled IT: A Flood or Disruption for the Data Center? While far from mainstream, direct liquid cooling has become a business-case foundation for several new companies, and the amount of data centers with cooling requirements close to those of high performance computing is growing.

A UK company called Iceotope, which floods an insulated server chassis with dielectric fluid, was founded in 2009, received several industry awards and raised $10 million in funding earlier this year. Another company, Denmark’s Asetek, which brings coolant directly to the processor via a small pipe, had a successful IPO on Oslo Stock Exchange in 2013. There is also a number of U.S.-based companies in the space, including LiquidCool Solutions and Green Revolution Cooling. Overall, 451 counted between 15 and 20 suppliers that have products that qualify as direct liquid cooling. Most of the smaller ones make from $5 million to $10 million in annual sales.

One of the biggest suppliers of dielectric fluid for liquid cooling of electronics is 3M. Its fluid is called Novec.

One of the biggest suppliers of dielectric fluid for liquid cooling of electronics is 3M. Its fluid is called Novec.

The big IT vendors have not lost interest in the technology either. IBM continues doing R&D in the space, and so do Fujitsu, HP, Dell and Intel. While the majority of these efforts are focused on the HPC market, there are signs of increasing use of “HPC-like systems” by enterprises. Driven by Big Data and other demanding applications, this trend could potentially open a door for direct liquid cooling vendors to help companies stand up high-performance systems in their existing traditional enterprise data centers.

Not the most popular kid in school

While the companies’ approaches to direct liquid cooling vary, the concept of making cooling more efficient by using a more efficient heat-exchange medium and by coupling cooling more closely with the heat source is the same, and so are barriers to adoption. Customers worry about repairs and maintenance of the systems, lack of standards and the general risk of introducing any disruptive technology.

The report cites a survey conducted by the Uptime Institute, a 451 Group subsidiary, saying that of all cooling technologies considered “advanced,” adoption of liquid-cooled IT was the lowest: 11 percent of the whole. The category includes things like cold-aisle and hot-aisle containment, higher inlet-air temperatures, variable frequency drives, air-side economization and water-side economization, among others.

Here’s a detailed look at Uptime’s results:

Advanced cooling adoption Uptime

Direct liquid cooling has a number of market trends working in its favor, however. One of them is the emergence of the web-scale data center operator category. Companies like Facebook and Google have “legitimized a less conservative approach to the design and operation of facilities and paved the way for use of technologies such as [direct liquid cooling] by other operators,” Andrew Donoghue, European research manager at 451 and the report’s author, wrote.

Rack densities not going up as expected

What this technology does not have going for it is the industry’s overall power-density trend. Rack densities are not going up as quickly as they were expected to a few years ago. Median rack density in enterprise and colocation data centers is lower than 5 kW per rack, the densest racks reaching 8 kW, according to the Uptime Institute. Such densities are well within air-based cooling systems’ capabilities, making the business case for direct liquid cooling not as compelling.

Inside an Asekek-cooled server. (Image: Asetek)

Inside an Asekek-cooled server. (Image: Asetek)

Other barriers to adoption include fear of bringing liquid into an environment with lots of electronics and high-voltage power. There are also more and more data centers that to a great extent rely on outside air for cooling and run at higher ambient temperatures than normal. This is both good and bad for liquid cooling. On one hand, it rules out use of liquid cooling for low-density environments, which are much cheaper to cool using outside air. On the other hand, if operator of a low-density data center that is using outside air wants to install some high-density racks, they can do so using direct liquid cooling without affecting the rest of the environment.

There are other concerns, such as growing interest in low-power servers, powered by ARM or Intel’s Atom chips, and introduction of a single point of failure in the form of cooling-liquid supply to a rack, but one of the biggest barriers is all the investment companies have already made in air-cooled data centers. “Operators do not want to risk moving to an unknown technology despite its efficiencies, and suppliers are reluctant to jeopardize sales of existing air-based equipment,” the report’s author wrote.

Iceotope sells both AMD- and Intel-based modules filled with 3M's Novec fluid. (Image: Iceotope)

Iceotope sells both AMD- and Intel-based modules filled with 3M’s Novec fluid. (Image: Iceotope)

Physics will not be enough

As Donoghue put it, “Even air-cooling apologists will admit that if the clock could be turned back liquid cooling is a more efficient way to cool data centers. Compelling physics alone, however, is not enough for most data center operators.”

Success of the technology will depend on continued backing from the IT vendors the industry trusts – the Dells, HPs and IBMs of the world – and whether they, web-scale data center operators and HPC-facilities operators, will demonstrate business benefits of direct liquid cooling compelling enough for the rest of the industry.

About the Author

San Francisco-based business and technology journalist. Editor in chief at Data Center Knowledge, covering the global data center industry.

Add Your Comments

  • (will not be published)

One Comment

  1. If we look at the Uptime graph presented we see that 7 of the 9 options are all optimizing what is in place or existing. Behavioral changes vs wholesale technology changes. Use and optimize what we have vs. throw the baby out with the bathwater and start out new. I think that has to do as much if not more with the data center building itself and the incumbent systems than it does with a shift in what cooling technologies are most efficient. I would use Lasik surgery as a comparative example. Using lasers shot into your eyes to make you see better? No f-ing way. For many that is still the sentiment. However for those that do their due diligence and are able to look at actual costs and upside the choice is educated and clear as their vision when they're done, however they have to be willing to understand the technology, the risks and the results before they jump. Some data center operators are seeing liquid cooling clearly, others are saying no f-ing way. It all comes down to how soon you want the benefit if the benefit is much bigger than the risk. Not everyone wants to ditch their Coke bottle glasses, and not everyone has vision issues. Same for liquid cooling. For most of the business, good enough is just that, and for those that have challenges they need to look at different solutions, even ones that involve shooting lasers in your eyes.