The U.S. Energy Act of 2020 called for the Department of Energy to update a 2016 Lawrence Berkeley National Laboratory (LBNL) study entitled "United States Data Center Energy Usage Report." At Data Center World 2024, event organizers invited the report’s authors to provide an update on progress as well as reveal any initial findings. Once published, this report should provide an interesting comparison to the 2016 study.
Arman Shehabi, Energy/Environmental Policy Staff Scientist at LBNL presented the assumptions being used to compile the latest report based on the rise of specialized hardware, artificial intelligence, and edge computing, as well as rapidly growing data demand in an increasingly interconnected world.
He highlighted the gains made by data centers over the last twenty years compared to the first few years of the millennium. Between 2000 and 2005, data centers doubled their electricity use in the US. The bulk of this consumption came from volume servers and data center cooling/power.
“Most projections at that time expected data center electricity usage to continue to expand rapidly,” said Shehabi. “We believed we would eventually run out of power.”
Fortunately, consumption flattened out. Between 2010 and 2018, power use in data centers rose by only 6% globally – despite a massive rise in compute and storage.
“Compute instances jumped by 550%, but we became far more efficient,” said Shehabi.
Modeling of data centers back in 2020 predicted that further work on energy efficiency could potentially accommodate a doubling of computational demand. However, this failed to predict the AI frenzy and the roll out of advanced GPUs. Researchers are now taking these factors into account as they rush to complete a major update to the report this year.
A New Study for a Rapidly Changing World
Shehabi works closely with Sarah Smith, Energy and Environmental Policy Research Scientist at LBNL. She has been compiling data and adding more factors into the analysis, including, crypto, carbon consumption, water usage, and other parameters. Smith is also wrapping her wits around thermal design power (TDP) and how rated power and maximum power might vary for different server types.
“We had been assuming that the maximum operational power of AI servers equaled TDP, but we are also considering that our models might be more accurate if we calculate maximum power at 80% of TDP,” said Smith at Data Center World.
In addition, researchers are trying to estimate average server utilization rates over an entire year – taking downtime into account. The current assumption is 70% utilization, but the uncertainty factor ranges from 30% to 80%. More work needs to be done to define this correctly for model accuracy.
Arman Shehabi (left) and Sarah Smith (right) at Data Center World 2024.
“Average server utilization rates are different based on the use case and the type of data center” said Shehabi. “We are trying to narrow our range of uncertainty.”
AI makes modeling and prediction even harder. But regardless of a wide zone of uncertainty, everyone agrees that electricity consumption is going to surge between 2024 and 2030. AI power consumption will dwarf conventional server consumption.
“Storage and networking electricity consumption are also growing but nowhere near CPU and GPU rates,” said Shehabi.
Cooling System Assumptions
The 2024 report takes into account the impact of liquid and other forms of cooling. This is yet another area of uncertainty for researchers.
“AI is driving the use of liquid cooling,” said Shebabi. “But water shortage and sustainability concerns are moving large data centers away from chiller water systems toward direct expansion systems.”
He requested assistance from industry associations and owners to plot exactly where all the data centers are in the US, particularly those of hyperscalers and cypto miners. Further, estimates are being made of power usage effectiveness (PUE) and water usage effectiveness (WUE) for small, midsize, hyperscaler and liquid cooled AI data centers.
Despite all the areas still to be calculated and compiled accurately, preliminary data center electricity estimates for the US are as expected: A steep rise in electricity usage across the boards.
“We don’t yet know how much AI will increase electricity demand, but we do know it will increase quickly,” said Shehabi.
Data Gaps Remain
As noted above, there remain many data gaps holding up the analysis. Other areas undergoing study include how much electricity data centers use from the grid, and how much from other sources of onsite power.
Still, excitement is already building about the findings revealed in the final report. The advent of AI means planners can see how it impacts the grid, how the grid responds and how quickly efficiency and technological advances can keep pace with growth. This will provide researchers and officials with insight into how the current push to transition to EVs and to bring about electrification of buildings and other industries will impact the grid.
“In the past, utilities have often overestimated capacity needs,” said Shehabi. “We want to see if that still holds with AI.”
Some believe that there is enough power in the US, either currently available or under development, to cover any increase due to AI. The problem is that power is not where it needs to be.
“We foresee electricity shortfalls as more of a local problem rather than a national problem,” said Smith.
She pointed out an opportunity for data centers in demand response. Many data centers have generators and other forms of backup power that are rarely used. Some regions are willing to pay data centers a fee to have their backup generation sources on standby. If the grid needs more power, it notifies the data center to switch to backup generation. The data center is paid all year for a service it may only provide for a few hours here and there.
Streamlining AI
What the report may not be able to take into account is the impact of innovation on AI electricity usage. Software is being streamlined to work better with GPUs. More integrated AI infrastructure and interconnects are being created that facilitate AI without using up so much power.
Michael Azoff, chief analyst at Omdia, noted that enterprises are more likely to gravitate to smaller, focused models than they are to develop ChatGPT-like behemoths.
“Whereas ChatGPT has 1.8 trillion data points, smaller models of perhaps 2.5 billion data points are emerging that can provide enterprises with good results,” said Azoff. “One business successfully built such a model using CPUs, not GPUs which greatly reduced processing requirements.”
