This article originally appeared on the Omdia blog.
According to Omdia’s Data Center Server Tracker, total revenue for 3Q23 decreased 7% year-over-year (YoY) and 1% quarter-over-quarter (QoQ) to $25.5 billion.
There was a significant slowdown in shipments of general-purpose servers with top OEMs feeling the heat of macroeconomic headwinds and changing investment priorities. The weak demand is primarily felt in the enterprise and communication service provider segments. Server useful life is at an all-time high with the average enterprise now using seven-year-old servers and delaying refresh.
Deployments of servers configured for AI are growing in line with Omdia’s forecast. NVIDIA’s 3Q23 data center revenue came in at $14.5 billionn (nearly four times growth year-over-year) and aligned with Omdia’s expectations. This indicates to us that, as expected, NVIDIA shipped nearly half a million H100 and A100 GPUs in 3Q23. We predict this to cross the half a million mark in 4Q23.
Within the cloud service provider market segment deployments of highly configured AI servers are offsetting delays in general-purpose server refresh. This is not the case in the enterprise and communication service provider market segment as server OEMs have struggled to get enough GPU allocation from NVIDIA. Server OEMs indicated a lead time of 36-52 weeks for servers configured with NVIDIA H100 GPUs.
With high-cost AI servers growing in penetration and general-purpose server refresh getting delayed until at least 2024, server shipments in 3Q23 decreased 26% YoY. We now expect server shipments of 11.4 million units for the full year of 2023, a decrease of 19% from 2022.
During the 2024 refresh cycle, our expectation is that significant server consolidation will occur with several installed servers being replaced with one new server. This is enabled by higher core-count CPUs and new server architectures with application optimization taking center stage.
This month we updated our long-range server forecast to reflect the 2023 momentum and continuing demand for highly configured servers. We now expect the server market to be worth $195.6 billion in 2027.
Within this forecast, the server processor and co-processor contribution is increasing in share driven by a move towards hyper heterogeneous computing, i.e., application-optimized server configurations with many co-processors. Examples include:
Servers running AI training and inference
- The most popular server configuration for large language model training is the NVIDIA DGX server, configured with 8 H100/A100 GPUs
- Amazon’s servers for AI inference are configured with 16 custom-built co-processors (called Inferentia 2)
Video transcoding servers with many custom-built co-processors
- Google's video transcoding server has 20 custom-built co-processors (called VCUs)
- Meta’s video processing server has 12 custom-built co-processors (called Meta Scalable Video Processor)
Omdia envisages this trend to only grow as demand for some applications matures to the scale that makes it cost-efficient to build an optimized custom processor. Media and AI are the early benefactors of hyper heterogeneous computing, but we expect other workloads like databases and web services to see a similar optimization push.
Visit Omdia’s Cloud and Data Center practice to read more insights and analysis covering market trends and industry forecasts.
