Editor's note: This article originally appeared in the Summer 2023 issue of The AFCOM Journal. AFCOM members are welcome to access the full issue here. Non-members can gain access to The AFCOM Journal (including back issues) by joining today.
The influence of AI is everywhere. ChatGPT has captured the popular imagination. The latest Mission Impossible movie even cast AI as the villain of the story. The Entity, as it is known, breached just about every computer network on the planet and threatens to subvert humanity – stay tuned for Part II to find out who wins.
Beyond the hype, there are some realities to face for data center managers. Many fear the loss of duties and responsibilities as AI facilitates an even higher degree of automation. There is no avoiding that. Indeed, some welcome it, as they have been required for many years to do a lot more with fewer and fewer people. AI, then, may spell trouble for some manually repetitive roles. But it may have arrived just in time to make life tenable in the data center.
AI Wariness
The Uptime Institute Global Data Center Survey 2023 revealed persistent staffing issues as well as a wary attitude toward the adoption of AI technologies.
“AI in data center facilities will be adopted cautiously,” said Douglas Donnellan, Senior Research Associate at Uptime Institute. “Operators are distrustful of its ability to make reliable operational decisions.”
That said, data center operators are under pressure to address staffing shortfalls. According to the Uptime Institute survey, staffing rivals energy efficiency as a top concern. 33% of data center personnel surveyed expressed “deep concern” about how to improve the energy performance of facilities equipment, with another 35% being somewhat concerned. Just behind came a lack of qualified staff: 30% were very concerned about it, 41% were somewhat concerned, and another 21% were slightly concerned. Next came improving the energy performance of IT, with 30% being very concerned and 35% somewhat concerned. These pressures very much open the door to AI-based solutions.
However, AI’s looming specter has hung over the data center industry for years. Nearly three-quarters of respondents to the 2023 Uptime survey believed that AI-based software tools would eventually reduce data center operations staffing levels. This attitude has been observed consistently for the past five years. Back in 2019, 29% of respondents said they believed that AI would reduce the need for human data center staff in the next five years. Nearly five years later, there is little evidence of this occurring, and that shows up in the survey results. Now only 25% expect AI to lower staffing needs in the near term. In 2019, 42% thought it would take more than five years for AI to impact staffing levels. That number has increased to 48% in the 2023 report.
“Perception of AI’s looming influence does not reflect reality,” said Donnellan.
Where AI Can and Will Steal Duties
Much like a series of innovative technologies that came before it, AI has the potential to address the skills shortage in data centers. By leveraging AI technologies, there is no doubt that data centers can improve operations and efficiency and do more with less.
“AI can automate repetitive and mundane tasks, such as server monitoring and resource allocation, thereby reducing the workload on human operators and allowing them to focus on more strategic and complex aspects of data center management,” said Steve Santamaria, CEO of Folio Photonics.
Miroslav Klivansky, Global Practice Leader of Analytics & AI at Pure Storage, also believes that AI can materially relieve the workforce stress in data center.
“From the challenges of remote work to the need for advanced skills that can impact the talent pool size, data centers are experiencing obstacles when it comes to talent acquisition and retention,” he said. “As a result, today’s data center operators are few in number - and are often faced with significantly larger workloads with much too little support from their teams.”
AI and automation, therefore, can be used to address these pain points by creating more efficient workflows and relieve the burdens facing data center operators. For example, data center operators can use AI to automate low-level tasks that enable them to free up more of their time to address strategic initiatives. By optimizing workloads for datacenter operators, AI can prove to be an effective tool for organizations to prevent burnout among employees.
Klivansky sees definite evidence that AI is unlocking new levels of efficiency across the data center. For application developers, for example, AI enables a DevOps co-pilot that assists in the dev process and simplifies workflows. Platform engineers, too, can reap the benefits of AI via predictive insights and recommendations to support accelerated time-to-deployment and an overall enhanced engineer experience.
“For those tasked with maintaining the integrity of the data center, embedded AIOps helps to monitor data stores and detect anomalies to ensure that data is stored properly and fed into AI models,” said Klivansky. “Built-in activities like these provide proactive insights on data workflows, automate remediation and empower those across functions with suggestions for improved outcomes.”
Such capabilities will result in some casualties. Think about how the invention of the automobile displaced so many jobs related to horse- and carriage-based transportation. With AI, the lost positions will include those who go from server to server filling out a maintenance checklist and taking readings from equipment. That is a field ripe for AI-driven automation. Data center infrastructure management (DCIM) systems are already on the market that harness basic AI capabilities to ensure the high availability of the data center, that they operate with energy efficiency, that they have properly utilized capacity, and to assist in the prediction of future needs within the constraints of the budget. By placing many sensors inside equipment and on individual components, data centers are given an enormous amount of data that can alleviate the manual maintenance workload.
Similarly, data wranglers that primarily move data between storage systems to manage capacity and even out the workload are in danger. A number of factors are driving this trend, according to Klivansky. These include the recent maturity of scale-out storage with huge namespaces and flash-native architectures that support a broad range of workloads on the same system, smarter automated tiering driven partly by AI and partly by the applications themselves, and more standardized open data formats that rarely require conversion. What should at-risk personnel do when they see the AI writing on the wall?
“Today's data wranglers need to transform themselves into full-fledged data engineers who can build automation to maintain data pipelines and a steady flow of clean data to applications,” said Klivansky.
Overall, AI will alleviate some very definite pain points in the data center. Cooling systems and server management platforms are following the lead of DCIM by incorporating AI. This is good news for overworked data center managers but bad news for those who have made a career out of walking around the data center tinkering with equipment and taking readings.
AI in a Financial Services Data Center
It stands to reason that the banking sector might be among the leaders in applying AI to the data center. The industry has deep pockets. The ability to process transactions faster and spot fraud quicker can add up to millions in a matter of minutes.
Take the case of JP Morgan Chase. Its electronic trading systems span the globe. They move billions around daily. Bottlenecks within its data centers concern fraud detection and credit card approvals. On the fraud side, think about how you interact with banks and credit card companies related to fraudulent transactions or potential anomalies. Typically, something is flagged or a customer calls in. These alerts tie up internal personnel resources in getting to the bottom of the situation. The same thing holds true on the authorization of credit card charges. Banking systems generally set thresholds for automatic approvals on charges. Above a certain amount, the transaction is flagged and queried instead of being approved. However, many of these transactions are valid. Only a few are fraudulent. But if the good ones are not approved fast, impatient users may switch to another card or move on to another provider. Large amounts of good business can be lost. That’s one of the reasons why JP Morgan Chase has already invested heavily in AI in the data center.
“With AI, we were able to approve hundreds of millions of credit card transactions that would otherwise have been declined,” said Greg Johnson, Executive Director or Global Electronic Trading Services at JP Morgan Chase. “AI made it possible to verify that they were valid transactions in a very short time.”
The company supports its AI infrastructure in the data center with a bare metal design that is 100% Linux based, as well as 100 Gb networks to provide the performance needed for applications that interact with the New York Stock Exchange. Data is stored locally to minimize latency on all-flash arrays that scale out to satisfy growing demand.
AI Opportunities
AI may lead to job losses and functions disappearing. But opportunity knocks for those who prepare for it. Uptime Institute’s Donnellan listed tasks such as dynamic optimization of power and cooling, anomaly detection, predictive maintenance and predictive analytics as some of the main use cases for AI in the data center to date. Each represents an area where existing data center personnel can add to their skills sets to a) head off the possibility of being laid off and b) to find a new role that is suddenly in demand. Those who can combine existing power and cooling know-how with AI monitoring or development skills won’t lack job openings. Similarly, those who know how to use AI to enhance data center security are likely to have long and successful careers. There are plenty of certification courses and other training avenues around that either directly cover AI or dovetail AI with areas such as cybersecurity, data center optimization and analytics.
“The properties of AI mean that this technology could be applied to almost every area of data center management and operations,” said Donnellan.
He pointed out some further findings from the recent Uptime Institute survey. Forecasting future data center capacity requirements came up as a serious concern among 27% of respondents, with a further 39% being somewhat concerned about it and 24% being slightly concerned. Clearly, it is an area rife with challenges in the data center.
Another area that came up in the survey was procuring the equipment needed to meet higher capacity demands. With the industry facing annual data growth rates of 30% or more, It is hardly surprising that 26% are very concerned about this function and 42% are somewhat concerned. Anyone who can marry storage, capacity planning, and AI skills can bring major change to data center operations.
“The integration of AI in data centers will create new job opportunities in areas like AI engineering, machine learning (ML), data science, and AI ethics, cultivating an expanded, diverse, and specialized workforce ready to embrace the benefits of AI in the data management industry,” said Santamaria.
Perhaps the Biggest Opportunity
According to a recent survey from Pure storage, IT buyers cited that their top investments planned for the next five years were AI and ML at 52%. But 90% of respondents said they felt pressured by leadership to adopt technology that their current infrastructure couldn’t support.
“AI is not magic: it requires significant planning and strategic implementation to be beneficial,” said Klivansky. “The consequences of adopting AI without taking the proper steps to modernize infrastructure can lead to significant challenges and burnout among data center operators.”
This is an area where existing data center smarts can really come to the fore in facilitating our AI futures. Data center personnel live and breathe infrastructure. They have been over the hurdles many times when it comes to figuring out how to implement new technologies, how to eliminate the many bottlenecks that invariably crop up. AI perhaps can be looked upon as just another technology that the data center needs to integrate – and that needs those with hard-won data center experience to implement.
AI requires so much juice that highly dense racks must be configured. It needs a lot more cooling and power to support it. The network interconnects and buses to other systems must be capable of operating at AI speed. Thus, AI may well be a far bigger opportunity for the data center than the potential job killer it is currently considered.
The average data center is estimated to be 12 years old, and many are more than 20 years of age. There are many data centers out there that need to be brought up to the velocity of AI to be able to compete in the modern world. That adds up to a whole lot of jobs. Perhaps the AI glass may be half full rather than half empty for the data center industry.
