If weather forecasts play out the way they are supposed to this week, San Francisco Bay Area data center operators will get a chance to show their customers why they are paying the big bucks.
Data center uptime, or keeping servers running regardless of rough conditions, is of course the core part of any data center provider’s value, and disaster preparedness is something data center operators spend a good chunk of their time on throughout the year.
In the U.S. the biggest test for resiliency in recent years – a test not every data center passed – was Hurricane Sandy, which made landfall on the East Coast in late October 2012. Sandy caused widespread and prolonged power outages, and flooding that followed damaged critical infrastructure.
The torrential rain and hurricane-strength winds meteorologists are saying will besiege the Bay Area Thursday may cause power outages, according to utilities PG&E and Silicon Valley Power. The former serves most of the region and the latter serves Santa Clara – the heart of the Silicon Valley and home to a dense cluster of data centers.
While data center providers do a lot of preparation and drills to maintain data center uptime during outages as part of their regular routine throughout the year, they usually take a few extra precautions if they know that a storm is on its way.
Staff at a Telx data center located within a Digital Realty Trust facility at 200 Paul Avenue in San Francisco have contacted all contractors that serve the facility, from generator fuel suppliers to UPS and generator vendors, to make sure they have resources on standby, Paul Sidore, director of west region operations at Telx, said.
The staff also checked the facility’s roof to make sure there weren’t any loose objects there or objects that would cause water build-up. “Our engineering staff will be there, ready to go 24 by seven,” Sidore said.
All maintenance activities scheduled for Thursday have been postponed, he said.
Telx has taken similar precautions to ensure data center uptime in Santa Clara, where it leases space from Vantage Data Centers.
Chris Yetman, senior vice president of operations at Vantage, which has a massive campus in Santa Clara, has put together a list of 10 steps every data center operator should take to maintain data center uptime and staff safety when they know a particularly bad storm is coming.
All Vendors on Standby
Digital Realty staff have also checked their buildings’ roofs and confirmed availability of fuel with the company’s suppliers, David Schirmacher, senior vice president of portfolio operations at Digital Realty, said in an email.
“In the event of a power outage, Digital Realty’s fuel supplier program is one of the largest in the industry; we have first-response status on par with the U.S. Federal Emergency Management Agency and the Department of Defense,” Schirmacher said. “So, if the need arises, we have ready access to diesel fuel to power the generators in our data centers.”
The data center landlord also notified all building systems vendors that they may be required to respond on short notice. Vendors contacted include suppliers of generators, UPS systems, chillers, fuel delivery systems, and switchgear.
The company also put on hold all non-essential maintenance activities.
Equinix Tests Gensets, May Book Nearby Hotels for Staff
Operations personnel at Equinix have tested generators and verified fuel levels, David Morgan, the company’s vice president of IBX operations, said. The company also made special arrangements to make sure staff is available on site to address any unforeseen problems.
If the situation becomes extreme, Equinix has cots for staff to sleep in at the sites, and will secure hotel rooms for staff nearby if necessary.
CoreSite Confident in Facilities Staff Competence
Billie Haggard, senior vice president of data centers at CoreSite, another major Silicon Valley data center provider, said the company was monitoring the situation.
“We are closely monitoring weather forecasts in preparation for the large amount of rain that is predicted to fall in the Silicon Valley area over the next few days,” he said in an emailed statement. “CoreSite facilities are constructed to withstand heavy amounts of rain, and we have highly-skilled facilities staff on-site that have been trained to deal with weather-related situations.”