Early on April 30, a fire broke out in one of the data center electrical rooms at Terremark’s NAP of the Capital Region in Culpeper, Va. The fire department was on site for hours, and the event was covered by local media. But the facility remained online throughout the entire event, according to Terremark, with no downtime for customers.
The incident happened at about 12:30 a.m. Friday in a basement electrical room in Data Center B, the second of the two operational facilities at the NAP of the Capital Region. ”We had a fault in the medium voltage room,” said Norm Laudermilch, Senior VP of Infrastructure for Terremark. A preliminary investigation points to a malfunction in a transformer, but the company has not made a final root cause determination.
“The UPS and generators responded as they were designed to, and the customers upstairs didn’t realize anything had happened,” said Laudermilch.
Terremark staff were able to isolate the room from the rest of the facility’s operations, and along with the local fire department, used halon extinguishers to contain and put out the fire. Laudermilch said it was “small fire, with a small duration.”
But he also noted the value of a good relationship with local emergency officials. When the local fire department arrives at a data center, they usually want to power down as large as area as possible to eliminate any potential safety risk for fire fighters. That can even mean using the Emergency Power Off (EPO) button to power down an entire data center.
“When we opened this facility, we invited the fire department and local emergency officials to come and tour the place and ask any questions they had,” said Laudermilch. “The fire department knows what risks exist, and in which areas. We were able to isolate the problem, and prove to them that there was no power to that room, and they didn’t make us shut down.”
Data Center B remains on generator power as Terremark repairs and replaces the equipment in the electrical room where the incident occurred. In the meantime, the generators are supported by more than 520,000 gallons of on-site diesel storage.