A power surge crashed the state’ of Washington’s s primary data center in Olympia last month, according to local media reports. The Aug. 21 outage occurred as a new high-voltage power line was being added to expand the power capacity of the office campus. During the installation, an electrical switch in an underground vault short-circuited.
The equipment failure resulted in a power surge that caused a “hard shutdown” of the state’s data center, according to The Seattle Times. Joanne Todd, a spokeswoman for the Department of Information Services, said generators initially kicked on to maintain the system but that a second surge forced the generators to shut down. Todd said the generators are designed to shut down in that situation to avoid damage to equipment.
The incident occurred on a Sunday afternoon, and the data center was back online by early Monday. The downtime delayed the processing of unemployment claims, but checks went out as scheduled.
The Department of General Administration says the outage will cost the state $400,000 to $500,00 to replace the electrical switch, which was custom-made, and hire an engineering firm to analyze the incident and improve safety and redundancy.
“Equipment in the underground vault remains a life-safety issue and there may be additional changes needed in another vault,” the GA said in a fact sheet, according to The Olympian. “GA has locked the vault and will bar entry to anyone until the equipment is replaced. The replacement work is not expected for about 16 to 20 weeks since the switch must be custom made. The repair will require a weekend power outage.”
The incident occurred at the existing data center for the state, which is also building a new data center in Olympia.