Computer Outages Hobble Services in Virginia
August 27th, 2010 By: Rich Miller
Many critical services in the state of Virginia were crippled Thursday by computer failures in a state data center in Chesterfield, state officials said. More than 220 servers were offline, leaving at least 24 state agencies without full IT support, according to local media.
The outages disrupted the state’s ability to process child support payments and aid to needy families, or accept new claims for unemployment benefits or applications for driver’s licenses, according to the Richmond Times-Dispatch.
“We’re disappointed to have a failure, an outage of this magnitude,” Samuel Nixon Jr., head of the Virginia Information Technologies Agency, told the paper. “No matter what you do, it’s going to happen on occasion.”
While stuff happens in data centers, this week’s outage is bound to prompt fresh scrutiny of the state’s approach to its IT infrastructure. Virginia made news last fall when state agencies reported rolling outages after the state apparently neglected to include network redundancy as a requirement in a 10-year, multi-billion dollar outsourcing deal won by Northrop Grumman. The contract was restructured earlier this year.
This week’s failure appears to be linked to a hardware failure in a storage area network. Nixon said 228 of 3,600 servers were affected when technicians for a storage vendor were checking for faulty equipment. Nixon said he believes state computer data “are largely intact.”
The affected sites included the Virginia Information Technologies Agency itself. “The VITA website is temporarily experiencing technical difficulties,” a message read on Friday morning. “Restoration activities are ongoing. More information will be shared as it becomes available.”
[...] up today is the piece about the state of Virginia having a major data center outage. Here’s yet another catastrophe that’s the result of poorly planned and executed [...]
This issue highlights the importance of having strict, enforceable Service Level Agreements (SLAs) in place with any company which is providing a collocation or an outsourcing service. Although this outage is regrettable, I’m sure that Northrop Grumman isn’t going to be paying the cost of downtime for the state. I’m sure that the State of Virginia didn’t calculate the cost of their downtime when determining if they should outsource this service or keep it in-house. I think they might think twice about this in the future.
Steven JacobsPosted August 31st, 2010
Typical Virginia Bureaucracy and inefficiency. You should see what goes on at Virginia Tech on a routine basis while the ignorant masses spend millions on hokie paraphernalia.