-
Water Main Break Floods Dallas Data Center
IT systems in Dallas County were offline for more than three days last week after a water main break flooded the basement of the Dallas County Records Building, which houses the UPS systems and other electrical equipment supporting the data center on the fifth floor of the building. The county does not have a backup data center, despite warnings that it faced the risk of service disruption without one.
The 90-year-old water main ruptured at about 10:30 p.m. last Monday, flooding the basement-level mechanical room with six feet of water, forcing the shutdown of the data center. Power was not restored to the facility until early Friday, as it took several days for the county to pump out the basement. The water was mixed with oil and mud, and needed to be pumped into taker trucks rather than the sewer system, delaying the recovery.
“The county’s criminal justice system nearly ground to a halt, as paper processing from another era led to lengthy delays,” the Dallas News reported. Several courts have gone paperless in recent months, and the district clerk’s office has been converting files to electronic form and destroying paper versions, the paper said.
The county had been warned in a 2008 consultant’s report that it needed to address its lack of a backup data center, but never followed through on the recommendation. The county IT staff is now developing a failover plan and expects to submit an updated plan in two weeks.
A portable generator and UPS were used to restore power to the facility early Friday, and the county IT staff began checking to ensure servers were properly restarted. The Texas county said it believes no critical data was lost.
About Rich Miller
Rich Miller is the founder and editor-in-chief of Data Center Knowledge, and has been reporting on the data center sector since 2000. He has tracked the growing impact of high-density computing on the power and cooling of data centers, and the resulting push for improved energy efficiency in these facilities.
10:30 p.m. Monday, How? Its only 5:50PM Monday here in the East coast, Its only 4:50 there now.
Water Main Break Floods Dallas Data Center | JetLib News
Posted June 7th, 2010[...] writes “IT systems in Dallas County were offline for three days last week after a water main break flooded the basement of the Dallas County Records Building, which houses the UPS systems and other electrical equipment [...]
Matt
Posted June 7th, 2010I know of at least five states in the US with Dallas counties. Does this story refer to the one in Texas?
uh
Posted June 7th, 2010“I know of at least five states in the US with Dallas counties. Does this story refer to the one in Texas?”
rtfa bro
Mel
Posted June 7th, 2010SimonTek – … offline for three days “last week”.
So, 10:30pm last Monday night. That’s how.
Matt – Yes, the one in Texas
Captain Obvious
Posted June 7th, 2010Yeah you might want to read all the way to the end there.
“The Texas county said it believes no critical data was lost.”
Yes, it’s the one in Texas.
Larry Jansch, CBCP
Posted June 8th, 2010Their servers survived a soaking in six feet of water, mud, and oil with no data loss? Where can I order fifty of these for our data center?
In a recession, business continuity is one of the first “extravagances” to be cut and one of the last to be restarted, despite insurance carrier’s and legislative requirements of continuous BC coverage.
Larry: The flood affected the power room. The servers were on an upper floor.
Tuesday data center tidbits:bailing out the data center basement, the great data center tax giveaway, and more! « The Server Room
Posted June 8th, 2010[...] } First up today is the piece about the flood in the Dallas County data center, caused by a ruptured water main. Not having backup servers is bad enough, but putting your backup power equipment below grade is [...]
TechYa Daily #6 « Andrew Horton
Posted June 9th, 2010[...] hacker who hacked into NASA to search for UFO secrets, Chinese boot camp for computer addiction, Water main in a data center breaks, flooding the building, backing up using an FTP service and more! — I backup using Interact [...]
One 90-year old water main, One data center, One criminal justice system « The Water Monitor
Posted June 13th, 2010[...] 13, 2010 Haggai, our CTO, stumbled upon the story of a 90-year old water main in Dallas (pre-PC era! imagine that) that burst and flooded a basement, [...]
RESOURCE LINKS:
Building A Cloud-Savvy Model for TCO and ROI
How Storage is Shaping The Cloud Data Center
Bringing Colo to the Customer: Modular Gets Local
Microsoft’s $1 Billion Data Center
June 7th, 2010