Comments on: Storms KO NaviSite San Jose Data Center

By: Backfires, Misfires, and Duds… « Enterprise Strategy Group

Backfires, Misfires, and Duds… « Enterprise Strategy Group — Fri, 05 Feb 2010 16:05:41 +0000

[...] Misfire – Storms KO NaviSite San Jose Data Center: [...]

By: Backfires, Misfires, and Duds… « IT Artillery

Backfires, Misfires, and Duds… « IT Artillery — Fri, 05 Feb 2010 15:55:16 +0000

[...] Misfire – Storms KO NaviSite San Jose Data Center: [...]

By: Guy Hammer

Guy Hammer — Tue, 26 Jan 2010 19:56:22 +0000

Losing backup generators is pretty bad for a company in the “high-availability” hosting business — and it has a huge trickle down impact for clients like Prostores:
“The generators are now up & we are working to get all servers online. We anticipate having service fully restored w/in two to three hours.” (http://twitter.com/prostores)

Navisite has had this facility for several years, so they might be over-subscribing their power capacity. If not, I doubt they would have had this outage if they had been doing full load generator testing once a week — which is the standard for Tier-4 data-centers.

By: Alfredo

Alfredo — Wed, 20 Jan 2010 16:26:59 +0000

As a business, it’s important to have redudant data center site(s) that can fail over with little to no impact and to insure there are heavy financial penalities in the contract when a colo doesn’t meet it’s obligations. From a distance (speculation) there are a number of reasons why this load didn’t fail over to generator. Perhaps they didn’t perform a full comission of the building before going online, or perhaps they reduced their preventative maintenance cost. To Nates point, I don’t think anyone should fear the flywheel. I believe a quarterly maintenance that requires a full load transfer under a controlled environment (electricians, mechanical engineers, generator tech, facility staff, on site etc) and following a MOP (Method of Procedure) would have caught the problem and been repaired so when a real utlity power loss happens, you don’t drop power to your customers.

By: nate

nate — Wed, 20 Jan 2010 00:32:09 +0000

issues like this that sends shivers down my spines when people talk about using flywheel UPS systems and having a matter of seconds for backup generators to respond. Assuming on site personnel I’d want at least 15-20 minutes of backup power in the event a generator fails to start, gives some time to troubleshoot at least.

By: Tuesday data center tidbits. « The Server Room

Tuesday data center tidbits. « The Server Room — Tue, 19 Jan 2010 23:54:00 +0000

[...] } In the news today is the crash of the NaviSite data center in San Jose as the result of backup power failure. This is the best example I’ve seen why the move to [...]