The Daily WTF has an item today titled The Big Red Button recounting one company’s experience with Take Your Child to Work Day.
Imagine how exciting a modern data center like the one at Robert’s organization would be for a child. Bright lights, multiple backup UPS systems, redundant AC units, and a very powerful generator out back to ensure there can never be an outage. Several giant systems housing multi-terabyte storage arrays, four huge IBM P695’s, and a few hundred other servers. Super high tech security to restrict physical access to the datacenter and a round-the-clock staff of fourteen administrators to lord over it all. I’m sure quite a few of you were getting excited just thinking about all that. Really, could there be a more awesome place to take the kiddies to see what millions of dollars of computer equipment looks like? … They loved it, and there were never any real problems. Well, until that one year when one little tyke couldn’t resist the temptation of The Big Red Button.
Most of you can imagine what the red button does. There are a lot of stories involving mishaps with Emergency Power Off (EPO) switches that have circulated in the data center world over the years. Back in 2004 Data Center Journal had an article about Emergency Power off switches, which noted that “the stories that EPO buttons evoke are likened to fish or golf tales. They contain some truth, but are mostly embellishment.” I can’t speak for the Daily WTF article, but it’s likely the number of mishaps with shutdown switches is probably somewhat smaller than the number of stories about them, particularly with the availability of molly guards. But these stories pique our interest, and are a reminder that Murphy lurks in every data center, waiting for opportunities to make his presence known.