7×24: Generators Are Key to Improving Reliability
November 15th, 2011 By: Rich Miller
When it comes to reliability, diesel generators are far and away the most important pieces of equipment in a data center, and regulatory mandates on fuel may be creating new problems that could raise generator failure rates.
Those were the key points made by Steve Fairfax, President of MTechnology, in a provocative keynote presentation Tuesday at the 7×24 Exchange Fall Conference in Phoenix. Fairfax, whose firm does “science risk” consulting work for both vendors and end users, said in-depth analyses of failure rates in data center components and systems yields counter-intuitive results, especially when it comes to maintenance.
“Generators are the most critical systems in the data center,” said Fairfax, whose studies of failure data found generators played a role in between 45 and 65 percent of outages in data centers with an N+1 configuration (with one spare backup generator). ”Reducing generator failures has more than 10 times the impact of reducing other component rates. This is where you should be focusing your attention – on generators. That’s what will take you down.”
Fairfax identified three key threats to generator reliability – fuel quality problems due to old fuel mixing with newer fuel, quality issues with new Ultra-Low Sulphur Diesel and biodiesel fuels, and wear and tear from efforts to start cold generators as quickly as possible.
Fuel Tanks and the “Diesel Solera”
Fairfax said the leading problem with generators is not the failure to start, but the failure to run properly once the generator has started. A key factor in the “failure to run” scenario is fuel quality.
Fairfax highlighted a phenomenon he calls the diesel “solera,” a term for the process for aging wine by mixing small amounts of older vintages with newer wine. While the solera process can help improve wine, it can introduce reliability challenges when it occurs in a tank of diesel fuel – which happens when older fuel remains in the bottom of a tank when it is refilled.
“Every year we take some of the diesel fuel out and add fresh fuel,” said Fairfax. “When was the last time you emptied that tank and cleaned it out?”
Ultra-Low Sulphur Diesel and BioDiesel
Another factor is the regulatory requirement to use Ultra Low-Sulphur Diesel Fuel (ULSD). While ULSD improves the emissions profile of generators, Fairfax said data center operators should pay close attention to fuel quality and tank conditions. Ultra-Low Sulphur Diesel is less stable than older distillate diesel fuels, he said, with a maximum storage time of 6 months. “Stabilizers can extend that, but then you have an interesting chemstry experiment going on in your diesel tank,” said Fairfax, who says this could result in a higher incidence of leaks and accelerated wear on seals.
Fairfax offered three recommendations on managing this challenge:
- Empty and inspect fuel tanks whenever possible.
- Change your generator testing policies. “Test them as you will run them,” said Fairfax, who said tests should run for 24 to 72 hours to simulate an extended utility outage, which will draw down diesel supplies in ways not seen in shorter periods.
- Sample your diesel fuel on a regular basis to track fuel quality.
Biodiesel, which is mandated in several states (including Massachusetts, Washington and New Jersey), poses additional challenges. “Biodiesel has a completely different chemistry” than older diesel fuels, said Fairfax, who said biodiesel can dissolves sediments that could clog filters, and has even worse stability than ULSD.
10-Second Start Times
Fairfax noted the NFPA guide to Emergency Power Supply Systems calls for generators to be able to start in 10 seconds for life safety applications. While not all data centers are required to adhere to this, many follow the NFPA guidelines by default. Farfax said Mtechnology’s research found no basis for the 10-second guidance, which he said places high stress on the generators that can shorten lifespan and impact reliability. The 10-second target requires cold equipment to start with a wide-open throttle, creating the highest possible thermal and mechanical stress.
“You get a huge benefit by reducing stresses. One of the best things you could do to improve the reliability of generators is to increase the start time to 30 seconds.”
Using a diesel rotary UPS (DRUPS) with enough kinetic stored energy can help reduce the number of times the engine must start, and can allow the engine to start unloaded. This reduces wear on the engine and typically makes starting in under 10 seconds a non-issue.
I think UPSs are more important than generators myself. Of the data centers I have been hosted in the number of times the generators actually got used (outside of testing) has been very very small, compared to continual use of the UPSs conditioning the power providing rock solid stable power feeds to all of the equipment.
generators are of course really important I wouldn’t go to a data center without them but if I was forced to choose – UPS or generator I’d choose UPS. At least with a UPS I can get enough batteries to allow for a graceful shutdown of the systems before the batteries run out.
By the same token I would never host equipment in a facility that is using a flywheel UPS – it may be green, but I love me my 15 minutes of runtime on batteries, not 15 seconds on flywheel.
The initial statement of “diesel generators are far and away the most important pieces of equipment in a data center,” supports the idea behind Preventive Maintenance Services. Regular equipment testing and a well-planned comprehensive preventive maintenance plan should protect your critical facility from costly repairs later on. It’s especially important if your generator set is used frequently or is subject to extreme operating conditions, as with data centers. This site is very informative and helpful:
KevinPosted November 17th, 2011
I find the author makes some good points. There are other fuel maintenance options like a permanently installed fuel polishing system, or contracting this service. Depending on the climate I am not sure if completely emptying the fuel tanks is realistic considering the logistics involved. I am a proponent of running the generator sets, but air permitting and the new EPA Tier 4i emissions standards must be taken into consideration when running generators for readiness testing. Depending on the environmental conditions and the number of run hours per year I am not sure the 10 second start is all that stressful on the equipment. Large diesel engines are used in many other much more demanding applications aside from power generation, and compression engines like to work hard.
[…] than a power outage to interrupt critical systems than affect uptime and necessary services. The standard configuration for any data centre is for them to use normal mains power from the National Grid but, in the event of an outage, to […]
[…] Generators are Key to Improving Reliability “Fairfax identified three key threats to generator reliability – fuel quality problems due to old fuel mixing with newer fuel, quality issues with new Ultra-Low Sulphur Diesel and biodiesel fuels, and wear and tear from efforts to start cold generators as quickly as possible.” […]