Blackrock: From Wall Street to Wenatchee
June 6th, 2012 By: Rich Miller
East Wenatchee is a long way from Wall Street. But the small town in central Washington state serves as the unlikely home for a data center for one of the major players in the financial industry. The financial services firm Blackrock operates a new data center at a mission critical campus operated by Sabey Corp.
Blackrock is the world’s largest asset management firm, overseeing more than $3.5 trillion in funds for its clients, which include pension funds and insurance companies. The company has also emerged as a leader in the development of Exchange Traded Funds (ETFs), investments that track indices of stocks or bonds. ETFs allow investors many of the advantages of mutual funds, but with a lower fee structure.
“We’ve experienced huge amounts of growth,” said Joshua Vallario, Backrock’s Director of Global Data Center Operations, who discussed the Washington state project at the recent DataCenterDynamics New York conference. “Managing the growth, from a data center perspective, has been no small task.”
Major Data Center Consolidation in 2010
The East Wenatchee data center is part of a broader IT consolidation begin in 2010 that saw Blackrock downsize from 32 global data centers to just 12 sites, with about 5.9 megawatts of IT capacity. “We migrated 3 megawatts (of IT load) around the globe without downtime,” said Vallario. “We’ve been in this program of ‘build to consolidate.’”
That includes the data center on the Sabey campus in East Wenatchee, which is about 2,750 miles from Wall Street but is one of the most affordable places in the world to operate a data center. Renewable hydro-electric power is available for as low as 2 cents per kilowatt hour, and the cool climate supports free cooling – the use of fresh air instead of chilled water to cool the servers, which drastically lowers the power bill.
That proved compelling to Blackrock, which opened its data center in January 2011 after a five-month construction process. Vallario said the facility operates with a Power Usage Effectiveness (PUE) of between 1.13 and 1.21, even at its current partial load.
Savings: $3.5 Million Per Year
Vallario said the East Wenatchee location, along with the energy efficient design, saves Blackrock about $3.5 million per year compared to the cost of running the same server capacity in the company’s second-most affordable site.
“We scratch our heads about why more financial firms don’t go there,” said Vallario, who worked closely on the initiative with Blackrock Global Datacenter Manager Barry Novick. “But not everyone can stand the latency that we can at Blackrock.” The latency between New York and East Wenatchee makes the location impractical for high-frequency trading, for example.
Sabey’s Intergate.Columbia, is a 438,000 square foot data center campus housing both single and multi-tenant data centers. Phase I was completed in December 2008 and is now fully leased. Sabey hopes to break ground on Phase II later this year.
One Pod Deployed, One for Expansion
Blackrock leased two of the six pods of data center space in its building, with its space physically demised to separate it from other tenants. Each pod supports 1.6 megawatts of IT capacity. One pod is completed, with the second to be added as the firm needs additional capacity.
The facility uses hot aisle containment in the server area to separate hot and cold air. For cooling,Blackrock uses indirect evaporative cooling units on the rooftop, which can pull warm air from the hot aisle if the weather gets too cold.
So what’s it like to have key data center capacity so far from Wall Street? Operating a remote site is made possible by Blackrock’s proprietary data center management software, Vallario said, which allows it to maintain the East Wenatchee facility with as few as six full-time staffers.
“We knew this would require creativity, since the data center team was in New York,” he said. “I can assure you that Blackrock is among the biggest control freaks, and we’re proud of it.”
Microsoft is getting their power for 1.9 cents per KW/h, so it’s only a matter of time until others gravitate to the area.
But… What happens when Mt. St. Helens blows again, or if environmental regulations require the dams on the Columbia river (the main power source) to increase their spill another 10%? How will these data centers survive 6 inches of ash? How reliable of a source of power are the dams when their are so many self-interest groups involved in their operation?
Is the low power cost number blinding companies to the risks?
Saving $3.5 million per year buys you a lot of geographic redundancy, especially when the risks you fear are measured on geological timescales. Volcanic eruptions in the Cascade range have averaged 1 per ~150 years for minor events and 1 per ~1000 years for major events. The last major event was in 1980, and the last two minor events were in the 19th century. At ~$3.5 million per annum you could save up $33.8 trillion dollars to use towards building and maintaining a datacenter outside of the range of those six inches of ash.
Any data center site selection must consider disaster risk, and the level of risk in this area of Washington is probably less than in many other areas of the country. There’s earthquake risk in California and Missouri, hurricane risk in Florida and Texas, and tornado risk in the Midwest.The kind of math Chuck describes is part of the question. Given that Microsoft, Yahoo, Intuit, Dell, Sabey and Vantage are all in Quincy, and Amazon, Google and Facebook are in Oregon, I’d say the industry seems to have found a comfort level with the disaster profile of the Pacific Northwest.
@chuck Mt. St. Helen’s latest eruption ended in 2008. However, I doubt that St. Helens will ever blow its top again the way it did in 1980. But it is starting to bulge again. I remember watching the ash fall, and so perhaps I’m a little sensitive to the chance of it recurring again.
@Rich Great article – No doubt the industry is comfortable with the risks, and the pricing is right. I hear that the east side of Quincy is sold out of power until 2013 – perhaps creating even more Wenatchee projects?
Tony HamdanPosted June 8th, 2012
There is no question that occasional consolidation makes good sense and saves companies a lot of money. As for risk, which other commenters brought up, this is the main game at BlackRock. Risk managent, including redundancy, disaster recovery, and business continuity are at the heart of operation. When your business is asset and investment management then risk must be a factor in everything you do from projects to develop new products, to infrastructure upgrades and migration.
All the best