Facebook’s Response to Greenpeace

The environmental group Greenpeace International says Facebook should rethink plans for its new Oregon data center and is urging Facebook users to join a group calling for the new facility to run entirely on renewable energy, rather than using utility power that generates a majority of its power from coal. After we noted Facebook’s response in our comments earlier this week, the issue has been getting attention from environmental groups and green blogs, which then sparked interest from Greenpeace.

“Given the massive amounts of electricity that even energy-efficient data centers consume to run computers, backup power units, and power related cooling equipment, the last thing we need to be doing is building them in places where they are increasing demand for dirty coal-fired power,” the group said in a statment, which was published on the Greenpeace web site and also on The Huffington Post. “Facebook and the cloud should be run on clean renewable energy … Facebook could and should be championing clean energy solutions, and not relying on the dirty fuel sources of the past to power their new data center.”

Greenpeace has urged Facebook users to join a Facebook group titled Tell Facebook to use Clean Energy for its Data Center.

“It is simply untrue to say that we chose coal as a source of power,” Facebook said in response to Greenpeace. “The suggestions of ‘choosing coal’ ignores the fact that there is no such thing as a coal-powered data center. Similarly, there is no such thing as a hydroelectric-powered data center. Every data center plugs into the grid offered by their utility or power provider.”

Here is Facebook’s updated response, in its entirety:

Overall, we’re thrilled at our choice in Oregon and that we’re challenging the industry to think creatively to meet the standards we’ve set in efficiency. As we continue to grow, we’re committed to environmental responsibility and will be seeking and evaluating more ways to minimize and offset our impact on the planet. In selecting Oregon, we chose a region that offers a uniquely dry and temperate climate. This climate enables us to design what we believe to be one of, if not the most, energy efficient data centers in the world. Specifically, most data centers use mechanical chillers or large air conditioners for part, if not all, of the year to cool the computers within the facility. These mechanical chillers use a lot of energy and are only exceeded in their energy use by the thousands of computers inside the data center. Because of the climate around Prineville and our unique design, we won’t use any mechanical chillers. None. We won’t even build any. Instead, the data center will use an innovative evaporative cooling system.

Facebook Responds on Coal Power in Data Center

An architectural rendering of the new Facebook data center planned for Prineville, Oregon.

Facebook has responded to growing criticism of its power choices for its new data center in Prineville, Oregon. This is one of the first cases in which a data center’s energy sourcing has attracted this kind of public attention, but it won’t be the last.  

Earlier this month we noted a report that Facebook’s new Oregon data center, which has been designed to be highly energy-efficient, would be getting its power from a local utility that uses coal to generate the majority of its power. This news, initially reported by SearchDataCenter, has been getting attention from environmental groups and green blogs. The issue was highlighted on TreeHugger and Change.org, and has even led to the creation of a Facebook group (Tell Facebook to use Clean Energy for its Data Center).

Facebook’s Response 
Yesterday Facebook responded via our comment section on the original post, with a statement from spokesperson Lee Weinstein. Since Facebook has taken its lumps on this issue, I felt its response should get equal visibility. Here’s Weinstein’s comment:

I’m writing on behalf of Facebook to share their response to the issues you’ve posed. Most electrical commercial and residential power in the United States comes from a variety of sources. Our new data center will be receiving our power through PacifiCorp, which like most utilities has a diverse generation portfolio including hydro, geothermal, wind and coal. PacifiCorp is now the #1 utility owner operator of renewables, having grown their portfolio 2,400 percent over the past three years.

When it comes online in early 2011, the new Facebook data center will also be one of the most energy efficient in the world, featuring an innovative cooling system created for the unique climate characteristics in Prineville, Oregon.

The new, world class energy-efficiency technologies the Facebook data center will utilize include an evaporative cooling system; an airside economizer that will bring colder air in from the outside; re-use of server heat to warm office space in the colder months; and new patent pending highly efficient electrical design will reduce electricity usage by up to 12 percent. The entire facility will be built to LEED Gold standards.

It’s Official: Facebook is Oregon’s Company X

An architectural rendering of the new Facebook data center planned for Prineville, Oregon.

Facebook’s first company-built data center will be in Prineville, Oregon, the company confirmed today. The new facility will be among the most energy efficient in the industry, Facebook said, and will provide the social network with headroom for its fast-growing server and storage infrastructure.

As we noted yesterday (Facebook to Build Its Own Data Centers), the fast-growing social network has previously leased server space from wholesale data center providers, but has grown to the point where the economics favor a shift to a custom-built infrastructure.

LEED Gold, PUE of 1.15
Facebook says the 147,000 square foot Prineville data center will be designed to LEED Gold standards and is expected to have a Power Usage Effectiveness (PUE) rating of 1.15.

“After a rigorous review process of sites across the West Coast, Facebook concluded that Prineville offered the best package of resources – including a suitable climate for environmental cooling, renewable power resources, available land, talented regional workforce and supportive business environment,” said Tom Furlong, Director of Site Operations for Facebook.

Facebook also left open the possibility that it will build more than one data center in Prineville. “Additional construction phases may be possible in the future, depending on business needs,” according to a press release issued by the state of Oregon.

A Long Way from a Single Server
Facebook’s extraordinary growth has forced the company to continually invest in its infrastructure.

“We have come a long way from our roots in a Harvard dorm room, when Facebook was only available at some colleges and run on a single server,” said Jonathan Heiliger, Facebook’s vice president of technical operations, in a blog post. “When Facebook first began with a small group of people using it and no photos or videos to display, the entire service could run on a single server.

“Now with more than 350 million people worldwide and our service and business continuing to grow, we must constantly scale our technical infrastructure to meet the demand and deliver you a fast, reliable experience,” said Heiliger. “An important step along the way is to build a custom data center so that we can design it to meet our unique needs.”

The social network has previously expanded by leasing space from “wholesale” data center landlords,including Digital Realty Trust, DuPont Fabros Technologies and Fortune Data Centers.

More Cost, But More Customization
Other huge Internet companies that build their own data centers include Google, Microsoft, Yahoo, eBay and Oracle. This typically requires a larger up-front investment in construction and equipment, but allows greater customization of power and cooling infrastructure.

The Prineville data center will use evaporative cooling instead of a chiller system, continuing a trend towards chiller-less data centers and water conservation. “This process is highly energy efficient and minimizes water consumption by using outside air,” said Heiliger.

The facility will also re-use excess heat expelled by servers, which will help heat office space in the building, a strategy also being implemented by Telehouse and IBM.

Novel UPS Design
The new design foregoes traditional uninterruptible power supply (UPS) and power distribution units (PDUs) and adds a 12 volt battery to each server power supply. This approach was pioneered by Google, which last year revealed a custom server that integrates a 12 volt battery, which the company cited this design as a key factor in the exceptional energy efficiency data for its data centers.

Facebook did not say how much it expected to spend on the project, which is expected to create more than 200 jobs during its 12-month construction phase, and will employ at least 35 full-time workers and dozens more part-time and contract employees. Those jobs are welcomed by Oregon officials.

Excitement in Prineville
“This is great news for Prineville and really the entire state,” said Gov. Ted Kulongoski. “The stable, family-wage jobs and economic stimulus they will provide to this area during construction are a bright spot as this nation and this state climb out of this recession.”

“I thank Facebook for choosing Oregon and am hopeful this decision will act as a big “open for business sign” for other companies in this quickly growing field,” said Kulongoski. “This is a great example of how the public and private sector work together to create economic opportunity for Oregon communities.”

Here’s a look at some of our past coverage of the growth of Facebook’s infrastructure:

• Facebook Follows Google to Data Center Savings: Facebook says it’s streamlining its servers, and also plans to adopt a novel power distribution design pioneered by Google.
• Facebook Goes Green With New Data Center Space: The breakout growth of Facebook is turning out to be an economic stimulus plan for data center landlords. And the fast-growing social network’s infrastructure isn’t just getting bigger, it’s getting greener.
• Facebook Makes Big Investment in Data Centers: The company has just locked down a large chunk of data center space in northern Virginia that will provide room for thousands of additional servers for growth in 2011 and beyond.
• Facebook: Managing Epic Growth in Real-Time: CTO Jonathan Heiliger presents an overview of Facebook’s back-end operations at the Velocity 2009 conference in June.
• Facebook Spending $20 Million a Year on Data Centers: An analysis of Facebook’s data center leases as of May 2009.
• A Look Inside Facebook’s Data Center: A recruitment video provides a glimpse inside the server-packed racks and aisles of a Facebook data center.
• Facebook Expanding Its Data Centers, Again: In early 2009 the company leased additional space in its East Coast hub in Ashburn, Virginia.
• Facebook Pushes Limits on Memcached: Caching is key to massive web scalabilty. Here’s how Facebook is extending a popular caching technology.

A sketch of the entrance of the new Facebook data center in Prineville, Oregon.

Facebook to Build Its Own Data Centers

A look at the fully-packed racks inside a Facebook data center facility.

Facebook has decided to begin building its own data centers, and may announce its first facility as soon as tomorrow. The fast-growing social network has previously leased server space from wholesale data center providers, but has grown to the point where the economics favor a shift to a custom-built infrastructure.

“Facebook is always looking at ways to scale our infrastructure and better serve our users,” said Facebook spokesperson Kathleen Loughlin said last week. “It should come as no surprise that, at some point, building a customized data center will be the most efficient and cost effective way to to do this. However, we have nothing further to announce at this time.”

UPDATE: Facebook has confirmed that it will build a 147,000 square foot data center project in Prineville, Oregon.

Not Google or Yahoo
The data center is being built by Vitesse LLC on behalf of an unidentified tenant. But Vitesse has said Company X is not either Google or Yahoo. Data center industry chatter suggests the tenant is a large social networking site – which usually means Facebook.

How Many Servers Can One Admin Manage?

There’s a discussion today at Slashdot about how many servers or users an admin can manage. The question seeks to establish whether a department in which sysadmins each manage 900 user machines is understaffed. As in any discussion of IT or data centers, practices vary widely. But here are a couple of relevant factoids from our travels tracking the data center industry.

Jeff Rothschild, the vice president of technology at Facebook, said in a recent presentation that Facebook has 230 engineers supporting data for more than 300 million users. He says Facebook seeks to maintain a ratio of one engineer for 1 million or more users. Facebook is vague about exactly how many servers it has, saying it’s “more than 30,000.” But 30,000 servers and 230 engineers works out to a ratio of about 130 servers per admin.

Microsoft says it has automated its data center operations to the point where its admins can each manage between 1,000 and 2,000 servers. That matters, as the company may pack more than 300,000 servers into its new container data center in Chicago. It expects to support that facility with about 30 employees, including admins and facility maintenance staff.

Should Servers Come With Batteries?

Will the data center of the future have no central UPS units, and be filled with servers with on-board batteries? The data center team at Facebook believes it should, and is pledging to share its best practices - and perhaps wield some of its clout with vendors and data center operators - as it presses its case for change.   

Facebook recently disclosed its plans to adopt a novel power distribution design pioneered by Google that removes uninterruptible power supply (UPS) and power distribution units (PDUs) from the data center. The new design shifts the UPS and battery backup functions from the data center into the cabinet by adding a 12 volt battery to each server power supply.

While many best practices shared by Google, Microsoft and Facebook can help other data center operators save energy and money, other customizations are impractical.   

Big Companies, Big Innovation 
“A lot of the innovation in the field is being driven by companies with thousands of servers who really care about the efficiency of these things,” said Facebook’s Amir Michael, who previously worked on Google’s data center team. “We have capital to be able to afford engineers to solve these problems. It’s not really benefiting the rest of the industry. Smaller companies who might deploy fewer servers can’t go and design their own systems.”

In discussing Facebook’s plans for on-board batteries, Michael discussed ways these innovations might become more widely available.

Facebook Follows Google to Data Center Savings

A graphic of the new data center power distribution system being implemented at Facebook, which replaces a central UPS with a battery built into the power supply.

Facebook is stepping up its efforts to make its data centers cheaper and more efficient, and is following in the footsteps of Google in several of its key initiatives. Facebook says it’s streamlining its servers, and also plans to adopt a novel power distribution design pioneered by Google.

The social network’s plans were discussed by Amir Michael, a server and data center engineer at Facebook, in a Nov. 17 engineering Tech Talk in Palo Alto, Calif. Michael joined Facebook in March after six years on the data center team at Google, where he designed cooling and electrical systems.

“The industry has discussed many ways of optimizing servers and data centers,” Michael said. “We don’t claim to be the original thought leader on many of these things, but we are gonna be one of the few companies who’s actually going to begin implementing them.”

Shift to On-Board Battery
That’s true of the most significant change outlined by Michael: a new power distribution design that removes traditional uninterruptible power supply (UPS) and power distribution units (PDUs) from the data center. The new design shifts the UPS and battery backup functions from the data center into the cabinet, adding a 12 volt battery to each server power supply.

Roundup: Force 10 Networks, Avocent, Facebook

Here’s a roundup of news announcements from the data center and hosting industry:

• Force 10 demonstrates at Interop. Force 10 Networks partnered with AFORE Solutions and Sun Microsystems at INTEROP in New York last week to demonstrate data center virutalization products and cloud computing initiatives focused on driving greater network agility. The 3 vendors demonstrated virtualized resource allocation for cloud-oriented applications as well as the AFORE ASE3300 Virutalization Extension Platform.  The ASE3300 and Force 10 switch and router solutions combine to enable a multi-site, virtual data center enabling migration to cloud computing environments.
• Avocent upgrades data center management software. Announced at Interop last week, Avocent is upgrading its MergePoint Infrastructure Explorer to include several new management capabilities.  The company said these enhancements will provide a unique view into capacity planning, bringing additional return on investment and total cost of ownership benefits.  Avocent CTO Ben Grimes said that the software will allow “customers to know where their assets are, as well as plan for different ‘what if’ scenarios, and manage their data centers to reduce risk -  all while bringing improved ROI and total cost of ownership benefits to customers.”  New features include rack timeline and an enhanced change management and capacity search capabilities.

Facebook Goes Green with New Data Centers

The breakout growth of Facebook is turning out to be an economic stimulus plan for data center landlords. And the fast-growing social network’s infrastructure isn’t just getting bigger, it’s getting greener. 

Facebook has confirmed that it has leased additional data center space in San Jose, Calif. in a new facility operated by Fortune Data Centers. This is the fourth data center expansion of 2009 for Facebook, which has signed two leases for additional data center space in Virginia and two more in Silicon Valley.

All LEED Gold Data Centers
All four of the leases are in facilities designed to attain the highest energy efficiency ratings under the LEED (Leadership in Energy and Environmental Design) program operated by the U.S. Green Building Council. Each of the data centers hosting Facebook’s servers are on track for Gold certification under the LEED rating scale. 

As with most companies seeking out LEED data centers, Facebook’s motivations are financial as well as environmental. The social network’s racks are packed with servers, and efficient  data centers allow them to do more with the same square footage.