The 365 Main flagship data center in San Francisco.

In the largest deal yet in its acquisition spree, Digital Realty Trust will buy the 365 Main portfolio of five data center properties from Rockwood Capital for about $725 million, the company said this morning. The transaction adds 919,000 square feet of data center space in five major markets, along with more than 200 customers.

The deal also provides Digital Realty Trust (DLR), the world’s largest operator of wholesale data center space, with 250,000 square feet of additional space in Chandler, Arizona that can be developed into turn-key data centers. Digital Realty is also acquiring $13 million of uninstalled data center infrastructure improvements.

‘Extending Our Leadership’
“Upon closing, the addition of these high quality, mission critical data center facilities to our operating portfolio will further extend our leadership position as the largest wholesale data center provider in the U.S.,” said Michael Foust, CEO of Digital Realty Trust. “The portfolio is leased to a diverse roster of over 200 tenants in various industries.”

The deal comes more than two years after the first reports that the 365 Main properties were for sale. There has been periodic industry chatter that a deal might be in the works, with Digital Realty often mentioned as a potential suitor. Big deals were difficult to come by during the economic downturn and accompanying credit crunch, but as industry deal activity picked up this spring, reports emerged that a deal was near.

Formed from AboveNet Assets
365 Main was formed to operate a San Francisco data center after AboveNet filed for bankruptcy in 2003. Rockwood Capital, which owns the building, has backed 365 Main in an expansion that has created a national network of five data centers:

• 365 Main Street in San Francisco
• 2260 East El Segundo Boulevard in El Segundo, California
• 720 2nd Street in Oakland, California
• 2121 South Price Road in Chandler, Arizona
• 4030-4050 Lafayette Center Drive in Chantilly, Virginia

365 Main also owned a development property in Newark, Calif., which was not mentioned in today’s announcement.

Retail / Wholesale Split

The San Francisco facility is unique in 365 Main’s portfolio in its retail colocation model, in which it leases cabinets and cages directly to customers. In its other markets, 365 Main employed a wholesale data center model, leasing large chunks of data center space to customers, who can use it for their own operations or to sell cabinets and cages to others.

The acquisition is scheduled to close on July 2. Digital Realty said it will fund the acquisition with “available cash, borrowings under the company’s revolving credit facility and the sale of debt or equity securities.” The company announced plans to sell 4.25 million shares of common stock.

The 365 Main flagship data center in San Francisco.

How do you engineer a data center for high availability in an earthquake zone? That’s a special challenge in San Francisco, which is home to a vibrant community of Internet businesses but experienced widespread devastation from a 1906 quake and sustained another big hit in the 1989 Loma Prieta (“World Series”) earthquake.

365 Main houses mission-critical equipment for more than 200 companies in its data center nestled alongside the Bay Bridge. When original owner AboveNet was converting the former tank turret factory for data center use in 2000, it confronted the earthquake risk, installing a base isolation designed to keep the entire building stable when the earth moves.

“This is one of the safest buildings in San Francisco,” said Miles Kelly, senior VP of marketing for 365 Main.

Many data center companies use rack-level earthquake isolation units, which are installed under racks and cabinets and employ a ball-and-cone system to allow the equipment to gently roll back and forth during an earthquake. Providing earthquake protection at the building level involves similar concepts, but a lot more engineering.

The 365 Main building is built atop bedrock, and the 98 columns supporting the building are equipped with the base isolation systems, which are visible in the facility’s underground parking garage. Each of the massive  columns is topped with a special joint known as a “friction pendulum” consisting of a plate and rubber bearings that absorbs the shock created by seismic movements. (See this video for an overview of a similar system).

A “Floating” Building
In an earthquake, this will allow the entire building to effectively float above the shifting ground beneath it. The piping, cabling and utility connections join the building above the isolation joints to protect their integrity in the event of an earthquake.

The seismic engineering extends outside the building, which is surrounded by an 18-inch wide “mini-moat” that will allow the sidewalk areas to fold and absorb the shock of a large quake.

Like many large metro-area data centers, 365 Main would be a priority customer for diesel fuel deliveries to keep the facility online after a disaster. The city of San Francisco has granted 365 Main “essential facility” status, allowing continued occupancy without the inspections required of buildings with less sturdy quake protection.

365 Main was formed to operate the San Francisco data center after AboveNet filed for bankruptcy in 2003. Rockwood Capital, which owns the building, has backed 365 Main in an expansion that has created a national network of five data centers, including California facilities in San Francisco, Oakland and El Segundo (LA market) as well as Chandler, Arizona and Chantilly, Virginia. The company also owns a property in Newark, Calif. for future expansion.

Retail vs. Wholesale Space
The San Francisco facility is unique in 365 Main’s portfolio in its retail colocation model, leasing cabinets and cages directly to customers. In its other markets, 365 Main employs a wholesale data center model, leasing large chunks of data center space to customers, who can use it for their own operations or to sell cabinets and cages to others.

Each of 365 Main’s buildings is designed with pods of 10,000 to 12,000 square feet of spoace. There are eight pods in the 227,000 square foot San Francisco facility. Kelly said 93 percent of available floor space is filled, but the facility is at 98 percent of its power capacity. During the 2000 retrofit, the company added a fifth floor to house generators and rotary UPS units.

The generators continue to be a point of interest for prospects aware of a 2007 outage resulting from the failure of multiple generators. But Kelly says the company’s response, including the release of an unusually detailed incident report, helped 365 Main implement additional procedures to ensure the generators’ reliability.

The 365 Main flagship data center in San Francisco.

Colocation provider 365 Main says it has space available in its flagship data center in San Francisco for the first time since late 2006. The new capacity is a result of several large customers “graduating” to the company’s new facility across the bay in Oakland, and energy savings created by server consolidations by several customers.

San Francisco has historically been one of the strongest markets for colocation space, according to 365 Main VP of marketing Miles Kelly. “We’re in deep conversations with existing customers to take some of that capacity,” said Kelly. “We expect it to go quickly. San Francisco is a tight data center market.”

Two large customers have relocated to 365 Main’s Oakland data center, which opened in early 2007 and has more than 80,000 square feet of raised-floor technical space. Kelly wouldn’t say which customers, but previous announcements identify blog hosting provider Six Apart and social network Hi5 as customers that have migrated from San Francisco to Oakland.

Those migrations took place some months ago. But data center capacity is guided by power as well as space, and the recent consolidations have freed up power capacity for additional customers. 365 Main has encouraged its San Francisco customers to take advantage of incentives from PG&E that offer up to $200 per server for consolidations that lower overall server counts. Kelly said one of the customers has reduced its total power usage by 20 percent.  

“Usually what tenants will do when they save space (through a consolidation) is consider it additional headroom to fail before they grow and have to look for a new facility,” said Kelly. 

Kelly said the space available at the San Francisco data center wasn’t a sign of weakening demand in the local colocation market, noting recent projections from Tier 1 Research. In a November report, Tier 1 predicted that colocation demand in Silicon Valley and the Bay Area will continue to outpace supply, boosting data center utilization from the current 70 percent to 95 percent by 2012.   

“The demand side of the equation continues to be healthy,” said Kelly. “Our executives were part of the first Internet building boom in the 1990s. What we found is that in tight times, an investment in technology is an investment in productivity.”

The San Francisco facility was the first data center for 365 Main, which bought the building from AboveNet in a Chapter 11 sale. Its business model also differs from the rest of 365 Main’s portfolio, which includes properties in El Segundo, Calif.; Chandler, Ariz.; and Chantilly, Va. in addition to San Francisco and Oakland.

“365 Main has always been a traditional colo model in San Francisco, where you can lease a a half-rack or a rack or server,” said Kelly. “A cage was really the upper extent for most customers here.  Every other facility in our portfolio is a wholesale colocation model, where we lease 2,000 to 25,000 square feet of space. That’s why in San Francisco, we have 200 customers in the facility, where we have just eight in an equal amount of space in El Segundo.”

hi5 was founded in 2003 and is now the 8th most-trafficked website in the world according to Alexa. The company will lease an additional 2,500 square feet of space at the Oakland data center, which opened in September 2007.

“365 Main has been responsive to our needs and has provided a secure home for our servers in San Francisco for more than three years,” said hi5 Founder and CEO, Ramu Yalamanchi. “With our rapidly expanding service and necessity for multiple high-performance data center locations, expanding our partnership and leasing additional space from 365 Main in Oakland made sense for our business.”

“Social networking sites like hi5 promise a reliable, 24X7 experience to their users, and to deliver on that promise requires modern data center infrastructure,” said Chris Dolan, CEO of 365 Main. “Our Oakland facility was designed for exactly this type of requirement, and we’re proud to be adding this world-class organization to our existing Oakland tenant roster.”

365 Main has existing facilities in San Francisco, Oakland, El Segundo, Calif., Chandler, Ariz. and Chantilly, Va. The company plans to open a seventh data center in Vernon, Calif. in 2008.

The specialized nature of data centers makes it more difficult to measure their energy efficiency using programs designed for office buildings. These include the LEED “green building” certification system from the U.S. Green Building Council (USGBC) as well as the Energy Star program for buildings. While it is best known for its energy ratings for appliances, Energy Star has also has certified more 4,350 buildings as meeting its guidelines.

As an Energy Star partner, 365 Main agrees to measure and track energy performance using tools offered by Energy Star, and implement a plan to achieve energy savings consistent with the program’s guidelines.

“EPA would like to congratulate 365 Main for joining ENERGY STAR as a partner,” said Jean Lupinacci, Chief of EPA’s ENERGY STAR Commercial Buildings program. “Strategic energy management is a critical element in our efforts on climate change, and by joining ENERGY STAR, 365 Main is helping lead the way to greater energy savings in the data center industry and real results for the environment.”

“365 Main brings to the table our expertise from operating more than one million square feet of data center space nationwide across five facilities, as well as our experience with reducing energy use through green programs offered by PG&E and our involvement with groups such as the Green Grid and U.S. Green Building Council,” said J.P. Balajadia, SVP of engineering for 365 Main.

365 Main also said it will work with EPA in developing an Energy Star Infrastructure Rating for Data Centers. The effort, still in its early stages, is designed to make it possible for data center facilities to earn Energy Star certification for superior energy-efficiency. Energy Star is gathering feedback from data center operators about the power usage of these facilities and how best to develop appropriate rating criteria.

Energy Star was introduced by EPA in 1992 as a voluntary program to reduce greenhouse gas emissions through energy efficiency. The Energy Star label can be found on more than 50 different kinds of products, new homes and commercial and industrial buildings. The EPA estimates that the initiative has saved Americans about $14 billion on their energy bills while reducing the greenhouse gas emissions equivalent to those of 25 million vehicles.

The specialized nature of data centers makes it more difficult to measure their energy efficiency using programs designed for office buildings. These include the LEED “green building” certification system from the U.S. Green Building Council (USGBC) as well as the Energy Star program for buildings. While it is best known for its energy ratings for appliances, Energy Star has also has certified more 4,350 buildings as meeting its guidelines.

As an Energy Star partner, 365 Main agrees to measure and track energy performance using tools offered by Energy Star, and implement a plan to achieve energy savings consistent with the program’s guidelines.

“EPA would like to congratulate 365 Main for joining ENERGY STAR as a partner,” said Jean Lupinacci, Chief of EPA’s ENERGY STAR Commercial Buildings program. “Strategic energy management is a critical element in our efforts on climate change, and by joining ENERGY STAR, 365 Main is helping lead the way to greater energy savings in the data center industry and real results for the environment.”

“365 Main brings to the table our expertise from operating more than one million square feet of data center space nationwide across five facilities, as well as our experience with reducing energy use through green programs offered by PG&E and our involvement with groups such as the Green Grid and U.S. Green Building Council,” said J.P. Balajadia, SVP of engineering for 365 Main.

365 Main also said it will work with EPA in developing an Energy Star Infrastructure Rating for Data Centers. The effort, still in its early stages, is designed to make it possible for data center facilities to earn Energy Star certification for superior energy-efficiency. Energy Star is gathering feedback from data center operators about the power usage of these facilities and how best to develop appropriate rating criteria.

Energy Star was introduced by EPA in 1992 as a voluntary program to reduce greenhouse gas emissions through energy efficiency. The Energy Star label can be found on more than 50 different kinds of products, new homes and commercial and industrial buildings. The EPA estimates that the initiative has saved Americans about $14 billion on their energy bills while reducing the greenhouse gas emissions equivalent to those of 25 million vehicles.

The measures were implemented through Savings By Design, an energy conservation initiative backed by four of California’s power companies: PG&E, San Diego Gas and Electric, Southern California Edison, and the Sacramento Municipal Utility District (SMUD). “Energy-efficient development is a top priority for us not only because it helps keep energy costs down, but also because we know we’re ultimately reducing our company’s carbon footprint,” said Chris Dolan, CEO of 365 Main.

Under the Savings By Design process, PG&E and a third party reviewed design drawings of 365 Main’s 111,000 square foot Oakland data center, and recommended design changes that would reduce power usage. Among the suggestions that were implemented:

• High-efficiency motors: Installing efficient motors throughout the data center improves the overall output of each device without drawing additional power. With an estimated annual cost savings of $4,335 the expected payback period is less than 2 years.
• Variable speed drives on the cooling tower fan motors: Cooling towers circulate the water throughout the data center’s air conditioning system. Using variable speed drives allows the motors to ramp up and down as needed, compared to the “always on” state of traditional fan motors. The payback period for the premium variable speed drives is 3 years, with an annual energy savings of 66,828 kWh and estimated annual cost savings of $8,019.
• High-efficiency pumps: High-performance pumps circulate 10 percent more water per minute than traditional pumps, but use the same amount of electricity. Estimated annual cost savings: $8,253.
• High-efficiency chillers: Boosting cooling capacity 20 percent above that of traditional chillers will save 404,274 kWh and produce cost savings of $48,513.
• High-efficiency uninterruptible power supply (UPS): The biggest gains were found in at the UPS level with a flywheel that achieved a 93-96% efficiency, for an estimated energy savings of 1.09 million kWh and estimated annual cost savings of $131,339

These measures were deemed by Savings By Design to have produced total estimated annual energy savings of 1.67 million kWh, and estimated annual cost savings of $200,460.

“As an industry leader in energy efficiency, PG&E helps our customers implement building design that makes good sense – to the owner, to the design team, to the environment and to the bottom line,” said Helen Burt, senior vice president and chief customer officer at PG&E. “We are excited to see 365 Main showing such leadership and commitment to the environment.”

There was only one problem: customers rejected the idea, even though it would have meant no additional cost. The reason? By the company’s calculations, the natural-gas-powered generators alone could deliver only a 94 percent guaranteed uptime. Although grid backup would bring the uptime projection to 99.8 percent, customers were unwilling to support a greener solution if it had even a fractional slippage on uptime guarantees.

365 Main abandoned the plans, but expects to have its Newark site certified as a LEED building. 365 Main’s Miles Kelly said it illustrates the challenges of making IT greener. “Everyone talks about the beauty of the intersection between business needs and green,” Kelly told Samson. “Here we are bringing a real option to the table with tangible benefits, and all of a sudden, we’re back to using traditional energy sources.”

The bottom line message from customers appears to be this: Green is great. Just don’t get any of it on my uptime.

365 Main bought the Oakland facility in January and leased one-third of its real estate within the following 90 days. “While we’ve enjoyed tremendous leasing activity at the Oakland building since day one, we’ve made significant upgrades to meet our strict standards for power, cooling, connectivity and security,” said Chris Dolan, 365 Main CEO. “The building isn’t entirely finished, but demand is so high that the decision to officially open our doors for business was an easy one.”

Six Apart has been a tenant at 365 Main’s San Francisco data center for nearly two years, and recently expanded into space at the Oakland facility, which is less than 10 miles away. “Leasing additional space in Oakland allows us to support our growth in what is a rapidly expanding market,” said Lisa Phillips, Director of Operations Engineering at Six Apart. The company’s expansion at 365 Main is notable, as Six Apart’s LiveJournal, Vox and TypePad services were briefly knocked offline during a July outage at 365 Main’s San Francisco data canter.

365 Main also announced that AboveNet and IP Networks, Inc. (IPN) have extended their networks and have laid fiber from 365 Main’s San Francisco data center to the new building in Oakland, providing Six Apart access to the more than 20 connectivity providers available at the San Francisco facility.

Oakland officials and company executives will hold a grand opening event today at the facility. “By welcoming companies like 365 Main into the fabric of our economy, we have made a significant step forward in our efforts to become a twenty-first century Model City,” said Oakland Mayor Ronald Dellums. “Oakland has made a substantial investment in technology infrastructure and is excited about the business this world-class data center is already bringing to our community.”

365 Main’s final report confirmed preliminary findings that the outage was caused by flaws in a component known as a Detroit Diesel Electronic Controller (DDEC), a system which monitors a diesel engine’s status, and can activate alarms or shut down a generator. Erroneous data from the DDEC caused 365 Main’s diesel engines to malfunction, failing to start properly after a grid outage.

“In the days since the incident occurred, we identified and corrected the root source of the problem and are taking steps to prevent this type of problem from happening again,” said Dolan. “We are also making our comprehensive findings available to other data centers to try to prevent the same problem from recurring elsewhere.”

365 Main’s VP of marketing, Miles Kelly, said the company is focused on communicating with its customers. “This is definitely a humbling experience and a painful experience,” said Kelly. “We’re following a very predictable emotional cycle after a traumatic event. Right now our customers are looking for assurance that the issues are fully addressed, and that something like this won’t affect them again in the future people. This is about stabilizing the credibility we’ve established with our customers.”

While there are reports that some customers may consider relocating their equipment – or at least establishing backup sites – Kelly said the company has benefited from its long-term performance. Kelly said that that over the last five years, 365 Main has maintained uptime of 99.9967% across its entire fiv-facility data center footprint.

After sharp criticism in the initial hours folliowing the outage, 365 Main has posted regular updates of its investigation on its public web site. The amount of information released was unusual compared with most previous industry outages. Kelly believes this has helped address customer concerns.

“We are an open book, frankly,” said Kelly. “We hope we’ve now set the communications standard for dealing with power outages.”