A Closer Look at the Kaleidoscope Supercomputer

Yesterday we wrote about the Kaleidoscope Project, which is being hosted in the Houston data center of managed hosting provider CyrusOne. Kaleidoscope harnesses 120 teraflops of computing power to generate digital images of oil reserves buried thousands of feet below the seabed. At right is a closer look at the Kaleidoscope installation, which is powered by 600 IBM PowerXCell 8i processors in eight cabinets that occupy just 22 square feet of floor space, resulting in a power load of 750 watts a square foot in that area of the CyrusOne data center. Kaleidoscope is a collaboration between Repsol; the Barcelona Supercomputing Center (home to the MareNostrum supercomputer); 3DGeo, a Houston-based imaging company formed by Stanford University professor and seismic imaging pioneer Biondo Biondi; and Stanford University’s Stanford Exploration Project (SEP).

Imaging the Earth’s Depths, at 750 Watts a SF

The search for oil on the ocean floor begins on the data center floor. Long before drilling gets underway, the data-crunching commences in places like the CyrusOne data center in Houston, where the Kaleidoscope Supercomputer applies 120 teraflops of computing power to generate digital images of oil reserves buried thousands of feet below the seabed.

Technology plays a critical role in finding oil and gas reserves at Spanish energy company Repsol YPF, which uses Kaleidoscope to identify drilling targets and reduce the risk of an expensive “dry hole.” Offshore wells typically cost at least $15 million, and some large platforms cost many times more. Computer modeling allows oil companies to analyze seismic data and produce 3D images that identify the best location and trajectory for drilling wells. These modeling applications require enormous computing power.

Not every data center can handle the demands of supercomputers like Kaleidoscope, which is powered by 600 IBM PowerXCell 8i processors in eight cabinets that occupy just 22 square feet of floor space in the CyrusOne data center. That works out to 750 watts a square foot of power.

“To meet the Kaleidoscope Supercomputer’s ultra-dense requirements and guarantee optimal installation and future-proof performance, we needed a colocation solution that delivered superior high-density capabilities across the board,” said Francisco Ortigosa, Repsol’s chief geophysicist and project leader.

Design Your Own Virtual Supercomputer

Ever wondered what it’s like to design a supercomputer? Computer science students from Purdue University have created an online game that can provide a virtual taste of the experience. Rack-A-Node challenges players to design and operate a simulated research supercomputer. You must stay within your budget as you fill racks with hardware and complete computing tasks. But watch out for those power loads!

The game was built to highlight Purdue’s participation in the Cluster Challenge at the SC ‘08 supercomputer conference on Nov. 15-21 in Austin, in which college teams compete to see who can build the best supercomputer in a day.

“Rack-A-Node is a game that captures the essence of the supercomputer challenge,” says Kyle Bowen, informatics manager for Information Technology at Purdue. “The player has to optimize the supercomputer for the type of science being performed.”

RFID in the Data Center

Bank of America said last week that it is using radio frequency identification chips (RFID) to keep track of servers and other IT equipment, and is working to advance a standard for RFID tracking in financial data centers. The huge bank, which just got even bigger with its acquisition of Merrill Lynch, has deployed RFID in 14 of its 28 data centers, the company told RFID Journal (link via Zero Downtime).

RFID allows information to be stored and retrieved on small devices called RFID tags, The technology is used in enterprise supply chain management, allowing companies to keep track of the location and status of products and orders. RFID has obvious utility in data center consolidations and migrations and in managing server sprawl in large organizations.  

Hardware Financing Terms Tightening

Loans used to purchase servers and other data center hardware are becoming harder to find, which could leave some customers unable to finance IT purchases, or more inclined to buy from well-heeled vendors who can offer financing. The Wall Street Journal (subscription) reports:

Troubles are brewing in the technology-financing business, the credit that greases many technology sales. Defaults on tech financings, loans that allow companies to purchase computers, software and other products, have spiked this year. The problems are surfacing after years in which such loans flowed freely. Now the banks and specialty lenders that most tech companies rely on to finance customer sales are retrenching, and financing terms are getting tougher. Some big tech companies, such as International Business Machines Corp., Oracle Corp. and Cisco Systems Inc., are stepping into the void — lending more of their own money to customers and taking on new risks.

Who stands to benefit? IBM, Cisco and Oracle all have large customer financing operations. How bad is the problem? In September, 0.86% of equipment loans were written off as losses, up from 0.48% a year earlier, according to the Equipment Leasing and Finance Association, an industry group for 700 lenders. Tech-financings will reach $88 billion, about 14% of the total amount spent on computer hardware and software this year, estimates research firm IDC.

Sun, Cisco Veterans Team on Arista Networks

Sun Microsystems co-founder Andy Bechtolsheim is leaving the company to become chairman of Arista Networks, a maker of 10 Gigabit Ethernet switches, which has also recruited former Cisco Systems data center executive Jayshree Ullal as CEO. Arista, previously known as Arastra, announce their hiring today. Already on the story are Om Malik at GigaOm and Ashlee Vance at The New York Times, who reports that Stanford’s David Cheriton will also join Arista as chief scientist.

Ullal headed Cisco’s Data Center 3.0 initiative, while Cheriton teamed with Bechtolsheim on early investments in Google (GOOG) and VMware (VMW).

“While a number of companies sell competing gear, the pedigree of Arista’s management and its modular, easy-to-update software have given the four-year-old firm instant credibility in Silicon Valley,” The Times writes. Arista’s early customers include content delivery network BitGravity and Lawrence Livermore National Laboratory and International Center for Advanced Internet Research at Northwestern University (iCAIR).

Beautifying Your Blackbox

Data center containers aren’t going to win any beauty contests. Containers are all about horsepower, efficiency and speed-to-market, right? Perhaps, but they’re not all going to be parked in garages in mammoth data centers. A case in point: the Sun MD units housed at Sun’s Menlo Park campus, which Sun’s Katy Dickinson writes about in her blog post, Blackbox Landscaping.  

“As a dedicated gardener, for the last six months I have been watching with interest the creation of a handsome landscape setting for the new Black Box now inside of Sun’s Menlo Park campus,” writes Dickinson, who has posted photos of the unusual setting. While some Sun MD customers are going with white containers, Sun has stuck with the original black version, as the green logo matches the plants and trees. Check it out.

Microsoft: PUE of 1.22 for Data Center Containers

Microsoft says its testing shows that the data center containers it is installing in its new Chicago data center are extraordinarily energy efficient. The 40-foot shipping containers packed with servers can deliver a Power Usage Effectiveness (PUE) energy efficiency rating of 1.22, which rivals recent PUEs reported by Google. Microsoft’s Mike Manos revealed the PUE numbers in a blog post about the Chicago data center, which has just completed its first phase of construction.

“The facility is already simply amazing and it’s a wonder to behold,” Manos writes of the $550 million, 500,000 square foot facility in Northlake, Illinois. “The joke we use internally is that this is not your mother’s data center.  You get that impression from the first moment you step into the ‘hangar bay’ on the first floor. The hangar’s first floor (pictured above) will house the container deployments and I can assure you it is like no data center you have ever seen.  It’s one more step to the industrialization of the IT world, or at least the cloud-scale operations space.”

The 40-foot CBlox containers can house as many as 2,500 servers, and Manos said they allow Microsoft to achieve a density “10 times the amount of compute in a traditional data center.” The company says it will pack between 150 and 220 containers in the first floor of the Chicago site, meaning the massive data center could house between 375,000 and 550,000 servers in the container farm.   

It also is helping Microsoft meet its goals for extraordinary energy efficiency. “Now I want to be careful here as the reporting of efficiency numbers can be a dangerous exercise in the blogosphere,” Manos writes. “But our testing shows that our containers in Chicago can deliver an average PUE of 1.22 with an AVERAGE ANNUAL PEAK PUE of 1.36. I break these two numbers out separately because there is still some debate (at least in the circles I travel in) on which of these metrics is more meaningful.  Regardless of your position on which is more meaningful, you have to admit those numbers are pretty darn compelling.” 

Are Containers Helping Google’s Low PUE?

Google’s patented “data center in a box” appears to be among the innovations helping it achieve exceptional energy efficiency ratings. The news is the latest sign of containers’ potential to deliver higher density and better energy efficiency than traditional raised floor data centers.

Google said Oct. 2 that its six company-built facilities have an average Power Usage Effectiveness (PUE) rating of 1.21, and that one facility had reached a PUE of 1.13. Erik Teetzel, an Energy Program Manager at Google, told Data Center Knowledge that at least one of the six Google-built data centers “could in fact be a container data center.”

Google has never publicly discussed its data center container project. Last October Google was awarded a patent on a portable data centerin a shipping container, confirming a 2005 report from PBS columnist Robert Cringley that the company was building prototypes of container-based data centers in a garage in Mountain View. Containers also featured prominently in Google’s recently-disclosed patent filing for a floating data center that generates its own electricity using wave energy.

Teetzel’s comment suggests that Google has not only deployed its data center containers, but has done so ahead of Microsoft, which is currently putting the finishing touches on a huge new data center near Chicago. The bottom floor of the $550 million facility will house at least 150 data center containers packed with servers.

How Digg Works: A Look Under The Hood

The social media site Digg is known for the huge volume of traffic it sends to sites that are linked on its front page. How does Digg manage its web site to support all that activity - 22 million users and 220 million page views in August, according to QuantCast)? Joe Stump provides an overview of How Digg Works in the first post on the Digg Technology Blog, which made its debut Thursday. An excerpt:

Ask Ron - our Systems Engineering Lead - the exact number of servers we have in production and he’ll probably respond with, “I don’t honestly know.” I can say we’ve got dozens of web servers and dozens more DB servers. I can say with certainty it takes six specialized graph database servers to run the Recommendation Engine and we have another six to ten machines that serve files from MogileFS. But really, the numbers are the least interesting part of the equation. What makes Digg an interesting place to work are what the pieces are and how they fit together.

Stump’s post shares some details on Dogg’s architecture and how the various pieces work together to process typical requests from Diggers like me.

Digg isn’t alone among social networking sites sharing information about their efforts on the back-end. The Engineering @ Facebook blog provides an inside look at some of the infrastructure challenges face by Facebook.