Facebook, Google Ready for Faster Ethernet

As Facebook was announcing that it has reached 300 million users last week, network engineer Donn Lee was making the case for faster Ethernet networks – definitely 100-Gigabit Ethernet, and perhaps even Terabit Ethernet.

It is “not unreasonable to think Facebook will need its data center backbone fabric to grow to 64 terabits a second total capacity by the end of next year,” Lee told attendees of a seminar hosted by the Ethernet Alliance, an industry consortium focused on the expansion of Ethernet technology.  

10 Gigabit Ethernet is currently the fastest rate available, although higher capacity can be achieved by aggregating 10 Gigabit links. The Ethernet Alliance event last week brought together networking developers working on the IEEE 802.3ba standard for 40/100 Gigabit Ethernet. Presentations from Facebook’s Lee and Google network architect Bikash Koley offered a deep dive into network technologies, architecture and the future of Ethernet standards. The next phase of development envisioned is Terabit Ethernet, although some stakeholders see 400 Gigabit Ethernet as a likely interim step.

For huge operations like Facebook and Google, the timetable for higher capacity networks is important. Lee’s 21 minute presentation outlined some of the growing pains Facebook has experienced, and discusses the company’s vision of what its ideal future equipment and architecture might look like. Facebook’s network is so complex that in order to diagram it, Lee had to write a Visio script to make 92 connections into a network device.

Lee’s key points:

• 100-Gigabit Ethernet is urgently needed.
• The bottleneck is the data center fabric
• There is a need for a layer 2 scale fabrics.
• After solving the data center problem, they are looking at the metro area fiber.

Google network architect Koley also had an interesting presentation. He went much deeper into Ethernet technologies and the findings they have seen in fabric technologies, physical complexities and total cost of ownership.  Koley discusses fabric size versus port speed and what you are trying to optimize when building fabrics.  He then gets into the data center, explaining that lower port speed equals lower power consumption. Koley said

“We all need fatter pipes, but there are other ways of moving forward if the speeds come at the cost of too much power consumption, physical complexity and cost of ownership,” Koley said. He also discusses the fiber link budget and ways to use existing technology to push must faster speeds.

The Optoelectronics Industry Development Association is already starting on terabit Ethernet research. Chris Cole from Finisar said that terabit technology would require new and complex modulation schemes and other advanced techniques.

Rich Miller contributed to this article. Photo by Argonne National Library via Flickr (CC BY-SA 2.0

RELATED POSTS:

• Facebook Chat: 1 Billion Messages A Day
• Facebook Adding 600,000 Users A Day
• Consolidation Drives 10 Gig Ethernet Growth
• Video Drives 10 Gigabit Ethernet Adoption
• Equinix Exchanges to Offer 10 Gigabit Ethernet

About John Rath

John Rath is a veteran IT professional and regular contributor at Data Center Knowledge. He has served many roles in the data center, including support, system administration, web development and facility management.