Record World Cup Traffic Slams Twitter

4 comments

The 2010 World Cup has lived up to its billing as a watershed event for the Internet, with web traffic from early game easily exceeding previous records from Election Night 2008. The leading casualty has been Twitter, where Tweetstorms during  World Cup games have exposed deeper scalability challenges.

Traffic spikes over the past week have overwhelmed Twitter’s internal network capacity, and the performance problems are likely to continue until a long-term fix is found, the company said last night, meaning outages and maintenance downtime are likely to continue through the remainder of the World Cup.

“From a site stability and service outage perspective, it’s been Twitter’s worst month since last October,” Twitter’s Sean Garrett wrote on the company blog. “We’re working through tweaks to our system in order to provide greater stability at a time when we’re facing record traffic. We have long-term solutions that we are working towards, but in the meantime, we are making real-time adjustments so that we can grow our capacity and avoid outages during the World Cup.”

Problems Run Deeper
Twitter has been turning off features during traffic spikes to ease the load. On Tuesday, for example, Twitter’s trend tracking was turned off during the Brazil-North Korea game. But those tweaks are addressing symptoms of larger scalability problems.

“As we go through this process, we have uncovered unexpected deeper issues and have even caused inadvertent downtime as a result of our attempts to make changes,” wrote Garrett, who said Twitter had fully expected capacity challenges during the World Cup. “What we didn’t anticipate was some of the complexities that have been inherent in fixing and optimizing our systems before and during the event.”

Traffic Hits All-Time High
Web traffic on Internet news sites hit a record 12.1 million visitors per minute on June 11 during the opening games of the World Cup, according to Akamai’s Net Usage Index.  That easily exceeded the previous record 8.6 million visitors per minute on Nov. 4, 2008 as Barack Obama was elected president. Yesterday’s match between Brazil and North Korea hit 10.6 million visitors, suggesting that record traffic may be a daily event through the World Cup finale on Sunday, July 11.

That traffic has surfaced structural problems with the design of Twitter’s internal network, the engineering team wrote last Friday. Jean-Paul Cozzatti reported that the company put two fast-growing, high-bandwidth components on the same segment of its internal network, which wasn’t adequately monitored.

In response, Twitter has doubled its internal network capacity and improved its load balancing and network monitoring. That still hasn’t been enough to avert performance problems, and there are more outages ahead as Twitter seeks to address its challenges.

“Over the next two weeks, we may perform relatively short planned maintenance on the site,” said Garrett. “During this time, the service will likely be taken down. We will not perform this work during World Cup games, and we will provide advance notification.”

The long-term fix? Twitter has announced plans to operate its own data center. We’re likely to hear more about Twitter’s problems and plans at next week’s Velocity 2010 conference, where Twitter’s John Adams is scheduled to present.

Here’s more of our coverage of Twitter’s infrastructure:

About the Author

Rich Miller is the founder and editor at large of Data Center Knowledge, and has been reporting on the data center sector since 2000. He has tracked the growing impact of high-density computing on the power and cooling of data centers, and the resulting push for improved energy efficiency in these facilities.

Add Your Comments

  • (will not be published)

4 Comments

  1. The World Cup event brings for the biggest world wide companies the best revenues. Not only socializing media have a lot of full access, but especially betting site companies.

  2. Twitter traffic these days is enormously high. I've myself observed lots of downtimes on twitter in the past few weeks. I wonder why don't they move to a better and reliable host.