The social media site Digg is known for the huge volume of traffic it sends to sites that are linked on its front page. How does Digg manage its web site to support all that activity – 22 million users and 220 million page views in August, according to QuantCast)? Joe Stump provides an overview of How Digg Works in the first post on the Digg Technology Blog, which made its debut Thursday. An excerpt:
Ask Ron – our Systems Engineering Lead – the exact number of servers we have in production and he’ll probably respond with, “I don’t honestly know.” I can say we’ve got dozens of web servers and dozens more DB servers. I can say with certainty it takes six specialized graph database servers to run the Recommendation Engine and we have another six to ten machines that serve files from MogileFS. But really, the numbers are the least interesting part of the equation. What makes Digg an interesting place to work are what the pieces are and how they fit together.
Stump’s post shares some details on Dogg’s architecture and how the various pieces work together to process typical requests from Diggers like me.
Digg isn’t alone among social networking sites sharing information about their efforts on the back-end. The Engineering @ Facebook blog provides an inside look at some of the infrastructure challenges face by Facebook.