Twitter Down, Overwhelmed by Whales
January 20th, 2010 By: Rich Miller
Twitter was offline this morning, experiencing its longest sustained downtime since an Aug. 8 outage from a denial of service attack. Reliability has been on ongoing project for Twitter as the service has scaled up to handle growing traffic. This popular microblogging service has been offline for about an hour this morning, according to the Pingdom uptime monitoring service. UPDATE: Looks like the Twitter.com site is available again as of about 7:55 a.m. Eastern time.
“We are experiencing an outage due to an extremely high number of whales,” reports the Twitter status page. “A sudden failure coupled with problems in switching to a backup system produced a high number of errors for around 90 minutes. This made the site largely inaccessible. No data was lost or compromised during this outage.”
The “whales” comment refers to the “Fail Whale” – the downtime mascot that appears whenever Twitter is unavailable. The appearance of the Fail Whale indicates a server error known as a 503, which then triggers a “Whale Watcher” script that prompts a review of the last 100,000 lines of server logs to sort out what has happened.
When at all possible, Twitter tries to adapt by slowing the site performance as an alternative to a 503. In some cases, this means disabling features like custom searches. In recent weeks Twitter.com users have periodically encountered messages that the service was over capacity, but the condition was usually temporary. At times of heavy load for more on how Twitter manages its capacity challenges, see Using Metrics to Vanquish the Fail Whale.
Twitter’s last major downtime event was a 3 hour, 40 minute outage on August 6, when Twitter was among the social networking sites targeted by an electronic attack, which prompted the service to beef up its network defenses.
While some Twitter-watchers continue to debate whether its growth is continuing, co-founder Ev Williams posted Jan. 12 that “across all metrics that matter, yesterday was Twitter’s highest-usage day ever. (And today will be bigger.)”
[...] outage response goes to Twitter for their Whale Watcher script. In automatic response to Fail Whales, this script automatically reviews the last 100,000 lines of [...]
[...] Follow this link: Twitter Down, Overwhelmed by Whales « Data Center Knowledge [...]