Google Turns to Tape to Rescue Lost Gmail

Last month we had an article on whether tape backup was obsolete, which discussed some of the reasons why many organizations continue to use tapes. One of the outfits  is Google, and this week those tape backups proved invaluable to a group of Gmail users whose email boxes disappeared during a software update. When a bug wiped out Gmail boxes, Google turned to the tape to restore the data.

“I know what some of you are thinking: how could this happen if we have multiple copies of your data, in multiple data centers?” wrote Ben Treynor, VP Engineering and Site Reliability Czar. “Well, in some rare instances software bugs can affect several copies of the data. That’s what happened here. Some copies of mail were deleted, and we’ve been hard at work over the last 30 hours getting it back for the people affected by this issue.

“To protect your information from these unusual bugs, we also back it up to tape. Since the tapes are offline, they’re protected from such software bugs. But restoring data from them also takes longer than transferring your requests to another data center, which is why it’s taken us hours to get the email back instead of milliseconds.”

Google says it will release a full incident report once its analysis is completed. The company said the bug affected 0.02 percent of Gmail users.

Get Daily Email News from DCK!
Subscribe now and get our special report, "The World's Most Unique Data Centers."

Enter your email to receive messages about offerings by Penton, its brands, affiliates and/or third-party partners, consistent with Penton's Privacy Policy.

About the Author

Rich Miller is the founder and editor at large of Data Center Knowledge, and has been reporting on the data center sector since 2000. He has tracked the growing impact of high-density computing on the power and cooling of data centers, and the resulting push for improved energy efficiency in these facilities.

Add Your Comments

  • (will not be published)


  1. It should be noted that Google offers these services at little to no charge. Clients who demand their mission critical data to be redundantly stored have the peace of mind to know that reputable data centers are in lock step with their needs. Sure there is a cost to a higher level of service, but a data loss can result in a company going out of business so it is referred to as a value. How palatable is a free/low cost service if one event can be the end of your business?