The entrance to the 111 8th Avenue carrier hotel in New York City

The entrance to the 111 8th Avenue carrier hotel in New York City

Internap Data Center Outage Takes Down Livestream, StackExchange

2 comments

Internap experienced a power outage at its data center located in the Google-owned building at 111 8th Avenue in New York City. There appears to have been a utility power outage, followed by a problem with uninterrupted power supply system that supported the Internap facility. The outage affected about 20 customers, according to Internap.

The outage  brought down a few notable Internap customers, including Livestream, a popular streaming video platform, for most of the morning Friday. It also affected the StackExchange network of sites, popular among developers. So far, the customers have been more vocal about the outage than Internap has been.

Livestream first reported issues at 3:49 a.m. EST. The company continued to report problems both on its status page and through live tweets throughout the morning. As of 11:30 EST the company reported that power was restored and systems were being brought back online.

“We have experienced a power outage with our data center provider Internap. We are still working to restore power as quickly as possible,” Livestream said in an update. The data center engineers are trying to restore power.

StackExchange reported issues on its status page as well as its Twitter feed. The company first failed over to its Oregon data center, but incurred further problems when migrating back to the main New York data center. “We are in a holding pattern since the UPS system serving both our primary and backup power feeds (yes, we’re very excited to learn this too) is still being worked on,” Stack Exchange Network said.

After the hard power outage, the problem appeared to be with the UPS system, something that has affected Internap in the past. Utility power to the building has been restored, but Internap said engineers were still working with its electrical vendors to stabilize UPS power.

In 2009, Internap suffered an outage in Boston that resulted from poor battery maintenance. The outage caused finger pointing between Internap and VoIP provider Ooma that resulted in some bad publicity.

How Internap handles this morning’s situation will be integral in minimizing the negative impact of the outage. Livestream is a sizeable player and can raise a ruckus if the situation is not handled correctly.

Outages are never a good thing, but transparency, frequent communication and taking responsibility goes a long way in retaining customers. If the provider does not come clean about causes of an outage, chances are a customer will be more vocal about it.

In addition to 111 8th Avenue, Internap’s New York metro data centers include a facility at 75 Broad Street in New York and another one across the Hudson River at 1 Enterprise Avenue North in Secaucus, New Jersey.

UPDATE: Internap issued the following statement: On Friday, May 16 at approximately 3 a.m. ET, Internap experienced a power outage in a portion of its data center located at 111 8th Avenue in New York City. Utility power was restored at approximately 6 a.m. ET. The outage impacted approximately 20 customers using a combination of colocation and IP connectivity services. We are still working to determine the root cause of the disruption, but the site has been returned to normal N+1 redundant configuration. We have been providing customers with hourly status notifications since the beginning of the event, and our technicians are onsite 24×7 to assist them. Customers can contact noc@internap.com with questions.

About the Author

Jason Verge is an Editor/Industry Analyst on the Data Center Knowledge team with a strong background in the data center and Web hosting industries. In the past he’s covered all things Internet Infrastructure, including cloud (IaaS, PaaS and SaaS), mass market hosting, managed hosting, enterprise IT spending trends and M&A. He writes about a range of topics at DCK, with an emphasis on cloud hosting.

Add Your Comments

  • (will not be published)

2 Comments

  1. Internap also had a large number of outages in their facility at Fisher plaza several years ago (seemed to be once a year at least until the big fire - at least one was due to poor maintenance on batteries). Also another big one at their facility in Sabey near Seattle back in 2008. They are a good IP provider though(said as a current customer). In the past 15 years myself I've only been through one or two data center outages(both at Fisher plaza/Internap - I moved my company out within 6 months roughly 3 years before the big fire). Most facilities are operated properly it seems Internap is still trying to figure out how to do it.

  2. Alex

    true, as shown on these graphs. Stackexchange was quicker to recover http://www.currentlydown.com/stackexchange.com#chart-2014-05-16 http://www.currentlydown.com/livestream.com#chart-2014-05-16