The first traffic spikes came in February. They were modest and limited to the Asia-Pacific region, Alagu Periyannan, CTO at BlueJeans by Verizon, recalled.
By March traffic started spiking 50 to 75 percent, and the growth was getting increasingly global, as more and more governments issued lockdown orders, and more people stopped going to work.
In the third week of March traffic on the BlueJeans network was reaching above 350 percent its normal levels. Luckily, Periyannan said, his team had a hybrid cloud infrastructure strategy in place, using a combination of its own data centers and public cloud.
“This hybrid approach really saved us,” he said, speaking on a panel Tuesday during the DCD New York conference, held virtually. “There was no way” the network could handle the surge, had it been hosted only in the company’s own data centers or colocation sites.
His story echoed what other operators of large global networks have said since the pandemic shifted much of people’s lives online. Technical infrastructure executives from companies including Zoom, BlueJeans’s largest competitor, and Netflix, said back in March that they were able to scale capacity on their networks without service disruption in response to the record spikes in demand thanks to their use of public cloud infrastructure.
In January and February about 80 percent of BlueJeans’ traffic was served out of its own data centers and 20 percent out of public cloud. That proportion flipped upside down starting in the second half of March, Periyannan said. Today “we’re about 70 percent in public cloud.”
This doesn’t mean that hybrid cloud is the ideal strategy for all IT shops. There are examples of large organizations that successfully use public cloud for nearly all their computing needs. (See Capital One.)
There are also examples of the opposite. Take Dropbox, which operates its own infrastructure out of colocation facilities, and which has managed to keep its network from falling over despite the massive pandemic-induced shift in traffic patterns.
But a well architected hybrid platform can save a lot of headaches for organizations that must keep a portion of their workloads under their own control while having the freedom to deploy the rest in public clouds, which have proven their ability to absorb massive spikes in demand, anticipated or otherwise.
This also doesn’t mean Periyannan’s team at BlueJeans sat back and watched as their platform automatically scaled to serve as much traffic as needed at any given moment. It was all-hands-on-deck around the clock.
Like many other IT organizations, BlueJeans’ infrastructure team had a “24/7 war room going on, with global handoff,” Periyannan said. One of the war-room’s tasks was to get on daily calls with AWS, the company’s cloud provider, almost daily to optimize cloud workload placement tactics.
BlueJeans was competing for badly needed AWS capacity with all the other AWS customers who were grappling with unprecedented spikes in demand on their networks. “We had to deal with their traffic patterns and really work closely with [AWS] to understand where they had capacity available, where they did not,” the CTO said.
Trying to contain the cost of ramping up cloud capacity, the team experimented with using spot instances on AWS. “We were using a heavy amount of on-demand instances, which are very, very expensive.”
The cloud provider offers spot instances at a 90 percent discount over on-demand VMs as a way to utilize unused capacity. But it can take them back when needed – at no more than a couple minutes’ notice.
Spot instances worked well for BlueJeans for about one week. In the second week the team started losing too many instances every day, around the end of workday in New York. (Their best guess for why this was happening was people logging in to stream video after work.) The team even tried using spot instances out of AWS’s Ohio data centers instead of the East Coast ones, in Northern Virginia, but in the end found that they couldn’t effectively rely on this type of cheap but unreliable cloud capacity.
