Amazon and IBM appear to be the only major cloud infrastructure providers to have been affected by the DDoS attack of potentially unprecedented scale on DNS servers operated by Manchester, New Hampshire-based Dyn last week.
While it’s unclear exactly why the other providers, such as Microsoft, Google, and Rackspace, didn’t suffer service disruptions (the most likely reason is that they don’t use Dyn), companies whose computing and storage infrastructure underpins so many of the top internet services will have to take a hard look at their existing DDoS mitigation strategies.
Attributed at least in part to a piece of open source software that enables attackers to use poorly secured connected devices – such as some CCTV cameras and DVRs – the series of attacks on Dyn last week shows that it will be difficult to predict just how big future attacks may be and how much bandwidth headroom companies will need to maintain in their networks to fight IoT botnets.
“In the past month, we’ve seen a doubling of the largest DDoS attacks,” Lawrence Orans, research VP at Gartner, told Data Center Knowledge in an interview. “I think the infrastructure providers are going go be looking for ways to beef up their DDoS mitigation” capabilities.
The scale of DDoS attacks is typically measured in Gigabits per second, since they work by flooding target networks with enough requests to exhaust their bandwidth resources. Before last September’s attack on the website of cybersecurity journalist Brian Krebs, the size of the largest known attack had been 363 Gbps. The attack on KrebsOnSecurity.com was close to 620 Gbps, Krebs wrote. That attack, however, was quickly followed by an attack on the French hosting company OVH, which was “roughly twice the size of the assault on KrebsOnSecurity,” according to Krebs.
Dyn has not yet disclosed the size of the multi-stage attack on its infrastructure on October 21st. As of Monday afternoon, the company was still busy conducting root-cause analysis of the incident and expected to have more details to report by the middle of the week, its spokesman, Adam Coughlin, wrote in an email to DCK.
“At this point we know this was a sophisticated, highly distributed attack involving 10s of millions of IP addresses,” Kyle York, Dyn’s chief strategy officer, wrote in a statement posted on the company’s website over the weekend. Tens of millions of discrete IP addresses used in the attack were associated with Mirai, the software that automatically detects poorly secured IoT devices and enlists them into a botnet used to conduct a massive-scale DDoS attack.
AWS Cloud Hit in US and Ireland
Some Amazon Web Services customers whose infrastructure is hosted in Amazon’s Northern Virginia data centers (the Amazon cloud’s largest data center cluster) could not reach “a small number of AWS endpoints” in the early hours of the morning Eastern Time, when the first attack, directed at Dyn’s East Coast data centers, took place. The second attack, which was more global in nature, caused similar impact on AWS users hosting applications in Amazon data centers in Ireland. There was a third attack that day, according to Dyn, but the company was able to prevent it from affecting customers.
A summary of the incident is posted on the AWS Service Health Dashboard. An Amazon spokesperson pointed us to the summary in response to a request for comment. The company didn’t point to Dyn specifically, saying only that the errors resolving DNS hostnames for some AWS endpoints were caused by an “availability event” with one of its third-party DNS service providers. AWS uses several such providers, in addition to its own DNS service called Amazon Route53.
Asked why other cloud providers managed to avoid being hit last week, a spokesperson for the cybersecurity and intelligence firm Flashpoint said it was “because AWS is the only major cloud provider that heavily relies on Dyn for its infrastructure services.”
Microsoft and Google cloud status dashboards did not show any disruptions during the attacks on Dyn. A Microsoft spokesperson declined to comment, while a Google spokesperson said there had been no Google Cloud Platform disruptions in connection with the incident.
IBM PaaS Users Affected
IBM, it appears, is another provider that relies on Dyn for at least one of its cloud services. Depending on who you ask, IBM is or isn’t one of the top cloud providers. While it has an extensive cloud services business, its cloud revenue doesn’t come close to the amount of money AWS reels in each quarter, which often causes it to be excluded from discussions about top cloud providers.
Users of Bluemix, IBM’s Cloud Foundry-based Platform-as-a-Service, experienced DNS resolution issues in Australia, US, and Europe within the timeframe of the attack on Dyn, according to updates on the Bluemix System Status dashboard. One of the updates pointed to Dyn’s website for details on the incident.
The health dashboard for IBM’s Watson IoT service does not list any issues for October 21. SoftLayer, IBM’s IaaS cloud, doesn’t offer a status dashboard for non-customers. An IBM spokesperson did not respond to a request for comment.
Rackspace Sticks to DDoS Best Practices
Rackspace, another Infrastructure-as-a-Service cloud provider, also was not directly affected by the attacks. “Of course, as customers of companies that leverage Dyn, our customers and Rackers alike experienced degraded connectivity and/or weren’t able to access many prominent websites on Friday,” a Rackspace spokesperson wrote in an email.
Because there was no direct impact on its infrastructure, Rackspace is not planning to take any extra measures specifically in the incident’s aftermath, she said. The company conducts regular bandwidth assessments and upgrades to ensure it can withstand DDoS attacks.
“As a best practice for potential DDoS attacks, we leverage global redundancy, high bandwidth, and both internal and external mitigation systems to protect Rackspace’s infrastructure and our ability to provide authoritative DNS services to customers.”
How to Fight IoT Botnets?
But extra measures will most likely be needed, as connected devices continue to proliferate, offering hackers a way to build DDoS botnets of unprecedented scale. Armed with these IoT botnets, bad actors can inflict ever more damage and launch attacks at a higher rate than before, Gartner’s Orans warned. Nothing suggests that an attack on Dyn today couldn’t be followed by an attack on UltraDNS or another service provider tomorrow, he said.
Cloud companies and others who provide internet infrastructure services will have to invest in more bandwidth, DDoS mitigation equipment, and expertise to address new attack capabilities, Orans said. “That’s what they need to do to mitigate impact from these types of attacks.”