On Monday, several services on the internet ground to a halt because of an outage at some Amazon Web Services cloud servers. The outage affected Netflix, Disney Plus, PUBG, League Of Legends, Ring security cameras, as well as Amazon products and delivery infrastructure. People were not able to see pictures of their favorite McDonald’s coffee, nor use their Roomba vacuum cleaners. On Reddit, users reported they were not able to charge their electric vehicles. Even here at Motherboard, we were briefly unable to post new stories, or share them on social media because the outage impacted some of the tools we use.
The outage lasted just a few hours, but it showed the world just how much the internet depends on Amazon’s infrastructure.
Steven Bellovin, a computer science professor at Columbia University, said that one of the issues with the internet’s dependency on AWS is that there is now a single point of failure for thousands of websites.
“If an attacker can gain control of AWS infrastructure, they could do very great damage. It's likely that that's much harder than penetrating individual companies, because AWS is very, very good at running a secure shop, but of course it's not impossible,” Bellovin told Motherboard in an email.
“This is yet another glimpse of how interconnected our services have become, with the immense complexity of cloud deployments impacting large numbers of enterprises and consumers,” Ed Skoudis, the president of the SANS Technology Institute, which focuses on cybersecurity, told Motherboard in an email.
Rob Graham, a cybersecurity expert who’s known for creating tools to scan the whole internet, said that when you depend on a cloud provider like AWS you may set up your website or service to keep running even when the provider is down, which may impact some security services.
Graham mentioned the example of Parler. Earlier this year, AWS and Twilion pulled the plug on the right-wing social media site, which caused the site’s SMS verification system to fail, allowing hackers to bypass two-factor authentication and get into users’ accounts.
“There are real but imponderable risks to one company controlling so much of the net.”
“Just like how people were surprised about their dependencies on AWS when it went down, people would be surprised about how much their security was dependent on AWS. However, they are less likely to notice that their security now has a hole. I mean, if it's an unexpected dependency on availability, people notice immediately. If it's an unexpected vulnerability on security, they won't notice—unless the hackers notice it :),” Graham said in an online chat. “You aren't going to have a big breakdown in obvious security thingies, but a subtle vulnerability in unexpected ways.”
On Tuesday, Amazon said that it was “seeing impact to multiple AWS APIs in the US-EAST-1 Region. This issue is also affecting some of our monitoring and incident response tooling, which is delaying our ability to provide updates.”
“The root cause of this issue is an impairment of several network devices in the US-EAST-1 Region,” the company added on its official status page.
The other risk, Bellovin added, is that AWS is a monoculture, and if a hacker figures out a way to phish AWS customers, they can use that technique with every one of them. Still, AWS is a great choice for most companies.
“For small and medium companies, I generally recommend cloud services, because they don't have the people or the skills to run their own systems as securely,” Bellovin said. “But there are real but imponderable risks to one company controlling so much of the net.”
AWS is a single point of failure, but the fact that it doesn’t fail too often has lulled many into a sense of security where they should prepare better.
“Single points of failure are all over, and those affected could have engineered with failover if it really is critical for them to keep working,” Nicholas Weaver, a senior researcher at the International Computer Science Institute at UC Berkeley, told Motherboard in an online chat. “If anything AWS is too reliable, it's reliable enough that people don’t bother doing the engineering for when it fails, so when it does fail it is a shock.”