Last week, eager Christmas celebrators across the world hooked up their brand new Xboxes and PlayStations only to find that both online networks were down, leaving countless new games totally unplayable.
It was a particularly large piece of coal in gamers' stockings, and while Microsoft's service recovered after 24 hours, the PlayStation Network suffered a prolonged outage that lasted two full days. Even today we're still feeling the effects; almost a week later, PSN's service remains intermittent thanks to what Sony said this weekend was a Distributed Denial of Service (DDoS) attack—an intentional flood of traffic designed to cripple a network's servers.
A group called Lizard Squad has claimed responsibility for the attack, though the "who" doesn't seem as important as the how: Just how did this happen? What knocked the PlayStation Network offline for so long? Why was Sony so unprepared? How can they prevent this from happening again?
Though Sony has not responded to repeated requests for comment over the past week, I spoke to two cybersecurity analysts in an attempt to at least make an educated guess as to how this could have happened. And although it's hard to parse the specifics of this particular attack without hearing directly from Sony, those analysts made one thing very clear: Today, it's harder than ever to stop these sort of attacks.
"There are a few reasons why DDoS attacks are bigger, more frequent, and in some cases more difficult to stop than they were in the past," said Dan Shugrue, director of product marketing at the cloud service provider Akamai Technologies. Though Shugrue couldn't comment on the specifics of the PlayStation Network attack, as Sony is an Akamai client, he said there are increasingly powerful tools that anyone can download and use to trigger denials of service.
For example: the High Orbit Ion Cannon (HOIC), a free piece of software that allows anyone to flood a website with overwhelming amounts of dummy traffic created by custom scripts. Anyone with a computer can download this program, type in the URL of a website, and watch the HOIC generate fake user after fake user in hopes of overloading that site's servers and bringing it down. And when multiple people use the HOIC at once on the same target, the damage can grow exponentially higher.
Taking on a multi-billion-dollar corporation like Sony requires more sophisticated methods, though. David Larson, CTO of the cybersecurity firm Corero Network Security, said he suspects that this PSN attack was the result of some sort of combination of DDoS tools that may have included botnets—collections of computer servers designed to connect and perform a unified action. Anyone can rent a botnet, Larson said—and combining botnets and Ion Cannon-like flooding programs can cause a lot of devastation across the web.
Just picture it: a thousand computers all using the same DDoS tools to generate countless fake accounts, all flooding the same website or server with thousands of gigabytes of data per second. "It's tremendously easy," Larson told me on the phone this afternoon. "Anybody can afford it; anybody can do it."
Larson also pointed out that if the attackers specifically went after the PlayStation Network's login servers—the ones that recognize and accept usernames and passwords—rather than the servers that host PSN's content, it might have been easier to overload Sony's network with fake requests. To do this, the attackers may have manipulated outside Domain Name System (DNS) servers—the servers that translate domain names into IP addresses—into barraging the PlayStation Network with data. The internet is full of these servers, many of which are easy to target and use for malfeasance.
"It's tremendously easy. Anybody can afford it; anybody can do it."
"You can download freeware tools that are basically a database of known vulnerable DNS servers on the internet," Larson said. "I can send a request, a very small packet request to a vulnerable DNS server. I can say, 'Hello vulnerable DNS server, I am the PlayStation login server—please send me a record.' And that record may be several kilobytes long. So with one smaller 64-byte packet, I can request several tens of thousands of bytes of information, and that server will respond as if I was the Sony site, and it will send that packet at Sony."
Imagine this happening hundreds or thousands or millions of times in a second and you can immediately see the problem here: even the most sophisticated servers can only deal with so much traffic at a time. Overloading a computer with requests can slow it down or even cripple it entirely, and it can be very difficult for a machine to parse the real traffic from the fake, especially when attackers are using some combination of botnets and fake IPs.
"For someone who knows how to get those tools and use them, you can hide on a small bandwidth connection and create tens of gigabytes of traffic," Larson said.
But Sony has been down this road before. In fact, as Larson points out, networks like Xbox Live and PSN are likely hit by attempted DDoS attacks every few months at the very least. So how did they not thwart this one?
"Without the specific details, and not knowing specifically how this attack was done, since we're not at Sony, it seems to me that you could do this with a lot of requests that actually don't occupy a lot of bandwidth if what you're attacking is the login servers," Larson said. "If all you're trying to do is prevent access to the login server, that kind of a starvation or occupation attack in either a spoofed way or botnet-driven or any other way is a relatively low-bandwidth thing that can have a massive outage across a large-scale network like the PlayStation Network or Xbox Live.
"There's no size network that cannot be overrun—you can basically overrun any size network you want."
Sony is no stranger to high-profile security breaches, most recently in the wake of this attack and the Sony Pictures leak in early December, but spanning as far back as 2011, when the PlayStation Network suffered a major hack that exposed user data for millions of its customers. So it almost seems hard to believe that they weren't prepared for what happened last week. (Of course, it's worth noting, a DDoS attack is not a security breach—it's simply a flood of traffic.)
But as Larson explains, these attacks have grown increasingly sophisticated over the past few months, to the point where security methods installed as recently as 2012 or 2013 might no longer be effective.
"One of the historical ways that people have handled these attacks is they notice that the spike is occurring against them and they're seeing an outage, and then they work with their providers to do something called 'blackholing,'" Larson said. "Meaning as they're sending [fake traffic] you just wanna throw it into the ground, you wanna throw it away.
"The tools are much more dynamic now. Humans and human processes are not fast enough to keep up with the change."
"A year and a half ago, DDoS attacks were the kind of things that happened against you—you noticed there was a DDoS attack, and then you blackholed and moved on. The tools are much more dynamic now. Humans and human processes are not fast enough to keep up with the change—you need a machine system that's basically looking for and identifying the attacks in real time, then taking action against them immediately."
OK. So how do companies like Sony stop attacks like this today, as we enter 2015?
"The best defense is to have plan," said Shugrue. "Companies need runbooks for DDoS attacks, they need to practice DoS drills, and of course they need to investigate DDoS mitigation provider options before the attack takes place. If there is no defense in place it is very difficult to restore functionality to servers until such time as the attacker decide to let up the pressure."
Larson suggests that anyone who runs a network use several safeguards both to identify unusual traffic patterns and to mitigate the damage. He also recommends that companies use cloud services that can offload excessive traffic while DDoS attacks are happening, therefore preventing those companies' networks from having to deal with the overload.
"We recommend people take the steps of protecting on their premises," Larson said. "Meaning any DDoS traffic that makes it into their data center, they should be able to deal with it on the very edge of their network immediately. They should be able to utilize a cloud solution so in the event of these large-scale attacks, these massive supersaturation events, you need to find a place that has the bandwidth to absorb that, get rid of it, and pass good traffic onto you."
It's unclear just what solutions Sony was using, and it's unlikely we'll hear clarification from them about what kind of attack this was and what safeguards they had in place, but it's become clear that cyberwarfare is just getting more powerful. DDoS attacks are growing more sophisticated with every passing month, and one security firm recently estimated that there are some 28 of these large-scale attacks per hour. The future is scary.
"This is a class of security threat to the Internet generally that needs to be taken seriously with specific proactive defenses because it's just growing," Larson said. "You see DDoS in the news associated with everything, not just PlayStation and Xbox—nation states, other forms of mischief against any number of brands. What we're seeing is DDoS being used more and more as a tool for any kind of exploit activity."
Top image via this rad visualization of DDoS attacks.
You can reach the author of this post at email@example.com or on Twitter at @jasonschreier.