GitHub has revealed it was hit with what may be the largest-ever distributed denial of service (DDoS) attack.
The first portion of the attack against the developer platform peaked at 1.35Tbps, and there was a second 400Gbps spike later. This would make it the biggest DDoS attack recorded so far. Until now, the biggest clocked in at around 1.1Tbps.
In a post on its engineering blog, the developer platform said that, on Feb. 28, GitHub.com was unavailable from 17:21 to 17:26 UTC and intermittently unavailable from 17:26 to 17:30 UTC due to the DDoS attack.
Github said that at no point "was the confidentiality or integrity of your data at risk."
"Between 17:21 and 17:30 UTC on February 28th we identified and mitigated a significant volumetric DDoS attack. The attack originated from over a thousand different autonomous systems across tens of thousands of unique endpoints. It was an amplification attack using the memcached-based approach described above that peaked at 1.35Tbps via 126.9 million packets per second," GitHub said.
The DDoS attack featured an unusual way of amplifying its power, relying on UDP-based memcached traffic.
Memcached is a tool meant to cache data and reduce strain on heavier data stores, like disk or databases. It is only intended to be used on systems that are not exposed to the internet, as there is no authentication required. However, there are currently more than 50,000 known vulnerable systems, according to Akamai.
When a system receives a memcached get request, it forms a response by collecting the requested values from memory and then sending them back in an uninterrupted stream.
However, DDoS attackers have now realised they can use this protocol to launch attacks. First, the attacker implants a large payload on an exposed memcached server. Then, the attacker spoofs the "get" request message with a target's source IP, and thus, a short request to the exposed server can result in a pile of traffic arriving at the victim.
"15 bytes of request can trigger a 134KB of response sent to the unfortunate target. This is amplification factor of 10,000x! In practice we've seen a 15 byte request result in a 750kB response (that's a 51,200x amplification)," said Cloudflare in a post explaining the potential threat.
GitHub explained how such an attack could generate vast amounts of traffic: "Spoofing of IP addresses allows memcached's responses to be targeted against another address, like ones used to serve GitHub.com, and send more data toward the target than needs to be sent by the unspoofed source. The vulnerability via misconfiguration described in the post is somewhat unique amongst that class of attacks because the amplification factor is up to 51,000, meaning that for each byte sent by the attacker, up to 51KB is sent toward the target," it said.
GitHub said that, because of the scale of the attack, it decided to move traffic to Akamai, which could help provide additional edge network capacity. It said it is now investigating the use of its monitoring infrastructure to automate enabling DDoS mitigation providers and will continue to measure its response times to incidents like this -- with a goal of reducing mean time to recovery.
Akamai said: "Because of its ability to create such massive attacks, it is likely that attackers will adopt memcached reflection as a favorite tool rapidly. Additionally, as lists of usable reflectors are compiled by attackers, this attack method's impact has the potential to grow significantly."