The 50,000x Problem
In February 2018, GitHub was hit by the largest DDoS attack recorded at the time: 1.35 Tbps of inbound traffic generated by memcached amplification. The attack exploited a simple but devastating property of the memcached protocol: a tiny request can produce an enormous response. With amplification factors reaching 50,000x, an attacker sending 30 Mbps of spoofed UDP requests can generate 1.5 Tbps of attack traffic directed at a single target.
The attack vector is straightforward. Memcached instances listening on UDP port 11211 respond to get and stats commands without any authentication. Attackers send small requests with spoofed source IPs (set to the victim's address), and the memcached servers dutifully send their large cached responses to the victim. Because memcached can store values up to 1 MB per key, a single 15-byte request can trigger a response hundreds of thousands of times larger.
What the Traffic Looks Like
Memcached amplification has a distinctive packet-level signature that separates it from other UDP floods. Understanding these characteristics is critical for building effective detection rules.
Source Port 11211
All amplified responses originate from UDP port 11211 on the reflector. Unlike DNS amplification (source port 53) or NTP amplification (source port 123), memcached responses come from a port that should almost never appear in legitimate inbound traffic for most organizations. This makes initial detection relatively straightforward if your monitoring is packet-aware.
$ tcpdump -nn 'udp src port 11211' 14:32:01.442 IP 203.0.113.45.11211 > 198.51.100.10.80: UDP, length 1400 14:32:01.442 IP 198.51.100.78.11211 > 198.51.100.10.80: UDP, length 1400 14:32:01.443 IP 192.0.2.33.11211 > 198.51.100.10.80: UDP, length 1400 14:32:01.443 IP 203.0.113.112.11211 > 198.51.100.10.80: UDP, length 1400
Fragment Floods
Because memcached responses often exceed the typical 1500-byte MTU, they arrive fragmented. A single logical response might be split into dozens of IP fragments. Your PPS counter will spike dramatically, but the initial fragment is the only one carrying the UDP header with port 11211. Subsequent fragments lack transport-layer headers entirely, which means port-based firewall rules will miss them unless you also filter on IP fragment offset.
Flowtriq tracks fragment reassembly rates as a separate metric. A sudden spike in IP fragments with no corresponding increase in complete UDP sessions is a strong signal of amplification traffic.
Multiple Source IPs, Single Destination
Unlike a traditional botnet flood where thousands of compromised hosts send traffic directly, amplification attacks use reflectors. The traffic comes from legitimate memcached servers worldwide. This means source IPs belong to real hosting providers, cloud instances, and sometimes enterprise networks. IP reputation lists alone will not save you because these are not malicious hosts; they are misconfigured ones being abused.
How Flowtriq Detects It
Flowtriq uses a layered detection approach that catches memcached amplification within the first second of an attack, well before it can saturate your link.
Layer 1: PPS and BPS Baselines
Flowtriq's agent samples /proc/net/dev counters every second and maintains dynamic baselines using exponentially weighted moving averages. When inbound PPS or BPS exceeds the learned p99 threshold, an anomaly event is generated. For memcached amplification, the BPS spike is typically far more dramatic than the PPS spike due to the large packet sizes, which is itself a useful signal.
Layer 2: Protocol Distribution Analysis
Flowtriq continuously tracks the protocol distribution of inbound traffic. Under normal conditions, most servers see a predictable mix of TCP and UDP. A sudden shift where UDP jumps from its normal 5% to 95% of inbound bytes triggers a protocol anomaly alert. This detection fires independently of absolute thresholds, so it catches attacks even when the total bandwidth is still below your baseline.
Layer 3: IOC Pattern Matching
Flowtriq maintains a library of attack-specific indicators of compromise. For memcached amplification, the agent looks for:
- Inbound UDP packets from source port 11211
- High IP fragmentation rates on inbound traffic
- Memcached response headers (
VALUE,STAT) in packet payloads - Multiple source IPs converging on a single destination port
When two or more of these indicators fire simultaneously, Flowtriq classifies the event as a memcached amplification attack with high confidence and includes the classification in all alert notifications.
Layer 4: PCAP Evidence
Flowtriq maintains a rolling 500-packet pre-attack buffer. When detection fires, the agent captures a PCAP snapshot that includes packets from before the detection threshold was crossed. This gives you forensic evidence to share with your upstream provider or ISP when requesting mitigation, without requiring you to have been running a manual capture at the time of the attack.
Detection time in practice: In our internal testing with replayed memcached amplification captures, Flowtriq fires the initial anomaly alert in 800ms and delivers the classified attack notification with PCAP link within 2.4 seconds of the first attack packet arriving.
Immediate Mitigation Steps
Once Flowtriq alerts you to a memcached amplification attack, the following steps can help contain the damage while you wait for upstream mitigation:
- Rate-limit UDP port 11211 at the host firewall. Since legitimate memcached traffic is almost never inbound from the internet, you can safely drop it:
iptables -A INPUT -p udp --sport 11211 -j DROP - Request a null-route or RTBH from your upstream provider. Share the PCAP evidence Flowtriq captured to expedite the process. Most providers have automated systems that respond to BGP blackhole requests within minutes.
- Enable fragment filtering. Drop all IP fragments for protocols where you do not expect fragmentation. This eliminates the bulk of amplified traffic that bypasses port-based filters.
- Engage your DDoS mitigation provider. If you use a scrubbing service, Flowtriq's webhook integration can automatically trigger traffic diversion via API.
Long-Term Prevention
If you operate memcached servers, ensure they are not exposed to the internet on UDP. Memcached should be bound to localhost or a private network interface. If UDP support is not needed, disable it entirely with -U 0 in the memcached startup flags. The memcached project disabled UDP by default starting in version 1.5.6, but older installations and Docker images may still have it enabled.
For defenders, memcached amplification is a reminder that volumetric attacks can arrive with virtually no warning. The time between "everything is normal" and "our link is saturated" can be measured in seconds. That is why Flowtriq samples every second rather than using 60-second polling intervals. When an attack can fill a 10 Gbps link in under 5 seconds, polling every minute means you are 55 seconds too late.
Flowtriq plans start at $9.99/mo per node and include memcached amplification detection out of the box. Start your free trial and see how fast detection can be.
Back to Blog