Flowtriq at scale: what we learned
monitoring 1M+ endpoints

Back to Blog

The Dataset

Running Flowtriq across a large and diverse node population creates a dataset that is genuinely unusual in the DDoS monitoring space: per-second traffic telemetry from thousands of distinct infrastructure configurations, spanning residential-adjacent VPS nodes, bare-metal dedicated servers, cloud-hosted workloads across AWS/GCP/Hetzner/OVH, and everything in between. When you accumulate enough of this data, patterns emerge that you simply cannot see at smaller scale — and several of them challenged assumptions we had built into the detection engine.

This post shares what we found and what it changed. It is not a marketing piece — we are sharing the data because it is genuinely useful for infrastructure operators thinking about their own exposure.

Attack Patterns by Infrastructure Type

VPS Nodes (shared-host, budget providers)

VPS nodes on budget providers — Vultr, DigitalOcean, Linode, Hetzner Cloud — show the highest attack frequency by a significant margin. The attack profile is predominantly UDP floods (61% of incidents) and SYN floods (28%), with application-layer attacks making up the remaining 11%. The high UDP flood rate is consistent with these nodes frequently running game servers and small game hosting operations, which attract the most automated botnet fire.

Critically, VPS nodes also show the shortest attack durations — median 3.1 minutes — because attackers targeting game servers are usually testing whether a host is protected before moving on. A node that absorbs 3 minutes of flood without going down is often abandoned in favor of more vulnerable targets. This "scanning" behavior means per-second detection is particularly valuable here; a 60-second polling interval would miss a significant percentage of these events entirely.

Dedicated/Bare-Metal Servers

Dedicated servers show a markedly different profile. Attack frequency is lower, but attack duration and intensity are substantially higher. The median attack duration on dedicated hardware is 8.4 minutes, and the mean is 41 minutes — roughly double the VPS figures. Amplification attacks (DNS, NTP, Memcached reflection) appear far more frequently on dedicated hardware, accounting for 34% of incidents versus 9% on VPS.

The likely explanation is targeting specificity. Dedicated server operators tend to run infrastructure with known external dependencies — public game server IPs, hosting company nodes, financial API endpoints — that attract more deliberate attack campaigns rather than spray-and-pray botnet traffic.

Cloud-Hosted Workloads

Cloud nodes (EC2, GCE, Azure VMs) show the lowest raw attack rate but the highest proportion of application-layer attacks — 38% of incidents, versus 11% on VPS and 19% on dedicated. This is consistent with cloud nodes predominantly running web applications and APIs rather than game servers or raw UDP services. HTTP floods and Slowloris-style connection exhaustion attacks are the dominant vectors.

Cloud nodes also show a distinctive false-positive pattern: cloud provider internal traffic — health checks, load balancer probes, autoscaling events — can trigger baseline deviation alerts on newly provisioned instances. This drove a specific change to ftagent's baseline initialization window, discussed below.

Attack Types by Industry Vertical

Segmenting by the self-reported industry of the workspace reveals strong specialization in attacker tooling:

• Game hosting: 74% UDP floods, predominantly targeting UDP 27015-27030 (Steam/Source engine), 7777 (Unreal/Ark), and 19132 (Minecraft Bedrock). Packet sizes cluster tightly at 1400-1480 bytes — maximum UDP payload before fragmentation — indicating purpose-built amplification avoidance.
• E-commerce: 61% HTTP floods. The majority target /, /search, and /cart endpoints with high-frequency GET requests from distributed IPs. Notably, 23% of HTTP flood attacks in this segment use valid TLS sessions, bypassing many simple TCP-layer filters.
• Financial services / payment processing: 58% SYN floods, significantly higher than any other segment. The hypothesis is that SYN floods against payment processors are often competitive disruption attempts rather than pure extortion, given the targeted precision of the source IP patterns.
• Managed hosting / ISPs: The most diverse mix — roughly even split across volumetric, protocol, and application attacks. These nodes are attacked across all their customer IPs simultaneously during campaigns, making the aggregate PPS figures by far the highest in the dataset.

Time-of-Day Attack Distribution

Attack frequency follows a pronounced diurnal pattern with two distinct peaks. The primary peak is 2:00-4:00pm EST, which correlates with peak afternoon internet usage in North America and the school dismissal window — a well-documented correlation given the demographics of booter service customers. The secondary peak is 1:00-3:00am EST, corresponding to peak evening hours in Asia-Pacific time zones and late-night North American gaming activity.

The troughs are 5:00-8:00am EST (early North American morning) and Sunday mornings. Attack frequency on Sunday 6am EST is approximately 34% of peak Thursday afternoon frequency.

Attack Duration: The Median/Mean Gap

Attack duration statistics are heavily influenced by a small number of long-running campaigns. The overall median attack duration in the dataset is 4 minutes. The mean is 23 minutes. The gap is explained by a distribution with a heavy right tail: roughly 8% of attacks run longer than 60 minutes, and 2% run longer than 6 hours. These long-duration attacks pull the mean far above the median despite representing a minority of incidents.

For infrastructure planning, this means you should design for the median case (attack stops in under 5 minutes without intervention) but maintain mitigation posture for the long-tail case. An attack that has run for 30 minutes is much more likely to continue for another 30 minutes than one that has run for 5 minutes. Duration is predictive: if you have not seen resolution by the 20-minute mark, plan for a multi-hour event.

Detection Accuracy Tuning

What Causes False Positives

Before the analysis, we assumed false positives would cluster around legitimate traffic spikes — product launches, viral content, flash sales. In practice, the leading causes were:

1. Backup jobs: Large rsync or database backup operations over the network can cause PPS spikes that look identical to low-volume flood patterns. These are the most common false positive source and were solved by making the baseline window asymmetric — rapid increases that track a sustained ramp rather than a step function are weighted differently.
2. Cloud provider health checks: As mentioned above, newly provisioned cloud instances receive heavy health check traffic from load balancers and autoscalers during the first 90 seconds. We now suppress alerting during the first 120 seconds of agent operation on a new node.
3. NTP synchronization: NTP traffic bursts from kernel clock discipline can trigger UDP anomaly detection on nodes with very low baseline UDP traffic. Port 123 is now excluded from UDP flood scoring unless it is the dominant UDP port — which itself is a reliable NTP amplification attack indicator.
4. Container churn: Nodes running Docker or Kubernetes with high container creation/destruction rates generate ephemeral connection patterns that can look like connection floods. This drove improvements to the connection rate baseline windowing.

The ftagent CPU Budget

Keeping ftagent's CPU consumption under 0.1% of a single core was a hard design constraint. The approach is to read from kernel counters rather than doing any packet inspection. /proc/net/dev, /proc/net/snmp, and /proc/net/netstat are read once per second — these are in-memory kernel structures, not disk I/O, and the read cost is effectively zero. The computational work is entirely in the sliding window arithmetic for baseline calculations, which is O(1) per metric per tick regardless of traffic volume.

# Verify ftagent's resource usage on your node:
top -b -n1 -p $(pgrep ftagent) | tail -n1

# Or check the last 60 seconds via pidstat:
pidstat -p $(pgrep ftagent) 1 60 | awk '{print $7, $8}'
# %usr and %system should both be well under 0.1

What Changed After the Analysis

The scale analysis drove five concrete changes to the detection engine:

• Industry-aware baseline defaults: New nodes are now onboarded with a baseline profile that matches their reported use case. A game server node starts with a higher UDP baseline than a web server node, reducing the false-positive rate during the learning period.
• Duration-aware alert severity: Alerts that fire at the 5-minute mark are escalated to a higher severity level than initial detection alerts, because duration is predictive of total impact.
• Asymmetric baseline windows: Baselines adapt faster to sustained decreases than to sustained increases. This prevents a slow-ramp attack from gradually shifting the baseline upward and hiding itself.
• Port 53 and 123 special handling: DNS and NTP traffic are scored separately from general UDP flood detection, with amplification-specific signatures (source port matching, packet size distribution) rather than raw PPS thresholds.
• Multi-vector co-detection: When two distinct attack signatures trigger within 90 seconds of each other on the same node, the incident is classified as multi-vector regardless of individual severity. This catches campaigns that use a low-rate secondary vector to probe defenses.

Back to Blog

Related Articles