Workspace

Congestion control: from Reno to BBR

3 day streakD

Congestion control: from Reno to BBR

Nobody tells a TCP sender how fast the network is. Learn how slow start and AIMD probe for capacity, why CUBIC and BBR changed the curve, and what bufferbloat and L4S are about.

CurrentLesson 7 of 722 min

A sender cannot see the path. It might cross a 10 Gbps data-centre link and then a crowded café Wi-Fi. If every sender transmitted at its own link speed, routers would overflow, drop most packets, and everyone would retransmit into a collapse. Congestion control is each sender's algorithm for finding a fair share by watching what happens to its packets.

Slow start

A new connection starts with a small congestion window, typically 10 segments, about 14 KB. Every ACK grows the window by one segment, so the window doubles every round trip. It sounds slow but is exponential: from 14 KB it reaches 1 MB in about six round trips. It ends at the first loss or when it reaches a threshold.

Worked exampleThe cold-start tax on a 1 MB response
Initial window
10 segments × 1,448 bytes ≈ 14.5 KB
Round trip 1, 2, 3, 4, 5, 6
14.5 → 29 → 58 → 116 → 232 → 463 KB per round trip
Cumulative after 6 round trips
≈ 912 KB
Total at 100 ms RTT
About 7 round trips of data, ~700 ms, plus handshakes

The same transfer on a warm connection with a large window takes about one round trip. Connection reuse beats almost any code optimisation.

AIMD: Reno's sawtooth

After slow start, classic Reno enters congestion avoidance: add one segment per round trip (additive increase), and halve the window on loss (multiplicative decrease). The window traces a sawtooth. Competing Reno flows converge to a fair share. But on a long, fast path, adding one segment per round trip takes minutes to recover from a single halving.

Figure 7.1

Three ways to probe a network

Reno averages 74% of capacity. CUBIC averages 86% of capacity. BBR averages 93% of capacity050100path capacityround trips →window (segments)
Reno · 74% of capacity usedCUBIC · 86% of capacity usedBBR · 93% of capacity used
The congestion window over 60 round trips on a path whose capacity is the dashed line. Reno grows linearly and halves on loss. CUBIC regrows along a cubic curve that flattens near the last peak. BBR estimates bandwidth and minimum RTT and paces near the capacity line, with periodic probes. Raise random loss to see which one suffers.

CUBIC and BBR

CUBIC, Linux's default since 2006, makes the window a cubic function of time since the last loss: it climbs quickly, slows as it approaches the previous peak, then probes carefully beyond it. It still treats loss as the congestion signal.

BBR takes a different view. It continually estimates the bottleneck bandwidth and the minimum round-trip time, and sends at roughly their product, pacing packets evenly. Random loss that does not reflect congestion, common on Wi-Fi and cellular links, barely affects it. Its later versions added fairness fixes after the first version was found to crowd out loss-based flows.

Bufferbloat, ECN and L4S

Loss-based algorithms keep increasing the window until a buffer overflows. If routers have large buffers, they fill completely before any loss, and every packet waits in a long queue. Latency on a busy home link can jump from 20 ms to over a second: bufferbloat. A video call stutters because a download filled the queue.

ECN lets routers mark packets instead of dropping them, signalling congestion early. L4S, standardised in 2023, pairs very shallow queues with fine-grained ECN marks and scalable congestion controllers, aiming for consistently low queueing delay. Deployment is growing in cable and mobile networks.

Go deeperMultipath TCP and HTTPS over many links

Multipath TCP lets one connection use several paths at once, such as Wi-Fi and cellular, with a separate congestion window per subflow. Apple has used it for Siri and Maps for years, and Linux has supported it upstream since 5.6. It keeps connections alive when one interface disappears.

QUIC reaches a similar goal differently: connection IDs let it migrate from one network to another without a new handshake (Module 5). Both reflect the same reality: phones change networks mid-request, and a protocol bound to one IP address and port loses work every time.

congestion.shShell
# Which algorithm does this Linux machine use, and which are available?
sysctl net.ipv4.tcp_congestion_control net.ipv4.tcp_available_congestion_control

# Simulate a bad path on an interface: 100 ms delay and 2% loss
sudo tc qdisc add dev eth0 root netem delay 100ms loss 2%

# Compare algorithms with iperf3 (server: iperf3 -s)
iperf3 -c server -C cubic -t 20
iperf3 -c server -C bbr   -t 20

Run the comparison with and without loss. The gap between CUBIC and BBR grows with random loss.

Where you will meet it

CDNs

Several large CDNs and video services run BBR on their edge servers to keep throughput high on lossy last-mile links.

Router firmware

Queue management such as fq_codel and CAKE in modern routers fights bufferbloat by keeping queues short.

Agent traffic

Token streams are small but latency-sensitive. When they share a congested link with large uploads, bufferbloat shows up as bursty, delayed tokens.

Words to know

Slow start
Doubling the congestion window every round trip until loss or a threshold.
AIMD
Additive increase, multiplicative decrease: Reno's probing pattern.
Pacing
Spreading packets evenly over time instead of sending bursts.
ECN
Explicit congestion notification: routers mark packets instead of dropping them.

Check yourself

Answer out loud before you open each one.

How does slow start grow the window?

By one segment per ACK, which doubles the window every round trip.

What signal do Reno and CUBIC use, and what does BBR use instead?

Reno and CUBIC treat loss as congestion. BBR models bottleneck bandwidth and minimum RTT.

What is bufferbloat?

Oversized buffers that fill before any loss, adding large queueing delay for every packet.

Try it

Run ping to a nearby host while a large download saturates your connection. Watch the round-trip time rise. Then test your network at a bufferbloat test site and compare the loaded and unloaded latency.

A good place to start

Explore a pattern

A little guidance

Start with a pattern, read through a problem, and mark it solved when the idea clicks. Save anything you want to revisit.

New tutorials are added one at a time. Your bookmarks and solved checkmarks stay in this browser.

Explore the roadmap