Windows, bandwidth-delay and small-write stalls
Throughput is window divided by round-trip time. Learn the bandwidth-delay product, why a fat pipe can feel slow, and the classic Nagle plus delayed-ACK stall.
TCP never sends without limit. At any moment, the sender may have at most a window of unacknowledged bytes in flight. Then it must wait for acknowledgements. Two windows apply, and the smaller wins: the receiver's flow control window, which protects the receiver's buffer, and the sender's congestion window, which protects the network.
The bandwidth-delay product
If the window is W bytes and a round trip takes RTT seconds, the sender can deliver at most W per RTT. To keep a link of bandwidth B full, the window must be at least B × RTT. That product is the bandwidth-delay product: the number of bytes that fit “in the pipe” at once.
Why a fat pipe can feel slow
- Bandwidth-delay product
- 13 MB
- Throughput
- 5.2 Mbps
- Link used
- 1%
Window-limited: the sender waits on ACKs with the link 99% idle. It needs a 13 MB window.
- Bandwidth
- 1 Gbps = 125,000,000 bytes/s
- Round trip
- 100 ms
- Bandwidth-delay product
- 125,000,000 × 0.1 = 12.5 MB must be in flight
- Actual window
- 64 KB = 65,535 bytes
- Throughput
- 65,535 ÷ 0.1 s = 655 KB/s ≈ 5.2 Mbps
About 0.5% of the link. Window scaling, auto-tuned buffers and parallel connections exist to fix exactly this.
The original TCP header has a 16-bit window field, limiting windows to 64 KB. The window scale option, negotiated in the handshake, multiplies it by up to 2^14. Modern kernels auto-tune receive buffers into the megabytes. A missing or stripped window scale option, or a small fixed SO_RCVBUF, brings the 64 KB ceiling back.
Nagle's algorithm and delayed ACKs
Two old optimisations each make sense alone and stall together. Nagle's algorithm on the sender: if there is unacknowledged small data in flight, hold new small writes until the ACK arrives, then send them together. Delayed ACK on the receiver: wait up to tens of milliseconds before acknowledging, hoping to piggyback the ACK on a reply.
- 01Client writes a small header
Sent immediately; nothing is in flight yet.
- 02Client writes the small body
Nagle holds it: the header is still unacknowledged.
- 03Server receives the header
No full request yet, so no reply to piggyback on. It delays its ACK, often 40 ms on Linux and up to 200 ms elsewhere.
- 04Delayed-ACK timer fires
Server ACKs; Nagle releases the body; the request finally completes.
The fix is to stop doing small separate writes: build the whole request and write it once. Or set TCP_NODELAY to disable Nagle on latency-sensitive connections. Node, Go and most HTTP libraries set it by default; custom protocols often forget.
import net from "node:net";
const socket = net.connect({ host, port, noDelay: true }); // disable Nagle
// Better still: one write per message, not one per field
socket.write(Buffer.concat([header, body]));noDelay sends small writes immediately. Combining writes still helps: fewer packets, fewer system calls.
Go deeperReceive buffers, zero windows and slow readers
The receiver advertises how much free buffer it has. If the application stops reading, the buffer fills and the advertised window drops to zero. The sender stops and periodically probes. This is TCP's backpressure, and it reaches all the way into your program: a Node socket.write eventually returns false, and the stream asks you to wait for drain (Module 3).
That chain is why a slow client can hold a server's memory. Each stalled connection keeps its unsent bytes in user space if the server ignores backpressure. Servers bound per-connection buffers and drop clients that stay at zero window for too long.
Where you will meet it
Copying data between continents with one TCP stream is often window-limited. Tools such as rclone and cloud SDKs open several parallel streams.
Drivers that write a query's parts separately can hit Nagle and delayed-ACK stalls, showing up as a suspicious 40 ms floor on every query.
SSH and terminal sessions set TCP_NODELAY so each keystroke is sent immediately.
Words to know
- Flow control window
- How much the receiver can buffer; it protects the receiver.
- Congestion window
- How much the sender allows in flight; it protects the network.
- Bandwidth-delay product
- Bandwidth × RTT: the bytes needed in flight to keep a link full.
- TCP_NODELAY
- A socket option that disables Nagle's algorithm.
Check yourself
Answer out loud before you open each one.
Why is a large download slow on a long link?
Throughput is capped at window ÷ RTT. If the window is smaller than the bandwidth-delay product, the link sits idle.
What is the bandwidth-delay product of 100 Mbps at 50 ms RTT?
12,500,000 bytes/s × 0.05 s = 625 KB.
Describe the Nagle and delayed-ACK stall.
The sender holds a small write until the previous one is acknowledged, while the receiver delays the ACK hoping to piggyback it. Both wait until the delayed-ACK timer fires.
Try it
On Linux, add 100 ms of delay with sudo tc qdisc add dev lo root netem delay 50ms and run iperf3 over loopback with -w 64K and then without -w. Remove the delay with tc qdisc del dev lo root.