Workspace

The handshake, sequence numbers and retransmission

3 day streakD

The handshake, sequence numbers and retransmission

How TCP turns lossy packets into a reliable stream: numbered bytes, acknowledgements, timers and selective repair. Then the leftovers: TIME_WAIT and running out of ports.

DepthLesson 5 of 722 min

TCP numbers every byte. Each side picks a random initial sequence number, and every segment says which byte number it starts at. The receiver replies with an acknowledgement number: the next byte it expects. From those two numbers, both sides always know exactly what has been delivered.

The three-way handshake

Before data flows, the two sides agree on starting sequence numbers. The client sends SYN with its number. The server replies SYN-ACK with its own number and an acknowledgement of the client's. The client sends ACK. That costs one full round trip before the first request byte can be sent.

Figure 5.1

A connection from SYN to FIN

No packets yetClientServer

The client wants to send an HTTP request. First it must agree on sequence numbers with the server.

Step 01 / 13
Step through a short HTTP exchange. Sequence numbers count bytes; each ACK names the next byte expected. Note the round trip spent on the handshake, the lost segment repaired by retransmission, and the TIME_WAIT left behind by whoever closes first.

Detecting loss

TCP learns about loss in two ways. Timeouts: if an acknowledgement does not arrive within the retransmission timeout, computed from measured round-trip times, the sender resends. Timeouts are slow, at least 200 ms on Linux and often more. Duplicate ACKs: when segment 3 is lost but 4, 5 and 6 arrive, the receiver keeps acknowledging “I still need 3”. Three duplicate ACKs trigger fast retransmit without waiting for the timer.

Selective acknowledgement, SACK, lets the receiver say exactly which later ranges it already has, so the sender resends only the holes. Without SACK, a burst of losses can force the sender to resend data the receiver already holds.

Worked exampleWhy one lost packet delays everything after it
Sent
Segments 1–10, each 1,448 bytes
Lost
Segment 3
Arrived
1, 2, 4, 5, 6, 7, 8, 9, 10
Delivered to the app
Only 1 and 2. Segments 4–10 wait in the kernel's buffer.
After fast retransmit (about 1 RTT later)
3 arrives; 3–10 are delivered at once

This is TCP head-of-line blocking. It is invisible in a single stream, and it becomes a big deal when HTTP/2 puts many streams on one connection (Module 5).

Closing, and TIME_WAIT

Each side closes its direction with FIN. The side that closes first enters TIME_WAIT and keeps the connection's 4-tuple reserved for twice the maximum segment lifetime, 60 seconds on Linux. This stops a delayed packet from an old connection being mistaken for part of a new one with the same ports.

Worked exampleRunning out of ephemeral ports
Default Linux ephemeral range
32768–60999, so 28,232 ports
Each closed connection holds its port in TIME_WAIT for
60 seconds
Sustainable new connections to one destination IP and port
28,232 ÷ 60 ≈ 470 per second
A client making 1,000 fresh connections per second to one API
Exhausts the range in under 30 seconds

Symptom: EADDRNOTAVAIL or connect timeouts under load. Fix: reuse connections with keep-alive pools rather than tuning kernel flags.

Go deeperSYN floods, SYN cookies and TCP Fast Open

A server must remember each half-open connection between SYN and the final ACK. An attacker sending millions of SYNs from spoofed addresses can fill that queue. SYN cookies let the server encode the needed state into its own initial sequence number and keep nothing until the final ACK proves the client is real.

TCP Fast Open lets a returning client put data in its SYN, saving a round trip. It saw limited deployment because middleboxes dropped or mangled SYNs carrying data, an example of ossification: a protocol becoming impossible to change because devices in the path assume its current shape. QUIC's encrypted headers were designed to avoid that fate.

inspect-tcp.shShell
# Connection states on this machine, counted
ss -tan | awk 'NR>1 {print $1}' | sort | uniq -c

# Per-connection internals: RTT, congestion window, retransmissions
ss -ti dst api.example.com

# Watch a handshake and close on the wire
sudo tcpdump -ni any 'host example.com and tcp port 443' -c 20

ss -ti shows rtt, cwnd and retrans per connection, which turns guesses about a slow connection into numbers.

Where you will meet it

Connection pools

HTTP clients such as undici keep connections alive and reuse them, avoiding both handshake latency and TIME_WAIT buildup.

Load testers

Naive load tests that open a new connection per request often measure their own port exhaustion instead of the server.

Packet capture

Wireshark marks retransmissions and duplicate ACKs in colour, which is often the fastest way to prove network loss.

Words to know

SYN / ACK / FIN
TCP flags: synchronise sequence numbers, acknowledge data, and finish a direction.
Retransmission timeout
How long the sender waits for an ACK before resending, derived from measured RTT.
SACK
Selective acknowledgement: reporting which later byte ranges have already arrived.
TIME_WAIT
The state that keeps a closed connection's 4-tuple reserved for a while.

Check yourself

Answer out loud before you open each one.

What does an ACK number mean?

The next byte the receiver expects. Everything before it has arrived.

Why does fast retransmit need three duplicate ACKs?

Duplicate ACKs can also come from reordering. Three is a heuristic that loss, not reordering, is likely.

Why does TIME_WAIT exist, and when does it hurt?

To stop old delayed packets confusing a new connection on the same 4-tuple. It hurts clients that open many short connections to one destination.

Try it

Capture curl -s https://example.com > /dev/null with tcpdump and open it in Wireshark. Find the SYN, SYN-ACK, ACK, the TLS handshake and the FINs. Note the relative sequence numbers.

A good place to start

Explore a pattern

A little guidance

Start with a pattern, read through a problem, and mark it solved when the idea clicks. Save anything you want to revisit.

New tutorials are added one at a time. Your bookmarks and solved checkmarks stay in this browser.

Explore the roadmap