Workspace

The cold-connection tax and connection reuse

3 day streakD

The cold-connection tax and connection reuse

Add up every round trip before the first byte of a response, and you will see why connection reuse matters more than most code optimisation.

DepthLesson 3 of 615 min

A brand-new HTTPS request pays for several things before the server even sees it: a DNS lookup, a TCP handshake, a TLS handshake, then the request itself. Each costs at least one round trip, and round trips are set by distance, not by your code.

Figure 3.1

Time to first byte, cold and warm

  1. TCP + TLS 1.2DNSTCPTLSRequest750 ms
  2. TCP + TLS 1.3DNSTCPTLSRequest600 ms
  3. QUIC (HTTP/3)DNSQUIC + TLSRequest450 ms
  4. QUIC 0-RTT resumeDNSRequest with handshake300 ms
  5. Reused connectionRequest150 ms

A reused connection reaches the first byte 450 ms sooner than a cold TLS 1.3 connection at this distance. Server time is extra in every row.

Choose the round-trip time and the connection type. Each bar is one phase. A reused connection skips straight to the request. QUIC folds transport and TLS into one round trip, and 0-RTT resumption sends the request immediately, with replay risk.
Worked exampleThe 150 ms cold-connection tax
DNS (cold)
~1 RTT to the resolver, often less if cached
TCP handshake
1 RTT = 150 ms
TLS 1.3 handshake
1 RTT = 150 ms
Request and first response byte
1 RTT = 150 ms, plus server time

450 ms before any server work, ignoring DNS. On a warm connection, the same request costs 150 ms.

Connection pools

HTTP clients keep a pool of open connections per origin and hand them out to requests. A good pool has a maximum size, an idle timeout shorter than the server's (so you do not reuse a connection the server is about to close), and health checks. In Node, fetch is built on undici, which pools by default; older http.Agent code needs keepAlive: true.

pool.tsTypeScript
import { Agent, setGlobalDispatcher } from "undici";

// One pool for the whole process, tuned for a model API
setGlobalDispatcher(new Agent({
  connections: 64,             // max sockets per origin
  keepAliveTimeout: 30_000,    // close idle sockets before the server does
  keepAliveMaxTimeout: 600_000,
  connect: { timeout: 10_000 },
}));

// Every fetch() now reuses warm connections when one is free
const response = await fetch("https://api.example.com/v1/messages", { method: "POST", body });

Creating a new client or agent per request silently throws the pool away. Create it once and share it.

Warming and prefetching

Browsers can open connections before they are needed with <link rel="preconnect">. Servers can do the same at startup: make one cheap request to each upstream so the first user request finds a warm, already-slow-started connection. For an agent harness that calls the same model API all day, the pool is warm after the first call; what matters is not throwing it away.

Go deeperWhen reuse goes wrong: stale connections

A pooled connection may have been closed by the server, a load balancer or a NAT while it sat idle. The client only finds out when it writes: the request fails with ECONNRESET or socket hang up.

Good clients retry such failures once on a fresh connection, but only for requests that are safe to repeat, because the server may have received the first attempt. Keep client idle timeouts below the server's and the balancer's to make this rare.

Where you will meet it

Serverless cold starts

A function instance's first request pays for its own connections. Platforms that reuse instances, and code that creates clients outside the handler, keep connections warm.

Database proxies

Connection poolers such as PgBouncer exist because opening a Postgres connection costs handshakes plus a process fork.

Browser preconnect

Sites preconnect to their API and font origins in the HTML head, saving hundreds of milliseconds on mobile.

Words to know

Time to first byte
Time from starting a request until the first response byte arrives.
Connection pool
A set of open connections reused across requests.
Preconnect
Opening a connection before the request that needs it.

Check yourself

Answer out loud before you open each one.

How many round trips does a cold HTTPS/TCP request cost before the first response byte, ignoring DNS?

Three with TLS 1.3: TCP, TLS and the request.

Why set the client's keep-alive timeout below the server's?

So the client never reuses a connection the server has just closed.

What does a new HTTP client per request cost?

A new connection each time: handshakes, slow start and possibly TIME_WAIT buildup.

Try it

Use curl -w '%{time_namelookup} %{time_connect} %{time_appconnect} %{time_starttransfer}\n' -o /dev/null -s https://example.com twice in one command line with two URLs, and compare the first and second request.

A good place to start

Explore a pattern

A little guidance

Start with a pattern, read through a problem, and mark it solved when the idea clicks. Save anything you want to revisit.

New tutorials are added one at a time. Your bookmarks and solved checkmarks stay in this browser.

Explore the roadmap