Workspace

Resumable streams and stateless agent protocols

3 day streakD

Resumable streams and stateless agent protocols

Make the stream a view of a durable log, and a dropped connection becomes a reconnect. Then see why the latest MCP revision went the other way: stateless requests, handles and polling.

CurrentLesson 6 of 618 min

A stream that lives only in a connection dies with it. The fix is to make the connection a view of something durable. The server appends every event to an append-only log keyed by run ID, with an increasing offset. A client streams from an offset; when it drops, it reconnects with the last offset it saw and receives everything after it.

Figure 6.1

Drop the connection at any point

Server log · 0 of 24 events produced
123456789101112131415161718192021222324
Client · connected
123456789101112131415161718192021222324
Received liveCaught up on reconnectLost for good

Run the model, drop the connection part-way, then reconnect.

The run writes events to a log as the model produces them. Disconnect the client, let the run continue, then reconnect. With a log, the client resumes from its last offset and loses nothing. Without one, everything produced during the gap is gone.
resumable.tsTypeScript
// Server: the model loop writes to the log, never directly to a socket.
async function runAgent(runId: string) {
  for await (const event of modelStream()) await log.append(runId, event);   // offset assigned by the log
  await log.append(runId, { type: "done" });
}

// Server: any connection reads from the log, starting where the client left off.
app.get("/runs/:id/events", async (req, res) => {
  const from = Number(req.header("last-event-id") ?? -1) + 1;
  res.writeHead(200, { "content-type": "text/event-stream", "cache-control": "no-cache" });
  for await (const { offset, event } of log.read(req.params.id, from, { follow: true, signal: req.signal })) {
    res.write(`id: ${offset}\ndata: ${JSON.stringify(event)}\n\n`);
  }
  res.end();
});

The model loop and the HTTP connection are now independent. A refresh, a network change or a second device all just read the same log.

What this buys, and what it costs

  • Resume after any drop, including a phone changing networks or a page refresh.
  • Multiple viewers of one run, such as a second device or a teammate.
  • Replay and audit: the log is the record of what the agent did (Module 10 builds durable execution on the same idea).
  • Cost: storage per event, and a log service to run. Redis streams, Postgres tables or a managed durable stream all work.

Idempotency: the other half of resuming

A client that retries a request after a drop may cause the action twice. Starting a run must therefore be idempotent: the client generates the run ID or an idempotency key, and the server returns the existing run if it has seen that key. Any side-effecting tool call inside the run needs the same protection.

Go deeperWhy a protocol would drop resumability

Resumable streams require the server to keep per-session state and route a reconnecting client to whoever holds it. That fights horizontal scaling: any server should be able to answer any request.

Stateless designs move state out of the connection into explicit handles: a task ID the client can poll from any server, backed by shared storage. It is the same durable-log idea, expressed as resources instead of a stream. Short calls stay simple; long calls become tasks.

Where you will meet it

Chat apps

Production chat products persist generation events so a refreshed tab picks up the answer mid-stream instead of losing it.

Durable streams

Several platforms now offer resumable streams as a primitive, backed by Redis or a database, for exactly this pattern.

Task APIs

Long agent jobs return a task ID immediately; clients poll or subscribe for status and results.

Words to know

Append-only log
A sequence of events that is only ever added to, never edited.
Offset
An event's position in the log, used to resume reading.
Idempotency key
A client-chosen ID that makes repeating a request safe.
Task handle
An ID representing long-running work, used to poll status and results.

Check yourself

Answer out loud before you open each one.

How does a stream survive a network change?

A durable event log, client-tracked offsets, and an idempotent resume request.

Why do side-effecting tools need idempotency keys once streams are not resumable?

A broken response loses the in-flight request; the client cannot tell whether the action ran, so a retry must be safe.

What does a stateless protocol replace sessions with?

Explicit handles, such as task IDs backed by shared storage, that any server can answer.

Try it

Build the log-backed endpoint above with an in-memory array, then use curl -N -H 'Last-Event-ID: 41' to resume from the middle. Kill the curl at random moments and confirm nothing is lost.

A good place to start

Explore a pattern

A little guidance

Start with a pattern, read through a problem, and mark it solved when the idea clicks. Save anything you want to revisit.

New tutorials are added one at a time. Your bookmarks and solved checkmarks stay in this browser.

Explore the roadmap