Resumable streams and stateless agent protocols
Make the stream a view of a durable log, and a dropped connection becomes a reconnect. Then see why the latest MCP revision went the other way: stateless requests, handles and polling.
A stream that lives only in a connection dies with it. The fix is to make the connection a view of something durable. The server appends every event to an append-only log keyed by run ID, with an increasing offset. A client streams from an offset; when it drops, it reconnects with the last offset it saw and receives everything after it.
Drop the connection at any point
Run the model, drop the connection part-way, then reconnect.
// Server: the model loop writes to the log, never directly to a socket.
async function runAgent(runId: string) {
for await (const event of modelStream()) await log.append(runId, event); // offset assigned by the log
await log.append(runId, { type: "done" });
}
// Server: any connection reads from the log, starting where the client left off.
app.get("/runs/:id/events", async (req, res) => {
const from = Number(req.header("last-event-id") ?? -1) + 1;
res.writeHead(200, { "content-type": "text/event-stream", "cache-control": "no-cache" });
for await (const { offset, event } of log.read(req.params.id, from, { follow: true, signal: req.signal })) {
res.write(`id: ${offset}\ndata: ${JSON.stringify(event)}\n\n`);
}
res.end();
});The model loop and the HTTP connection are now independent. A refresh, a network change or a second device all just read the same log.
What this buys, and what it costs
- Resume after any drop, including a phone changing networks or a page refresh.
- Multiple viewers of one run, such as a second device or a teammate.
- Replay and audit: the log is the record of what the agent did (Module 10 builds durable execution on the same idea).
- Cost: storage per event, and a log service to run. Redis streams, Postgres tables or a managed durable stream all work.
Idempotency: the other half of resuming
A client that retries a request after a drop may cause the action twice. Starting a run must therefore be idempotent: the client generates the run ID or an idempotency key, and the server returns the existing run if it has seen that key. Any side-effecting tool call inside the run needs the same protection.
Go deeperWhy a protocol would drop resumability
Resumable streams require the server to keep per-session state and route a reconnecting client to whoever holds it. That fights horizontal scaling: any server should be able to answer any request.
Stateless designs move state out of the connection into explicit handles: a task ID the client can poll from any server, backed by shared storage. It is the same durable-log idea, expressed as resources instead of a stream. Short calls stay simple; long calls become tasks.
Where you will meet it
Production chat products persist generation events so a refreshed tab picks up the answer mid-stream instead of losing it.
Several platforms now offer resumable streams as a primitive, backed by Redis or a database, for exactly this pattern.
Long agent jobs return a task ID immediately; clients poll or subscribe for status and results.
Words to know
- Append-only log
- A sequence of events that is only ever added to, never edited.
- Offset
- An event's position in the log, used to resume reading.
- Idempotency key
- A client-chosen ID that makes repeating a request safe.
- Task handle
- An ID representing long-running work, used to poll status and results.
Check yourself
Answer out loud before you open each one.
How does a stream survive a network change?
A durable event log, client-tracked offsets, and an idempotent resume request.
Why do side-effecting tools need idempotency keys once streams are not resumable?
A broken response loses the in-flight request; the client cannot tell whether the action ran, so a retry must be safe.
What does a stateless protocol replace sessions with?
Explicit handles, such as task IDs backed by shared storage, that any server can answer.
Try it
Build the log-backed endpoint above with an in-memory array, then use curl -N -H 'Last-Event-ID: 41' to resume from the middle. Kill the curl at random moments and confirm nothing is lost.