PHASE 02 · MODULE 06
Streaming and RPC.
Choose by who talks when and what sits in the path. LLM streaming is one-way, so Server-Sent Events usually wins by being ordinary HTTP that proxies already understand.
Every streaming protocol answers three questions: which side may send, what happens when the connection breaks, and how the middleboxes in the path will treat it. Most production bugs in token streaming are not model bugs; they are framing, buffering and reconnection bugs.
Start lesson 1Basics
- Chunked transfer
- The SSE wire format
- The WebSocket upgrade
- Request-response versus streaming
Depth
- WebSocket framing and masking
- gRPC over HTTP/2 with trailers
- Deadlines and cancellation propagation
- L4 versus L7 load balancing
- Resumable streams
Current
- Connect RPC
- WebTransport
- Stateless agent protocols
- Append-only event logs as the source of truth for a stream
Lessons
- 01Request-response, streaming and the options betweenLong polling, SSE, WebSocket, gRPC, WebTransport and job handles each fit a different shape of conversation. Learn to pick by direction, reconnection and what sits in the path.
- 02Server-Sent Events, byte by byteSSE is a tiny text format inside an HTTP response. Learn every field, how to parse it correctly from arbitrary chunks, and how reconnection with Last-Event-ID works.
- 03WebSocket: upgrade, frames and backpressureA WebSocket starts as HTTP and becomes a two-way message channel. Learn the handshake, the frame format and its masking, and the operational traps of stateful connections.
- 04gRPC, deadlines and why load balancing surprises peoplegRPC is typed RPC over HTTP/2 streams. Learn its message framing and trailers, how deadlines and cancellation should flow through a call tree, and why one long connection defeats an L4 balancer.
- 05Everything that breaks an LLM streamA worked catalogue of the six ways a token stream fails between the GPU and the screen, with the fix for each and a client that survives them.
- 06Resumable streams and stateless agent protocolsMake the stream a view of a durable log, and a dropped connection becomes a reconnect. Then see why the latest MCP revision went the other way: stateless requests, handles and polling.
Module lab
- Write an SSE parser and fuzz it with random chunk boundaries, including splits inside a character.
- Build a resumable stream backed by an append-only log, and kill the connection at random points.
- Put a buffering proxy in front of your stream, observe the lump, and fix it.
Interview questions
SSE or WebSocket for a chat UI?
SSE, unless the client must send on the same channel during generation.
How does a stream survive a network change?
Durable event log, offsets, idempotent resume.
Why does gRPC load balancing surprise people?
One long HTTP/2 connection carries many calls, so an L4 balancer cannot spread them.
Read
- The Server-Sent Events section of the WHATWG HTML standard
- RFC 6455: The WebSocket Protocol
- gRPC's PROTOCOL-HTTP2 document
- The current Model Context Protocol specification