Workspace

Streaming & RPC

3 day streakD
Systems syllabus
PHASE 02 · MODULE 06

Streaming and RPC.

Choose by who talks when and what sits in the path. LLM streaming is one-way, so Server-Sent Events usually wins by being ordinary HTTP that proxies already understand.

Every streaming protocol answers three questions: which side may send, what happens when the connection breaks, and how the middleboxes in the path will treat it. Most production bugs in token streaming are not model bugs; they are framing, buffering and reconnection bugs.

6 lessons114 minutes of reading6 illustrations
Start lesson 1
Basics
  • Chunked transfer
  • The SSE wire format
  • The WebSocket upgrade
  • Request-response versus streaming
Depth
  • WebSocket framing and masking
  • gRPC over HTTP/2 with trailers
  • Deadlines and cancellation propagation
  • L4 versus L7 load balancing
  • Resumable streams
Current
  • Connect RPC
  • WebTransport
  • Stateless agent protocols
  • Append-only event logs as the source of truth for a stream

Lessons

  1. 01Request-response, streaming and the options betweenLong polling, SSE, WebSocket, gRPC, WebTransport and job handles each fit a different shape of conversation. Learn to pick by direction, reconnection and what sits in the path.Basics16 min
  2. 02Server-Sent Events, byte by byteSSE is a tiny text format inside an HTTP response. Learn every field, how to parse it correctly from arbitrary chunks, and how reconnection with Last-Event-ID works.Basics22 min
  3. 03WebSocket: upgrade, frames and backpressureA WebSocket starts as HTTP and becomes a two-way message channel. Learn the handshake, the frame format and its masking, and the operational traps of stateful connections.Depth18 min
  4. 04gRPC, deadlines and why load balancing surprises peoplegRPC is typed RPC over HTTP/2 streams. Learn its message framing and trailers, how deadlines and cancellation should flow through a call tree, and why one long connection defeats an L4 balancer.Depth20 min
  5. 05Everything that breaks an LLM streamA worked catalogue of the six ways a token stream fails between the GPU and the screen, with the fix for each and a client that survives them.Depth20 min
  6. 06Resumable streams and stateless agent protocolsMake the stream a view of a durable log, and a dropped connection becomes a reconnect. Then see why the latest MCP revision went the other way: stateless requests, handles and polling.Current18 min

Module lab

  1. Write an SSE parser and fuzz it with random chunk boundaries, including splits inside a character.
  2. Build a resumable stream backed by an append-only log, and kill the connection at random points.
  3. Put a buffering proxy in front of your stream, observe the lump, and fix it.

Interview questions

SSE or WebSocket for a chat UI?

SSE, unless the client must send on the same channel during generation.

How does a stream survive a network change?

Durable event log, offsets, idempotent resume.

Why does gRPC load balancing surprise people?

One long HTTP/2 connection carries many calls, so an L4 balancer cannot spread them.

Read

  • The Server-Sent Events section of the WHATWG HTML standard
  • RFC 6455: The WebSocket Protocol
  • gRPC's PROTOCOL-HTTP2 document
  • The current Model Context Protocol specification

A good place to start

Explore a pattern

A little guidance

Start with a pattern, read through a problem, and mark it solved when the idea clicks. Save anything you want to revisit.

New tutorials are added one at a time. Your bookmarks and solved checkmarks stay in this browser.

Explore the roadmap