Workspace

Readiness, completion and io_uring

3 day streakD

Readiness, completion and io_uring

How one thread waits on 10,000 sockets. Learn thread-per-connection, epoll's readiness model and io_uring's completion model, and why io_uring is often disabled.

DepthLesson 4 of 617 min

The oldest server design gives each connection its own thread that blocks in read. It is simple and works well for hundreds of connections. At 10,000 mostly idle connections, it means 10,000 stacks and a scheduler juggling 10,000 threads that are almost all asleep.

Figure 4.1

Three ways to wait for I/O

Thread per connection

  • T1read(fd 7)asleep
  • T2read(fd 8)asleep
  • T3read(fd 9)asleep
  • T6read(fd 12)asleep
  • T9read(fd 15)asleep
  • … 10,000 threads, 10,000 stacks

Simple code. Memory and scheduling cost grow with every idle connection.

Readiness · epoll, kqueue

  1. epoll_wait()one thread sleeps on all sockets
  2. → fd 7, fd 12 readykernel reports what can be read now
  3. read(7) · read(12)your code does the reads
  4. ↻ back to epoll_waitloop

One thread holds thousands of idle connections for a few KB each.

Completion · io_uring

Submission queueread fd 7 → buf Aread fd 12 → buf Bwrite fd 3 ← buf C
Completion queuefd 7: 512 bytes in buf Afd 3: 4096 bytes written

Batches many operations per system call, and handles files too.

Thread-per-connection blocks a thread per socket. Readiness APIs such as epoll tell one thread which sockets can be read now; the thread then reads them. Completion APIs such as io_uring take the read request and report when the data is already in your buffer.

Readiness: select, poll, epoll, kqueue

select and poll take the full list of descriptors on every call and scan it, which is O(n) per wait. epoll on Linux and kqueue on BSD and macOS keep the interest list in the kernel and return only descriptors that are ready. The program then calls read itself. libuv, Go's netpoller, Tokio and nginx all sit on these.

Level-triggered notification repeats as long as data remains. Edge-triggered notification fires once when the state changes, so you must read until EAGAIN or you will never hear about the leftover bytes.

Completion: io_uring

io_uring shares two ring buffers between your process and the kernel. You place requests such as read this socket into this buffer in the submission queue. The kernel performs them and posts results in the completion queue. Many operations can go in with one system call or none, and it works for files as well as sockets, which epoll never handled well.

ModelAnswer to “what now?”Who does the readExamples
BlockingBlock until doneYour thread, sleepingThread-per-connection servers
ReadinessThis descriptor can be read nowYou, after the notificationepoll, kqueue, libuv, Tokio
CompletionThis read is done; here is the resultThe kernelio_uring, Windows IOCP
epoll_loop.cC
int ep = epoll_create1(0);
struct epoll_event ev = { .events = EPOLLIN, .data.fd = listen_fd };
epoll_ctl(ep, EPOLL_CTL_ADD, listen_fd, &ev);

for (;;) {
  struct epoll_event ready[64];
  int n = epoll_wait(ep, ready, 64, -1);        // sleep until something is ready
  for (int i = 0; i < n; i++) {
    int fd = ready[i].data.fd;
    if (fd == listen_fd) add_client(ep, accept4(listen_fd, NULL, NULL, SOCK_NONBLOCK));
    else handle_readable(fd);                   // read until EAGAIN
  }
}

This is the heart of every event loop. libuv wraps it with timers, a thread pool and cross-platform backends.

Where you will meet it

nginx and HAProxy

A few worker processes, each with an epoll loop, serve hundreds of thousands of connections.

Long-lived streams

SSE and WebSocket connections are mostly idle, so readiness-based servers hold them for kilobytes each instead of a thread each.

Storage engines

Databases such as TigerBeetle and ScyllaDB can use io_uring for disk I/O on hosts they control, where the sandbox concern does not apply.

Check yourself

Answer out loud before you open each one.

What is the difference between readiness and completion I/O?

epoll says you can act now and you do the read. io_uring says the read is done and the data is in your buffer.

Why does edge-triggered epoll require reading until EAGAIN?

It notifies only on a change. Leftover data does not cause a new notification.

Why might a sandbox block io_uring?

seccomp cannot inspect operations submitted through the rings, so io_uring can perform actions the filter meant to forbid.

Try it

Build a TCP echo server three ways: thread per connection, epoll, and io_uring with liburing. Hold 10,000 idle connections against each and compare resident memory.

A good place to start

Explore a pattern

A little guidance

Start with a pattern, read through a problem, and mark it solved when the idea clicks. Save anything you want to revisit.

New tutorials are added one at a time. Your bookmarks and solved checkmarks stay in this browser.

Explore the roadmap