Workspace

Blocking the loop and the thread pool

3 day streakD

Blocking the loop and the thread pool

One slow synchronous call freezes every user on the process. Learn to spot event-loop blockers and the hidden four-thread pool behind fs, DNS and zlib.

DepthLesson 3 of 615 min

Because one thread runs all JavaScript, any synchronous work holds up everyone. A 200 ms JSON.parse of a 50 MB tool result stops the loop for 200 ms. Every other user's token stream freezes for exactly that long, and every pending timer and socket read waits.

Figure 3.1

Three streams and one big parse

Token delivery over 400 msWith the parse on the main thread, no user receives tokens between 100 and 300 ms; delayed tokens arrive in a burst at 300 ms.0 ms100 ms200 ms300 ms400 msUser AUser BUser CMain threadJSON.parse · 200 ms · loop blocked
Longest gap between tokens
≈ 200 ms
Users affected
All 3
Main thread during parse
Busy
Three users receive tokens every 20 ms. A synchronous parse on the main thread stops all three for 200 ms. Move the parse to a worker thread and the streams keep flowing while the worker parses.

Fixes, in order of preference

  1. 01
    Do less

    Do not parse what you will not read. Cap tool output by bytes before parsing, or keep it as text.

  2. 02
    Stream it

    Parse incrementally as bytes arrive, so no single step is long.

  3. 03
    Chunk it

    Break the work into pieces and yield between them with setImmediate.

  4. 04
    Move it

    Run it in a worker_thread or a separate process. Workers have their own event loop and heap, and communicate by message passing or SharedArrayBuffer.

The libuv thread pool

Not all I/O in Node is non-blocking in the kernel. File system calls, dns.lookup, zlib compression and some crypto functions run on libuv's thread pool, which has 4 threads by default. If four slow DNS lookups occupy the pool, an unrelated fs.readFile waits in line behind them.

Figure 3.2

Four threads, one queue

Main thread · event loopRuns JavaScript, waits in epoll for socketsHTTP sockets bypass the pool entirely
Pool queue
  1. fs.readFile("config.json")waiting 1.6 s
  2. crypto.pbkdf2(…)waiting
libuv thread pool · 4 threads
  • Thread 1dns.lookup("api.slow.example")2.1 s and counting
  • Thread 2dns.lookup("cdn.slow.example")1.8 s and counting
  • Thread 3dns.lookup("auth.slow.example")1.6 s and counting
  • Thread 4zlib.gzip(8 MB)40 ms
Three slow DNS lookups and a gzip job hold all four threads, so fs.readFile waits in the queue though the disk is idle. Network sockets do not use the pool; they go through epoll on the main loop.
threadpool.shShell
# Raise the pool size before Node starts (read once at startup)
UV_THREADPOOL_SIZE=16 node server.js

# Find event-loop blockers with a CPU profile
node --cpu-prof server.js        # writes a .cpuprofile; open it in Chrome DevTools

# Or warn on blocking sync calls during development
node --trace-sync-io server.js

dns.resolve* uses c-ares on the network instead of the thread pool. dns.lookup uses the system resolver through the pool.

Where you will meet it

Large tool results

Agent servers cap and stream tool output, and parse big JSON in workers, so one heavy run does not stall others on the same instance.

Slow DNS incidents

Services have stalled file I/O because a resolver became slow and dns.lookup calls filled the thread pool.

CPU-heavy endpoints

Image resizing and PDF parsing are usually moved to worker pools or separate services, keeping the request loop free.

Check yourself

Answer out loud before you open each one.

Why can one JSON.parse hurt every user?

It runs synchronously on the single event-loop thread, so nothing else runs until it finishes.

Which Node operations use the libuv thread pool?

File system calls, dns.lookup, zlib and some crypto such as pbkdf2 and scrypt.

How would you find what is blocking the loop?

Measure loop delay, then take a CPU profile or flame graph during the stall and look for long synchronous frames.

Try it

Start a Node HTTP server with a /stream endpoint that writes a line every 20 ms and a /parse endpoint that parses a 50 MB JSON string. Hit /stream with curl, then call /parse and watch the stream pause. Move the parse into a worker and repeat.

A good place to start

Explore a pattern

A little guidance

Start with a pattern, read through a problem, and mark it solved when the idea clicks. Save anything you want to revisit.

New tutorials are added one at a time. Your bookmarks and solved checkmarks stay in this browser.

Explore the roadmap