Pipes, buffers and EOF
A pipe is a small kernel buffer with a reader and a writer. When nobody reads, the writer blocks. When anybody still holds the write end, the reader never sees EOF.
pipe() returns two descriptors: a read end and a write end. Bytes written to one come out of the other, in order. Between them sits a kernel buffer, 64 KiB by default on Linux. When it fills, write blocks until a reader makes room. When it is empty, read blocks until a writer adds bytes or every write end is closed, which the reader sees as EOF, a read that returns 0.
Fill a pipe until the writer blocks
Each step the child writes 8 KiB and the harness reads up to 12 KiB.
write and never exits, while a harness that is waiting for exit waits forever.Hang 1: the full pipe
A classic deadlock: the harness waits for the child to exit and only then reads its output. The child produces more than 64 KiB, blocks in write, and never exits. Neither side moves. The fix is to read stdout and stderr continuously while the child runs, even if you only keep the first part.
Hang 2: EOF never arrives
EOF arrives only when every descriptor for the write end is closed. Suppose the model runs npm run dev & and the main command exits. The background server inherited stdout, so it still holds the write end. The harness reads forever, waiting for an EOF that comes only when the server dies. Listen for process exit as well as stream end, and decide what to do with descendants that outlive their parent.
const MAX = 256 * 1024; // keep at most 256 KiB
let kept: Buffer[] = [], keptBytes = 0, totalBytes = 0;
child.stdout.on("data", (chunk: Buffer) => {
totalBytes += chunk.length; // always drain the pipe
if (keptBytes < MAX) {
const slice = chunk.subarray(0, MAX - keptBytes);
kept.push(slice); keptBytes += slice.length;
}
});
// 'exit' fires when the process ends; 'close' waits for all stdio to end.
// A background grandchild can delay 'close' forever.
child.on("exit", () => setTimeout(() => child.stdout.destroy(), 500));Draining and keeping are separate decisions. Always drain; keep only what you will use.
Where you will meet it
The documentation warns that wait() with stdout=PIPE can deadlock on large output, and recommends communicate(), which reads both pipes while waiting.
In producer | head -1, head exits after one line. The producer's next write fails with SIGPIPE, which is how pipelines stop early.
Production tools stream output to the model in chunks and cap what they keep, while draining everything so the child can finish.
Check yourself
Answer out loud before you open each one.
Why does a child hang when the harness reads only after exit?
Its output fills the 64 KiB pipe buffer, write blocks, and the child cannot exit.
The main command exited but the read never ends. Why?
A background descendant inherited the write end of the pipe and still holds it open, so EOF never arrives.
What happens when a process writes to a pipe whose readers have all closed?
It receives SIGPIPE, which terminates it by default, or write fails with EPIPE if the signal is ignored.
Try it
Write a tiny harness that spawns yes | head -c 200000 and calls waitpid before reading. Watch it hang with strace -p. Then fix it by reading while waiting.