PHASE 01 · MODULE 01
Machine reality.
Big-O counts operations. Real speed depends on how far the data is from the CPU.
Two O(n) loops can differ by 10x or more because one stays in cache and the other chases pointers. Scale the numbers until 1 ns feels like 1 second, and it becomes obvious why a harness should spend its effort on round trips first and CPU second.
Start lesson 1Basics
- Binary and two's complement
- IEEE 754 floats
- Endianness
- ASCII, UTF-8 and UTF-16
- Stack and heap
- Pointers and alignment
Depth
- 64-byte cache lines
- Locality and prefetching
- False sharing
- The TLB
- Branch prediction
- SIMD
- Memory ordering on x86 and Arm
- Generational garbage collection
Current
- SIMD parsing as in simdjson
- Arm servers
- NUMA on large hosts
- GPU memory as the limit for inference
Lessons
- 01Bits, integers and floatsEvery value is a bit pattern plus a rule for reading it. Learn the three rules that matter: unsigned, two's complement and IEEE 754.
- 02Text is bytes: ASCII, UTF-8 and UTF-16A string has at least four different lengths. Learn which one each limit means, and how a careless slice ships broken text to a model.
- 03Stack, heap, pointers and alignmentWhere values live decides how fast you can reach them and how long they survive. Learn the two regions, and why struct field order changes its size.
- 04The latency ladderEvery operation has a distance. Put them on one scale and you will know where to spend optimisation effort before you profile anything.
- 05Caches, locality and false sharingMemory moves in 64-byte lines. Code that uses every byte of each line runs fast; code that jumps around pays the full trip to DRAM again and again.
- 06SIMD, memory ordering, GC and GPU memoryWhere hardware is in late 2026: wide vector instructions, weaker memory models on Arm, garbage collectors that trade pauses for throughput, and GPUs limited by memory bandwidth.
Module lab
- Sum an array, then walk a shuffled linked list, at sizes from 1 KB to 1 GB.
- Plot nanoseconds per element and find the steps at each cache size.
- Confirm the cause with
perf stat -e cache-misses. - Write a truncation helper that respects UTF-8 boundaries and graphemes, and fuzz it with random emoji.
Interview questions
Why can two O(n) loops differ by 10x?
Locality and pointer chasing. Sequential access uses whole cache lines and the prefetcher; pointer chasing waits on DRAM for most elements.
Why is str.length wrong for byte limits and token budgets?
It counts UTF-16 code units. Bytes depend on encoding, tokens on the tokenizer. Separate the three explicitly.
What is false sharing, and how would you find it?
Two threads write different variables on one cache line. Use perf c2c, or pad hot per-thread fields to 64 bytes and measure.
Read
- Computer Systems: A Programmer's Perspective, Bryant and O'Hallaron
- What Every Programmer Should Know About Memory, Ulrich Drepper
- Algorithms for Modern Hardware, Sergey Slotin