PART 1
0/6Sampling Primitives
Implement the decoding-time math: stable softmax, temperature scaling, top-k/top-p filters, and stochastic vs greedy token selection.
Mini LLM Inference Server
Mini LLM Inference Server
Construct sampling, tokenization, KV caching, paged allocation, continuous batching, streaming, and benchmarking.
Every source step is its own lesson with intuition, concepts, correctly rendered MathJax mathematics, implementation, tests, mistakes, and a checkpoint.
PART 2
0/3Tokenization
Build a tiny vocabulary plus encode/decode utilities to move between strings and token id sequences.
PART 3
0/7Tiny Transformer with KV Cache
Implement embeddings, linear projections, a basic KV cache, causal attention, and the prefill/decode forward passes.
010define tiny transformer with kv cache+5 XP011derive tiny transformer with kv cache+5 XP012prepare tiny transformer with kv cache+5 XP013implement tiny transformer with kv cache+5 XP014connect tiny transformer with kv cache+5 XP015validate tiny transformer with kv cache+5 XP016debug tiny transformer with kv cache+5 XP
PART 4
0/9Paged KV Cache
Design a block-based KV allocator with allocation/free, paged appends, gather, paged attention, and usage accounting.
PART 5
0/8Sequences and Static Batching
Model per-request sequence state, drive single-sequence generation, then batch multiple sequences through synchronized decode steps.
026define sequences and static batching+5 XP027derive sequences and static batching+5 XP028prepare sequences and static batching+5 XP029implement sequences and static batching+5 XP030connect sequences and static batching+5 XP031validate sequences and static batching+5 XP032debug sequences and static batching+5 XP033optimize sequences and static batching+5 XP
PART 6
0/8Continuous Batching and Scheduling
Add capacity checks, priority queues, admission, preemption, and a scheduler that mixes prefill and decode across in-flight requests.
034define continuous batching and scheduling+5 XP035derive continuous batching and scheduling+5 XP036prepare continuous batching and scheduling+5 XP037implement continuous batching and scheduling+5 XP038connect continuous batching and scheduling+5 XP039validate continuous batching and scheduling+5 XP040debug continuous batching and scheduling+5 XP041optimize continuous batching and scheduling+5 XP
PART 7
0/5Serving API
Expose a request/response interface with streaming chunks, submission, a driver loop, output collection, and completion responses.
PART 8
0/5Benchmarking
Measure TTFT, inter-token latency, throughput, and latency percentiles with an end-to-end benchmark runner.