Build Your Own teenygrad

Construct a lazy buffer, reverse-mode autodiff engine, tensor API, neural primitives, and train a small MLP.

7 parts58 individual lessons4.8 estimated hours290 XP available

Build Your Own teenygrad project artwork

0%0 of 58 complete

No compressed chapters.

Every source step is its own lesson with intuition, concepts, correctly rendered MathJax mathematics, implementation, tests, mistakes, and a checkpoint.

PART 1

Shape Utilities

Implement the small helper functions for working with shape tuples and index ordering that the rest of the engine relies on.

0/2

001prod+5 XP 002argsort+5 XP

PART 2

The Lazy Buffer Backend

Build the numpy-backed LazyBuffer with constant and random initialization plus elementwise, reduce, and movement operations.

0/10

003make op enums+5 XP 004LazyBuffer+5 XP 005lazybuffer const+5 XP 006rand+5 XP 007lazybuffer unary e+5 XP 008lazybuffer binary e+5 XP 009lazybuffer r+5 XP 010lazybuffer reshape+5 XP 011lazybuffer expand+5 XP 012lazybuffer permute+5 XP

PART 3

Autograd Functions

Define the differentiable Function base class and implement forward and backward passes for all unary, binary, reduce, and movement operations.

0/21

013Function+5 XP 014function forward backward stubs+5 XP 015apply+5 XP 016Neg+5 XP 017Relu+5 XP 018Log+5 XP 019Exp+5 XP 020Sqrt+5 XP 021Sigmoid+5 XP 022Add+5 XP 023Sub+5 XP 024Mul+5 XP 025Div+5 XP 026sum function forward+5 XP 027sum function backward+5 XP 028max function forward+5 XP 029max function backward+5 XP 030Reshape+5 XP 031expand function forward+5 XP 032expand function backward+5 XP 033permute function forward backward+5 XP

PART 4

The Tensor API

Create the Tensor wrapper, creation helpers, topological backward pass, and method bindings for every supported operation.

0/11

034Tensor+5 XP 035tensor from data+5 XP 036tensor creation helpers+5 XP 037tensor randn+5 XP 038build topological order+5 XP 039tensor backward+5 XP 040bind unary tensor methods+5 XP 041broadcasted+5 XP 042bind binary tensor methods+5 XP 043bind movement tensor methods+5 XP 044bind reduce tensor methods+5 XP

PART 5

Functional and Composite Ops

Compose primitives into higher-level operations like mean, matmul, softmax, log_softmax, and cross entropy.

0/6

045tensor mean+5 XP 046tensor transpose+5 XP 047tensor matmul 2d+5 XP 048tensor softmax+5 XP 049tensor log softmax+5 XP 050sparse categorical cross entropy+5 XP

PART 6

Neural Network Modules and Optimizer

Implement Linear layers, an MLP, and the SGD optimizer with gradient zeroing.

0/4

051Linear+5 XP 052MLP+5 XP 053sgd step+5 XP 054zero grad+5 XP

PART 7

Training and Evaluation

Generate a toy dataset, train the MLP end to end, and evaluate its accuracy on a held-out split.

0/4

055make toy digit dataset+5 XP 056accuracy+5 XP 057train mlp+5 XP 058evaluate mlp+5 XP