01MediumMachine Learning
Random Forest from Scratch
Build a complete Random Forest classifier from impurity math to a bagged ensemble with bootstrapping and feature subsampling.
Start project →LVL 01SK
Explore the journey
Every project is open. Start anywhere, follow your curiosity, and learn at your own pace.
26 projects · all openProgress and bookmarks are stored locally
01
02MediumMachine Learning
Support Vector Machine from Scratch
Build a linear SVM using hinge loss, a regularized objective, and gradient-based optimization.
Start project →
03HardDeep Learning
Build a Trainable CNN from Scratch
Assemble a LeNet-style convolutional network with im2col convolutions, gradients, Adam, and a complete training loop.
Start project →
04HardDeep Learning
Transformer from Scratch
Reimplement the original encoder-decoder Transformer with multi-head attention, scheduling, and beam search.
Start project →
05HardDeep Learning
Tiny GPT From Scratch
Grow a character-level language model from a bigram baseline into a working GPT with multi-head attention and sampling.
Start project →
06EasyReinforcement Learning
Q-Learning on FrozenLake
Train a tabular Q-learning agent with epsilon-greedy exploration and greedy evaluation.
Start project →
07HardReinforcement Learning
AlphaZero on Connect-4
Build the game engine, policy-value network, PUCT MCTS, self-play generation, training, and baseline evaluation.
Start project →
08HardLLMs
RAG Pipeline
Construct ingestion, chunking, embeddings, hybrid retrieval, grounded generation, evaluation, and conversational memory.
Start project →
09HardAlignment
DPO from Scratch
Implement log-prob utilities, Bradley–Terry preferences, DPO gradients, IPO variants, and a full evaluation pipeline.
Start project →
10HardLLMs
RLHF from Scratch
Build decoding, SFT, LoRA, reward modeling, PPO, preference optimization, evaluation, and a model comparison interface.
Start project →
11HardGenerative AI
Vision-Language Model
Build a ViT encoder, multimodal projector, causal decoder, training loop, and caption generation from raw tensor operations.
Start project →
12HardCUDA
Flash Attention in CUDA
Build a tiled, IO-aware Flash Attention kernel with online softmax and causal masking.
Start project →
13HardGPU / CUDA
Fused LLM Inference Kernels
Implement reductions, activations, fused RMSNorm, Softmax, RoPE, and SwiGLU kernels for efficient inference.
Start project →
14HardML Systems
Mini LLM Inference Server
Construct sampling, tokenization, KV caching, paged allocation, continuous batching, streaming, and benchmarking.
Start project →
15HardDeep Learning
VeriLLM
Wrap cached GPT inference in Merkle commitments and spot checks, then simulate a decentralized committee with rewards and slashing.
Start project →
16EasyDeep Learning
MLP in JAX from Scratch
Implement initialization, forward passes, loss, autodiff, and pure functional SGD updates in JAX.
Start project →
17HardDeep Learning
Build Your Own teenygrad
Construct a lazy buffer, reverse-mode autodiff engine, tensor API, neural primitives, and train a small MLP.
Start project →
18MediumFederated Learning
Federated Averaging
Implement client partitioning, local SGD, weighted aggregation, partial participation, and non-IID experiments.
Start project →
19HardDistributed Training
DiLoCo Distributed Training
Train workers locally, aggregate pseudo-gradients with an outer optimizer, and quantify communication savings.
Start project →
20HardDistributed Training
Memory-Constrained Trainer
Implement accumulation, checkpointing, mixed precision, all-reduce, and ZeRO-style optimizer sharding.
Start project →
21HardGPU Programming
Trainable MoE in CUDA
Build CUDA kernels for a complete sparse mixture-of-experts forward, backward, and training loop.
Start project →
22HardGenerative AI
Multimodal Image Generator
Train a VQ-VAE image tokenizer and an autoregressive text-conditioned transformer with guided sampling.
Start project →
23EasyFine-tuning
LoRA Fine-Tune a Chat Model
Load a 4-bit Qwen model, attach LoRA adapters, format instructions, run SFT, and generate with the tuned model.
Start project →
24MediumMachine Learning
A/B Testing & Causal Inference
Build proportion tests, sample sizing, multiple-testing corrections, difference-in-differences, and synthetic control.
Start project →
25MediumOptimization
Market-Making Simulator
Build expected-value games and a quoting engine that manages inventory, adverse selection, and P&L.
Start project →
26HardReinforcement Learning
Tic-Tac-Toe: Minimax to DQN
Build the game engine, minimax, tabular self-play, and a DQN agent, then compare value and policy learners.
Start project →