Build the game engine, policy-value network, PUCT MCTS, self-play generation, training, and baseline evaluation.
Every source step is its own lesson with intuition, concepts, correctly rendered MathJax mathematics, implementation, tests, mistakes, and a checkpoint.
Build the board representation, move mechanics, terminal detection, and environment step function for Connect-4.
Encode boards as tensors and assemble a convolutional backbone with policy and value heads.
Mask illegal moves and turn network logits into sampled or greedy column actions.
Implement nodes, PUCT selection, network-guided expansion, backup, and full MCTS rollouts.
Use MCTS to play games against itself, recording (state, policy, outcome) training tuples.
Define the policy, value, and L2 losses and run minibatched training over the self-play buffer.
Alternate self-play generation and network training across many iterations.
Build baseline agents, run head-to-head matches, and measure win rate against a random policy.
