Tic-Tac-Toe: Minimax to DQN

Build the game engine, minimax, tabular self-play, and a DQN agent, then compare value and policy learners.

6 parts92 individual lessons7.7 estimated hours460 XP available

Tic-Tac-Toe: Minimax to DQN project artwork

0%0 of 92 complete

No compressed chapters.

Every source step is its own lesson with intuition, concepts, correctly rendered MathJax mathematics, implementation, tests, mistakes, and a checkpoint.

PART 1

Board Representation & Game Engine

Encode the 3x3 board, implement move legality, win/draw detection, turn tracking, and wrap everything into a reusable Game class with hard-coded and interactive play.

0/18

001create empty board+5 XP 002encode player+5 XP 003print board+5 XP 004is cell empty+5 XP 005place move+5 XP 006get legal moves+5 XP 007check row win+5 XP 008check column win+5 XP 009check main diagonal win+5 XP 010check anti diagonal win+5 XP 011is winner+5 XP 012is draw+5 XP 013get game status+5 XP 014get current player+5 XP 015switch player+5 XP 016play hardcoded game+5 XP 017play interactive game+5 XP 018TicTacToeGame+5 XP

PART 2

Random and Minimax Baselines

Implement a random-move agent, run head-to-head rollouts, then build a full minimax (with alpha-beta pruning) and verify optimal play against random and itself.

0/12

019random move agent+5 XP 020play random vs random game+5 XP 021play random vs random matches+5 XP 022compute outcome rates+5 XP 023minimax terminal score+5 XP 024minimax value+5 XP 025minimax recursive+5 XP 026minimax max min step+5 XP 027minimax best move+5 XP 028minimax alpha beta+5 XP 029play minimax vs random matches+5 XP 030play minimax vs minimax matches+5 XP

PART 3

Tabular Q-Learning Foundations

Design state hashing with symmetry canonicalization, hyperparameters, epsilon-greedy action selection over legal moves, reward shaping, and the full Q-learning update and episode loop with learning curves.

0/24

031encode board state key+5 XP 032canonical board key+5 XP 033initialize q table+5 XP 034get q value+5 XP 035set q value+5 XP 036choose learning rate alpha+5 XP 037choose discount factor gamma+5 XP 038choose initial epsilon+5 XP 039epsilon decay schedule+5 XP 040epsilon greedy explore move+5 XP 041epsilon greedy select action+5 XP 042greedy argmax over legal actions+5 XP 043random tie break argmax+5 XP 044tic tac toe reward+5 XP 045q learning nonterminal target+5 XP 046q learning terminal target+5 XP 047q learning update+5 XP 048episode reset game+5 XP 049episode agent pick action+5 XP 050episode apply action+5 XP 051episode apply q update+5 XP 052episode check terminate+5 XP 053train q learning agent+5 XP 054compute batched outcome stats+5 XP

PART 4

Self-Play, Evaluation & Persistence

Train the Q-agent through self-play with perspective flipping, evaluate against random and minimax opponents, support human-vs-agent play, and save/load the Q-table.

0/9

055self play episode+5 XP 056flip board perspective+5 XP 057perspective reward sign+5 XP 058train q agent self play+5 XP 059evaluate q agent vs random+5 XP 060evaluate q agent vs minimax+5 XP 061inspect q values for state+5 XP 062serialize q table to dict+5 XP 063deserialize q table from dict+5 XP

PART 5

Deep Q-Network Agent

Move from a table to a neural function approximator: encode the board for an MLP, mask illegal actions, build a replay buffer and target network, train DQN end-to-end, and compare against earlier agents.

0/21

064encode board flat length nine+5 XP 065encode board one hot length eighteen+5 XP 066build mlp architecture+5 XP 067initialize mlp parameters+5 XP 068mlp forward pass+5 XP 069mask illegal actions neg inf+5 XP 070argmax action from q values+5 XP 071mse loss on chosen action+5 XP 072mlp backward pass+5 XP 073adam update step+5 XP 074create replay buffer+5 XP 075append transition to buffer+5 XP 076cap buffer size drop oldest+5 XP 077sample minibatch from buffer+5 XP 078build target network copy+5 XP 079compute target q with target network+5 XP 080sync target network periodically+5 XP 081dqn select action+5 XP 082dqn train step+5 XP 083train dqn agent+5 XP 084compare dqn tabular random minimax+5 XP

PART 6

Policy Gradients & Extensions

Implement a SARSA variant, a REINFORCE policy-gradient learner, compare value-based vs policy-based methods, and accelerate learning via 8-fold symmetry data augmentation.

0/8

085sarsa on policy update+5 XP 086train sarsa agent+5 XP 087reinforce log prob of action+5 XP 088reinforce collect episode returns+5 XP 089reinforce policy gradient update+5 XP 090train reinforce agent+5 XP 091compare value vs policy learners+5 XP 092symmetry augmented training+5 XP