A Q-learning agent that starts with zero knowledge and learns Tic Tac Toe over 10,000 games against a random opponent, in one Python file with no neural network.
By Sarah Schürch
Key takeaways.
1. One Python file, no neural network: a Q table maps every board-and-move pair to an expected payoff.
2. Rewards are +1 for a win, 0.5 for a draw and −10 for an invalid move, so illegal moves get trained out fast.
3. Epsilon-greedy keeps 10% of moves random; across 10,000 games the win rate climbs and losses fall.
Chapters:
0:00 AlphaGo’s move 37
0:43 What we’re building
1:07 Reinforcement learning basics
2:08 State, actions, rewards
2:43 Exploration vs exploitation
4:34 The environment class
5:36 The Q-learning agent
7:01 Training and the learning curve
source