Skip to content
ML to LLMs
Labs / Foundations

Q-learning on a grid world

Step an agent through a small grid world, watch the Q-table and greedy policy update, and tune learning rate, discount, and exploration.

Theory and worked examples: Q-learning and SARSA

Q-learning on a cliff grid

A synthetic 5×5 world: −1 per move, −100 for a pit. Watch the Q-table fill in, one temporal-difference update at a time.

Q-values per cell: longer, darker arrows are better actionsS pit −100pit −100pit −100goal
best action (greedy policy)greedy pathagent
Return per episode01020304050-100-75-50-250episodereturnoptimal −6
20-episode meanone episode (clipped at −120)
Q-learning: press Step to see one update written out.
All Q-values start at 0. Press Step to watch one move and one update, or Run 50 episodes to see the values spread back from the goal.