Project · Reinforcement learning

Learn2Slither

Teaching a Snake to play on its own through trial and error and a handcrafted Q-table. A project at 42.

C++17 · SDL342 school

.zip — private GitHub repository, as required by 42

How the snake sees

No image, just four rays and a direction

The snake does not see the board. On each turn, a ray extends from its head in each of the four cardinal directions until it finds a wall or its own body (W), a green apple (G), a red apple (R), or nothing (0). These four letters and the current direction form a short key such as "0G0WU", which is exactly the Q-table input. No neural network or combinatorial explosion: just a hash map that assigns a value to a situation.

Terminal output showing the snake's vision in four directions and the score of each move

Learning through rewards

A Bellman equation applied to every move

On every move, the expected value of the chosen action is compared with what it should actually have been worth (the immediate reward plus the best value reachable from the new position), and the Q-table is adjusted by the difference:

Q_observed = reward + γ · max(Q(next_state))
Q(state)  += α · (Q_observed - Q(state))

α = 0.1   (learning rate)
γ = 0.92  (discount factor)

Wall-500

Its own body-100

Red apple (shrinks)-20

Empty cell (per move)-1

Green apple (grows)+20

To choose a direction, the snake samples from a softmax over the four values rather than using simple epsilon-greedy selection. A temperature of 2.65 encourages extensive exploration during the first 30% of training sessions, then drops to 1.25 to favor what already works. During normal play, outside training, a temperature sampled between 1.65 and 5.0 keeps things a little unpredictable.

Handcrafted from start to finish

The game had to be built too

The game engine

Snake, a 10×10 board, green and red apples, collisions: all the Snake logic was written before even thinking about AI.

The SDL3 interface

A small custom toolkit built on raw SDL3 (Screen, Board, Button, Text, Image), without Dear ImGui, and a bitmap font drawn one letter at a time.

The flag parser

A small command-line argument parser (--training, --sessions, --load...) written by hand instead of using getopt.

The Q-table is automatically saved at the end of each session in two formats: a compact binary for quick loading and a .st text file (one key value line per learned situation) for manual inspection.

On video

Watch the snake play

Command line

Train, then watch

./l2s -t -s 10000 --visual --speed 200      # train for 10,000 games
./l2s --load model-10000.s --visual         # replay a trained model

-t, --trainingEnable training (updates the Q-table)

-s, --sessions <n>Number of games to play

--visualDisplay the SDL3 window during games

--load <model>Load a saved Q-table (.s or .st)

--step-by-stepAdvance manually, frame by frame

--speed <fps>Simulation speed (120 by default)

The program's command-line help, listing all available flags

Tech stack

What powers Learn2Slither

C++17Language, compiled with -O3
SDL3 · SDL3_image · SDL3_ttfWindow, rendering, textures
Custom Makefilepkg-config on macOS, vendored libraries on Linux
Tabular Q-learningA hash map as the Q-table, no neural network

home