Self-play RL · runs entirely in your browser
Sit across the board from a neural network.
A 587,306-parameter graph transformer that learned Ludo by playing itself — small enough to run right here in the page. No server, no account, no tracking: your moves never leave your browser. It plays a sharp, patient game. See if you can beat it.
In the tradition of TD-Gammon · AlphaGo · AlphaZero
Where to start
Interactive
Take on the AI in a real game of Ludo. The whole network runs locally in your browser — no backend, no waiting, your moves never leave your machine.
The story
From a naive baseline that couldn't tell its own four tokens apart, through hand-engineered features and a symmetric encoder, to a graph transformer a fifth the size that plays better than all of them.
Inspirations
AlphaGo lit the spark. TD-Gammon wrote the playbook for games with dice. AlphaZero is the obvious next step but needs more compute than this side project has. Here's the full reading list.
AI's predicted winner
Behind the model
From a naive baseline that couldn't tell its own four tokens apart, to a graph transformer a fifth the size that beats every earlier version of itself — and the proof that what's left between it and perfection is mostly the dice. The short tour, including what didn't work.
From V1 to V15.2
The model saw the board as eight stacked black-and-white maps — but it couldn't tell its own four tokens apart. They all collapsed into one blob, so the AI was guessing which piece to move. It lost a lot.
We started feeding the network "tactical hints" we computed by hand — danger maps, capture opportunities, safe landing squares. It got better, but plateaued at 73–77% wins against scripted opponents and stopped improving.
Added a "token attention" layer — letting the network reason about its four tokens as separate entities, with awareness of how often each had been ignored. First model to consistently win >80% against scripted opponents.
Stripped most of the hand-engineered features back out and gave the network mostly raw board positions plus 3 static board hints (safe cells, home stretches). Beat every earlier version of itself by a small but statistically real edge. Held this site for two weeks as the strongest version — until V13.5.
Every encoder so far had given each of the four own tokens its own input channel — forcing the network to learn from scratch that the rules treat them identically. V13.5 collapses them into a single count-per-cell channel and re-routes the model's "which token to move" output through a rank-indexed gather. Same parameter budget, same training pipeline, same opponent pool — 51.7% wins over 3,000 games vs V13.2, and 90.4% vs the competing V13.4 temporal experiment. The first version to clear the V13-class plateau, and this site's previous champion.
Instead of a CNN scanning the board like an image, V15.2 treats the 225 board cells as nodes in a graph transformer — each cell carries a three-number summary, attention layers pass information along the routes, and the policy picks the cell to move from. No history frames, no hand-engineered hints, no per-token channels: one frame in, one decision out. At 587K parameters — a fifth of V13.5 — it beats V13.5 52.7% over 2,000 games and the strongest V13.6 experiment 52.5%, using the reward pipeline that finally worked: win/loss only, spread over the game by GAE. This is the model you play against.
End-to-end
Generate millions of practice games between scripted bots — heuristic, aggressive, defensive, expert. The network learns by watching them play.
The new student network is trained to copy the previous best AlphaLudo's decisions. By the end of this stage it already plays as well as the teacher.
The student plays thousands of games against itself and various opponents, gradually adjusting its strategy to win more. Once it's consistently strong, we add the previous AlphaLudo versions back in as sparring partners.
For most of the project we shaped behaviour with hand-tuned reward penalties — don't ignore the laggard token, don't walk into capture range. The final pipeline drops all of it: the only reward is winning or losing, and GAE (generalized advantage estimation) spreads that single signal fairly across every move of the game. The bad habits disappeared on their own — they were artifacts of the crutches, not the game.
Win rate against scripted bots saturates in the mid-80s, so it stops being useful as a measure. We compare versions directly — thousands of games, head to head, same dice for both sides where possible. That's the only test that distinguishes the strongest models.
From the journal
An early reward-shaping scheme looked elegant on paper but quietly subtracted a tiny amount of reward every turn. Over a 150-move game it added up to about a fifth of a "loss" — the model became convinced every game was unwinnable. Took 155,000 games to figure out what was happening.
In long games, even tiny systematic biases compound. Always check what the reward looks like end-to-end.
Mid-project, quieting the intermediate rewards cratered win rate from 67% to 33%: the dice noise drowned the learning signal, and loud hand-shaped rewards looked like the answer. The real answer arrived later — drop the shaping entirely and let GAE spread the pure win/loss signal across the game. Same architecture, cleaner rewards, stronger play. The shaping was scaffolding, not structure.
Shaped rewards are a way to survive dice noise, not the way to beat it. Credit assignment done properly (GAE over terminal-only reward) replaced every crutch we'd built.
For most of the project we believed the 80–83% plateau was about the opponent pool. Three architectural designs (CNN + attention, pure CNN, a temporal transformer over 8-turn history) all sat at the same ceiling. Then V13.5 broke it — by attacking the input instead of the model: collapsing the four own-token channels into a single permutation-symmetric count view. V15.2 finished the thought — erase token identity entirely, describe each board cell with three numbers, and let attention do the rest. Every step that helped made the input more honest about the game's symmetry, not the model bigger.
When three different architectures hit the same ceiling, the bottleneck is upstream of the architecture. We were giving the model an asymmetric view of a symmetric game.
The strongest models all settle around 85% against the scripted pool — so is that a skill ceiling or a luck ceiling? We tested it six independent ways: head-to-head between champions lands at coin-flip, adding 2-ply search at play time adds nothing, an adversary trained specifically to exploit the model manages 45%, a per-decision "equity loss" audit measures ≈0 regret against rollout ground truth, retraining with richer value targets ties, and human play-testing finds no repeatable pattern to exploit. Six arrows, one answer.
In a dice game, a single game is mostly luck between near-optimal players. ~85% against the pool is the ceiling — the remaining 15% is variance, not weakness. Knowing when to stop is a result too.
Mechanistic interpretability
A sister project — a battery of probing experiments asking: what has the network actually learned? Built around the V13.x generation (five experiments, ~600–2,000 board states each), plus a controlled representation study run on the way to V15. Findings from this work directly shaped the encoders that followed.
V13.5 · CHANNEL ABLATION RE-RUN
Re-ran channel-ablation on V13.5 with 600 stratified states. The four "Tok→Rank" planes (the constant channels that route the rank-indexed output back to which token to actually move) are the dominant channels globally — Policy KL 0.60–0.76, higher than any other channel. In late-game the pattern shifts: own-token-count and the leader-token rank mask take over (KL 0.68–1.01), and safe-zone reasoning becomes critical for landing.
The symmetric-encoder bet isn't just a numerical tie — it's an active mechanism the model uses on almost every decision. Phase-specialized policy: rank routing in opening, leader-token + safe-zone reasoning in endgame. The kind of structure V13.5 was designed to expose.
V13.2 · CHANNEL ABLATION (original finding)
Zero out one input channel at a time and measure how much the policy distribution shifts (KL divergence). On 600 stratified states with the multi-legal filter — i.e. only states where the network actually has a choice — Token 3's channel comes out at KL ≈ 0.85, more than 2× the next-highest token (T1 at 0.38).
This was a surprise. The four tokens are interchangeable under the rules, so a symmetric encoder shouldn't single one out. The asymmetry was real and pointed at the input encoding itself — directly motivating V13.5's token-symmetric collapse, and ultimately V15.2's identity-free encoding.
EXPERIMENT 3 · LINEAR PROBES
Train a logistic regression on the network's GAP features to decode hand-labelled concepts. Numbers are balanced accuracy on a held-out test set; baseline is the chance-level for that label distribution.
Strategic context (phase, lead, who's winning) lives clearly in the features. Per-token spatial concepts (which token is closest, how many are nearly home) don't — the network appears to re-derive these from the input each forward pass rather than maintaining them in the residual stream.
EXPERIMENT 2 · DICE SENSITIVITY
Hold the board fixed, sweep the dice channel through 1–6, see how the chosen token shifts. ~78% of states flip the preferred token when the dice value changes — and the JS divergence between roll-1 and roll-6 distributions is large.
The network behaves as f(board, dice) → action, with dice values acting as broadcast modifiers rather than something integrated into a temporal plan. Same pattern across V6, V10, and now V13.2 — annealed PPO didn't change it. Tree-search-style planning would look very different.
EXPERIMENTS 4–5 · CAPACITY USE
Layer-knockout (skip individual ResBlocks) and channel-activation (which of the 160 channels actually fire) ran together. Result: 0 globally dead channels at any threshold — every channel produces some activation. But channel-importance is heavily long-tailed: a handful of channels dominate the policy gradient, and the bulk are weakly redundant.
The model isn't wasting parameters in the obvious sense (no dead neurons), but it's also not packing them densely. There's likely room to compress 10× without losing strength — a prediction V15.2 later confirmed by matching the champions at a fifth of the parameters.
CONTROLLED A/B · REPRESENTATION
A one-variable experiment: train two identical networks on identical data, one with a learned per-token identity embedding, one without — then linearly decode each token's position and danger from the internal features. Across six seeded pairs, the paired difference in decodability was 0.00 ± 0.01: the identity embedding, though actively used by the network, scrambles nothing. A positive control (forcing a network to store properties in such an embedding) showed the probe detects real scrambling when it exists (r drops to ~0.2).
Free per-object codes only garble a network's knowledge when they store the property; as pure identity tags they're harmless. In practice every strong AlphaLudo dropped them anyway — computing properties from the board beat memorising them, every time.
The scoreboard
Every line here comes from a logged, reproducible run in the training journal.
DEPLOYED · this site
225 board cells as graph nodes, three numbers per cell, attention along the routes, a 225-way "move from this cell" policy. Trained with the pipeline that finally worked everywhere: terminal-only reward spread by GAE. 587K parameters — a fifth of the model it replaced.
Head-to-head results: 52.7% vs V13.5 over 2,000 games (this site's previous champion) · 52.5% vs V13.6, the strongest CNN experiment · 68.3% vs an MCTS baseline. Smallest and strongest model of the project, in the same breath.
DIDN'T WORK · resolved
Three separate bets on "there must be more skill to extract" each came back empty. A temporal transformer over 8-turn history lost 9.6% of head-to-head games against the stateless V13.5 — history adds no signal a single frame doesn't carry. Bolting 2-ply expectimax search onto the deployed model at play time: 49.8%, a tie — the search can't out-rank the policy it reads. Training an adversary against the frozen champion: 45.4% — it found nothing to exploit.
SETTLED · six ways
Champion-vs-champion head-to-heads land at coin-flip. Play-time search adds nothing. A dedicated exploiter fails. A per-decision equity audit against double-sampled rollouts measures ≈0 lost win-probability per move. Richer value targets tie. Human play-testing finds no repeatable flaw. Two-player Ludo, for this project, is closed — the residual is dice.
PARKED · compute-bound
A stripped-down variant — shallow expectimax + the current network as leaf evaluator, distilled into a fresh student — lost 89/10 to its teacher. Not a refutation of AlphaZero; a sign that 2-ply search over the network we're trying to beat doesn't generate strong-enough targets. The full recipe (deep MCTS, millions of games from random init) stays parked until there's a real compute budget — though with the variance ceiling now proven, the expected payoff is small.
Inspirations & dead ends
AlphaLudo borrowed liberally and parked one idea for compute reasons. Here's the full reading list, in chronological order.
The original "neural net plays a dice game at world-class level" result, written when most of the modern field didn't exist yet. Thirty years later, AlphaLudo rediscovered Tesauro's central lesson the hard way: in dice games, the small rewards along the way matter more than the final win/loss signal. Scale them down too far and learning collapses.
Read on Wikipedia →AlphaZero's full recipe — train from a random network, generate millions of self-play games with deep MCTS search at every move, distill the search-improved policy back into the network — is the obvious next step after AlphaGo. We didn't run that loop. The blocker isn't the idea; it's the compute. Generating millions of search-augmented games on a single GPU is months of wall-clock time we don't have.
We did try a stripped-down variant: shallow expectimax search using our best existing network as the leaf evaluator, distilled into a fresh student. That student lost 89 / 10 to its teacher — not a refutation of AlphaZero, but a sign that 2-ply search over the same network we're trying to beat doesn't generate strong-enough targets. With a stronger leaf evaluator (or a real compute budget for full self-play search), it's still on the table.
Original paper →A 2017 idea that lets a neural network reason about a "set" of things — like the four tokens you control — without caring what order they're in. We used this for one experiment in AlphaLudo, building a much smaller network with no convolutional layers at all. It hit the same ceiling as the bigger models, which is what convinced us the model itself wasn't the limit.
arXiv:1703.06114 →The reinforcement-learning algorithm doing the heavy lifting in every AlphaLudo run. PPO is the workhorse of modern RL — boring, reliable, well-understood. We didn't try to be clever with the optimiser; the interesting part of AlphaLudo is what we feed into it, not how we update the weights.
arXiv:1707.06347 →Project meta
A long-running side project on what it actually takes to learn Ludo from self-play. Built end-to-end — engine, training, mechanistic interpretability, and this site.
Runtime
Model
By the numbers
Privacy
Nothing. There is no backend. Your moves never leave your browser. The page loads Google Fonts and (on the Lineage page) one YouTube embed via youtube-nocookie.com; that's the only third-party traffic. No analytics, no cookies, no telemetry.
Connect
Built by Sumit Pal as a side project. The full training pipeline — engine, model code, every experiment, the journal — is open-source. Reach out if you want to chat about it.
Ready?
The network is already loaded. Pick up the dice.