Teaching a network to play a game
nobody has theory for

The variant

Extinction chess keeps every rule of standard chess — castling, en passant, promotion — and replaces the win condition. There is no check and no checkmate. Kings are ordinary pieces: they can be captured, and you can promote a pawn into one.

You win by driving any one of your opponent's piece types to zero. Take their last knight, and the game ends immediately. That single change cascades through everything.

The strategic consequences are strange. Pieces you would happily trade in normal chess become liabilities the moment they're your last of a kind. A rook you'd usually cash in becomes untouchable. And because every piece type is a potential losing condition, the engine has to track seven simultaneous extinction races at once.

Because the variant has no literature, there was no way to check whether the engine was playing well — only whether it was playing better than its former self.

The problem: nothing to learn from

Training a chess engine normally starts with data. Millions of human games, or positions labelled by an engine that already plays well, or at minimum a hand-written evaluation function encoding what centuries of theory established — that a rook is worth five pawns, that doubled pawns are weak.

Extinction chess has none of that. No corpus, no reference engine, no accepted evaluation. A rook might be worth five pawns or it might be a liability, and nobody has worked out which. So before you can train anything, you have to answer an awkward question: what do you regress against?

Bootstrapping from your own search

The answer is to make the network predict the output of a search that used the network. Two targets, one per head:

The first term is the strange one. π is not ground truth — it is the same network's opinion, run through 800 simulations of lookahead. The network is being trained to agree with a slightly better-informed version of itself.

That only works because search genuinely adds information. A single forward pass is one guess; eight hundred simulations actually play moves out, reach positions the network evaluates separately, and feed those results back. The resulting distribution is measurably stronger than the raw policy that seeded it. Distilling it back into the weights makes the next forward pass better, which makes the next search better still. The gap between the network and its own search is the entire training signal.

The value target looks more solid — somebody really did win. But those games were played by this same network against itself, so z only ever answers "what tends to happen when this model plays this model." Nothing in the loop is anchored to an outside standard.

That is the structural risk, and it is not hypothetical. If the model develops a blind spot, self-play will not reveal it: both players share it. Every strength measurement in this project is relative — against older checkpoints, against random play, against a fixed puzzle set — which detects regression reliably and a shared blind spot not at all.

Architecture

The engine follows the AlphaZero template: a single network with a policy head and a value head, guiding Monte Carlo tree search. Both heads train purely on the outcomes of the network's own games.

The input encoding is where the variant shows up. Alongside the usual piece planes, board history, castling rights and move clock, there's a dedicated endangered plane — an explicit signal for which piece types are close to extinction. The network could in principle infer this by counting pieces, but making it an input rather than something to be learned lets the residual tower spend its capacity on strategy instead of arithmetic.

What comes out of it

The tower is ordinary. What matters is the two heads, because they are what the search consumes — and neither one decides a move on its own.

The policy head emits 4,864 logits, which is 76 move planes × 64 origin squares. The plane encodes what kind of move it is — a queen-like ray in one of eight directions, a knight jump, or a promotion — and the square says where it starts. Extinction chess needs more planes than standard chess does, because promoting to a king is legal and has to be representable. Masked to the legal moves and softmaxed, this becomes P(a), a prior probability over moves.

The value head collapses the same tower output to a single number through a tanh, so v ∈ [−1, 1], read from the perspective of whoever is to move. +1 means the player about to move is winning.

The distinction that matters: the policy is a prior, not a decision, and the value replaces playing the game out. Classical MCTS estimated a position by playing random moves to the end thousands of times. Here a single forward pass substitutes for all of that, and the policy tells the search which branches deserve attention first. The search then overrules both — routinely.

Terminal-position drilling

Pure self-play produced a specific, persistent weakness: the engine would build a winning position and then fail to actually take the final piece. Extinction wins are single, exact captures, and they show up too rarely in ordinary self-play for the value head to weight them properly.

The fix was a supplementary training phase. After each iteration's normal training, the trainer generates synthetic near-terminal positions — boards one capture away from extinction — and drills the network on them at a reduced learning rate, with a second, harder pool trained more gently still. It's a targeted correction for a failure mode that self-play alone doesn't sample often enough to fix.

Inside the search

One "simulation" is four steps: walk down the tree to a position nobody has evaluated yet, expand it, evaluate it with one forward pass, and push that result back up to the root. Repeat 800 times and the distribution of visits across the root's children is the move the engine plays — and, during training, the target π that the policy head learns from.

Choosing which branch to walk down

Every step of the descent picks the child with the highest PUCT score:

The second term is what makes this work. It starts large for high-prior moves and decays as that move accumulates visits — the 1 + N(a) denominator means the hundredth visit to a move is worth far less attention than the first. Meanwhile √N(s) grows as the whole search deepens, so exploration never dies off entirely. A move the network dismissed can still get examined, just later.

So the search begins by trusting the network and ends by trusting the results. Early simulations follow the policy almost exactly; after a few hundred, Q dominates and the visit distribution reflects what actually happened when the moves were played out.

Two selections, worked through

Three legal moves, priors 0.60 / 0.30 / 0.10, with c_puct = 2.5. After ten simulations the root has learned something the policy did not know:

B wins, despite half A's prior. A was the network's favourite and got explored first, but seven visits shrank its exploration term and the results came back mediocre. B looks better where it has been tried. C, never tried at all, keeps its full exploration term and is already ahead of A.

Simulation 11 goes to B and the leaf evaluates at −0.50, dragging B's average down:

One bad result is enough to drop B below A, and now C finally gets its turn. That is the self-correcting part: a move only stays favoured while the simulations keep agreeing, and a move the network rated at 10% still gets examined rather than being written off.

Noise, and where it goes

Trained on its own search output, a network can quietly stop considering moves entirely — the prior collapses toward zero, the exploration term never lifts them, they never get visited, and nothing generates evidence to the contrary. During self-play the priors at the root only are perturbed before searching:

Root-only is deliberate. Noise at the root changes which lines get explored; noise deeper in the tree would corrupt the evaluations those lines produce. And it is applied once per root — a detail that turned out to matter more than it sounds, because the search runs in repeated chunks while pondering, and re-drawing the noise on every chunk quietly flattened the priors toward uniform until it was caught.

A last detail that looks like a mistake and isn't: the engine plays the most-visited move, not the highest-valued one. A single simulation returning +0.9 is noise; a move that attracted three hundred visits earned them by repeatedly surviving. Visit count is the more robust statistic, and it is the one that becomes π.

Training at scale

Every iteration plays ~400 complete games against itself, harvests roughly 15,000 positions, and trains on a rolling buffer of the last five iterations. Then it does it again.

That works out to well over 400,000 self-play games and several billion network evaluations, spread across months of cluster time on a mix of RTX 3090, 4090 and A10G nodes. Self-play generation is the bottleneck by an enormous margin — the training step itself takes about two minutes of each three-and-a-half-hour cycle.

The infrastructure is most of the work

A training run this long is really an exercise in operations. Jobs hit SLURM wall-time limits and have to resubmit themselves mid-run, preserving the replay buffer and optimizer state. Nodes fail. GPUs crash mid-benchmark. At one point a driver upgrade drained the node that generated supplementary games, and the loss curves quietly improved — not because the model got better, but because a smaller training set is a cleaner, weaker signal. Reading that correctly, rather than celebrating it, mattered.

A separate incident cost a full iteration of logs: a job died with a node failure, SLURM auto-requeued it, and the requeued run truncated the original log file on open. The fix was one flag — appending rather than truncating — but finding it meant reconstructing what happened from checkpoint timestamps and the raw .npz buffers on disk.

Knowing whether it's improving

With no external opponent to measure against, strength is assessed entirely through self-reference. Every tenth checkpoint triggers an automated battery: head-to-head matches against recent iterations, against distant historical anchors, a fixed tactical suite, and a "win-taking" test that checks whether the engine converts positions where an extinction capture is available.

This is noisier than it sounds. Head-to-head results between adjacent checkpoints sit inside a ±7% noise floor, so a single matchup proves nothing. Distant anchors are the load-bearing signal — if the current model still beats a checkpoint from 900 iterations ago at the same rate, it hasn't regressed, even when neighbouring comparisons look flat.

Reading a plateau correctly

Around iteration 1,020 progress stalled — the model scored 51% against its own predecessor from ten iterations earlier, statistically indistinguishable from a tie. The diagnosis was that MCTS exploration had narrowed: the search kept re-confirming what the network already believed instead of testing alternatives.

Widening the root exploration parameters produced exactly the pattern you'd hope for and exactly the one that looks like failure. Policy loss rose. Head-to-head results against recent checkpoints dipped below 50%. But the tactical suite jumped 15 points, and performance against distant historical anchors held steady. The model wasn't worse — it was playing differently, which depresses head-to-head scores against its own recent lineage. Reverting on the loss curve alone would have thrown away a real improvement.

The bug that was hiding inside the fix

Then, for about twenty iterations, the model started losing to itself. Iteration 1,120 scored 42.5% against iteration 1,110. Iteration 1,130 managed 41.5% against iteration 1,090 and 42.5% against 1,070 — a model consistently worse than versions of itself from forty iterations earlier.

What made it hard to act on is that nothing looked broken. The absolute markers held: still 100% against random play, still ~89% against the iteration-100 baseline, tactical suite steady, win-taking at 94%. Only the comparisons against its own recent lineage were sliding, which is exactly the signature of harmless stylistic drift — the thing the previous section says not to panic about. The case for "wait and see" was genuinely strong right up until the absolute markers began slipping too.

The cause turned out to be the drilling phase described above — the fix for one weakness was quietly creating another. Drilling runs last in each iteration, on curated near-terminal positions whose feature distribution looks nothing like ordinary midgame play. It ran with the network in training mode, so those final batches updated BatchNorm's running mean and variance. At the default momentum of 0.1, the last twenty batches carry 1 − 0.9²⁰ ≈ 88% of the exponential weight in those statistics at the moment the checkpoint is saved.

So every checkpoint was being written with normalisation statistics calibrated to drill positions, then loaded for the next round of self-play where BatchNorm uses exactly those stored statistics to normalise ordinary midgame positions. The network was being quietly miscalibrated against the positions it actually plays, once per iteration, compounding.

The fix is to put the BatchNorm layers in eval mode before drilling: the running statistics freeze, while the learnable scale and bias keep training normally. The next iteration's model.train() restores them, so nothing needs unfreezing.

Iteration 1,140 was the first checkpoint fully trained under the fix, and it produced the largest single-benchmark improvement in the project's history: 82.5% against iteration 1,130, and 25–30 point swings against every recent ancestor. The absolute markers set new all-time peaks — 96.0% against the iteration-100 baseline, beating a record that had stood since iteration 1,070. The record has since been raised again, to 98.5% — set at iteration 1,190 and matched by iteration 1,300, the checkpoint running on this site.

One honest cost came with it. Win-taking — converting positions where an extinction capture is available — dropped from 80% to 72% at low simulation counts. The distorted statistics had been giving the model artificially sharp recognition of immediate wins, and correcting them took that edge away. The bug was doing something useful. It was just doing far more damage everywhere else.

What I'd do differently

Batched evaluation from the start. The evaluator scores one position at a time, which leaves the GPU badly underutilised during self-play. Batching across parallel games is the single biggest available speedup and it's structurally awkward to retrofit.

A fixed external benchmark. Everything is measured against the model's own history, which detects regression well but can't detect a shared blind spot. Even a modest hand-written alpha-beta opponent would have provided an absolute reference point.

A serialisation format on day one. There's still no FEN equivalent for extinction positions, so positions move around as replayed move histories. That was fine for a training pipeline and is the first thing that needs solving to put the engine on the web.