Amortized Planning with Large-Scale Transformers: A Case Study on Chess

• Chess is a landmark planning problem where naive search is computationally intractable and brute-force memorization is futile, even at scale.Source: §1 · p. 1
• The paper asks whether scaling modern neural networks can push amortized planning far enough for searchless engines to match engines using test-time search.Source: §1 · p. 2
• ChessBench contains 530M board states from 10M Lichess games, annotated by Stockfish 16 with state-values, best actions, and 15B action-values.Source: §1 · p. 2
• The largest transformers generalize to novel boards, achieve Lichess blitz Elo 2895 against humans, and solve puzzles up to Lichess Puzzle Elo 2867 without test-time search.Source: §1 · p. 2
• The authors perform ablations across dataset and model size, architecture, and prediction targets, and release the dataset, weights, and training and evaluation code.Source: §1 · p. 3

2.1 ChessBench Dataset

• The training dataset uses 10 million Lichess games downloaded in February 2023; Stockfish 16 annotates states and every legal action with values, using 50 ms per board and per state-action pair, respectively.The largest training dataset contains 15.3B action-value estimates and approximately 530M state-value estimates and oracle best-actions. The puzzle test set contains 10K challenging board states.Source: §2.1 · p. 3

2.2 Data Preprocessing and Training

• FEN positions are flattened to fixed-length inputs, while move actions use UCI tokens; this representation omits history needed to capture threefold-repetition draws.Predictors use decoder-only transformers, and value targets are binned into 128 classes by default; training minimizes cross-entropy with HL-Gauss smoothing for value prediction.Source: §2.2 · pp. 3–4

2.3 Predictors and Policies

• The paper compares action-value prediction, state-value prediction, and behavioral cloning, deriving policies by maximizing predicted action-values, minimizing successor-state values, or selecting the highest-probability action.For state-value policies, each legal move is evaluated through its resulting state, from the opponent’s perspective.Source: §2.3 · pp. 4–5

2.4 Evaluation

• Evaluation measures best-action accuracy, Kendall’s τ action-ranking correlation, exact puzzle-solution accuracy, and Elo from tournaments and Lichess games.Puzzle accuracy requires matching the full solution sequence; Lichess games include human-only and bot-only opponents.Source: §2.4 · p. 5

2.5 Engine Comparisons

• Comparisons include Stockfish, AlphaZero, Leela Chess Zero, and GPT-3.5, but cross-engine conclusions are limited by differences in history access, training, and test-time search.Tournament Elo and Lichess Elo are distinct measures and cannot be directly compared.Source: Table 1 · p. 6, §2.5 · p. 6

3 Case-Study Results

• The results use separate large-scale settings for main results and more computationally efficient settings for ablations.The main results cover Sections 3.1 and 3.2; ablation settings cover Sections 3.3 and 3.4.Source: §3 · p. 6

3.1 Main Result

• 2895 Lichess blitz Elo against humans places the 270M model in grandmaster territory; scores consistently improve with model size across metrics.The models were trained on the full 10M-game dataset, and none of the three tested sizes overfit that training set.Source: §3.1 · p. 6

3.2 Puzzles

• The 270M model nearly matches AlphaZero with tree search on 10K Lichess puzzles, though Stockfish leads across difficulty categories.Puzzle solving requires a correct move sequence, which the searchless policy must produce using greedy value estimates.Source: §3.2 · p. 7, Figure 2 · p. 7

3.3 Scaling Analysis

• Larger datasets consistently improve final puzzle accuracy, while larger architectures generally improve performance; ≥7M models overfit with 10K games, but not with 100K or 1M.Puzzle accuracy correlates with other metrics and overall playing strength.Source: §3.3 · p. 7

3.4 Variants and Ablations

• Ablations favor action-value prediction, HL-Gauss loss, depth 16, and 128 value bins among the tested configurations.Action-value prediction leads on action-ranking, action accuracy, and puzzle accuracy, potentially because it uses more training data; HL-Gauss outperforms cross-entropy and L2 loss. Performance rises with depth but saturates around 16 layers, while more bins help up to a point.Source: Table 2 · p. 8, §3.4 · p. 8

4 Discussion

• Without game history, state-based predictors cannot detect threefold-repetition risk; checking the next move and assigning it 50% win probability reduces such draws but cannot plan ahead to avoid them.Source: §4 · p. 8
• When several moves receive maximum value, the policy may switch strategies rather than finish a winning game, risking a draw or loss from accumulated mistakes.The workaround checks whether all top five predicted moves exceed 99%, verifies this with Stockfish, then uses Stockfish’s top move for consistency.Source: §4 · p. 9
• The human–bot Elo discrepancy has several proposed explanations: human resignations, potentially miscalibrated player pools, and tactical mistakes that bots may punish more severely.The authors say investigating this discrepancy is not central and does not affect their main claims.Source: §4 · p. 9

4.1 Limitations

• The largest model does not close the gap to Stockfish 16, and it remains unclear whether scaling or other innovations would close it.Source: §4.1 · p. 9
• The models are impractical in speed-limited computer chess tournaments, and “grandmaster-level” play is established only against human opponents on Lichess.Only the biggest model was evaluated against humans there; repeated play could expose weaknesses in its deterministic policy.Source: §4.1 · p. 9
• A direct engine comparison is qualified by differences in history inputs, training protocols, and test-time search.Source: §4.1 · p. 9
• Leela Chess Zero’s self-play/RL networks achieve higher Elo without explicit test-time search, while the project targets strong chess performance at low computational cost through domain-specific adaptations.The paper refers readers to [6] for a detailed comparison.Source: §4.1 · p. 9

5 Related Work

• Early computer chess paired explicit search with heuristics, progressing from Turing’s explorations and NeuroChess to Deep Blue and early Stockfish.Source: §5 · p. 10
• AlphaZero [7] and Leela Chess Zero used deep reinforcement learning with Monte Carlo tree search, followed by work on self-play enhancements [33] and model-free reinforcement learning [23].Source: §5 · p. 10
• ChessFormer models comparable in size to the paper’s models outperform its vanilla transformers while using fewer FLOPS through domain-specific adaptations [6].Source: §5 · p. 10
• Large-scale game datasets also supported supervised or unsupervised learning; Maia used behavioral cloning on 12M human games to predict players’ moves rather than distill Stockfish.Source: §5 · p. 10
• Foundation-model work has addressed chess rules, move quality, playing strength, state tracking, and play from visual inputs, with chess-specific fine-tuning also improving performance.Source: §5 · p. 10

6 Conclusion

• ChessBench supports supervised distillation of an approximation of Stockfish 16 into feed-forward transformers, with strong generalization and chess play emerging at sufficient dataset and model scale.The authors frame this as evidence that large transformers can approximate sophisticated algorithms, while noting that closing the remaining performance gap may require architectural innovations.Source: §6 · p. 10

7 Impact Statement

• The authors argue that chess has limited real-world impact and that supervised single-task training requires a strong oracle, limiting prospects for rapidly introducing substantial unknown capabilities.Source: §7 · p. 10

A.1 Tokenization

• FEN board tokenization uses 64 characters, padded fields for castling and move counters, and a fixed representation for empty squares and en passant targets.Source: §A.1 · p. 14

A.2 Main Setup

• The main experiments train for 10 million steps on 10M games and vary architecture across three decoder-only transformer configurations.The setup specifies batch size 4096, action-value prediction with 128 return buckets, and evaluation on 1k games and 10k puzzles; the three configurations vary embedding dimension and layer count.Source: §A.2 · p. 14

A.3 Ablation Setup

• All ablations use the same basic setup and vary only the ablation parameters.The setup trains an action-value policy on 1 million games for 5 million steps, using a batch size of 1024 and 3.19 epochs.Source: §A.3 · p. 14

A.4 Dataset Statistics

• Stockfish annotation time-outs occasionally cause state records to be dropped from the dataset.The passage attributes the time-outs to the authors’ internal infrastructure setup.Source: §A.4 · p. 14

A.5 Playing-Strength Evaluation

• The largest model’s human-only Lichess evaluation was used for better calibration because standard Elo ratings were dominated by bot games.In standard evaluation, fewer than 4.5% of games for the 270M model were against humans.Source: §A.5 · p. 14

A.6 Stockfish Setup

• The Stockfish oracle evaluates each legal move for 50 ms, without clearing its cache between moves.This comparison setup differs from standard 50 ms per move because each legal move receives 50 ms of evaluation.Source: §A.6 · p. 15

A.7 AlphaZero Setup

• AlphaZero was evaluated as policy-only, value-only, and with 400 MCTS simulations; its network had been trained on 44M games.The authors used the 2020 AlphaZero version and network.Source: §A.7 · pp. 15–16

A.8 Leela Chess Zero Setup

• Leela Chess Zero was evaluated as policy-only, value-only, and with 400 MCTS simulations using its T82 Large network.The network architecture and training protocol for T82 were not fully available to the authors.Source: §A.8 · p. 16

A.9 Computational Resources

• Large models used 128 TPU V5 chips each, while each Elo-tournament agent used one TPU V3 chip.Ablation experiments used four TPU V5 chips per model.Source: §A.9 · p. 16

B.1 Additional Ablations

• The 0.05-second Stockfish annotation limit gives only marginally worse puzzle performance and is chosen as a compromise between computation and model performance.Annotating the 10M-game dataset requires about 15B action-evaluation calls, or roughly 8680 days of unparallelized Stockfish evaluation.Source: §B.1 · p. 16
• Uniform sampling from filtered training boards performs significantly better than weighted sampling that restores the natural game distribution.The authors hypothesize that greater data diversity under uniform sampling explains the improvement; common openings are seen only a handful of times in training.Source: §B.1 · pp. 16–17
• The tested vanilla convolutional model does not achieve strong results, but the authors did not extensively tune its setup.They expect more domain-specific architectures and training setups to perform better, so the experiment compares vanilla configurations.Source: §B.1 · p. 17

B.2 Inference Times

• Searchless agents are orders of magnitude faster than agents using tree search at test time.Value networks are slower because they evaluate every legal action sequentially before taking the maximum.Source: §B.2 · p. 17

B.3 Predicting Legal Moves

• The large models show no overfitting, and larger models improve both training and test loss.On the smallest training set, models with at least 7M parameters begin to overfit; larger training sets avoid this in the reported scaling experiment.Source: Figure A2 · p. 18
• Behavioral cloning almost always predicts legal moves, including on out-of-distribution puzzles, whereas action-value prediction is not trained to define outputs for illegal moves.Legal-move accuracy is undefined for state-value prediction because it is evaluated only on legally reachable boards.Source: §B.1 · p. 17, §B.3 · p. 18

B.4 Loss Curves

• Larger models improve training and test loss, except when models with at least 7M parameters overfit on the smallest training set.With model size held constant, larger training sets improve test loss.Source: §B.4 · p. 18

B.5 Predictor-Target Comparison

• With equal game counts, action-value learning outperforms state-value and behavioral cloning in the long run, but action-value’s advantage over state-value largely disappears when data points are equalized.Action-value training uses roughly 1.6B values versus roughly 55M states or best actions; at equal data points, state-value slightly outperforms action-value except on Kendall’s τ action-ranking correlation.Source: §B.5 · p. 19, Figure A4 · p. 19, Table A4 · p. 19
• Behavioral cloning performs below state-value prediction despite comparable data-point counts; the authors hypothesize that best-action labels discard information contained in values.Training on the oracle’s full action distribution rather than only its best action is suggested as a way to close the gap, but is outside this paper’s scope.Source: §B.5 · pp. 19–20

B.6 Fischer Random Chess

• The 9M transformer reaches 1539 Elo against bots in Fischer random chess, below its 2054 blitz Elo in classical chess, suggesting difficulty with this distribution shift.The authors use Fischer random chess as a measure of out-of-distribution generalization.Source: §B.6 · p. 20

B.7 Tactics

• The 270M transformer sacrifices material when doing so is advantageous for building a longer-term advantage.This behavior was observed in analysis of a game against a human rated 2145 blitz Elo.Source: §B.7 · p. 20

B.8.1 King weakening game

• 17 .. Bg5 encouraged the opponent to weaken their king position.Source: §B.8 · p. 20

B.8.2 Wing pawn push game

• The agent pushes wing pawns in the middlegame when conditions permit, a motif the reviewers associate with recent neural engines [7].Source: §B.8 · p. 20

B.8.3 Exchange sacrifice game

• In game B.8.3, the agent executes a correct long-term exchange sacrifice.Source: §B.8 · p. 20

B.8.4 Long term sacrifice game

• In game B.8.4, a back-rank pin supports a pawn sacrifice for long-term pressure.Source: §B.8 · pp. 20, 22

B.8.5 Expose king game

• The piece sacrifice in game B.8.5 is not justified by Stockfish, though the opponent blunders six moves later.Source: §B.8 · p. 22

B.8.6 Stockfish disagreement game

• In game B.8.6, masters judged Stockfish correct about 18 .. Rxb4; the agent reversed its opinion on the next move.Source: §B.8 · p. 22

B.8.7 Blunder game

• 2895 Lichess blitz Elo against humans contrasts with an estimated 2299 against engines, where tactical blunders are often refuted.The paper gives Bxh3 followed by g4 as one example.Source: §B.8 · p. 23

2. Limitations

• The paper reports no theoretical results or proofs.Source: §2 · p. 25
• Experimental details are provided in Section 2 and Appendix A, while compute resources are discussed in Section 4.1.Source: §2 · p. 25
• The authors report open-sourcing the dataset, model weights, and code.Source: §2 · p. 25