Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzles

1 Introduction

• Existing LRMs struggle with human-solvable puzzles requiring logical reasoning rather than professional knowledge, while prior puzzle work has largely emphasized evaluation.The introduction identifies limited diversity and scalability, with little control over data generation or difficulty.Source: §1 · pp. 1–2
• Enigmata is presented as a comprehensive puzzle suite intended to improve LLM logical reasoning through scalable, controllable data and automatic evaluation.Its generator-verifier design is intended for integration into RL pipelines.Source: §1 · pp. 2–3
• Enigmata-Data covers 36 task types across seven categories, each with a generator and verifier for scalable, difficulty-controlled puzzle generation.The design supports unlimited self-verifying RLVR prompts, difficulty-ratio experiments, and arbitrary per-task sample counts.Source: §1 · pp. 2–3
• Enigmata-Model surpasses state-of-the-art LRMs on Enigmata-Eval and ARC-AGI, with reported generalization to out-of-domain puzzles, mathematics, and STEM.Scaling to Seed1.5-Thinking, the authors report transfer from knowledge-orthogonal puzzle data to advanced math and STEM tasks.Source: §1 · pp. 2–3
• Enigmata pairs its data suite with Enigmata-Eval for comprehensive puzzle assessment and Enigmata-Model, a recipe for training high-performing puzzle solvers.The benchmark is described as challenging even state-of-the-art LRMs.Source: §1 · pp. 2–3

2 Related Work

• Puzzles offer automatically generated, rule-verifiable solutions and controllable difficulty, but prior RLVR research has largely overlooked them in favor of other domains.RLVR assigns rewards from objectively verifiable answers without requiring a reward model [2] [5] [18].Source: §2 · p. 3
• Existing puzzle benchmarks trade off diversity against scalable generation or difficulty control, leaving no comprehensive balance of all three.Prior benchmarks target abstract [9] [12], deductive [10], or compositional reasoning [13].Source: §2 · p. 3
• Puzzle-solving methods have used external tools or RLVR training on single puzzle types, limiting direct internal reasoning enhancement or generalization.Tool-based methods include code and symbolic solvers, while RLVR examples focus on countdown or zebra logic [15].Source: §2 · p. 3

3.1 Puzzle Categories

• Enigmata spans crypto, arithmetic, logic, grid, graph, search, and sequential puzzles, covering distinct forms of reasoning.Examples include decoding, arithmetic under constraints, deduction, grid placement, path-finding, state-space search, and step sequences.Source: §3.1 · pp. 3–4

3.2 Data Construction

• The dataset comprises 36 designed tasks; 30 have custom generators, while six draw from existing datasets, and all 36 have manually validated auto-verifiers.Generator parameters control difficulty; each task has Easy, Medium, and Hard levels based on performance trends across settings.Source: §3.2 · p. 4

3.3 Task Statistics

• Table 1 presents task statistics for Enigmata-Data, including the distribution of tasks across the dataset.The table caption identifies its scope as task statistics; no specific cell values are given in these passages.Source: Table 1 · p. 4, §3.3 · p. 5
• Enigmata is the only compared resource described as covering multiple task categories, offering scalability and automatic verification, and being publicly available.The authors also state that it uniquely uses RLVR to enhance puzzle reasoning.Source: §3.3 · p. 5, Table 2 · p. 5

3.4 Enigmata-Eval

• Enigmata-Eval contains 4,758 puzzle instances rather than the planned maximum of 5,400, with strict separation protocols used to prevent train–evaluation leakage.Some tasks yielded fewer than 50 instances per difficulty level, while others relied on manually collected and annotated data.Source: §3.4 · p. 5

4 Enigmata-Model: The Training Recipe

• The training recipe has two stages: rejection fine-tuning establishes foundational reasoning patterns, followed by multi-task RL to develop skills intended to transfer across domains.The stated aim is to build diverse reasoning skills while avoiding overfitting to specific problem types.Source: §4 · p. 5

4.1 Rejection Fine-tuning

• Rejection fine-tuning uses high-quality solutions in supervised fine-tuning, combining puzzles and mathematics in a balanced 1:1 ratio.Puzzle examples are sampled across tasks and difficulty levels, and ARC-AGI training data are included; DeepSeek-R1 generates eight candidate solutions per puzzle instance, from which a correct solution is selected.Source: §4.1 · p. 5

4.2 RL with Verifiable Puzzles

• Each of the 36 tasks has an automated verifier, and 30 have generators; generated examples receive verifier rewards that VC-PPO uses to update the policy.For fixed-pool tasks, the number of selected examples is capped by the available pool; sample counts can adjust task proportions, difficulty mix, and dataset size.Source: §4.2 · p. 6

4.3 Multi-task Training

• Mix-training combines Enigmata, ARC-AGI 1 and 2 training data, and AIME problems at a 1:1 puzzle-to-mathematics ratio; multi-stage RL introduces Enigmata after ARC-AGI and AIME training stabilizes.The authors explore both strategies because single-task training can lead to narrow expertise and poor transfer, while mixed-task training may involve conflicts between tasks.Source: §4.3 · p. 6

5.2 Results

• Our 32B-parameter model outperforms most public models on Enigmata-Eval and surpasses strong reasoning models on ARC-AGI; after RL training, it maintains comparable math reasoning abilities gained from rejection fine-tuning.Rejection fine-tuning and multi-task RL yield significant performance gains on OOD benchmarks.Source: §5.2 · p. 7

5.3 Generalization with Scaling: Free Lunch from Enigmata

• On the larger Seed1.5-Thinking model, additional Enigmata training generally improves AIME 2024 and 2025, BeyondAIME, and GPQA Diamond performance.The comparison uses the same base model and RL data except for 20K Enigmata-Data, with comparable PPO steps.Source: §5.3 · pp. 7–8, Table 5 · p. 8

5.2 Results

• Crypto, Arithmetic, and Logic are the strongest Enigmata-Eval categories for Qwen2.5-32B-Enigmata, while spatial and sequential tasks remain harder.The model also performs competitively on search tasks, outperforming most baselines.Source: §5.2 · p. 7, Table 4 · p. 8

5.4 Ablation Studies

• Small amounts of second-stage Enigmata-Train improve Enigmata-Eval performance while better preserving first-stage knowledge and OOD performance; excessive data causes catastrophic forgetting and slightly degrades OOD performance.Increasing the data progressively improves in-domain Enigmata-Eval performance.Source: §5.4 · p. 8, Figure 2 · p. 9
• A balanced difficulty ratio of 1:1:1 supports more robust complex reasoning, while difficulty-tag selection matches HRV on Enigmata-Eval and does better on OOD benchmarks.The comparison uses 400 examples per task at each difficulty level and the same stage-1 checkpoint with 150 second-stage steps.Source: §5.4 · pp. 8–9, Table 6 · p. 9
• Multi-Stage RL develops deeper task-specific reasoning and transfers better to unseen complex tasks with limited pre-training data, whereas Mix-Training generalizes better to OOD tasks.The approaches show complementary strengths under equal total training steps.Source: §5.4 · p. 9, Table 7 · p. 10

5.5 Analysis

• For simple tasks, training approaches improve similarly; on Minesweeper, Multi-stage RL learns without SFT while Mix-Training struggles, and Light Up benefits most from comprehensive SFT plus Multi-stage RL.When tasks are included in SFTAll, both approaches start higher and quickly approach perfect accuracy; Mix-Training without relevant SFT fails on Light Up.Source: §5.5 · pp. 9–10, Figure 3 · p. 10
• On ARC-AGI 1, more concise reasoning responses are associated with higher accuracy; code use is associated with worse puzzle-task performance.Mix-Training and Multi-Stage RL reach similar final rewards, but Mix-Training has smoother response-length curves and more consistent outputs.Source: §5.4 · p. 9, §5.5 · pp. 10–11, Table 8 · p. 10

• Enigmata-Data provides 36 tasks across seven reasoning categories with scalable generation, automatic verification, and adjustable difficulty.Enigmata-Eval assesses puzzle reasoning, while Enigmata-Model is trained with RLVR.Source: §6 · p. 11
• The trained model demonstrates superior puzzle performance, robust generalization, and reasoning skills.Source: §6 · p. 11
• For larger models such as Seed1.5-Thinking, synthetic puzzle data brings additional math and STEM benefits over a state-of-the-art model.Source: §6 · p. 11

Core Contributors†

• Jiangjie Chen, Qianyu He, Siyu Yuan, and Aili Chen are listed as core contributors.Source: Core Contributors† · p. 12

Supervisors

• Hao Zhou and Mingxuan Wang are listed under supervisors.Source: Supervisors · p. 12

Affiliation

• The affiliations listed include Tsinghua AIR, Shanghai Jiao Tong University, and SIA-Lab of Tsinghua AIR and ByteDance Seed.Source: Affiliation · p. 12

A Reward Curves Across Individual Tasks in Enigmata

• Reward curves show gradual mastery, difficulty-stratified learning, and stagnant or low learning across different Enigmata tasks.Light Up and Zebra Logic improve steadily; Car Painting, Star Battle, and Hitori separate by difficulty; Big Bench Symbolic and Magic Square show little or no improvement, particularly on challenging settings.Source: A Reward Curves Across Individual… · p. 16
• Light Up and Zebra Logic gain reward steadily, consistent with learning tasks that require complex reasoning chains or integrating multiple constraints.The passage suggests extended exploration progressively refines decision-making on these tasks.Source: A Reward Curves Across Individual… · p. 16
• Car Painting, Star Battle, and Hitori show early learning on easy instances, while medium and hard variants require significantly more training to improve.The authors interpret this separation as evidence that difficulty scaling yields meaningful differences in learning complexity.Source: A Reward Curves Across Individual… · p. 16
• Big Bench Symbolic and Magic Square show little or no improvement across difficulty levels, especially on challenging instances.Sparse rewards, long-term dependencies, or overly complex solution spaces are suggested as possible challenges for PPO without additional guidance.Source: A Reward Curves Across Individual… · p. 16
• The authors suggest using early-mastered tasks as warm-ups and introducing harder tasks later to push reasoning boundaries.They also identify understanding positive transfer and potential interference as important for optimizing multi-task training.Source: A Reward Curves Across Individual… · p. 16

B.1 Rejection Fine-tuning.

• The puzzle RFT set contains 12,041 samples, selected by generating eight DeepSeek-R1 solutions per instance and keeping one correct solution.It samples 1,000 instances from each Enigmata task and also includes synthetic ARC-AGI data.Source: §B.1 · p. 16
• The mathematical RFT set contains 12,533 problems from light-R1 stage 2 and an additional 3,000 sampled from stage 1.These are problems answered by DeepSeek-R1.Source: §B.1 · p. 16

B.2 Reinforcement with Verifiable Puzzles.

• Mix-Training combines 400 Enigmata-Train samples per task, evenly distributed across difficulty, with upsampled ARC-AGI 1 and 2 and AIME 1983–2023 data.Eight Enigmata tasks were excluded: six deemed too difficult, Game24 deemed too simple by pass@k, and FOLIO for lack of training data.Source: §B.2 · p. 17
• Multi-stage RL first trains on upsampled ARC-AGI 1 and 2 plus AIME, then adds Enigmata-Train while retaining those datasets without further upsampling.The second-stage choice is described as intended to prevent catastrophic forgetting.Source: §B.2 · p. 17

C Implementation Details

• VC-PPO is used to train on verifiable puzzles, with modifications intended to improve stability and performance over long sequences.The implementation removes the KL constraint and pretrains the value model using Monte Carlo returns from a fixed SFT policy before initializing the PPO critic.Source: C Implementation Details · p. 17
• Decoupled GAE uses λcritic = 1.0 for unbiased value estimation and λpolicy = 0.95 to improve sample efficiency and learning speed over long token sequences.The model is fine-tuned on the balanced dataset for 2 epochs with a maximum sequence length of 32768 tokens and a learning rate of 1e-5.Source: C Implementation Details · p. 17
• PPO training runs for 425 steps with prompt and response limits of 6,144 and 26,624 tokens, respectively, using batch sizes of 4,096 and 512.Actor and critic learning rates are 1 × 10^-6 and 2 × 10^-6; value pretraining precedes PPO, and rollouts use temperature τ = 1.0.Source: C Implementation Details · p. 18

D Baselines

• Evaluation compares general-purpose LLMs and reasoning-specialized models, including GPT-4o-1120 and o1, using temperature sampling τ = 1.0.The baseline descriptions also include Gemini 2.5 Pro and reasoning models such as DeepSeek-R1.Source: D Baselines · p. 18

Limitations

• Training was limited to Qwen backbones and VC-PPO, and Enigmata contains single-turn textual puzzles rather than multi-turn or visual puzzles.The authors also note that they trained with only hundreds of competitive math problems alongside Enigmata-Data.Source: Limitations · p. 18

E Enigmata Details

• The appendix's Enigmata details cover task specifications, difficulty estimation methodology, and example cases.The supplied figure caption describes reward curves by task, training step, average reward, and difficulty color.Source: Figure 6 · p. 19, E Enigmata Details · p. 20

E.1 Task Details

• The task-detail tables cover 36 tasks, listing each task’s category, data source, difficulty-control variables, and rules.The tables present Enigmata’s seven reasoning categories.Source: §E.1 · p. 20, Table 10 · p. 21, Table 11 · p. 22

E.2 Difficulty Estimation

• GPT-4o pass@k at k=1, 10, and 100 is used to establish easy, medium, and hard puzzle tiers.Table 13 provides examples at different difficulty levels.Source: §E.2 · p. 20, Table 13 · p. 23
• GPT-4o pass@k performance clearly distinguishes easy, medium, and hard levels across various tasks.The passage reports distinctions but does not provide specific scores.Source: §E.2 · p. 20

E.3 Task Cases

• Magic Square requires equal row, column, and diagonal sums without duplicating filled numbers, while Sudoku examples enforce nonrepeating values in rows, columns, and subgrids.The listings include 3x3 Magic Square, 4x4 Sudoku, and 9x9 Sudoku cases.Source: §E.3 · pp. 25, 28
• Skyscraper puzzles use row-and-column uniqueness plus visibility counts, while Sum Skyscraper clues total the heights of visible buildings.The examples distinguish counting visible buildings from summing their heights.Source: §E.3 · pp. 25–26
• Star Battle requires one nonadjacent star per row and column; Campsite pairs each tree with a tent under adjacency and row, column, and total-count constraints.Star Battle also forbids diagonal adjacency; a tent may be adjacent to multiple trees but is associated with only one.Source: §E.3 · pp. 24, 27
• Crosswords require valid across and down words whose intersecting letters agree, whereas Symbolic Hard asks solvers to infer a pattern and apply it to a test case.The crossword rules map across clues by row and down clues by column.Source: §E.3 · pp. 29–30
• Crypto KKA asks for ciphertext decryption from supplied rules, while Stack Permutation tests whether an output sequence can be generated through push and pop operations.The example describes stack operations as adding input elements and removing them from the stack’s top.Source: §E.3 · pp. 31, 33
• Car Painting seeks an ordering that minimizes color switches while limiting each car’s movement to K positions; Twiddle restores a 3x3 sliding puzzle through rotations.The car sequence can be rearranged only within the specified range of its original positions.Source: §E.3 · p. 32
• Hitori removes duplicate numbers while keeping black cells nonadjacent and white cells connected; the minesweeper case asks for mine locations that can be definitively inferred.Hitori requires reporting coordinates of blacked-out cells.Source: §E.3 · pp. 36, 38
• The cases include solving Slant, checkmate-in-one, Tic Tac Toe, and maze puzzles.The listings identify these task types but provide little further case detail in this section.Source: §E.3 · pp. 38, 39, 42
• Game24 requires using each supplied integer once with arithmetic operators to form 24; Countdown similarly targets a number using five integers, with positive-integer intermediate results.Both cases permit parentheses; Game24 allows four, five, or six integers, while Countdown specifies five.Source: §E.3 · p. 40
• The graph cases ask whether an undirected graph contains a Hamiltonian cycle visiting every vertex once and returning to the start, or a Hamiltonian path visiting every vertex once.The path case requests a vertex-index list when one exists and "NO" otherwise.Source: §E.3 · pp. 40–41
• NL Navigation asks for the shortest route from a binary-tree starting landmark to the nearest landmark of a specified type.The cases use 7–10 landmarks and request the path as uppercase landmark letters.Source: §E.3 · pp. 41–42
• Knights-and-knaves cases use truth-telling knights and lying knaves to assess a statement, while FOLIO prompts classify a conclusion as true, false, or unknown.The sample asks whether Ted is a knight; the specified knights-and-knaves output options are Entailment, Contradiction, and Unknown.Source: §E.3 · p. 43
• Zebra Logic requires deducing a consistent grid arrangement from background information and clues, then presenting the answer in the prescribed table format.The instructions specify a Year, Wine, and Type table and prohibit transposing the given uncompleted table.Source: §E.3 · p. 44