PaperlayerView exact revision on arXiv

Source-linked AI summary

The Imitation Game: When LLMs Learn to Reason Like Programs via Code-Centric Reasoning Data Synthesis

Jinyang Zhang, Weibin Liao, Keqin Bao, Sihang Li, Shaobo Wang, Muyang Ye, Hongxin Ding, Yue Fang, Tianyi Tang, Fei Huang, Kexin Yang, Xingzhang Ren, Dayiheng Liu

arXiv:2609.16076v1cs.CLcs.AI

TL;DR

LLMs can be strong at programming yet struggle with deterministic, fine-grained natural-language reasoning, and transferring code’s procedural rigor without triggering memorized coding shortcuts remains an open question. MIMIC uses deterministic code execution to synthesize reasoning trajectories, and the paper reports gains across general reasoning, mathematics, and fine-grained symbolic tasks. Its data and synthesis setting is currently limited to Codeforces problems, a subset of pretraining corpora, and about 20 contextualized problems per seed.

  • Problem

    LLMs’ programming strengths do not reliably extend to deterministic natural-language reasoning, and transferring code’s procedural rigor without prompting memorized coding patterns remains an open question.

    Source: §1 · pp. 1–2

  • Method

    MIMIC synthesizes natural-language reasoning trajectories by separating language-model articulation of logic from computation guaranteed by deterministic code execution.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    MIMIC-trained SFT and GRPO models achieve consistent gains across mathematical, general-reasoning, and fine-grained symbolic benchmarks.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The paper concludes that translating executable code and state transitions into natural-language trajectories can enhance mathematical and logical capabilities.

    Source: §5 · p. 9

    The current data uses only Codeforces problems, a subset of pretraining corpora, about 20 contextualized problems per seed, and Qwen3.5-27B for synthesis.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large Language Models (LLMs) excel at programming tasks but frequently fail at deterministic, fine-grained reasoning in natural language, relying heavily on semantic approximations rather than robust symbolic execution. To bridge this gap, we propose MIMIC, a framework that leverages executable code as a rigorous medium for reasoning data synthesis. MIMIC fundamentally transforms algorithms into verifiable reasoning trajectories through narrative fusion, code-guided test synthesis, and dynamic code instrumentation. Crucially, these explicit intermediate execution states naturally form a Code-Instrumented Reward (CIR), providing dense, high-fidelity process supervision for reinforcement learning without external reward models. Extensive evaluations reveal that models trained via SFT and GRPO on our synthesized dataset achieve substantial, consistent gains. Our method significantly elevates accuracy across general reasoning, complex mathematical benchmarks, and fine-grained deterministic tasks, demonstrating that the procedural rigor of executable code can effectively unlock and enhance the generalized reasoning capabilities of LLMs. Our code and data are available at https://github.com/zjy1298/MIMIC.

1 Introduction

MIMIC addresses the gap between LLMs’ programming strengths and their failures on deterministic natural-language reasoning by using executable code to synthesize reasoning trajectories. The paper reports gains across general reasoning, mathematics, and fine-grained symbolic benchmarks.

1 Introduction

  • LLMs can excel at complex programming yet fail deterministic natural-language tasks, suggesting approximate pattern matching rather than internalized symbolic execution.Examples include comparing 9.9 with 9.11, tracking state transitions, and counting characters.Source: §1 · p. 1
  • The research question is how to transfer executable code’s procedural rigor into natural-language reasoning while preventing shortcuts based on memorized coding patterns..Source: §1 · p. 2
  • MIMIC separates language-model articulation of logic from deterministic code execution, grounding generated reasoning in computation.Its stated innovations are narrative fusion, code-guided test synthesis, and trace-grounded chains of thought.Source: §1 · p. 2
  • Narrative fusion removes programming artifacts and combines algorithmic logic with real-world contexts to create natural-language exam questions.This is intended to reduce recall of programming templates.Source: §1 · p. 2
  • Code-guided tests target control-flow boundaries, while instrumented execution supplies intermediate states that ground reasoning and provide dense process rewards.The reward signal does not require a separate reward model.Source: §1 · p. 2
  • MIMIC-trained models show consistent gains across mathematical, general-reasoning, and fine-grained symbolic benchmarks.The introduction reports improvements for both SFT and GRPO models trained on the synthesized data.Source: §1 · pp. 2–3

2 Related Work

Related work covers synthetic reasoning data and verifiable-reward training, uses of code to support reasoning, and failures on tasks that are simple for deterministic programs. MIMIC distinguishes itself by transplanting executable logic into natural narratives while retaining verifiable structure.

2 Related Work

  • Prior synthetic-data and verifiable-reward work often uses mathematical verifiers or separate process reward models, whereas this paper uses programs for gold outputs and intermediate signals.The cited areas include mathematical-domain data synthesis, rule-based logic and puzzle generation, and outcome- or process-based reinforcement learning.Source: §2 · p. 3
  • Prior code-assisted reasoning includes inference-time tools, programming-benchmark evaluation, and input–output tasks; MIMIC instead places executable logic in natural narratives.This is intended to reduce programming-specific biases while preserving verifiable reasoning structure.Source: §2 · p. 3
  • LLMs remain weak on basic string, character-level, and text-maze tasks, with some studies attributing these failures partly to tokenization mismatches.These code-trivial but reasoning-fragile tasks motivate program-grounded supervision.Source: §2 · p. 3

3 Methodology

MIMIC turns competitive-programming problems into verified natural-language reasoning pairs through a four-stage pipeline, from kernel extraction and narrative synthesis to targeted inputs and trace-grounded reasoning.

3.1 Framework Overview and Notations

  • MIMIC sequentially classifies and extracts a problem’s mathematical kernel, fuses it with a narrative, synthesizes inputs, and produces trace-grounded reasoning.The stages project the algorithmic logic away from recognizable programming artifacts while retaining executable reasoning and deterministic intermediate states.Source: §3.1 · p. 3, Figure 2 · p. 4

3.2 Stage 1: Problem Classification and Mathematical Kernel Extraction

  • MIMIC assigns each problem a reasoning category and extracts a kernel that preserves the accepted code’s constraints, boundary conditions, and objective.The resulting representation combines the kernel, problem type, and accepted code.Source: §3.2 · p. 4

3.3 Stage 2: Problem Synthesis via Narrative Fusion

  • MIMIC samples a type-compatible educational context and fuses it with the extracted kernel to create a refined natural-language exam question.The story-wrapped question implicitly embeds a valid default input.Source: §3.3 · pp. 4–5

3.4 Stage 3: Code-Guided Test Synthesis and Narrative Input Fusion

  • MIMIC inspects code control-flow decisions to generate adversarial and guided-random inputs, then integrates each input into the narrative question.This targets distinct execution paths beyond the initially embedded default input while preserving the story context.Source: §3.4 · p. 5

3.5 Stage 4: Trace-Grounded Chain-of-Thought Synthesis

  • MIMIC instruments accepted code and executes it to obtain ground-truth outputs and intermediate states, which ground the generated reasoning pairs.The synthesis function translates execution traces into natural-language chain-of-thought linking each instantiated question to its output.Source: §3.5 · p. 5

4 Experiment

The experiments evaluate MIMIC across general, mathematical, coding, and fine-grained deterministic tasks, and test its synthesis choices and reward strategy. Results report gains across reasoning domains and show that narrative fusion, execution-trace supervision, and CIR are central to the evaluated approach.

4.1 Experimental Setup

  • The evaluation uses Qwen3-4B, Qwen3-8B, and DeepSeek-R1-Distill-8B with SFT and GRPO across general, math, coding, and fine-grained benchmarks.The setup also uses MIMIC alongside mathematical datasets to examine composability.Source: §4.1 · p. 5

4.2 RQ1: Generalization of Reasoning Capabilities

  • Qwen3-4B GRPO raises general and mathematical averages by 3.8 and 4.8 points; DeepSeek-R1-Distill-8B gains 9.7 math-average points.Qwen3-4B AIME rises from 15.6 to 28.3, while Qwen3-8B MBPP rises from 40.5 to 44.0.Source: Table 1 · p. 6, §4.2 · p. 6
  • Raw Codeforces SFT lowers Qwen3-4B general reasoning by 16.5 points and math by 21.6 points, unlike contextualized MIMIC synthesis.The authors attribute the contrast to syntax overfitting in raw code data and transferable reasoning from narrative fusion.Source: §4.2 · pp. 6–7
  • Mixing MIMIC with mathematical data yields Qwen3-4B scores of 93.2 on GSM8K and 30.0 on AIME without compromising general reasoning.GRPO, GPG, RLOO, and Reinforce++ also produce largely consistent gains over the base model.Source: Table 1 · p. 6, §4.2 · p. 7
  • MIMIC outperforms TeaR, CoE, ExecGrounded, SynLogic, and Enigmata across most evaluated settings under SFT and GRPO.The comparison is reported in Table 2.Source: §4.2 · p. 7, Table 2 · p. 8

4.3 RQ2: Solving Fine-grained, Rule-based Deterministic Problems

  • Qwen3-8B GRPO reaches 40.0% on MazeBench, 39.5% on StringBench, and 99.2% on CharBench, with none of these benchmark formats used in training.The authors describe these as zero-shot generalizations to deterministic rules.Source: §4.3 · p. 7, Table 3 · p. 8

4.4 RQ3: Reasoning vs. Pattern Matching

  • With the fused format, code rate drops to 0.7% and reasoning accuracy rises to 42.2%, versus 23.8% for the original format, indicating the model is driven toward step-by-step deduction rather than shallow pattern matching.The unconstrained fully synthesized format yields 61.8% accuracy but produces overly simple, stereotyped test cases.Source: §4.4 · p. 8

4.5 RQ4: Effectiveness of the Process Reward

  • CIR achieves gains of 5.0 points on general reasoning and 3.8 on math over baseline, surpassing binary reward and learned process reward models.The method uses deterministic execution checkpoints for dense, graded rewards; the authors report that learned PRMs struggle with their structured algorithmic trajectories.Source: §4.5 · pp. 8–9, Table 5 · p. 9

5 Discussion and Conclusion

The discussion frames code as a cognitive scaffold for grounding language reasoning in verifiable logic and as a source of step-level process supervision. The conclusion presents MIMIC as a framework for translating executable code and state transitions into natural-language reasoning trajectories.

5 Discussion and Conclusion

  • Program execution states provide cost-free, high-fidelity step-level verification, offering process supervision without human annotations or closed systems.The discussion presents this as a scalable pathway for process reward models.Source: §5 · p. 9
  • The discussion casts programs as a cognitive scaffold that connects symbolic execution with semantic understanding to mitigate hallucination.It argues that deterministic program structure can ground linguistic reasoning in verifiable logic.Source: §5 · p. 9
  • MIMIC translates executable code and state transitions into natural-language reasoning trajectories to improve mathematical and logical capabilities.The conclusion describes this as a self-verifiable, sustainable data engine for more interpretable and reliable AI reasoning.Source: §5 · p. 9

Limitations

The limitations identify a simple, equally weighted reward formulation and a bounded data-synthesis configuration. The appendix describes the pipeline’s synthesis stages and its initial problem-classification and mathematical-kernel extraction steps.

Limitations

  • CIR uses a simple k/K step-counting reward that weights all execution checkpoints equally; the authors do not propose it as an optimal reward model.The stated aim is to demonstrate the effectiveness of program-derived intermediate results as training signals.Source: Limitations · p. 9
  • The current data configuration uses only Codeforces problems, a subset of pretraining corpora, about 20 contextualized problems per seed, and Qwen3.5-27B for synthesis.The authors state that this configuration supports approximately 114K SFT samples and 10K RL prompts.Source: Limitations · p. 9

A Data Synthesis Pipeline

  • The synthesis pipeline has separate stages for narrative-background synthesis, code-guided test-case synthesis, and trace-based chain-of-thought synthesis.Execution checkpoints in the traces provide process-level reasoning signals.Source: A Data Synthesis Pipeline · p. 14

A.1 Stage 1: Problem Classificaton and Mathematical Kernel Extraction

  • Stage 1 classifies problems by reasoning skill, then removes narrative and contest framing while preserving task semantics and constraints.The extraction step produces a concise mathematical kernel for later reasoning-focused synthesis.Source: §A.1 · pp. 14, 16

A.2 Stage 2: Problem Synthesis — Story Wrapping

This section describes how executable problems are wrapped in coherent narratives and expanded with code-guided tests while preserving their logic. It then instruments correct programs to produce verified execution traces that ground stepwise natural-language reasoning trajectories in the story and can support SFT and process-level RL supervision.

A.2 Stage 2: Problem Synthesis — Story Wrapping

  • Type-aware matching pairs each problem only with passages suited to its reasoning structure, drawing background text from FineWebEdu’s diverse educational and informational corpus.Passages are screened for structures such as coherent prose, extractable numbers, structured records, or explicit sequential rules, reducing forced or artificial story connections.Source: §A.2 · p. 17
  • Each rewrite uses the paired passage as its narrative world, removes programming traces, and preserves the original problem’s mathematical structure, decision rules, and stopping conditions.The question must include all information needed to solve it in prose, with contextual detail that makes it resemble a genuine reading-based exam task.Source: §A.2 · pp. 18–19
  • The process generates 20 story variants per source problem, yielding roughly 200K questions that retain the same executable logic while varying domain, discourse, and surface realization.This variation is intended to increase the diversity and generalizability of the training data.Source: §A.2 · p. 19
  • Refinement reduces average problem length by approximately 35% while noticeably improving narrative coherence.The second pass targets drafts that read as corpus text followed by an abruptly attached question, rewriting them as a unified exam item while preserving the hidden input exactly.Source: §A.2 · p. 19
  • Representative Type 2 and Type 4 examples illustrate how refinement preserves executable inputs while making stitched-together or context-heavy drafts more coherent and exam-like.The examples target a visible seam between borrowed context and a decoding task, and the risk that long descriptive context can overwhelm a data-processing question.Source: §A.2 · p. 20
  • Story wrapping removes abrupt narrative seams while preserving the hidden input, as archival telegraph decoding and butterfly conservation funding become naturally motivated tasks.The refinement also embeds the survey log and funding amounts directly in the conservation scenario.Source: §A.2 · p. 21

A.3 Stage 3: Test Case Synthesis and Narrative Input Fusion

  • Because each story-wrapped problem supplies only one instance and corpus or released judge data can lack useful edge cases, MIMIC synthesizes additional inputs from accepted-code decision points.The resulting small, sharp tests target logical vulnerabilities rather than raw computational burden.Source: §A.3 · pp. 21, 24
  • Every synthesized input is run against the original accepted code for a deterministic gold output, then its values are woven into the existing story while its scenario and question logic remain unchanged.The rewritten question must expose all new input values and update dependent narrative details consistently.Source: §A.3 · p. 23

A.4 Stage 4: Code Instrumentation, Trace Collection, and CoT Synthesis

  • Stage 4 exposes accepted code’s process supervision by adding sparse, read-only diagnostic traces at meaningful execution points and converting them into natural-language reasoning trajectories.Instrumentation adds only printing statements, without modifying, deleting, or reordering original code, and avoids tracing immaterial computations.Source: §A.4 · pp. 24–25
  • Read-only instrumentation adds structured [TRACE] output at selected checkpoints, and execution on verified inputs yields intermediate states while preserving the original answer when trace lines are filtered.The prompt limits traces to 20 lines and favors event-driven output; the instrumented code must remain runnable and preserve the original program’s output after filtering traces.Source: §A.4 · pp. 25–26
  • Before generating reasoning, the pipeline checks that the trace-stripped execution output matches the known answer and discards samples that fail verification.For retained samples, a model generates a numbered reasoning chain whose steps each perform one logical operation and whose final step states the answer.Source: §A.4 · p. 26
  • The traces guide the explanation implicitly, while each reasoning step uses the story’s terminology and avoids programming or trace references.The prompt asks for informative checkpoints, explicit computations, and a clear final answer; the resulting format supports SFT and step-level reward assignment or process supervision.Source: §A.4 · pp. 26–27
  • The resulting supervision combines executable correctness, instrumented intermediate structure, and the story scenario, providing multi-step targets or partial and step-level supervision units.A representative string-ordering example translates a trace’s sorted order into a narrative explanation without programming terminology.Source: §A.4 · pp. 27–28

B.1 Evaluation Framework

The appendix specifies how benchmarks are evaluated, how subset accuracy is aggregated, and which inference and training settings are used.

B.1 Evaluation Framework

  • Code benchmarks use sandboxed execution and unit-test verification, while other experiments use lm-evaluation-harness with vLLM for standardized evaluation and batched generation.The code pipeline follows the benchmarks’ official repositories.Source: §B.1 · p. 28

B.2 Benchmarks

  • The evaluation covers mathematical reasoning, general reasoning and knowledge, and code generation benchmarks.Table 7 summarizes the benchmarks.Source: §B.2 · p. 28

B.3 Metric Definition and Subset Aggregation

  • Per-task accuracy is the share of extracted answers matching ground truth; multi-subset benchmarks aggregate subset accuracy by example count.The aggregation follows lm-evaluation-harness’s weight_by_size strategy.Source: §B.3 · p. 28

B.4 Inference Configuration

  • Inference uses separate decoding configurations for Qwen3 think and non-think models, and a shared configuration for DeepSeek-R1-Distill-LLaMA; results are mean@6.Both Qwen3 configurations use max_gen_toks=4096; the DeepSeek setting matches the Qwen3 think configuration.Source: §B.4 · p. 28

C Training Setting

  • The training-setting section covers model specifications, hyperparameters, and dataset configurations for both SFT and RL.Source: C Training Setting · p. 28

C.1 Base Models

The experiments use instruction-tuned models spanning Qwen3 scale variants and a reasoning-distilled model, with MIMIC and mathematical data used across SFT and GRPO. The setup evaluates binary and step-based reward strategies for reinforcement learning.

C.1 Base Models

  • The study uses Qwen3-4B, Qwen3-8B, and DeepSeek-R1-Distill-Llama-8B, disabling Qwen3 thinking mode while retaining DeepSeek’s chain-of-thought behavior.The models represent two Qwen3 scales and a distinct architecture with distilled reasoning capabilities.Source: §C.1 · p. 28

C.2 Training Datasets

  • MIMIC supplies approximately 114K verified SFT samples and 10K selected RL samples; the default SFT mixture combines MIMIC and math data.The RL examples are selected for partial correctness across eight rollouts, while the SFT mixture uses equal portions of MIMIC and math data.Source: §C.2 · p. 30

C.3 Supervised Fine-Tuning (SFT)

  • SFT mixes equal portions of MIMIC and NuminaMath-CoT to mitigate forgetting, with approximately 114K total training instances.The study also compares Math-only and original Codeforces data without story context.Source: §C.3 · p. 30

C.4 Reinforcement Learning (GRPO)

  • GRPO compares binary answer-match rewards with CIR, which grades progress through reference-solution steps using a lightweight judge.CIR assigns a reward from 0 to 1 based on correctly completed steps relative to the total steps.Source: §C.4 · p. 31

D Distribution of Question Types in the Original Dataset

The cleaned Stage-I dataset covers five top-level reasoning categories, but its instances are concentrated in mathematical computation and rule/state simulation. The taxonomy includes smaller categories to broaden capability coverage.

D Distribution of Question Types in the Original Dataset

  • Type 5 accounts for 46.02% and Type 3 for 40.64% of instances, together covering 86.66% of the 10,008-instance dataset.Table 10 reports the hierarchical distribution across top-level categories and subtypes.Source: D Distribution of Question Types in… · p. 32
  • Type 2 represents 6.47% of instances, while Type 4 represents 4.80% and Type 1 represents 2.07%.Type 2 is dominated by character manipulation, while Type 4 is concentrated in aggregation and sorting.Source: D Distribution of Question Types in… · p. 32
  • Despite the skew, every predefined category is represented, including smaller capability-critical types, using a single publicly available source, Codeforces.The authors describe the resulting coverage as useful for training and fine-grained diagnosis.Source: D Distribution of Question Types in… · p. 32

E.1 Difficulty-Dependent Effects Across Ratings

Across rating bands, fused and synthesized problem formats suppress code-oriented responses, while fused-format performance improves relative to refined formats on harder problems. The taxonomy analysis also identifies difficult reasoning categories and finds that generated Level-C tests can approach official-test difficulty.

E.1 Difficulty-Dependent Effects Across Ratings

  • Across ratings, fused and synthesized formats keep code-writing below 1%, while fused gains over refined grow from negative below 1200 to substantial above 2000.The reported code-writing rate rises to nearly half of cases for original and refined formats in the 1200–1999 range.Source: Table 11 · p. 32, §E.1 · p. 33, Table 12 · p. 33

E.2 Performance Across Taxonomy Types

  • Visual/Spatial and Rule/State reasoning each reach only 37.2% accuracy, among the categories where the model performs poorly.The authors associate these tasks with fine-grained structural interpretation, local manipulation, and multi-step state tracking.Source: §E.2 · p. 34, Table 13 · p. 34

E.3 Quality of Random Guided Synthetic Test Cases

  • Generated Level-C tests often approach official-test difficulty despite smaller constraints, and induce similar failure patterns.The authors report that these tests can support evaluation and training when official tests are unavailable.Source: §E.3 · p. 35, Table 14 · p. 35

F Token Distribution Shift

SFT changes a broader portion of token preferences than GRPO, whose shifts concentrate more narrowly on computation-related vocabulary. Training dynamics analyses report rising rewards and improvements on MIMIC and several other benchmarks.

F Token Distribution Shift

G.1 RL Training Dynamics Analysis

  • During 190 GRPO steps, mean reward rises from 0.61 to 0.81, while smoothed KL divergence stays below 0.004.The reported reward curve improves rapidly during steps 1–40 and more gradually thereafter.Source: §G.1 · p. 37
  • Mean response length grows from approximately 850 to 1,330 tokens, which the authors interpret as more elaborate reasoning rather than length hacking.The reward evaluates the final answer rather than explicitly rewarding longer responses.Source: §G.1 · p. 37
  • MIMIC improves by +9.8%, while BBH gains +1.9%, GSM-Plus +4.0%, and MBPP +8.9% during training.MIMIC rises from 41.2% to 51.0% in the reported evaluation.Source: §G.1 · p. 37

G.2 SFT Training Dynamics Analysis

SFT converges quickly but its benchmark gains saturate, whereas GRPO continues improving across reasoning, math, and code benchmarks. The task-type ablation setup measures category contributions by training on individual types or leaving each type out.

G.2 SFT Training Dynamics Analysis

  • SFT benchmark gains rise rapidly for about 800 steps, then plateau or mildly regress; its losses fall quickly before converging with little train-validation gap.SFT training loss drops from 0.92 to about 0.45 within 400 steps and converges to 0.38; validation loss decreases from 0.44 to 0.39.Source: §G.2 · pp. 37–38, Figure 6 · p. 38
  • GRPO improves MIMIC by +9.8% and MBPP by +8.9%, versus SFT’s +3.3% MIMIC gain and negligible code improvement.SFT gains are concentrated in format-sensitive ARC (+5.5%), while GRPO also improves BBH (+1.9%) and GSM+ (+4.0%).Source: §G.2 · p. 38
  • SFT mainly learns the training data’s output distribution, while GRPO optimizes outcome correctness and improves continuously without saturation.GRPO can explore diverse reasoning paths and retain those that yield correct answers.Source: §G.2 · p. 38

H Task Type Ablation Analysis

  • The ablation compares GRPO models trained on one task category alone with models trained on all categories except one, under identical hyperparameters and evaluation benchmarks.The study uses five task categories and evaluates the resulting models on the same benchmark suite.Source: H Task Type Ablation Analysis · p. 38
  • LOO Impact is the full-model score minus the score without a category; positive values indicate a beneficial contribution, while negative values suggest interference.The study visualizes this metric for individual benchmarks and category averages.Source: H Task Type Ablation Analysis · p. 39, Figure 10 · p. 39

H.1 Per-Type Contribution Analysis

Task-type ablations identify distinct contributions across reasoning, math, and code, while showing that no single category matches the full mixture’s balance. The analysis also finds overlapping contributions and a trade-off between Type 4’s code gains and math performance.

H.1 Per-Type Contribution Analysis

  • Type 3 has the broadest impact, with LOO contributions of Reasoning +1.8, Math +2.4, and Code +2.6, including +10.0 pp on AIME.Type 3 alone reaches Reasoning Avg 63.5 and Math Avg 68.5, near the full model’s 63.8 and 69.4.Source: §H.1 · pp. 39–40
  • Type 5 is the strongest single-type contributor to reasoning: it reaches Reasoning Avg 63.6, Math Avg 69.7, and MIMIC 53.8, while its removal lowers MIMIC by 6.5 pp.Its removal also lowers BBH by 1.7 and GSM+ by 2.0; its tasks involve graph traversal, constraint satisfaction, and iterative state simulation.Source: §H.1 · p. 40
  • Type 1’s moderate category-average impacts include a +6.7 pp LOO impact on AIME, second only to Type 3.The passage also reports +3.8 pp on CruxEval and +2.5 pp on GPQA.Source: §H.1 · p. 40
  • Type 2 has the smallest LOO impacts across dimensions, yet its single-type model achieves the highest AIME score among single types at 28.3.Its reported category impacts are Reasoning +0.3, Math +0.8, and Code +0.4.Source: §H.1 · pp. 40–41
  • Type 4 contributes most to Code Avg (+3.3 pp) but has a negative Math Avg impact (-0.9 pp); alone, it is the weakest single-type model.Its per-benchmark impacts include +4.8 on CruxEval, +3.5 on MBPP, and -3.3 pp on AIME; mixed with other types, it supports coding ability.Source: §H.1 · p. 41

H.2 Cross-Type Interactions and Complementarity

  • The full mixture reaches Reasoning 63.8, Math 69.4, and Code 48.0, while the best single-type results come from different categories.Type 5 has the best single-type Reasoning and Math averages, while Type 3 has the best Code average; no single category matches all three maxima.Source: Table 16 · p. 40, §H.2 · pp. 41–42
  • LOO impacts are sub-additive: their sum on Reasoning Avg is 5.3 pp, exceeding the full model’s 3.6 pp gain, consistent with transfer and redundancy among types.Each type’s marginal contribution is smaller when other types are already present.Source: §H.2 · p. 42

H.3 Summary

  • The study retains all five categories to favor balanced performance, since removing Type 4 raises Math Avg to 70.2 but sacrifices its largest Code improvement of +3.3 pp.The summary of the ablation reports positive overall contributions from all five categories, with Types 3 and 5 most impactful.Source: §H.2 · p. 42, §H.3 · p. 42

I Case Studies of Base Model and Trained Model

The case studies contrast base-model errors on a maze and a string operation with more systematic, correct solutions after GRPO training on MIMIC data. They attribute the base model’s failures to inefficient search and incomplete procedural execution.

I Case Studies of Base Model and Trained Model

  • On the maze, the base model cycles among failed paths and exhausts its token budget, while the GRPO-trained model systematically tests directions and reaches the target.The base response contains 13,831 characters; the trained model’s example proceeds through the route to the target.Source: I Case Studies of Base Model and… · p. 43
  • For the string task, the base model removes only the initial “5f” and misses the following “f,” while the trained model returns the correct result.The correct operation strips three leading characters before the first nonmatching character.Source: I Case Studies of Base Model and… · p. 43
  • The base model’s verbose reasoning fails through error accumulation, including losing track of maze positions and miscounting string boundaries.The case discussion describes over 13K characters for a seven-step maze and over 2K for a simple string operation.Source: I Case Studies of Base Model and… · pp. 42–43
  • The trained model converges faster in maze search and continues probing characters in the string task, illustrating more systematic exploration of task procedures.The discussion links these examples to precise, systematic computation that programs handle trivially but challenges approximate language-model inference.Source: I Case Studies of Base Model and… · pp. 43–44

Related papers

Loading 2609.16076v1…