AdaR: A Framework for Equipping LLMs with Adaptive Reasoning

1 Introduction

• Spurious reasoning ties model outputs to superficial correlations, causing failures in robustness and generalization when problem conditions change.The paper illustrates this with a model repeating reasoning tied to a memorized variable value; it describes the resulting reasoning as non-compositional along causal relations.Source: §1 · p. 1
• AdaR defines adaptive reasoning as correlating a query template with its reasoning process while disentangling specific variable values from the query.Its synthesis process varies variable values and query templates while preserving problem-solving logic, then uses code execution and sanity checks to verify generated pairs.Source: §1 · pp. 1–2
• AdaR gains an average of +8.50 points across in-domain robust and out-of-domain tasks using only 9K synthetic examples.The authors report gains on both math-specialized and general base models.Source: §1 · p. 2
• Perturbing variable values is more effective than perturbing query templates for improving model performance.Source: §1 · p. 2
• Attention to templates and improved algebraic thinking provide evidence that AdaR equips models with adaptive reasoning.Source: §1 · p. 2

2 Method

• AdaR decomposes synthesis into LLM-extracted metadata and logic, externally controlled perturbations, and checks intended to ensure correct query-answer pairs.The framework’s method overview describes perturbing variable values while keeping problem-solving logic fixed, followed by a sanity check.Source: §2 · p. 2, §2.1 · p. 3

2.1 Data Synthesis with Executable Code and Verifiable Answers

• For data synthesis, AdaR extracts a query template and translates the original chain of thought into executable code, then perturbs variables and runs the code to produce answers.Sanity checks test variable alignment, code execution and reproduction of the original answer, and agreement with a mathematical LLM to assess whether a valid query exists; failed instances may be retried or discarded. Paraphrasing adds template diversity, and the synthetic data contains queries and answers without chain-of-thought traces.Source: §2.1 · p. 3

2.2 Training Strategy

• AdaR combines RLVR with synthetic perturbed queries so reward feedback can distinguish reasoning that generalizes across changed values from spurious reasoning.The authors contrast this with single-query outcome rewards, which can reward spurious reasoning that happens to produce the correct answer.Source: §2.2 · p. 4

3.1 Experimental Setup

• The setup synthesizes 9K ORCA-AdaR training instances from ORCA-MATH seeds and holds out 2.5K non-overlapping instances for testing.Qwen2.5-72B-Instruct generates templates and problem-solving code; the synthesis uses α = 500 and a maximum of τ = 50 attempts.Source: §3.1 · p. 4
• Experiments cover Qwen2.5-Math-7B and DeepSeekMath-7B as math-specialized models, plus LLaMA3-8B as a general base model.Source: §3.1 · p. 4
• Evaluation covers in-domain competence and robustness plus out-of-domain generalization across seven benchmarks, with Table 1 comparing performance across in-domain and out-of-domain tasks.The setup reports pass@1 except for AIME 2025, which uses avg@32; GSM-SYM main, p1, and p2 increase in difficulty.Source: §3.1 · p. 4, Table 1 · p. 5

Appendix A.4.3.

• Comparisons include 9K-seed Initial SFT, Standard-RLVR on another 9K ORCA-MATH subset with disjoint templates, MetaMATH, and MathGenie; RLVR uses DAPO.Source: Appendix A.4.3 · p. 5

3.2 Main Results

• With 9K synthetic data, AdaR surpasses other methods across base models, improving over MetaMATH by 8.50 points and MathGenie by 11.44 points on average.Source: §3.2 · p. 5
• Across base models, gains on seen-query and unseen-query perturbations are +4.66 / +4.75 points; Qwen2.5-MATH-7B averages +8.56 points on out-of-domain tasks.Source: §3.2 · p. 5
• Additional rollouts for correct and incorrect responses rise from Qwen2.5-MATH (458.00) to DeepSeekMath (563.20) to Llama3 (942.83), while gains fall (+17.69, +10.53, +5.26 points).The authors interpret sufficient mathematics-related knowledge as necessary for sampling both response types.Source: §3.2 · p. 6

3.3 Ablation study

• Without executable-code checking (EC), subsequent EVQ retains only 0.2% of data; rule-based EC is 218× more computationally efficient, while paraphrasing complements value perturbation.Source: §3.3 · p. 6
• AdaR-RFT gains overall versus the baseline on perturbed data, yet improves computation while degrading reasoning versus Initial-SFT, consistent with memorizing superficial computation patterns.Source: §3.3 · p. 6

4.1 AdaR Enables Adaptive Reasoning

• After SFT, attention to irrelevant numerical tokens rises; AdaR strengthens attention to the logic-bearing template and reduces attention to variable values, with logical-order dependence checked by ILO.Source: Figure 3 · p. 7, §4.1 · p. 7
• Structural-code CoTs rise from 55% to 90% without code training; standard RLVR lowers code-generation accuracy, whereas AdaR synthetic data improves it.Source: §4.1 · p. 7

4.2 Effect of Scaling Different Dimensions

• Figure 4 shows steady in-domain and out-of-domain gains with scale, while fitted marginal-return coefficients satisfy kL < kT < kx.Source: Figure 4 · p. 8, §4.2 · p. 8

4.3 Effect of Seen Perturbed Queries

• AdaR trained on SFT-seed-conditioned perturbations and non-overlapping AdaR-Lite×4 data both outperform Standard-RLVR.Source: Table 4 · p. 8, §4.3 · p. 8

4.4 Effect of Perturbation Magnitude

• Figure 5 shows performance rising with larger variable perturbations up to a point, then slightly declining as excessive perturbations create more invalid candidates before sanity checks.Source: Figure 5 · p. 8, §4.4 · p. 8

4.5 AdaR is Applicable to Instruct Model

• Using 2507 as the base model, AdaR still yields substantial performance gains (+5.86) and effectively mitigates robustness failures that persist under test-time scaling (+1.51).Source: §4.5 · p. 8

• Prior findings link invalid or spurious reasoning demonstrations to comparable performance and reduced sensitivity to logical-order variations, while benchmarks report robustness and generalization weaknesses.Source: §5 · p. 9
• Earlier synthesis diversifies templates through paraphrasing or generation and verification, but often preserves mathematical structure or relies on error-prone verification, limiting adaptive-reasoning gains.Source: §5 · p. 9
• AdaR is presented as complementary to online synthesis from model responses; incorporating the model under training for template and code generation may further improve performance.Source: §5 · p. 9

• AdaR targets mathematical robustness and generalization failures attributed to spurious reasoning based on superficial features.Source: §6 · p. 9
• The framework combines data synthesis and model training to encourage correct correlations between query templates and reasoning processes.Source: §6 · p. 9
• Experiments report substantial improvements with small amounts of data on in-domain and out-of-domain tasks, and analyses indicate adaptive reasoning, scalability, and broad applicability.Source: §6 · p. 9

• AdaR currently scales query templates and variable values, but not diversity in problem-solving logic; expanding logic diversity remains future work.Source: Limitations · p. 9
• Each seed query must have a fully executable problem-solving program and easily perturbed input variables, so broader settings such as theorem proving need more sophisticated designs.Source: Limitations · p. 9
• Computational constraints prevented evaluation on large reasoning models and base models with larger parameter counts.Source: Limitations · p. 9

A.1 The use of LLMs

• LLMs are used only to polish writing, focusing on grammatical corrections and more appropriate word choices.The authors say they thoroughly double-check all content refined by LLMs.Source: §A.1 · p. 11
• The authors double-check LLM-refined content to prevent harmful information from being included.Source: §A.1 · p. 11
• The verification process is intended to ensure the final content’s accuracy, reliability, and appropriateness.Source: §A.1 · p. 11

A.2.1 SFT & RFT

• SFT trains on query–response pairs using negative log-likelihood, RFT retains high-scoring sampled outputs, and RLVR uses a verifier that checks predicted answers against gold answers.The reported SFT setup runs three epochs; RL training uses 16 responses per prompt and takes approximately two days.Source: §A.2.1 · p. 11, §A.2.2 · p. 11, §A.4.2 · p. 12

A.3 Prompts

• The appendix lists prompts for template/code generation, valid-query checks, code generation, and code execution, and inspects 20 GSM-SYM responses for structural snippets.It also reports that sanity-check generation costs were measured on a single compute node.Source: §A.3 · p. 12, §A.7 · p. 14, §A.8 · p. 14

A.4.1 Data Synthesis

• With Qwen2.5-7B rather than Qwen2.5-72B for synthesis, scaling data along variable values reduces the performance gap from 8.14 to 3.09.The appendix also describes ORCA-MATH sampling and specifies Qwen2.5-72B as the primary synthesis model.Source: §A.4.1 · p. 12, Table 7 · p. 17

A.4.3 Evaluation

• Evaluation reports pass@1 and avg@32, whose differences stay within one percentage point on test sets of about 1,000 samples, with conclusions consistent with avg@1.The appendix also describes seen- and unseen-template test sets of 1,000 instances each for the Instruct-model experiment and says Table 8 gives its full results.Source: §A.4.3 · p. 13, §A.9 · p. 14

A.5 Analysis on Influence to Logical Order

• 114.24% vs. 221.87%: CoTs failing versus succeeding across all variable perturbations have lower versus higher ILO, respectively.The authors interpret lower ILO as less sensitivity to logical sequence and associate it with spurious reasoning.Source: §A.5 · p. 13
• 150.49% vs. 119.22%: AdaR training improves ILO over Initial SFT, indicating greater reliance on rigorous logical sequencing than superficial feature matching.Source: §A.5 · p. 13

A.6 Theoretical Analysis

• Variable perturbation decorrelates superficial features from logic, so empirical risk minimization on the perturbed distribution favors the invariant logic mapping over shortcut prediction.The theoretical analysis describes static-data shortcuts as exploiting correlations between specific numbers and reasoning paths.Source: §A.6 · p. 14

A.10 Training Dynamic Analysis

• During RLVR, generated CoTs lengthen; entropy first decreases as short patterns stabilize, then rises as longer sequences accompany more adaptive reasoning.AIME 2024 scores improve rapidly early in training, then plateau with minor fluctuations.Source: §A.10 · pp. 14–15, Figure 7 · p. 15, Figure 8 · p. 15