Efficient Agentic Reasoning Through Self-Regulated Simulative Planning

1 Introduction

• Undifferentiated chain-of-thought lacks controls over planning presence, horizon, and structure, increasing token consumption without necessarily yielding better answers [28] [92].It also lacks an explicit planning structure that can be analyzed, regulated, or improved independently of the remaining reasoning process.Source: §1 · pp. 1–2
• The proposed architecture combines reactive execution, world-model-based simulative planning, and learned self-regulation that determines when and how deeply to plan.System I handles fine-grained reasoning and direct action, System II predicts action consequences, and System III configures planning. The paper argues prior approaches do not combine all three.Source: §1 · p. 2
• SR2AM implements its configurator and simulative planner as distinct chain-of-thought stages, using the LLM itself as the world model.Each turn, the configurator chooses whether to make a new plan, continue an existing one, or act directly. Planning explicitly represents proposed actions and predicted future states alongside free-form reasoning and execution.Source: §1 · p. 2, Figure 1 · p. 2
• 25.8–95.3% fewer reasoning tokens than similar-scale competitive agentic LLMs are used by SR2AM-v1.0-30B, whose overall Pass@1 is competitive with 685B–1T-parameter systems.SR2AM-v0.1-8B achieves overall Pass@1 competitive with 120–355B-parameter systems. Evaluations cover math, science, tabular analysis, and web information seeking.Source: §1 · p. 2
• 22.8% longer average planning horizons follow RL, while planning frequency rises only 2.0 percentage points, indicating learning to plan further ahead rather than more often.The reported frequency change is in percentage points, not a relative percentage increase.Source: §1 · p. 2
• SR2AM-v0.1 records decisions from a prompted multi-module system, while v1.0 reconstructs structured plans from pretrained reasoning LLM traces for better scalability.Both undergo supervised learning followed by RL, producing SR2AM-v0.1-8B and SR2AM-v1.0-30B.Source: §1 · p. 2

2.1 Agent-Environment Model and Simulative Reasoning

• Simulative reasoning predicts belief-state transitions for candidate actions through a world model and selects sequences maximizing expected long-term goal progress.The agent infers beliefs from observations rather than accessing the true world state, and seeks to maximize goal-dependent reward. Verifiable next-state prediction provides a general-purpose planning mechanism without domain-specific procedures.Source: §2.1 · p. 3
• A forthcoming analysis claims that augmenting a baseline with a reasonably accurate world model yields a mixed policy no worse than that baseline.The claimed improvement is strict when simulative reasoning identifies a better choice.Source: §2.1 · p. 3
• An approximate planner produces structured plans containing the current belief, selected actions, and predicted future states because exact optimization is intractable.Predicted states support progress assessment and detection of violated expectations; planned actions guide execution when anticipated states occur. The reactive actor conditions concrete actions on the current belief and plan.Source: §2.1 · p. 3

2.2 From Unregulated Deliberation to Self-Regulation

• Unregulated deliberation leaves planning implicit, without explicit current-state beliefs, predicted future states, or contingency plans to ground action selection.Planning is expected to emerge through task training, including end-to-end RL [85] [41]. Reasoning can lengthen dramatically without necessarily improving task success [28] [92].Source: §2.2 · pp. 3–4
• Invoking simulative reasoning at every step [20] [100] [119] explicitly models decision ingredients but can be prohibitively costly when replanning is unnecessary.Urgent situations and simple continuations illustrate why explicit planning alone does not remove the need to regulate its invocation.Source: §2.2 · p. 4

2 Formalizing Self-Regulated Simulative Reasoning

• The framework separates planning control (System III), simulative planning (System II), and reactive execution (System I).A belief-conditioned configurator chooses whether to create, continue, or skip a plan, while the actor handles fine-grained actions. The formulation also permits iterative plan refinement and free-form reasoning within components.Source: §2.1 · p. 3, §2 · p. 3, §2.2 · p. 4

2.2 From Unregulated Deliberation to Self-Regulation

• Prior effort-adaptive, mode-routing, and workflow-distillation approaches each implement only part of the proposed three-system decomposition.Effort-adaptive approaches [e.g., 52, 99] select fixed thought modes without explicit planning; mode-routing approaches [e.g., 13] decide only at task onset. Workflow distillation [e.g., 48] internalizes predefined routing but lacks simulative planning and free-form reasoning.Source: §2.2 · p. 4

3 Instantiating Self-Regulated Simulative Reasoning

• SR2AM realizes the configurator and simulative planner as distinct chain-of-thought stages alongside free-form reasoning and acting, with the LLM serving as world model.Supervised finetuning teaches this structure before reinforcement learning refines task success.Source: §3 · p. 4

3.1 Environment and Tools

• The agent forms beliefs from reasoning context, actions, and tool outputs, then calls web search, a goal-directed browser, or stateless Python, or produces a final response.Following prior work [41] [110], these tools support interactive reasoning; trajectories terminate with a final answer or at the maximum action count.Source: §3.1 · p. 4

3.2 Supervised Data Construction

• The feasibility-focused v0.1 approach records a prompted configurator invoking separate planning and belief-formation modules before selecting actions.Training traces interleave configurator thoughts with module outputs and are filtered for answer correctness and minimum reasoning complexity. The approach is model-agnostic; main experiments use o4-mini.Source: §3.2 · p. 5
• The primary v1.0 approach uses an annotator to reconstruct planning decisions and, when needed, structured plans from DeepSeek-V3.2 thinking–acting traces.Plans preserve the original reasoning and support hierarchical time scales; generating them jointly infers the current state, proposes actions, and predicts consequences in one pass. Plans for highly uncertain web-browsing operations are capped at two steps.Source: §3.2 · p. 5

3.3 Reinforcement-Learning-Based Refinement

• Reinforcement learning coordinates the three systems using correctness, trajectory-structure compliance, and final-answer extractability rewards, optimized through adapted Group Relative Policy Optimization [GRPO, 85].The reward prioritizes correctness while retaining structural feedback for unsuccessful trajectories; optimization uses asymmetric clipping [121] and group-normalized advantages. For models of 30B and above, truncated trajectories are filtered to prevent format collapse [111].Source: §3.3 · p. 5

3.4 Training Data and Hyperparameters

• 4,845 supervised examples train v0.1, while 10,787 train v1.0, drawn from open-source math, science, tabular, and web reasoning datasets.The starting models are Qwen3-8B for SR2AM-v0.1-8B and Qwen3-30B-A3B-Thinking-2507 for SR2AM-v1.0-30B. RL difficulty filtering [90] retains questions with intermediate Pass@K rates to provide informative gradient signals.Source: §3.4 · pp. 5–6

4.1 Experiment Setup

• 11 benchmarks span math, science, tabular analysis, and web information seeking, evaluated using overall Pass@1 and reasoning tokens per trajectory.Overall Pass@K [12], following [111], averages benchmark scores without weighting; reasoning tokens exclude environment observations and tool outputs. Comparisons cover pretrained reference systems and unregulated or partially regulated agents; analyses normally use 330 sampled examples repeated three times, totaling 990 runs.Source: §4.1 · p. 6, §4.3 · p. 7

4.2 Main Results

• 71.3 overall Pass@1 makes SR2AM-v1.0-30B competitive with tool-equipped DeepSeek-V3.2 (685B, 73.2) and Kimi-K2.5 (1.0T, 70.9).It exceeds text-only GPT-5.4-xhigh (68.4) while approaching its tool-equipped performance (78.3). SR2AM-v0.1-8B achieves 57.0, outperforming same-scale systems and competing with 30–32B unregulated agents and 120–355B pretrained LLMs with tools.Source: §4.2 · p. 6, Figure 2 · p. 7
• 25.8–95.3% fewer reasoning tokens accompany better or competitive accuracy for SR2AM-v1.0-30B versus 30–32B agentic models with overall Pass@1 of at least 60.0.Against MiroThinker-v1.5-30B, it uses 51.2% fewer tokens (5,518 versus 11,295) with competitive Pass@1. SR2AM-v0.1-8B averages 3,698 tokens, fewer or comparable to most same-scale systems (601–11,206), while outperforming them in Pass@1.Source: §4.2 · pp. 6–7, Figure 3 · p. 7

4.3 Analysis of Self-Regulated Simulative Reasoning

• 46.8 versus 66.6 Pass@1 after removing free-form reasoning is the largest ablation drop, supporting complementarity between structured plans and fine-grained reasoning.Disabling selective planning increases token consumption most (4,925 to 5,451), while removing simulative structure or uncertainty-aware horizon control also reduces accuracy. The full decomposition outperforms direct SFT on teacher CoT (66.6 versus 65.3).Source: §4.3 · pp. 7–8, Table 1 · p. 8
• 5.3% to 14.9% growth in two-or-more-step plans after RL accompanies nearly stable unplanned turns (15.6% to 13.6%), indicating deeper rather than more frequent planning.Average planning horizon increases across all categories, most in science (32.7%) and least in web (20.9%), consistent with web uncertainty limiting feasible horizons. Representative trajectories show error detection and anticipatory fallback strategies, but occasional overplanning on simple tasks remains.Source: Figure 4 · p. 8, §4.3 · p. 8
• 34.1% fewer reasoning tokens accompany a higher step-400 pass rate for SR2AM-v0.1-8B than unregulated Qwen3-8B (56.2 versus 47.6).Unregulated accuracy peaks at step 200 and then declines as context overflow rises to 22.4%, whereas SR2AM has negligible overflow. For v1.0, RL raises Pass@1 from 66.6 to 72.8 with moderate token growth from 4,925 to 5,414.Source: §4.3 · pp. 8–9, Figure 5 · p. 9
• 1.2–3.5% Pass@1 and 1.5–3.4% Pass@3 gains favor structured v0.1 training data over unregulated reconstruction across data scales with the same o4-mini teacher.The comparison fine-tunes Qwen3-8B on 800–3,200 examples and evaluates 300 held-out examples over six runs, isolating structure from teacher quality. Using Qwen3-Coder-480B-A35B-Instruct improves results further, with Pass@1 rising from 60.9 to 72.9 as data increases from 1,600 to 3,200 examples.Source: §4.3 · p. 9, Figure 6 · p. 9

5 Related Work

• Frontier reasoning models [19] [114] apply extended chain-of-thought uniformly without mechanisms to modulate reasoning horizon or structure.Responses to this inefficiency include length penalties [4] [2], supervised compression [43] [108], and user-specified effort controls [50]. These impose external constraints without modeling internal autonomy.Source: §5 · pp. 9–10
• Effort-adaptive approaches [52] [124] [23] [86] regulate reasoning amount along a single axis without constructing simulative plans.Recent extensions apply effort adaptation to agentic settings [99] [34].Source: §5 · p. 10
• World-model planning grounds action selection in predicted future states but invokes simulation obligatorily.This includes classical MPC, learned latent world models [84] [32], and visual simulators [5] [109] [119].Source: §5 · p. 10
• LLM-based planners [117] [33] [127] [20] demonstrate that LLMs can serve as world models, but lack selective planning invocation [74].Planning remains obligatory.Source: §5 · p. 10
• Workflow distillation [48] internalizes multi-agent orchestration [106] [37] but retains rigid sequencing and lacks free-form reasoning and variable-horizon planning.These limitations concern reactive execution (System I) and simulative reasoning (System II), respectively.Source: §5 · p. 10
• Mode routing [13] [40] makes a one-time decision at task onset rather than reassessing the reasoning mode each turn.The comparison identifies per-turn reassessment as a self-regulation capability (System III).Source: §5 · p. 10

6 Conclusion

• Four interactive reasoning task categories demonstrate competitive accuracy at substantially lower reasoning cost through complementary systems for simulative planning, learned self-regulation, and reactive execution.These correspond to System II, System III, and System I, respectively; ablation studies confirm their complementary roles.Source: §6 · p. 10
• RL with self-regulated simulative reasoning produces longer-horizon plans rather than more frequent plans, with controlled growth in token consumption.The authors distinguish this improvement pattern from RL without self-regulated simulative reasoning.Source: §6 · p. 10
• A single simulative planning framework performs strongly across diverse task categories without per-domain procedures, supporting its use as a general-purpose planning mechanism.The framework spans four task categories.Source: §6 · p. 10
• The authors suggest that ever-longer reasoning traces may reflect undifferentiated reactive policies lacking internal control over planning.This is an interpretation of the findings rather than an established explanation for all agentic models.Source: §6 · p. 10
• The learned configurator illustrates autonomous regulation of reasoning processes, a principle the authors expect could extend to agents governing their own learning and adaptation.This extension beyond inference-time planning is a proposed direction, not a demonstrated result.Source: §6 · p. 10

7 Limitations and Future Work

• Four language-based task categories delimit evaluation; future work includes embodied and multi-agent settings, multimodal world models, and isolated component diagnostics.LLM world models may predict physical and social dynamics inadequately; perceptual next-state prediction would test richer environments and coordination constraints. Evaluation currently measures overall accuracy and efficiency; proposed diagnostics compare planning decisions with oracle hindsight and predicted states with observed outcomes.Source: §7 · p. 10
• Engineering improvements include context management, targeted synthetic training data, and richer tool interfaces.Current models retain full interaction histories, train on public datasets, and use standard tools; proposed changes include recent-response retention, synthetic web-reasoning data, terminal access, and Agent Skills.Source: §7 · p. 10
• Self-regulation could extend beyond planning to govern world-model updates, mental-simulation learning, self-identity revision, and long-term goal decomposition.Regulating multiple interacting processes is proposed as a path toward broader autonomy and adaptability, rather than demonstrated here.Source: §7 · p. 10

A.1 Unregulated Deliberation and External Controls

• External length penalties and reasoning compression reduce average cost without learning situational regulation; user-selected effort tiers likewise leave control outside the model.Length-penalized RL [4] [2] [120] and supervised compression [43] [59] [108] impose external targets; Qwen3 [114], o1/o3, and K2-V2 series [50] provide user controls rather than autonomous per-step adjustment. Reasoning length can grow rapidly during RL without commensurate accuracy gains; the paper classifies these approaches as System I without explicit Systems II or III.Source: §A.1 · pp. 17–18

A.2 Partially Regulated Deliberation

• Effort-adaptive methods regulate reasoning situationally without explicit simulative plans, while mode routers generally choose execution regimes once rather than reassessing each turn.Effort control includes uncertainty gating [14], RL switching [52] [124] [23], and within-trace meta-actions in Satori [86], with extensions to multi-turn agentic settings. A2FM [13] and Large Hybrid Reasoning Models [40] distinguish reasoning regimes but lack explicit simulative planning.Source: §A.2 · p. 18
• Developer-orchestrated workflows prescribe deliberation, while Chain-of-Agents [CoA, 48] internalizes workflows but retains rigid sequencing and action-directive plans rather than predicted state transitions.AutoGen [106], MetaGPT [37], LangGraph, DeerFlow, and MiroFlow organize specialized stages through human-authored control logic. Workflow distillation removes external orchestration at inference time but constrains free-form execution and lacks variable-horizon simulative plans.Source: §A.2 · p. 18

A.3 Simulative Planning Without Self-Regulation

• Existing simulative planners lack autonomous planning regulation; LLM-based approaches often use rule-based orchestration, while VAGEN [100] retains obligatory, shallow, fixed-horizon simulation.World-model planning evaluates actions through predicted state transitions, but agents can struggle to invoke available simulations appropriately [74]. RAP [33] simulates transitions within Monte Carlo Tree Search, while SiRA [20] predicts natural-language latent states for long-horizon web browsing.Source: §A.3 · p. 19

B Environment and Tool Details

• 28,000 tokens is the webpage-content limit in an interactive environment combining search, browser summaries, and stateless Python execution.Agents receive observations, call tools, and eventually answer; search follows [110] using SerpAPI or Serper.Dev, while goal-directed browser summaries use Qwen3-30B-A3B-Instruct-2507. The SandboxFusion [16] implementation, following, runs self-contained scripts with preinstalled libraries; training enables effective stateless execution despite models sometimes assuming persistence.Source: B Environment and Tool Details · pp. 19–20

C.1 Multi-Module Inference Details (v0.1)

• More than 3 reasoning-module calls or actions combined are required for retained v0.1 trajectories, alongside answer correctness.This threshold preserves sufficient reasoning behavior.Source: §C.1 · p. 20
• Task-specific reasoning capabilities improve validation performance: v0.1 omits the summary module for tabular tasks and additionally omits the user intent module for web tasks.The authors suggest these omissions may help because the questions have simpler structure.Source: §C.1 · p. 20
• 30 actions and up to 3 retries bound v0.1 collection with o4-mini, seeking a returned final answer.Answer correctness is checked afterward during trajectory filtering.Source: §C.1 · p. 20
• GPT-4.1 enriches the v0.1 configurator’s reasoning before web-task action selection, improving performance in practice.The authors also explore Qwen3-Coder-480B-A35B-Instruct as an open-source alternative in the same pipeline.Source: §C.1 · p. 20

C.2 Plan Reconstruction Details (v1.0)

• Structured plan annotations make the v1.0 planning horizon explicit and controllable across task categories.DeepSeek-V3.2 in thinking mode jointly annotates each trajectory in a single pass.Source: §C.2 · p. 20
• 100 steps and up to 5 retries bound v1.0 collection using DeepSeek-V3.2 in thinking mode with 128K context.Collection retries seek a correct answer, and trajectories are filtered for answer correctness.Source: §C.2 · p. 20

D.1 Reward Function

• The reward follows [19] and [41], combining three binary signals to prioritize answer correctness while rewarding structural compliance even on unsuccessful trajectories.Reward values remain fixed across experiments without tuning.Source: §D.1 · p. 20

D.2 GRPO Objective

• Adapted Group Relative Policy Optimization [GRPO, 85] uses group-normalized advantages and maximizes a clipped surrogate objective with asymmetric clipping bounds [121].Trajectory groups are sampled from the old policy. Token advantages use group-level z-score normalization, excluding environment-observation tokens.Source: §D.2 · p. 21

D.3 Additional Training Considerations

• On-policy updates and truncation filtering for models of 30B and above stabilize training and guard against format collapse, following [111].Dynamic sampling and GSPO [126] did not improve performance, so the original GRPO objective was retained. Token-level loss aggregation [121] and mean-only group normalization remain future work.Source: §D.3 · p. 21

E.1 Dataset and Data Construction

• 10,787 supervised examples totaling 39.5M tokens remain for v1.0, compared with 4,845 examples totaling 20.7M tokens for v0.1.Self-regulation construction uses 12,800 questions for v1.0, combining 6,400 new questions with v0.1’s 6,400. For RL, v1.0 combines its new datasets with half of the v0.1 dataset.Source: §E.1 · p. 21
• v1.0 expands v0.1’s training sources while filtering noisy answers and restricting web tasks to sufficiently difficult, reachable questions.MegaScience [22] questions require answers shorter than 1,024 characters; TaskCraft [87] requires valid_hop at least 3 and excludes image-tool tasks. MegaScience is sampled to 42,863 examples to match the new web examples, yielding 86,087 additional examples after deduplication.Source: §E.1 · p. 21

E.2 RL Data Filtering

• Difficulty-based filtering before RL follows [90], retaining questions with varied rewards because uniform rewards produce zero advantages and uninformative gradients.Pre-RL rollouts retain questions whose Pass@K lies within a target interval, stopping early when eligibility is established. For expensive agentic tasks, strong instruct-model answers estimate difficulty from parametric knowledge alone, following [48].Source: §E.2 · p. 22
• 151,218 examples form the final v0.1 RL mix, while the final v1.0 mix contains 20,701 examples after filtering and web-data balancing.v0.1 retains 110,086 non-web and 124,596 web questions before downsampling web data to 33.3%. v1.0 adds web examples equal to 33.3% of v0.1 non-web examples and 100% of v1.0 non-web examples, refreshing filtering with intermediate checkpoints during RL.Source: §E.2 · p. 22
• 32 instruct-model answers per web question support filtering at c = 0.3 in both versions; v1.0 also imposes stricter pre-RL rollout thresholds.Web filtering uses Qwen3-Next-80B-A3B-Instruct for v0.1 and Qwen3-235B-A22B-Instruct-2507 for v1.0. The stronger v1.0 pre-RL checkpoint uses K = 8, a = 1/8, and b = 6/8.Source: §E.2 · p. 22

E.3 Training Hyperparameters

• 400 RL steps (51.2K examples) train SR2AM-v0.1 from Qwen3-8B; modest gains from Qwen3-32B motivated v1.0’s plan-reconstruction approach.SFT uses four epochs, batch size 32, context length 32,768, and initial learning rate 2.5e-5; RL uses batch size 128, eight samples per prompt, and at most 40 action steps. The 32B variant instead receives 300 RL steps, with its modest improvement suggesting data construction rather than scale was the primary bottleneck.Source: §E.3 · p. 22
• 200 RL steps (25.6K examples) train SR2AM-v1.0 from Qwen3-30B-A3B-Thinking-2507, introducing difficulty-filtered data for the final 40 steps.SFT runs four epochs with batch size 32, context length 131,072, and initial learning rate 1e-5. RL allows 16,384 response tokens, 122,880 context tokens, and 100 tool steps, using Qwen3-235B-A22B-Instruct-2507 as the reward judge.Source: §E.3 · p. 22
• Both SR2AM instantiations continue improving in reward and evaluation at training’s end, suggesting they may be undertrained.The authors identify scaling training data and compute through more efficient fully asynchronous training as future work.Source: §E.3 · pp. 22–23

F Baseline Details

• Configurable baselines use their highest reasoning-effort tiers: GPT-5.4 uses xhigh, while GPT-OSS-120B and K2-Think-V2 use high.Baselines are organized into three categories according to how they produce behavior.Source: F Baseline Details · p. 23

F.1 Reference Systems

• Reference systems separate tool-free pretrained reasoning from pretrained models given the trained agents’ tool harness, isolating the contribution of tool access.Tool-free reasoning models answer in one step without environment interaction or a maximum completion-token limit. Tool-enabled pretrained models receive no agentic training and use the tools and inference settings described in §3.1.Source: §F.1 · p. 23

F.2 Unregulated Deliberation

• Unregulated agentic baselines learn tool-augmented reasoning and action through unconstrained chain-of-thought, without explicit control over planning presence, horizon, or structure.Any planning behavior must emerge implicitly from end-to-end training.Source: §F.2 · p. 23

F.3 Partially-Regulated Deliberation

• Partially regulated baselines implement only subsets of the proposed decomposition: A2FM routes once at task onset, while AFM distills per-turn routing among predefined capabilities.A2FM lacks per-turn regulation and explicit simulative planning. AFM lacks free-form reasoning (System I) and simulative planning (System II).Source: §F.3 · p. 23

F.4 Inference Settings

• Agentic baselines retain their default inference parameters and routines, sharing the §3.1 browsing summarization model whenever one is required.Source: §F.4 · p. 23

G.1 Evaluation Protocol

• 10-minute per-turn, 5-minute per-tool, and 60-minute overall-response timeouts bound inference time.Following [111], consistent inference settings support stability and reproducibility. SR2AM and pretrained LLMs running through the tool harness have a per-turn completion limit of 16,384 tokens.Source: §G.1 · p. 23
• 0.8 and 1.0 are the rollout temperatures used for SR2AM-v0.1-8B and SR2AM-v1.0-30B evaluation, respectively, both with top-p=0.95.Pretrained LLMs with tools and reasoning LLM baselines use their default generation hyperparameters.Source: §G.1 · p. 23
• 40,960-token contexts and 50-turn limits apply to SR2AM-v0.1-8B and Qwen3-8B; all other models receive 131,072-token contexts and 100-turn limits.Source: §G.1 · p. 23
• 32× test-set duplication covers AIME-24 and AIME-25; 4× covers GPQA-Diamond, GAIA-103, and XBench-DeepSearch, following [13] to stabilize small-sample evaluation.Reported metrics average all duplicated runs. HLE uses the same 500-question subset as [48], while all other benchmarks use their full test sets.Source: §G.1 · p. 23

G.2 Evaluation Metrics

• Overall Pass@K [12] is the unweighted average of dataset-level Pass@K scores.Table 2 summarizes benchmark-specific judge LLMs and scoring methods. Following, every evaluated system receives an instruction to present its final answer in boxed notation.Source: §G.2 · pp. 23–24
• Reasoning efficiency measures average reasoning tokens per trajectory, including free-form reasoning, configurator decisions, and plans, but excluding environment observations, actions, and tool outputs.The count covers tokens generated by the agent.Source: §G.2 · p. 24

H Evaluation Function Sources

• Adapted evaluation functions include those from TIGER-AI-Lab/General-Reasoner and the GAIA scorer from WebSailor (Alibaba-NLP/WebAgent).WebSailor also adapted an evaluation function from the official openai/simple-evals implementation.Source: H Evaluation Function Sources · p. 25
• Additional evaluation functions are adapted from the official implementations at centerforaisafety/hle and xbench-ai/xbench-evals.Source: H Evaluation Function Sources · p. 25
• Evaluation-function sources across all benchmarks include rule-based grading, adapted scorers, and official evaluation functions.Rule-based grading uses PRIME-RL/PRIME functions via verl, derived from hendrycks/math, Microsoft- /ToRA, and OpenAI/prm800k.Source: Table 2 · p. 25, H Evaluation Function Sources · p. 25

• Table 3 compares reference systems and agentic reasoning paradigms benchmark by benchmark on the final test set.Source: Table 3 · p. 26
• “# Reas. Tokens” measures the average number of reasoning tokens per problem.Source: Table 3 · p. 26
• Systems are grouped by paradigm and ordered by parameter size, with ties broken by ascending overall performance and GPT-5.4 listed last where applicable.Source: Table 3 · p. 26

Example 1: Web Research (GAIA)

• 6 was SR2AM-v1.0-30B’s final GAIA answer after verification, versus 7 from both SR2AM-v1.0-30B (SFT) and Unregulated CoT (SFT).All three initially tallied six Nile and one Morelet’s crocodile. The RL model planned additional verification and ultimately consulted a GAIA dataset page showing the recorded answer.Source: Example 1: Web Research (GAIA) · pp. 27–28

Example 2: Mathematical Reasoning (AIME 2024)

• SR2AM-v1.0-30B correctly answered 315 in 11 messages, versus 61 for SR2AM-v1.0-30B (SFT); Unregulated CoT (SFT) was incorrect after 57 messages and ended with a malformed tool call.Unregulated CoT (SFT) initially assumed vertex-only rectangles but later recognized that diagonal-intersection vertices also counted. SR2AM-v1.0-30B anticipated fallbacks, while its SFT counterpart spent 25+ turns on blocked AoPS workarounds without a fallback strategy.Source: Example 2: Mathematical Reasoning… · pp. 28–29

Example 3: Over-Verification (HLE)

• 0/3 HLE trials succeeded for SR2AM-v1.0-30B after over-verification, versus 3/3 for both SR2AM-v1.0-30B (SFT) and Unregulated CoT (SFT).SR2AM-v1.0-30B used 73 messages and omitted required hyphens; the two SFT comparators used 13 and 11 messages, respectively. The authors identify stopping-criterion calibration as unresolved.Source: Example 3: Over-Verification (HLE) · pp. 29–30

K Example Trajectories from Trained Instantiations

• Trained trajectories retain SFT-established structure while RL refines planning content, distinguishing configuration, simulated plans, reflection, and tool execution.v0.1 uses explicit configurator, planning, and reflection blocks, whereas v1.0 uses free-form reasoning and “Plan:” text. The v0.1 example critiques and revises its initial plan before executing its first action.Source: K Example Trajectories from Trained… · pp. 30–31
• SR2AM-v1.0-30B adapts planning to task progress: a detailed initial plan, skipped planning during routine error recovery, and shorter horizons near completion.The GPQA Diamond trajectory demonstrates these choices through Plan Reconstruction, illustrating control over both planning occurrence and horizon.Source: K Example Trajectories from Trained… · p. 31
• Example trajectories combine multi-step planning, selective replanning, planning skips, and shorter final plans while answering web-search and physics questions.The web trajectory identifies Jacob W. Crisp; the physics trajectory selects option C, corresponding to an energy uncertainty of order 10^-16 J.Source: K Example Trajectories from Trained… · pp. 34–35

L SR2AM-v0.1 Data Collection Routine

• 30 turns per attempt and 3 retries bound v0.1 collection; only correct trajectories with more than 3 tool calls are retained.At each turn, the teacher configurator may invoke intent, planning, reflection, and summary modules before selecting an action.Source: L SR2AM-v0.1 Data Collection Routine · p. 37

M Supervised Data Collection Prompts for Multi-Module Inference (v0.1)

• The v0.1 collection prompts prescribe intent analysis, planning, reflection-driven plan revision, tool execution, and summaries that guide subsequent steps.Specialist prompts identify user needs and challenges, diagnose earlier tool mistakes, and summarize useful information or milestones.Source: M Supervised Data Collection Prompts… · p. 38

N Supervised Data Annotation Prompts for Plan Reconstruction (v1.0)

• The v1.0 annotator reconstructs optional high-level state–action plans, leaving plans empty when execution follows an existing plan.Annotations must preserve original actions and available thoughts without revealing specific future observations, calculations, or answers; placeholders and contingencies are allowed. A predicted step may represent multiple real steps.Source: N Supervised Data Annotation Prompts… · p. 39

O System Prompt

• The system prompt requires at least one tool call, search or browsing verification before using information, and Python for all computation or data analysis.Available tools support web search, webpage browsing, and Python execution.Source: O System Prompt · p. 39