PaperlayerView exact revision on arXiv

Source-linked AI summary

Agent-Conditioned Routing: Diverse Reasoning Paths Within One Frozen Model

Ananth Kalyanasundaram, Ankush Kadu, Athira Gopal, Aswanth Krishnan

arXiv:2610.37375v1cs.AIcs.LGcs.MA

TL;DR

Additional generations can repeat a dominant reasoning approach, leaving uncertainty about whether they produce distinct methods. This paper introduces Agent-Conditioned Routing, which varies expert eligibility across trajectories in one frozen MoE. On a screened benchmark, ACR yields more problems with at least two correct primary methods than the strongest temperature setting on both evaluated backbones.

  • Problem

    Repeated generations can paraphrase one dominant approach, and answer-level agreement does not establish whether derivations use meaningfully different methods.

    Source: §1 · p. 1

  • Method

    ACR assigns persistent shared/private expert-eligibility masks to parallel trajectories in one frozen MoE while preserving the learned token router.

    Source: §1 · p. 1

  • Results

    ACR yields approximately twice as many problems with at least two correct primary methods as the strongest temperature setting under GLM and 1.67 times as many under GPT-5.6 Sol.

    Source: §6 · p. 9

  • Takeaways & Limitations

    ACR provides an inference-time control for reasoning diversity within one frozen MoE without user-assigned roles or prescribed methods.

    Source: §6 · p. 9

    The screened benchmark is conditioned on multiple solution paths and at least one correct solution, and broader task and model coverage remains future work.

    Source: §4.1 · p. 4, §5.5 · p. 9

Abstract

from arXiv · show

Generating several solutions can expose alternative methods, but additional trajectories often paraphrase one dominant approach. We investigate whether the conditional capacity of a frozen sparse mixture-of-experts (MoE) model can provide such diversity from one problem prompt, without user-assigned roles, prescribed methods, fine-tuning, or model copies. Agent-Conditioned Routing (ACR) assigns $m$ parallel trajectories persistent, untrained shared/private expert-eligibility masks while preserving the learned, input-dependent token router. On a screened 126-question multi-mode mathematics and physics benchmark, with model judges reading the reported derivations in full, we count the number of problems yielding at least two distinct primary methods among soft-correct solutions in four trajectories. ACR reaches approximately twice as many such problems as the strongest temperature setting under GLM 5.3 (37.3 versus 18.7), and 1.67 times as many under GPT-5.6 Sol (45.00 versus 27.00), at matched output budgets. A blinded four-expert audit on a stratified sample of 30 problems also finds more multi-method solutions with ACR on both backbones, with lower correctness. Matched-eligibility ablations show modest multi-method count advantages for ACR, while token-resampled masks attain higher correctness. An informed sequential prompt elicits more diversity still, using previous solutions and explicit requests for alternative methods. ACR adds about 160 KiB of mask state. In a three-problem microbenchmark capped at 1,024 tokens, its latency and peak VRAM usage remain within 3% of batched temperature sampling.

1 INTRODUCTION

The paper asks whether frozen-MoE routing constraints can diversify executed reasoning methods while preserving correctness, rather than merely changing generated text. It introduces ACR and a 126-question multi-mode benchmark to evaluate distinct primary methods among correct trajectories.

1 INTRODUCTION

  • ACR uses expert eligibility as a training-free inference control, seeking to change executed derivations while retaining correctness.The method targets autonomous diversification from one problem prompt, without requiring users to specify alternative techniques.Source: §1 · pp. 1–2
  • Four route-conditioned trajectories run in one batched request on a single frozen Qwen3.6-35B-A3B instance.Each trajectory receives the same prompt and deterministic decoding, with persistent shared/private eligibility masks and the learned router selecting top-k experts within each eligible set.Source: §1 · pp. 1–2
  • The benchmark measures how often four trajectories yield at least two distinct primary methods among soft-correct solutions, assessing correct method diversity under a fixed generation budget.It comprises 126 questions across 12 mathematics and physics categories; full-benchmark evaluations by two model judges are complemented by a blinded four-expert audit on a stratified 30-problem subset across three seeds.Source: §1 · pp. 1–2

2 RELATED WORK

The related work distinguishes answer-level inference-time scaling from methods that target reasoning-path diversity. ACR varies expert eligibility across independent trajectories while keeping the prompt, weights, and decoding rule fixed.

2.1 INFERENCE-TIME DIVERSITY FROM ONE MODEL

  • Answer-level success from repeated samples does not establish that the sampled derivations execute different methods.Prior work includes self-consistency, tree search, self-refinement, and test-time compute scaling.Source: §2.1 · p. 2
  • ACR holds the prompt, weights, and decoding rule fixed while varying expert eligibility, and evaluates primary executed methods among correct trajectories.This differs from approaches that target output variation through decoding, prompting, or training.Source: §2.1 · p. 2

2.2 PARALLEL-AGENT REASONING

  • Unlike parallel-agent systems that combine outputs, exchange critiques, or share partial state, ACR differentiates trajectories through routing constraints within one frozen backbone.The trajectories generate independently without separate weights, role prompts, or inter-agent communication.Source: §2.2 · p. 2

2.3 EXPERT ROUTING AS AN INFERENCE-TIME CONTROL

  • ACR uses persistent shared/private eligibility masks, with the pretrained router selecting experts within each mask; matched-cardinality controls examine persistence separately from mask construction.The paper contrasts this with token-level routing-noise controls RoE and Expert-Sample.Source: §2.3 · p. 3

3 METHOD

ACR partitions expert eligibility into shared and trajectory-specific sets while retaining the frozen model’s learned router. The masks persist through a routed prefix, and trajectories are executed in a shared batched model instance.

3.1 PERSISTENT SHARED/PRIVATE ELIGIBILITY

  • At each MoE layer, a seeded permutation selects a shared core and divides remaining experts into disjoint private sets for the trajectories.The construction constrains distributed computation rather than assuming that individual experts represent distinct semantic skills.Source: §3.1 · p. 3
  • The pretrained router selects its usual top-k experts within each trajectory’s eligible set, and the mask persists throughout the routed prefix.At P = E, trajectories reduce to deterministic copies of the base model; decreasing P increases private capacity but risks excluding useful experts.Source: §3.1 · p. 3
  • The four trajectories share weights and execute along the batch dimension while retaining separate token histories and KV caches.In the hybrid configuration, constrained routing ends after a 2,000-token prefix, after which unfinished trajectories continue with unrestricted routing in the same model.Source: §3.1 · p. 3

4 EXPERIMENTS

The experiments use a screened multi-mode mathematics and physics benchmark, assess solution correctness and method diversity, and report resource comparisons. The selected benchmark population and short-generation resource measurements bound the scope of the evaluation.

4.1 TASKS, MODEL, AND CONDITIONS

  • The 126-question benchmark retains problems that showed multiple solution paths and at least one correct solution in six temperature samples.It is drawn from a 200-problem, 12-category candidate pool and is a diagnostic set conditioned on evidence of multiple paths in temperature-generated outputs.Source: §4.1 · pp. 3–4

4.2 JUDGING PROTOCOL

  • Judges distinguish soft-correct answers from method diversity, counting distinct substantive mathematical routes rather than rephrasing or reordering equivalent steps.UCM@m counts distinct routes among soft-correct trajectories; evaluations use raw predictions, with GLM 5.3 and GPT-5.6 Sol serving as separate judges.Source: §4.2 · p. 5

4.3 RESOURCE ACCOUNTING

  • In a 1,024-token microbenchmark, ACR remains close to batched temperature sampling in latency and peak memory, while RoE and sequential prompting take longer.The cap does not reach the 2,000-token handoff, so the measurements do not characterize the full 4,096-token hybrid configuration; Table 1 covers four completions on three screened problems.Source: §4.3 · p. 5, Table 1 · p. 5

5 RESULTS

Across a screened 126-problem benchmark, ACR yields more problems with at least two correct primary methods than the strongest temperature setting on both model judges. Human audit results also favor ACR, while ablations and scope limitations qualify the findings.

5.1 BENCHMARK SCALE: THE SCREENED 126

  • 37.3 versus 18.7 multi-method problems for ACR P = 160 versus temperature T = 0.9 under GLM 5.3; GPT-5.6 Sol reports 45.00 versus 27.00.Both comparisons average three seeds; Figure 2 shows the GLM correctness–diversity comparison across pool sizes.Source: §5.1 · p. 6, Figure 2 · p. 7
  • 14.29 percentage points (95% CI: [8.20, 20.37]; sign-flip p = 1.49 × 10−5) separate ACR from temperature T = 0.9 in paired GPT-5.6 Sol comparisons.Intervals favor ACR over each listed blind comparator but favor informed DivPrompt over ACR; routing-noise controls produce fewer multi-method problems despite high correctness, while informed sequential prompting yields the highest GLM diversity.Source: §5.1 · p. 7, Table 4 · p. 7

5.2 BLINDED HUMAN AUDIT

  • In the blinded 30-problem audit, Qwen ACR (P = 160) yielded 12.0 vs 9.0 problems with at least two methods (96.4% vs 98.3% correctness); Gemma ACR (P = 80) yielded 14.0 vs 7.0 (86.2% vs 100.0%).Four experts evaluated outputs with model and setting identities hidden; counts are means across three seeds. Qwen P = 96 had the largest three-method count but substantially lower correctness.Source: §5.2 · p. 8, Table 5 · p. 8

5.3 READING THE DERIVATIONS: ROLLING DOWN AN INCLINE

  • All eight rolling-incline trajectories reach a = g sin α/(1 + κ), but ACR includes Lagrangian derivations while all temperature samples use force and torque balance.Energy checks appear in ACR texts but receive no extra primary-method credit.Source: §5.3 · p. 8, Table 6 · p. 9

5.4 ABLATION STUDIES

  • At matched eligibility, ACR has 5.33 more multi-method solutions than independent masks and 2.67 more than resampled masks, but correctness is 1.52 percentage points below resampling.Switching adds eight EOS completions and one Sol-judged correct trajectory on 20 problems, with unchanged aggregate diversity.Source: §5.4 · p. 8, Table 7 · p. 9
  • Both judges rank P = 160 above P = 192 and P = 96 in diversity; small pools can induce looping, while larger pools restore correctness but reduce diversity.On Gemma-4-26B-A4B, fraction-matched P = 80 yields 52 versus 21 multi-method problems with a 2.7-percentage-point correctness reduction; this is a one-seed comparison.Source: §5.4 · pp. 8–9

5.5 LIMITATIONS AND FUTURE WORK

  • The screened benchmark targets problems with multiple solution methods, the human audit covers 30 problems without quantified reviewer agreement, and broader task and model coverage remains future work.Matched-eligibility contrasts are descriptive; paired uncertainty and expert assessment of intermediate-step validity are outside the evaluation.Source: §5.5 · p. 9

6 CONCLUSION

ACR increases multi-method outcomes relative to temperature sampling at matched output budgets, while correctness is lower; informed sequential prompting yields greater diversity still.

6 CONCLUSION

  • ACR yields at least two correct primary methods on approximately twice as many problems as the strongest temperature setting under GLM and 1.67 times as many under GPT-5.6 Sol.A blinded four-expert audit supports the direction of these gains on both backbones, with lower correctness for ACR.Source: §6 · p. 9
  • Matched-eligibility controls retain much of the diversification effect, with modest multi-method count advantages for the full construction.The matched handoff comparison covers 20 problems at P = 160, with correctness and diversity judged by GPT-5.6 Sol.Source: §6 · p. 9, Table 8 · p. 9
  • Informed sequential prompting achieves greater diversity using previous attempts, whereas ACR explores alternatives independently in one batched request.The backbone-ablation table covers Gemma-4-26B-A4B on one seed and 126 problems; its caption does not report outcome values.Source: §6 · p. 9, Table 9 · p. 10

AI STATEMENT

The paper used AI tools for manuscript polishing, development-set generation, and solution judging. Its reproduced LLM evaluation prompt asks judges to assess full solutions and count distinct correct methods.

AI STATEMENT

  • AI tools polished the manuscript, generated the synthetic 11-problem development set, and served as judges for solution evaluation.The paper distinguishes model-based evaluations from the blinded human audit.Source: AI STATEMENT · p. 11

A LLM JUDGING PROMPT

  • The evaluation prompt asks models to read raw predictions fully and count unique correct methods, treating rephrased or reordered steps as the same method.It allows soft-correct solutions when the answer appears in the reasoning despite formatting or boxed-answer issues.Source: A LLM JUDGING PROMPT · p. 14
  • The prompt requests pool-wise UCM@4 and counts of problems with at least two and at least three solutions.Each solution is assigned to one trajectory, so a problem can have at most four distinct trajectories.Source: A LLM JUDGING PROMPT · p. 14

B PAIRED MEAN-UCM@4 COMPARISONS

Paired mean-UCM@4 comparisons favor ACR over the listed blind comparators and favor informed DivPrompt over ACR. A selected rolling-incline example contrasts executed methods across routed and temperature trajectories.

B PAIRED MEAN-UCM@4 COMPARISONS

  • Paired-bootstrap intervals exclude zero in favor of ACR against the listed blind comparators, and in favor of informed DivPrompt against ACR.Table 10 reports paired UCM@4 comparisons for ACR at P = 160 over three seeds, with mean UCM@4 approximately 1.3915.Source: B PAIRED MEAN-UCM@4 COMPARISONS · p. 14, Table 10 · p. 14

C ROLLING-INCLINE TRAJECTORY COMPARISON

  • The selected comparison uses four trajectories per problem under a 4,096-token cap, with ACR at P = 160 and temperature sampling at T = 0.9.The ACR mask seed is 2; the temperature settings are top-p = 0.95 and seed 0.Source: C ROLLING-INCLINE TRAJECTORY COMPARISON · p. 14
  • In the selected rolling-incline example, ACR trajectories execute multiple derivations, while every temperature trajectory eliminates friction using the same Newtonian equations.The ACR routes include a constraint-and-multiplier derivation, an energy derivation, and force-and-torque balance; the example is not an aggregate benchmark result.Source: C ROLLING-INCLINE TRAJECTORY COMPARISON · p. 14

D DEVELOPMENT AND HELD-OUT PROBES

Development probes compare method counts and correctness, while case studies illustrate different routes and a held-out problem. A rendered inclusion–exclusion example separates primary proof routes from additional checks.

D DEVELOPMENT AND HELD-OUT PROBES

  • At P = 192, the 11-problem development set reaches 93% soft-correctness and 2.00 distinct methods per problem, versus 95% and 1.27 for temperature T = 0.5.These method counts include all trajectories and are separate from correctness-filtered benchmark UCM@4; the development set was used for configuration selection.Source: D DEVELOPMENT AND HELD-OUT PROBES · pp. 14–15
  • On a held-out Putnam problem, 50 temperature samples yield two observed methods while eight routed trajectories expose a third.Figure 3 also reports a development comparison; its caption does not provide the plotted values.Source: Figure 3 · p. 15

E DEVELOPMENT CASE STUDIES

  • Case studies show ACR using reciprocal symmetry, series, Mellin differentiation, and an odd-function transformation for an integral, and four proof approaches for Tsirelson’s bound.For Basel, cosine-series and sine-series/Parseval derivations count separately only when used as primary derivations.Source: E DEVELOPMENT CASE STUDIES · p. 15

F INCLUSION–EXCLUSION: RENDERED SEED-0 SOLUTIONS

  • In the inclusion–exclusion example, ACR completes three primary routes—element counting, disjoint-region algebra, and recursive two-set identity use—while all temperature trajectories complete the first route.Later checks receive no extra primary-method credit, and all four trajectories from each method reach the identity.Source: F INCLUSION–EXCLUSION: RENDERED SEED-0… · pp. 15–16

G FREQUENCY OF THE 2,000-TOKEN HANDOFF

Saved-run analysis counts explicit routing handoffs and estimates near-cap prefixes when handoff flags are unavailable. Across three seeds at P = 160, about a quarter of trajectories and nearly half of problem-runs meet the near-cap criterion.

G FREQUENCY OF THE 2,000-TOKEN HANDOFF

  • 101 switched trajectories (20.04%) occur on 48 problems (38.10%) in the P = 192, seed-2 run with explicit switch flags.The run records both prefix-token counts and a switch flag for each trajectory.Source: G FREQUENCY OF THE 2,000-TOKEN HANDOFF · p. 16, Table 11 · p. 16
  • Earlier-run cap attainment is estimated by retokenizing prefixes, so counts are not exact switch counts; continuation text alone does not identify cap-triggered handoffs.Decoding removes special tokens and retokenization can change token boundaries; the analysis does not measure looping or correctness gains from switching.Source: G FREQUENCY OF THE 2,000-TOKEN HANDOFF · pp. 16–17
  • 382 of 1,512 trajectories (25.26%) and 183 of 378 problem-runs (48.41%) meet the 1,995-token near-cap criterion at P = 160 across three seeds.Problem-runs count each problem separately in each seed; the table distinguishes estimated near-cap rows from a recorded-flags row.Source: Table 11 · p. 16, G FREQUENCY OF THE 2,000-TOKEN HANDOFF · p. 16
  • 373 trajectories (24.67%) and 180 problem-runs (47.62%) meet a stricter 2,000-token threshold; a 1,990-token threshold gives the same counts as 1,995 tokens.For the run with explicit flags, the 1,995-token estimate agrees with the recorded counts.Source: G FREQUENCY OF THE 2,000-TOKEN HANDOFF · pp. 16–17

H MATCHED SWITCH-VERSUS-NO-SWITCH EVALUATION

In a matched continuation comparison on 20 sampled problems, switching routing eligibility yielded more EOS completions but only one additional soft-correct trajectory, with aggregate diversity metrics unchanged.

H MATCHED SWITCH-VERSUS-NO-SWITCH EVALUATION

  • The comparison samples 20 benchmark problems and uses P = 160, mask seed 0, four trajectories, greedy decoding, repetition penalty 1.05, thinking disabled, and a 4,096-token output cap.The 20 problems include one randomly selected problem from each of 12 categories and eight sampled from the remainder, without consulting outcomes.Source: H MATCHED SWITCH-VERSUS-NO-SWITCH… · p. 17
  • Both conditions share a masked prefix of up to 2,000 tokens, then continue unfinished trajectories for up to 2,096 tokens with either unrestricted routing or the original eligibility mask.EOS-completed trajectories are retained unchanged; both conditions rebuild the prefix cache and use the same active batch, controlling restart mechanics while varying continuation routing eligibility.Source: H MATCHED SWITCH-VERSUS-NO-SWITCH… · p. 17
  • Among 28 paired unfinished continuations, 22 reach EOS with the switch versus 14 without it, yielding eight additional EOS completions and one additional soft-correct trajectory.Aggregate diversity metrics are equal across the two conditions.Source: H MATCHED SWITCH-VERSUS-NO-SWITCH… · p. 17
  • Only 28 of 80 trajectories remain unfinished at the prefix boundary, across 10 of the 20 problems; the subset supports a completion benefit, not universal necessity for correctness.The repetition heuristic flags three full trajectories in the switch condition and four in the no-switch condition; EOS completion is distinct from mathematical correctness.Source: H MATCHED SWITCH-VERSUS-NO-SWITCH… · p. 17

Related papers

Loading 2610.37375v1…