PaperlayerView exact revision on arXiv

Source-linked AI summary

Think More or Read More? A Token-Budget Theory of Reasoning with Shared Context

Manh Nguyen, Sunil Gupta, Hung Le

arXiv:2610.23246v1cs.CLcs.LG

TL;DR

When reasoning systems reuse generated text, reading consumes tokens that could otherwise support reasoning in a shared window. The paper develops a token-budget theory to predict debate group size from single-agent measurements, then finds that self-consistency matches or beats deployed debate at most matched operating points.

  • Problem

    Debate and self-refinement add context but consume part of the same fixed token window needed for reasoning, raising the question of when that context is worth its cost.

    Source: §1 · p. 1

  • Method

    The theory models agent ability from remaining reasoning tokens and combines votes with a correlation-aware Condorcet model to predict optimal group size from single-agent measurements.

    Source: §1 · p. 2

  • Results

    Self-consistency matches or beats deployed debate at 93% of 980 matched operating points, with the gap largest under tight budgets and nearly disappearing at a loose budget.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Under a shared window, the theory predicts that communication pays only when its gain in accuracy ceiling exceeds the reasoning room spent reading.

    Source: §1 · p. 3, §5 · p. 10

    TB-MAD is calibrated per model and task, and a group size chosen outside the regime covered by its probe sweep can lose accuracy without signaling that it has.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Reasoning systems increasingly improve answers by incorporating additional context into the model's input, from multi-agent debate, where agents read their peers' responses, to self-refinement, where a model revisits its own drafts. In these systems, communication consumes context as one agent's output becomes another's input. We study the regime where token reading and generation draw on one fixed token window, so every token spent reading is a token that cannot be spent reasoning. We develop a \textbf{token-budget theory} that formalizes this trade-off, modeling accuracy as a function of the reasoning budget remaining after reading and aggregating votes with a Condorcet jury model that permits correlated errors. For multi-agent debate, the theory gives a closed-form prediction for the optimal number of agents. This quantity is estimated from single-agent measurements, allowing the optimal group size to be determined without running debate, and the same accounting extends to a heterogeneous set of different models. We empirically verify our theory and show that, at the predicted group size, our method outperforms strong baselines under the tightest calibration spend, where debate-calibrated rules can afford only one or two questions. Self-refinement obeys the same accounting, with its own draft in place of a peer message. The theory further predicts that communication pays only when the gain in the accuracy ceiling exceeds the reasoning room spent on reading, with this condition tightening as the window shrinks. Across five open models and four reasoning tasks, self-consistency matches or beats deployed debate at $93$\% of $980$ matched window-and-group-size settings, with the gap being largest under tight windows and nearly disappearing at a loose one, consistent with the theory. Under a shared window, how much room a model has left to \textit{think matters more than what it reads}.

1 INTRODUCTION

The paper frames peer messages and self-feedback as useful context that competes with reasoning for a shared token window. It develops a closed-form way to choose debate group size and finds that self-consistency often matches or beats deployed debate under matched windows.

1 INTRODUCTION

  • Under a fixed token budget, reading peer messages uses tokens that otherwise could support reasoning, so larger debate groups can leave too little room to think.Prior group-size searches require debate runs and do not explain why an operating point is optimal or when adding agents stops helping.Source: §1 · pp. 1–2
  • The token-budget theory predicts an optimal debate size in closed form from single-agent measurements, without running debate, and extends to heterogeneous model sets.It models ability as a function of remaining reasoning room and aggregates votes with a Condorcet jury model that permits correlated errors.Source: §1 · pp. 2–3
  • At the predicted group size, the method incurs only a 1.4% accuracy gap from a per-cell oracle, using 26–79 calibration questions depending on the cell, while debate-calibrated rules use one or two at matched spend.Source: §1 · p. 3
  • Self-consistency matches or beats deployed debate at 93% of 980 matched operating points; the gap is largest under tight budgets and nearly disappears at a loose budget.Self-refinement’s accuracy collapses faster than debate’s as rounds accumulate because it rereads drafts without voting amplification.Source: §1 · p. 3
  • The paper’s contributions include tests indicating that reading cost drives debate collapse and the interior optimum, with robustness tests covering longer reasoning and practical token pricing.Its theory also characterizes when debate’s voting benefits outweigh communication cost relative to self-consistency.Source: §1 · p. 3

2 RELATED WORK

Related work addresses debate communication costs, budget-matched comparisons, and group-size selection, but often treats reading and reasoning costs separately. This paper positions its approach as predicting the operating point from single-agent calibration rather than debate runs.

2 RELATED WORK

  • Prior debate work reduces communication through sparse speakers, group partitioning, or retaining conflicting messages, while other studies compare debate with self-correction or long reasoning chains under matched budgets (Huang et al., 2024; Tran & Kiela, 2026; Bertalanic & Fortuna, 2026).Work on heterogeneous panels explores diversity as another route to improving debate (Zhang et al., 2025; Zhu et al., 2026; Nilayam et al., 2026).Source: §2 · p. 3
  • Existing methods search or learn the debate operating point but require debate runs; Chen et al. (2024) refit by budget, whereas this paper transfers from a single-agent cap sweep.Liu et al. (2026) derive a closed-form collapse threshold, but their simulation-calibrated concave ability curve cannot capture near-zero accuracy below a solution-length threshold, and their fixed per-agent generation allowance excludes reading costs.Source: §2 · p. 3
  • Token-budget research allocates a fixed budget across samples or reasoning length for one agent, while debate comparisons report that voting can outperform communication under matched sampling or search budgets (Kaesberg et al., 2025; Wang et al., 2024a).These approaches leave peer text outside the budget account; Wunderlich et al. (2026) report the opposite ordering under per-agent rather than shared contexts.Source: §2 · pp. 3–4

3 METHOD

The method treats reading and generation as competing uses of one token window, estimates per-agent ability from capped single-agent reasoning, and combines agents with a correlation-aware voting model. This yields an interior optimal group size when the window is scarce and clarifies the comparison with self-consistency and self-refinement.

3 METHOD

  • TB-MAD uses a single-agent probe sweep to predict a debate group size without running debate.The method formalizes the shared-window budget and converts per-agent ability into a group-accuracy prediction.Source: §3 · p. 4

3.1 PROBLEM FORMULATION

  • Each debate agent retains the question and n−1 peer messages, leaving g = W − H − c tokens for reasoning; retaining full transcripts would make the measured penalty a lower bound.Self-refinement instead rereads its own last draft, while n=1 debate without peer messages corresponds to self-consistency.Source: §3.1 · p. 4

3.4 THEORY: THE COLLAPSE REGIME

  • With positive reading cost, adding agents reduces each agent’s thinking room and can produce collapse; when conditions for an interior optimum hold, the synergy index has a unique interior maximum.As the window grows without bound, thinking room stops binding and the classical correlation ceiling returns.Source: §3.1 · p. 4, §3.4 · pp. 6–7

3.2 PER-AGENT ABILITY AND THE THINKING GAIN

  • Per-agent ability βeff(g) is measured by capping a single agent’s generation at g tokens, and the thinking-gain function ϕ captures the fraction of ceiling ability reached at that cap.The paper selects a sigmoid HILL form based on held-out error; the cap sweeps are near zero below solution length, then rise and saturate.Source: §3.2 · pp. 4–5

3.3 GROUP ACCURACY AND THE SYNERGY INDEX

  • A correlation-aware Condorcet model combines homogeneous agents’ votes, assuming equicorrelated errors with ρ estimated at round zero before peer reading.Its aggregation head maps per-agent ability to group accuracy; the signed synergy index balances the declining per-agent edge against voting amplification.Source: §3.3 · pp. 5–6

3.4 THEORY: THE COLLAPSE REGIME

  • At matched aggregation, self-consistency’s initial-round accuracy is at least debate’s; debate can exceed it only if peer text raises the accuracy ceiling enough.For self-refinement, rereading a draft pays a reading cost without vote amplification, so its accuracy is bounded by the untouched-agent ceiling unless rereading raises pmax.Source: §3.3 · p. 6, §3.4 · p. 7

4 EXPERIMENTS

The experiments evaluate TB-MAD against calibration baselines under equal token spending, then test how debate, self-refinement, and reading costs behave across settings. TB-MAD performs best under tight calibration budgets, while ablations link group-size collapse and the interior optimum to reading costs.

4.1 SETUP

  • The evaluation covers five open instruction models across four reasoning benchmarks, compares eight alternatives, and reports held-out regret against a per-cell oracle.The test set is split into 40% for calibration and 60% for evaluation; runs use three seeds.Source: §4.1 · p. 7

4.2 MAIN RESULTS

  • 1.40% average regret, below 2% across all models and eight equal-spend comparisons, makes TB-MAD the strongest reported calibration rule.It stays within 1% of the best rule at six of seven windows, trailing by 1.16% at W=2500.Source: §4.2 · p. 8
  • At equal spend, TB-MAD has the lowest regret, while debate-based baselines improve as spend grows and Successive Halving overtakes it at the largest budget.With equal questions, its plain regret is among the lowest at 10 questions and stays roughly flat beyond 60.Source: §4.2 · p. 8
  • In heterogeneous settings, probe-calibrated methods are strongest, and Learned βeff edges ahead of TB-MAD.TB-MAD trades a small amount of regret for a closed form and an explicit model-selection gate.Source: §4.2 · p. 8
  • Self-refinement can reduce accuracy under tight windows before refinement helps, and useful gains require both headroom and self-correction ability.The resulting gains are modest relative to the added token cost.Source: §4.2 · p. 8
  • Self-consistency matches or beats debate at 93% of 980 matched operating points, with its advantage largest under tight windows and nearly gone at loose windows.A paired bootstrap finds self-consistency significantly better in 57% of cases and debate in 0.5%; the remaining 42.5% are indistinguishable.Source: §4.2 · p. 9

4.3 ABLATIONS

  • Setting the peer-reading token cost m to zero removes the interior optimum and collapse, predicting the largest feasible group at every window.With m > 0, reading cost reduces shared thinking room as agents are added.Source: §4.3 · p. 9
  • Making input tokens more expensive reduces performance, while TB-MAD retains the lowest regret at each tested price ratio; moving the question changes but does not eliminate the collapse cell.At fixed H and g, moving the question to the transcript end changes accuracy by less than 1% on average.Source: §4.3 · pp. 9–10

5 CONCLUSION

The conclusion presents a shared-window theory that predicts optimal debate group size and when a single agent can outperform debate. Experiments find lowest regret for the method under tight budgets, while self-consistency often matches or outperforms debate at lower cost.

5 CONCLUSION

  • The paper places reading and thinking in one token window to predict optimal group size and identify when a single agent can outperform debate.It concludes that self-consistency often matches or outperforms debate at lower cost, while TB-MAD achieves the lowest regret under tight budgets.Source: §5 · p. 10

AI USE STATEMENT

  • The authors report using generative AI only for writing support, with all assisted text reviewed and verified by the authors.They state that AI was not used to develop research ideas, conduct experiments, or generate substantive content.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • TB-MAD is calibrated per model and task, so choosing a group size outside the probe-sweep regime can lose accuracy without a warning signal.The work uses public benchmarks and open-weight models, with no human subjects or personal data.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The appendix contains proofs, implementation details, heterogeneous-setting analyses, and additional results supporting the main text.The authors state that they will release code, banked debate transcripts, and figure and table generation scripts upon publication.Source: REPRODUCIBILITY STATEMENT · p. 10, APPENDIX · p. 15

A.1 SINGLE-PEAKEDNESS (PROP. 1)

The theory establishes a unique interior optimal group size when adding agents helps, and characterizes how the optimum changes with correlation and available window. It also identifies the large-window limit and a condition under which self-consistency weakly dominates debate.

A.1 SINGLE-PEAKEDNESS (PROP. 1)

  • The accuracy objective has a unique interior maximum n* when some larger group helps, under the stated log-concavity conditions.The shifted edge δ crosses zero once, defining the feasible interval for the interior optimum.Source: §A.1 · pp. 15–16
  • As error correlation ρ rises, the optimal group size n* strictly decreases because extra votes provide less amplification.More correlated errors buy less from an extra vote at every group size.Source: §A.1 · p. 17

A.2 THE LARGE-WINDOW LIMIT

  • As the window grows, n* is non-decreasing but grows no faster than the feasibility cap, widening the gap between them.This bound holds on window ranges where the maximizer remains interior.Source: §A.2 · p. 17
  • When the window is no longer scarce, the accuracy objective becomes monotone increasing; with ρ > 0 it saturates at the classical correlation ceiling.For ρ = 0, effective sample size is unbounded and there is no ceiling.Source: §A.2 · p. 17

A.3 SELF-CONSISTENCY WEAKLY DOMINATES DEBATE (PROP. 2)

  • Self-consistency weakly dominates the initial-round debate vote because it leaves at least as much room for reasoning, and equality holds at n = 1.The ability bound carries through the aggregation head.Source: §A.3 · p. 17

B IMPLEMENTATION DETAILS

The implementation fits per-agent ability from single-agent reasoning-cap sweeps, estimates error correlation, and compares operating-point selection methods under matched calibration costs. The heterogeneous extension selects model sets using model-specific abilities and within- and cross-model error correlations.

B.1 IMPLEMENTING TB-MAD

  • TB-MAD fits (pmax, g0, κ) to single-agent cap-sweep accuracies using sample-weighted nonlinear least squares, then uses RMSE to choose among candidate forms.R2 is at least 0.993 in every cell but is not used alone to select the form.Source: §B.1 · p. 18
  • On Qwen2.5-7B and Falcon3-7B, fitted Hill g0 is within 4% of median completion length on GSM8K and 15% on MMLU-Pro.The median completion lengths are measured at the largest generation cap, 4096 tokens.Source: §B.1 · p. 18, Table 6 · p. 19
  • Estimated round-0 error correlation varies by at most 0.05 across group sizes and windows in the evaluated model-task cells.The paper reports that correlation appears to be a property of the model and task on these cells.Source: §B.1 · p. 19

B.2 IMPLEMENTING BASELINES

  • Baselines include debate-calibrated search methods and probe-only alternatives, including learned βeff and the Star exponent.The debate-calibrated methods include empirical argmax, successive halving, and a quadratic surface; the probe-only alternatives use single-agent measurements.Source: §B.2 · pp. 19–20

B.3 CALIBRATION COST

  • Calibration costs are charged for correlation estimates, and debate re-reads peer messages, giving it the largest input-token multiple among the listed rules.Each correlation-calibration question requires at least three independent round-0 answers.Source: §B.3 · p. 21, Table 11 · p. 22

C GENERALIZATION TO HETEROGENEOUS SETTINGS

  • The heterogeneous setting assigns equal numbers of agents to each selected model, with each model's ability determined by its own fitted parameters.The objective is to choose both the total number of agents and the model set.Source: C GENERALIZATION TO HETEROGENEOUS… · pp. 21–22

C.1 EXTENDED METHOD

  • With model blocks, effective sample size depends on within- and cross-model error correlations; mixing raises the asymptotic ceiling when cross-model errors are less correlated.The stated condition is ρb < ρw.Source: §C.1 · pp. 22–23
  • When model edges have mixed signs, the selection gate permits positive-edge singletons; otherwise, it reduces to the homogeneous rule.A negative-edge model can dilute the group edge when mixed with positive-edge models.Source: §C.1 · pp. 22–23

C.2 UPDATED THEORY ANALYSIS

The heterogeneous aggregation result gives an accuracy head and an equivalent optimization objective when the average edge is positive. The extension's dominance and interior-optimum results have stated conditions, and the authors note that the latter results are not experimentally isolated.

C.2 UPDATED THEORY ANALYSIS

  • Under uniform allocation, common variance, and within- and between-model correlations, the heterogeneous aggregation head gives group accuracy and preserves the accuracy-maximizing objective when the average edge is positive.In that case, maximizing accuracy is equivalent to maximizing effective sample size times squared average edge.Source: §C.2 · p. 23
  • Self-consistency dominance still holds when no model's accuracy ceiling rises from peer text, but can fail if a weaker member gains ceiling from information it lacks.The latter case is described by Δpmax,i > 0 for some member.Source: §C.2 · p. 23
  • For heterogeneous averages, the interior-optimum argument extends under stated restrictions, but log-concavity need not hold away from the feasibility boundary.The authors report no experiments isolating these two results.Source: §C.2 · pp. 23–24

C.3 RESULTS

Across heterogeneous model sets, probe-calibrated methods are strongest at every K, with the method’s relative standing depending on set size. The evaluation uses model sets of differing sizes and capabilities across four reasoning tasks.

C.3 RESULTS

  • The evaluation covers model sets of similar and differing capability on GSM8K, MATH500, MMLU-Pro, and CSQA, with members speaking cyclically by set position.The tested K=2 sets use n ∈ {2, 4, 6, 8}; the K=4 sets use n ∈ {4, 8}.Source: §C.3 · p. 24
  • Probe-calibrated methods are strongest at every K; at K=2 the method is second best, while for K > 2 the star exponent matches learned βeff.Removing the gate worsens average regret from 1.49 to 1.54 on pairs and from 1.66 to 2.75 on larger sets.Source: §C.3 · p. 24, Table 13 · p. 25
  • At the saturation window, both R=2 and R=3 self-refinement remain within 7% of first-draft accuracy.Figure 3 plots self-refinement accuracy against the first draft as the window grows; R=1 is generation without refinement.Source: Figure 3 · p. 25

D.1 SELF-REFINEMENT, FULL RESULTS

Self-refinement can reduce accuracy when extra drafts leave too little room for generation. Its gains depend on model capability and available headroom, and remain small relative to added token cost.

D.1 SELF-REFINEMENT, FULL RESULTS

  • At the smallest window, increasing refinement from R=1 to R=3 reduces accuracy by 1.3–44.3 points, with losses recovering as the window grows.More refinement steps require a larger window before they become beneficial.Source: §D.1 · p. 25
  • A genuine interior best round appears mainly for mid-capability models with both headroom and self-correction ability, including Falcon3 and Gemma-2.Most models on MATH500 also show an interior best round.Source: §D.1 · p. 25
  • Self-refinement gains remain small relative to the additional token cost, while Llama3.2-3B gains almost nothing and ceiling models have little headroom.The ceiling examples are Qwen and Gemma-3 on GSM8K.Source: §D.1 · p. 25

D.2 VOTING AGAINST TALKING, AND THE SINGLE AGENT

Under tight windows, peer-reading costs can make self-consistency outperform debate, while the comparison narrows as windows grow. The section also tests when debate helps, where its predictions fail, and how token pricing affects operating choices.

D.2 VOTING AGAINST TALKING, AND THE SINGLE AGENT

  • Self-consistency weakly dominates debate, and forming a group is not always worthwhile when δ ≤ 0.The self-consistency margin is largest when the generation window binds and shrinks as the window loosens.Source: §D.2 · p. 25
  • The self-consistency margin falls as the window loosens, from +42.0% to +13.7% when clipped cells are excluded, and is under 1pp from W=6k onward in the extended comparison.The large tight-window margin is partly a clipping effect; from W=6k through 32k, margins stay within seed noise.Source: §D.2 · pp. 25–26, Table 16 · p. 27
  • On CSQA, the closed form poorly recovers the δ sign, most severely for Gemma3-12B, where a steep fit to seven calibration points overfits and c_w estimates are least stable.All five models show the same direction of error on CSQA.Source: §D.2 · p. 26
  • Peer reading changes per-agent accuracy ceilings by only 2–4% in magnitude, with positive changes for Falcon3-7B and negative changes for Qwen2.5-7B on both tested tasks.The comparison fixes g=2048 and isolates peer text from voting.Source: §D.2 · p. 26, Table 15 · p. 26
  • At matched generated-token budgets, the debate group leads a single-agent long-CoT on every cell, while self-consistency recovers the same advantage without message exchange.The comparison is therefore one of breadth versus depth rather than communication.Source: §D.2 · pp. 26–27, Table 17 · p. 27

D.3 REPRICING AT API RATES

  • The interior cost-optimal group size persists under realistic pricing, with cheaper input shifting n∗ upward at matched budget; on Qwen2.5-7B/MMLU-Pro, n=1 remains optimal regardless of price ratio.All n ≥ 2 points collapse far below n=1 there due to the generation-floor effect.Source: §D.3 · p. 27
  • Discounted input costs slightly improve debate-calibrated methods’ coverage and performance, while TB-MAD retains the lowest regret at budgets of 1, 2, and 4M across every price ratio.Source: §D.3 · p. 27

D.4 IS THE ERROR CONCENTRATION cw CONSTANT?

The appendix tests assumptions and sensitivities in the theory’s predictions, finding that error concentration can vary with reasoning room and that the generation floor’s effects depend on task requirements.

D.4 IS THE ERROR CONCENTRATION c_w CONSTANT?

  • Eighteen of twenty cells’ error-concentration estimates drift by at most 0.05 over group sizes, but concentration rises with reasoning room and a fixed c_w overstates it at the bottom of the window range.Using c_w(n) moves n∗ at 26 of 140 operating points and cuts mean held-out regret from 1.16pp to 0.80pp.Source: §D.4 · p. 27

D.5 DOES σ DEPEND ON n?

  • The derived n-dependent σ changes predictions at only 7 of 140 operating points, always by one agent, so the paper retains the simpler fitted constant.Six shifts are downward and one is upward.Source: §D.5 · p. 28

D.6 SENSITIVITY OF THE PREDICTION TO m.

  • With no per-agent reading cost (m=0), the prediction selects nmax at every window; increasing m lowers n∗ at fixed W, and m=320 is closest to the oracle among the three positive values swept.The empirical argmax stays flat and low across the sweep.Source: §D.6 · pp. 28–29

D.7 GENERATION-FLOOR RECOVERY.

  • Accuracy recovers as the generation floor crosses g0, with GSM8K near ceiling by gmin=512 while MMLU-Pro continues climbing past 1024.The result supports a task-dependent reasoning requirement captured by g0, rather than an effect unique to the reported gmin=64 floor.Source: §D.7 · p. 29

E LIMITATIONS AND FUTURE WORK

The analysis has several scope boundaries: its aggregation model is an approximation, experiments use relatively small models with short solutions, and the accounting assumes a shared window for reading and generation.

E LIMITATIONS AND FUTURE WORK

  • The aggregation model approximates group behavior using equicorrelation, fixed error concentration, and an unweighted average-based decision threshold; consensus-weighted aggregation is not considered.Heterogeneous model sets are investigated, but allocation across agent counts remains uniform; adaptive allocation is left for future work.Source: E LIMITATIONS AND FUTURE WORK · p. 29
  • The analysis covers self-generated computation, including debate and self-refinement, but not externally retrieved or supplied context such as retrieval-augmented generation.Source: E LIMITATIONS AND FUTURE WORK · p. 29
  • The experiments cover five models of at most 12B parameters with fitted solution lengths of 137–478 tokens, so operating points may differ for models with thousand-token reasoning chains.The authors expect the qualitative shape to transfer, but not the specific operating points.Source: E LIMITATIONS AND FUTURE WORK · p. 29
  • Additional self-refinement rounds increase visibility and may shift the trade-off if their revision benefit does not offset the added context cost.The paper explores up to three rounds for self-refinement.Source: E LIMITATIONS AND FUTURE WORK · p. 30
  • The account assumes one window pays for both reading and generation; under per-agent contexts, reading does not consume generation room and the interior optimum disappears.In the authors’ accounting, per-agent contexts correspond to m = 0.Source: E LIMITATIONS AND FUTURE WORK · p. 30
Loading 2610.23246v1…