The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs

• Inference-time scaling marks a shift from training-time scaling, with longer test-time reasoning improving complex-task performance, while deployment budgets remain finite and expensive.Source: §1 · p. 1
• The paper formalizes a three-phase, S-shaped compute-utility pattern: uniform limits can leave difficult queries below emergence thresholds while giving easy queries diminishing-return compute.The phases are Strict, Surge, and Ample.Source: §1 · pp. 1–2, Figure 1 · p. 1
• At the batch optimum, active queries receive tokens until their marginal utility equals a global shadow price, while queries below that price receive no budget.This frames allocation over instance-specific, non-concave latent utility curves as a global constrained optimization problem.Source: §1 · p. 2
• CLEAR estimates query thresholds, discovers the market-clearing price by bisection, and uses a Lambert W closed-form policy to allocate tokens, truncate, or rationally abandon queries.It is a plug-and-play inference wrapper requiring no backbone retraining.Source: §1 · p. 2
• CLEAR improves cost-accuracy Pareto efficiency on mixed-complexity mathematical reasoning benchmarks and is robust to hyperparameter choices and predictor noise.Source: §1 · p. 2

2. Empirical Motivation

• The motivation is that each query may have a favorable reasoning length, with utility rising after a minimum threshold before saturating or declining when generation becomes excessive.Source: §2 · p. 2
• The controlled experiment sampled diverse reasoning paths with Qwen2.5-Math-7B at T = 1.0 and measured conditional Pass@1 across trajectory-length bins.It used N = 50 responses for AIME-24 and N = 4 for GSM8K and MATH-500.Source: §2 · p. 2
• Reasoning utility follows three nonlinear regimes: short trajectories fail, performance surges past a minimum threshold, and additional generation brings diminishing returns or may degrade quality.The passages describe these as the Strict, Surge, and Ample phases.Source: §2 · pp. 2–3

3. Problem Formulation

• Per-query reasoning utility follows a Strict–Surge–Ample structure, making token allocation a global constrained optimization over an instance-dependent, non-concave utility landscape.Source: §3 · p. 3

3.1. Modeling the Physics of Reasoning

• The model treats binary correctness as governed by a continuous, unobservable reasoning utility ϕ(t).Source: §3.1 · p. 3
• In the Strict regime, utility remains zero until generation length crosses the latent emergence threshold τi.Source: §3.1 · p. 3
• In the Surge regime, reasoning potential rises with initial velocity αi after the threshold is crossed.Source: §3.1 · p. 3
• In the Ample regime, the exponential decay term e−βi∆t dominates as generation extends excessively, reflecting diminishing returns.Source: §3.1 · p. 3

3.2. The Global Optimization Objective

• The objective allocates tokens across N queries to maximize aggregate reasoning potential under a global budget Btotal, with token cost Ci(t) = t.Source: §3.2 · p. 3

4.1. Shadow Price Parity

• At an optimum, each active query’s marginal reasoning gain equals the global shadow price, determining whether it remains in Strict, enters Surge, or is capped in Ample.The shadow price is the marginal gain in total potential from relaxing the budget by one token.Source: §4.1 · p. 3, §4 · p. 3
• Queries whose maximum net surplus is negative receive zero tokens under the Rational Abandonment Condition.The per-query interior solution is valid only when its net surplus is positive; otherwise, the optimum is the zero-allocation boundary.Source: §4.1 · p. 3, §4.2 · p. 4

4.2. Individual Optimal Allocation Policy

• For a query in its active reasoning phase, equating marginal surge potential with the price yields an allocation policy solved in closed form using the principal Lambert W branch.The surge function is ϕi(t) = αi∆te^−βi∆t, with ∆t = t − τi; the closed-form policy is for a given shadow price λ.Source: §4.2 · pp. 3–4

4.3. Global Equilibrium and Phase Transitions

• The hard global budget is met by finding a unique market-clearing price, guaranteed because each query’s allocation decreases strictly as the price rises.The total budget inversely determines the equilibrium price.Source: §4.3 · p. 4, policy · p. 4

policy

• As the budget tightens, the higher shadow price favors queries near their Surge threshold; queries with λ exceeding their initial velocity αi are abandoned.Such difficult queries cannot generate enough marginal surplus to justify crossing the threshold and remain before Strict.Source: policy · p. 4
• With an abundant budget, the shadow price approaches zero and active queries approach the peak of their utility curves near the Surge-to-Ample transition.The allocation converges to ti → τi + 1/βi.Source: policy · p. 4

5. Methodology

• CLEAR numerically solves for a global shadow price that equilibrates batch demand with token supply.This translates the theoretical optimal policy into a practical inference-time control algorithm.Source: §5 · p. 4

5.1. Threshold Prediction and Parametric Scaling

• DeBERTa-v3-base predicts query-specific emergence thresholds, while global α sets the reservation price and β adapts to aggregate budget surplus.CLEAR treats α and β as backbone-level global hyperparameters, with query heterogeneity primarily captured by predicted τi.Source: §5.1 · p. 5

5.2. Global Shadow Price Optimization

• CLEAR bisects λ from 0 to α, evaluates Lambert W allocations with solvency-based abandonment and Tmax caps, then adjusts price according to excess demand or supply surplus.The individual allocation is monotonic in λ, enabling the bisection search for the price that clears the global budget.Source: §5.2 · p. 5

Models and Training Data

• Qwen2.5-Math-7B-Instruct and Qwen3-30B-A3B-Instruct are frozen backbones, while a DeBERTa-v3-base predictor estimates emergence thresholds from solution-length training data.The predictor regresses logarithmic generated-solution length and is trained exclusively on GSM8K and MATH splits.Source: §6 · p. 5

Evaluation Datasets and Streams

• Evaluation uses a mixed oracle pool spanning six reasoning datasets and four synthetic streams: Balanced, Mostly-Easy, Mostly-Hard, and U-Shaped.Each stream contains n=500 queries sampled from the 7B oracle pool.Source: §6 · pp. 5–6

Baselines and Allocation Policies

• CLEAR is compared with Uniform, Predictor, TALE-EP, two internal ablations, and Oracle, which provides an upper bound using ground-truth solution lengths.Predictor allocates tokens proportional to predicted thresholds without abandonment; TALE-EP estimates token demand and renormalizes allocations to the same global budget. CLEAR (Heuristic) uses affine allocation with a median-based rejection cutoff, while CLEAR (Auction) greedily admits queries by predicted return on investment.Source: §6 · pp. 5–6
• The designation CLEAR (Lambert) refers to the full proposed pipeline, distinguishing it from the internal CLEAR variants used for ablation.The paper uses CLEAR (Lambert) 1 to mark the full pipeline.Source: §6 · p. 5

7.1. Latent Threshold Modeling

• The predictor’s key allocation role is ranking tasks by latent threshold, while the global shadow price λ absorbs absolute token-consumption prediction errors.Figure 4 analyzes log-scale regression performance and threshold-tier token-count distributions.Source: §7.1 · p. 6, Figure 4 · p. 6

7.2. Main Performance Comparison

• CLEAR consistently improves over uniform allocation across all three code-generation benchmarks under aligned best-of-4 completion-token budgets.The evaluation uses Qwen2.5-Coder-7B with an in-domain length regressor; the allocation algorithm remains unchanged.Source: §7.2 · p. 6, Table 2 · p. 8
• At 256 tokens per query, CLEAR beats Uniform by +11.6 accuracy points in Balanced, +24.0 in Mostly-Easy, +5.2 in Mostly-Hard, and +14.2 in U-Shaped streams.Absolute gains shrink as budgets rise, when most methods can cross the latent emergence threshold.Source: §7.2 · p. 6, Table 1 · p. 7
• With 256 tokens, CLEAR abandons queries whose cost outweighs potential gain and redistributes their resources to admitted queries, adjusting extra allowance by decay rate β.Uniform and predictor-based policies instead spread insufficient tokens across queries, leaving most below their ground-truth reasoning lengths.Source: §7.2 · pp. 6, 8

7.3. Robustness and Sensitivity

• CLEAR retains a significant advantage over Uniform even at high predictor-noise levels, although accuracy declines as log-normal noise increases.Static β cannot suit both low-budget strictness and high-budget extended reasoning; adaptive β tracks aggregate surplus and reaches the Pareto-optimal frontier across regimes.Source: §7.3 · p. 8
• Varying initial velocity α causes negligible accuracy fluctuations while the market-clearing shadow price λ∗ scales linearly with α.The allocation depends on λ/α, so the optimizer adjusts λ∗ to offset changes in α.Source: §7.3 · p. 8

7.4. Structural Utility Variants

• Both triangular and quadratic utility variants remain substantially above Uniform in the tight-budget regime, indicating gains do not depend on one utility curve.Both preserve threshold-aware global budget clearing and rational abandonment.Source: §7.4 · p. 9, Table 3 · p. 9

Resource Allocation for LLM Inference.

• Prior LLM inference work allocates scarce resources through model routing, query-allocation formulations, and analyses of hardware-level throughput, latency, and energy tradeoffs.Examples include FrugalGPT (Chen et al., 2023), RouteLLM (Ong et al., 2024), LLM-Bandit and TREACLE (Zhang et al., 2024), and hardware analyses (Kwon et al., 2023; Sheng et al., 2023; Wilhelm et al., 2025).Source: §8 · p. 9

Efficient CoT.

• Efficient CoT methods adapt reasoning traces within queries, including difficulty- and confidence-based sampling, instance-specific budgets, and prompt-guided token budgets.DSC (Wang et al., 2025) varies sampled paths; SelfBudgeter (Li et al., 2025) estimates and follows instance-specific budgets; TALE (Han et al., 2025) estimates a token budget and injects it into the prompt. CoT and inference-time scaling can improve reasoning performance (Wei et al., 2022; Snell et al., 2024; Wu et al., 2024; Wan et al., 2025).Source: §8 · p. 9
• In contrast, CLEAR allocates a fixed global token budget across a batch with fixed decoding behavior, deciding each query’s budget and when to rationally abandon it.Source: §8 · p. 9

9. Conclusion

• CLEAR improves the cost-accuracy Pareto frontier, with its largest gains in resource-scarce regimes where uniform allocation spreads computation too thinly.The conclusion identifies improving robustness of utility modelling as future work toward narrowing the gap to oracle-level allocation.Source: §9 · p. 9

A.1. Pseudocode; A.2. Allocation Policy Details

• The appendix defines per-query allocation t_i under total budget B_total, distinguishes predicted thresholds from oracle-only ground-truth lengths, and presents pseudocode for computing the optimal allocation.Source: §A.2 · p. 11, §A.1 · p. 11

A.2.1. UN I F O R M POLICY

• Uniform allocation distributes resources equally across queries, without accounting for task complexity.Source: §A.2.1 · p. 11

A.2.2. PR E D I C T OR-BASED PROPORTIONAL POLICY

• The predictor-based proportional policy scales budgets to predicted lengths under a linear threshold-to-token assumption and does not abandon queries.Source: §A.2.2 · p. 11

A.2.3. TALE-EP POLICY

• TALE-EP uses a stronger teacher model to estimate each query’s token need, renormalizes estimates into budget-constrained caps, then regenerates with soft budget prompts and hard decoding caps.Source: §A.2.3 · p. 11

A.2.4. CLEAR(HE U R I S T I C) POLICY

• When average budget is below 0.8 times the mean predicted threshold, CLEAR (Heuristic) drops queries above the median threshold and reallocates the full budget among the rest.The policy’s selected queries share the budget using an affine allocation rule scaled to conserve the total budget.Source: §A.2.4 · p. 11

A.2.5. CLEAR(AUCTION) POLICY

• CLEAR (Auction) sorts queries by predicted threshold, retains the largest prefix whose thresholds fit the total budget, and shares the budget among survivors using the heuristic’s affine rule.Nonselected queries receive zero budget.Source: §A.2.5 · p. 12

A.2.6. OR A C L E POLICY

• The Oracle upper-bound baseline uses unavailable-at-test-time ground-truth solution lengths to spend tokens on the shortest solvable queries first, maximizing completed tasks under the budget.Source: §A.2.6 · p. 12

B. Appendix: More Results

• Up to +2.4 accuracy gains over uniform are reported for CLEAR across tested supply-demand conditions, with the largest gains in strictly constrained budgets.Table 4 details downstream allocation performance across varying supply-demand scenarios.Source: B. Appendix: More Results · p. 12
• On Qwen3-30B-A3B-Instruct, Figure 10 evaluates the threshold predictor’s accuracy as a test of framework generalizability to larger reasoning models.The passage says Figure 10 validates predictor accuracy but gives no specific accuracy value.Source: B. Appendix: More Results · p. 12, Figure 10 · p. 12

C. Appendix: Data Composition and Statistics

• The evaluation suite is intentionally OOD relative to the threshold predictor’s training set.Tables 6 and 7 provide detailed breakdowns of training and evaluation datasets.Source: C. Appendix: Data Composition and… · p. 12, Table 6 · p. 14, Table 7 · p. 14
• CLEAR maintains a performance advantage over Uniform even under significant predictor noise σ.Figure 11 presents CLEAR versus Uniform under increasing predictor noise; the supplied passages do not give numerical values.Source: C. Appendix: Data Composition and… · p. 13, Figure 11 · p. 13

3. CLEAR Allocation Mechanism

• CLEAR uses α = 2.0 and finds the allocation with bisection search over 40 iterations at ϵ = 1e −6.Source: §3 · p. 14

4. Evaluation Scenarios

• Evaluation simulations use N = 5, 00 queries per stream and draw from MATH-500, OlympiadBench, AIME (24/25), AMC-23, and Minerva.Source: §4 · p. 14
• Dataset statistics cover Qwen-2.5-math-7B-Instruct with greedy decoding and a 4K new-token constraint, and Qwen3-30B-A3B-Instruct with greedy decoding and a 16K constraint.For Qwen3-30B-A3B-Instruct, threshold tiers are assigned using the average reasoning length of the solutions.Source: Table 6 · p. 14, Table 7 · p. 14
• The allocation algorithm uses a batch-adaptive decay rate and sets a query’s allocation to zero when λ ≥ α or its computed Δt_i is nonpositive.Source: §4 · p. 15