The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs

Xu Wan; Speed Zhu; Jianwei Cai; Guang Chen; XiMing Huang; Wiggin Zhou; Mingyang Sun

Source-linked AI summary

The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs

Xu Wan, Speed Zhu, Jianwei Cai, Guang Chen, XiMing Huang, Wiggin Zhou, Mingyang Sun

arXiv:2606.03092v2Jun 2, 2026cs.AI

TL;DR

Under strict global compute budgets, the paper asks how to allocate inference-time reasoning across heterogeneous queries to maximize utility. It formulates allocation economically and introduces CLEAR, which improves the cost-accuracy Pareto frontier, especially when resources are scarce.

Problem
Strict global compute budgets make maximizing utility across heterogeneous reasoning queries the central inference-time allocation challenge.
Method
CLEAR models threshold-aware reasoning utility and uses global shadow-price discovery with a closed-form Lambert W policy to allocate or abandon queries.
Results
CLEAR improves cost-accuracy Pareto efficiency across mixed-complexity mathematical reasoning benchmarks, with the largest gains in resource-scarce regimes.
Takeaways & Limitations
Targeted allocation can avoid spending tokens on queries unlikely to be solved and redirect computation toward queries more likely to benefit.
Takeaways & Limitations
The method treats utility-shaping parameters as global backbone-level hyperparameters because predicting query-specific parameters is computationally intractable and high variance.

Abstract

from arXiv · show

Inference-time scaling has emerged as a critical avenue for enhancing Large Language Models' performance, yet real-world deployment is constrained by strict computational budgets. In this work, we formulate inference budget allocation as a global constrained optimization problem governed by economic principles. By modeling per-query reasoning utility with a shifted-surge function, we derive an optimal allocation policy based on a global shadow price that equilibrates marginal utility under resource scarcity. Based on this theory, we propose Constrained Latent-utility Equilibrium Allocation for Reasoning (CLEAR). It performs rational abandonment and reallocates resources from insolvent queries to solvable queries near their emergence thresholds. Extensive experiments on several reasoning tasks with different traffic streams demonstrate that CLEAR significantly improves the Pareto frontier of total token cost versus mean accuracy. In resource-scarce regimes, CLEAR achieves up to a 3x improvement in global accuracy compared to uniform allocation.

1. Introduction

The introduction frames inference-time reasoning as a powerful but budget-constrained resource, motivating global allocation policies that account for heterogeneous query utility. It presents CLEAR, an economic allocation framework governed by a global shadow price and validated for improved cost-accuracy efficiency.

Motivation: Inference-time scaling can deliver substantial reasoning gains, but deployment faces finite, expensive compute and strict global budget constraints.The paper situates inference-time scaling as a shift from training-time scaling and links longer thinking to improvements on complex reasoning tasks.
Motivation: Uniform token limits conflict with heterogeneous reasoning tasks because queries follow different S-shaped compute-utility profiles.The introduction highlights that difficult queries may fail to reach useful reasoning regimes under uniform allocation, while easy queries can be over-provisioned.
Economic formulation: The allocation problem maximizes aggregate expected utility under a fixed total token supply, with active queries receiving tokens until marginal utility matches a common global shadow price.This principle applies to instance-specific, non-concave latent utility curves despite the overall objective being non-convex.
CLEAR: CLEAR models each query with a surge-shaped latent utility curve, estimates its emergence threshold, and uses bisection search to discover the unique market-clearing shadow price.The framework combines threshold modeling and price discovery rather than assigning every query the same utility or token limit.
Contributions: CLEAR improves cost-accuracy Pareto efficiency and remains robust to hyperparameter choices and predictor noise on mixed-complexity mathematical reasoning benchmarks.The paper also derives a closed-form Lambert W allocation policy and presents CLEAR as a plug-and-play framework.

2. Empirical Motivation

The empirical motivation shows that reasoning-token utility is query-dependent and nonlinear: performance may rise after a minimum solvable threshold, then exhibit diminishing or negative returns with excessive generation.

Utility dynamics: Per-query reasoning utility is expected to rise after a minimum threshold, then saturate or decline as generation becomes excessive.This motivates modeling an instance-specific favorable reasoning length rather than assuming longer reasoning is uniformly beneficial.
Experimental setup: A controlled Qwen2.5-Math-7B experiment sampled diverse reasoning paths at T = 1.0 and grouped trajectories by length to compute conditional Pass@1.The study generated N = 50 responses for AIME-24 and N = 4 for GSM8K and MATH-500.
Utility dynamics: Reasoning utility follows three regimes: Strict failure on short trajectories, Surge after the solvable threshold, and Ample diminishing or negative returns.The Ample phase indicates that additional generation can eventually degrade solution quality.

3. Problem Formulation

The paper models per-query reasoning utility as a Strict–Surge–Ample process and formulates token assignment as a global constrained optimization problem. The objective maximizes aggregate reasoning potential under a fixed total token budget using a pure token-based cost model.

3. Problem Formulation: Per-query reasoning utility follows a Strict–Surge–Ample structure, producing an instance-dependent, non-concave utility landscape.This structure motivates treating token assignment as a global constrained optimization problem.
3.1. Modeling the Physics of Reasoning: Binary task outcomes are modeled as governed by a continuous, unobservable reasoning utility ϕ(t).For query s_i, accumulated potential after t generated tokens follows a Shifted Surge Function.
3.1. Modeling the Physics of Reasoning: Strict behavior keeps utility at zero until generation length crosses the latent emergence threshold τ_i.The threshold marks when reasoning potential can begin to accumulate.
3.1. Modeling the Physics of Reasoning: Surge behavior begins after threshold crossing, with initial velocity α_i capturing rapid accumulation of valid reasoning.The parameter α_i represents the initial rise in reasoning potential.
3.1. Modeling the Physics of Reasoning: Ample behavior reflects diminishing returns when excessive generation makes the exponential decay term e^−β_i∆t dominant.This regime captures declining utility from extending generation too far.
3.1. Modeling the Physics of Reasoning: The fitted latent reasoning potential provides a continuous utility mapping from rollout outcomes observed at different generation lengths.Figure 2 visualizes this mapping with a red curve.
3.2. The Global Optimization Objective: Under a pure token-based cost model, C_i(t) = t, tokens are allocated across N queries to maximize aggregate reasoning potential under budget B_total.The allocation vector is t = [t_1, . . . , t_N].

4. Theoretical Analysis

Theoretical analysis formulates optimal inference allocation through a global shadow price that equalizes marginal reasoning potential under a relaxed budget constraint. It combines Lambert W-based per-query allocation with rational abandonment and market-clearing resource allocation across operating regimes.

Shadow-price allocation: Lagrangian relaxation establishes equalization of marginal reasoning potential as the economic principle governing optimal inference under a global budget.The Lagrange multiplier represents the marginal gain in total potential from relaxing the budget by one token.
Shadow-price allocation: KKT conditions require every active task’s marginal gain to match the global shadow price, determining whether queries enter Surge or are capped in Ample.This parity also governs whether a query is lifted from Strict.
Rational abandonment: Queries with nonpositive maximum net surplus are economically insolvent and receive zero budget under the Rational Abandonment Condition.The interior allocation is valid only when its resulting net surplus is positive.
Individual allocation policy: For the surge potential ϕ_i(t) = α_i∆te^−β_i∆t, the fixed-price optimal allocation is obtained in closed form using the principal branch W_0 of the Lambert W function.The solution follows from equating marginal potential with the shadow price and solving the resulting transcendental equation.
Global market clearing: A unique market-clearing price λ* enforces the hard global budget because allocated tokens decrease strictly with λ, while budget size determines the operating regime.Large budgets drive λ* toward zero and allocations toward τ_i + 1/β_i; when λ* > α_i, difficult tasks are abandoned.

5. Methodology

CLEAR operationalizes the theoretical optimal policy by numerically finding a global shadow price that balances batch token demand with supply. It predicts query emergence thresholds, shapes utility curves with global hyperparameters, and uses bisection with Lambert W allocations, rational abandonment, and context limits.

Algorithm overview: CLEAR numerically solves for the shadow price λ∗ that equilibrates aggregate batch token demand with global token supply.This replaces heuristic truncation or abandonment rules with a practical inference-time control algorithm.
Threshold prediction: CLEAR estimates each query’s emergence threshold τi—the minimum computation needed to enter productive reasoning—using a DeBERTa-v3-base predictor fθ.The threshold is the core input to the Lambert W policy.
Utility parameterization: CLEAR uses initial velocity α as a reservation price and decay rate β to control marginal-utility decline beyond the predicted threshold.Queries with λ∗ exceeding α receive zero tokens, while β is adapted dynamically from aggregate budget surplus.
Utility parameterization: Because query-specific βi prediction is intractable and high-variance, CLEAR treats α and β as global hyperparameters while modeling task heterogeneity through predicted τi.The hyperparameters characterize average reasoning dynamics of the LLM backbone rather than individual queries.
Market clearing: CLEAR finds λ∗ by bisection over [λmin, λmax], evaluating Lambert W allocations, rational abandonment, and the physical context limit Tmax at each candidate price.The bounds are λmin = 0 for saturation and λmax = α for total abandonment; excess demand raises the price, while supply surplus lowers it.
Market clearing: Market clearing updates the search interval by comparing total consumption Ctotal with the global budget Btotal.When Ctotal exceeds Btotal, CLEAR sets λmin ← λmid; with a supply surplus, it sets λmax ← λmid.

6. Experimental Settings

The experiments use two frozen Qwen backbones, a DeBERTa-v3-base threshold predictor trained on GSM8K and MATH, and six reasoning benchmarks across four synthetic traffic streams. CLEAR is compared with uniform, predictor-based, external, internal ablation, and oracle allocation policies.

Models and Training Data: Experiments use Qwen2.5-Math-7B-Instruct and Qwen3-30B-A3B-Instruct as frozen backbones for all reasoning tasks.A DeBERTa-v3-base encoder predicts latent emergence thresholds by regressing logarithmic generated-solution length.
Models and Training Data: The threshold predictor is trained exclusively on GSM8K and MATH splits, using generated-solution token length as a proxy for emergence threshold.The predictor estimates τ(s) through logarithmic token-length regression.
Evaluation Datasets and Streams: Evaluation spans MATH-500, AMC-23, AIME-24, AIME-25, Minerva, and OlympiadBench across Balanced, Mostly-Easy, Mostly-Hard, and U-Shaped streams.Each synthetic stream contains n=500 queries sampled from the 7B oracle pool.
Baselines and Allocation Policies: CLEAR is benchmarked against Uniform, Predictor, TALE-EP, CLEAR (Heuristic), CLEAR (Auction), and Oracle allocation policies.Uniform evenly divides the global budget; Predictor allocates proportional to predicted threshold without abandonment; Oracle uses ground-truth solution lengths.
Baselines and Allocation Policies: The internal ablations apply median-based rejection with affine allocation or greedily admit queries by predicted return on investment under the budget constraint.CLEAR (Lambert) denotes the full proposed pipeline, distinguishing it from internal CLEAR variants.

7. Results and Analysis

CLEAR improves resource-constrained reasoning by ranking queries through latent thresholds, abandoning insolvent tasks, and reallocating saved resources to solvable queries. Its advantages extend across mathematical reasoning, code generation, predictor noise, adaptive hyperparameters, and alternative utility structures.

Predictor Performance: High rank correlation indicates the predictor captures relative reasoning thresholds, while the global shadow price absorbs absolute token-prediction errors.Allocation therefore depends primarily on correctly ranking tasks by latent threshold rather than exactly predicting token consumption.
Mathematical Reasoning: +24.0 accuracy points is CLEAR’s gain over Uniform in the Mostly-Easy stream at the 256-token budget, exceeding gains of +11.6, +5.2, and +14.2 points in Balanced, Mostly-Hard, and U-Shaped streams.The largest improvements occur under the most constrained budget, where uniform allocation wastes scarce tokens on queries unable to produce sufficient utility.
Generalization to Code Generation: CLEAR improves over uniform allocation across all three code benchmarks under aligned best-of-4 completion-token budgets.The code-generation evaluation uses Qwen2.5-Coder-7B with an in-domain length regressor while keeping the allocation algorithm unchanged.
Robustness and Sensitivity: CLEAR maintains a significant advantage over Uniform even at high predictor-noise levels and adaptively calibrates β to remain on the Pareto-optimal frontier.Low budgets favor strict, high β, whereas high budgets favor lenient, low β; static β cannot satisfy both regimes.
Structural Utility Variants: Both Triangular and Quadratic utility variants remain substantially above uniform allocation under the tight-budget setting, indicating benefits derive from budget clearing and abandonment-aware allocation.The variants preserve threshold-aware allocation while changing the latent utility shape.

8. Related Work

Related work studies LLM inference as a constrained resource-allocation problem and improves reasoning efficiency by adapting model routing, query allocation, or chain-of-thought computation. Existing approaches include cascades, preference-based routing, contextual allocation methods, and difficulty- or confidence-aware reasoning procedures.

Resource Allocation for LLM Inference: LLM inference research frames computation, latency, and tokens as scarce assets allocated under global constraints.This line of work treats deployment as a resource-allocation problem.
Resource Allocation for LLM Inference: FrugalGPT uses model cascades to reduce cost, while RouteLLM learns preference-based routing policies across models.Both systems address deployment efficiency through query routing or model selection.
Efficient CoT: CoT prompting and inference-time scaling improve reasoning performance by allocating more computation at test time.This work motivates efficient reasoning methods that adapt how inference computation is used.
Efficient CoT: DSC allocates sampled reasoning paths according to question difficulty and posterior confidence.It improves token efficiency by adapting the reasoning procedure itself.

9. Conclusion … A.2.6. OR A C L E POLICY

The paper frames fixed-budget token allocation across heterogeneous reasoning queries as a constrained optimization problem and introduces CLEAR to redirect resources toward queries more likely to be solved. The appendix specifies CLEAR and comparison policies, including uniform, predictor-based, teacher-based, heuristic, auction, and oracle allocations.

9. Conclusion: CLEAR addresses wasted tokens by assigning zero budget to queries with negative expected surplus and reallocating resources toward queries that targeted allocation can solve.This policy is motivated by the mismatch between uniform budgeting, hard queries that cannot be solved within budget, and overlooked solvable queries.
A. Appendix: Algorithm Details; A.1. Pseudocode: Algorithm ?? presents pseudocode for CLEAR, which computes an optimal resource-allocation vector under a specified total budget constraint.The pseudocode supports the batch-level allocation formulation described in the appendix.
A.2. Allocation Policy Details: The allocation framework defines per-query tokens t_i for a batch S under total budget B_total, using predicted thresholds τ̂_i and average budget B̄ = B_total/N.Ground-truth solution lengths d_i are reserved for the Oracle policy.
A.2.1. UN I F O R M POLICY; A.2.2. PR E D I C T OR-BASED PROPORTIONAL POLICY: Uniform distributes resources equally across queries, while predictor-based proportional allocation scales budgets with predicted required length without abandonment.Uniform is agnostic to task complexity; proportional allocation assumes a linear relationship between latent threshold and required tokens.
A.2.3. TALE-EP POLICY: TALE-EP uses a stronger teacher model to estimate each query’s token need, renormalizes estimates under the total budget, and regenerates with soft prompt constraints and hard decoding caps.The teacher estimate may come from Qwen3-30B-Instruct.
A.2.4. CLEAR(HE U R I S T I C) POLICY: CLEAR(Heuristic) activates when η = B̄/E[τ̂] < 0.8, drops queries above Median(τ̂), and reallocates the full budget among the remaining queries.The selected queries use exactly B_total tokens, with κ chosen to enforce budget conservation.
A.2.5. CLEAR(AUCTION) POLICY: CLEAR(Auction) sorts queries by increasing predicted threshold, keeps the largest prefix fitting within B_total, assigns zero to nonselected queries, and shares the full budget among survivors.Its survivor allocation uses the same affine rule as CLEAR(Heuristic), rescaled to satisfy Σ_i t_i = B_total.
A.2.6. OR A C L E POLICY: The Oracle is an upper-bound baseline that uses unavailable ground-truth lengths d_i to spend tokens on the shortest solvable queries first.It sorts by true length and greedily allocates each query exactly d_oi tokens until the next allocation would exceed B_total.

B. Appendix: More Results

Additional experiments with Qwen3-30B-A3B-Instruct demonstrate the framework’s generalizability to larger-scale reasoning models. CLEAR consistently outperforms uniform allocation across tested supply-demand conditions, with gains of up to +2.4 accuracy in strictly constrained budgets.

Larger-Scale Model Results: Experiments using Qwen3-30B-A3B-Instruct demonstrate the framework’s generalizability to larger-scale reasoning models.Figure 10 evaluates the threshold predictor, while Table 4 examines downstream allocation performance.
Allocation Performance: CLEAR consistently outperforms the uniform baseline across all tested supply-demand conditions.Table 4 reports downstream allocation performance across varying supply-demand scenarios.
Allocation Performance: +2.4 accuracy gains are achieved in strictly constrained budget environments.These gains occur where efficient resource distribution is most critical.

C. Appendix: Data Composition and Statistics

The appendix documents the datasets, evaluation settings, and predictor configuration used to assess CLEAR. The evaluation suite is designed to be Out-of-Distribution relative to the threshold predictor’s training data, with threshold tiers based on average solution reasoning length.

Data Composition: The evaluation suite is intentionally Out-of-Distribution relative to the threshold predictor’s training set.Training and test datasets are detailed in Tables 6 and 7.
Experimental Configuration: The threshold predictor uses DeBERTa-v3-base with 86M parameters, left-truncated inputs retaining the last 512 tokens, and a 512-token maximum sequence length.Training uses mean squared error on log-length, AdamW with LR=2e-5 and weight decay=0.01, batch size 32, for 10 epochs.
Dataset Statistics: Qwen-2.5-math-7B-Instruct datasets are evaluated under greedy decoding and 4K new token constraints.The threshold tiers are assigned based on the average reasoning length of these solutions.
Dataset Statistics: Qwen3-30B-A3B-Instruct datasets are evaluated under greedy decoding and 16K new token constraints.The threshold tiers are assigned based on the average reasoning length of these solutions.