PaperlayerView exact revision on arXiv

Source-linked AI summary

EAGer: Entropy-Aware GEneRation for Adaptive Inference-Time Scaling

Daniel Scalena, Leonidas Zotos, Elisabetta Fersini, Malvina Nissim, Ahmet Üstün

arXiv:2510.11170v2cs.LGcs.AIcs.CL

TL;DR

Parallel reasoning can spend computation regenerating predictable prefixes while allocating uniform sampling effort to prompts with different exploration needs. EAGER branches at high-entropy tokens and reallocates saved budget; it reports performance gains alongside reduced token use.

  • Problem

    Parallel reasoning can redundantly regenerate predictable prefixes while allocating the same sampling effort to simple and complex prompts with different exploration needs.

    Source: §1 · p. 2

  • Method

    EAGER monitors token-level uncertainty, branches at high-entropy tokens, and reallocates unused sampling budget toward more challenging prompts.

    Source: §1 · p. 2

  • Results

    Across math, science, and coding tasks, tested LMs show up to 37% performance boost over FULL PARALLEL; with known answers, EAGER raises Pass@k by up to 37% while using up to 65% fewer tokens.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    EAGER directs saved sampling budget toward challenging prompts, aiming to maximize inference-time scaling benefits for difficult problems.

    Source: §1 · p. 2

    Full EAGER requires target labels to identify failing prompts for budget reallocation, restricting direct use to settings with verifiable answers.

    Source: Limitations · p. 8

Abstract

from arXiv · show

With the rise of reasoning language models and test-time scaling methods as a paradigm for improving model performance, substantial computation is often required to generate multiple candidate sequences from the same prompt. This enables exploration of different reasoning paths toward the correct solution, however, allocates the same compute budget for each prompt. Grounded on the assumption that different prompts carry different degrees of complexity, and thus different computation needs, we propose EAGer, a training-free generation method that leverages model uncertainty through token-wise entropy distribution to reduce redundant computation and concurrently improve overall performance. EAGer allows branching to multiple reasoning paths only in the presence of high-entropy tokens, and reallocates the saved compute budget to instances where exploration of alternative paths is most needed. We validate EAGer across multiple open-source models on complex reasoning benchmarks, with gains specifically demonstrated on AIME 2025. When target labels are accessible -- as in RLVR training pipelines -- EAGer achieves up to +37% in Pass@k and 59% fewer tokens; in test-time settings it still yields +12% in Pass@k and 64% fewer tokens compared to Full Parallel Sampling.

1. Introduction

EAGER targets redundant parallel reasoning by branching at high-entropy tokens and reallocating unused sampling budget toward harder prompts. Across tested models and domains, it reports performance gains over FULL PARALLEL while reducing token use.

  • Parallel paths can redundantly regenerate identical prefixes, while simple prompts often converge with little variation and complex prompts may need more generations.Source: §1 · p. 2
  • EAGER monitors token-level uncertainty and branches only at high-entropy tokens, then reallocates saved sampling budget toward more challenging prompts.Source: §1 · p. 2
  • Across math, science, and coding tasks, tested 3B–20B LMs show up to 37% performance boost over FULL PARALLEL sampling with EAGER.Source: §1 · p. 2
  • Token-wise entropy peaks serve as online uncertainty proxies for when more exploration is needed, reflecting prompt difficulty for the model.Source: §1 · p. 2
  • EAGER generates up to 65% fewer tokens and saves up to 80% of the generation budget across benchmarks and models.Source: §1 · p. 2
  • On AIME 2025, EAGER improves performance by up to 12% without target labels and up to 37% when labels are available.Source: §1 · p. 2

2. Preliminaries

The preliminaries motivate entropy-guided compute allocation by linking uncertainty during reasoning to potential need for additional path exploration. They define top-K token entropy and examine how peak entropy relates to reasoning accuracy.

2. Preliminaries

  • FULL PARALLEL sampling generates multiple candidate sequences to explore reasoning paths, while the stated objective is to allocate sequence-generation resources according to uncertainty.The approach is used for diverse solutions and RLVR pipelines, where correct generations are selected for policy updates.Source: §2 · p. 2

2.1. Uncertainty in LLMs’ Generations

  • Top-K token entropy approximates full-vocabulary entropy using the K most probable tokens, with minimal computational overhead.The paper uses K = 20 as a trade-off between efficiency and fidelity; the top 20 tokens typically capture most probability mass.Source: §2.1 · pp. 2–3, §2.2 · p. 3

2.2. Entropy in Long Chain-of-Thought Reasoning

  • The analysis treats peak entropy as a proxy for perceived problem difficulty: high peaks indicate uncertainty about the next reasoning step, while low entropy indicates greater confidence.The authors focus on the sequence’s highest entropy peaks and hypothesize that additional exploration during these phases may help performance.Source: §2.2 · pp. 3–4
  • For each sampled candidate sequence, the method records token entropy during generation and defines peak entropy as the mean of values in that sequence’s pth-percentile entropy distribution.Experiments use five standard reasoning benchmarks in math, science, and code generation, with M = 32 parallel sequences.Source: §2.2 · p. 3
  • r = −0.547 between Pass Rate and average entropy peak across sequences; sequences with higher entropy are less likely to answer correctly.Figure 2 reports Qwen3 4B with FULL PARALLEL sampling (M = 32), using pth = 99.9.Source: Figure 2 · p. 3

3. Entropy-Aware GEneRation Explained

EAGER uses token entropy to prune unnecessary parallel generations and redirect saved compute toward prompts needing more exploration. Its stages combine entropy-triggered branching with budget reallocation.

3.2. EAGER: Dynamically Allocate the Saved Compute

  • EAGER reallocates compute saved by EAGER-init, using model signals without labels or Pass@k failures when target labels are available.Without labels, EAGER-adapt gives extra budget to prompts that reached the branching cap; with labels, failing prompts receive uniform shares of the saved budget, capped at 2M.Source: §3 · p. 4, §3.2 · p. 5

3. Entropy-Aware GEneRation Explained

  • EAGER is a two-stage, training-free inference-time method: EAGER-init prunes unnecessary generations, then saved compute is reassigned to challenging prompts.For label-based reallocation, underutilizing prompts get a 20% lower entropy threshold, while saturating prompts can receive additional sequences up to M + b_i; the method remains below an equivalent fixed-budget FULL PARALLEL token baseline.Source: §3 · p. 4, §3.2 · p. 5

3.1. EAGER-init: Save Compute via Token Entropy

  • EAGER-init branches when token entropy exceeds threshold θ, creating continuations from the two most likely tokens until the per-prompt sequence limit M is reached.Branches reuse preceding tokens; easy prompts may finish with one sequence, saving M − 1 generations. Branching is restricted to long CoT sequences, and monitoring stops after 1000 tokens without a branch.Source: §3.1 · p. 4

4. Experimental Setting and Results

The evaluation compares EAGER with FULL PARALLEL sampling across reasoning models, benchmarks, and compute and performance metrics. Results report token savings alongside improvements in Pass Rate and Pass@k, with budget reallocation and scaling experiments examining when the gains occur.

4. Experimental Setting and Results

  • The evaluation spans models from 3B to 20B parameters and math, science, and code benchmarks; it measures compute with #Seq and #Token and quality with Pass@k, Cons@k, and Pass Rate.Source: §4 · pp. 5–6
  • The entropy threshold trades compute for branching frequency, and each model’s value is calibrated on 10 MATH-500 examples before being applied unchanged across benchmarks.Across 30 validation-set runs, 27 recovered the identical threshold.Source: §4 · p. 6

4.1. Results

  • Without target labels, EAGER-adapt redirects budget to prompts reaching M=32 sequences as a difficulty proxy, saving tokens on average while raising Pass@k over FULL PARALLEL.Table 2 reports EAGER-adapt’s reallocation on saturating prompts.Source: §4.1 · p. 6, Table 2 · p. 7
  • With target labels, EAGER redirects budget to saturating sequences and still achieves the second-best performance in most settings.Source: §4.1 · pp. 6–7
  • Across benchmarks and model sizes, EAGER and EAGER-init use half as many FULL PARALLEL tokens in most cases while improving Pass Rate; EAGER achieves the highest Pass@k.The comparison covers AIME-2025, GPQA-Diamond, and HumanEval Plus.Source: §4.1 · pp. 6–7, Table 1 · p. 6, Figure 3 · p. 7
  • EAGER-init’s equal-or-higher Pass Rate than FULL PARALLEL can come with lower Pass@k, reflecting reduced exploration; Pass@k matters when finding at least one correct answer is critical.Pass Rate and Cons@k capture a different quality dimension from Pass@k.Source: §4.1 · p. 7
  • As the allowed sequence count M increases, EAGER improves Pass@k while using fewer tokens for correct solutions, shifting the performance–efficiency Pareto frontier.Source: §4.1 · p. 7, Figure 4 · p. 8

5. Related Works

Prior work explores reward-guided branching and confidence-based generation controls, while EAGER uses token-level rather than sequence-level uncertainty. Related methods differ in whether uncertainty guides branching, truncation, or answer selection.

5. Related Works

  • REBASE expands trajectories rated highly by a reward model, while DeepConf truncates sequences below a confidence threshold set during warm-up; EAGER instead uses certainty to guide branching.The paper characterizes DeepConf’s warm-up and truncation as yielding greater theoretical token use than EAGER.Source: §5 · p. 8
  • ESC generates in small sequential windows and halts early when trajectories converge on the same final answer to reduce unnecessary continuation.The paper provides an empirical comparison with ESC and DeepConf in Appendix C.Source: §5 · p. 8
  • Unlike Kang et al.’s (2025) sequence-level self-certainty, EAGER uses token-level certainty to modulate reasoning generation’s performance and efficiency.Source: §5 · p. 8

6. Conclusion and Future Directions

EAGER variants improve efficiency and performance against FULL PARALLEL sampling across math, science, and coding. The paper identifies lightweight uncertainty quantification as important for future alternatives to token-level entropy.

6. Conclusion and Future Directions

  • EAGER-init exceeds FULL PARALLEL Pass@k by up to 37% with target labels while using up to 65% fewer tokens; EAGER-adapt outperforms it with up to 40% fewer tokens without a verifier.The approaches are reported as domain- and temperature-agnostic.Source: §6 · p. 8
  • EAGER’s reported scope includes math, science, and coding, with temperature-agnostic behavior.Source: §6 · p. 8
  • Future uncertainty measures, such as KL Divergence, must remain lightweight or their computational cost could undermine generation efficiency.Source: §6 · p. 8

Limitations

The full EAGer variant relies on target labels for fine-grained budget reallocation, while EAGer also incurs per-model threshold-selection costs. Evaluation scope and threshold generalizability remain bounded.

Limitations

  • Full EAGer relies on target labels to identify failing prompts for reallocation, restricting direct applicability to tasks with verifiable answers.Open-ended reasoning would require an external verifier or reward model for the same fine-grained reallocation; EAGer-init and EAGer-adapt avoid this constraint.Source: Limitations · pp. 8–9
  • EAGer requires selecting entropy threshold θ per model, adding upfront cost compared with FULL PARALLEL sampling.The protocol uses 10 validation examples, but whether threshold consistency holds for much larger models needs further testing.Source: Limitations · p. 9
  • Evaluation covers verifiable math, science, and code tasks using models from 3B to 20B parameters.Generalization to substantially larger models, agentic settings, and non-verifiable open-ended generation remains an open question.Source: Limitations · p. 9

Impact Statement

The Impact Statement frames EAGER as a way to reduce resource costs for inference-time scaling. The appendix notes that Table 3 provides a complete overview of experimental results.

Impact Statement

  • EAGER aims to reduce the energy and infrastructure costs of test-time scaling workloads, without introducing capabilities beyond the underlying models.The stated potential benefit is improved accessibility for researchers and practitioners with limited compute budgets.Source: Impact Statement · p. 9

A. Complete Results

B. Threshold Selection Robustness

The threshold-selection robustness experiment tests whether validation-source choice changes EAGer’s selected threshold. Across 30 Qwen3-4B runs, most selections matched the main-experiment threshold.

B. Threshold Selection Robustness

  • The robustness test addresses whether choosing a different validation benchmark would substantially alter the selected threshold and reduce practicality.Source: B. Threshold Selection Robustness · p. 11
  • The study compared MATH-500, GSM8K, and ZebraLogic validation sources, spanning in-domain math, easier grade-school math, and out-of-domain logic puzzles.Source: B. Threshold Selection Robustness · p. 11
  • For each source, five 10-example sets were tested on Qwen3-4B for EAGer-init and full EAGer, yielding 30 selection runs.Source: B. Threshold Selection Robustness · p. 11
  • In 27 of 30 runs (90%), threshold selection recovered the main-experiment value regardless of validation source.The two deviations selected θ = 2.1 instead of 2.0 for one GSM8K EAGer-init run and θ = 2.1 instead of 2.5 for one full EAGer MATH-500 run.Source: B. Threshold Selection Robustness · p. 11

C. Related Approaches

The AIME 2025 comparison finds EAGER-adapt the strongest overall fair-comparison approach across models, balancing performance and token use. ESC saves tokens but is slower, while EAGER’s branch-and-reuse strategy avoids redundant generation, especially on easy prompts.

C. Related Approaches

  • The AIME 2025 comparison includes ESC, which targets lower token use, and DeepConf, which stops generation when sequence confidence falls below a threshold.ESC uses its original default parameters; DeepConf uses its online variant with a per-prompt warm-up counted in Pass Rate and generated tokens, and both have maximum budget M = 32.Source: C. Related Approaches · p. 11
  • Under fair comparison, EAGER-adapt is strongest overall across models, balancing Pass@k and Pass Rate with generated-token efficiency.Full EAGER is excluded from direct comparison because it accesses target labels.Source: C. Related Approaches · p. 11
  • ESC has higher wall-clock times than EAGER, DeepConf, and FULL PARALLEL because sequential windows and repeated evaluations restrict parallelization.ESC uses small windows and may continue up to budget M; it reduces token use relative to FULL PARALLEL.Source: C. Related Approaches · p. 11
  • EAGER reuses generated tokens, avoids extra sequences when the model is confident, and needs no warm-up; on easy prompts it often uses only one or two branches.DeepConf-online can generate all 32 sequences on easy prompts despite identical, correct answers, producing redundant tokens.Source: C. Related Approaches · pp. 11–12

D. Effect of Temperature

On AIME 2025, temperature effects differ by model: SmolLM 3B generally performs better at high temperature, while DeepSeek 8B shows the opposite pattern. The generation configurations use model- and benchmark-specific entropy thresholds, with separate threshold goals for EAGER-init and EAGER.

D. Effect of Temperature

  • SmolLM 3B generally performs best at high temperature, whereas DeepSeek 8B generally performs best at low temperature; EAGER is competitive with or often surpasses the default baseline at both settings.Figure 5 compares Pass@k and Cons@k for varying entropy thresholds against default-sampling performance.Source: D. Effect of Temperature · p. 12, Figure 5 · p. 12
  • For SmolLM 3B, high temperature yields 59.1% fewer repeated-token generations on average, and EAGER and EAGER-init partially mitigate this issue.The pattern was observed only with SmolLM 3B; its HumanEval Plus results were all zero under FULL PARALLEL because it solved no tasks.Source: D. Effect of Temperature · pp. 12–13
  • At high temperature, DeepSeek 8B generates less than half as many tokens as at low temperature, while SmolLM 3B generates more; higher performance occurs at the setting with more tokens for each model.Source: D. Effect of Temperature · p. 13

E. Generation Parameters

  • Experiments use the models’ longest thinking configuration, a 32k-token context window, temperature 0.60, and top-p 95%; thresholds are selected per benchmark and model.EAGER-init favors a lower threshold to save budget without significantly compromising performance, while EAGER favors a higher threshold to preserve performance for reuse.Source: E. Generation Parameters · p. 13
  • Tables 3–7 cover model, benchmark, and threshold configurations; threshold-selection robustness; AIME 2025 comparisons; temperature and threshold results; and held-out calibration thresholds.Table 3 reports Pass@k, Cons@k, Pass Rate, and token counts for FULL PARALLEL, EAGER-init, and full EAGER; Table 4 repeats validation-set selection five times across three sources.Source: Table 3 · p. 14, Table 4 · p. 15, Table 5 · p. 15

Related papers

Loading 2510.11170v2…