Source-linked AI summary
FLEET: From Logits Entropy to Enhanced Trajectories in Text Generation
Oleksii Streltsov, Oleksandra Vitko
TL;DR
Sampling can waste computation at critical branching points, while static temperature also perturbs otherwise stable steps. FLEET adds memory for targeted exploration of generation trajectories. Under ground-truth verification, it improves LiveCodeBench Pass@32 from 59.9% to 66.2%.
Problem
At critical branching points, standard sampling can overallocate probability to unviable candidates, and uniform temperature increases variance even at stable steps.
Source: §1 · p. 2Method
FLEET is a memory-augmented search framework that targets high-uncertainty states and uses VectorDSU to map hidden states into compact search-state components.
Source: §1 · p. 4, §3.1 · p. 7Results
Under ground-truth verification, FLEET outperforms standard stochastic temperature sampling across both evaluated domains and raises LiveCodeBench Pass@32 from 59.9% to 66.2%.
Source: §4.2 · p. 19, §5 · p. 19Takeaways & Limitations
FLEET provides a practical approach for augmenting large language models with structured completion-space search.
Source: §5 · p. 19The method may not synergize with reasoning models that rely heavily on reflective processes, where deterministic selection can be detrimental.
Source: §3.2 · p. 11
Abstract
from arXiv · showhide
Solutions based on large language models (LLMs) often rely on temperature sampling to improve accuracy and stability by aggregating multiple samples from the completion distribution. However, this memoryless approach is inherently suboptimal: because it lacks awareness of prior generations and their evaluations, it produces an increasing proportion of semantically duplicate answers as more samples are drawn, leading to diminishing returns. To address this limitation, we introduce FLEET, a novel method that integrates a memory mechanism into the generation process. FLEET represents each generation as a sparse trajectory through states whose entropy exceeds a predefined threshold and uses these trajectories to infer per-token utility scores that adjust the logits. Benchmark evaluations demonstrate that FLEET achieves the same accuracy as the repeated sampling baseline, with a 3x speedup, and substantially improves accuracy on complex coding tasks (LiveCodeBench Pass@32 increases from 59.9% to 66.2%) under the same budget. Furthermore, in the greedy-decoding configuration evaluated here, the approach is deterministic and uses a single calibration pass to derive its principal hyperparameters, requiring only minimal modifications to existing LLM pipelines.
1 Introduction
The introduction argues that conventional sampling can waste computation at critical decision points, while uniform exploration and brittle tuning leave key limitations unresolved. FLEET adds memory to target exploration, with evaluations reporting better scaling than repeated sampling under the same budget.
a) Sample Inefficiency
- At branching points, sampling can overallocate probability to unviable candidates, while raising temperature also destabilizes otherwise stable steps.The passage says the joint probability of a globally correct trajectory decays exponentially as branching points become more frequent.Source: §1 · p. 2
b) Lack of Selective Exploration
- Ideally, stochastic sampling would focus on high-entropy decision points and semantically meaningful actions rather than low-entropy syntactic steps.Top-k, top-p, and min-p are listed as heuristic truncation strategies for excluding some unviable tokens.Source: §1 · p. 3
c) Low Robustness and Poor Cross-Task Generalization
- Temperature and truncation parameters lack a unified theoretical basis, so empirical tuning is sensitive to prompts, architectures, and task complexity.The passage says the resulting configurations do not generalize well across downstream domains.Source: §1 · p. 3
Proposed approach
- The authors attribute sample inefficiency to sampling methods’ inability to use completion rewards to distinguish when to explore from when to exploit.They propose a lightweight memory mechanism and evaluate it on two benchmarks with verifiable tasks.Source: §1 · p. 3
Contributions
- VectorDSU maps hidden states into unified search states to prevent trajectory duplication and preserve state-utility history.FLEET is presented as a memory-augmented, deterministic search paradigm for targeted exploration of the completion space.Source: §1 · pp. 3–4
Empirical evaluation
- FLEET accuracy is reported to scale better than repeated sampling on mathematical and coding problems under the same budget.The evaluation compares both approaches using ground-truth verifiers and reward models.Source: §1 · p. 4
2 Related Work
Related work describes adaptive sampling as a way to modulate decoding parameters by state, but notes that sample inefficiency at critical decisions remains. It also questions whether scalar temperature adjustment can selectively amplify valid tokens without increasing variance across the distribution.
Adaptive sampling
- Adaptive sampling methods detect salient or high-entropy states and adjust decoding parameters such as temperature or truncation thresholds accordingly.The cited examples include Zhu et al. (2023) and Chang et al. (2025).Source: §2 · p. 4
Learnable parameter adjustment
- Learnable modules have been proposed to govern dynamic parameter adjustments, coupling tuning to the target-task objective rather than manual task-agnostic searches.The passage attributes this approach to Dang et al. (2026).Source: §2 · p. 4
Limits of adaptive sampling
- State-dependent modulation addresses selective exploration, but sample inefficiency at critical decision nodes persists.The passage identifies that remaining vulnerability as limitation (a).Source: §2 · p. 4
Temperature adjustment
- Scalar temperature adjustments cannot selectively amplify valid tokens without also increasing variance across the entire distribution.The passage also says the generality of adaptive-temperature results across broader domains remains unclear.Source: §2 · p. 5
3 FLEET Algorithm
FLEET steers generation with uncertainty signals and memory of prior trajectories, using search-guided logit penalties and reusable search nodes. Its calibration selects a layer based on logit-lens match and entropy-varentropy clusterability, then derives uncertainty thresholds and search parameters from empirical distributions.
3 FLEET Algorithm
- FLEET detects high-uncertainty states as branching points and uses historical trajectory evaluations to penalize actions on suboptimal paths.The framework uses memory to guide candidate paths rather than relying only on global scalar hyperparameter adjustments.Source: §3 · p. 5
- Intermediate-layer logits from the Logit Lens provide sharper uncertainty signals than final-layer logits, which may be smoothed.Conditional entropy measures distributional uncertainty but can overflag synonym clusters or deterministic reasoning; varentropy spikes when probability mass splits across distinct candidate clusters.Source: §3 · p. 6
3.1 Online Hidden State to Search State Mapping via Vector Disjoint Set Union
- VectorDSU maps hidden states online to compact representative-based clusters, retaining cluster metadata without storing every historical vector.Cosine similarity guides assignment, and access to internal activations and output distributions supports subsequent search.Source: §3.1 · pp. 7–8
- FLEET uses pUCT to balance observed rewards with language-model priors, restricts candidate actions to top-k tokens, and soft-penalizes evaluated tokens that are suboptimal.For autoregressive generation, action utility averages rewards across known outcomes; unexplored top-k tokens are grouped into an exploration action.Source: §3.1 · p. 8
3.2 FLEET Search
- A running buffer tracks the joint entropy–varentropy trigger rate, relaxing thresholds when it falls below a cube-root-scheduled target that rises across iterations.This helps search continue expanding when initial thresholds become too selective; coordinated workers can reserve distinct high-scoring branches, though parallel speedup was not quantified.Source: §3.2 · p. 9
- VectorDSU records attempted token actions, their empirical rewards, and comparisons with alternatives, yielding trajectory data richer than a sequence-level correctness label.The authors suggest it may support downstream SFT or RL, but do not evaluate that training value.Source: §3.2 · p. 10
- Current limitations include patching the model architecture, poor fit for reflective reasoning states under deterministic selection, and occasional unintelligible output noise.The authors suggest a secondary lightweight language model might mitigate syntactic or formatting corruption, but this was not evaluated.Source: §3.2 · p. 11
3.3 Hyperparameter Initialization
- FLEET settings were derived from a single calibration pass, while the temperature-sampling baseline used repeated generations during Bayesian temperature optimization.The authors note that FLEET has no universal default settings, but its parameters map to interpretable mechanisms in the search architecture.Source: §3.3 · p. 11
- The final layers are suboptimal because their higher logit-lens match ratio coincides with lower topological clusterability and degraded entropy-varentropy separability.Calibration selects a target layer, then samples continuous hidden states and corresponding output logits across multiple prompts.Source: §3.3 · pp. 12–13
- GMM clustering reveals cascading joint entropy-varentropy clusters, so thresholds are derived from marginal points on the joint decision frontier rather than a quadrant partition.The thresholds partition states across the two-dimensional entropy-varentropy space.Source: §3.3 · p. 13
- At T ≈2.9, the exploratory candidate tail’s cumulative mass strictly surpasses top-1 mass, with the temperature plot parameterized at k = 32.The resampling temperature is annealed to the minimum value meeting this condition; normalized entropy is also plotted as a measure of proximity to a uniform noise prior.Source: §3.3 · p. 13
- τdsu is the minimum similarity beyond which normalized divergence stays below the 0.05 error tolerance, using the transition-divergence upper bound of mean plus two standard deviations.The upper confidence bound is evaluated as state proximity increases.Source: §3.3 · p. 13
4 Experiments
Experiments compare FLEET with temperature sampling on GSM8K and LiveCodeBench under ground-truth verification and ORM guidance. Ground-truth-guided FLEET performs better on both benchmarks, while ORM selection reveals task-dependent ranking limits.
4 Experiments
- Evaluation uses LiveCodeBench and GSM8K, with ground-truth rewards or continuous ORM scores guiding search and 32 trajectories generated per problem.LiveCodeBench ground-truth rewards reflect the fraction of passed tests, whereas GSM8K rewards are binary; the LiveCodeBench evaluation includes 222 tasks.Source: §4 · p. 14
4.1.1 Ground-Truth Verification
- Under ground-truth verification, FLEET gains 6.31 percentage points and solves 14 more LiveCodeBench tasks; on GSM8K, it solves 7 more tasks than temperature sampling.The Pass@32 comparison covers both evaluated domains; GSM8K’s baseline accuracy is already 97.27%.Source: §4.1.1 · p. 16, Table 4.2 · p. 16
- Under ground-truth guidance, FLEET avoids redundant GSM8K paths through VectorDSU and scales more steeply and robustly than temperature sampling on LiveCodeBench.Figure 4.1 includes ground-truth evaluation of candidates generated with ORM feedback.Source: §4.1.1 · pp. 16–17, Figure 4.1 · p. 17
4.1.2 Outcome Reward Model (ORM) Guidance
- Under ORM selection, LiveCodeBench solution rate rises from 19.36% (43 problems) to 25.22% (56 problems), while GSM8K temperature sampling scores 0.8544 versus FLEET’s 0.8362.FLEET’s ground-truth-evaluated candidate pool reaches 0.6937 on LiveCodeBench and 0.9780 (1290 solved problems) on GSM8K.Source: §4.1.2 · p. 17, Table 4.3 · p. 18
- With ORM guidance, GSM8K scaling plateaus and diverges from ground-truth generation capability, while FLEET’s advantage over sampling widens with candidate budget on LiveCodeBench.The reported interpretation links the GSM8K gap to proxy-ranking errors on structurally diverse trajectories.Source: §4.1.2 · pp. 18–19
4.2 Discussion and Synthesis of Experimental Findings
- Under ORM selection, temperature sampling slightly leads on GSM8K, but FLEET leads by 13 LiveCodeBench tasks; dense ORM scores also help its search routing, adding 7 tasks in a hybrid configuration.The hybrid uses ORM score memory during search and ground truth for final evaluation, despite suboptimal ORM terminal ranking.Source: §4.2 · p. 19
5 Conclusion
FLEET adds deterministic, state-aware trajectory exploration to repeated generation in its greedy-decoding configuration. Across mathematical reasoning and code generation, ground-truth-guided FLEET improves LiveCodeBench Pass@32 from 59.9% to 66.2%.
5 Conclusion
- FLEET augments repeated generation with deterministic, state-aware trajectory exploration in its greedy-decoding configuration.Its structural memory, VectorDSU, organizes intermediate paths while entropy–varentropy triggers dynamically adjust exploration.Source: §5 · p. 19
- LiveCodeBench Pass@32 rises from 59.9% to 66.2% with ground-truth-guided FLEET; ORM-guided candidate generation reaches 69.4% under ground-truth evaluation.The paper reports that the ORM does not reliably recover this candidate-pool accuracy during final selection.Source: §5 · p. 19
- The evaluation covers mathematical reasoning and code-generation benchmarks, and the paper presents FLEET as a practical approach to structured completion-space search.The paper reports competitive performance under high-baseline saturation on GSM8K.Source: §5 · p. 19
A Prompts and Generation Configuration
The appendix gives the coding and math prompts, describes GSM8K task and few-shot prompt construction, and specifies decoding limits and settings.
A.1 LiveCodeBench system prompt
- The LiveCodeBench prompt requests a correct Python program that passes the tests and requires one standalone top-level function, without a class or multiple functions.The problem specification is supplied in the user message.Source: §A.1 · p. 24
A.2 GSM8K system prompt
- The GSM8K prompt asks for transparent substeps and requires a numeric final answer on the last line, preceded by ####.Source: §A.2 · p. 24
A.3 GSM8K task and few-shot prompt construction
- For each GSM8K question, the routine independently selects two training examples offline and fixes the resulting prompt for that evaluation run.It loads test tasks and few-shot examples from the official GSM8K dataset.Source: §A.3 · p. 24
- Few-shot answers are cleaned by removing text enclosed in double angle brackets, then formatted as question-answer examples separated by blank lines.Source: §A.3 · pp. 24–25
- GSM8K task records receive task IDs and prompts formed from the few-shot prompt, question, and answer prefix.The task IDs use the gsm8k_ prefix.Source: §A.3 · p. 25
A.4 Decoding configuration
- Both decoding configurations allow at most 1,024 generated tokens; temperature sampling is stochastic, while FLEET uses greedy decoding on raw logits.Trajectories that reach the limit are evaluated unfinished.Source: §A.4 · p. 25