PaperlayerView exact revision on arXiv

Source-linked AI summary

ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn Language Models

Haziq Mohammad Khalid, Athikash Jeyaganthan, Timothy Do, Yicheng Fu, Sean O'Brien, Vasu Sharma, Kevin Zhu

arXiv:2510.14077v1cs.CL

TL;DR

LLMs can lose performance in multi-turn conversations, posing challenges for real-world deployment. ERGO monitors next-token entropy and reconstructs prompts when uncertainty spikes; in incrementally revealed-instruction simulations, it improved average performance by 56.6% over standard multi-turn baselines, while also increasing aptitude and reducing unreliability.

  • Problem

    LLMs often lose performance in multi-turn conversations compared with single-turn interactions, posing challenges for real-world deployment.

    Source: §1 · p. 1

  • Method

    ERGO monitors Shannon entropy over next-token distributions and triggers prompt reconstruction when uncertainty spikes.

    Source: §1 · p. 1

  • Results

    In incrementally revealed-instruction simulations, ERGO improved average performance by 56.6% over standard multi-turn baselines, increased aptitude by 24.7%, and reduced unreliability by 35.3%.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    ERGO restructures prompts through an external policy rather than modifying model internals or rewriting outputs, shifting correction upstream to input respecification.

    Source: §2.2 · p. 2, §2.3 · p. 3

    The stated limitations are framed as extensions toward broader deployment in richer, more adaptable dialogue settings.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large Language Models (LLMs) suffer significant performance degradation in multi-turn conversations when information is presented incrementally. Given that multi-turn conversations characterize everyday interactions with LLMs, this degradation poses a severe challenge to real world usability. We hypothesize that abrupt increases in model uncertainty signal misalignment in multi-turn LLM interactions, and we exploit this insight to dynamically realign conversational context. We introduce ERGO (Entropy-guided Resetting for Generation Optimization), which continuously quantifies internal uncertainty via Shannon entropy over next token distributions and triggers adaptive prompt consolidation when a sharp spike in entropy is detected. By treating uncertainty as a first class signal rather than a nuisance to eliminate, ERGO embraces variability in language and modeling, representing and responding to uncertainty. In multi-turn tasks with incrementally revealed instructions, ERGO yields a 56.6% average performance gain over standard baselines, increases aptitude (peak performance capability) by 24.7%, and decreases unreliability (variability in performance) by 35.3%, demonstrating that uncertainty aware interventions can improve both accuracy and reliability in conversational AI.

1 Introduction

ERGO targets multi-turn performance degradation by monitoring uncertainty and dynamically reconstructing prompts when entropy spikes. In incremental-instruction simulations, it improves performance, aptitude, and reliability metrics over standard multi-turn baselines.

1 Introduction

  • Multi-turn conversations can substantially degrade LLM performance, challenging real-world deployment.Reported manifestations include reduced accuracy, lower confidence, and a 112% increase in unreliability.Source: §1 · p. 1
  • Prior mitigation strategies based on task classification, retrieval, or context compression are described as lacking generality or requiring fine-tuning.Source: §1 · p. 1
  • ERGO monitors Shannon entropy over next-token distributions and dynamically reconstructs prompts when uncertainty spikes, preserving useful context while discarding accumulated noise.The authors hypothesize that uncertainty spikes signal conversational drift.Source: §1 · p. 1
  • ERGO improves average performance by 56.6% versus standard multi-turn baselines, raises aptitude by 24.7%, and reduces unreliability by 35.3%.Aptitude denotes best-case performance capability; unreliability denotes variability in response consistency.Source: §1 · pp. 1–2

2 Background and Related Works

The section situates ERGO among work on multi-turn degradation, uncertainty estimation, inference-time control, and backtracking. ERGO uses token-level entropy as a temporal signal to restructure user input rather than modify model internals or generated output.

2 Background and Related Works

  • Laban et al. (2025) report average multi-turn performance drops of 39% across six domains, while Gupta et al. (2024) formalize task-switch sensitivity; prior mitigations and agent frameworks have constraints.Laban et al. (2025) mitigated average losses by 15–20% but faced verbosity and practicality constraints; Wu et al. (2023) explore system-level solutions without targeting fundamental generation limitations.Source: §2 · p. 2

2.1 Entropy Based Uncertainty Estimation

  • ERGO uses next-token predictive entropy as a low-cost temporal signal for detecting context degradation and triggering prompt restructuring, unlike entropy methods used mainly for evaluation or filtering.Prior work also studies semantic-level measures and trainable proxies, which may rely on sampling or auxiliary models.Source: §2.1 · p. 2

2.2 Inference-Time Interventions

  • Unlike inference-time interventions that manipulate activations or logits, ERGO uses an external policy layer to monitor uncertainty and restructure user input without modifying model computation or sampling.Inference-time methods cited include activation-level interventions and activation engineering.Source: §2.2 · p. 2

2.3 Backtracking and Prompt Restructuring

  • Rather than backtracking over generated content or iteratively revising outputs, ERGO updates the user prompt using rising entropy as a trigger, shifting correction upstream to input respecification.The authors describe this as avoiding cumulative reasoning errors and as, to their knowledge, the first entropy-based mid-conversation user-input restructuring method.Source: §2.3 · p. 3

3 Entropy-Guided Context Resetting

ERGO tracks turn-level token entropy and its change across consecutive turns. When the increase exceeds a calibrated threshold, the system initiates a context reset.

3.1 Rise in Average Token Level Entropy

  • ERGO calculates average token-level entropy over generated tokens in each turn to quantify overall uncertainty; higher values indicate more diffuse token distributions.The change in this average is monitored across subsequent turns.Source: §3.1 · p. 3

3.2 Threshold-Based Trigger for Context Reset

  • A calibrated threshold τ marks an entropy increase beyond the acceptable margin, interpreted as a possible signal of compounding uncertainty or conversational drift.The threshold selection and sensitivity analyses are referenced in Appendices A and B.Source: §3.2 · p. 3

3.3 Context Reset Protocol

  • When the change in average entropy exceeds τ, ERGO initiates its automated context reset protocol.Source: §3.3 · p. 3

I. Prompt Rewriting:

The prompt-rewriting step consolidates prior user inputs into a single-turn prompt that preserves relevant task information while reducing ambiguity and redundancy. ERGO then generates from that rewritten prompt and continues on a new dialogue branch.

I. Prompt Rewriting:

  • The model rewrites the user’s inputs through turn t as one optimized single-turn prompt, retaining relevant task information while reducing ambiguity and redundancy.Source: §I · p. 3
  • The rewritten prompt is passed to a new model instance with no memory of prior turns, which generates a response to that prompt.Source: §I · p. 3
  • A new dialogue branch begins from the rewritten prompt and its response, continuing from the optimized state rather than the potentially degraded original context.Source: §I · p. 3

4 Experimentation Background

The experiments compare ERGO with single-turn and sharded multi-turn settings across five instruction-following tasks, using repeated runs to assess task success, aptitude, and unreliability.

4.1 Simulation Scale & Parameters

  • Five instruction-tuned models were evaluated without additional fine-tuning, with standardized generation settings and three runs per dataset; Data-to-Text used 50-question samples, while other datasets used 100.OpenAI models expose only top-20 log probabilities, limiting entropy-estimate precision, particularly on shorter Actions and Data-to-Text responses.Source: §4.1 · p. 4

4.2 Tasks

  • The evaluation covers Math, Code, Database, Actions, and Data-to-Text tasks using 220–325 constructed prompts per task and incrementally presented instruction shards.Tasks use exact-match scoring for GSM8K Math, execution-based validation for Code and Database, API-call validation for Actions, and BLEU for Data-to-Text.Source: §4.2 · pp. 4–5

4.3 Metric Selection

  • Task success scores range from 0 to 100 and are collected across repeated simulations for each instruction.The evaluation follows the prior simulation protocol, with ERGO as the changed implementation.Source: §4.3 · p. 5

4.4 Per-Run Scoring

  • Binary-correctness tasks score 100 when any turn is correct and end the run, while Data-to-Text uses the final output’s BLEU score rescaled to 0–100.An incorrect binary-task run receives 0.Source: §4.4 · p. 5

4.5 Aggregate Metrics

  • Aggregate metrics are average performance, 90th-percentile aptitude as peak capability, and the 90th-to-10th-percentile unreliability range as response variability.Aptitude and unreliability are averaged across tasks; lower unreliability indicates greater consistency.Source: §4.5 · p. 5

5 Results & Discussion

ERGO improves average performance, aptitude, and consistency over multi-turn baselines, while outperforming reset alternatives and existing intervention strategies. Its gains vary by task and model, and entropy changes were not systematically associated with response-length changes in the Phi-4 analysis.

5.1 Aptitude and Unreliability Improvements

  • ERGO increased average aptitude by 24.7% and reduced unreliability by 35.3% versus SHARDED, with aptitude sometimes exceeding single-turn performance.Figure 3 compares model performance under FULL and SHARDED settings with ERGO; the text reports recovery beyond single-turn aptitude and greater stability than multi-turn baselines.Source: §5.1 · pp. 5–6, Figure 3 · p. 5

5.2 Average Performance Gains

  • 56.6% average performance improvement over baseline multi-turn setups was reported across models, with nearly every model-dataset combination improving.Several combinations exceeded 100% gains; Data-to-Text improved less than other datasets, with model-specific prompt-rewriting and entropy-estimation constraints limiting benefits.Source: §5.2 · p. 6

5.3 Evaluating Entropy-Guided Resets vs. Random Resets and Fixed Resets

  • Entropy-guided resets outperformed random and quintet resets while adapting reset frequency to task uncertainty.In Database, ERGO gained 0.400 with 96 resets versus the quintet baseline’s 0.261 gain with 44 resets; in Actions, it gained 0.125 with 41 resets versus 0.045 and 0.085 for random and fixed approaches.Source: §5.3 · pp. 6–7, Figure 4 · p. 7

5.4 Comparison to Existing Intervention Strategies

  • ERGO outperformed SNOWBALL and RECAP across both model variants, nearly closing GPT-4o-mini’s SHARDED-to-FULL gap and bringing GPT-4o within 3.2 points of FULL.Unlike SNOWBALL, ERGO avoids prompt growth at each turn; unlike RECAP, it does not require knowing the final input turn.Source: §5.4 · pp. 7–8, Table 2 · p. 8

5.5 Evaluating Length Bias in Entropy-Based Reset Triggers

  • For Phi-4, entropy fluctuations were not systematically associated with response-length changes, supporting entropy as a signal distinct from verbosity.The reported Spearman result showed no meaningful monotonic relationship, while Pearson’s statistically significant coefficient was negligible and negative.Source: §5.5 · p. 8, Table 3 · p. 8

6 Conclusion

The conclusion presents ERGO as an entropy-triggered approach to prompt restructuring for mitigating multi-turn performance degradation. It reports gains across performance, aptitude, and unreliability, and identifies advanced context consolidation as future work.

6 Conclusion

  • ERGO uses Shannon entropy to detect model confusion and trigger prompt restructuring to mitigate multi-turn performance degradation.The conclusion describes Shannon entropy as a signal for targeted context consolidation that minimizes unnecessary resets.Source: §6 · p. 8
  • ERGO achieved 56.6% performance gains over standard baselines, improved aptitude by 24.7%, and reduced unreliability by 36.3%.The conclusion also reports that correlation analysis linked entropy-based resets to model uncertainty rather than response length.Source: §6 · p. 8
  • The conclusion describes ERGO as a practical, model-agnostic framework and identifies multi-stage summarization and adaptive long-form techniques as future work.Source: §6 · p. 8

Limitations

ERGO’s limitations concern which conversational context resets preserve and how well a fixed threshold generalizes; the authors describe these as steps toward broader dialogue settings.

Limitations

  • Resets consolidate user inputs but omit assistant responses, which may limit fidelity in open-ended dialogues and discard key entities or reasoning.The authors identify full dialogue-trace consolidation as a possible extension.Source: Limitations · p. 9
  • ERGO calibrates model-specific entropy thresholds on GSM8K and keeps them fixed across datasets.The authors suggest more dynamic or task-aware thresholds could improve precision.Source: Limitations · p. 9
  • The authors frame these limitations as progressions toward broader, more general-purpose deployment rather than challenges to ERGO’s core mechanism.Source: Limitations · p. 9

A Threshold Selection Procedure

ERGO selects a model-specific entropy-rise threshold using held-out GSM8K examples, then fixes that threshold across datasets to assess generalization without per-task tuning.

A Threshold Selection Procedure

  • For each model, the threshold is calibrated on approximately ∼80 held-out GSM8K shard-level examples outside the final evaluation set.The calibration computes changes in average token-level predictive entropy during standard multi-turn interactions.Source: A Threshold Selection Procedure · p. 11
  • The threshold is chosen from the entropy-rise distribution at a percentile aligned with the model’s baseline GSM8K aptitude.For example, GPT-4.1’s ∼90% single-turn aptitude corresponded to a 90th-percentile threshold.Source: A Threshold Selection Procedure · p. 11
  • Each model’s calibrated threshold is fixed across datasets to test generalization without dataset-specific adjustment.Table 4 summarizes model versions, thresholds, and calibration percentiles.Source: A Threshold Selection Procedure · p. 11, Table 4 · p. 11
  • Despite sharing a 90th-percentile threshold, Phi-4 triggered significantly more evaluation resets than GPT-4.1.The authors attribute the difference to Phi-4’s strong GSM8K performance but weaker performance on the broader task set.Source: A Threshold Selection Procedure · p. 11

B Sensitivity to Entropy Threshold (τ)

Across Database, Actions, and Math, τ = 0.03 produced the highest gains; lower and higher thresholds were less effective, supporting a balance between timely intervention and reset overhead.

B Sensitivity to Entropy Threshold (τ)

C Computational Cost and Reset Overhead Analysis

ERGO adds computational overhead through entropy monitoring and context resets, while reset frequency varies with model confusion and prompt length is lower in runs with resets. The section also describes how success scores are constructed and aggregated across repeated simulations.

C Computational Cost and Reset Overhead Analysis

  • Table 5 reports average ERGO performance, shards per reset, and the selected threshold percentile for each model, averaged across datasets.The passage estimates that one question corresponds to approximately six shards on average.Source: Table 5 · p. 13, C Computational Cost and Reset… · p. 13
  • ERGO adds the cost of predictive-entropy computation and two forward passes per reset, although token-level Shannon entropy adds negligible cost beyond decoding.The reset passes rewrite accumulated user context and respond to the consolidated prompt; costs scale with reset count.Source: C Computational Cost and Reset… · p. 13
  • 260 tokens was the average prompt length when resets occurred, compared with 309 tokens when no resets occurred.The reduction may partly offset reset costs by reducing input size on subsequent turns, but does not eliminate the reset cost.Source: C Computational Cost and Reset… · p. 13
  • Thresholds derived from one reasoning-heavy dataset generalize across diverse tasks, while reset frequency scales with model confusion.More capable models experience fewer high-entropy turns and require fewer resets than less capable models such as Phi-4.Source: C Computational Cost and Reset… · p. 13

D.1 Metric Selection

  • Each instruction is evaluated through repeated simulations, with each run’s final task-success score ranging from 0 to 100.The scores are collected across independent runs for a single instruction.Source: §D.1 · p. 14

I. Binary-correctness tasks (Code, Database,

  • For binary-correctness tasks, a run scores 100 if any turn produces a correct solution and 0 otherwise.For refinement tasks, the final output’s native metric is rescaled to [0, 100].Source: §I · p. 14

D.1.2 Aggregate metrics

  • The aggregate metrics are mean performance, 90th-percentile aptitude, and unreliability measured by the gap between the 90th and 10th percentiles.Aptitude and unreliability are computed per instruction and averaged across tasks; binary-correctness accuracy is mapped to a 0–100 scale.Source: §D.1.2 · p. 14

Related papers

Loading 2510.14077v1…