PaperlayerView exact revision on arXiv

Source-linked AI summary

Drift No More? Context Equilibria in Multi-Turn LLM Interactions

Vardhan Dongre, Ryan A. Rossi, Viet Dac Lai, David Seunghyun Yoon, Dilek Hakkani-Tür, Trung Bui

arXiv:2510.07777v2cs.CLcs.AI

TL;DR

Context drift gradually separates multi-turn responses from goal-consistent behavior, and its temporal course is not captured by static evaluation alone. The paper measures divergence from a reference policy and models it as a bounded stochastic process; experiments find finite equilibria and reduced divergence with reminders.

  • Problem

    In multi-turn interactions, model responses can drift from user preferences and constraints, while common evaluations do not capture temporal misalignment across turns.

    Source: §1 · p. 1

  • Method

    The paper measures turn-wise divergence from a goal-consistent reference policy and models its evolution as a stochastic process whose equilibrium can be influenced by interventions.

    Source: §1 · p. 1, §3 · p. 2

  • Results

    Across synthetic rewriting and realistic multi-turn settings, drift stabilizes at finite, noise-limited equilibria, and goal reminders reduce divergence.

    Source: §1 · p. 1, §8 · pp. 5–6

  • Takeaways & Limitations

    Multi-turn drift can be studied as a controllable equilibrium phenomenon rather than inevitable decay.

    Source: §8 · p. 6

    The study covers a small set of models and domains, including synthetic rewriting tasks and two goal-oriented τ-Bench scenarios.

    Source: §9 · p. 6

Abstract

from arXiv · show

Large Language Models (LLMs) excel at single-turn tasks such as instruction following and summarization, yet real-world deployments require sustained multi-turn interactions where user goals and conversational context persist and evolve. A recurring challenge in this setting is context drift: the gradual divergence of a model's outputs from goal-consistent behavior across turns. Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics. In this work, we present a study of context drift in multi-turn interactions and propose a simple dynamical framework to interpret its behavior. We formalize drift as the turn-wise KL divergence between the token-level predictive distributions of the test model and a goal-consistent reference model, and propose a recurrence model that interprets its evolution as a bounded stochastic process with restoring forces and controllable interventions. We instantiate this framework in both synthetic long-horizon rewriting tasks and realistic user-agent simulations such as in $\tau$-Bench, measuring drift for several open-weight LLMs that are used as user simulators. Our experiments consistently reveal stable, noise-limited equilibria rather than runaway degradation, and demonstrate that simple reminder interventions reliably reduce divergence in line with theoretical predictions. Together, these results suggest that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay, providing a foundation for studying and mitigating context drift in extended interactions.

1 Introduction

The introduction frames context drift as a gradual loss of user intent across turns that static evaluations often miss. The paper measures drift, models its dynamics, and reports that reminders reduce divergence.

1 Introduction

  • Context drift is a gradual divergence from users’ preferences, instructions, or constraints, while end-task and per-turn evaluations miss temporal misalignment.The introduction contrasts this with single-turn errors and gives examples including tone loss, aesthetic drift, and user simulators forgetting goals.Source: §1 · p. 1
  • Across synthetic and realistic multi-turn settings, drift stabilizes at finite levels and can be shifted downward by goal reminders.The proposed stochastic recurrence interprets this behavior as stable equilibria under mild assumptions about memory decay and stochasticity.Source: §1 · p. 1
  • The study measures temporal divergence between test LLMs and a reference policy in synthetic rewriting tasks and τ-Bench user simulators.It presents this as one of the first systematic analyses of drift trajectories.Source: §1 · p. 1
  • Targeted reminders reduce equilibrium divergence and improve alignment quality across tasks and models, in line with the framework’s predictions.Source: §1 · p. 1

2 Related Works

Prior work describes context loss, investigates ways to diagnose drift, and develops dynamical-systems and memory-management approaches. This paper positions its contribution as modeling drift as a bounded process with an estimable, influenceable equilibrium.

2 Related Works

  • Context drift is degradation or distortion of conversational state, distinct from alignment drift, which is deviation from intended policies or values.Source: §2 · p. 2
  • Prior drift studies diagnose and quantify long-horizon change, while this work models it as a bounded stochastic process with stable equilibrium divergence.The paper frames its KL comparison against a goal-consistent reference policy as a way to analyze those dynamics.Source: §2 · p. 2
  • Dynamical-systems studies analyze prompt controllability and output stabilization; this work focuses on restoring forces and interventions that shape drift equilibria.The authors state that prior studies had not explicitly analyzed drift as a bounded stochastic process with stable fixed points.Source: §2 · p. 2
  • Memory-augmented models and context compression aim to preserve salient information, but often lack a principled account of long-horizon dynamics.This work complements them by treating drift as a process whose equilibrium can be estimated and influenced.Source: §2 · p. 2

3 Dynamics of Context in Multi-Turn Interactions

The paper models multi-turn drift through per-turn divergence between a test model and a reference policy. It treats divergence as a bounded stochastic trajectory and studies how interventions can shift its equilibrium lower.

3 Dynamics of Context in Multi-Turn Interactions

  • The setup compares a test language model and a reference policy exposed to the same evolving conversation history over multiple rounds.The reference serves as a stable, high-quality proxy for goal-consistent behavior.Source: §3 · p. 2
  • Contextual divergence is a measurable proxy for drift, and the paper represents its turn-wise evolution as a bounded dynamical process rather than inevitable unbounded accumulation.Perfect context alignment corresponds to D_t = 0 at every turn.Source: §3 · p. 2
  • The divergence metric compares full token-level predictive distributions, while GPT-4.1 is an alignment anchor rather than ground truth.The analysis is triangulated with semantic similarity and LLM-judge quality judgments; the recurrence includes controllable inputs and stochastic variability.Source: §3 · p. 2
  • The study examines interventions that shift equilibrium divergence toward lower levels, reframing the goal as influencing long-run context alignment.Source: §3 · p. 3

4 Modeling Drift Dynamics

The paper models turn-wise divergence as a bounded stochastic process with interventions and interprets its long-run behavior through equilibria. Under stated conditions, interventions can shift the equilibrium downward, though the model is not claimed to describe every LLM or setting.

4 Modeling Drift Dynamics

  • A recurrence model represents divergence as the prior turn’s divergence plus systematic bias, bounded stochastic perturbation, and a corrective-intervention term.The model is proposed to capture empirically observed bounded fluctuations rather than monotonic growth.Source: §4 · p. 3
  • Restoring dynamics may reduce large divergence, pulling trajectories toward a finite equilibrium.The paper gives reliance on salient context as an example of a stabilizing force.Source: §4 · p. 3

4.1 Equilibrium Interpretation

  • A contextual equilibrium is a fixed point representing the long-run divergence sustained under an interaction protocol; convergence follows if the bias is monotone and noise is bounded..Source: §4.1 · p. 3

4.2 A Simple Bound

  • Without interventions, divergence settles near a noise-limited equilibrium; when average intervention strength exceeds the noise bound, the equilibrium can shift downward.The paper presents this as an interpretive bound under mild assumptions, not a universal description of LLM behavior.Source: §4.2 · p. 3
  • The framework treats drift as a bounded, controllable equilibrium process and identifies estimating that equilibrium and designing minimal interventions as key challenges.Source: §4.2 · p. 3

5 Experimental Setup

The experiments pair a controlled synthetic rewriting task with realistic goal-oriented interactions, measuring divergence against a fixed reference policy. GPT-4.1 serves as that anchor, while an LLM judge rates alignment quality in multi-turn user simulations.

5 Experimental Setup

  • The evaluation combines a synthetic task with explicit constraints and intensifying conflicting instructions, and realistic τ-Bench retail and airline conversations with user goals and success criteria.The synthetic runs test three models over eight turns; both settings include freerunning and intervention-controlled conditions with turn-wise divergence logging.Source: §5 · p. 3

5.1 Reference policy definition.

  • GPT-4.1’s predictive distribution conditioned on the original goal and full interaction history is the reference; its self-divergence is reported as KL < 0.05 across T = 10.The reference is used as a stable anchor for relative drift, not as an absolute ground truth.Source: §5.1 · pp. 3–4, Figure 1 · p. 4

5.2 LLM-as-Judge

  • An LLM judge scores user-simulator alignment from 1 to 5 on profile consistency, task-goal alignment, and context appropriateness.The judge receives the original profile, task goal, and full conversation history.Source: §5.2 · p. 4

6 Results

Across the tested models, baseline divergence remains bounded, while goal reminders lower divergence and improve judged alignment. The reminder effect varies by model, and drift can resume after an intervention.

Baseline dynamics

  • Across LLaMA 3.1 8B, LLaMA 3.1 70B, and Qwen 2 7B Instruct, baseline divergence stabilizes around a noise-limited equilibrium rather than growing unboundedly.Semantic similarity and LLM-judge scores remain stable or mildly improve over turns.Source: §6 · p. 4

Effect of reminders as control interventions

  • Figure 3 shows reminders at turns 4 and 7 lowering divergence for most models; reported gains include up to 30% divergence reduction for LLaMA 3.1 8B and a late-turn judge score of 5.0 for Qwen 2 7B Instruct.The reminder setting also raises judge scores for LLaMA 3.1 8B by +0.5 points over baseline.Source: Figure 3 · p. 5, §6 · p. 5
  • Reminder effects are model-specific: divergence may resume in later turns, while reminders still yield measurable judge-score improvements for LLaMA 3.1 70B despite its low baseline divergence.Source: Figure 3 · p. 5, §6 · p. 5

Interpretation via equilibrium dynamics

  • The results support a bounded, controllable drift interpretation: interventions reliably lower equilibrium divergence but do not eliminate drift entirely.The paper describes the improvement as occurring at modest cost.Source: §6 · p. 5

7 Analysis of Equilibrium Dynamics

Reminder interventions shift estimated equilibrium divergence downward, while residual and correlation diagnostics support bounded noise and restoring dynamics. The analysis estimates the equilibrium by modeling turn-to-turn divergence change as a function of current divergence.

Estimating the equilibrium

  • A diagnostic regression models turn-to-turn divergence change as a function of current divergence; a negative slope indicates a restoring force.The estimated equilibrium is the fixed point where expected divergence change is zero.Source: §7 · p. 5

Effect of reminder interventions

  • Reminders consistently shift estimated equilibria downward; for LLaMA-3.1-8B, the estimate falls from 20.4 to 17.6.Observed KL divergence and LLM judge scores also improve by +0.2 to +0.6 points across models.Source: §7 · p. 5

Noise and residual diagnostics

  • Residuals are bounded and light-tailed, with no heavy-tail pathologies; divergence and its subsequent change are strongly negatively correlated (ρ < −0.7).These diagnostics support noise-limited equilibrium and restoring dynamics.Source: §7 · p. 5

8 Conclusion

Across synthetic rewriting and τ-Bench, drift stabilizes at finite, noise-limited equilibria, and goal reminders reduce divergence. The paper presents an explanatory framework for measuring and shaping equilibrium behavior, not a definitive solution to multi-turn drift.

8 Conclusion

  • Across synthetic rewriting and τ-Bench, drift stabilizes at finite, noise-limited equilibria, and goal reminders reduce divergence.The conclusion frames drift as an equilibrium process whose level can be shifted through interventions.Source: §8 · pp. 5–6
  • The conclusion’s τ-Bench evidence includes all-turn baseline metrics, reminder averages at turns 4 and 7, estimated equilibria, and KL-divergence and LLM-judge equilibrium shifts.The table captions identify the scope of these results but do not provide values in the supplied passages.Source: Table 1 · p. 6, Table 2 · p. 6, Table 3 · p. 6
  • The framework is offered as an explanatory lens for measuring and shaping long-run drift, rather than as a definitive solution to multi-turn drift.The authors describe the work as combining empirical evidence with a simple explanatory model.Source: §8 · p. 6

9 Limitations

The results are bounded by the reference policy, the limited model and domain sample, and the use of simple goal reminders.

9 Limitations

  • The study uses GPT-4.1 as an imperfect reference, covers a small set of models and domains, and tests only simple goal reminders.Different references could yield different divergence estimates; experiments cover synthetic rewriting and two goal-oriented τ-Bench scenarios.Source: §9 · p. 6

10 Future Work

Future work would extend equilibrium analysis to higher-stakes and more varied settings, test richer interventions, and improve evaluation of long-run reliability.

10 Future Work

  • Extend equilibrium-dynamics analysis to multimodal interactions and safety-critical settings.The authors identify these as domains where drift may have higher stakes.Source: §10 · p. 6
  • Explore interventions beyond goal reminders, including adaptive prompting, retrieval-augmented memory, and reinforcement-based alignment signals.The aim is to better understand how different mechanisms shape long-run equilibrium behavior.Source: §10 · p. 6
  • Develop standardized metrics and benchmarks for estimating equilibrium divergence and evaluating multi-turn reliability across models.Source: §10 · p. 6
  • Investigate how equilibrium dynamics relate to value drift and user-preference shifts.The authors suggest this could deepen insight into how interactive agents maintain trust and effectiveness over extended horizons.Source: §10 · p. 6

11 Appendix

The appendix sketches why bounded noise can yield finite drift equilibria, defines a linear diagnostic for their location, and tests the diagnostic’s reliability. It also describes synthetic and τ-Bench settings, including a turn-level example in which a reminder restores constraint compliance.

11.1 Proof Sketch of Bound

  • With monotone restoring dynamics and bounded noise, the proof sketch shows convergence to a finite equilibrium; positive interventions δt shift that equilibrium downward.Source: §11.1 · p. 8

11.2 Linear Drift Diagnostic

  • The empirical equilibrium Dhat* = -a/b estimates the fixed point where E[ΔDt] = 0.Source: §11.2 · p. 8

12 Statistical Reliability of Fitted Coefficients

  • Across settings, fitted b stayed negative within 95% confidence bounds; average R2 values ranged from 0.28–0.72, with OLS intervals bootstrapped over trajectories.Source: §12 · p. 8

13.1 Synthetic constrained multi-turn generation task

  • The synthetic task isolates drift in a controlled multi-turn setting with an unambiguous goal and gradually conflicting instructions against fixed constraints.Source: §13.1 · p. 8
  • τ-Bench supplies realistic goal-driven dialogues in retail and airline domains, with tool-using agents, user profiles, and task-success criteria.Source: §13.2 · p. 9
  • GPT-4.1 served as reference for open-weight user simulators; turn-wise KL/JS divergence, similarity, and judge alignment were measured, with reminders at t = 4 and t = 7.Source: §13.2 · p. 9
  • LLaMA-3.1-8B exceeded the 100–130-word constraint on Turns 2–3, then complied after a Turn-4 reminder, while GPT-4.1 remained compliant.Source: §13.1 · p. 9, Table 5 · p. 11

Related papers

Loading 2510.07777v2…