PaperlayerView exact revision on arXiv

Source-linked AI summary

LLMs Get Lost In Multi-Turn Conversation

Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, Jennifer Neville

arXiv:2505.06120v1cs.CLcs.HC

TL;DR

LLM evaluation largely uses fully specified, single-turn instructions despite frequent underspecification in real conversations. Through large-scale single- and multi-turn simulations, the paper finds significantly degraded multi-turn performance, driven mainly by increased unreliability when models make early wrong turns.

  • Problem

    LLMs are commonly evaluated with fully specified, single-turn instructions even though underspecified requests and multi-turn clarification are prevalent.

  • Method

    The paper conducts large-scale simulations comparing LLM performance across single- and multi-turn conversations and analyzes aptitude, unreliability, and failure mechanisms.

  • Results

    Multi-turn underspecified conversations averaged 65% performance versus 90% when the full instruction was provided initially, with degradation primarily reflecting increased unreliability.

  • Takeaways & Limitations

    When LLMs make incorrect assumptions or propose premature solutions, they often rely on those earlier responses and fail to recover across turns.

  • Takeaways & Limitations

    The fully automated simulations are not representative of natural human-AI conversations and impose a relatively narrow conversational structure.

Abstract

from arXiv · show

Large Language Models (LLMs) are conversational interfaces. As such, LLMs have the potential to assist their users not only when they can fully specify the task at hand, but also to help them define, explore, and refine what they need through multi-turn conversational exchange. Although analysis of LLM conversation logs has confirmed that underspecification occurs frequently in user instructions, LLM evaluation has predominantly focused on the single-turn, fully-specified instruction setting. In this work, we perform large-scale simulation experiments to compare LLM performance in single- and multi-turn settings. Our experiments confirm that all the top open- and closed-weight LLMs we test exhibit significantly lower performance in multi-turn conversations than single-turn, with an average drop of 39% across six generation tasks. Analysis of 200,000+ simulated conversations decomposes the performance degradation into two components: a minor loss in aptitude and a significant increase in unreliability. We find that LLMs often make assumptions in early turns and prematurely attempt to generate final solutions, on which they overly rely. In simpler terms, we discover that *when LLMs take a wrong turn in a conversation, they get lost and do not recover*.

1 Introduction

The paper addresses the gap between prevalent underspecified user instructions and evaluations centered on single-turn, fully specified tasks. Large-scale simulations find substantial multi-turn degradation, driven especially by unreliability and models’ tendency to commit prematurely to assumptions and answers.

  • LLM evaluations typically use single-turn, fully specified instructions despite underspecification being prevalent in real conversation logs.
  • The sharded simulation environment transforms existing single-turn instructions into smaller instructions revealed gradually across conversation turns.
  • 65% average multi-turn performance represents a 25-point drop from 90% single-turn performance across the tested tasks.
  • The performance degradation combines aptitude loss with increased unreliability, which affects all tested LLMs in multi-turn settings regardless of aptitude.
  • Models generate verbose responses, propose final solutions prematurely, make incorrect assumptions, and over-rely on previous incorrect attempts.
  • The findings indicate that multi-turn performance degradation may particularly affect novice users who provide less complete instructions initially.

2 Background and Related Work

Prior multi-turn evaluations often treat conversations as isolated episodic subtasks, while this paper focuses on generative tasks requiring models to integrate incrementally disclosed information. It uses scalable LLM-based user simulation to study that setting.

  • Earlier conversational systems were evaluated mainly through single-turn tasks, human protocols, or competitions because prior models were not equipped for multi-turn conversation.
  • Post-ChatGPT evaluations expanded multi-turn benchmarks through crowd-sourced judging, longer conversations, finer granularity, naturalness, and tool-use assessments.
  • Episodic evaluations isolate related subtasks, but underspecified conversations require fusing information and adapting when new information arrives.
  • The paper argues that episodic tasks overestimate multi-turn performance because they do not require answering underspecified instructions through active information integration.
  • Long-form multi-turn evaluation is needed for generative programming and natural-language use cases where responses must adapt and refine as users provide information.
  • The study adopts an LLM-based user simulator to improve scalability and reproducibility compared with involving real users.

3 Simulating Underspecified, Multi-Turn Conversation

The paper builds a sharded-conversation simulation environment by dividing fully specified instructions into incrementally disclosed shards and using an LLM simulator to manage the dialogue. It compares several single- and multi-turn simulation types across programming and natural-language tasks.

  • The method repurposes existing single-turn benchmark tasks by sharding fully specified instructions and running conversations based on the resulting shards.
  • 3.1 Sharding Process: A sharded instruction preserves the original information while separating its high-level intent from subsequent clarifications.
  • 3.1 Sharding Process: Figure 2 pairs each fully specified instruction with an equivalent sharded instruction for single-turn and multi-turn simulation.
  • 3.2 Simulating Sharded Conversations: The simulation includes an LLM assistant, an LLM user simulator that reveals shards, and a system that classifies and evaluates assistant responses.
  • 3.2 Simulating Sharded Conversations: The user simulator selects and naturally rephrases shards using the conversation state, while the assistant is not told that the interaction is underspecified or multi-turn.
  • 3.2 Simulating Sharded Conversations: Simulation errors from the user simulator, strategy classifier, and answer extractor occurred in fewer than 5% of inspected conversations and disadvantaged assistants in fewer than 2%.
  • 3.3 Simulation Types: FULL provides the original instruction in one turn, while SHARDED reveals information across turns as the primary underspecified-conversation evaluation.
  • 3.3 Simulation Types: CONCAT preserves sharding rephrasing while presenting all shards together as a verification baseline for separating underspecification effects from information loss.

4 Task and Metric Selection

The paper evaluates six diverse generation tasks using instructions transformed from fully specified single-turn benchmarks into multi-turn, underspecified conversations. It measures mean performance, best-case aptitude, and stochastic unreliability.

  • The sharding process converts each fully specified instruction into smaller instructions that jointly preserve the original information.
  • Six tasks span programming, database querying, API actions, elementary math, data-to-text generation, and long-context summarization.
  • The first four tasks use binary correctness, while Data-to-Text and Summary use continuous scores from 0 to 100.
  • Average performance P estimates mean score, aptitude A90 estimates the 90th-percentile best-case score, and unreliability U90 measures the gap between best- and worst-case simulations.
  • Metrics are computed per instruction and can then be averaged across an instruction corpus; reliability is defined as R90/10 = 100 − U90/10.
  • In box-plot terms, aptitude corresponds to the upper whisker’s height, while unreliability corresponds to the distance between upper and lower whiskers.

5 Simulation Scale and Parameters

The simulation study evaluates 15 LLMs across a large instruction set and repeated conversation simulations. It includes models spanning families, sizes, weight-access regimes, and reasoning capabilities.

  • 15 LLMs were evaluated across six tasks using 10 simulations for each model, instruction, and simulation type.
  • The simulations totaled more than 200,000 conversations and used a default generation temperature of T = 1.
  • Repeated simulations improve averaged-performance estimates and enable detailed analysis of aptitude and reliability.
  • The model set includes 15 models from eight model families, covering small 8B models through models exceeding 300B parameters.
  • The study includes both open- and closed-weight models and two reasoning models, o3 and Deepseek-R1, to examine additional test-time compute.

6 Results

Across six tasks, multi-turn underspecified conversations substantially reduce performance for every tested model, while the main degradation reflects increased unreliability rather than lost aptitude. The effect appears across domains and begins with conversations split into two shards.

  • 6.1 Average Performance Findings: −39% average performance degradation occurs for every model on every task when comparing FULL and SHARDED settings.Models achieving 90%+ in fully specified single-turn evaluation struggle on the same tasks when instructions are underspecified and multi-turn.
  • 6.1 Average Performance Findings: CONCAT performance averages 95.1% of FULL performance, indicating that SHARDED losses are not explained by information removed during sharding.Smaller models show more pronounced CONCAT degradations of 86–92.
  • 6.1 Average Performance Findings: More capable models such as Claude 3.7 Sonnet, Gemini 2.5, and GPT-4.1 show 30–40% average degradation, comparable to smaller models.The paper attributes part of this comparability to metric definitions and the smaller models’ lower FULL scores.
  • 6.1 Average Performance Findings: Multi-turn capability varies by domain: Command-A degrades least on Actions, Claude 3.7 Sonnet and GPT-4.1 conserve Code performance, and Gemini 2.5 Pro conserves Data-to-Text performance.
  • 6.1 Average Performance Findings: Reasoning models deteriorate similarly to non-reasoning models, while their responses average 33% longer and tend to contain more assumptions.The analysis links longer responses with assumptions that can confuse user requirements and prior model responses.
  • 6.2 Aptitude vs. Reliability Analysis: Aptitude drops 16% on average between FULL and SHARDED settings, whereas unreliability increases 112% and performance varies by 50 percentage points between best and worst runs.The authors conclude that degradation is driven largely by unreliability rather than aptitude loss.
  • 6.2 Aptitude vs. Reliability Analysis: The identified causes include premature full-answer attempts, incorrect assumptions, reliance on prior answers, loss of middle-turn information, and verbose responses.
  • 6.3 Gradual Sharding Experiment: With GPT-4o and GPT-4o-mini, lost-in-conversation behavior appears from two shards onward, while one shard is the only setting that improves reliability.

7 Implications

The paper argues that agent-style repetition can partly mitigate multi-turn degradation, but current systems remain less reliable than in fully specified interactions. It therefore calls for LLMs to improve native multi-turn reliability, while noting task and language coverage limits.

  • 7.1 Implications for System and Agent Builders: 15-20%: SNOWBALL mitigates the FULL-to-SHARDED performance deterioration through realistic user-turn repetition.RECAP performs better but is unrealistic because it intervenes on the final turn, which is unknown during real conversations.
  • 7.1 Implications for System and Agent Builders: Agent-like information processing remains inferior to FULL or CONCAT, suggesting that offloading multi-turn handling can be limiting.Both RECAP and SNOWBALL improve over SHARDED simulations but do not match the stronger settings.
  • 7.2 Implications for LLM Builders: 50-80%: Lowering assistant temperature improves reliability in FULL and CONCAT settings, but SHARDED reliability improves little or not at all.In SHARDED simulations, GPT-4o-mini shows no improvement as assistant temperature decreases, while GPT-4o improves only about 15-20%; unreliability remains around 30% even when both temperatures are 0.0.
  • 7.2 Implications for LLM Builders: The paper calls on builders to jointly optimize aptitude and reliability rather than focusing only on intellectual task performance.Its proposed reliability target includes similar aptitude across single- and multi-turn settings, low multi-turn unreliability, and robustness at unmodified temperature T = 1.0.
  • 7.3 Implications for NLP Practitioners: The evaluated task set is limited to six mainly analytical tasks with English-language instructions, and preparing 100 sharded instructions required about three hours of author work.The authors manually validated and finalized samples after automating parts of the sharding process.
  • 7.3 Implications for NLP Practitioners: Translation showed no SHARDED degradation for the two tested models, with BLEU scores within 10% across settings.The authors attribute this result to translation being largely achievable at the sentence level and note that BLEU may not adequately capture document-level quality.
  • 7.4 Implications for Users of Conversational Systems: Starting a new conversation or consolidating requirements into one instruction can improve outcomes when an ongoing conversation has gone astray.The paper presents consolidation before retrying as an effective strategy for improving aptitude and reliability, based on CONCAT experiments.

8 Conclusion

The paper finds that LLM performance on fixed tasks degrades significantly in multi-turn, underspecified conversations. Models become less reliable as they struggle to maintain context, make premature assumptions, and over-rely on earlier responses.

  • 8 Conclusion: Large-scale simulations show significant performance degradation in multi-turn, underspecified conversations on a fixed set of tasks.The paper characterizes this behavior as LLMs getting lost in conversation.
  • 8 Conclusion: Models struggle to maintain context, make premature assumptions, and over-rely on previous responses, producing a significant decrease in reliability.These mechanisms are presented as the materialization of the lost-in-conversation phenomenon.

9 Limitations

The paper identifies three scope limitations: fully automated simulation, analytical tasks, and English text-only tasks. These constraints limit how broadly the findings can be generalized to natural, creative, multilingual, and multimodal conversations.

  • Simulation scope: Fully automated simulations are scalable and repeatable but do not represent the full range of natural human-AI conversation dynamics.The simulation reveals one new shard per turn, ends with all task information specified, and keeps the high-level intent in the first turn.
  • Task scope: The experiments cover analytical programming and natural-language tasks, leaving open whether models also get lost during creative writing.The authors state that creative-task degradation and its magnitude remain to be established.
  • Language and modality scope: The findings are restricted to English, text-only tasks and therefore do not establish whether the degradation extends to other languages or modalities.The authors identify multilingual and multimodal evaluation as needed to determine the scope of the observed effect.

Appendices

The appendices detail prior work on underspecification, define valid sharded instructions, and document the semi-automatic procedure used to construct them. Sharding preserves the original task intent while distributing its information across conversational turns.

  • Related work: Prior evaluation often treats multi-turn conversations as separable subtasks, whereas this work targets human-like underspecified interaction.Related work also identifies underspecification as common in human language and studies model responses to ambiguous or underspecified inputs.
  • Sharded instructions: A sharded instruction is a set of shorter instructions that jointly preserves the original query’s intended output.The sharding process identifies atomic content units and constructs shorter shards usable for multi-turn simulation.
  • Sharded instructions: Validity requires information preservation and a first shard that states the conversation’s high-level intent.The formal properties specify that no necessary information is lost and that the initial shard defines the overall objective.
  • Sharding process: The authors transform fully specified instructions through automated segmentation, rephrasing, and verification, followed by manual inspection and editing.The manual step is used because LLM-generated outputs do not guarantee the quality required for precise, large-scale experiments.
  • Sharding process: Segmentation extracts non-overlapping atomic content units, while rephrasing decontextualizes them and reorders shards to satisfy the validity properties.Rephrasing resolves dependencies between segments and can move the overall-intent segment into the first shard.
  • Sharding process: Verification compares original and sharded simulations, including shuffled concatenation, and rejects instructions whose degradation exceeds the acceptable threshold.The shuffled variant tests order insensitivity, while degradation below 80% indicates possible information loss or inadequate decontextualization.
  • Sharding process: Manual inspection typically takes 1–3 minutes per instruction, about an order of magnitude less than writing shards from scratch.The authors release the prompts used in the automated sharding stages to support construction for additional tasks.

Appendix D Inspection of Simulated Sharded Conversation

The inspection evaluates whether simulated user and assistant turns faithfully implement sharded conversations across four tasks. It finds the environment largely reliable, with most inspected conversations judged successful despite component-level errors.

  • Inspection design: The inspection covers 100 simulated sharded conversations across Actions, Code, Math, and Database.Summary and Data-to-text are excluded from the main inspection because they require answer attempts at every turn and do not use an LLM-based user simulator.
  • Inspection design: Inspectors annotate whether user turns reveal exactly one shard and whether each shard is appropriately contextualized.They flag turns revealing multiple or partial shards and assess contextual fit with the preceding conversation.
  • Inspection design: Assistant turns are checked for strategy-classification accuracy and successful extraction of answer attempts.For clarification labels, inspectors verify that the response actually asks a clarification question.
  • Results: Roughly 98% of inspected conversations are labeled successful overall, indicating that the simulation environment is highly reliable.Overall success is assigned when simulation errors do not affect the validity of the conversation.
  • Results: User simulation fully reveals one shard around 96% of the time and contextualizes it 98% of the time.These are component-level inspection results for the simulated user turns.
  • Results: Assistant strategy classification is 95% accurate, while answer-attempt extraction reaches 97% accuracy.These component errors do not necessarily invalidate the overall simulation because later turns can correct earlier errors.
  • Results: Some early-turn simulation errors are corrected later, so utterance-level mistakes do not always compromise conversation-level validity.The inspection observed both later user correction and successful extraction on a subsequent answer attempt.

Appendix E Concrete Example of Loss in Aptitude vs. Reliability

The concrete examples show that identical average performance drops can arise from reduced aptitude, increased unreliability, or both. In the paper’s practical analyses, performance degradation is more strongly associated with reliability losses than aptitude losses.

  • Baseline: P = 90% in FULL simulations, where perfect performance occurs on nine of ten instructions and all runs fail on the tenth.This yields A = 90% and U = 0% because each instruction’s 10th- and 90th-percentile scores are equal.
  • Situation 1: Drop in Aptitude: P = 60%, A = 60%, and U = 0% represent a performance drop explained entirely by reduced aptitude.The model succeeds perfectly on six of ten instructions and fails on the remaining four, while reliability is unchanged.
  • Situation 2: Drop in Reliability: P = 60%, A = 90%, and U = 90% represent a performance drop explained entirely by increased unreliability.Sharded and fully specified aptitude are equal, but performance varies substantially across simulations.
  • Situation 3: Combined Drop: P = 60%, A = 80%, and U = 60% represent a combined drop in aptitude and increase in unreliability.Compared with fully specified simulations, unreliability rises from 0% to 60%, whereas aptitude falls from 90% to 80%.
  • Behavioral Analyses: Later first answer attempts achieve 64.4 versus 30.9 when attempts occur in the final versus first 20% of conversations.Clarifying instructions or discussing the problem before generating a complete answer is associated with higher performance.
  • Behavioral Analyses: Final SHARDED answer attempts are 20-300% longer than FULL and CONCAT solutions as models retain incorrect assumptions.The analysis also finds middle-turn information receives less citation attention as conversations progress, although model-specific magnitudes are not reported.

F.4 Overly-verbose Assistant Responses

The verbosity analysis examines whether response length relates to performance across six tasks. Shorter responses generally perform better, while longer responses can introduce assumptions and burden users.

  • Method: The analysis regularizes verbosity by calculating response-length tags for each LLM-instruction pair.This accounts for task differences, such as Python functions requiring more text than SQL queries.
  • Performance and Verbosity: On five of six tasks, shortest-response conversations achieve performance 10-50% higher than longest-response conversations.Performance generally declines as assistant responses become longer; Actions is the exception.
  • Interpretation: Longer responses likely contain more assumptions or hypotheses, whereas short turns tend to stay focused on clarification.The authors connect this difference to maintaining the conversation’s direction across turns.
  • User Impact: Longer responses both deteriorate underlying model performance and take users longer to read.The paper characterizes this as undesirable for models and end-users.

Appendix K Gradual Sharding Implementation

The gradual sharding experiment varies how many conversational shards represent the same underlying instruction. It enables comparison across equivalent instructions with different information granularity.

  • Instruction Construction: Each underlying instruction is paired with one CONCAT version and seven SHARDED versions containing two to eight shards.The resulting 248 instructions are balanced across shard counts from one to eight.
  • Experimental Setup: The experiment uses 248 instructions, 10 conversations per instruction and model, and two models: GPT-4o and GPT-4o-mini.The instructions derive from 31 underlying instructions selected across Code, Math, and Data-to-Text.

Appendix L Temperature Experiment Implementation

The temperature experiment evaluates how decoding temperature affects aptitude and reliability in single- and multi-turn settings, while related simulations implement alternative ways to repeat prior information. The implementation uses controlled task, model, temperature, and conversation-setting combinations.

  • Temperature Experiment: The temperature experiment uses 40 instructions from Code, Database, Actions, and Math with GPT-4o and GPT-4o-mini.Each instruction is evaluated in FULL, CONCAT, and SHARDED settings at temperatures 0.0, 0.5, and 1.0.
  • Temperature Experiment: Each temperature condition runs for 20 simulations to improve percentile estimates for aptitude and reliability.The experiment uses fewer models and instructions than the main study, making the additional runs practical.
  • Alternative Simulations: RECAP is simulated by appending a recapitulation turn to each SHARDED run, allowing direct comparison with SHARDED results.The added turn gathers all previous user utterances.
  • Temperature Findings: Even at T = 0, multi-turn settings remain highly unreliable because subtle token- and turn-level non-determinism compounds.The paper notes that greedy decoding does not guarantee identical outputs in practice.
Loading 2505.06120v1…