PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond the Assistant Turn: User Turn Generation as a Probe of Interaction Awareness in Language Models

Sarath Shekkizhar, Romain Cosentino, Adam Earle

arXiv:2604.02315v2cs.AI

TL;DR

Assistant-only benchmarks leave unanswered whether language models anticipate conversational consequences after producing an answer. The paper uses user-turn generation to probe this interaction awareness and finds it largely decoupled from task performance, while controlled perturbations and targeted post-training change the metric.

  • Problem

    Assistant-only benchmarks evaluate response correctness but do not measure whether a model generates a grounded conversational consequence under the user role.

  • Method

    The paper generates an assistant answer, switches to the user role, and evaluates whether the resulting turn is grounded in the query and answer across models and datasets.

  • Results

    Interaction awareness is largely decoupled from task performance: Qwen3.5 GSM8K accuracy rises from 41.6% to 96.8% while grounded follow-up remains near zero at every scale.

  • Takeaways & Limitations

    User-turn generation captures an interaction-awareness dimension that assistant-only evaluation misses, and its distinct model-specific patterns suggest training recipes shape it.

  • Takeaways & Limitations

    The probe measures grounded follow-up rather than recovery of a held-out human continuation, and multilingual, code-generation, and longer-horizon settings remain untested.

Abstract

from arXiv · show

Standard LLM benchmarks evaluate the assistant turn: the model generates a response to an input, a verifier scores correctness, and the analysis ends. This paradigm leaves unmeasured whether the LLM encodes any awareness of what follows the assistant response. We propose user-turn generation as a probe of this gap: given a conversation context of user query and assistant response, we let a model generate under the user role. If the model's weights encode interaction awareness, the generated user turn will be a grounded follow-up that reacts to the preceding context. Through experiments across $11$ open-weight LLMs (Qwen3.5, gpt-oss, GLM) and $5$ datasets (math reasoning, instruction following, conversation), we show that interaction awareness is decoupled from task accuracy. In particular, within the Qwen3.5 family, GSM8K accuracy scales from $41\%$ ($0.8$B) to $96.8\%$ ($397$B-A$17$B), yet genuine follow-up rates under deterministic generation remain near zero. In contrast, higher temperature sampling reveals interaction awareness is latent with follow up rates reaching $22\%$. Controlled perturbations validate that the proposed probe measures a real property of the model, and collaboration-oriented post-training on Qwen3.5-2B demonstrates an increase in follow-up rates. Our results show that user-turn generation captures a dimension of LLM behavior, interaction awareness, that is unexplored and invisible with current assistant-only benchmarks.

1 Introduction

The paper identifies a gap in assistant-only evaluation: models may answer correctly without generating a grounded conversational consequence. It proposes user-turn generation as a probe of interaction awareness and reports that this capability is decoupled from task accuracy.

  • Motivation: 96% GSM8K accuracy can coexist with a user-role continuation that completely falls apart, exposing a gap in assistant-only evaluation.Observed failures include repeating an earlier turn or continuing as the assistant.
  • Approach: User-turn generation appends the assistant answer to the conversation and generates under the user role to probe anticipated reactions.The generated user turn is treated as a behavioral readout from the same model weights.
  • Motivation: Interaction awareness matters for multi-agent collaboration, self-play, interactive deployment, interpretability, and collaborative applications.The paper also connects user reactions to training for better multi-turn conversational models.
  • Contributions: The study formalizes the probe, jointly measures assistant accuracy and follow-up quality, and evaluates 11 models spanning 0.8B to 397B parameters.Experiments cover cross-family comparisons, temperature sampling, controlled perturbations, and collaboration-oriented post-training.

2 User Turn Generation

The paper defines user-turn generation as producing a user continuation conditioned on a query and assistant response, then evaluates whether that continuation is grounded. It measures this across models and datasets using an LLM judge, controlled perturbations, and post-training interventions.

  • 2.1 Problem Formulation: Given query q and assistant response a, user-turn generation produces a continuation u under the user role conditioned on the preceding context.The study considers both self-generated assistant responses and held-out real conversational responses.
  • 2.2 Measuring Interaction Awareness: A genuine follow-up references, reacts to, or builds upon specific content in the conversation; degenerate turns restate, continue, plan, or launch another task.The criterion tests whether the user continuation is grounded in (q, a).
  • 2.2 Measuring Interaction Awareness: The evaluator returns a binary genuine-follow-up judgment σ, a descriptive label ℓ, and a rationale, with genuine-follow-up rate as the primary metric.Labels include previous-turn restate, new task prompt, assistant-turn restate, malformed artifact, meta planning, degenerate short, plausible followup, and other.
  • 2.2 Measuring Interaction Awareness: The LLM judge assesses whether u is semantically grounded and pragmatically plausible across datasets, achieving human-annotation agreement of κ = 0.726.The judge is preferred over embedding classifiers because it generalizes without per-domain classifier training.
  • 2.3 Experimental Setup: The evaluation covers 11 open-weight models, 5 benchmark datasets, 2 held-out conversational datasets, controlled perturbations, and collaboration-oriented post-training.The model set includes eight Qwen3.5 models, two gpt-oss models, and GLM-4.7.
  • 2.3 Experimental Setup: Truncation and appended generic questions serve as positive controls for sensitivity to incomplete assistant responses and explicit conversational cues.The perturbations test whether follow-up rates respond to causal changes in the assistant turn.
  • 2.3 Experimental Setup: The cross-family results show task accuracy and follow-up rate remain largely uncorrelated under deterministic generation.gpt-oss reaches 17–21% follow-ups on GPQA Diamond despite lower answer accuracy than higher-accuracy GLM-4.7 and Qwen3.5-397B.

3 Results

Across model families, task accuracy and follow-up quality remain largely uncorrelated. Temperature sampling can surface latent interaction awareness, but it does not scale monotonically with model size; perturbations and post-training further show that the behavior is sensitive to training and generation conditions.

  • 3.1 Task Accuracy Does Not Predict Follow-Up Quality Across Families: Task accuracy and follow-up rates remain largely uncorrelated across representative Qwen3.5, gpt-oss, and GLM-4.7 models under deterministic generation.The paper evaluates accuracy and follow-up quality jointly, finding distinctive behavior across the two measures.
  • 3.1 Task Accuracy Does Not Predict Follow-Up Quality Across Families: gpt-oss models reach 17-21% follow-ups on GPQA Diamond despite 61-67% answer accuracy, whereas GLM-4.7 and Qwen3.5-397B achieve 66.4% and 86.1% accuracy with 1.5% and 0.5% follow-up rates.These cross-family results show that stronger answer accuracy does not imply stronger follow-up quality.
  • 3.2 Interaction Awareness Is Latent in the Distribution: Qwen3.5-27B rises from 0% to 22% on GSM8K, 1.5% to 35.9% on GPQA Diamond, and 1% to 30.7% on IFBench as temperature increases.Qwen3.5 and GLM show strong temperature scaling, while gpt-oss responses vary by dataset and model.
  • 3.2 Interaction Awareness Is Latent in the Distribution: Higher-temperature sampling reveals interaction awareness in Qwen and GLM distributions, while deterministic generation leaves the capability near the mode only rarely.The paper contrasts latent awareness with contexts where temperature does not elicit interaction-aware continuations.
  • 3.3 Within-Family Analysis: Model Size Does Not Predict Interaction Awareness: GSM8K accuracy climbs from 41.6% to 96.8% across Qwen3.5-0.8B to 397B-A17B, while five of eight models produce 0.0% follow-up at T=0.The within-family comparison separates model size from interaction-awareness behavior.
  • 3.3 Within-Family Analysis: Model Size Does Not Predict Interaction Awareness: At T=1.0 on IFBench, Qwen3.5-397B-A17B reaches 43.7%, but Qwen3.5-9B reaches 36% and Qwen3.5-0.8B reaches 27.7%, showing non-monotonic scaling.On GPQA Diamond, Qwen3.5-27B reaches 35.9%, while Qwen3.5-0.8B peaks at 32.3% at T=0.7.
  • 3.3 Within-Family Analysis: Model Size Does Not Predict Interaction Awareness: Both Qwen3.5-9B and Qwen3.5-27B answer the same GPQA chemistry question correctly, but only the smaller model generates a user turn engaging with the reasoning.The larger model restates the original prompt verbatim, illustrating divergent interaction awareness despite equal task correctness.
  • 3.4 Controlled Perturbations Confirm the Metric: Truncating assistant responses sharply increases genuine follow-up for GLM-4.7 from 1.0% to 55.0% and for gpt-oss-120b from 0.0% to 24.2% on GSM8K.The effect also appears on GPQA Diamond, while Qwen3.5-27B remains unchanged on GSM8K and flat on GPQA Diamond.

4 Discussion and Conclusion

Across 11 open-weight models and five benchmark datasets, interaction awareness is largely decoupled from task performance. The probe is sensitive to controlled perturbations and post-training, but its evaluation and generalization remain limited.

  • Interaction awareness remains largely decoupled from task performance across 11 open-weight models and five benchmark datasets.
  • Within Qwen3.5, GSM8K accuracy rises from 41.6% to 96.8%, while grounded follow-up rates remain near zero across model sizes.
  • Controlled perturbations and collaboration-oriented post-training move follow-up rates in interpretable directions, indicating that interaction awareness is latent, measurable, and increaseable through targeted training.
  • The evaluation relies on an LLM judge and remains untested for multilingual settings, code generation, and longer-horizon multi-turn interactions.
  • Genuine follow-up remains minority behavior even in the strongest cells, while downstream uses such as reranking and self-play training data remain future work.

A Full Results Tables

The appendix provides complete result tables for Qwen3.5 scaling and temperature sweeps across the benchmark datasets. These tables complement the main-text figures and report genuine-follow-up rates alongside task accuracy.

  • Tables 3 and 5 provide complete results across all five benchmark datasets, complementing the three-dataset figures in the main text.
  • Table 3 reports Qwen3.5 GSM8K answer accuracy and genuine-follow-up rates across all five benchmark datasets.
  • Table 4 contains the complete Qwen3.5 temperature-sweep data underlying Figure 4.

B Failure Mode Taxonomy

The three model families exhibit systematic, qualitatively distinct failure profiles rather than one shared interaction-awareness failure. These profiles support the view that training recipes shape a family of related behaviors.

  • Qwen3.5 is dominated by previous-turn restatement at 78.5%, with degenerate short outputs comprising 16.6%.Restatement is consistent across model sizes, while degenerate short outputs are particularly common for Qwen3.5-397B-A17B.
  • The dominant failure mode for each family accounts for 43–78% of generations, indicating systematic family-level patterns.
  • gpt-oss primarily restates the assistant turn at 70.2%, accompanied by moderate meta planning at 6.8%.The model copies or paraphrases its assistant response under the user role rather than restating the original prompt.
  • GLM-4.7 is dominated by meta planning at 43.0% and assistant-turn restatement at 23.0%, with malformed artifacts at 10.8%.Its meta-planning behavior attends to the role-header change but produces assistant-style reasoning instead of a user-appropriate response.
  • Truncation increases genuine follow-up for gpt-oss and GLM but not Qwen, consistent with different attention to the assistant turn.

C Representative Generation Examples

Representative generations show that each family’s dominant failure has a distinct form, while genuine follow-ups react to specific content in the assistant response. Examples include grounded subproblem continuation, clarification, and revision requests.

  • Qwen3.5 commonly reproduces the original prompt verbatim under the user role, ignoring the assistant response.This previous-turn restatement is the modal Qwen failure, occurring in 78.5% of generations.
  • Qwen3.5-9B generates a grounded clarification request about an unstated distance used in an astrophysics calculation.It produces 13.1% genuine follow-ups on GPQA Diamond, unlike most other Qwen sizes.
  • gpt-oss can emit internal verification text under the user role instead of a human follow-up.The example retains an analysis framing and checks formatting and solution constraints rather than reacting conversationally.
  • A gpt-oss example nevertheless drills into a specific subproblem left partially unresolved by the assistant, qualifying as a genuine continuation.
  • GLM-4.7 frequently produces assistant-style planning text under the user role, but can also issue a specific revision request.The examples contrast “Here’s a thinking process” with a concise request to shorten an overly long cover letter.
  • Genuine follow-ups are grounded in assistant-response content, including clarification requests, factual challenges, and revision instructions.

D LLM Judge Validation and Human Annotation

The evaluation system’s reliability was assessed through judge comparisons and blinded human review, covering agreement, annotation procedures, and correlation analysis.

  • The study evaluates LLM-judge reliability using same-family comparisons and blinded manual review with human annotators.The validation covers judge selection, cross-judge agreement, human annotation, and correlation analysis.
  • Validation examines agreement between automated judges and between judges and blinded human annotators.
  • The analysis reports both agreement metrics and the procedures used to obtain them.

D.1 Same-Family Judge Validation

Same-family validation supports using gpt-5.4-mini for paper-facing tables while retaining nano results for broad trend checks, not robust rankings or difficult borderline cases.

  • 81.7% genuine-followup agreement with κ = 0.480 accompanies a rate drop from 26.6% under nano to 18.5% under mini.Nano remains usable for large sweeps on natural data, but the judges are not equivalent.
  • Qwen rows remain stable or rise slightly under mini, whereas gpt-oss and GLM-4.7 rates shrink sharply.gpt-oss-120b falls from 21.2% to 4.8% on HealthBench and from 43.0% to 16.1% on Coval.
  • 82.5% genuine-followup agreement with κ = 0.477 and moderate file-level correlations show that the prevalence-preserving slice tracks the full rerun closely.Pearson r = 0.642 and Spearman ρ = 0.685 are reported for file-level rates.
  • On balanced hard cases, nano agrees with mini and full gpt-5.4 on about 73% of genuine-followup decisions, with κ ≈0.37.Nano marks 37.9% as genuine, compared with 22.4% for mini and 18.1% for full gpt-5.4.
  • Mini is used for paper-facing tables, while older nano runs remain useful for broad trend checks and disagreement-slice construction.

D.2 Manual Review Methodology

The manual review study uses blinded, standardized annotation packets to test whether benchmark conclusions survive comparison with human judgment.

  • The study conducts blinded manual review with human annotators to validate the LLM-based evaluation system beyond judge-to-judge comparison.
  • Annotation protocol: Annotation packets contain 100 examples sampled either from judge disagreements and agreed cases or from the natural prevalence of the full panel.
  • Annotation protocol: Annotators view the conversation context and generated user turn without model identity, dataset metadata, reference turn, or judge labels.They use the same evaluation criteria and primary-label inventory as the LLM judge.
  • Correlation analysis: Human-versus-judge agreement is reported at the binary genuine-followup level, while inter-annotator agreement is not computed.The analysis emphasizes packet coverage and correlation with the LLM judge.
  • Correlation analysis: 90.5% genuine-followup agreement with κ = 0.726 is achieved by mini across blinded packets, versus 73.0% / 0.311 for nano.Primary-label agreement for pooled mini is lower at 58.5% with κ = 0.481, supporting the binary choice rather than full label equivalence.

E Related Work

The paper positions its probe between theory-of-mind, user-simulation, multi-turn optimization, and interactive-benchmark research while emphasizing a distinct focus on the assistant model’s own interaction awareness.

  • Theory of mind in LLMs: Prior theory-of-mind work finds that explicit belief attribution can exceed behavior prediction and judgment, motivating a knowing-versus-applying comparison.
  • Theory of mind in LLMs: This paper’s probe differs by testing whether a model encodes the consequence of its own response through first-person behavioral generation.SimpleToM instead probes third-person inference about another agent’s mental state.
  • User simulation: Unlike dedicated user-simulation work, the paper does not build a reusable user proxy but probes interaction awareness in the assistant model’s own weights.It also catalogs prompt restatement, meta-planning, and identity leakage as failure modes when assistant models are repurposed as user proxies.
  • Multi-turn optimization and interaction-aware training: Multi-turn optimization methods extend training beyond the current response using future collaboration quality, post-action user states, or hierarchical multi-turn reinforcement learning.
  • Interactive benchmarks: Interactive benchmarks embed consequence modeling in tool-mediated environments, whereas this probe uses a narrower design to attribute differences to model weights and training.

F Continued Discussion

The discussion tests whether low follow-up rates reflect training-distribution or template artifacts and finds evidence for context-dependent interaction awareness, while limiting claims to behavioral output.

  • Single-turn training data: Collaboration-oriented multi-turn training raises follow-up rates, supporting training distribution as a causal factor in interaction awareness.The authors frame this as complementary to the identified gap: without multi-turn continuations, training may not instill interaction awareness.
  • Chat template conventions: Qwen3.5 models sharing a chat template show follow-up rates from 0.0% to 13.1%, challenging a template-only explanation.
  • Chat template conventions: Perturbations substantially change gpt-oss and GLM follow-up rates while leaving Qwen largely unaffected, arguing against a purely template-level explanation.
  • Role-header semantics: HealthBench and Coval follow-up rates reach 19–48%, exceeding benchmark-dataset rates and indicating that richer conversational context can elicit grounded continuations.The authors interpret this pattern as consistent with interaction awareness being latent but context-dependent rather than absent.
  • Behavioral versus representational claims: The probe measures behavioral output under the user role, not internal representations, so its claims concern generated text under the experimental setup.Representation-level analyses could complement the behavioral probe.
Loading 2604.02315v2…