PaperlayerView exact revision on arXiv

Source-linked AI summary

When Patients Cut In: Extending Clinical Conversational AI Safety to Interruptions

Zachary Ellis, Spencer Hazel, Adam Brandt, Yajie Vera He, Ernest Lim, Jared Joselowitz

arXiv:2608.29241v1cs.CL

TL;DR

Clinical voice-agent evaluations largely assume cooperative turn-taking, despite truncated turns potentially dropping required clinical content. This paper evaluates transcript-based interruption recovery across clinical phases, interruption types, cells, and four deployment-oriented LLM configurations. Recovery varies by cell and model, so the authors argue for content-grounded, per-cell evaluation matched to deployment interruption profiles.

  • Problem

    Existing clinical conversational-AI evaluations largely assume cooperative turn-taking, leaving interruption-induced loss of clinically required content insufficiently tested.

  • Method

    The paper builds a transcript-based evaluation using three operational overlap types, four clinical cells, reference-based content scoring, and four non-reasoning LLM agents.

  • Results

    Interruption recovery varies across cells and models; every model fails under at least one condition, and a brief apology marker shifts recovery inconsistently.

  • Takeaways & Limitations

    Clinical interruption robustness should be content-grounded, reported per cell, and aligned with the deployment's interruption profile rather than reduced to one aggregate score.

  • Takeaways & Limitations

    The text-transcript simulator omits prosodic, acoustic, wall-clock-timing, and ASR-error cues, and the findings concern cascaded-truncation architectures.

Abstract

from arXiv · show

Clinical voice agents are now deployed in routine care, where real patients do not wait their turn: they interrupt. These systems typically use a cascaded architecture (speech-to-text -> LLM -> text-to-speech), so when a patient cuts the agent off mid-utterance, clinically required content can be lost even when the model handles cooperative transcripts well. Yet clinical conversational-AI benchmarks almost universally assume patients wait for the agent to finish, missing interruption-induced loss of required content. We present a transcript-based evaluation of interruption recovery, adapting conversation-analytic overlap categories into three operational types (recognitional, competitive, transitional sub-unit) and testing four deployment-oriented, non-reasoning LLM configurations across four cells spanning history-taking (information gathering) and FAQ (information provision), scored on whether the agent preserves the clinically required content. In the gathering cells, target-question failure varied across models; in the provision cells, where arms are directly comparable, failure rose for every model. Rankings differ across cells, and competitive FAQ interruption produced 30/30 provision-coverage failures for all four models (Wilson 95% CI: 88.6-100.0%; baseline 0/30 for three, 4/30 for Llama). A brief apology marker ("sorry to interrupt") shifts recovery by tens of percentage points, inconsistently across models, and for one it reduces recovery. Interruption robustness therefore cannot be a single score: evaluation must be content-grounded, reported per cell, and matched to the deployment's interruption profile.

1 Introduction

Clinical voice agents increasingly operate in care settings where interruption robustness matters because cascaded systems pass truncated dialogue to the LLM. Existing clinical evaluations largely assume cooperative turn-taking, leaving interruption-induced loss of required content insufficiently tested.

  • Cascaded clinical voice agents use speech-to-text, LLM, and text-to-speech stages, making LLM robustness to received dialogue context part of the safety case.
  • Interruption recovery requires deciding whether to abandon, resume, or incorporate new input after an agent turn is truncated.The downstream recovery problem is textual once the interruption has been detected and the turn truncated.
  • Clinical conversational-AI evaluations typically score against reference rubrics but largely assume cooperative turn-taking.

EXAMPLE · COMPETITIVE FAQ INTERRUPTION

A competitive FAQ interruption can cut off safety-netting advice, after which the agent answers the patient's new question without restoring the dropped escalation guidance. The paper evaluates this clinical-content failure using interruption categories, four cells, and reference-based scoring across four LLM agents.

  • A competitive FAQ interruption cuts off safety-netting advice, and the agent then answers a new question without re-delivering the escalation advice.
  • The study adapts overlap categories into transitional sub-unit, recognitional, and competitive interruption types.
  • The evaluation spans four cells across history-taking and FAQ phases, scoring responses against reference clinical content units.
  • Interruption recovery differs across cells and models, and a brief apology preface can shift recovery rates by tens of percentage points for some models.

2 Related Work

Prior clinical conversational-AI evaluation focuses on reference-based safety and content scoring, while voice-agent benchmarks study interruption recovery in nonclinical tasks. Conversation analysis supplies interruption categories, but their use as a clinical-AI perturbation axis remains underdeveloped.

  • Clinical evaluation work scores specific safety obligations or atomic content points rather than general response appropriateness.
  • Existing interruption benchmarks examine reorientation, response-side failures, or task fulfilment primarily in consumer, enterprise, and task-oriented settings.
  • Conversation analysis distinguishes overlap by turn position and floor competition, providing a vocabulary for structurally different interruptions.
  • Prior clinical interruption studies are observational, and these categories had not been operationalised as a perturbation axis for evaluating clinical conversational AI.

3 Methodology

The methodology models interruptions in cataract follow-up conversations, classifies three clinically meaningful overlap types, and evaluates recovery against phase-specific reference content. Trials use a text-based simulator, binary judges, non-reasoning agents, baseline-versus-interruption arms, and marker ablations.

  • 3.1 Clinical Interruption Types: The taxonomy uses transitional sub-unit, recognitional, and competitive overlap, excluding between-turn transitional and terminal overlap because they pose less semantic risk.
  • 3.2 Conversation Phases and Cells: The study evaluates history-taking and FAQ provision in four cells within post-operative cataract follow-up phone conversations.
  • 3.3 Simulator: The simulator classifies whether to interrupt, selects a literal word-boundary truncation, and generates a patient turn consistent with the interruption type.
  • 3.4 Reference-Based Scoring: Reference-based scoring checks whether provision answers convey all required content points and whether gathering turns explicitly ask the canonical top-level question.
  • 3.5 Models and Run Protocol: The study evaluates reasoning-disabled agents because deployed real-time voice systems cannot afford the latency variance of dynamic thinking.
  • 3.6 Marker Ablation: Each cell compares baseline and interruption arms, while marker-ablation arms vary only the interruption utterance and inherit the parent cell's sample size.

4 Results

Interruption recovery varies by clinical phase, interruption cell, and model rather than following a single robustness ranking. Provision failures were universal under competitive FAQ interruption, while apology markers shifted recovery inconsistently across models.

  • Cross-cell pattern: Provision cells failed more severely than gathering cells, with every model showing increased coverage failure when baseline and interrupt arms were directly comparable.The provision-transitional cell instead showed graded failure rates: 23.3% for Haiku, 73.3% for Gemini, 83.3% for Llama, and 100% for GPT-5.4 mini.
  • Gathering cells: 83.3% gathering-recognitional failure for Gemini 2.5 Flash fell to 23.3% under competitive interruption, a 60pp swing for the same target question.Haiku fell from 40.0% to 0.0%, GPT-5.4 mini from 76.7% to 50.0%, and Llama from 36.7% to 30.0%.
  • Gathering cells: Gathering-recognitional results formed lower- and higher-failure model clusters, but competitive interruption changed the ordering and produced uneven per-model decreases.Baseline target-question failure was near zero, while interrupt-arm rates were yield-filtered and ranged from 0% to 83.3%.
  • Provision cells: 100% provision-coverage failure occurred for all four models under competitive FAQ interruption (30/30; Wilson 95% CI: 88.6–100.0%).Baseline failure was 0/30 for three models and 4/30 (13.3%) for Llama, producing +100pp and +86.7pp changes, respectively.
  • Marker ablation: Apology markers changed recovery by up to 67 percentage points in gathering and by 60pp for Haiku and 50pp for Gemini in provision.GPT-5.4 mini increased from 60.0% to 76.7% failure with the marker, whereas Llama changed little; effects were therefore model-dependent.

5 Discussion

Interruption recovery is a structured clinical safety problem: failures depend on interruption form, clinical obligation, and model, so a single robustness score is inadequate. Evaluation should therefore be cell-based, content-grounded, and matched to deployment-specific interruption profiles.

  • Structured recovery: Interruption robustness is conditional on the interactional form of the interruption and the clinical obligation at risk.The same models can exhibit different failure profiles across gathering cells that differ only in the patient’s interactional move.
  • Lost clinical obligations: Clinical evaluation should score lost obligations, such as unasked symptom questions or dropped safety-netting and escalation content, rather than surface fluency.A recovery can sound coherent while still failing to preserve required clinical content.
  • Lost clinical obligations: 100% provision-coverage failure occurred for all four models under competitive topic-change, versus near-zero baseline failure on the same content.The result was 0/30 for three models and 4/30 for Llama at baseline, followed by complete failure under interruption.
  • Recovery mechanisms: The strongest defensive prompt still left three of four models at 100% failure and moved only the fourth to 90% in the competitive provision case.Prompt-level instructions to resume after interruption did not reliably recover the unfinished clinical content.
  • Recovery mechanisms: A brief apology marker changed failure rates by up to 67 percentage points in gathering and up to 60 percentage points in provision, inconsistently across models.The marker reduced failure for some models but increased it for GPT-5.4 mini in gathering, so it is not a generic mitigation.
  • Scope and future design: The study evaluates one clinical pathway, four cells, and one default patient persona, limiting generalization of cross-model rankings.Deployment benchmarks should cover likely interruption types with pathway-appropriate reference content before generalizing these rankings.
  • Scope and future design: Tracking unresolved clinical obligations is proposed as a future architecture, not a tested fix.An explicit state record could keep interrupted questions and content points open until they are completed, unlike prompts alone.

6 Conclusion

The paper frames interruption recovery as preserving required clinical content, not merely maintaining conversational flow. Across history-taking and FAQ cells, recovery varied by cell and model, supporting cell-specific, content-grounded evaluation.

  • The evaluation treats recovery as a clinical safety problem by scoring whether interrupted agents preserve required content to elicit or convey.
  • Every evaluated model failed under at least one interruption condition across the tested history-taking and FAQ cells.
  • A brief “sorry to interrupt” marker shifted recovery by tens of percentage points within a cell, inconsistently across models.
  • Interruption robustness should be content-grounded, reported per cell, and aligned with the deployment’s interruption profile rather than summarized by one score.

Limitations

The study’s evidence is bounded by a transcript-based simulator, a single cataract follow-up pathway, English-only evaluation, and one default patient persona. It therefore does not directly establish performance across real-time voice conditions, other pathways, languages, or varied patient characteristics.

  • The simulator uses text transcripts with simulated truncation and omits prosodic, acoustic, and wall-clock-timing cues involved in real voice deployments.
  • The findings generalize to transcript-truncating barge-in architectures, not systems preserving full agent turns with marker tokens or avoiding truncation.
  • The evaluation covers only cataract follow-up, so reference content points would need re-derivation for other clinical pathways.
  • All evaluation components operate in English, and the study does not test whether interruption-failure patterns generalize outside English clinical interactions.
  • The patient simulator uses one default persona without demographic or paralinguistic variation such as age, accent, anxiety, or hesitation.

Ethics Statement

The ethics statement addresses simulated patients, LLM-based judging, and the dual-use potential of releasing interruption stress tests. It distinguishes defensive evaluation value from deployment claims and documents judge validation.

  • All evaluation dialogues were LLM-simulated, with one default persona and no demographic or paralinguistic variation.
  • Marker-dependent recovery is treated as an equity hypothesis for future evaluation rather than a deployable mitigation.
  • LLM judges were validated against two independent annotators on 60 dialogues per judge, with κ = 0.942 for target-question and κ = 0.933 for provision coverage.
  • The released taxonomy and simulator support defensive pre-deployment stress testing, while deployment-specific corpus content is withheld.

A Corpus Distribution of Overlap Types

The mini-corpus summarizes overlap instances by operational type and conversational phase in 35 cataract follow-up calls. Recognitional and transitional sub-unit overlaps dominate the observed instances, while competitive overlaps are uncommon.

  • 20 of 35 calls contained at least one overlap during a safety-critical sequence, yielding 27 observed overlap instances.
  • Recognitional and transitional sub-unit overlap accounted for 24 of 27 instances, while competitive overlap accounted for 3 of 27.
  • Table 3 distributes overlap instances across overlap types and conversational phases in the cataract follow-up mini-corpus.

B.1 Judge Validation Datasets Breakdown

The validation materials comprise annotation packs organized by cell and arm, with 60 dialogues distributed across four evaluated agent models.

  • 60 dialogues were distributed across four evaluated agent models in two labelling packs, one for each judge.
  • Table 4 organizes annotation-pack composition by cell and arm.

B.2 Annotator and Judge Agreement Data

The study used binary gathering and provision judgments, independently annotated and reconciled where necessary, with strong agreement for gathering and lower agreement for provision.

  • Gathering: κ = 1.000, with 60/60 trials unanimously agreeing on the gathering target-question verdict.The LLM judge matched resolved annotator ground truth on 59/60 trials, κ = 0.942 and 98.3% agreement.
  • Provision: κ = 0.831, with 55/60 provision trials unanimous and 91.7% annotator agreement.The five disagreements clustered around the strictness of escalation-criteria thresholds.
  • Judgment criteria: Gathering target-question judgments asked whether the agent asked or otherwise covered the canonical eye-redness question.Truncated or interrupted utterances did not count as asking the question.
  • Judgment criteria: Provision coverage required substantive conveyance of all three reference points for the bright-light-sensitivity FAQ.The three points were normal-sensitivity reassurance, escalation criteria, and mild-case advice such as wearing sunglasses outside.
  • Procedure: Annotators were blinded during initial passes, and disagreements were reconciled in discussion into resolved ground truth.Hazard verdicts were recorded regardless of interruption realism.
  • Defensive prompt follow-up: Three of four models still failed on 100% of defensive-strong interrupt trials, while Gemini 2.5 Flash fell to 90%.The standard prompt used n = 30 per arm per model; defensive-strong used n = 10.

D Methodology Supplements

The supplements define interruption examples, prompt-driven patient simulation, provision content decomposition, and prompt-variant comparisons for the evaluated clinical cells.

  • Interruption examples: The three interruption types are illustrated using Jeffersonian transcript conventions, including pauses, elongation, overlap onset, and simultaneous talk.
  • Provision content: Provision cells evaluate coverage of three clinician-reviewed content points for a light-sensitivity-after-cataract-surgery FAQ.
  • Patient prompts: Patient interruption prompts specify scenario state, interruption type, placement, transcript context, and whether the agent utterance matches the interruption rule.
  • Interruption placement: The simulator’s interruption-placement prompt outputs a literal prefix of the agent utterance before the cut.It requires exact words, no paraphrase, and no surrounding explanation.
  • Generation modes: Patient-turn generation uses directive, free, or hard-coded modes depending on the evaluation cell.Hard-coded mode is used for marker-ablation arms.
Loading 2608.29241v1…