PaperlayerView exact revision on arXiv

Source-linked AI summary

Jarvis: A Proactive Speech Agent for Multi-Party Conversations

Seunghyun Oh, Hirotaka Hiraki, Shuyue Stella Li, Yulia Tsvetkov, Shyamnath Gollakota

arXiv:2610.07506v1cs.HCcs.AI

TL;DR

Speech agents are typically reactive and dyadic, leaving open how one can contribute selectively to multi-party conversation. Jarvis follows discussion against a shared document and intervenes on unresolved gaps or errors using deterministic checks and cited evidence. It reports high correctness and deference to group self-repair in a synthetic benchmark, with real-time interventions also tested in Zoom sessions.

  • Problem

    Speech agents are typically reactive and dyadic, and existing settings lack a way to objectively judge whether an unaddressed intervention in multi-party conversation is correct, timely, or needed.

    Source: §1 · p. 2

  • Method

    Jarvis monitors discussion against a shared document, waits for unresolved gaps or errors to persist despite self-repair, and grounds spoken interventions in cited source sentences.

    Source: §1 · p. 2

  • Results

    On CHI-180-proactive, Jarvis stayed silent in 97% of self-corrected or self-answered cases; in live Zoom sessions, 163 of 164 interventions were correct or partially correct.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Document-grounded conversations make it possible to measure proactive interventions against ground truth for correctness and timing.

    Source: §7 · p. 16

    Jarvis answers only from the uploaded document, so it cannot answer questions outside that source.

    Source: §6 · p. 16

Abstract

from arXiv · show

Speech agents are reactive and dyadic: they speak when spoken to, and to one person at a time. We ask what it takes for a speech agent to instead take part in a conversation among several people and speak up only when it can help. We introduce Jarvis, a real-time proactive speech agent that audibly participates in multi-party human conversations. Grounded in a document shared beforehand, Jarvis follows the discussion and intervenes when the group misses or misstates a fact and does not correct itself within a few turns. We make three contributions: a problem setting based on epistemic breakdowns that makes proactive intervention measurable, realized as CHI-180-proactive, a synthetic multi-party dataset seeded with known gaps, errors, and self-corrections; a proactive backbone that harnesses a small, open-weight model with deterministic checks and grounds every claim in a source sentence; and interaction techniques for taking the floor in live speech and showing the cited evidence on screen. On CHI-180-proactive, Jarvis is correct on most events it addresses and stays silent 97% of the time when the group resolves an issue itself. A live study with 23 participants confirms these trends with real-time interventions.

1 INTRODUCTION

Jarvis is designed to join multi-party speech without waiting to be addressed, intervening on document-checkable gaps or errors only after giving the group a chance to self-correct. The paper introduces a measurable setting, a deterministic document-grounded backbone, and live interaction techniques, and reports results in a synthetic benchmark and Zoom study.

Motivation

  • Existing speech agents are mainly reactive and dyadic, while multi-party agents must decide when an unaddressed intervention helps without disrupting the group.The paper frames this as a problem of deciding whether and what to say, when and how to take the floor, and how to evaluate intervention.Source: §1 · pp. 1–2

Contributions

  • CHI-180-proactive pairs 180 CHI papers with synthesized discussions containing 4,320 planted events over 50,400 turns, each annotated with a fixed correct action.The setting targets checkable epistemic breakdowns grounded in a document shared with Jarvis beforehand.Source: §1 · p. 2
  • The deterministic harness uses a monitor, self-repair check, and gate to defer intervention until a document-checkable problem persists unresolved, with spoken claims grounded in document sentences.The paper also explores waiting for an opening, raising a hand for permission, and displaying cited evidence.Source: §1 · p. 2

Findings

  • Jarvis provided correct or partially correct information with 95–97% accuracy on CHI-180-proactive and stayed silent in 97% of cases when participants self-corrected or answered their own questions.When it intervened, it acted within 2–3 turns in 98% of cases.Source: §1 · p. 2
  • In live Zoom sessions with 23 participants, 163 of 164 interventions were correct or partially correct, and Jarvis and participants resolved 74% of planted errors, 72% of spontaneous errors, and 83% of information gaps.Eighty-five percent of interventions arrived within two turns of the problem, despite transcription errors from the speech pipeline.Source: §1 · p. 2
  • Participants preferred an agent that raises its hand before speaking and overwhelmingly preferred having cited sources shown on screen (21 of 23).The displayed source let participants check claims and trust answers.Source: §1 · p. 3

2 RELATED WORK

Prior work addresses proactive assistance, turn-taking, group conversation, and grounded generation, but does not combine autonomous spoken intervention in human multi-party conversation with content-level evidence and objective ground truth. Jarvis targets that gap by focusing on document-checkable epistemic breakdowns.

Proactive Agents and the When-to-Speak Problem

  • Proactive-agent research has examined timing and user attitudes, but much work remains formative or Wizard-of-Oz rather than autonomous spoken intervention in multi-party conversation.Earlier notification systems deliver information on screen, while spoken interventions consume shared airtime.Source: §2 · p. 3
  • Multi-party text agents can select moments to contribute, but do not ground intervention in whether the group missed or misstated a verifiable fact; audio benchmarks test reactions to predefined sounds instead.Private AR or whispered cues are not spoken into the group conversation.Source: §2 · p. 3

Turn-Taking, Interruption, and Full-Duplex Spoken Dialogue

  • Turn-taking and full-duplex systems model floor availability, but assume utterances are addressed to the agent and do not determine whether content warrants intervention.Jarvis treats the agent as a deliberately dispreferred other-repairer that waits through self-repair.Source: §2 · p. 3

Agents in Meetings and Group Conversations

  • Meeting assistants typically summarize or answer queries rather than autonomously correct errors, while group-conversation agents are largely text-based, formative, or lack ground truth.ProMediate uses turn-based simulated negotiation, where the agent receives its turn and interventions lack an objective answer key.Source: §2 · p. 3

Grounded Generation and Verifiable Attribution

  • Grounded-generation methods usually assume a written response to an explicit question, checked offline by the person who asked.Jarvis instead targets proactive spoken corrections during ongoing group discussion.Source: §2 · p. 3

3 SYSTEM DESIGN

Jarvis separates speech processing, a document-grounded proactive backbone, and interaction delivery. Its backbone checks for actionable problems and self-repair before a deterministic gate permits a cited response; the interaction module manages openings, interruptions, permission, and visible evidence.

3 SYSTEM DESIGN

  • The system separates turn and audio processing, proactive decision and grounding, and interaction delivery; speech processing supplies timestamped speaker-attributed turns and plays responses into the meeting.The interaction module waits for an opening, yields when someone speaks, and can raise a hand or display the cited page.Source: §3 · p. 3, §3.1 · p. 4

3.2 Proactive Backbone: Deterministic harness of small models

  • A deterministic harness uses multiple model stages with logic checks, so no single model decision can trigger an intervention on its own.The backbone has a monitor, self-repair check, grounded generator, direct answerer, and gate; all stages use numbered document sentences.Source: §3.2 · p. 4

3.2.2 Monitor.

  • The monitor identifies actionable missing facts, incorrect facts, and misattributions, while distinguishing them from on-track, off-topic, and document-unanswerable turns.It normalizes the discussion point as a claim and returns source sentence IDs when it can ground the issue.Source: §3.2.2 · p. 5

3.2.3 Self-Repair Check.

  • The self-repair check tests whether participants stated the correct fact for an open claim, and a deterministic guard closes it only when the value appears in the recent turns.The guard rejects questions and admissions of not knowing; for misattributions, it accepts a retraction.Source: §3.2.3 · p. 6

3.2.4 Claim Ledger and Gate.

  • The ledger tracks claims across turns, and the gate speaks only when persistence reaches a threshold, self-repair has not closed the claim, and cooldown and repeat-speech conditions permit it.The threshold gives participants turns to correct a problem before the gate fires; an ungroundable generated response may be rolled back.Source: §3.2.4 · p. 6

3.2.5 Grounded Generation.

  • Grounded generation produces concise corrections or document answers with cited sentence IDs, and the system discards responses when valid citations cannot be resolved.The spoken utterance can orient listeners with a topic and point them to the cited page and keywords.Source: §3.2.5 · p. 6

3.3.1 Proactive Entry, Barge-in, and Recovery.

  • Jarvis queues interventions until a long silence, no participant speech, and low input energy indicate an opening, then yields to sustained participant speech and replays, resumes, or drops interrupted output according to how much was delivered.Hand-raise mode instead waits for a participant to give Jarvis the floor and discards the response if permission does not arrive within 60 seconds.Source: §3.3.1 · pp. 6–7, §3.3.2 · p. 7

3.3.3 Visually communicating agent state and evidence.

  • The interface signals whether Jarvis is idle, listening, thinking, or speaking, and shows the cited page and passage in a shared PDF viewer so participants can check its spoken claim.In hand-raise mode, the raised hand signals that Jarvis has something to say.Source: §3.3.3 · p. 7

4 SYNTHETIC DATASET GENERATION

CHI-180-proactive makes proactive intervention measurable through synthetic conversations grounded in CHI papers, with planted gaps, errors, and self-corrections. Its event design tests when Jarvis should speak or remain silent, while the generation pipeline builds and screens the conversations.

4 SYNTHETIC DATASET GENERATION

  • CHI-180-proactive contains 4,320 events across 50,400 turns, generated from 180 CHI papers with 24 planted problems per paper.The papers span 2008–2025, with ten papers from each of 18 subcommittees.Source: §4 · p. 7

4.1 Events and Ideal Actions

  • Each event marks an open question or false claim, its ideal action, and a usefulness window; about 256 of roughly 280 turns per paper are ordinary turns where silence is correct.Late corrections do not count once the group has moved on.Source: §4.1 · p. 7
  • Matched silent counterparts pair open questions with correctly resolved questions and false claims with self-corrections, separating detection failures from unnecessary interruptions.The trigger is present in both cases; whether the group resolves it differs.Source: §4.1 · p. 7
  • Two planted error types test fine-grained grounding: nearby same-unit number swaps and distinctive facts misattributed from another CHI paper.The cross-paper errors test whether the agent stays bound to the shared document rather than accepting a similar fact from elsewhere.Source: §4.1 · p. 7

4.2 Generation Pipeline

  • The generation pipeline extracts 12 question-answer items, plants eight false claims, then synthesizes conversations with events spaced ten turns apart.The question-answer items also benchmark question answering; self-corrections require Jarvis to hold back for two turns. Resolved-question events occupy a separate 60-turn conversation, apart from the 220-turn main conversation.Source: §4.2 · pp. 7–8

4.1 Events and Ideal Actions

  • The five event types and their ideal actions are organized in Table 1.Source: Table 1 · p. 8

4.2 Generation Pipeline

  • Two researcher roles are synthesized with an Inner-Thoughts-style engine, and GPT-OSS-120B was adopted as one adequate configuration after screening four open-weight models across 16 role assignments.Fifteen of 16 conditions fell below ArgSciChat’s 0.14–0.19 split-half divergence range, so screening did not reliably discriminate among configurations at this sample size.Source: §4.2 · p. 8

5 EVALUATION

The evaluation examines Jarvis in synthetic conversations and live interventions, assessing its responses to group-resolved issues, yielding behavior, visible evidence, model size, and participant feedback on intervention costs and broader support needs.

5.1.1 Benchmarking our small models.

  • On QASPER and CHI-180, the 9B model trailed the 35B models and the 4B model trailed further, while Strict+Partial scores spread by at most .028.For the deployed model on QASPER, the cited-sentence match rate was .907, rising to .980 when manual review counted near-gold or otherwise supporting sentences as valid. The 4B model had the shortest TTFT and total time across all three datasets; the 9B model was slower end-to-end than the 35B models.Source: Table 3 · p. 9, §5.1.1 · p. 9

5.1.2 Benchmarking Agent Yielding Behavior When a Person Speaks.

  • With barge-in yielding enabled, Jarvis stopped speaking in 51 of 54 trials (94.4%), versus 0 of 54 trials (0.0%) when yielding was disabled.Among the 51 trials in which it stopped, median yield latency was 486 ms (range 465–521 ms).Source: Figure 4 · p. 10, §5.1.2 · p. 10

5.2.2 Scoring.

  • On CHI-180-proactive, Jarvis addressed 90.6% of gap and 76.9% of wrong events, with correct or partially correct contributions in 97.0% and 95.3%, respectively.Good outcomes were 88.3% for gap events and 76.1% for wrong events; Jarvis stayed silent in 97.4% of cases where participants resolved issues and invented answers to 2.5% of unanswerable questions.Source: §5.2.3 · p. 10, Figure 5 · p. 11

5.2.3 Results.

  • Jarvis stayed silent after participant self-corrections 96.8% of the time and acted within two turns in more than 95% of addressed gap and wrong events.Most misses were silences rather than off-target interventions; removing the combined persistence gate and self-repair check reduced silence after self-correction to 16% and caused speech before repair in 80% of cases. Cross-paper imported facts accounted for 117 of 166 wrong-event misses, versus 49 number swaps.Source: §5.2.3 · p. 11, §5.2.4 · p. 11, §5.2.5 · p. 11

5.3.2 Study design.

  • The live study involved 23 participants across 14 sessions, and its five feature conditions were presented in sequence rather than counterbalanced.Because each later condition followed more exposure to Jarvis, the authors treated paired ratings as impressions in sequence, not controlled comparisons between settings.Source: §5.3.1 · p. 12, §5.3.2 · p. 12

5.3.3 Intervention Timing and Correctness.

  • In live meetings, 139 of 164 intervention decisions (85%) arrived within two turns, and 148 of 164 interventions (90%) matched the paper.From source display onward, the page and quote matched the evidence in 84 of 87 cases. The agent acted on 52 of 84 planted errors, 19 of 32 spontaneous errors, and 84 of 107 questions; when it fell short, it generally stayed silent rather than speaking wrongly.Source: §5.3.3 · p. 13

5.3.4 Participant ratings.

  • Participants rated source display as improving their ability to check answers (2.26 to 4.65) and trust them (2.48 to 4.74), while 18 reported looking at the source versus 8 without it.Participants also rated the agent helpful at 3.91 and trustworthy at 4.22; 10 preferred hand-raising, 5 preferred speaking without it, and 8 said it depended. These were sequential impressions, not controlled comparisons.Source: §5.3.4 · p. 13

5.3.5 Qualitative feedback.

  • Participants most often named routine work meetings as a setting for Jarvis, while formal or high-stakes settings and private conversations were common places they would not use it.Other suggested uses included document-detail discussions and information lookup.Source: §5.3.5 · p. 13
  • One participant found being corrected “over and over” about the paper “a little embarrassing.”This feedback came from P3.Source: §5.3.5 · p. 14
  • Participants wanted reminders when stuck or forgetful (5), new ideas or related work (3), basic information during silence (3), and flags for off-topic discussion (3).Two wanted external sources; individual requests included showing a page to someone lost, stepping in on unresolved questions, or answering only on a wake word. Two said the current scope was enough.Source: §5.3.5 · p. 14
  • 21 of 164 interventions stated a fact already said; 10 cases involved a participant repeating it during the hand-raise delay, versus one such case in Speak.In one instance, the lead corrected themselves to “90 participants” and Jarvis gave the same number three seconds later.Source: §5.3.5 · p. 14
  • In hand-raise parts, participants sometimes read Jarvis’s pending on-screen answer and said the fact first, leaving Jarvis to confirm it.The hand-raise indicator itself became a channel for information.Source: §5.3.5 · p. 14
  • 11 interventions repeated prior content (7 played), including 7 triggered by one wrong utterance and 3 when a subtopic returned.The cooldown allowed intervening again on the same topic, but Jarvis did not omit information it had already said.Source: §5.3.5 · p. 14

6 DISCUSSION AND LIMITATIONS

The discussion bounds Jarvis’s evidence to controlled and remote-meeting settings, while identifying limitations in its coverage, reliability, and deployment. It also frames hand-raising as a way to manage the social cost of public correction.

6 DISCUSSION AND LIMITATIONS

  • The evaluation uses deliberately controlled planted errors; CHI-180-proactive’s regular event structure makes correctness and timing decidable but may not reflect messier natural errors.The live study also included spontaneous errors, which the authors use to assess whether performance depends on the planted events’ structure or phrasing.Source: §6 · pp. 14–15
  • 32 spontaneous live-study errors provide a less-confounded check on planted-error results, though the lead researcher ran every session and may have made planted errors unusually easy to catch.The authors report that Jarvis handled spontaneous errors at a rate comparable to planted ones.Source: §6 · p. 15
  • By withholding public corrections until a participant grants the floor, hand-raising lets the group decide whether a correction is worth its interpersonal cost.The authors frame this as face management as well as floor management.Source: §6 · p. 15
  • Jarvis’s silence does not establish that a statement is correct: it deliberately abstains when uncertain and can miss reworded claims, speech-recognition errors, or facts in tables it cannot cite.Its interventions are not 100% comprehensive, so participants remain responsible for their own claims.Source: §6 · p. 16
  • Jarvis answers only from the uploaded document, prioritizing verifiable source-backed claims over questions outside that document.Long or multiple documents can exceed the small models’ context capacity; larger models increase latency pressure, while retrieval adds a step.Source: §6 · p. 16
  • The backbone uses empirical fixed thresholds and timing windows; appropriate values may vary across groups, documents, and speaking styles.Learning or adapting these values is left for future work.Source: §6 · p. 16
  • The system runs on datacenter GPUs, and local deployment for offline or privacy-sensitive settings would depend on smaller models.In online meetings, audio already leaves participants’ devices for the meeting platform’s servers.Source: §6 · p. 16
  • The evaluation covers CHI-paper discussions in Zoom with per-participant audio; co-located groups pose harder speaker-separation and diarization challenges.The authors say the technique could apply to contracts, filings, specifications, and other meeting platforms, but those settings need evaluation.Source: §6 · p. 16

7 CONCLUSION

Jarvis joins multi-party conversation using a shared document, intervening when the group misses or misstates a fact and giving participants a chance to correct it first. Its document-grounded claims and timing can be checked against evidence and ground truth.

7 CONCLUSION

  • Jarvis follows a group discussion against a shared document, waits for participants to correct missed or misstated facts, and backs interventions with on-screen source evidence.Document grounding also makes intervention correctness and timing objectively measurable.Source: §7 · p. 16

A SPEECH PROCESSING MODULE: IMPLEMENTATION DETAILS

The speech-processing module handles separate participant audio streams, detects speech and wake words, segments turns, and delivers synthesized responses into the meeting. Its components combine per-speaker processing with streaming detection and playback.

A SPEECH PROCESSING MODULE: IMPLEMENTATION DETAILS

  • Each participant’s audio stream is processed in parallel by voice-activity detection, wake-word detection, and automatic speech recognition.The Zoom implementation receives 16-kHz mono audio in 20-ms frames.Source: A SPEECH PROCESSING MODULE:… · p. 19
  • Per-participant voice-activity detection emits acoustic events, using speech onset, sustained speech, and short- or long-silence thresholds to track activity.The implementation uses 30-ms frames, 0.5-second sustained-speech and hangover thresholds, and 0.5- and 2.0-second silence thresholds.Source: A SPEECH PROCESSING MODULE:… · p. 19
  • A two-stage “hey agent” detector proposes wake-word candidates, then verifies them, with contextual filtering and transcript fallback to reduce false triggers and recover acoustic misses.On 285.4 hours of held-out AMI and CANDOR audio, it achieved 99.08% recall with 0.077 false accepts per hour.Source: A SPEECH PROCESSING MODULE:… · p. 19
  • Per-speaker streaming ASR uses turn-end detection after 500 ms of silence, with a 700-ms watchdog as backup if audio delivery stalls.Median latency from speech end to turn-end detection was 224 ms.Source: A SPEECH PROCESSING MODULE:… · p. 19
  • Speech synthesis streams sentence-bounded text in chunks and uses a jitter buffer before sending audio packets into the meeting.After warm-up, latency from the first text input to audible playback is 0.70–0.95 seconds.Source: A SPEECH PROCESSING MODULE:… · p. 19

B MONITOR MODULE: ANSWERING DIRECT QUESTIONS BASED ON CONTEXT

The direct-question monitor answers questions from the document while filtering out fragments and participant-to-participant talk. It uses recent conversational turns to resolve references, and cites document sentences for spoken answers.

B MONITOR MODULE: ANSWERING DIRECT QUESTIONS BASED ON CONTEXT

  • The monitor answers complete questions supported by the document, abstains when it cannot answer, and skips incomplete fragments or participant asides.Its fixed abstention reply is “That isn’t covered in the document”.Source: B MONITOR MODULE: ANSWERING DIRECT… · pp. 19–20
  • It uses the last eight turns as context to resolve references, then gives a concise spoken answer citing the document sentences.Answers are limited to at most three sentences.Source: B MONITOR MODULE: ANSWERING DIRECT… · p. 20

C INTERACTION MODULE: ANSWERING DIRECT QUESTIONS

The interaction module binds wake-word events to a speaker’s ASR turns, assembles complete multi-turn questions, and starts answers speculatively to reduce waiting.

C INTERACTION MODULE: ANSWERING DIRECT QUESTIONS

  • A direct question is answered as soon as it is ready, and short follow-up turns remain part of the question until a long gap, another participant’s substantive turn, or a timeout.The wake event is matched to an ASR turn from the same speaker by timing overlap or a shortly following start.Source: C INTERACTION MODULE: ANSWERING DIRECT… · p. 20
  • After a brief pause, speculative generation begins; continued speech triggers regeneration, while a new question cancels queued output and active playback from the old one.The continuation window closes with a <Complete> marker.Source: C INTERACTION MODULE: ANSWERING DIRECT… · p. 20
  • Wake events bind when the same-speaker turn overlaps the wake position within ±300 ms or begins within 5 s; unbound events expire after 20 s.A wake-only turn contains at most three tokens and lasts at most 2.5 s.Source: C INTERACTION MODULE: ANSWERING DIRECT… · p. 20
  • A wake-only turn opens a 5 s grace window; continuation is accepted within 4.5 s or while VAD remains voiced, up to a 30 s session limit.Speculative <Silence> is sent after 0.5 s of silence; generation closes immediately if transmitted text covers the VAD endpoint within 700 ms, otherwise 1.5 s later.Source: C INTERACTION MODULE: ANSWERING DIRECT… · p. 20

D SYNTHETIC DATASET CONSTRUCTION DETAILS

The construction details describe the CHI source papers, paired conversational events, and the fact and question items used to build CHI-180-proactive.

D.1 Source Documents

  • The dataset draws 180 papers from ten examples in each of the 18 CHI subcommittees listed in the CHI 2026 guide.The authors describe this selection as covering CHI topic areas without bias toward any one subcommittee.Source: §D.1 · p. 20

D.2 Event Pairing and Placement

  • Each speak-event has a silent control: resolved questions reuse a gap item, while self-corrections use a different claim and replace adoption turns with a participant repair.Resolved-question events are isolated in separate conversations to avoid silence being attributable to repetition suppression; gap events cover six sub-types and share a cue, failed resolution, and document-based answer structure.Source: §D.2 · p. 20

D.3 Stage 1 Details: Facts and QA Items

  • Each paper yields 300–470 extracted, passage-anchored facts; the pipeline uses Qwen3-235B-A22B-Instruct-2507 [58] and verifies unanswerable items against the full text.A passage-text hash flags mismatches if later parsing changes passage boundaries.Source: §D.3 · p. 20
  • Each paper contributes up to 12 questions in a balanced 4/4/2/2 answer-type mix, and each item serves both as a QA item and as a conversational event.Ten answerable items become gap events, two unanswerable items become unanswerable events, and four gaps are reused as resolved-question events.Source: §D.3 · p. 21

D.4 Stage 2 Details: Error Validity Checks

Planted errors and self-corrections are constrained so their validity does not depend on conversational context or leak the document’s answer.

D.4 Stage 2 Details: Error Validity Checks

  • In self-correction events, a participant repairs the false claim two turns after it; each planted claim must pass four deterministic validity and leakage checks.The checks require the claim to be false, not decidable from conversation alone, non-interfering with QA items, and not revealed elsewhere in the conversation.Source: §D.4 · p. 21
  • A wrong value must be absent from extracted facts, and entities in a misattributed fact must be absent from the paper.These checks ensure the planted claim is false under the paper’s stated criteria.Source: §D.4 · p. 21
  • The claim cannot contradict earlier conversation, interfere with the twelve QA items, or reveal the correct value elsewhere in the conversation.Self-correction events use a brief intervening reply before the original speaker repairs the claim.Source: §D.4 · p. 21

D.5 Stage 3 Details: Plan, Personas, and Generator Screening

The dataset-generation pipeline plans conversational events, screens generated dialogue and leakage, and specifies the prompts and judging procedures used across system stages and evaluations.

D.5 Stage 3 Details: Plan, Personas, and Generator Screening

  • The main conversation has 220 turns with gap, unanswerable, wrong, and self-correction events; a separate 60-turn conversation contains resolved-question controls.Plans specify cue, failed-resolution, and decision-use turns; generated turns are restricted to local fact pools, and a validator rejects turns that violate answer or planted-error constraints.Source: §D.5 · p. 21
  • The authors screen four open-weight dialogue generators across 16 speaker-role conditions, comparing generated eight-turn windows with real-conversation references.Naturalness was worst when Qwen3-235B asked, while engagingness was worst whenever Qwen3.8-27B appeared; both patterns were reported against ArgSciChat divergences.Source: §D.5 · p. 21

PROMPTS OF THE PROACTIVE BACKBONE

Monitor

  • The monitor runs on every ordinary turn, receiving the recent turns and the new utterance alongside the shared document block.The monitor’s system and user messages are reproduced in Tables 9 and 10.Source: Monitor · p. 21

Self-repair check

  • When an open claim exists, the self-repair check runs in parallel with the monitor and uses a deterministic test before closing a proposed resolution.Open claims are listed as [C#] entries.Source: Self-repair check · p. 21

Grounded generation

  • Grounded generation runs only after the gate fires, derives its intent from monitor status, and resolves cited sentence IDs to page text before delivery.An answer-bias instruction is included when the monitor supplies answer sentence IDs; the spoken response points listeners to a page and keyword.Source: Grounded generation · p. 21

Direct answering

  • Direct answering handles wake-word turns outside the monitor, ledger, and gate, removes the wake phrase, and supplies the last eight turns as context.Its prompt is reproduced in Table 13.Source: Direct answering · p. 21

LLM JUDGE PROMPTS / User-study intervention judging

Judge for proactive-behavior events

The appendix specifies how proactive-event judgments are made and catalogs related prompts, rubrics, and labeling instructions for other evaluation tasks.

Judge for proactive-behavior events

  • Each planted event is scored with one Claude Sonnet 5 call using the Table 20 rubric, full paper, event details, and a transcript window spanning two turns before to five after onset.The evaluated intervention is inserted where it arrived and marked [EVALUATED].Source: Judge for proactive-behavior events · p. 22, Table 20 · p. 30
  • Judgments are based on support in the paper rather than agreement with the event’s gold string.The rubric also omits three unused output fields: a grounding flag, evidence quote, and note.Source: Judge for proactive-behavior events · p. 22
  • The prompt materials cover monitoring, self-repair checking, grounded generation, and direct answering.Tables 9–13 identify the monitor system and user messages, self-repair prompt, grounded-generation prompt, and direct-answering prompt.Source: Table 9 · p. 23, Table 10 · p. 24, Table 11 · p. 24
  • The direct-answer accuracy judge uses one call per 130-item batch and the same prompt across three datasets.Tables 16–18 cover its rubric, gold-origin strings, and the user-study intervention judge rubric.Source: Table 14 · p. 27, Table 15 · p. 27, Table 16 · p. 28
  • The appendix also includes instructions for judging live-study interventions and labeling problem utterances.Tables 17–19 identify the intervention-judge instruction and rubric, and problem-utterance labeling instruction.Source: Table 17 · p. 29, Table 18 · p. 29, Table 19 · p. 30

Related papers

Loading 2610.07506v1…