PaperlayerView exact revision on arXiv

Source-linked AI summary

Thinking While Speaking: Inference-Time Knowledge Transfer for Responsive and Intelligent Conversational Voice Agents

Vidya Srinivas, Zachary Englhardt, Vikram Iyer, Shwetak Patel

arXiv:2511.07397v3cs.CL

TL;DR

Voice agents must balance slow, capability-enhancing reasoning and retrieval with millisecond conversational responsiveness. ConvFill uses a small Talker to provide immediate grounded responses and integrate streamed Reasoner knowledge, delivering responsive performance while retaining frontier-level task capability.

  • Problem

    Voice agents must balance iterative, time-consuming reasoning, tool calls, and retrieval with the millisecond response times required for seamless conversation.

  • Method

    Conversational infill uses a small Talker to generate grounded filler responses during Reasoner latency and transform streamed knowledge chunks into natural conversation.

  • Results

    ConvFill consistently delivers millisecond-level time-to-first-response across three task types, retains QA accuracy relative to frontier models, and is preferred outright for RAG.

  • Takeaways & Limitations

    ConvFill places voice agents at a new latency–capability Pareto point by coupling lightweight-model responsiveness with foundation-model task performance.

  • Takeaways & Limitations

    The evaluation did not capture Reasoner timeouts or excessive latency, and fillers could become disruptive during prolonged delays.

Abstract

from arXiv · show

Voice agents face a fundamental tension: the reasoning, retrieval, and tool use that make foundation models capable are iterative and slow, while conversational interaction demands responses on a millisecond timescale. Smaller, real-time models meet the latency bar but cannot match foundation models on complex tasks, leaving current voice agents to trade away either responsiveness or capability. We introduce conversational infill, where a small talker model both immediately generates contextually grounded responses to hide the latency of an external reasoner model and fluently integrates streamed reasoner knowledge into its responses during inference. We curate a 290,571-example synthetic dataset spanning six domains and demonstrate that this task is learnable across seven widely used small language models ranging from 135M to 1.7B parameters. Our system implementation, ConvFill, sustains millisecond-level time-to-first-response while closing the accuracy gap to within 6.3% of the corresponding frontier reasoner performance. In a live user study (n=18) with talker deployments running on an Apple M2 SoC, participants rank ConvFill on par with frontier models overall, prefer it for retrieval-heavy tasks, and rate it significantly more responsive. These results show that conversational infill unlocks a new point on the latency-capability Pareto frontier, offering a practical path toward voice agents that are both responsive and highly capable. Code, models, and datasets are available at https://github.com/vysri/conversational-infill.

1 Introduction

Advances in large language models have enabled increasingly capable task-oriented systems across customer service, virtual assistance, tutoring, and interactive software. The paper provides dataset, code, and model resources for conversational infill.

  • Background: Large language model advances have enabled increasingly capable task-oriented systems.These systems support applications including customer service, virtual assistants, tutoring, and interactive software.
  • Application scope: Task-oriented LLM systems are used for customer service, virtual assistance, tutoring, and interactive software.
  • Resources: The paper provides a conversational-infill dataset, code, and models through public repositories.The listed resources are github.com/zenglhardt/convfill-dataset and github.com/vysri/conversational-infill.

A. Conversational Infill

Conversational infill separates immediate dialogue from slower reasoning: a lightweight Talker responds to the user while a frontier Reasoner supplies knowledge chunks that the Talker integrates during inference. ConvFill implements this design to preserve millisecond responsiveness while adding reasoning, retrieval, and tool-use capabilities.

  • Motivation: Conversational infill addresses the conflict between iterative reasoning, retrieval, and tool use and the millisecond response times required for seamless spoken interaction.Voice agents otherwise trade capability for responsiveness because these capabilities are inherently time-consuming.
  • Conversational infill: A lightweight Talker interacts directly with the user while a frontier Reasoner handles multi-turn reasoning and tool calls, returning concise knowledge chunks for conversational integration.The Talker transforms incoming Reasoner knowledge into natural conversation and can respond without external knowledge when necessary.
  • System implementation: ConvFill masks Reasoner latency with filler responses, lets the Talker operate from transcriptions and text-only training, and preserves independent Talker–Reasoner model selection.This retains cascaded-system flexibility without requiring the resource-intensive audio modeling used by full-duplex approaches.
  • Scope and feasibility: The released ConvFill dataset contains 290,571 rigorously validated training examples across six domains, and conversational infill is learnable across seven SLMs from four model families.The evaluated families are Qwen, Llama, Gemma, and SmolLM.

2 Hiding Latency in Conversations

Conversational systems hide latency by providing immediate acknowledgments or infill while slower processing continues. ConvFill extends dynamic infill with a single consistent Talker that incorporates streamed Reasoner knowledge throughout multi-turn dialogue.

  • Latency compensation: Immediate acknowledgments and generic infill can preserve conversational responsiveness because prolonged silence interrupts real-time conversational flow.Acknowledging that the user has been heard is described as essential for responsive, natural-feeling conversation.
  • Dynamic infill: Unlike approaches that allow lower-latency models to be corrected by higher-capability models, ConvFill keeps users interacting only with a Talker trained to incorporate external knowledge.This design avoids repeated revisions or inconsistencies that can harm trust, confidence, and engagement in spoken dialogue.
  • System architecture: ConvFill concurrently sends transcribed speech to the Talker and Reasoner, letting the Talker produce filler responses before ingesting streamed knowledge for conditioned replies.The Talker passes all responses to text-to-speech, while Reasoner outputs enter a streamed knowledge queue.

3 Task Formulation

The task pairs a low-latency Talker with a high-performance Reasoner, requiring the Talker to maintain conversational continuity while incorporating asynchronously streamed reasoning, retrieval, tool-use, and external knowledge. The Talker generates phrase-by-phrase responses from conversation context and either incoming knowledge chunks or silence signals, preventing stalls during variable Reasoner latency.

  • 3 Task Formulation: The Talker-Reasoner architecture assigns advanced, high-latency reasoning and tool use to the Reasoner, while the Talker provides low-latency conversational responses and consumes streamed information (Chris-takopoulou et al., 2024).The Talker must produce contextually contingent phrases when necessary to hide Reasoner latency.
  • 3.1 Task Setup: The Talker is a lightweight small language model for consumer hardware, whereas the Reasoner manages multi-turn context, tools, and external knowledge sources.A turn is defined as one exchange between the user and Talker.
  • 3.1.1 Talker Interface: At each turn, the Talker forms its response T_τ as a sequence of phrases conditioned on the current user utterance, conversation history, streamed Reasoner knowledge, and previously generated phrases.The conversation history uses a configurable lookback of ℓ turns, and i indexes the phrase position within the turn.
  • 3.1.2 Reasoner Interface: The Reasoner asynchronously emits a knowledge stream E_τ whose elements are either semantically complete reasoning, retrieval, inference, or tool-use chunks or dynamically inserted silence tokens.Silence tokens indicate that the Reasoner has not yet produced a knowledge chunk.
  • 3.1.3 Inference-Time Collaboration: The Talker and Reasoner operate concurrently: the Reasoner streams knowledge asynchronously while the Talker continuously consumes the next stream element at each phrase position.This interface supports inference-time collaboration rather than waiting for the Reasoner to finish its full response.
  • 3.1.3 Inference-Time Collaboration: When a knowledge chunk r_i arrives, the Talker conditions the next phrase on it alongside the user utterance, history, and prior phrases; when a silence token arrives, it uses only those conversational inputs.Knowledge and silence are mutually exclusive at each position.
  • 3.1.3 Inference-Time Collaboration: The one-element-per-position constraint |T_τ| = |E_τ| enables continuous output despite variable Reasoner latency and prevents the Talker from stalling while awaiting knowledge.Silence tokens allow generation to continue when no Reasoner chunk arrives within the threshold duration.

4 ConvFill Dataset

The ConvFill dataset trains Talker models on synthetic conversations pairing user utterances with factual Reasoner content and conversational rephrasings. Its construction combines realistic task-oriented dialogues with validation procedures that preserve turn alignment and inference-time grounding.

  • Dataset construction: The dataset pairs user utterances with factual responses and conversational rephrasings while restricting Talker grounding to past and current visible inputs.This prevents responses from referencing information unavailable to the Talker at inference time.
  • Dataset construction: Samples begin from DSTC8 task-oriented dialogues, retaining labeled user utterances and using labeled system responses as Reasoner thoughts padded with <sil> placeholders.Talker responses are generated conditioned on the resulting thoughts array, and a second corpus expands conversational topic variety.
  • Dataset validation: Candidate conversations are validated for turn counts, matching Talker–Reasoner phrase pairs, and ordering errors using pairwise BERTScore comparisons.The BERTScore check tests whether each thought aligns most strongly with its corresponding response entry within each turn.
  • Dataset validation: End-to-end dataset generation using Claude Opus 4.6 cost approximately $2,400, including retries caused by validation failures.Prompt templates, validation details, and additional dataset statistics are provided in Appendix B.

5 Infill Model Training and Inference

ConvFill fine-tunes Talker models with role-delimited conversational-infill examples and streams Reasoner knowledge through queued Talker and TTS responses. The implementation supports seven Talkers from 135M to 1.7B parameters, three Reasoners, and INT8 MLX inference on Apple M2 hardware.

  • Talker Training: ConvFill training adds a [KNOWLEDGE] control sequence to each Talker’s existing role and turn-boundary template.Most models retain [USER], [ASST], and [END] controls, while [KNOWLEDGE] marks external knowledge for infill training.
  • Talker Training: Training examples combine one-turn history, the latest user utterance, external knowledge, and optional prior Talker phrases.The format uses a history lookback of ℓ=1 to maintain consistency across turns, with additional history possible for larger ℓ.
  • ConvFill Inference: During inference, parallel Talker and Reasoner execution feeds knowledge and speech through separate queues, while <sil> tokens generate contingent filler when speech is unavailable.The first queue buffers Reasoner knowledge and the second buffers Talker responses for TTS; <sil> is inserted most often at turn starts while awaiting knowledge.
  • Model Configurations: The evaluation covers seven Talkers across four families, spanning 135M–1.7B parameters, paired with three Reasoners to form 21 ConvFill configurations.Talkers include Gemma 3, Qwen3, SmolLM2, and Llama 3.2; all use ℓ=1 in evaluation.
  • Model Configurations: All Talkers are converted to INT8 MLX and evaluated on an Apple MacBook Pro with an M2 SoC and 16 GB memory.This conversion enables benchmarking and evaluation directly on the target hardware.

6 Benchmark Evaluation

Benchmark evaluation spans single-turn and multi-turn tasks, using accuracy, phrase-level NLI, and turn-level LLM-as-a-Judge metrics to assess capability, grounding, and conversational usefulness. ConvFill Talkers approach their Reasoner accuracy while substantially improving over base SLMs, with performance scaling depending on task complexity and metric.

  • Evaluation framework: Evaluation combines accuracy with Entailment, Non-Contradiction, Coverage, Faithfulness, and Helpfulness to distinguish correctness, grounding, omissions, unsupported additions, and conversational usefulness.Entailment evaluates knowledge-conditioned Talker phrases, Non-Contradiction evaluates <sil>-conditioned filler phrases, and the judge metrics operate over complete dialogue turns.
  • Single-turn QA: ConvFill Talkers perform within 0%–6.3% of corresponding Reasoners, while improving over base SLMs by up to 51% on LLAMA1 and 63.4% on SimpleQA.Figure 4 compares ConvFill Talkers with Base SLMs, Reasoners, and Frontier Models; Reasoner accuracy serves as the corresponding Talker ceiling.
  • Single-turn scaling: SimpleQA shows positive scaling trends across Accuracy, Entailment, Coverage, Faithfulness, and Helpfulness, whereas LLAMA1 has uniformly high Accuracy, Entailment, and Coverage with limited room for scaling.Base SLM accuracy also correlates positively with parameter count across the two QA datasets.
  • Multi-turn scaling: MultiWOZ exhibits positive scaling for Coverage, Faithfulness, and Helpfulness, while Everyday Conversations shows no significant scaling trends, reflecting differing dataset complexity.This pattern parallels the contrast between the more challenging SimpleQA and easier LLAMA1 datasets.
  • Scaling exceptions: Non-Contradiction generally does not improve with Talker size and correlates negatively with size on MultiWOZ, although SmolLM2-derived models show positive size correlations on both QA datasets.Non-Contradiction is therefore the main exception to otherwise positive performance scaling.

7 User Evaluation

An end-to-end user study found that ConvFill matched Frontier overall on most quality ratings, exceeded it on latency, and substantially reduced response delay for retrieval-heavy interactions while maintaining comparable responsiveness to Base SLM.

  • User Ratings: ConvFill was rated equivalently to Frontier on Clarity, Fluency, Response Length, Coherence, Task Completion, and Satisfaction, exceeded Frontier on Latency, and fell below it on Naturalness.Base SLM was rated significantly below both ConvFill and Frontier across all metrics.
  • Interaction Conditions: The study collected 144 interactions and 635 total turns from 18 participants across Direct, RAG, and MCP tasks, with randomized system order and Latin Square task ordering.Participants completed all three task types; MCP omitted Base SLM because it could not make structured MCP calls.
  • Real-Time Interactive System: 542 ms vs. 617 ms, ConvFill Talker TTFR was similar to Base SLM for Direct tasks, while 976 ms vs. 3,812 ms, it was much lower for RAG tasks.ConvFill offloads retrieved-context processing to the Reasoner, reducing the Base SLM’s retrieval burden; Reasoner mean TTFR increased across Direct, RAG, and MCP tasks from 2,947 ms to 7,242 ms.

8 Discussion

ConvFill combines millisecond-level responsiveness with near-frontier QA performance, and users rank it on par with frontier systems while preferring it for retrieval-heavy interaction. Its modular, independently trainable design supports deployment across diverse small models and realistic hardware, though performance depends on model-specific prompting.

  • Dynamic Fillers for Perceived Responsiveness: ConvFill delivers millisecond-level TTFR across three task types, receives the strongest subjective latency ratings, and retains near-frontier QA accuracy.Talker TTFR is equal to or lower than the corresponding base SLMs while Reasoner assistance recovers almost all frontier-model-only performance.
  • Reducing Local Inference-Time Burden: Talker accuracy generally approaches the Frontier ceiling, but trails Gemini 3.1 Pro on SimpleQA by an average of 9.8% because the Reasoner output format degrades frontier performance.The evaluation uses one generic prompt structure across Reasoner models rather than model-specific prompt tuning, yielding a conservative uniform-deployment estimate; prompt choices can cause model-dependent differences (Zhuo et al., 2024; Lu et al., 2022; Sclar et al., 2024).
  • Live Interaction Takeaways: Users rank ConvFill on par with Frontier for Direct and MCP tasks and prefer it outright for RAG, where external-result latency is highest.The live evaluation introduced RAG and MCP tasks absent from training and measured the full system on real users.
  • Development Feasibility: Conversational infill is learnable across Gemma, Qwen, SmolLM, and Llama models ranging from 135M to 1.7B parameters, with synthetic-data generation costing $2,400 once.Evaluation and live interaction use INT8-quantized Talker models on an Apple M2 laptop with 16 GB of memory.
  • Development Feasibility: The independently trainable Talker supports task-specific conversational adaptation without directly fine-tuning frontier text or audio models, while Reasoners can be swapped without retraining or changing Talker prompts.This modularity enables drop-in use of pre-existing frontier-model tools and data sources without re-engineering them for real-time conversational behavior (Défossez et al., 2024; Xia et al., 2024; Jin et al., 2023; Luo et al., 2025).

9 Related Work

Recent work separates speech generation from reasoning or retrieval to balance conversational responsiveness with model intelligence. These approaches include token-level thinking, backend reasoning, Thinker–Talker architectures, speculative decoding, parallel tool prediction, and retrieval-aware voice systems.

  • 9 Related Work: Recent systems divide speech and reasoning across models to balance responsiveness and intelligence, including Mini-Omni-Reasoner (Xie et al., 2025) and SplitReason (Akhauri et al., 2025).
  • 9 Related Work: Qwen2.5-Omni (Xu et al., 2025) uses a Thinker–Talker architecture in which the Talker converts the Thinker’s high-level representations into speech tokens.
  • 9 Related Work: Related methods address interaction delays through speculative decoding, parallel tool-call prediction, oracle-based response refinement, and retrieval backends paired with full-duplex voice frontends.Chien et al. handle retrieval delays of up to two seconds.

10 Conclusion

The paper introduces conversational infill, enabling lightweight Talker models to respond promptly with context-aware fillers while incorporating external Reasoner knowledge during inference. ConvFill combines millisecond-level responsiveness, grounding, and task accuracy, placing it at a new latency–capability Pareto frontier (Lin et al., 2025).

  • Conclusion: Conversational infill lets lightweight and responsive SLM Talkers produce context-aware fillers while incorporating external knowledge from frontier-class Reasoners during inference.The task leverages a Talker–Reasoner approach to support prompt responses without giving up access to stronger model knowledge.
  • Conclusion: ConvFill consistently delivers millisecond-level time-to-first-response across tasks with variable Reasoner latency and across Reasoner models while maintaining external-knowledge grounding and task accuracy.This combines responsive Talker behavior with streamed knowledge from frontier-class Reasoner models at inference time.
  • Conclusion: These properties place ConvFill at a new latency–capability Pareto frontier (Lin et al., 2025) and support conversational AI for task-oriented and tool-use scenarios underserved by current state-of-the-art voice models.The approach couples lightweight SLM responsiveness with the raw task performance of foundation models.

Limitations

ConvFill’s independent Talker can generate fillers before the guardrailed Reasoner responds, leaving safety and prolonged-latency behaviors incompletely addressed. Further evaluation should examine fallback behavior, larger samples, and languages beyond English.

  • Safety coupling: The independently operating Talker may not immediately comply with safety constraints because it generates fillers before Reasoner responses are available.Future work should explore tighter safety coupling between the Talker and guardrailed Reasoner.
  • Latency failures: ConvFill was not evaluated under Reasoner timeouts or excessive latency, where prolonged filler sequences could become disruptive.Future work should investigate appropriate inter-filler pauses and fallback strategies.
  • Evaluation scope: Larger user evaluations could improve the statistical robustness and generalizability of the findings, while future work could test applicability beyond English.These limitations concern both evaluation scale and language coverage.

Ethical Considerations … D.3 Study Tasks

The appendices document ethical safeguards, dataset construction and validation, training and formatting details, and the user-study protocol and task categories. Together, they specify reproducible generation procedures, model configurations, released-data scope, and participant protections.

  • Ethical Considerations; D.1 Ethical Considerations: The study received institutional review-board approval; participants were informed, compensated, non-vulnerable, and exposed to no foreseeable risk, while synthetic data excluded personally identifiable information and inappropriate content.The appendix also states that participants were told participation was voluntary and that study data would appear in an academic publication.
  • A Appendix; A.1 AI Use Statement: The appendix organizes reproducibility materials across dataset generation, model training, user-study details, full benchmark results, and inference prompts, and states that authors—not LLM assistants—produced the scientific content and conclusions.LLM assistants supported programming, formatting, figure and table generation, and proofreading.
  • B Dataset Generation; B.1 Topic-Seeded Freeform Generation; B.2 DSTC8 Schema-Guided Scaffolded Generation: The dataset combines topic-seeded freeform conversations across six domains with DSTC8-reskinned task-oriented dialogues, using scaffolded JSON generation and response infills paired with concise thoughts.The freeform track uses seeded topics and prescribed conversation scaffolds, while DSTC8 preserves source user and system content and adds generated infill delivery.
  • B.2 DSTC8 Schema-Guided Scaffolded Generation: DSTC8 generation constrains infills to the previous turn and current turn so far, counteracting the source scaffold’s access to future dialogue content.Structural validation also requires source user and thought content to remain verbatim, preventing generation-time visibility from leaking unavailable information.
  • B.3 Validation Cascade: A four-stage validation cascade checks structure, thought-response semantics, positional alignment, and DSTC8 scaffold preservation before admitting a conversation.The cascade combines structural rules, NLI contradiction gates, BERTScore anchor and cross-position checks, and source-consistency and visibility checks.
  • B.4 Generation Settings; B.5 Output Format and Yield: The released JSONL dataset contains 8,443 conversations and 74,508 turns: 6,005 freeform conversations with 50,324 turns and 2,438 DSTC8-scaffolded conversations with 24,184 turns.Generation hyperparameters and per-domain configuration files are provided for reproduction.
  • C Model Training Details; C.1 Base SLM Models; C.2 Frontier Models; C.3 Control Sequence Expansions for Different Talker Models; C.4 Shared Training Parameters for Talker Models: Training details identify seven fine-tuned base SLMs, API-based frontier models, model-specific control-token expansions, and shared Talker training on 90/10 train/evaluation splits.The configurations include hardware and GPU-hour accounting, while [KNOWLEDGE] is mapped through each model’s existing control-token template.
  • D User Study Details; D.2 Study Instructions; D.3 Study Tasks: The user study lasted about 45 minutes per participant, compared three systems across three task blocks, and collected interaction ratings covering latency, clarity, fluency, response length, coherence, task completion, and naturalness.Participants conversed freely toward task goals, then ranked systems by preference; assigned tasks were categorized as Direct, RAG, and MCP.

D.3.1 Direct Tasks … D.9 ConvFill Speedup, TTFR, & Filler Details

The live study compared ConvFill with Base SLM and Frontier systems across direct, retrieval, and MCP tasks, while evaluating human judgments, rankings, and latency-related behavior. Results include a significant ConvFill preference for RAG, strong grading reliability, and task-specific speedup, latency, and filler analyses.

  • D.3.1 Direct Tasks: The study evaluated direct task dialogues for trip planning, recipes, brainstorming, and events using Base SLM, Frontier, and ConvFill configurations.Participants selected among three scenarios, including birthday-party planning, train travel, and Greek-yogurt cake preparation.
  • D.3.2 RAG Task: The RAG task used a two-stage retrieve-then-rerank pipeline over 367 document chunks, passing the top three reranked chunks to the language model.Queries used text-embedding-3-large embeddings and FAISS retrieval of the top 30 candidates before cross-encoder reranking.
  • D.3.3 MCP Task: The MCP task connected Frontier and ConvFill to an IMAP email server because Base SLM models could not reliably produce properly formatted MCP calls.Participants queried inbox counts, unread messages, event dates, handoff summaries, build failures, Rust updates, and roadmap deadlines.
  • D.4 Participant Condition Assignment: Eighteen participants evaluated Direct, RAG, and MCP conditions in a fully counterbalanced Latin square design, with configuration order also counterbalanced.Each condition appeared six times in every ordinal position; MCP included Frontier and ConvFill, while the other conditions also included Base SLM.
  • D.5 Live Interaction Setup; D.5.1 Live Interaction Example: The live system used voice input with faster-whisper transcription, Silero VAD, macOS speech synthesis, and INT8-quantized MLX models running on an Apple M2 SoC.Participants interacted through a press-and-hold recording interface and heard spoken responses rather than seeing model outputs.
  • D.6 Infill Grading: Human agreement was substantial for Coverage (α = 0.672), Faithfulness (α = 0.67), and Helpfulness (α = 0.751), with moderate human–LLM agreement across all dimensions.Human–LLM agreement was Kendall’s τb = 0.55 and 0.554 for Coverage and Faithfulness, and point-biserial r = 0.69 for Helpfulness.
  • D.7 TOST Test Details for Likert-Scale Ratings; D.9 ConvFill Speedup, TTFR, & Filler Details: The study’s TOST analysis compared Frontier and ConvFill ratings across all tasks, while latency analyses reported task-level speedup, user-experienced Talker latency, Reasoner latency, and <sil> filler-token counts.The supplied tables define speedup as Reasoner latency divided by Talker latency and report filler-token means, standard deviations, and 95% confidence-interval half-widths.

E Full Benchmark Evaluation Results · F Inference and Evaluation Prompts

The appendix reports benchmark-evaluation table scopes and documents the prompts used for live interaction and LLM-as-a-Judge assessment. Evaluation distinguishes thought coverage, faithfulness, and helpfulness while applying task-specific backend instructions.

  • E Full Benchmark Evaluation Results: The benchmark appendix covers QA accuracy on SimpleQA and LLAMA1, including ConvFill Talker conditional accuracy and comparisons with frontier, Reasoner, and Base SLM responses.Table 17 reports accuracy percentages with full 95% Wilson score confidence intervals over questions.
  • E Full Benchmark Evaluation Results: Additional evaluations measure thought-conditioned Talker quality on QA datasets using live hosted Reasoner thoughts and on sampled multi-turn dialogue turns using dataset-derived thoughts.Tables 18 and 19 report entailment, judge, and related metrics with full 95% bootstrap confidence intervals.
  • E Full Benchmark Evaluation Results: A scaling-test table reports only trend directions when Benjamini–Yekutieli-corrected results are significant, separating exploratory all-model trends from primary SmolLM and Gemma contrasts.The table identifies ConvFill, data/reference thoughts, and QA accuracy in its metric scope.
  • F Inference and Evaluation Prompts: The appendix provides exact per-task prompts for live interaction and the LLM-as-a-Judge prompt for single- and multi-turn evaluation.These prompts define the operational conditions underlying the reported evaluations.
  • F.1 LLM-as-a-Judge Prompt: Coverage credits relevant thought facts despite paraphrase or awkward framing, while omissions are judged against available thoughts rather than unanswered parts of the user query.Relation errors and distortions belong under Faithfulness, not Coverage.
  • F.1 LLM-as-a-Judge Prompt: Faithfulness penalizes contradictions, fabrications, unsupported claims, misleading action claims, and meaning-changing framings, while tolerating harmless surface-form variation and process filler accompanying an answer.The rubric scales from fully faithful responses to multiple contradictions or fabrications causing substantively wrong understanding.
  • F.1 LLM-as-a-Judge Prompt: Backend prompts constrain normal, retrieval, and tool-using tasks to concise information-supplier responses, with task-specific context, formatting limits, and tool-use instructions.The prompts include speech-transcribed queries, conversation context, retrieval excerpts, and directions for using provided tools when needed.
Loading 2511.07397v3…