Thinking While Speaking: Inference-Time Knowledge Transfer for Responsive and Intelligent Conversational Voice Agents

1 Introduction

• Voice agents face a tension between iterative reasoning, retrieval, and tool use that improve capability and the millisecond-scale responses required for seamless spoken interaction.The introduction notes that voice conversational models lag behind text-only counterparts (Lin et al., 2025).Source: §1 · p. 1
• Conversational infill lets a lightweight Talker respond immediately while a frontier Reasoner generates knowledge chunks, which the Talker integrates into fluent, contextually grounded conversation.When Reasoner knowledge is unavailable, the Talker generates contextual filler to mask latency and maintain conversational flow.Source: §1 · pp. 1–2, Figure 1 · p. 1
• ConvFill preserves cascaded-system flexibility while masking Reasoner latency; its text-input Talker can be trained without resource-intensive audio modeling, and Talker and Reasoner choices are modular.Unlike full-duplex systems, ConvFill retains a cascaded design while avoiding its usual latency; its components can be selected independently.Source: §1 · p. 2
• Talker models retain millisecond-level time-to-first-response while closing the accuracy gap to within 6.3% of corresponding Reasoner performance.The task is learnable across seven small language models spanning four model families.Source: §1 · p. 2
• In a live evaluation with 18 participants, users rated ConvFill on par with frontier models for interaction quality and significantly better for perceived latency.Talker deployments ran on an Apple M2 SoC.Source: §1 · p. 2
• The authors curate 290,571 validated training examples across six domains and demonstrate conversational infill across seven small language models from four model families.They release the dataset, generation and validation pipeline, training and inference code, and fine-tuned Talker weights.Source: §1 · p. 2

2 Hiding Latency in Conversations

• Simpler latency-hiding approaches use fixed generic infill phrases such as “I’m thinking” or heuristics that select phrases based on context (Gao et al., 2025).Initial responses can acknowledge that the user has been heard, while prolonged silence disrupts conversational flow.Source: §2 · p. 2
• Dynamic infill methods let a higher-capability model correct incorrect information from a lower-latency model, but repeated revisions or inconsistencies can harm trust, confidence, and engagement.These risks are especially consequential in spoken dialogue; ConvFill ensures users interact only with the Talker.Source: §2 · pp. 2–3
• ConvFill's Talker responds while the Reasoner streams knowledge chunks, then ingests them to produce conditioned responses that are passed to text-to-speech.Speech is transcribed and sent concurrently to both models; the Talker maintains a consistent voice across multi-turn conversations by incorporating external knowledge rather than relaying it verbatim.Source: Figure 2 · p. 3, §2 · p. 3

3 Task Formulation

• The Talker hides Reasoner latency with contingent phrases and transforms available Reasoner information into fluent, contingent conversation.The Reasoner uses tools, reasoning, and other high-latency capabilities to address user queries.Source: §3 · p. 3

3.1 Task Setup

• A Talker is a lightweight small language model for consumer-grade hardware, while a Reasoner is a high-performance model that manages multi-turn context and uses tools and external knowledge.A turn is one exchange between the user and Talker.Source: §3.1 · p. 3

3.1.1 Talker Interface

• Within each turn, the Talker generates its full response phrase by phrase, conditioning on conversation history, the user utterance, incoming Reasoner knowledge, and its prior phrases.The history lookback can vary; the Reasoner knowledge available at each phrase position is the stream received up to that position.Source: §3.1.1 · p. 3

3.1.2 Reasoner Interface

• The Reasoner asynchronously streams semantically complete knowledge chunks or silence tokens when no chunk arrives within a predetermined time.Chunks can convey reasoning, retrieval, inference, tool-call results, or other computation; the Reasoner uses the full conversation history and current user utterance as context.Source: §3.1.2 · p. 3

3.1.3 Inference-Time Collaboration

• The Talker and Reasoner run concurrently: the Reasoner streams knowledge asynchronously while the Talker continuously consumes the stream.At each position, the Talker observes the next stream element.Source: §3.1.3 · p. 4
• When a knowledge chunk arrives, the Talker conditions its next phrase on that chunk; when a silence token signals delayed Reasoner output, it responds using the user utterance, history, and prior phrases.The chunk and silence cases are mutually exclusive.Source: §3.1.3 · p. 4
• This collaboration maintains continuous output despite variable Reasoner latency by preventing the Talker from stalling, with one Talker phrase for each Reasoner stream element.The formulation enforces equal lengths for the Talker response sequence and Reasoner knowledge stream.Source: §3.1.3 · p. 4

Generation

• Each turn pairs Reasoner-thought phrases with Talker-response phrases, preceded by 0–3 <sil> placeholders that elicit conversationally contingent filler.Responses incorporate Reasoner meaning in a style consistent with prior conversation and must rely only on information visible to the Talker at inference.Source: §4 · p. 4
• Data combines DSTC8 task dialogues with LLM-generated 7–10-turn conversations seeded by 1,000 premises in each of six domains.DSTC8 retains labeled user utterances and uses system responses as Reasoner thoughts; the second corpus covers general advice, assistant queries, event planning, customer service, education, and medicine.Source: §4 · p. 4

Validation

• A four-stage validator checks turn and phrase-pair structure, temporal alignment, entailment, and proper-noun introductions before accepting conversations.Because Claude Opus 4.6 sometimes referenced future thoughts in earlier responses, conversations were regenerated until they passed; checks use pairwise BERTScore, MNLI-fine-tuned DeBERTaV3, and proper-noun ordering.Source: §4 · p. 4

Composition

• 8,443 conversations yield 290,571 English training examples: 111,552 silence-based fillers and 179,019 knowledge-grounded examples.The JSONL dataset’s end-to-end generation with Claude Opus 4.6 cost approximately $2,400, including validation retries.Source: §4 · pp. 4–5

5.1 Training Pipeline

• Training uses role and turn-boundary tokens plus [KNOWLEDGE] and <sil>; examples include a one-turn history, the latest user utterance, a knowledge chunk, and optionally prior Talker phrases.The training format is shown in Figure 3; [sil] marks cases where Reasoner knowledge is not yet available.Source: §5.1 · p. 5, Figure 3 · p. 5

5.2 Inference Pipeline

• ConvFill sends each transcribed utterance to the Talker and Reasoner in parallel, queues knowledge and TTS responses, and dynamically supplies <sil> when the TTS queue is empty.If streamed knowledge is empty while TTS responses are queued, the Talker waits for another knowledge chunk; <sil> is inserted most often at turn start while awaiting Reasoner output.Source: §5.2 · p. 5

5.3 Talker and Reasoner Models

• The study evaluates 21 ConvFill configurations combining seven Talkers from 135M to 1.7B parameters with three Reasoners; Talkers use ℓ=1 history and INT8 MLX on an Apple M2.Talkers include Gemma 3 270M and 1B (Team et al., 2025), Qwen3 0.6B (Yang et al., 2025), SmolLM2 135M, 360M, and 1.7B (Allal et al., 2025), and Llama 3.2 1B (Grattafiori et al., 2024). Reasoners are Claude Opus 4.7, GPT-5.5 (OpenAI, 2026), and Gemini 3.1 Pro (Google DeepMind, 2026).Source: §5.3 · p. 5

6 Benchmark Evaluation

• The evaluation compares single-turn, multi-turn, and live-interaction settings because accuracy, faithfulness, coverage, and user-intent alignment capture distinct failure modes in the dual-model system.This section presents benchmark evaluations; live interaction results are presented in Section 7.Source: §6 · pp. 5–6

6.1.1 Benchmark Metrics

• Phrase-level Entailment measures whether knowledge-conditioned Talker phrases are supported by corresponding Reasoner phrases, while Non-Contradiction checks whether <sil>-conditioned phrases contradict any Reasoner phrase.Both metrics are normalized by the number of relevant phrases per turn.Source: §6.1.1 · p. 6
• Turn-level judge metrics separately assess coverage of Reasoner knowledge, faithfulness to it, and helpfulness in addressing user intent.Coverage captures omissions, Faithfulness captures distortions or unsupported claims, and binary Helpfulness is based on a Likert judge score threshold of ≥3.Source: §6.1.1 · pp. 6–7

6.1.2 Evaluation Conditions

• Evaluation prepends a random 0–3 <sil> tokens before knowledge chunks to simulate Reasoner latency, matching or exceeding the count produced in real deployment.Inference in this section uses text inputs only.Source: §6.1.2 · p. 7

6.1.3 Single-Turn Evaluations

• Single-turn QA accuracy compares Base SLM (no conversational infill), ConvFill Talker, ConvFill Reasoner, and Frontier Model (no conversational infill); NLI and LLM-as-a-Judge metrics use ConvFill Talker outputs only.LLAMA1 models were evaluated on the full 300-question test set with the same prompting and judge model as SimpleQA.Source: §6.1 · p. 6, §6.1.3 · p. 7

6.2 Results

• Across SimpleQA and LLAMA1, ConvFill Talker models perform within 0%–6.3% of their corresponding Reasoner models.Reasoner accuracy represents a ceiling for the corresponding Talker because Reasoner knowledge chunks enable Talker performance gains.Source: Figure 4 · p. 8, §6.2 · p. 8

6.1.4 Multi-Turn Evaluations

• For multi-turn evaluation, MultiWOZ tests longer task-oriented dialogues and Everyday Conversations tests simpler 3–4-turn continuity; grading covers 991 and 119 turns, respectively.The Talker responds sequentially using user input, simulated Reasoner knowledge, and its previous response, and only Talker outputs are evaluated.Source: §6.1.4 · p. 8

• Reported gains reached as high as 51% on LLAMA1 and 63.4% on SimpleQA relative to corresponding base SLMs.Source: LLAMA1 SimpleQA · p. 9
• The standalone Reasoner is roughly 2% from Frontier-model-only accuracy overall, except on SimpleQA with Gemini 3.1 Pro, where the gap is almost triple.On LLAMA1, standalone Reasoner accuracy is 2.3% lower for Claude, 0.7% higher for Gemini, and 1.3% higher for GPT-5.5; on SimpleQA, it is 2.0%, 7.4%, and 1.4% lower, respectively.Source: LLAMA1 SimpleQA · p. 9
• Base SLM Accuracy correlates significantly positively with parameter count on both QA datasets, indicating stronger underlying question-answering capability in larger models.Source: LLAMA1 SimpleQA · p. 9
• On SimpleQA, Talker size has significant positive trends across Accuracy, Entailment, Coverage, Faithfulness, and Helpfulness; on LLAMA1, only Faithfulness and Helpfulness show positive trends.LLAMA1 Accuracy, Entailment, and Coverage are uniformly high across model sizes, leaving little room for scaling trends.Source: LLAMA1 SimpleQA · p. 9
• MultiWOZ shows significant positive scaling trends for Coverage, Faithfulness, and Helpfulness, while Everyday Conversations shows no significant scaling trends.The passage attributes the contrast likely to dataset complexity, with scaling more visible when model capability limits performance.Source: LLAMA1 SimpleQA · p. 9
• Non-Contradiction is an exception to positive Talker-size scaling: it has no clear overall relationship on QA or Everyday Conversations and correlates negatively on MultiWOZ.Within-family QA results reverse the cross-family trend for SmolLM2-derived models, which show significant positive correlations, while larger Gemma 3 models show worse Non-Contradiction.Source: LLAMA1 SimpleQA · pp. 9–10

7.1 Methods

• The interactive study involved 18 participants who completed Direct, RAG, and MCP tasks in Latin Square order; system configurations were randomized, and Base SLM was omitted from MCP because it could not make structured MCP calls.Source: §7 · p. 10, §7.1 · p. 10
• Participants graded conversational infills for Coverage and Faithfulness on 1–5 scales and Helpfulness as a binary judgment, creating data for correlation with LLM-as-a-Judge ratings.They were given a user utterance, external knowledge chunks, and candidate conversational responses.Source: §7.1 · p. 11

7.2 Results

• ConvFill was rated equivalent to Frontier on six metrics, exceeded it on Latency, and fell below it on Naturalness; Base SLM scored significantly below both across all metrics.The six equivalent metrics were Clarity, Fluency, Response Length, Coherence, Task Completion, and Satisfaction.Source: §7.2 · p. 11, Table 3 · p. 11
• Rank-1 tests found no ConvFill–Frontier preference on Direct or MCP, but strongly preferred ConvFill for RAG; 15/18 users ranked Base SLM last on Direct and RAG.The study pooled 54 ranking observations per system except Base SLM, which had 36 because it could not make MCP calls.Source: Table 4 · p. 11
• 542 ms vs. 617 ms (Direct) and 976 ms vs. 3,812 ms (RAG): ConvFill Talker TTFR matched Base SLM on Direct and was lower on RAG.ConvFill Reasoner mean TTFR ranged from 2,947 ms on Direct to 7,242 ms on MCP, while Talker responses stayed fast across tasks.Source: Table 4 · p. 11, Figure 5 · p. 12, §7.2 · p. 12

Dynamic Fillers for Perceived Responsiveness

• ConvFill delivers millisecond-level TTFR across three task types despite Reasoner latency, approaches frontier QA accuracy, and adapts filler volume without task-specific TTFR tuning.The Talker’s TTFR is equal to or lower than that of corresponding base SLMs; phrase-based, condensed Reasoner knowledge also reduces the Talker’s token-processing burden compared with independently processing full retrieved context.Source: §8 · p. 12

Accuracy and Frontier Model Parity

• 9.8% average SimpleQA performance gap: Talker performance trails the Gemini 3.1 Pro Frontier Model baseline because of Reasoner-format prompting sensitivity, not Talker integration.The evaluation uses a single generic prompt structure across Reasoner models, providing a conservative estimate; application-specific prompt adaptation could further test robustness and improve performance.Source: §8 · pp. 12–13

Task Difficulty and Metric Saturation

• Grounding metrics remain strong, but Faithfulness and Helpfulness vary more with model and task difficulty, with MultiWOZ scoring lower than QA and Everyday Conversations.Accuracy, Coverage, Faithfulness, and Helpfulness scale positively with model size on unsaturated tasks; Entailment scales on single-turn QA but not multi-turn benchmarks, where the NLI checker struggles with nuance. Easy-task metric saturation can obscure scaling, and Non-Contradiction trends differ between SmolLM and Gemma families.Source: §8 · p. 13

Live Interaction Takeaways

• In live unscripted interactions, ConvFill and Frontier significantly exceed Base SLM, while ConvFill is statistically equivalent to Frontier except on Latency and Naturalness.Users rank ConvFill on par with Frontier for Direct and MCP and prefer it for RAG; MCP averages 2.35 infill phrases per turn before the Reasoner response, versus 1.34 for RAG.Source: §8 · p. 13

Latency-Adaptive Behavior

• Infill may be most beneficial during medium-length delays; adapting its amount to expected latency could preserve responsiveness while avoiding naturalness costs from excessive filler.The discussion proposes waiting briefly when a fast Reasoner response is expected and using Talker-generated pauses for longer delays instead of continuous filler.Source: §8 · pp. 13–14

Development Feasibility

• 2.8–48.9 GPU hours per Talker and $133.70 total for seven Talkers show that fine-tuning requires no large-scale training infrastructure; dataset generation and verification cost $2,400.Deployment evaluation used INT8-quantized Talkers on an Apple M2 laptop with 16 GB memory; model graphs ranged from 136 MB to 1.69 GB.Source: §8 · p. 14

Decoupling Model Behavior and Capability

• Separating conversational behavior from reasoning lets Talkers be independently trained and customized, while Reasoners can be swapped without retraining Talkers or changing their prompts.A consistent customized Talker could preserve user experience across Reasoner upgrades, while the modular design can use existing tools and data sources integrated with frontier models.Source: §8 · p. 14

• Mini-Omni-Reasoner (Xie et al., 2025) uses token-level thinking-in-speaking; SplitReason (Akhauri et al., 2025) offloads reasoning; Qwen2.5-Omni (Xu et al., 2025) feeds Thinker representations to its Talker.Source: §9 · p. 14
• Speculative decoding verifies or refines faster generation with a larger model, and Li and Grover (2025) extend this approach to cascaded speech models.Source: §9 · p. 14
• Chen et al. retrieve from text-audio hybrid knowledge bases; Arora et al. predict tool calls during utterances; Kuroki et al. refine responses with oracle LLMs; Chien et al. pair RAG backends with full-duplex voice.Chien et al. handle retrieval delays of up to two seconds.Source: §9 · pp. 14–15

10 Conclusion

• Conversational infill is a novel task that enables SLM Talkers to respond promptly with context-aware fillers while incorporating frontier-class Reasoner knowledge at inference time.The conclusion describes this as a Talker-Reasoner approach.Source: §10 · p. 15
• ConvFill consistently achieves millisecond-level time-to-first-response across tasks with variable Reasoner latency and models while maintaining external-knowledge grounding and task accuracy.Source: §10 · p. 15
• The authors position ConvFill at a new latency–capability Pareto-frontier point (Lin et al., 2025) and release model weights, dataset, and code for training and inference.They describe conversational infill as a path toward task-oriented and tool-use conversational AI systems.Source: §10 · p. 15

Limitations

• The independent Talker may generate fillers before Reasoner responses are available and may not immediately comply with safety constraints, motivating tighter safety coupling.Most safety concerns in the two-stage architecture are mitigated by a guardrailed Reasoner.Source: Limitations · p. 15
• The evaluation did not capture Reasoner timeouts or excessive latency, when prolonged filler sequences could become disruptive; future work should examine pauses and fallback strategies.Source: Limitations · p. 15
• A larger user evaluation sample could improve the statistical robustness and generalizability of the findings.Source: Limitations · p. 15
• Future work could evaluate whether ConvFill applies to languages other than English.Source: Limitations · p. 15

Ethical Considerations

• Participants were informed and compensated; the IRB approved the study, and the synthetic dataset was screened for personally identifiable information and inappropriate content.The study involved no vulnerable populations and posed no foreseeable risk.Source: Ethical Considerations · p. 15

A.1 AI Use Statement

• LLM assistants supported programming, formatting, and proofreading; the authors state that scientific content, experimental design, and conclusions are theirs.Source: §A.1 · p. 20

B Dataset Generation

• The dataset combines topic-seeded freeform conversations with DSTC8 dialogues, using a shared scaffold-fill-validate-retry pipeline and parallel thoughts and response arrays.Both tracks use structural, NLI, and BERTScore validators; the scaffolded track adds source-preservation and inference-time visibility checks.Source: B Dataset Generation · p. 21

B.1 Topic-Seeded Freeform Generation

• Freeform generation covers six conversational domains, with per-topic scaffolds specifying 7–10 turns, 0–3 initial silence slots, and 1–5 substantive lines per turn.The response instructions require contextual infills before substantive information, paired with concise factual thoughts.Source: §B.1 · pp. 21–22

B.2 DSTC8 Schema-Guided Scaffolded Generation

• The DSTC8 track re-skins task-oriented dialogues while preserving user utterances and system content, and constrains infills to context available at deployment.The generator sees the full dialogue, so prompts specify an immediate visibility window and validation checks for leaked proper nouns; source content is required verbatim.Source: §B.2 · pp. 23–24

B.3 Validation Cascade

• The four-stage validation cascade checks structure, thought-response contradiction, positional alignment, and—on DSTC8—source preservation and proper-noun visibility.NLI rejects pairwise contradiction probabilities above 0.20 or whole-turn probabilities above 0.30; BERTScore enforces paired alignment, while DSTC8 adds source-thought and visibility checks.Source: §B.3 · pp. 25–26

B.4 Generation Settings

• Table 6 reports the hyperparameters needed to reproduce the dataset, and per-domain configuration JSON files are slated for release with it.Source: §B.4 · p. 26

B.5 Output Format and Yield

• The released dataset contains 6,005 freeform conversations (50,324 turns) and 2,438 DSTC8-scaffolded conversations (24,184 turns), totaling 8,443 conversations and 74,508 turns.Conversations are stored as JSONL objects; scaffolded conversations include metadata recording the DSTC8 service and dialogue ID.Source: §B.5 · p. 26, Table 7 · p. 27

C.1 Base SLM Models; C.2 Frontier Models

• Seven base SLMs were fine-tuned for conversational infill, while API-based models were used for development, inference, and testing with specific tagged versions.“IT” denotes a base model that was originally instruction-tuned.Source: Table 8 · p. 27, Table 9 · p. 27

C.3 Control Sequence Expansions for Different Talker Models

• The training and inference examples use existing model control tokens for user, assistant, and knowledge spans; [KNOWLEDGE] is a new role mapped through each model’s existing control-token template.Source: §C.3 · p. 27, Table 10 · p. 28

C.4 Shared Training Parameters for Talker Models

• Talker models used a 90/10 train/evaluation split and were trained on four NVIDIA RTX 6000 or four NVIDIA A40 GPUs; Table 11 documents training configurations and GPU hours.Source: §C.4 · p. 28

D.1 Ethical Considerations

• The study received institutional review board approval; participants were told participation was voluntary and their data would appear in an academic publication, and they were compensated.Interaction and prompt-grading sessions lasted approximately 45 minutes.Source: §D.1 · p. 29

D.2 Study Instructions

• Participants tried three systems across three task blocks, conversed freely for several turns, then surveyed and ranked the systems; ratings covered latency, clarity, fluency, response length, coherence, task completion, naturalness, and satisfaction.Participants were encouraged to assess information access, responses, helpfulness, conversational flow, and likes or dislikes.Source: §D.2 · p. 29

D.3 Study Tasks

• Assigned study tasks covered three categories: Direct, RAG, and MCP.Source: §D.3 · p. 29

D.3.1 Direct Tasks

• Direct tasks covered dialogues that could proceed without tools or external databases, with participants choosing among three scenarios and using Base SLM, Frontier, and ConvFill.The examples included birthday-party planning, train travel, and finding a Greek-yogurt cake recipe.Source: §D.3.1 · pp. 29–30

D.3.2 RAG Task

• The RAG task connected all three configurations to an institutional-document database using a retrieve-then-rerank pipeline, passing the top three reranked chunks to the language model.The index contained 367 embedded chunks; the system retrieved 30 candidates and reranked them before selecting the top three for grounded response generation.Source: §D.3.2 · pp. 30–31

D.3.3 MCP Task; D.4 Participant Condition Assignment

• MCP connected models to an IMAP email server and included Frontier and ConvFill only because Base SLM could not properly format MCP calls; 18 participants completed counterbalanced conditions.The three task conditions followed a Latin square so each appeared six times in each ordinal position; model configurations were also counterbalanced.Source: §D.3.3 · p. 31, §D.4 · p. 31

D.5 Live Interaction Setup; D.5.1 Live Interaction Example

• Participants interacted by voice through a press-and-hold web system, hearing spoken responses while viewing their transcribed text but not the model responses.The study used faster-whisper transcription and macOS say synthesis; Base SLM and ConvFill Talker ran with INT8 quantization through MLX on an Apple M2 SoC.Source: §D.5 · pp. 31–32, §D.5.1 · p. 32, Figure 6 · p. 32

D.6 Infill Grading

• Krippendorff’s α was 0.672 for Coverage, 0.67 for Faithfulness, and 0.751 for Helpfulness, indicating strong inter-rater reliability.Participants graded 108 samples, each reviewed by two participants, yielding 216 grading examples; 27 samples came from each of four datasets.Source: §D.6 · p. 33

D.7 TOST Test Details for Likert-Scale Ratings

• Table 13 reports Frontier-versus-ConvFill TOST results aggregated across tasks, with N = 54 per system and equivalence declared when both one-sided p-values are below .05.The test used δ = 0.5 and α = .05; the table defines separate lower- and upper-bound null hypotheses.Source: Table 13 · p. 34

D.8 Test Details for User Rankings across Tasks

• On Direct, Frontier received 10 vs. ConvFill’s 8 rank-1 votes (p = .740); on RAG, ConvFill received 12 vs. Frontier’s 5 (p = .044); on MCP, ConvFill received 11 vs. Frontier’s 7 (p = .481).MCP used a two-sided binomial test because neither system received any rank-3 votes; the RAG preference for ConvFill was significant.Source: §D.8 · p. 34

D.9 ConvFill Speedup, TTFR, & Filler Details

• The speedup and latency tables break results down by task and mode, while the filler table reports per-task <sil> tokens per turn.The speedup caption defines speedup as ConvFill Reasoner latency divided by ConvFill Talker latency; values above 1 indicate the filler arrived sooner than the Reasoner response would have.Source: Table 14 · p. 35, Table 15 · p. 35, Table 16 · p. 35

E Full Benchmark Evaluation Results

• SimpleQA (N = 1000) and LLAMA1 (N = 300) are evaluated with accuracy and 95% Wilson intervals, including conditional accuracy and comparisons to frontier, reasoner, and base-SLM answers.Conditional accuracy is restricted to questions where the corresponding Reasoner bullet output was judged correct; the caption supplies no accuracy values.Source: Table 17 · p. 36
• Thought-conditioned quality is evaluated on QA with live hosted Reasoner thoughts and on sampled multi-turn turns with dataset-derived thoughts, using entailment and judge metrics with bootstrap intervals.For multi-turn dialogue, one non-edge turn is graded per dialogue after earlier turns are generated as hidden warm-up to preserve ConvFill self-history.Source: Table 18 · p. 36, Table 19 · p. 37
• The scaling comparison uses conservative Benjamini–Yekutieli FDR correction, showing significant trend directions only; all-model trends are exploratory, SmolLM is the primary within-family trend, and Gemma is a two-point contrast.The caption identifies comparison dimensions but reports no specific significant trends.Source: Table 20 · p. 38

F Inference and Evaluation Prompts

• The appendix includes exact per-task prompts used in live interaction and an LLM-as-a-Judge prompt for single- and multi-turn evaluation.The judge evaluates one ConvFill turn on a 1–5 scale and is instructed to use streamed thoughts as the only factual source.Source: F Inference and Evaluation Prompts · p. 39, §F.1 · p. 39

F.1 LLM-as-a-Judge Prompt

• The judge rubric separates coverage of relevant thought content, faithfulness to those thoughts, and whether the response appropriately addresses the user's query.Coverage concerns omission of available information, faithfulness concerns unsupported or distorted meaning, and helpfulness concerns alignment with the query; the prompt asks for a score and behavior-specific rationale.Source: §F.1 · pp. 39–41
• The prompt limits replies to three sentences and one question at most, spells out numbers for dollar amounts and large numbers, and requires verbatim material to be quoted.Source: §F.1 · p. 44
• For information lookup, the prompt advises using likely keywords rather than long verbatim search phrases.Source: §F.1 · p. 44
• The example conversation discusses flavored matcha drinks, suggesting vanilla syrup or homemade strawberry puree as options.Source: §F.1 · p. 44