PaperlayerView exact revision on arXiv

Source-linked AI summary

Never Stop Thinking: Continuous-Time Language Agents

Bojie Li, Noah Shi

arXiv:2609.17416v1cs.AI

TL;DR

LLM voice agents use a rigid listen-think-speak loop, but evidence is limited on whether interleaved thinking and speaking improves interactive task completion under interruptions. The paper combines an interrupt-and-resume orchestrator for unmodified text LLMs with ReactiveBench and a five-stage study of verifiable training signals, reward structure, and optimizers. On a live voice pipeline, continuous-time cognition cuts mean latency by 19% overall and 49% on early-entity tool tasks.

  • Problem

    LLM voice agents use a rigid listen-think-speak loop, but evidence is limited on whether interleaved thinking and speaking improves interactive task completion under interruptions.

  • Method

    The paper combines an interrupt-and-resume orchestrator for unmodified text LLMs with ReactiveBench and a five-stage study of verifiable training signals, reward structure, and optimizers.

  • Results

    On a live voice pipeline, continuous-time cognition cuts mean latency by 19% overall and 49% on early-entity tool tasks.

  • Takeaways & Limitations

    Continuous-time interaction is enabled by orchestration, but its content-quality benefit appears only with verifiable objectives shaped to the task and optimized on-policy.

  • Takeaways & Limitations

    The benchmark’s pre-registered checklist items are authored from task specifications and frozen before scoring, rather than filed with an external registry.

Abstract

from arXiv · show

Voice agents built on LLMs follow a rigid listen-think-speak loop that inserts seconds of dead air before every reply. We show that continuous-time cognition (thinking while listening and thinking while speaking) emerges from an unmodified text model under a lightweight interrupt-and-resume orchestrator, cutting live-pipeline latency by 19% overall and by half in the regime the mechanism targets. To measure whether continuous-time thinking improves what agents accomplish, we introduce ReactiveBench: 120 interactive scenarios scored against pre-registered binary requirements, plus a verifiable streaming track scored by exact correctness. ReactiveBench exposes a pitfall with broad consequences: LLM judges reward visible reasoning; a large judged "advantage" of continuous-time thinking reverses sign under an independent judge, and judge-trained models objectively complete fewer requirements when they think. A five-stage training study then locates the right signal at three levels. Its source: verifiable objectives turn thinking from harmful to helpful. Its structure: whatever a uniform reward omits, optimization trades away; brevity everywhere erodes multi-hop tool chaining. Its optimizer: preference optimization can only trade conflicting sub-goals against each other, while on-policy RL over a type-shaped reward improves every correctness axis at once, raising streaming completion from 48% to 73+/-5% across seeds and replicating at larger scale and on a second model. Orchestration makes continuous-time interaction possible; a verifiable signal, correctly sourced, shaped, and optimized, makes it good.

1 Introduction

The paper introduces continuous-time language agents that use unmodified text LLMs to think during listening and speaking, then evaluates their latency, quality, and training requirements with objective benchmarks.

  • C1: Continuous-time cognition on unmodified text LLMs: Continuous-time agents maintain one persistent reasoning stream while injecting partial transcripts, playback deadlines, tool results, and barge-ins at interrupts.The design uses a lightweight interrupt-and-resume primitive without changing model weights, tokenizer, or chat template.
  • C1: Continuous-time cognition on unmodified text LLMs: 19% overall live-pipeline latency reduction and roughly half reduction in the targeted regime establish the main practical benefit of continuous-time cognition.A dual-process fast/slow policy separately cuts time-to-first-response by about two-thirds across model sizes.
  • C2: ReactiveBench: ReactiveBench evaluates interactive completion with 120 scenarios and objective binary requirements, alongside a verifiable streaming track scored by exact correctness.The benchmark addresses the absence of evaluation for interleaved thinking and speaking under interrupts.
  • C2: ReactiveBench: A judged continuous-time quality advantage reverses under an independent judge and largely disappears when reasoning is hidden, exposing visible-reasoning bias.This motivates treating LLM-judge-only claims about interactive agents with suspicion.
  • C3: The signal’s source decides whether thinking helps: Judge-trained models complete fewer verifiable requirements when thinking, whereas verifiable objectives eliminate the deficit and can turn continuous-time thinking into a measured gain.Coverage-preserving training retains efficient thinking while protecting spoken-output coverage.
  • C4: The signal’s structure and optimizer decide what survives training: Uniform brevity rewards erode omitted skills such as multi-hop tool chaining, while on-policy RL with type-conditional verifiable rewards improves every correctness axis simultaneously.The optimizer determines whether conflicting sub-goals are traded off or improved together.

2 Continuous-Time Cognition on an Unmodified LLM

The system turns an unmodified text model into a continuous-time agent by interrupting and resuming one reasoning stream as observations arrive. Dual-process reasoning provides broad latency gains, while think-while-listening is strongest when speech or playback provides enough compute time.

  • 2.1 One primitive, four observation sources: A single interrupt-and-resume mechanism handles ASR partials, TTS timeouts, asynchronous tool results, and user barge-ins in one persistent stream.It closes the current thinking block, injects the observation as a chat turn, and resumes with prior reasoning preserved.
  • 2.1 One primitive, four observation sources: The agent emits one persistent stream of <think>/<say>/<tool_call> tokens and resumes inside thinking after each interrupt.Figure 2 shows observations on the left and agent outputs on the right, with the stream continuing across interrupts.
  • 2.1 One primitive, four observation sources: When a partial names Denver, the agent calls weather(Denver) and speaks the grounded forecast before the user finishes speaking.This early-entity anticipation is the behavior measured as tool-lead.
  • 2.2 What the mechanism buys: latency: 49% early-entity, 15% late-intent, and 19% pooled mean latency reductions demonstrate regime-dependent gains in the live voice pipeline.The live pipeline uses streaming Whisper ASR, Qwen3-8B, and streaming TTS over 20 episodes.
  • 2.2 What the mechanism buys: latency: Dual-process reasoning cuts time-to-first-response by about two-thirds across models, while think-while-listening hides thinking only when it fits within the speech window.The window suffices for 88% of Qwen3-1.7B problems and 52% of Qwen3-4B problems in the scaled study.
  • 2.2 What the mechanism buys: latency: ReactiveBench contains 120 interactive scenarios and 200 exact-correctness streaming tasks, with continuous and baseline modes and three interrupt sources.It also measures tool-lead and speech coverage as behavioral metrics.

3 ReactiveBench

ReactiveBench evaluates continuous-time interaction with within-scenario baseline contrasts, objective checklists, and exact-correctness streaming tasks. It also shows that the apparent quality advantage under an LLM judge reverses under independent or reasoning-hidden evaluation.

  • 3 ReactiveBench: ReactiveBench fills a benchmark gap with interactive and verifiable streaming tracks that compare continuous and baseline modes within model and scenario.Track A uses 120 scripted scenarios; Track B uses 200 exact-correctness tasks and behavioral metrics.
  • 3 ReactiveBench: Track A scores 688 pre-registered binary requirements, excluding tone, style, and fluency from its primary objective metric.Requirements are authored from scenario specifications before model outputs are scored.
  • 3 ReactiveBench: Track B tests hard correctness across reasoning, single- and multi-hop tool QA, calculator, and early-entity tool tasks under streaming input.Its exact-correctness design complements Track A’s open-ended checklist coverage.
  • 3.3 Why the primary metric is not an LLM judge: Up to 8.4% judged gains for continuous mode reverse negative for all seven models under an independent judge, with rankings essentially uncorrelated at Spearman ρ=0.07.Hiding <think> blocks from the primary judge removes most of the advantage for six of seven models.

4 The Training Signal Decides

The training study progressively replaces judge-based supervision with verifiable objectives, then examines reward structure and optimization. It finds that efficient interleaved thinking alone is insufficient when training suppresses spoken content or omits task-specific behaviors.

  • 4.1 Signal source: the judge vs. the spoken checklist: Judge-reward training produces roughly 4 short interleaved think blocks instead of 2 long blocks, using 5–6× fewer think tokens per response.This efficient style persists across three SFT and three RL variants, but does not establish better task completion.
  • 4.1 Signal source: the judge vs. the spoken checklist: Judge-based optimization improves the judged style while training models speak less and complete fewer verifiable requirements in continuous mode.The failure arises because thinking displaces content from speech, which the judge does not penalize.
  • 4.1 Signal source: the judge vs. the spoken checklist: Coverage-preserving SFT instead rewards important points appearing in spoken output, eliminating the continuous-mode deficit while retaining concise thinking.RLVR then optimizes the objective checklist score on think-stripped speech.
  • 4.2 Dense verifiable signals: distill, then reject: On-policy distillation grades student-generated streaming trajectories against a verifiable track, avoiding exposure bias that caused naive trace cloning to regress the base model.Dense token-wise supervision still misses trajectory-level habits such as spoken meta-reasoning, wrong trajectories, and talking over users.
  • 4.3 Signal structure and optimizer: Uniform verifiable gates can still erode multi-hop tool chaining and allow unpriced anticipation timing to drift, because optimization trades away omitted behavior.Preference optimization cannot express “thorough here, brief there,” whereas direct on-policy optimization can target a type-conditional reward.

5 Results

Objective evaluation shows that judge-trained thinking can reduce spoken content, while coverage-preserving and verifiable training recover or improve correctness. The strongest results come from task-shaped rewards optimized on-policy, with robustness across seeds, scale, models, and interrupt settings.

  • 5.1 The judge-trained models, measured objectively: Judge-trained models complete fewer objective requirements while thinking, whereas Coverage-SFT removes the deficit and attains the highest completion of any tested model.Coverage-SFT combines brief thinking with comprehensive speech; its continuous-minus-baseline difference is statistically null at −0.004.
  • 5.3 The verifiable-signal lineage on Track B: 73±5% streaming completion is reached by Shaped-RL across seeds, rising monotonically from base through OPD and rejection fine-tuning on matched Track B evaluation.Multi-hop QA dips under rejection concision gates and partially recovers when the shaped reward restores thoroughness.
  • 5.4 Composing conflicting objectives: preference vs. shaped reward: DPO improves preference-aligned axes but sacrifices multi-hop QA, while on-policy GRPO over type-conditional verifiable rewards improves all four axes simultaneously.The comparison starts from the same rejection-fine-tuned checkpoint and replicates across three seeds and on Qwen3-4B.
  • 5.3 The verifiable-signal lineage on Track B: The released model preserves 100% speech coverage, fires tools during its own speech in 80% of episodes, and anticipates early-entity tools with a 3 s median lead in 61% of episodes.Repricing anticipation recovered performance from 21% to 61% at unchanged overall streaming correctness.
  • 5.5 Generality and robustness: Think-while-listening appears across thirteen architecture families in streaming mode but exactly 0% of turn-based tasks, while correctness effects vary by model and baseline speech coverage.Streaming helps under-speaking models and can hurt strong instruction-followers acting on partial input; turn-based speech coverage predicts the delta at r=−0.57.
  • 5.5 Generality and robustness: No evaluated chat-turn-boundary interruption caused reasoning leakage, panic answering, or self-doubt, separating interrupt robustness from spoken meta-reasoning learned during training.The latter is attributed to imperfect training trajectories rather than interrupt events.
  • 5.5 Generality and robustness: The paper rests its main claims on latency measurements, think-token efficiency, objective completion with verifier agreement, and replicated verifiable-lineage gains, not judged rankings.Judged rankings among trained variants remain within confidence intervals.

6 Related Work

Prior work either trains bespoke full-duplex or multimodal architectures, or studies incremental text reasoning without evaluating interleaved interaction under interrupts. This paper complements those efforts with an architecture-agnostic orchestrator and a benchmark for real-time thinking-and-speaking.

  • Native full-duplex and omni-modal models: Native full-duplex systems achieve token-level latency and acoustic grounding, but require bespoke trained architectures.The proposed text-channel approach is complementary and could also wrap a native model.
  • Streaming and concurrent reasoning in text: This work shows that interleaved reasoning, barge-in handling, TTS-paced cycling, and asynchronous tools can be added to an unmodified text model through orchestration.The orchestrator supplies behavioral capabilities that related native architectures obtain through specialized training.
  • Streaming and concurrent reasoning in text: Streaming text systems already reason over partial input or parallel processes, but this paper adds objective interrupt-focused evaluation and training methodology.The related line includes incremental reasoning, dual-thread reasoning, and learned fast–slow model coupling.
  • Benchmarks: ReactiveBench addresses a gap left by dialogue, tool-use, and audio benchmarks: evaluating interleaved thinking and speaking under interrupts.Existing benchmarks cover task-oriented dialogue, LLM-judged multi-turn quality, tool-use agency, or audio understanding, but not this combined setting.

7 Limitations

The evaluation is limited by simulated transcript-level interrupts, small and trace-composed latency measurements, incomplete interactive-quality metrics, and constrained training scale. The system also inherits text-pipeline realism limits and does not claim parity with native audio models.

  • System and realism: The text-based system inherits ASR/TTS latency, cannot use prosody or emotion, and does not claim native models’ token-level latency or speech naturalness.Its latency comparisons are within-system rather than cross-system.
  • System and realism: Track A simulates interrupts at the transcript level, isolating interrupt-content handling rather than real-time behavior under load.Live-pipeline measurements cover latency but not full interactive quality, use n=20 episodes, and compose conditions from per-episode timings rather than independent closed-loop runs.
  • Evaluation: Judged-quality results share model-family components, use one generation per scenario, and objective metrics omit timing grace and conversational repair.A blinded human A/B harness is released, but running it at scale remains future work.
  • Training: The RL study uses LoRA scale on one GPU with modest rollout budgets, hand-set reward weights, and a deployment regime specialized to streaming.The omitted-anticipation episode illustrates how deployment-critical behavior can be absent from the reward.

8 Conclusion

The paper argues that continuous-time cognition can be orchestrated on existing thinking models, but its content benefits depend on objective measurement and carefully designed training signals. The conclusion identifies the reward source, structure, and optimizer as the decisive factors.

  • 8 Conclusion: A two-hundred-line interrupt-and-resume orchestrator elicits continuous-time cognition from off-the-shelf thinking models without new architectures.It enables thinking while listening and speaking and substantially reduces conversational latency.
  • 8 Conclusion: ReactiveBench uses pre-registered verifiable criteria and shows that LLM-judge scores can reverse under judge changes and reasoning visibility.The benchmark is intended to objectively ground interactive-agent claims.
  • 8 Conclusion: Verifiable objectives make thinking beneficial, whereas uniform brevity can erode multi-hop tool chaining and omitted reward dimensions are traded away.The conclusion locates the right signal at its source and structure, not merely in the presence of reasoning.
  • 8 Conclusion: In a 10-problem pilot, dual-process reduced TTFR by approximately 67%, while TWL was regime-dependent and speculative reasoning affected model sizes differently.The pilot’s accuracy effects are directional because one task represents 10 percentage points; scaled studies and live-pipeline measurements carry the main quantitative claims.

B Live-Pipeline Latency Measurement

The live-pipeline study measures latency with real ASR, model, and TTS components, while pilot tables characterize when different inference-time policies help. Dual-process is consistently fastest; TWL helps only when inference speed lets hidden thinking fit within the speech window.

  • Inference-time policies: ≈67% TTFR reduction is the consistent dual-process result across both models, while TWL increases CPU TTFR by 10–22% because of pre-thinking overhead.The TWL penalty scales inversely with inference speed and is reported in the timing tables.
  • Inference-time policies: TWS self-correction depended on model and hardware: the 1.7B model improved from 60% to 90%, while the 4B result reversed from CPU failure to 20%→80% on GPU.The pilot also reports model-size differences in speculative reasoning and calibration.
  • B Live-Pipeline Latency Measurement: The latency comparison is trace-composed rather than based on two independent interactive runs, using measured speech overlap and shared generation traces.This removes within-episode generation variance but makes continuous latency depend on the stated overlap model.
  • B Live-Pipeline Latency Measurement: 19% mean latency reduction was measured across 20 live-pipeline episodes, with 49% on early-entity tasks and 15% on calculator tasks.The corresponding median reductions were 27% pooled, 49% early-entity, and 13% calculator.
  • B Live-Pipeline Latency Measurement: Six of twelve calculator episodes hit the 3,072-token cap before closing the think block, making turn-based latency a lower bound; ASR compute was excluded from both conditions.These caveats affect interpretation of the reported latency reductions.
  • Inference-time policies: TWL becomes beneficial when thinking fits within the remaining speech window; on GPU it reduced 8B TTFR by 21% (−226 ms).The GPU table also reports accuracy drops to 60% for 8B and 0% for 4B under partial-input confusion.

D Judged-Quality Results and Robustness Checks

LLM judging makes continuous-time thinking appear broadly beneficial, but independent and think-hidden checks reverse or largely remove that advantage.

  • The independent judge yields a negative continuous-mode advantage for all seven models, with rankings essentially uncorrelated with the primary judge.The reported Spearman correlation between judges is ρ=0.07.
  • The primary judge rates every model higher in continuous mode and five of seven gains significant, but these results are visibility- and judge-dependent.The primary judge shares a model family with the SFT teacher, while the RL reward model belongs to a third family.
  • Hiding <think> blocks from the primary judge reproduces intact-transcript scores but removes most of continuous thinking’s apparent advantage.

E Objective Task Completion: Details and Diagnostics

Objective checklist evaluation shows that trained models often lose requirements when thinking continuously because thinking displaces speech, while preserving coverage removes the deficit.

  • Every judge-trained model completes fewer objective requirements in continuous mode than baseline, significantly so for three models.The untrained base model instead gains slightly, by +0.04.
  • Speech displacement explains the deficit: trained models speak 6–18% less continuously, whereas the base model speaks 24% more and gains objective completion.Within continuous mode, spoken length correlates positively with completion at r=0.14–0.33 across models.
  • A best-of-3 base-model probe raises continuous completion from 0.731 to 0.791, a significant +0.060 over baseline when speech is not suppressed.
  • Coverage-preserving SFT eliminates the deficit at −0.004 and reaches the highest continuous completion, 0.763, while thinking with roughly five times fewer words.The model speaks 288 words continuously versus 285 at baseline, indicating no speech suppression.
  • RLVR preserves parity at 0.771 continuous versus 0.781 baseline because the coverage reward saturates near 0.77–0.79 and supplies no mode-asymmetry gradient.

F Human Evaluation Harness

The paper combines blinded human-preference infrastructure with objective, speech-only verification to test whether judged quality reflects actual task completion.

  • The released harness presents think-stripped continuous and baseline outputs in randomized A/B order and collects pairwise preferences with rationales.The harness is static and covers all 120 Track A scenarios.
  • ReactiveBench’s objective metric uses 688 pre-registered binary factual requirements across 120 scenarios, scored from spoken output with think blocks removed.Two independent verifiers agree on 87.9% of 9,632 item judgments.
  • The gated deployment streams early-entity lookups but answers late-intent tasks turn-based, recovering near-turn-based accuracy while retaining the tool lead.Streaming everything is worst for the base model on late-intent reasoning, whereas the OPD student specializes in streaming and regresses turn-based performance.
  • OPD trades longer, erratic zero-shot tool leads for shorter 3.0–3.8-second leads paired with greater reliability and correctness.
  • Training data use is separated from evaluation splits, with no overlapping dataset-backed questions verified by string intersection.Synthetic early-entity tasks are the deliberate exception described in the supplied passage.
  • The OPD recipe inherits reasoning leakage, lacks trajectory-level correctness filtering, and encourages talking over users because token-wise supervision misses trajectory habits.The training data contains 17.9% spoken meta-reasoning and 39% wrong on-policy trajectories.

I.2 Refining the recipe: coverage, drift, and reward structure

Recipe refinement shows that continuous-time cognition is not itself the limiting factor: coverage, capability preservation, reward structure, and on-policy optimization determine which behaviors survive.

  • Uniform brevity training collapses multi-hop tool use from five calls and 68% accuracy to one or two calls and 40% accuracy.Conditioning on episodes with at least three tools raises OPD accuracy to 75%, showing the lost capability is recoverable.
  • Conservative KL anchoring and balanced-checkpoint early stopping limit capability drift from behavior cloning, while preserving full speech coverage and the headline anticipation behavior.Early-entity tools fire before end-of-utterance 100% of the time with a 3–4-second lead.
  • Preference optimization trades conflicting objectives, pushing away a thorough correct four-tool episode, whereas type-conditional rewards assign it the highest group-relative advantage, +0.56.
  • On-policy RL with a shaped verifiable reward improves correctness axes together, while offline group-relative updates diverge as policy-gradient magnitude rises from 0.1 to 22.A KL-anchored clipped approach restores stability and avoids degeneration.
  • Omitting anticipation from the reward reduces early-entity tool-before-final behavior from 100% to 21% and tool-calls-during-speech from 86% to 54%.Repricing anticipation restores the behavior in the released checkpoint.
  • Across three seeds, online GRPO reaches multi-hop 64±6%, Track B streaming 73±5%, and multiple-choice streaming 72±12%, with replication at larger scale and on Qwen3-4B.
  • Across thirteen model families, 12 of 13 anticipate tools in streaming mode while every family does so in 0% of turn-based tasks.Streaming helps models with low turn-based speech coverage but can hurt models that already speak comprehensively; the cross-family correlation is r=−0.57.

J.2 A new verifiable task type: multiple-choice reasoning and yes/no comprehension

On newly added static-reasoning tasks, streaming generally fails to improve correctness over turn-based interaction, unlike regimes with early entities to anticipate. Training still transfers across modes, while scaled and replicated analyses support the benchmark’s main contrasts.

  • Static reasoning results: Streaming does not beat turn-based on static reasoning for models that already speak, and can reduce correctness when answers must be committed before all options arrive.Across ARC-Challenge, ARC-Easy, OpenBookQA, and BoolQ, the reported streaming deltas are negative for Qwen3-8B, Qwen3-14B, Coverage-SFT, OPD, and rejection-FT.
  • Static reasoning results: Streaming gains appear only for under-speakers, where speech elicitation outweighs the accuracy cost of partial input.Qwen2.5-7B improves by +16 and Qwen3-4B by +5 on the initial static-reasoning set.
  • Transfer across task types: Training on the streaming Track B distribution transfers to held-out static tasks, raising correctness over the base model in both streaming and turn-based modes.For the reported held-out comparison, streaming rises from 46→65 and turn-based from 54→80–81.
  • Broader replication: The finding broadens across seven datasets: streaming does not aid static single-shot reasoning, so its value there is limited to starting speech sooner.On MMLU, PIQA, and WinoGrande, paired deltas span −10 to a non-significant +9, with no model gaining significantly overall.
  • Scale and robustness: Five replicate passes show seed variation of only 1–3 points, small relative to the headline base-versus-OPD and streaming-versus-turn effects.This supports the claim that the main text’s single-sample confidence intervals do not materially understate uncertainty.
  • Scale and robustness: At n=448, the base model has a null streaming−turn delta, while on-policy distillation improves streaming correctness over the base and preserves the tool<final signature.The expanded set is a superset of the 200-task benchmark; the table reports roughly a 10-point streaming improvement with non-overlapping confidence intervals.
Loading 2609.17416v1…