PaperlayerView exact revision on arXiv

Source-linked AI summary

DiffuPlex: Accelerating Full-Duplex Spoken Dialog Models via Rolling Masked Diffusion

Heeseung Kim

arXiv:2610.12214v1cs.SDcs.CL

TL;DR

Fine-grained full-duplex models still require sequential backbone computation at each interaction frame. DiffuPlex predicts masked future user and assistant frames in one wake, consuming a confident prefix and revising only unplayed output; it reduces computation while largely preserving interaction behavior, with a speech-naturalness trade-off for SPEAK.

  • Problem

    Fine-grained full-duplex models update their backbone autoregressively at every interaction frame, requiring frequent sequential computation.

    Source: §1 · p. 1

  • Method

    DiffuPlex predicts masked future user and assistant frames in one backbone wake, consumes a confident prefix, and revises only unplayed output when user input diverges.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    1.46× and 1.59× deployment-path wall-clock speedups for LISTEN and SPEAK accompany interaction behavior that remains largely preserved.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Predicting interaction between backbone invocations better preserves full-duplex behavior than deferring calls while emitting silence in the evaluated baselines.

    Source: §5.3 · p. 8

    SPEAK's more aggressive future consumption comes with lower speech naturalness, partly associated with lost same-frame semantic conditioning for deeper future acoustic codebooks.

    Source: A LIMITATIONS · p. 17

Abstract

from arXiv · show

Recent full-duplex spoken dialog models enable simultaneous listening and speaking, but fine-grained models still advance their backbone autoregressively at every interaction frame. We introduce DiffuPlex, a rolling masked diffusion framework that reduces this sequential computation by predicting multiple future user and assistant frames in a single backbone wake. DiffuPlex consumes only a confident prefix of each predicted future while interaction continues at the original frame rate. As user speech arrives, it checks the corresponding user predictions and, when the interaction diverges, preserves already played assistant content while revising only the unplayed future. We consider two inference policies over the same predictor: DiffuPlex-LISTEN consumes multiple future frames when they predict assistant silence, whereas DiffuPlex-SPEAK can also consume predicted assistant speech. Across full-duplex interaction and spoken-language evaluations, DiffuPlex substantially reduces sequential backbone computation while largely preserving interaction behavior and general capability. DiffuPlex-LISTEN and DiffuPlex-SPEAK achieve $1.46\times$ and $1.59\times$ deployment-path wall-clock speedups and $1.61\times$ and $1.80\times$ Core LM speedups, with all measured backbone invocations completing within the 80ms interaction interval. Human evaluation shows that LISTEN preserves speech naturalness and conversational quality, while SPEAK retains conversational quality with some degradation in speech naturalness.

1 INTRODUCTION

Fine-grained full-duplex models can respond to overlapping speech, but their frame-by-frame autoregressive updates require frequent backbone computation. DiffuPlex predicts and revises future interaction frames in blocks, reducing computation while largely preserving interaction behavior.

1 INTRODUCTION

  • Fine-grained multi-stream models update user and assistant channels every interaction step, requiring a new backbone pass every 80ms in Moshi and PersonaPlex.This granularity supports prompt responses to interruptions, overlaps, and backchannels, but increases backbone computation.Source: §1 · p. 1
  • Future-frame prediction is motivated by long stretches with only one speaker and speech planning beyond individual frames, but future user speech has not yet arrived to verify an assistant draft.Across 38 hours of Switchboard conversations, 95.3% of active conversational time contains only one speaking participant.Source: §1 · pp. 1–2
  • DiffuPlex predicts a block of future user and assistant frames, plays a confident assistant prefix, and revises only unplayed output when observed user speech diverges.The paper calls this repeated predict-consume-revise process rolling masked diffusion.Source: §1 · p. 2
  • 1.46× and 1.59× deployment-path wall-clock speedups accompany largely preserved interaction behavior for DiffuPlex-LISTEN and DiffuPlex-SPEAK, respectively.The two policies also achieve 1.61× and 1.80× Core LM speedups, with all measured backbone invocations completing within 80ms.Source: §1 · p. 2

2 RELATED WORK

Prior work advances full-duplex interaction autoregressively, while parallel and diffusion-based generation address related but distinct settings. DiffuPlex applies masked future prediction to the challenge of jointly predicting full-duplex interaction before future user speech arrives.

2 RELATED WORK

  • Recent full-duplex systems use single-stream generation, including SyncLLM [Veluri et al., 2024] and OmniFlatten [Zhang et al., 2025], or parallel streams, including Moshi [D´efossez et al., 2024] and PersonaPlex [Roy et al., 2026], but remain autoregressive.Earlier dGSLM [Nguyen et al., 2023] jointly modeled conversational speech streams; later work improves turn-taking and timing through interactivity training [Wu et al., 2025; Hsiao et al., 2026; Ohashi et al., 2026; Li et al., 2026b].Source: §2 · pp. 2–3
  • Speculative decoding and parallel decoding reduce sequential token generation [Stern et al., 2018; Leviathan et al., 2023; Chen et al., 2023], while PredGen [Li & Grover, 2025] and RelayS2S [Mai & Liang, 2026] start responses before speech input is complete.Those speculative responses remain autoregressively generated, moving response computation earlier rather than reducing sequential steps for future full-duplex interaction frames.Source: §2 · p. 3
  • Masked and block diffusion predict multiple positions in parallel, with related methods for language [Austin et al., 2021; Lou et al., 2024; Sahoo et al., 2024; Arriola et al., 2025] and speech generation [Cheng et al., 2026b; Seo et al., 2026; Moon et al., 2026].In those settings, farther output positions can be predicted from already available input.Source: §2 · p. 3
  • Unlike prior speech diffusion settings, full-duplex dialogue requires predicting future assistant frames before the corresponding user speech arrives, so DiffuPlex predicts the missing interaction jointly and revises unplayed assistant output.DFlash [Chen et al., 2026a] and DiffuSpec [Li et al., 2026a] instead use diffusion models as parallel drafters for autoregressive verification.Source: §2 · p. 3

3 DIFFUPLEX

DiffuPlex reduces sequential backbone calls by predicting a masked future block at each wake, retaining a confident assistant prefix, and revising only unplayed future when incoming user speech conflicts with stored predictions.

3 DIFFUPLEX

  • At each backbone wake, DiffuPlex predicts assistant and user streams over a short masked future, then consumes a confident prefix and revises unplayed output as the interaction evolves.It is built on a 12.5Hz backbone that otherwise makes one call per 80ms interaction frame.Source: §3 · p. 3

3.1 MASKED FUTURE PREDICTION

  • DiffuPlex masks unavailable future inputs with learned stream-specific tokens and predicts W future positions, where h=1 is the ordinary next-frame prediction and larger h denotes farther positions.Training uses variable-length blocks sampled uniformly from {1, . . . , W}, with lookahead resetting to h=1 at each block start.Source: §3.1 · p. 3

3.2 ROLLING DRAFT GENERATION

  • LISTEN retains future positions only while predicted assistant text remains PAD, whereas SPEAK permits PAD, EPAD, and lexical states; both stop at the first disallowed state or confidence below τconf.For LISTEN, a short PAD run may represent a pause rather than listening, so a policy-specific minimum prefix length is required before consuming future positions.Source: §3.2 · p. 5
  • Retained assistant frames play at the original 80ms rate without additional backbone wakes, while their corresponding user predictions are stored for later consistency checks.At the next wake, received user frames and played assistant frames replace prior predictions and are processed with a newly masked future.Source: §3.2 · p. 5

3.3 USER-AWARE DRAFT REVISION

  • When observed user input and assistant-token evidence trigger revision, DiffuPlex discards only the unplayed future while retaining received user frames and already played assistant frames as context.The revision criterion uses how unlikely the received user token was under its stored prediction and how weakly the assistant preferred its selected token.Source: §3.3 · p. 5

4 EXPERIMENTS

DiffuPlex is evaluated with a frozen PersonaPlex-family backbone, using two inference policies and multiple benchmarks. Comparisons include autoregressive and deferred-execution baselines, while evaluation covers interaction, spoken-language capability, human ratings, and runtime efficiency.

4.1 EXPERIMENTAL SETUP

  • DiffuPlex uses frozen personaplex-rl-seamless, training only stream-specific MASK embeddings and future-gated LoRA parameters on Seamless Interaction.The setup uses maximum lookahead W = 16, confidence threshold τconf = 0.80, and revision threshold τrev = −0.5.Source: §4.1 · p. 5

4.2 EVALUATION

  • Evaluation measures interaction, general spoken-language capability, human-rated speech and conversation quality, and inference efficiency.Interaction tests cover pause handling, backchanneling, turn-taking, and interruption; efficiency uses a fixed 60-clip subset on one RTX A6000 GPU.Source: §4.2 · p. 6

4.3 BASELINES

  • Comparisons include Moshi, PersonaPlex, personaplex-rl-seamless, and three training-free deferred-execution baselines.PAD, SILERO, and SMART TURN skip backbone calls and emit assistant silence, with the interval capped at R ∈ {2, 4, 6, 8} interaction steps.Source: §4.3 · p. 6

5 RESULTS

DiffuPlex reduces backbone computation while largely preserving full-duplex interaction and spoken-language capability. Its results show a trade-off between greater future consumption, runtime savings, and speech naturalness, while confidence filtering and revision help constrain draft use.

5.1 FULL-DUPLEX INTERACTION

  • LISTEN and SPEAK advance 1.91 and 2.23 interaction frames per backbone wake while remaining close to personaplex-rl-seamless across key interaction behaviors.LISTEN has NMOS 3.92 ± 0.11 and DMOS 4.08 ± 0.13, versus 3.93 ± 0.12 and 4.09 ± 0.14 for personaplex-rl-seamless; SPEAK has DMOS 4.07 ± 0.13 and NMOS 3.54 ± 0.12.Source: §5.1 · pp. 6–7

5.2 RUNTIME EFFICIENCY

  • 1.455× and 1.590× deployment-path wall-clock speedups accompany 1.608× and 1.797× Core LM speedups for LISTEN and SPEAK, respectively.Backbone-wake p99 latency is 61.34ms for LISTEN and 61.41ms for SPEAK, with no measured wake frame exceeding 80ms; each wake has higher Core LM cost, amortized over multiple frames.Source: Table 2 · p. 7, §5.2 · p. 7

5.3 MASKED FUTURE PREDICTION VS. DEFERRED EXECUTION

  • At R = 6, PAD reaches 2.05× wall-clock speedup, but backchannel frequency falls from 0.163 to 0.048 and turn-taking latency rises from 0.124s to 0.423s.DiffuPlex predicts interaction between backbone invocations and achieves 1.455× and 1.590× wall-clock speedups while remaining much closer to original interaction behavior.Source: Figure 3 · p. 8, §5.3 · p. 8

5.4 GENERAL SPOKEN-LANGUAGE CAPABILITY

  • LISTEN remains nearly unchanged on VoiceBench and OpenAudioBench, while SPEAK shows moderate degradation, most notably on OpenAudioBench s→s.Table 3 reports general spoken-language capability, with higher scores better.Source: Table 3 · p. 8, §5.4 · p. 8

5.5 ANALYSIS OF FUTURE DRAFTS

  • Removing confidence filtering raises frames advanced per wake, while removing revision raises it further but reduces responsiveness.Without revision, backchannel frequency falls to 0.078 and 0.080 and turn-taking latency rises to 0.440s and 0.480s for LISTEN and SPEAK, respectively.Source: §5.5 · pp. 8–9, Table 4 · p. 9
  • Draft-exhaustion wakes usually follow a single consumed frame, whereas revision-triggered wakes follow several consumed draft frames.For exhaustion wakes, 98.2% of LISTEN and 81.5% of SPEAK wakes occur after one frame; revision-triggered wakes average 4.24 and 3.79 frames.Source: §5.5 · p. 9
  • Restoring autoregressive audio-token conditioning raises UTMOSv2 from 2.646 to 2.894 when regenerating only cb1–7.Regenerating cb0–7 with generated text fixed yields 2.903, compared with 2.919 for fully autoregressive generation.Source: §5.5 · p. 9

6 CONCLUSION

DiffuPlex predicts multiple future positions in one backbone wake, retaining a confident prefix and revising only unplayed future content when user speech diverges. The conclusion presents this as a path toward more compute-efficient real-time full-duplex generation.

6 CONCLUSION

  • DiffuPlex reduces sequential forward passes by predicting multiple future positions in one wake, consuming a confident prefix, and revising unplayed future when observed user speech deviates.The conclusion reports reduced backbone wakes and Core LM time while largely preserving interaction behavior and general spoken-language capability.Source: §6 · pp. 9–10

AI USE STATEMENT

  • The authors report using Claude Code for implementation and visualization support, and ChatGPT for writing assistance and supplementary related-work checks.They state that authors reviewed AI-assisted artifacts and take responsibility for the final content.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • The evaluation uses the backbone’s official voice prompt and does not aim to develop or evaluate impersonation, while acknowledging potential synthetic-speech misuse.The authors say use should follow appropriate safeguards and the licenses and usage conditions of the underlying models and datasets.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper documents its training, inference, data-processing, evaluation, and runtime-measurement procedures and plans to release implementation and evaluation scripts after review.External datasets, pretrained models, evaluation tools, licenses, and public sources are listed in Table 18.Source: REPRODUCIBILITY STATEMENT · p. 10

A LIMITATIONS

DiffuPlex's limitations concern its pretrained-backbone adaptation, separately controlled prediction and revision, future speech generation, and transfer across backbones and deployment environments.

A LIMITATIONS

  • DiffuPlex adapts a frozen pretrained autoregressive backbone with future-gated LoRA adapters and learned MASK embeddings, rather than training natively for masked-future generation.The authors suggest native masked-future training may make better use of additional future context.Source: A LIMITATIONS · p. 17
  • Future prediction is trained separately from inference-time retention and revision rules, which use prediction confidence and a fixed revision threshold.Jointly learning these decisions could improve trade-offs among computation, responsiveness, and prediction reliability.Source: A LIMITATIONS · p. 17
  • Aggressive future consumption is harder during assistant speech than listening, and SPEAK's lower wake frequency comes with lower speech naturalness.The authors identify lost same-frame semantic conditioning for deeper acoustic codebooks as one contributor; neither restoring cb0 conditioning nor autoregressive replay is a practical solution for future speech generation.Source: A LIMITATIONS · p. 17
  • Transfer is not automatic: the PersonaPlex variant reduces backbone wakes but degrades interaction behavior, especially pause handling and interruption behavior.The main experiments use personaplex-rl-seamless; substantially different full-duplex architectures have not been evaluated, and runtime speedups depend on hardware, software, and implementation.Source: A LIMITATIONS · p. 17

B DEMO PAGE

  • Selected audio examples and qualitative demonstrations are available on the DiffuPlex demo page.https://diffuplex.github.io/Source: B DEMO PAGE · p. 17

C.1 BACKBONE ARCHITECTURE

The backbone's parallel streams retain their native delays, and DiffuPlex trains masked future blocks that respect stream-specific availability while keeping temporal attention causal.

C.1 BACKBONE ARCHITECTURE

  • The backbone represents assistant text, eight assistant audio codebooks, and eight user audio codebooks; generated user codebooks are not played back.Its temporal backbone models successive positions, while a depth transformer predicts audio codebooks; during streaming, incoming user audio supplies the user streams.Source: §C.1 · pp. 17–18
  • The native one-position delay lets same-frame assistant cb0 enter the temporal representation before cb1–7 are predicted, and DiffuPlex preserves this alignment.Temporal generation otherwise remains autoregressive, with the backbone evaluated every 80ms; the method targets that schedule without changing stream organization or the next-frame shift.Source: §C.1 · p. 18, §C.2 · p. 18

C.2 EXACT STREAM ALIGNMENT

  • At model position p, streams with delay 0 receive frame p − 1 and predict frame p, while streams with delay 1 receive frame p − 2 and predict frame p − 1.Thus physical user frame f becomes available at position f + 1 for cb0 and f + 2 for cb1–7; a model position does not contain every stream's same physical frame.Source: §C.2 · p. 18

C.3 MASKED TRAINING LAYOUT

  • Masking follows input availability: assistant streams from h ≥ 2, user cb0 from h ≥ 3, and user cb1–7 from h ≥ 4; temporal attention remains causal.Each stream has its own learned MASK embedding; each block is a fresh prediction from realized clean context at its boundary, not a continuation of the preceding masked block.Source: Figure 4 · p. 19, §C.3 · pp. 19–20
  • Training concatenates clean and masked copies of each sequence, then divides the masked dialog region into variable-length blocks with lookahead reset at each block start.Block lengths are sampled uniformly from 1 to W, with W = 16 for the main model; the system-prompt region stays unchanged.Source: §C.3 · p. 19

D.1 DATA PROCESSING

The appendix details the conversation data and training objective, inference and revision policies, deferred-execution baselines, and evaluation protocols used for DiffuPlex.

D.1 DATA PROCESSING

  • The source corpus contains more than 4,000h of dyadic conversation; filtering leaves 42,613 sessions, with 42,413 used for training and 200 session-disjoint sessions for validation.The training split totals 2,728.5h, and valid voice references are selected to avoid exposing speech from the target interaction window.Source: §D.1 · p. 20

D.2 TRAINING

  • Training distills the autoregressive model over dialog positions, excludes invalid targets and prompt positions, and gives semantic-codebook and acoustic-codebook losses weights of 1 and 0.02.An additional assistant-text state distribution loss matches PAD, EPAD, and lexical interaction states; training uses future-gated LoRA and learned MASK embeddings.Source: §D.2 · pp. 20–21

D.3 INFERENCE

  • LISTEN retains future positions only for predicted PAD states and requires a candidate prefix of at least 6; SPEAK permits PAD, EPAD, or lexical states with a minimum of 2.Both use W = 16, confidence threshold 0.80, revision threshold −0.5, and always retain the ordinary h = 1 prediction; revision preserves played assistant and observed user frames, replacing only unplayed future.Source: §D.3 · pp. 21, 23

D.4 DEFERRED-EXECUTION BASELINES

  • Deferred-execution baselines postpone autoregressive updates only during silence, requiring both assistant PAD text and low decoded-audio RMS because PAD can occur within speech.Controllers include PAD confidence, streaming SILERO VAD, and SILERO plus SMART TURN; deferral is capped by run length, and controller cost is measured separately from streaming runtime.Source: §D.4 · pp. 23–24

E EVALUATION DETAILS

  • The appendix provides evaluation protocols for interaction, general spoken-language capability, and runtime results, with model names and prompts following benchmark conventions unless stated otherwise.The general spoken-language protocol constrains assistant output to silence during the instruction and begins free generation after its final frame.Source: E EVALUATION DETAILS · p. 24

E.1 FULL-DUPLEX BENCHMARKS

The evaluation appendix specifies continuous-interaction benchmarks, single-turn spoken-language protocols, runtime measurements, and statistical tests for comparisons with the autoregressive backbone.

E.1 FULL-DUPLEX BENCHMARKS

  • Full-Duplex-Bench v1.0 evaluates pause handling, smooth turn-taking, backchanneling, and interruption during continuous assistant generation; v1.5 uses paired inputs with and without overlap events.The v1.0 set contains 727 clips, while v1.5 contains 498 examples; its metrics include response behavior, latency, and changes in speech characteristics.Source: §E.1 · pp. 24–25

E.2 GENERAL SPOKEN-LANGUAGE BENCHMARKS

  • VoiceBench and English OpenAudioBench subsets are evaluated under a single-turn protocol that begins free generation after the complete instruction is received.The protocol suppresses assistant output during the instruction, then scores both native text and transcribed speech output.Source: §E.2 · pp. 25–26

E.3 RUNTIME MEASUREMENT

  • Runtime is measured on a fixed 60-clip subset using wall time, end-to-end GPU time, and Core LM time relative to the autoregressive reference.The evaluation also reports frames advanced per backbone wake and p99 wake-frame latency, including the fraction exceeding the 80ms interaction interval.Source: §E.3 · p. 26

E.4 STATISTICAL TESTING

  • For v1.0 interaction metrics, LISTEN and SPEAK are each compared with the autoregressive backbone using paired tests across 20 comparisons and Holm correction.A non-significant corrected result is treated as no detected evidence of a difference, not as formal equivalence.Source: §E.4 · p. 26

F HUMAN EVALUATION

The human-evaluation protocol separates ratings of post-interruption speech naturalness from ratings of complete-interaction flow. Additional analyses report paired benchmark comparisons, behavioral and speech-feature changes, and the scope of spoken-language benchmark results.

G.1 PAIRED SIGNIFICANCE RESULTS

  • None of the 20 paired Full-Duplex-Bench v1.0 comparisons remains significant after Holm correction, which does not establish formal equivalence.The table compares paired clips against the autoregressive personaplex-rl-seamless backbone; Holm correction covers all 20 comparisons.Source: Table 5 · p. 29, §G.1 · p. 30

F HUMAN EVALUATION

  • NMOS rates the assistant’s post-interruption speech, while DMOS rates the complete interaction’s flow and interruption handling; compared outputs share interactions and are presented in randomized, blinded order.NMOS uses 200 valid assignments from 120 workers, and DMOS uses 180 valid assignments from 84 workers.Source: Figure 5 · p. 27, Figure 6 · p. 28, F HUMAN EVALUATION · p. 29

G.2 FULL-DUPLEX-BENCH V1.5

  • For RESPOND interruption trials, all systems show increased speaking rate and mean intensity after interruption, while pitch variability and cross-system differences in change magnitude are not established.Mean pitch decreases significantly for both DiffuPlex policies but not the autoregressive backbone; UTMOSv2 increases slightly for both DiffuPlex policies.Source: Table 7 · p. 30, §G.2 · p. 31

G.3 VOICEBENCH AND OPENAUDIOBENCH

  • The VoiceBench and OpenAudioBench tables provide per-task results underlying the paper’s aggregates, with OpenAudioBench overall normalizing task scores to percentages before averaging.VoiceBench uses task-specific scores on a common percentage scale; its Overall value is the aggregate reported in the main paper.Source: Table 8 · p. 31, Table 9 · p. 31

G.4 DEFERRED-EXECUTION RUNTIME RESULTS

The deferred-execution runtime results report speedups relative to the autoregressive backbone, with controller decisions replayed rather than inferred during timed generation. The ablation sweep separately reports pooled LM speedup, which is distinct from controlled runtime measurements.

G.4 DEFERRED-EXECUTION RUNTIME RESULTS

  • Deferred-execution runtime measurements replay SILERO and SMART TURN controller decisions, excluding controller inference from the timed loop; Table 10 reports c60 on the fixed subset and c727 across all 727 clips.All speedups are relative to the autoregressive backbone.Source: §G.4 · p. 31, Table 10 · p. 32
  • The SMART TURN controller pipeline requires approximately 13.90ms of CPU processing per interaction second, about 80% attributable to the continuously running Silero frontend.This controller cost is measured separately from the reported timed-generation runtime.Source: §G.4 · pp. 31–32

H ADDITIONAL ABLATIONS

  • The ablation sweep reports pooled LM speedup for comparing configurations, distinct from controlled wall-clock, E2E GPU, and Core LM measurements.The ablations use 727 Full-Duplex-Bench v1.0 clips unless otherwise stated.Source: H ADDITIONAL ABLATIONS · p. 32

H.1 TRAINING OBJECTIVE AND ARCHITECTURE

The ablations examine how training choices, horizon, retention, revision, and attention affect advancement and interaction behavior. Across these tests, more aggressive future use does not consistently preserve interaction metrics, and the rolling-draft analysis distinguishes retained, consumed, and discarded future frames.

H.1 TRAINING OBJECTIVE AND ARCHITECTURE

  • Without the future gate, the learned adapters can alter the h = 1 path and TOR falls; hard-target cross-entropy shortens the usable horizon, while wake reduction transfers to PersonaPlex without automatic behavior preservation.For PersonaPlex, interruption TOR is .905 and .920, with latencies of 1.277s and 1.347s.Source: §H.1 · pp. 32–33, Table 11 · p. 33

H.2 MODEL AND TRAINING SCALE

  • Across rank and training-duration sweeps, frames advanced per wake remain relatively stable; increasing LoRA rank from 128 to 512 or training from 526 to 1,052 steps yields no consistent improvement, while interaction metrics vary.With matched training and inference horizons, W = 8 gives similar advancement but shifts several interaction metrics; W = 32 adds little advancement while increasing per-wake LM cost and reducing pooled LM speedup.Source: §H.2 · p. 34

H.3 INFERENCE HORIZON

  • With W = 16 training, inference W = 32 leaves c nearly unchanged but reduces pooled LM speedup from 1.56× to 1.42× for LISTEN and 1.77× to 1.64× for SPEAK.Across horizon mismatches, c remains near 1.86–1.91 for LISTEN and 2.19–2.27 for SPEAK.Source: §H.3 · p. 34

H.4 CONFIDENCE AND RETENTION

  • Disabling confidence filtering raises c from 2.23 to 2.80 for SPEAK but lowers its interruption Judge score from 4.357 to 3.550 and raises backchannel TOR from .145 to .236.For LISTEN, disabling confidence filtering raises c from 1.91 to 2.11 with comparatively modest interaction-metric changes; lower confidence thresholds increase advancement.Source: §H.4 · p. 35, Table 14 · p. 35

H.5 REVISION RULE

  • Without revision, SPEAK advances c from 2.23 to 3.08, but turn-taking TOR falls from 1.000 to .950 and turn latency rises from .156s to .480s.LISTEN turn latency also increases, from .173s to .440s; the authors conclude that revising unplayed drafts matters for responsiveness.Source: §H.5 · p. 35, Table 15 · p. 35

H.6 CAUSAL AND BIDIRECTIONAL FUTURE ATTENTION

  • Bidirectional future attention produces nearly the same c as causal attention but mixed interaction changes, with no consistent improvement at comparable advancement.The paper retains causal attention because it matches the autoregressive backbone’s causal structure and supports variable block lengths during training.Source: §H.6 · p. 36, Table 16 · p. 36

H.7 REVISION SIGNAL DECOMPOSITION

  • The full revision rule combines user surprisal and assistant uncertainty; single-signal variants show distinct interaction profiles and are not strictly matched for realized advancement.Assistant-only permits more advancement but accompanies longer interruption latency; user-only changes latency and backchannel measures despite similar LISTEN advancement.Source: §H.7 · pp. 36–37, Table 17 · p. 37

I ROLLING-DRAFT ANALYSIS

  • Average frames advanced per wake, c, does not explain short or long wake intervals; Figure 7 analyzes retained future, frames consumed, and unplayed future discarded by revision.The analysis connects the retained draft at each wake to what is consumed before the next wake.Source: I ROLLING-DRAFT ANALYSIS · p. 37

I.1 WAKE DYNAMICS AND REVISION

Wake diagnostics show that conservative prefix retention, rather than immediate revocation of long drafts, often limits acceleration. Revision can safeguard consistency while leaving already played interaction untouched.

I.1 WAKE DYNAMICS AND REVISION

  • Each wake selects a retained prefix r_w from future horizon W, while a_w counts realized frames before the next wake; exhaustion gives a_w = r_w, and revision gives a_w < r_w.Wake-level analyses exclude clip-end truncations, but the main frames-per-wake metric uses the complete interaction.Source: §I.1 · p. 37
  • 23.6% of LISTEN and 32.1% of SPEAK wake terminations are revision-triggered; exhaustion is dominated by short retained prefixes.Revision-triggered wakes generally follow several consumed frames, so short intervals more often reflect conservative retention than long drafts immediately discarded.Source: §I.1 · pp. 37–38, Figure 7 · p. 38
  • 92.4% of LISTEN and 82.5% of SPEAK revisions leave the next four fully revisable assistant states unchanged.Revision affects only unplayed interaction; a nontrivial subset of triggers changes the immediate assistant state.Source: §I.1 · p. 38

I.2 FUTURE-SPEECH MECHANISM

  • Restoring same-frame cb0 reduces acoustic-codebook NLL by 0.0912 during assistant speech, versus 0.0285 during silence.Restoring assistant text instead does not help (∆= −0.0176), linking the larger speech effect specifically to missing semantic audio input.Source: §I.2 · pp. 38–39
  • Autoregressive replay recovers most of the measured UTMOSv2 gap by regenerating cb1–7 with DiffuPlex text and cb0 fixed.Regenerating cb0 as well adds only a small gain; these are diagnostic analyses, not DiffuPlex inference stages.Source: §I.2 · pp. 39–40

I.3 QUALITATIVE EXAMPLES AND FAILURE CASES

  • During a stable silent interval, SPEAK consumes two consecutive 16-frame drafts; after revision, it discards nine unplayed frames from a 12-frame silence draft.The revision occurs after three frames have been consumed, before a new assistant continuation is generated.Source: Figure 8 · p. 39, Figure 9 · p. 39, §I.3 · p. 41
  • When the next predicted assistant state is lexical, SPEAK consumes nine frames while LISTEN advances one; stable silence can still yield 47 consecutive one-frame SPEAK passes.Limited acceleration can therefore reflect either revision of a longer retained future or repeatedly conservative one-frame retention.Source: Figure 10 · p. 40, Figure 11 · p. 40, §I.3 · p. 41

J ARTIFACTS, LICENSES, AND REPRODUCIBILITY

The paper documents its external artifacts and records provenance for scored generations and analysis outputs. It also reports training and runtime hardware and verifies a matched W = 1 boundary case.

J ARTIFACTS, LICENSES, AND REPRODUCIBILITY

  • Table 18 lists external datasets, models, and tools alongside their licenses and public sources.The artifacts are used for research consistent with their intended use.Source: J ARTIFACTS, LICENSES, AND… · p. 41
  • Scored generations retain their seed and inference configuration, while ablations and analysis outputs record provenance and are stored by artifact type.The recorded metadata supports tracing reported results to the configuration that produced them.Source: J ARTIFACTS, LICENSES, AND… · p. 41
  • DiffuPlex training uses four RTX A6000 GPUs; controlled runtime tests use one, and matched W = 1 runs reproduce the official frame-by-frame token sequence.No other jobs ran on the runtime-measurement node.Source: J ARTIFACTS, LICENSES, AND… · p. 41
Loading 2610.12214v1…