Integrate the Reward, Don't Sample It: Window Likelihood for Interruptions and Backchannels in Full-Duplex Speech Models

1 Introduction

• Full-duplex models must yield to interruptions but hold through backchannels, yet the two errors trade off as the model is trained to stop more or less often.Moshi yields to only a minority of real interruptions, while released Lychee-FD stops on half of user backchannels.Source: §1 · p. 1
• Binary window rewards produce zero group-relative advantage when all rollouts agree, affecting 23–34% of training windows on three measured models when all 16 rollouts fail.These are difficult windows where the base policy almost never stops on interruptions or almost always stops on backchannels.Source: §1 · p. 1
• Whole-rollout GRPO spreads one rollout score across listening-state decisions that do not determine the timing reward, causing floor re-opening or failure to answer in tested models.After GRPO, Lychee-FD re-opens the floor in every 100-s silence and Freeze-Omni stops answering the first question.Source: §1 · p. 1
• Window likelihood marginalizes sampled continuations’ binary control decisions in closed form and maximizes the resulting expected window reward as a log-likelihood.The approach targets rewards that are known functions of a few policy probabilities.Source: §1 · p. 2
• On HumDial-FDBench, window likelihood achieves the best yield/hold F1 on all six models, 13 to 62 points above base.Matched on event-window rollouts, GRPO trails where it trains stably and collapses elsewhere, with a side effect on three of four models.Source: §1 · p. 2

2 Related Work

• Open full-duplex models differ in how and whether they express stop decisions, spanning control branches, control tokens, VAD-gated classifiers, or no explicit control interface.The paper post-trains one model of each kind.Source: §2 · p. 2
• Chen et al. (2025) use online RL for barge-in and backchannel continuation on their 1.1B model and synthetic tests, while Ohashi et al. (2026) post-train Moshika and PersonaPlex with GRPO.Ohashi et al. use G=16 rollouts per group, rule-based rewards on four axes, and an LLM content reward.Source: §2 · p. 2
• Timing post-training approaches generally estimate rewards by sampling, including windowed GRPO, action-space projection, event-causal masking, and offline preference pairs.DuplexPO keeps credit local but still samples rewards and does not evaluate holding through a backchannel; HiPLEX routes sampled advantages through event-causal masks.Source: §2 · p. 2
• Fixes for zero-centred advantages in uniform binary-reward groups keep sampling, whereas this paper takes the timing reward out of the sampler.The cited fixes include sign advantage, group splitting, and dynamic sampling.Source: §2 · p. 2

3 Yield/Hold Decisions in Event Windows

• An event window is the two-second interval after an interruption or backchannel begins while the model is speaking, containing K binary control decisions at the model’s own decision rate.The rate varies by interface, from every 0.4 s for Lychee-FD to every text position, VAD frame, or 80-ms frame in other model families.Source: §3 · p. 2
• Each stop probability pk is conditioned on the dialogue prefix in which the model has kept speaking up to decision k.The window stop probability is described for a fixed sampled content path; the passage notes there is no independence assumption because each probability is conditioned on continued speaking.Source: §3 · p. 2
• Interruption stop rate I measures stops within two seconds after interruptions, while backchannel false-stop rate B measures stops within two seconds after backchannels.The yield/hold F1 is the harmonic mean of I and the hold rate 1 − B, and is high only when both yielding and holding are strong.Source: §3 · p. 3

4.1 Closed-Form Expected Window Reward as a Likelihood

• WL computes the expected window reward from visited stop probabilities in closed form and maximizes it as a likelihood, using noisy-OR for yielding and survival likelihood for holding.A teacher-forced pass evaluates one sampled continuation; its content and any early stop are held fixed, with no gradient through sampling.Source: §4.1 · p. 3
• Auxiliary timing terms apply survival to stop decisions before the event and noisy-OR or survival to post-utterance startspeaking decisions, as appropriate.The terms are used only where the model's control channel governs the decision; the Moshi family uses neither auxiliary term.Source: §4.1 · p. 3

4.2 Two Properties That Follow from the Form

• At S=0.02, a G=16 GRPO group is informative with probability 0.28, while the WL interruption gradient over K=5 equal-probability decisions has magnitude 0.99.WL's interruption gradient stays Θ(1) as S approaches zero, whereas GRPO groups are informative only intermittently at the extremes.Source: §4.2 · pp. 3–4, Figure 2 · p. 4
• At S=0.98, the WL backchannel gradient is 2.7 against the same GRPO informativeness probability of 0.28; WL's gradient fades only at the solved end.The logarithm prevents the vanishing gradient on difficult windows, and integration removes score sampling variance.Source: §4.2 · pp. 3–4
• Event terms put gradient on in-window stop decisions, not listening-state decisions; avoiding other listening-state drift additionally requires turn-taking terms or the Moshi gate.The figure contrasts GRPO's rollout-wide score with WL's closed-form computation from the window's stop probabilities.Source: §4.2 · p. 4, Figure 1 · p. 4

4.3 Instantiation for Models without a Control Token

• For models without a stop token, a stop is ten consecutive wordless 80-ms frames; overlapping blocks use a union-bound-based monotone surrogate, so both terms are surrogates rather than likelihoods.The block spans 0.8 seconds, matching the evaluation merge gap.Source: §4.3 · pp. 4–5
• The no-control-token adaptation trains only the PAD and end-of-text PAD output rows, while gating them to yield/hold decisions and adding post-turn rows to address re-opening.The event gate is active from user speech over the model until two seconds after the user stops; the second-stage rows leave I and B unchanged.Source: §4.3 · p. 5

5.1 Experimental Setup

• The six-model evaluation uses HumDial-FDBench real-recorded interruptions and backchannels, with training on the model’s test-language clips and disjoint training and test speakers and dialogues.Every configuration is a single training run; reported confidence intervals are 95% bootstrap intervals over test items.Source: §5.1 · p. 5

5.2 Closed-Loop Evaluation Protocol

• The closed-loop simulator injects real interjections 0.8, 1.6, or 2.4 seconds after the model starts answering, and drops items when the model is no longer speaking.Yielding and false stops are read from each model’s stop decision within two seconds; the silence test measures reopening and speech during a 100-second post-answer silence.Source: §5.2 · p. 6

5.3 Main Results: Yielding and Holding on Six Models

• WL raises F1 on every model, but its per-model trade-offs include lower PersonaPlex stop rate and persistently low absolute performance on Freeze-Omni.GRPO’s outcomes vary: BayLing interruption handling collapses, while MiniCPM-o over-stops on backchannels.Source: §5.3 · p. 6
• WL re-opens in 0% of silences on five models and 5.4% on Moshika, while BayLing’s 0% covers only dialogues whose first answer ends.On the other four models, every GRPO run re-opens in 100% of silences and fills 41–68 seconds of the window.Source: §5.3 · p. 6
• “Wait” is the hardest interruption category for every model, while WL exceeds 84% in every category on Lychee-FD and BayLing.WL trails a GRPO configuration on MiniCPM-o in every category, most on “wait” (54.8 vs 83.6), alongside 0.3% vs 34.7% false stops.Source: §5.3 · pp. 6–7
• WL has the highest F1 on every model, with test-item confidence intervals disjoint from the runner-up’s on all six; it beats stable GRPO by 7.2 and 5.7 F1 on Lychee-FD and MiniCPM-o.BayLing and Freeze-Omni GRPO runs collapsed.Source: §5.3 · p. 6, Table 1 · p. 7

5.4 Two GRPO Failures, Measured

• Whole-rollout GRPO’s all-identical rollouts occur on 72% of Lychee-FD, 19% of MiniCPM-o, 36% of BayLing, and 91% of Freeze-Omni training windows.Wrong-side windows where all 16 rollouts fail make up 23%, 33%, and 34% of windows for Lychee-FD, BayLing, and Freeze-Omni, respectively.Source: §5.4 · p. 7, Figure 3 · p. 8
• Only 15–47% of GRPO-credited decisions fall inside the event window, and 69% of Freeze-Omni’s credited decisions are listening decisions during the first question.The speaking-only ablation removes Lychee-FD’s silence reopening and BayLing’s collapse, but does not recover working GRPO or match WL; remaining confounds include auxiliary terms and more distinct windows per step for WL.Source: §5.4 · p. 7

6 Conclusion

• Across six open full-duplex models, WL gives the best yield/hold F1 and shows neither measured GRPO failure.The method trains a stop-decision head where available and attention LoRA otherwise; the authors argue it may also serve as a timing term inside windowed RL.Source: §6 · p. 8

7 Limitations

• Freeze-Omni’s F1 rises from 19.7 to 40.5 but remains low because its VAD-gated linear state head cannot distinguish an ongoing question from a just-finished one.The window data contain no information about that distinction.Source: §7 · p. 8
• In BayLing-Duplex, suppressing [EPAD] on backchannels also affects answer endings because the token serves both functions.Thirty percent of WL’s first answers do not end within 120 s in the silence test.Source: §7 · p. 8
• On PersonaPlex, the event-row readout shifts the operating point rather than improving discrimination; Moshika’s text-projection rows cannot control non-verbal audio.Moshika WL re-opens in 2 of 37 silences with non-verbal audio.Source: §7 · p. 8
• The reported numbers are not comparable with the official transcript-based HumDial leaderboard, and the study uses one benchmark and one training seed per configuration.Source: §7 · p. 8

A Training Details

• The six-model training table distinguishes four explicit-stop models with matched event-window GRPO comparisons from two models compared against released checkpoints.For Lychee-FD, the turn-taking term adds two windows per step; “a + b” denotes event-window and post-turn stages.Source: Table 2 · p. 11
• WL scores each model’s own answer continuation after inserting the interjection as at test time, then re-scores it with a teacher-forced pass.GRPO samples 16 continuations per window; its group advantage multiplies rollout log-probability, including decisions WL does not score.Source: A Training Details · p. 11
• Lychee-FD trains its control head for 554 steps; WL uses eight event windows with two continuations each per step, while GRPO uses one window with 16 rollouts.WL also uses two turn-taking windows per step; the speaking-only variant applies advantage only to speaking-state decisions.Source: A Training Details · p. 11
• MiniCPM-o and BayLing-Duplex use rank-16 attention LoRA; MiniCPM-o uses 150 WL steps × 8 windows and 75 GRPO steps × 16 rollouts, totaling 1,200 rollouts each.Freeze-Omni trains only its state head, with 150 steps and 16 windows or rollouts per step.Source: A Training Details · p. 11
• PersonaPlex and Moshika train two text-projection rows in an event stage and a post-turn stage, with model-specific step counts and learning rates.During post-turn training, dialogues run through the answer and 16 further seconds, with per-frame stay-silent likelihood on PAD and event rows frozen.Source: A Training Details · p. 11

B Evaluation Details

• Each language’s HumDial-FDBench set contains 300 interruptions per category and 300 backchannels, as reported for the interjection categories.Source: Table 3 · p. 12
• The silence test follows each model’s first answer with 100 s of silence and counts whether speech reopens and total speech duration.Dialogues with no answer or an answer that has not ended after 120 s are excluded; BayLing WL has 12 such cases out of 40.Source: B Evaluation Details · p. 12
• Figure 4 illustrates three Lychee-FD cases: follow-up interruption, backchannel, and silence, with stop decisions judged in a shaded 2-s window.WL and the base stop for the follow-up, while only the base falsely stops for the backchannel; GRPO reopens the floor nine times during silence.Source: Figure 4 · p. 12
• Figure 5 plots backchannel false-stop rate against interruption stop rate, with 95% CIs and F1 iso-lines; the ideal is the top-left corner.WL moves toward that corner on four controlled models and Moshika, while PersonaPlex trades stop rate for fewer false stops.Source: Figure 5 · p. 13
• Figure 6 compares speech during 100-s silence windows: whole-rollout GRPO and both released Kyutai checkpoints fill the window, whereas WL stays quiet.The re-opening is not attributed to GRPO as such for the released checkpoints; Freeze-Omni and MiniCPM-o are omitted because they do not reopen.Source: Figure 6 · p. 13

• Table 4 reports interruption stop rate (%) by category on the full test set.Source: Table 4 · p. 14

D The Moshi Family: Gate Mechanics and Gating Ablation

• Figure 7 separates event rows, active during yield/hold decisions, from post-turn rows, active in silence after an answer; frozen rows preserve base behavior elsewhere.Its panel (c) plots the share of dialogues that re-open the floor in the 100-s silence test.Source: Figure 7 · p. 14
• Event rows activate when recent output energy exceeds threshold as the user starts speaking over the model, while post-turn rows activate after 0.8 s of mutual silence.Training a LoRA on the whole text head with the same objective collapses into non-stop babbling within ten steps because emitting words trivially satisfies “keep talking.”Source: D The Moshi Family: Gate Mechanics and… · p. 14
• Ungated readout re-opened the floor in 97% of cases, an energy gate left 7.5% of first answers unfinished, and event plus post-turn gates avoided those side effects.The event gate matches the base outside events, and post-turn rows remove the base model’s re-opening without touching I or B.Source: D The Moshi Family: Gate Mechanics and… · p. 14, Table 5 · p. 14