Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models

1 Introduction

• Full-duplex models process user and system speech streams in parallel, unlike turn-based systems that wait for the user’s entire utterance before responding.This design supports learning turn-taking, backchanneling, and overlap handling within the model.Source: §1 · p. 1
• Token-level likelihood maximization does not directly optimize interaction-level timing, while training–inference input mismatch can undermine robust behavior in real conversations.Reported issues include excessive silence, badly timed turn-taking, and too few backchannels.Source: §1 · p. 1
• Prior RL studies target only subsets of conversational dynamics, often one model, and timing-only rewards can degrade semantic quality.The paper adds an LLM Judge reward for response quality while targeting all four axes.Source: §1 · pp. 1–2
• The method improves interactivity across Moshi and PersonaPlex on Full-Duplex-Bench v1, generalizes to real-time multi-turn dialogue, and is preferred by human raters.The reported axes are pause handling, turn-taking, backchanneling, and user interruption.Source: §1 · p. 2

2.1 Full-Duplex Spoken Dialogue Models

• End-to-end models process user and system audio in a unified model, while cascaded systems use external VAD and turn-control modules; most existing models lack interaction-level optimization.Cascaded designs can incur inter-module latency and lose paralinguistic information; this work applies RL post-training to end-to-end architectures.Source: §2.1 · p. 2

2.2 Reinforcement Learning for Spoken Language Models

• Speech RL has addressed semantic coherence, speech quality, QA, reasoning, and paralinguistics; full-duplex methods such as SALMONN-omni and ORISE target barge-ins and backchannels [Yu et al., 2025].Prior and concurrent methods cover only subsets of interactivity, and ASPIRin reports degradation in response latency or semantic quality [Hsiao et al., 2026].Source: §2.2 · p. 2

2.3 Benchmarks for Full-Duplex Spoken Dialogue Models

• Full-Duplex-Bench v1 evaluates responses to pre-recorded static audio, whereas v2 evaluates real-time interaction with an automated conversational partner.The work uses v1’s four axes for reward design and v2 to verify generalization to multi-turn settings.Source: §2.3 · pp. 2–3
• The method extracts human-conversation segments for each of four interaction axes, samples multiple model outputs, scores them with axis-specific rewards, and optimizes via GRPO.Segments may be paired with their dialogue context.Source: Figure 1 · p. 3

3 Method

• The method uses GRPO on short human-conversation segments to target pause handling, turn-taking, backchanneling, and user interruption.Each segment exemplifies one axis and receives an axis-specific reward.Source: §3 · p. 3, §3.2 · pp. 3–4

3.1 Full-Duplex Spoken Dialogue Modeling

• The model autoregressively predicts speaker Y’s tokens from speaker X’s stream and its own prior outputs, while a parallel text stream guides speech content and timing.The method assumes discrete audio tokens and joint text-token prediction.Source: §3.1 · p. 3

3.2 Reinforcement Learning Pipeline

• For each sampled axis, the policy generates multiple completions, scores their decoded audio, normalizes rewards into advantages, and updates with a clipped loss and reference-policy KL penalty.The objective is computed over text-token probabilities, while the loss excludes the prepended context window.Source: §3.2 · pp. 3–4

3.3 Training Data Curation

• Training segments are extracted from two-speaker recordings through utterance annotation and event-driven identification of examples for each interactivity axis.The method uses VAD-derived intervals and axis-specific segment criteria.Source: §3.3 · p. 4

3.4 Reward Design

• Pause rewards penalize speech intervals longer than one second, while turn-taking rewards penalize response delay after the user finishes.Backchannel reward uses F1 for short vocalizations near ground-truth positions, counting takeovers as false positives; interruption reward penalizes delay after the interruption ends.Source: §3.4 · p. 5
• An LLM judge scores contextual relevance and naturalness for turn-taking and interruption rewards, complementing delay-based rewards to address semantic degradation.Generated audio is transcribed before the judge scores the response on a three-point scale.Source: §3.4 · p. 5

4.1 Training Datasets

• Training data combines Fisher telephone conversations with Seamless Interaction’s improvised and naturalistic subsets, spanning distinct recording conditions and conversational populations.Fisher contains 2,000 hours; the Seamless subsets contain 1,300 and 2,700 hours, respectively.Source: §4.1 · p. 5

4.2 Evaluation Benchmarks

• The study evaluates models on pre-recorded audio and real-time multi-turn dialogue, using Full-Duplex-Bench v1 for four interactivity axes and v2 for streaming tasks.Version 1 measures takeover rate across axes plus axis-specific metrics; version 2 covers Daily, Correction, Entity Tracking, and Safety tasks.Source: §4.2 · pp. 5–6

4.3 Models and Training Details

• The method is applied to Moshi and PersonaPlex, with training run for 100 epochs using 32 segments per epoch and 16 completions per segment.Training uses 32 NVIDIA H100 GPUs and prepends randomly sampled context windows that grow to a maximum of 30 seconds.Source: §4.3 · p. 6

5.1 Results of Static Evaluation

• RL-trained models achieve the best or near-best scores across static metrics while matching strong turn-taking TOR with lower pause-handling TOR.Unlike dGSLM, which has high turn-taking TOR at the cost of poor pause handling, the results show these behaviors can improve together.Source: §5.1 · p. 7, Table 1 · p. 7

5.1 Results of Static Evaluation; 5.2 Results of Interactive Evaluation

• Across both model families, RL improves static-benchmark results and turn-taking fluency in all four real-time tasks.Static results also show better pause handling, turn-taking, backchanneling, and user-interruption measures; dynamic instruction-following and task-specific scores improve in most conditions.Source: §5.1 · pp. 6–7, §5.2 · p. 7, Table 2 · p. 7

5.2 Results of Interactive Evaluation

• Seamless training yields stronger improvements than Fisher across both model families, while Fisher training can reduce PersonaPlex’s Safety-task scores.The paper reports that a safety-aware LLM Judge criterion mitigates the Fisher-related degradation.Source: §5.2 · p. 8

5.3 Human Evaluation

• Moshi + RL is consistently preferred to base Moshi across human-evaluated axes and scenarios, although some differences are not statistically significant.Figure 2 covers seven Full-Duplex-Bench tasks, with significance tested on per-dialogue mean ratings.Source: §5.3 · p. 8, Figure 2 · p. 8

5.4 Ablations

• Ablations show that pause and turn-taking data prevent opposing extremes, while backchannel rewards, the LLM Judge reward, and preceding context support timing and content quality.Without pause data the model speaks at every opportunity; without turn-taking data it becomes overly conservative. Removing context especially hurts turn-taking fluency and instruction-following in multi-turn evaluation.Source: §5.4 · pp. 8–9, Table 3 · p. 8

5.5 Case Study

• In a Daily-task case study, Moshi + RL shows smooth turn transitions and well-timed backchannels, unlike the original Moshi’s long delays and greater speech overlap.The example uses Moshi + RL trained on Fisher.Source: §5.5 · p. 9

6 Conclusion

• The authors extracted human-conversation segments for pause handling, turn-taking, backchanneling, and user interruption, then optimized Moshi and PersonaPlex with axis-specific and LLM-judge-based semantic rewards.Source: §6 · p. 9
• The method outperformed baselines on pre-recorded static and multi-turn evaluations, and human raters perceived the improved interactivity as more natural.The evaluations covered Moshi and PersonaPlex.Source: §6 · p. 9
• The authors identify instruction-following and reasoning rewards as future directions.Source: §6 · p. 9

Limitations

• Axis-specific rule-based rewards require manual engineering, may overlook other conversational dynamics, and become increasingly difficult to scale as the number of axes grows.The authors suggest exploring data-driven, scalable reward modeling.Source: Limitations · p. 9
• The GRPO objective is computed over the parallel text-token stream; applying it to audio-token probabilities for architectures without that stream remains empirically unverified.Source: Limitations · p. 9

A Training Details

• Each of 100 epochs samples 32 segments, with G = 16 completions per segment; each group is assigned to one of 32 H100 GPUs and yields 16 gradient updates.Source: A Training Details · p. 13
• Training uses AdamW with learning rate 2 × 10^-7, cosine scheduling, gradient clipping at 2, and KL penalty coefficient β = 0.01.Text and audio sampling temperatures are 0.7 and 0.8, respectively; audio top-k is 250.Source: A Training Details · p. 13
• Context is prepended with probability 0.5 using a duration sampled from [0, lmax] seconds, while lmax increases linearly from 0 to 30 over 100 epochs.Source: A Training Details · p. 13

B.1 Full-Duplex-Bench v1

• Full-Duplex-Bench v1 defines a takeover as model speech other than silence or a short backchannel, and reports Takeover Rate as the proportion of samples with a takeover.Source: §B.1 · p. 13
• For pause handling, the benchmark measures Takeover Rate downward; smooth turn-taking uses Takeover Rate upward and response latency downward.Source: §B.1 · p. 13
• Backchanneling uses takeover rate, backchannel frequency, and timing-distribution JSD; user interruption uses takeover rate, response latency, and a GPT-4o semantic score.Source: §B.1 · p. 13
• The v1 evaluation follows the official pipeline and default test samples, with a correction to the backchanneling script’s audio resampling to 16 kHz.Source: §B.1 · p. 13

B.2 Full-Duplex-Bench v2

• Full-Duplex-Bench v2 evaluates real-time multi-turn interaction through staged goals in Daily, Correction, Entity Tracking, and Safety task families.Source: §B.2 · pp. 13–14
• The v2 benchmark uses 1–5 LLM-judge scores for turn-taking fluency and instruction following, plus task-specific competence for Correction, Entity Tracking, and Safety.The settings use GPT-Realtime as examiner, a 60-second maximum duration, and fast pacing; transcript scores use Gemini 2.5 Flash, and sessions with no model speech receive minimum scores.Source: §B.2 · p. 14

C Evaluation on MTR-DuplexBench

• MTR-DuplexBench evaluates four tasks using task success rate, response latency, and backchannel frequency in a multi-turn benchmark independent of the Full-Duplex-Bench series.The authors implemented Moshi inference themselves because the official evaluation code was not publicly available.Source: C Evaluation on MTR-DuplexBench · p. 14
• On MTR-DuplexBench, Moshi + RL outperforms base Moshi on nearly all metrics; Background-task success rate is the exception.The authors note that Background Speech was absent from the training segments and rewards, which likely limited generalization.Source: C Evaluation on MTR-DuplexBench · p. 14

D Mitigating Safety Degradation

• Safety-aware LLM judging recovers PersonaPlex’s Safety scores to the base level or above without changing the Fisher training data.The reward prompt instructs the judge to give the highest score to a clear, natural refusal of harmful requests.Source: D Mitigating Safety Degradation · pp. 14–15, Figure 5 · p. 15

• UTMOSv2 scores remain comparable to baselines for both models, while WER and matched-word onset delay slightly decrease after RL.The evaluations use Turn-Taking and User Interruption outputs from Full-Duplex-Bench v1.Source: Table 6 · p. 15, E Speech Quality and Text–Speech… · p. 15, Table 7 · p. 16
• The authors attribute this stability to audio-derived VAD rewards, ASR-based LLM judging, and KL regularization toward the pretrained policy.Although the GRPO objective updates text tokens only, the reward pipeline uses decoded audio and its ASR transcription.Source: E Speech Quality and Text–Speech… · pp. 15–16

F Human Evaluation Setup

• The study sampled 10 dialogues from each of seven tasks, yielding 70 pairs with identical user speech and different system recordings.Dialogue durations were restricted to 10–30 seconds for v1 and at most 60 seconds for v2.Source: F Human Evaluation Setup · p. 16
• Ten raters judged each task’s 10 pairs by choosing A, B, or tie, totaling 100 judgments per task and 700 overall.The v1 questions targeted each task’s interactivity axis; v2 used one question about conversational naturalness.Source: F Human Evaluation Setup · p. 16, Figure 6 · p. 16

G.1 Interactivity of Moshi Baseline

• In one Daily-task example, baseline Moshi waits about 8 seconds and overlaps extensively, whereas RL produces smoother turn-taking without excessive overlap.Figure 7 reports turn-taking fluency 2.75 and instruction-following 2.50 for the baseline example.Source: §G.1 · p. 16, Figure 7 · p. 16

G.2 Safety Degradation of PersonaPlex + RL

• The base model maintains structured turn transitions despite failing to refuse, while Fisher-trained RL produces cooperative fragments incompatible with harmful-request refusal.The authors connect this cooperative bias to the Safety-score decline for PersonaPlex trained on Fisher.Source: §G.2 · p. 17, Figure 8 · p. 17