PaperlayerView exact revision on arXiv

Source-linked AI summary

SteerDuplex: Steerable Duplex Speech Dialogue Models

Utkarsh Tyagi, Ramaneswaran Selvakumar, Advait Gosai, Sonal Kumar, Nikhil Barhate, Isabell Sagar, Steven Li, Miheer Bavare, Daniel Quigley, Fabiola Tapia Carrillo, Jose M Patron E, Diego Macías Gutiérrez, Paul Song, Ramani Duraiswami, Dinesh Manocha, Yunzhong He

arXiv:2609.12623v1cs.AIcs.CL

TL;DR

Full-duplex models can manage turn-taking and interruptions, but reliable instruction-based steering of conversational behavior remains underexplored. The paper introduces a steerability taxonomy, SteerDuplex, and SteerBench, then reports supervised steering gains and RL interaction refinements alongside reward-related limitations.

  • Problem

    Full-duplex systems can handle turn taking and interruptions, but whether they reliably follow instructions about tone, persona, and delivery remains an open question.

    Source: §1 · pp. 1–2

  • Method

    The paper combines a taxonomy and benchmark for spoken steering with MOSHI-based supervised fine-tuning and two-stage RL using continuity-aware hybrid rewards.

    Source: §1 · pp. 2–3, §3 · p. 4

  • Results

    Supervised fine-tuning supplies the main steerability gains, while RL refines interaction behavior and improves interruption response and pause handling with comparable or higher steering and aggregate task scores.

    Source: §1 · p. 3, §9 · p. 12

  • Takeaways & Limitations

    Explicit continuity rewards reduce timing credit from short or absent responses but do not eliminate the conflict between yielding and continuing, so conversational events should be evaluated alongside aggregate task success.

    Source: §9 · p. 12

    Rubric rewards are imperfect proxies that depend on LLM-judge recognition and reference-audio comparison, and automated safety checks do not establish deployment safety.

    Source: §8 · p. 11

Abstract

from arXiv · show

Full-duplex spoken dialogue models support low-latency turn taking, interruption handling, and backchanneling, yet a key capability remains underexplored: steerability, the ability to reliably shift conversational behavior along attributes such as tone, persona, speaking rate, and voice style in response to user instructions. We introduce a taxonomy of text- and audio-based steerability that identifies substantial gaps in current full-duplex models. To address this gap, we introduce SteerDuplex, a Moshi-based full-duplex speech model fine-tuned on natural conversations and synthetic dialogues targeting instruction following, vocal delivery, reasoning, and duplex interaction. We further apply two-stage reinforcement learning (RL) with hybrid rewards, combining verifiable interaction checks and judge-based semantic feedback to improve timing and response continuity. To evaluate full-duplex spoken steerability, we introduce SteerBench, a benchmark with 390 spoken prompts and 1,067 human-authored binary audio and text rubrics spanning tone, persona, style/accent, and speed/length. On SteerBench, supervised training improves audio-steering average pass rate by 44.5 percentage points over the strongest evaluated open baseline. On Audio MultiChallenge, task average pass rate improves by 7 points over its strongest evaluated open baseline. RL further raises source-clean interruption response from 72.5% to 82.5% and reduces synthetic pause barge-in from 26.5% to 9%. Steering and aggregate task scores remain comparable or higher, while reward probes reveal reward hacking through incomplete responses. Our model and benchmark support systematic research on spoken steerability, with reward analysis showing why timing gains must be evaluated alongside response completeness.

1 Introduction

Full-duplex systems can manage timing and interruption yet may not follow instructions about tone, persona, or vocal delivery. The paper organizes spoken steerability and introduces a benchmark and post-training approach to evaluate and improve it.

1 Introduction

  • Fluent turn taking does not guarantee that full-duplex systems follow user instructions about tone, persona, or delivery.Speech conveys emotion, accent, and timing cues that text transcripts alone do not preserve.Source: §1 · pp. 1–2
  • The taxonomy separates natural-language steering, acoustic understanding and adaptation, and duplex interaction, which require distinct checks.Task compliance and floor management require separate checks; evaluation covers requested content and delivery alongside intelligibility, timing, and task capability.Source: §1 · p. 2
  • STEERBENCH uses 390 spoken prompts and separate text and audio rubrics to assess requested content and delivery across four attribute groups.Fixed reference clips anchor requested acoustic style; the groups are tone, persona, style/accent, and speed/length.Source: §1 · p. 2
  • With the same judge and matched items, MOSHI and PERSONAPLEX achieve audio-steering average pass rates of 20.55% and 16.44%, respectively.The results motivate testing whether targeted post-training can improve steerability while preserving general conversational capability.Source: §1 · p. 3
  • STEERDUPLEX combines targeted supervised steering with continuity-aware, two-stage reinforcement learning and evaluates both alongside general task performance.The model is fine-tuned on recorded conversations and synthetic examples covering instruction following, requested delivery, reasoning, safety, and duplex interaction.Source: §1 · p. 3
  • The reward probes reveal that timing gains can mask incomplete responses, motivating evaluation of response continuity alongside timing.The approach combines programmatic interaction rewards with judge-based semantic feedback and incentives to sustain answers until the user takes the floor.Source: §1 · p. 3

2 Related Work

Prior work develops simultaneous listening and speaking, controllable behavior, spoken evaluation, and speech reinforcement learning. This paper connects these strands to study requested content and delivery alongside conversational timing.

Full-duplex speech models

  • Duplex dialogue work progressed from joint two-channel modeling to synchronization and stream interleaving, while MOSHI and later systems support real-time interaction.PERSONAPLEX adds role and voice conditioning, and F-ACTOR studies interactional control through supervised imitation.Source: §2 · p. 3

Steering and audio understanding

  • Controllable text generation covers style, sentiment, and persona, but controlling a model’s own speech delivery remains distinct from audio understanding and reasoning.Speech steering needs acoustic grounding, beyond the broader audio understanding and reasoning capabilities studied in audio-language models.Source: §2 · pp. 3–4

Evaluating spoken interaction

  • Spoken evaluations cover instruction following, multimodal cues, revisions, and duplex timing; STEERBENCH adds explicit content and delivery rubrics for steering requests.Duplex benchmarks separately assess event timing and multi-turn task completion.Source: §2 · p. 4

Reinforcement learning for speech

  • Speech reinforcement learning addresses alignment, timing, and joint interaction behaviors, but semantic feedback must reward responsiveness without encouraging incomplete answers.The design also requires acoustic validity and timing alongside semantic feedback.Source: §2 · p. 4

3 SteerDuplex: Post-Training for Speech Control

STEERDUPLEX builds on MOSHI, using supervised fine-tuning followed by two-stage reinforcement learning with hybrid rewards. The training and reward design targets steering, interaction timing, and sustained responses.

3 SteerDuplex: Post-Training for Speech Control

  • STEERDUPLEX builds on MOSHI; supervised fine-tuning produces STEERDUPLEX-SFT, and two-stage RL with hybrid rewards refines interaction behavior.The hybrid objective combines programmatic interaction rewards with judge-based semantic rewards.Source: §3 · p. 4

3.1 Supervised Fine-Tuning

  • The supervised mixture combines natural conversations with targeted steering, instruction-following, duplex, safety, and reasoning examples.Natural speech supplies prosody, overlap, and turn-structure variation; targeted examples provide explicit instructions and complete conversation state.Source: §3.1 · p. 4

3.2 Two-Stage Reinforcement Learning

  • The first RL stage starts from the supervised checkpoint and uses response continuity to discourage short answers that satisfy timing but fail to sustain a response.It also includes interaction-timing and transcript rewards, with a 4-second first-response target.Source: §3.2 · p. 5
  • The second RL stage adds a continuation-duration bonus on noise and user-backchannel events, rewarding speech through listener feedback that does not request the floor.Dedicated backchannel sampling emphasizes these events; sampled padding-token actions retain timing credit for pauses and speech onset.Source: §3.2 · pp. 5–6

3.3 Rewards and Audio Validity

  • The RL objective combines timing, continuity, transcript-judge, and waveform-integrity rewards; waveform checks reject silence, clipping, and invalid outputs.Interruption credit requires assistant speech before the interruption, preventing silence from earning yielding credit.Source: §3.3 · p. 6
  • Reference-audio rubrics assess requested delivery and task success but do not enter the reported RL objective.Judges evaluate prosody, rate, articulation, and naturalness against fixed target clips; speaker identity matters only when explicitly requested.Source: §3.3 · p. 6

4 SteerBench: Evaluating Spoken Steerability

STEERBENCH tests spoken steering alongside concrete tasks using separate text and audio rubrics. Under matched evaluation conditions, supervised STEERDUPLEX exceeds the evaluated open baselines on audio-steering pass rate.

4 SteerBench: Evaluating Spoken Steerability

  • STEERBENCH contains 390 prompts across tone, persona, style/accent, and speed/length, with 1,067 human-authored binary audio and text rubrics.The test set is disjoint from supervised and RL training data; audio rubrics use fixed reference clips validated by human reviewers.Source: §4 · p. 6
  • Audio-steering APR requires every applicable audio rubric to pass, while sample APR additionally requires every text rubric to pass.Rubric pass rate averages individual binary decisions, distinguishing delivery control from full task compliance.Source: §4 · p. 6
  • 65.10 ± 1.13% audio-steering APR for the supervised model exceeds 20.55% for MOSHI and 16.44% for PERSONAPLEX under matched items and judging.The cited comparison is reported in Figure 3.Source: §4 · p. 6

5 Evaluation Setup

The evaluation combines spoken-task, instruction-following, and duplex-interaction benchmarks, while using source-clean data to assess generalization. Checkpoint selection uses a separate development set, not benchmark test scores.

5 Evaluation Setup

  • The evaluation covers Audio MultiChallenge, VoiceBench, and FDB, with FDB-v1 testing events, FDB-v1.5 interruption and overlap, and FDB-v2 multi-turn sessions.Repeated evaluations average three decoding runs; published baseline scores retain their source protocols.Source: §5 · p. 6
  • Passing three of four rubrics yields a 75% rubric pass rate but still fails the all-constraints pass criterion.This distinguishes individual-rubric success from complete task compliance.Source: §5 · p. 6
  • On Figure 3, SFT reaches 65.10 ± 1.13% audio-steering APR across three runs; baseline scores retain their source protocols.The same runs provide sample APR and individual-rubric pass rates, judged by Gemini 3.6 Flash.Source: Figure 3 · p. 7
  • Some official FDB-v1 turn-taking and pause examples reuse CANDOR conversations from supervised training, so those scores are treated as diagnostics.Generalization is assessed with FDB-v2, the 498-example FDB-v1.5 split without Fisher/CANDOR material, and source-clean FDB-v1 tasks.Source: §5 · p. 7
  • Checkpoint selection uses a separate development set for interaction and task capabilities, and benchmark test scores do not affect checkpoint choice.The checkpoint is frozen before benchmark evaluation.Source: §5 · p. 7

6 Results

The supervised model establishes steering and task capability; RL improves interruption response and pause handling while keeping steering and aggregate task means comparable or higher. The results also show trade-offs across interaction measures.

6.1 Multi-Turn and General Spoken Capability

  • On Audio MultiChallenge, STEERDUPLEX reaches 13.64 ± 0.28% APR and 37.17 ± 1.33% ARS, versus PERSONAPLEX at 6.64% and 20.56%.On FDB-v2, the supervised model scores 4.17 under the slow examiner and exceeds MOSHI (2.59) and PERSONAPLEX (2.65) in every task family; VoiceBench mean is 40.87 ± 0.27, versus MOSHI at 38.55 and PERSONAPLEX at 30.51.Source: §6.1 · p. 7, Table 1 · p. 8

6.2 RL Improves Interruption Response and Pause Handling

  • On source-clean FDB-v1.5, RL raises correct response after interruption from 72.5% to 82.5%; on synthetic pauses, barge-in falls from 26.5% to 9%.On the synthetic interruption task, response rate rises from 96% to 97.7%, semantic score changes from 3.94 to 3.88, and mean takeover latency increases by 40 ms.Source: §6.2 · pp. 7–8, Figure 4 · p. 8

6.3 Steering, Task Scores, and Interaction Costs

  • RL-minus-SFT FDB-v2 mean task score changes by only −0.003, with per-event turn-taking and instruction-following differences of +0.018 and +0.134.VoiceBench rises from 40.87 to 41.38, and FDB-v2 safety from 4.65 to 4.81; Figure 5 colors show point-difference direction, not statistical significance.Source: §6.3 · pp. 8–9, Figure 5 · p. 9, Table 2 · p. 9
  • RL takes over the floor less often during user backchannels than SFT, 4.8% versus 9.1%, but aggregate task scores can conceal premature yielding elsewhere.The reported task-level means do not establish that every conversational behavior is retained.Source: §6.3 · p. 9

7 Reward Design and Optimization Dynamics

The section examines response continuity, reward hacking, and sensitivity to training choices. Its findings show that timing and response-continuity objectives can trade off, while reward probes expose incomplete or unresponsive outputs.

7.2 Reward Hacking in Duplex Speech

  • In separate reward probes, promptness-only optimization leaves 25 of 96 rollouts empty and scores zero on the shared duplex diagnostic; the joint composite leaves four empty outputs.Text-only, audio-rubric-only, and audio-quality-only objectives leave 47, 41, and 41 empty rollouts, respectively; these probes are separate from the reported interaction RL runs.Source: Figure 6 · p. 10, §7.2 · p. 10

7.1 Continuity Supports Sustained Responses

  • RL averages 23.0 words per utterance across FDB-v2 daily-task runs, versus 22.1 for SFT and 20.1 for stage 1, and development continuation after backchannel rises from 2.60 to 3.20 seconds.Despite longer responses, RL ends daily-task utterances while the examiner is speaking more often than SFT, 32.3% versus 23.6%; differences in interaction pools, optimization settings, and training budgets prevent attributing gains solely to continuity rewards.Source: §7.1 · pp. 9–10

7.4 Robustness and Behavioral Trade-offs

  • A training-seed replicate reaches 9% synthetic pause barge-in and 4.22 FDB-v2 turn taking, while applying the final reward directly from SFT yields 16.8% and 4.13 with less training.The staging effect is therefore not isolated.Source: §7.4 · p. 10

7.3 Continued Optimization Can Erode Response Continuity

  • During stage 2, interruption reward on natural development conversations rises from 0.450 to 0.793, while continuation after user backchannel falls from 3.20 to 2.00 seconds and noise-robustness reward declines from 1.96 to 0.77.These measurements suggest interference between interaction objectives without isolating its mechanism; a reward component constant within a rollout group supplies no gradient to resist other components.Source: §7.3 · p. 10, Figure 7 · p. 11

8 Limitations

The evaluation is bounded by controlled English requests with fixed references, backbone capacity, and imperfect judge and interaction measurements. Several reported comparisons also have selection, overlap, or measurement constraints.

Scope and selection

  • STEERBENCH tests controlled English requests with fixed reference clips, not unrestricted personalization across languages, dialects, or recording conditions.Some fine-grained styles remain below the stronger open baseline; repeated evaluations average the three highest-scoring runs from unequal pools, while FDB-v1.5 uses one decoding pass per checkpoint.Source: §8 · p. 11

Interaction trade-offs

  • The RL checkpoint yields too readily in some FDB-v2 contexts, and the single-stage comparison cannot isolate staging because it uses fewer updates.The first-stage FDB-v1.5 row lacks an acoustic check, and latency distributions beyond means are unavailable.Source: §8 · p. 11

Backbone capacity

  • The retained Moshi 7B model, codec, and streaming architecture constrain reasoning, acoustic representation, and latency, so results measure gains within this backbone rather than its ceiling.The effects of backbone, training data, and reward design have not been isolated.Source: §8 · p. 11

Judges and deployment

  • Rubric rewards depend on judges recognizing requested characteristics, and reference-audio comparisons can favor a particular style realization.Judge errors can affect learning signals and scores; automated safety and audio checks do not establish deployment safety.Source: §8 · p. 11

9 Conclusion

The conclusion frames steerability across content, delivery, and interaction, with supervised fine-tuning supplying the main gains and RL refining interaction behavior. The appendices specify training and reward settings.

9 Conclusion

  • Explicit continuity rewards reduce timing shortcuts from short or absent responses, but do not eliminate the conflict between yielding and continuing.Premature yielding calls for evaluating conversational events alongside aggregate task success.Source: §9 · p. 12

A Training Hyperparameters

  • The supervised run uses a Moshi-style 7B backbone, 80 H100 GPUs, and 3,600 steps; the reported SFT checkpoint is step 2,925.Both RL stages leave parameters trainable, with policy gradients through the text head and shared temporal transformer; decoded speech and transcripts supply rewards, while audio-codebook actions receive no direct policy loss.Source: A Training Hyperparameters · p. 14

A.1 Reward Weights

  • The stage-1 run activates its transcript-rubric judge only on turn and interruption strata, with within-group standard deviations of 0.14–0.43 and no recorded rollout parse failures.The reward table separates within-group variation from hard validity checks and held-out evaluation metrics.Source: §A.1 · p. 14

A.2 Compute Resources

This appendix reports compute, judge and checkpoint procedures, and caveats about CANDOR overlap and pause measurement. The synthetic pause probe addresses a measurement gap in the official clips.

A.2 Compute Resources

  • 5.08 and 4.01 hours of checkpoint-run time correspond to approximately 162.7 and 32.1 allocated GPU-hours for RL stages one and two, respectively.These intervals include rollout generation, reward computation, optimization, and checkpoint writes, but exclude queueing, setup, evaluation, hosted-model compute, and preliminary or later training; RL stages inherit the preceding policy and freeze a reference copy.Source: §A.2 · p. 14, Table 3 · p. 15

B.1 Judges and Checkpoint Selection

  • SteerBench, FDB-v2, AudioMC, VoiceBench, and FDB-v1 interruption ratings use different judges, and checkpoint selection uses development performance alone before benchmark reporting.The judge-sensitivity study rescored identical generations and did not change the reported checkpoint.Source: §B.1 · p. 15

B.2 Overlap and Pause Measurement

  • Strong transcript n-gram overlap links 100 of 216 CANDOR pause transcripts to 96 supervised conversations, making CANDOR turn and pause tasks diagnostic for the training lineage.Official pause clips end only 0.02–0.11 seconds after the user’s last word, so they cannot measure waiting followed by a response; the synthetic probe supplies that continuation, and the source-clean FDB-v1.5 comparison uses 498 examples.Source: §B.2 · pp. 15–16
  • The reward table separates components with within-group variation from hard validity checks and held-out evaluation metrics.Hard checks and held-out metrics do not substitute for missing reward variation.Source: Table 4 · p. 16

C.1 Complete Retention Results

The appendix documents retention comparisons, training controls, evaluation prompts and judges, human agreement, and reward-integrity probes. The diagnostic finds that isolated reward families retain substantial empty-rollout rates, while reference conditioning improves agreement with human labels.

C.1 Complete Retention Results

  • The retention appendix reports three-run means and standard deviations, per-control differences from SFT, same-generation judge sensitivity, and second-stage optimization snapshots.The tables describe comparison scope but the supplied passages do not provide their benchmark cell values.Source: §C.1 · p. 16, Table 6 · p. 17, Table 7 · p. 17

C.2 Training Controls; C.3 Reward and Continuation Snapshots

  • Training controls vary in recipe, budget, or seed, so they do not isolate every reward component; optimization snapshots report separate observations within one run without intermediate measurements or confidence intervals.The original-recipe control omits continuity additions, and the single-stage variant applies the final reward directly from SFT with a smaller training budget.Source: §C.2 · p. 17, §C.3 · p. 17, Table 8 · p. 17

E Reward-Integrity Diagnostic

  • Every isolated reward probe leaves the empty-rollout rate at ≥26% and duplex-FDB score at or near zero (≤0.055) across 96 held-out sampled rollouts per condition.These endpoint diagnostics establish failure modes within this study, not a general necessity for a particular reward combination.Source: Table 9 · p. 18, E Reward-Integrity Diagnostic · p. 19

D.1 System Prompts

  • RL rollout prompts prepend a training system prompt, while multi-turn scenarios expose dialogue history as alternating user and assistant lines ending with the latest user turn.The assistant is instructed to answer that latest turn using the preceding context.Source: §D.1 · p. 18

D.2 Reference-Audio Rubric Judge; D.4 Text Rubric Judge

  • Reference-audio judging compares output delivery against a target clip while using transcripts to verify content; text-rubric judging scores one criterion at a time with reasoning before rating.The reference judge evaluates audio qualities from audio and does not require identical speaker timbre unless the rubric requests it.Source: §D.2 · p. 18, §D.4 · p. 19

D.3 Benchmark Judge

  • The benchmark judge receives user requests, task rubrics, and assistant transcripts, then gives transcript-grounded reasoning before a calibrated 1–5 task score.When criteria are listed, it evaluates each independently with a reason and boolean flag.Source: §D.3 · p. 19

D.5 Agreement with Human Annotations

  • Approximately 76% agreement with human annotations using reference audio versus 62% without it yields an approximately 14-percentage-point gain, with agreement still imperfect.The comparison uses 200 LLM-generated labels and 200 corresponding human annotations.Source: §D.5 · p. 19

F.1 Configurations; F.2 Benchmark Breakdowns

  • The appendix identifies public Moshi and PersonaPlex checkpoints, notes that hosted-model protocols differ, and points to expanded comparison tables for RL and hosted-API context.The hosted AudioMC scores use text output for Gemini 2.5 Flash and audio output for GPT Realtime; protocol differences limit comparisons with open models.Source: §F.1 · p. 19, §F.2 · p. 20

G Data and Responsible Development

The appendices document SteerBench construction and scoring, training and interaction data, contributor safeguards, and the limits governing evaluation and artifact reuse.

G Data and Responsible Development

  • The supervised mixture contains 504,416 audio and 65,675 text records, with 8,510.9 audio hours and a fixed 2.304M sample-draw budget.Targeted subsets include instruction-following, steering, duplex, and six-turn mathematics records.Source: G Data and Responsible Development · pp. 20–21
  • Interaction data spans turns, interruptions, pauses, backchannels, noise, and speech-mirror scenarios; evaluation overlaps are filtered and test prompts and labels are excluded from optimization.Contributors were trained and their submissions independently reviewed; recordings required speaker permission and applicable ethics approval.Source: G Data and Responsible Development · p. 21, §G.1 · p. 21

G.2 Demographic and Linguistic Coverage

  • Evaluation is English-only and does not claim coverage of non-English languages, code-switching, children, clinical speech, or all regional dialects; safety measures reduce but do not eliminate misuse risks.Deployment still requires identity, consent, abuse-monitoring, and content-safety controls.Source: §G.2 · p. 21, §G.4 · p. 21

G.3 Artifact Licenses and Terms

  • Restricted corpora and benchmark assets are not redistributed, while code, SteerBench, and checkpoints are slated for research-license release upon acceptance; authors reviewed AI-assisted writing.Audio release follows consent and source terms.Source: §G.3 · p. 21, Table 15 · p. 22, §G.5 · p. 22

H SteerBench Documentation

  • SteerBench covers 390 single-turn English prompts across four steering categories, with separate binary text and audio rubrics and fixed reference clips.Its prompts comprise 100 tone, 136 persona, 104 style/accent, and 50 speed/length items.Source: H SteerBench Documentation · p. 22
  • Synthetic benchmark audio uses neutral user speech and fixed-pool voices, while reference scripts combine the request, steering category, and rubrics into transcript and delivery instructions.Fixed reference clips are reused across models, though regeneration can change the target.Source: H SteerBench Documentation · p. 22, §H.2 · p. 23
  • Audio and text criteria are judged separately, while item-level pass rates require all applicable criteria to pass and rubric pass rate counts individual decisions.Judge-call errors are excluded from denominators, and runs with more than 5% such errors are invalid.Source: H SteerBench Documentation · p. 22
  • Reference-script instructions encode tone in delivery notes, persona in transcript phrasing and vocal style, and style, accent, speed, or length through tailored transcript and speech directions.Length targets range from one short sentence to about five or six sentences.Source: §H.2 · pp. 23–24
Loading 2609.12623v1…