PaperlayerView exact revision on arXiv

Source-linked AI summary

DuraS2ST: Chain-of-Thought and Reinforcement Learning for Duration-Aligned Speech-to-Speech Translation

Yayue Deng, Dingdong Wang, Yuxuan Hu, Jinyu Li, Yanqing Liu, Yuanyuan Wang, Weidong Chen, Helen M. Meng, Shujie Liu, Xixin Wu

arXiv:2609.33742v1cs.SDcs.CLeess.AS

TL;DR

Time-sensitive speech-to-speech translation needs duration consistency, but existing systems generally lack explicit temporal planning. DuraS2ST plans target wording and phonetic length before speech generation, then optimizes with multi-modal reinforcement learning; on CVSS-T it achieves a strong balance across translation quality, acoustic preservation, and duration consistency.

  • Problem

    Most S2ST systems generate target speech without explicit temporal control, although duration mismatch can desynchronize audio and video, overlap speech, and degrade user experience in time-sensitive settings.

    Source: §1 · p. 1

  • Method

    DuraS2ST uses a duration-aligned CoT corpus to train explicit planning of target content and phonetic length before speech generation, followed by multi-modal reinforcement learning with DMR and MARA.

    Source: §1 · pp. 1–2

  • Results

    Across CVSS-T, DuraS2ST achieves the strongest balance across translation quality, acoustic preservation, and duration consistency; on EN-ZH, DuraS2ST-Think-RL improves AVGBLEU from 27.25 to 30.95.

    Source: §4.3 · pp. 6–7

  • Takeaways & Limitations

    The results support treating duration alignment as explicit planning of target wording and phonetic length rather than acoustic post-processing.

    Source: §5 · p. 8

    Evaluation covers bidirectional Chinese-English, and decoding the full CoT before acoustic output fits offline dubbing but does not yet support streaming or simultaneous interpretation.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Speech-to-speech translation (S2ST) in time-sensitive applications such as video dubbing requires not only semantic fidelity and speaker preservation, but also strict duration consistency to avoid audio-visual misalignment. However, existing S2ST systems largely generate target speech without explicit temporal planning, making duration control an unresolved challenge. We introduce DuraS2ST, a duration-aligned reasoning framework that enables a single speech language model to first generate an explicit chain-of-thought (CoT) for planning target wording and phonetic length, and then synthesize the corresponding speech tokens. To support this paradigm, we construct DuraSet-440K, a high-quality duration-aligned CoT corpus for supervised initialization. We further optimize the model with multi-modal multi-dimensional reinforcement learning, using a Duration Margin Reward to balance translation quality and duration consistency, and Modality-Aware Reward Attribution to assign rewards to appropriate token spans. Experiments on CVSS-T show that DuraS2ST achieves a strong balance between translation quality and duration consistency, outperforming competitive open-source and commercial baselines. Project page: https://github.com/Mia11939/DuraS2ST.

1 Introduction

DuraS2ST treats duration alignment as a planning problem: it reasons about target content and phonetic length before generating speech. On CVSS-T, it achieves a strong balance between translation quality and duration consistency while outperforming competitive baselines.

1 Introduction

  • Most existing S2ST systems lack explicit temporal control, and duration mismatch in dubbing or simultaneous interpretation can desynchronize audio and video, overlap speech, and degrade user experience.S2ST also aims to preserve speakers’ acoustic characteristics while translating across languages.Source: §1 · p. 1
  • Prior isochronous S2ST methods use cascaded post-hoc time-stretching or end-to-end speed, length, or duration controls, but typically treat duration as a low-level acoustic constraint.The paper contrasts this with deliberate paraphrasing before articulation in human dubbing.Source: §1 · p. 1
  • DuraS2ST plans target content and its phonetic footprint in an explicit reasoning trajectory before generating the final interleaved text-acoustic sequence.The paper frames duration alignment as a planning problem rather than acoustic post-processing.Source: §1 · pp. 1–2
  • The two-stage training uses DuraSet-440K for supervised planning of semantic content and phonetic length, then applies multi-modal reinforcement learning with DMR and MARA.DMR balances translation quality and duration consistency; MARA assigns rewards to modality-specific token spans to prevent cross-modality contamination.Source: §1 · p. 2
  • On CVSS-T, DuraS2ST achieves a strong balance between translation quality and duration consistency, outperforming competitive open-source and commercial baselines, with substantial gains in zero-shot RL.Source: §1 · p. 2

2 DuraS2ST

DuraS2ST treats duration alignment as a reasoning problem: a single multimodal language model plans target content and phonetic length before synthesizing speech. It trains this behavior with supervised fine-tuning followed by reinforcement learning that combines duration-aware rewards with modality-specific credit assignment.

2 DuraS2ST

  • DuraS2ST uses a single pre-trained multimodal language model to plan target content and phonetic length before producing speech tokens.The framework frames duration alignment as planning rather than acoustic post-processing.Source: §2 · p. 2, Figure 2 · p. 3

2.1 Phase I: SFT Training

  • Supervised fine-tuning on DuraSet-440K trains the model to generate a CoT rationale and then an interleaved text-acoustic sequence in one autoregressive output.The rationale selects target words and sentence structures to balance semantic correspondence with source-speech duration; the resulting speech tokens are conditioned on that rationale.Source: §2.1 · pp. 2–3

2.2 Phase II: GRPO Training

  • GRPO optimizes translation-quality, duration, and format signals because SFT cannot directly optimize non-differentiable sequence-level objectives.The RL design addresses the risk that strict duration matching over-penalizes small deviations and that sequence-level duration penalties affect reasoning tokens.Source: §2.2 · p. 3

2.2.1 Multi-Modal Multi-Dimensional Reward Design

  • Duration Margin Reward treats alignment as a bounded feasible region, smoothly rewarding near-target durations while tolerating minor deviations.Its reward decreases with duration gap, avoiding over-penalization of small differences and focusing learning on recoverable mismatches.Source: §2.2.1 · p. 4

2.2.2 Modality-Aware Reward Attribution

  • MARA computes group-relative advantages by reward component and assigns them to the token spans each component supervises.Translation-quality advantage applies to reasoning and acoustic spans, while duration advantage applies only to the acoustic span.Source: §2.2.2 · p. 4

3 DuraSet-440K

DuraSet-440K is a duration-aligned bilingual S2ST corpus built from English and Chinese LEMAS speech. Its pipeline selects and synthesizes aligned speech pairs, then adds reasoning annotations and bidirectional examples.

3 DuraSet-440K

  • DuraSet-440K contains 440,705 parallel utterances and roughly 1,000 hours of speech, designed for duration-aligned, translation-faithful, speaker-consistent target speech.The corpus is built from English and Chinese subsets of LEMAS using duration-controllable synthesis rather than post-hoc time-stretching.Source: §3 · p. 4
  • Figure 3 overviews the DuraSet-440K construction pipeline.Source: §3 · p. 4, Figure 3 · p. 5
  • The pipeline selects and filters English and Chinese LEMAS utterances, generates translation candidates, synthesizes and selects aligned speech pairs, then annotates and swaps retained pairs.Candidate selection considers transcription accuracy, duration alignment, and speaker similarity; synthesis uses source audio as a timbre prompt.Source: §3 · p. 5

4 Experiments

DuraS2ST is evaluated on CVSS-T for translation quality, acoustic preservation, and duration consistency. Its results show a strong balance across these dimensions, with subjective evaluation and Zero-GRPO analysis providing additional evidence.

4.1 Training Setup

  • DuraS2ST builds on Step-Audio-2-mini-Think and uses two-phase training, with Phase II GRPO for 1,000 steps at learning rate 1.5e−5 and KL coefficient 0.001; rewards combine BLEU/COMETKiwi quality and duration consistency.The duration reward uses a margin-based formulation with tolerance τ=5%, and MARA converts rewards into independent group-relative advantages. The format gate zeroes both rewards for malformed rollouts.Source: §4.1 · p. 5

4.2.1 Test Data

  • The out-of-domain CVSS-T test set contains 4,897 utterance pairs, totaling 8.2 hours of Chinese and 6.3 hours of English speech.It is disjoint from DuraSet-440K.Source: §4.2.1 · p. 5

4.2.2 Metrics

  • Evaluation measures translation quality, voice and prosody preservation, and duration consistency using BLEU/COMET, A.PCP/SECS, and SLC, MADE, and MRDE metrics.SLC-0.2 and SLC-0.4 measure the fraction of samples within ±20% and ±40% target-to-source duration ratios.Source: §4.2.2 · p. 6

4.2.3 Baselines

  • Comparisons include open-source and commercial S2ST systems, backbone variants isolating SFT, CoT, and GRPO, and inference using the official Step-Audio-2 sampling configuration.Generated acoustic tokens are converted to waveforms with Step-Audio-2’s token2wav module.Source: §4.2.3 · p. 6, §4.2.4 · p. 6

4.3 Main Results

  • DuraS2ST achieves the strongest balance across translation quality, acoustic preservation, and duration consistency on CVSS-T, with the highest overall translation performance on ZH-EN.On EN-ZH, Think-RL raises AVGBLEU from 27.25 to 30.95; its SLC-0.2 scores rise from 0.487 to 0.997 on EN-ZH and 0.305 to 0.934 on ZH-EN versus the backbone.Source: §4.3 · pp. 6–7, Table 1 · p. 7, Table 2 · p. 7
  • GRPO consistently improves translation metrics in both directions while preserving or further improving duration consistency; on ZH-EN, AVGBLEU rises from 24.88 to 27.30.MRDE falls from 0.039 to 0.038 on EN-ZH and from 0.079 to 0.068 on ZH-EN.Source: §4.3 · p. 7

4.4 Subjective Evaluation

  • Table 3 reports subjective MOS (1–5, higher is better) on CVSS-T; Think and Think-RL variants are consistently preferred or highly competitive, with DuraS2ST-Think-RL best overall on ZH-EN and competitive on EN-ZH.Source: §4.4 · p. 8, Table 3 · p. 8

4.5 Analysis

  • Without DuraSet-440K SFT, Zero-GRPO reduces duration errors, and removing the duration reward weakens gains, supporting sequence-level duration optimization.Figure 4 tracks duration, BLEU, COMETKiwi, and total rewards over 1,000 GRPO steps.Source: §4.5 · p. 8, Figure 4 · p. 8

5 Conclusion

DuraS2ST plans target wording and phonetic length before synthesizing speech, supported by DuraSet-440K and two-stage SFT plus multimodal GRPO. On CVSS-T it balances translation quality and duration consistency, while a zero-SFT ablation shows reward-driven reductions in MADE.

5 Conclusion

  • DuraS2ST balances translation quality and duration consistency on CVSS-T, while zero-SFT results show rewards reduce MADE beyond duration-aligned SFT.The Duration Margin Reward uses a saturated soft constraint, and Modality-Aware Reward Attribution restricts duration advantage to acoustic tokens.Source: §5 · p. 8

Limitations

The evaluation is limited to bidirectional Chinese-English translation, and the think-then-speak architecture is designed for offline use rather than streaming. The authors identify broader language validation and finer-grained interleaving of reasoning and acoustic emission as future work.

Limitations

  • Evaluation covers only bidirectional Chinese-English translation, leaving broader linguistic validation for future work.The authors describe Chinese and English as a representative pair of structurally divergent languages.Source: Limitations · p. 9
  • Because the model completes its chain-of-thought before emitting audio, it supports offline dubbing but not streaming or simultaneous interpretation.Future work will explore finer-grained interleaving of reasoning and acoustic emission.Source: Limitations · p. 9

Ethical Considerations

The authors flag potential misuse of synthetic speech with speaker preservation and state consent, licensing, and safeguards as conditions for application.

Ethical Considerations

  • Speaker-preserving speech synthesis may be misused for impersonation, unauthorized voice conversion, or misleading audio generation, so the authors call for consent, licensing, and safeguards.Experiments use public research datasets and anonymized samples, with human evaluation participants not exposed to identifying information beyond benchmark speech.Source: Ethical Considerations · p. 9

Supplementary Material

The supplementary material reports an auxiliary GRPO filtering experiment and details the DuraSet-440K construction pipeline, including candidate generation, synthesis, filtering, and phoneme-grounded rationales.

A.1 Pre-Rollout Filtering

  • Pre-rollout filtering discards prompts with near-zero reward variance and keeps all medium-reward prompts plus a small fraction of easy prompts; on ZH→EN, it improves reward curves over unfiltered training.The two GRPO runs share reward composition, rollout budget, and hyperparameters; Figure 5 shows faster climbs and higher plateaus across four reward components.Source: §A.1 · p. 11, Figure 5 · p. 12

B.1 Translation Candidate Construction

  • For each source utterance, GPT-4o generates five stylistically varied translation candidates to vary wording and phonetic footprint while preserving core semantics.The prompt spans verbose to concise styles.Source: §B.1 · p. 11

B.2 Target Speech Synthesis Configuration

  • Candidates are synthesized with zero-shot duration control at 24 kHz, using source audio as a timbre prompt; frame scaling is clipped to [0.5, 2.0] to limit excessive compression or expansion.Clipping also preserves measurable residual duration errors for rejecting candidates with incompatible phonetic lengths.Source: §B.2 · p. 11

B.3 Multi-Dimensional Filtering

  • Candidates must pass transcription-fidelity, duration-alignment, and speaker-similarity filters, after which selection minimizes duration error and uses similarity and WER as tie-breakers.The thresholds are τwer = 0.05, τabs = 5 s, τrel = 0.20, and τsim = 0.86.Source: §B.3 · pp. 11–12

B.4 Reasoning Rationale Construction

  • Each example receives a phoneme-grounded rationale that connects translation choices to phonetic footprints and duration constraints using source and target phoneme sequences and their lengths.The sequences are deterministically extracted with the BigCiDian lexicon using longest-match and mapped to IPA; GPT-4o rationales follow a 150–300-word structured template.Source: §B.4 · p. 12

B.5 Dataset Statistics

DuraSet-440K contains 440,705 parallel utterances across English-to-Chinese and Chinese-to-English directions. Its source and target durations are closely aligned, supporting its use for duration-aware S2ST training.

B.5 Dataset Statistics

  • DuraSet-440K contains 440,705 parallel utterances: 229,410 En→Zh and 211,295 Zh→En examples.Source: §B.5 · p. 12
  • The translation-candidate prompt encourages stylistically diverse options with different lexical densities and temporal footprints for each source utterance.Source: Figure 7 · p. 13
  • Target-to-source duration ratios center at µ = 0.995 with σ = 0.008, and SLC0.2 reaches 100.0%.Most source-target duration pairs cluster around the dtgt = dsrc diagonal.Source: §B.5 · p. 13

C.1 Training Hyperparameters

The appendix details the two training phases, online reward computation and attribution, baseline inference protocols, and evaluation metrics. Training combines supervised CoT-then-speech fine-tuning with GRPO optimization.

C.1 Training Hyperparameters

  • SFT fine-tunes Step-Audio-2-mini-Think for 4,000 steps on eight A100 GPUs, then GRPO trains LoRA adapters for 1,000 steps from the SFT checkpoint.The SFT sequence limit is 16,384 tokens; GRPO samples 16 rollouts per input.Source: §C.1 · pp. 13–14

C.2 RL Reward Implementation

  • Online rewards use the interleaved TA4 stream’s in-segment text tokens as the post-CoT translation, avoiding ASR during training.Source: §C.2 · p. 14
  • The Duration Margin Reward gives high reward below 5% duration gaps, focuses learning on the 5%–15% range, and saturates for larger deviations.Its constants were tuned on a small held-out DuraSet-440K subset.Source: §C.2 · p. 14
  • A deterministic format gate zeros quality and duration rewards for malformed outputs; MARA applies quality advantages to reasoning and acoustic spans but duration advantages only to acoustic tokens.Source: §C.2 · p. 14

C.3 Baselines

  • External baselines use default decoding with fixed direction-specific instructions and no extra preprocessing or post-processing; Qwen2.5-Omni filler artifacts remain in its waveform and affect reported metrics.Source: §C.3 · pp. 14–15

C.4 Evaluation Metrics

  • Evaluation groups measures into translation quality, voice/prosody preservation, and duration consistency, with corpus-level scores on the CVSS-T test split.Translation measures include text and ASR-based Speech-BLEU and COMET variants; voice/prosody uses A.PCP and SECS.Source: §C.4 · pp. 15–16
  • SLC-0.2 and SLC-0.4 count utterances within ±20% and ±40% duration ratios, while MADE and MRDE provide finer-grained duration errors.Source: §C.4 · p. 16

D Evaluation Examples

The qualitative evaluation compares randomly selected CVSS-T test utterances with a representative baseline in each translation direction. The examples show DuraS2ST tracking source duration while preserving reference-consistent translations.

D Evaluation Examples

  • The qualitative comparison randomly selects two CVSS-T test utterances per direction and jointly visualizes synthesized waveforms, duration alignment, and translation fidelity.Source: D Evaluation Examples · p. 16
  • In the ZH→EN example, reasoning over semantic content and phonetic length helps DuraS2ST follow the source temporal envelope while keeping the translation faithful to the reference.Source: D Evaluation Examples · p. 16
  • The examples are presented as evidence that reasoning-based generation and duration-aware optimization generalize to held-out test utterances.Source: D Evaluation Examples · p. 16

E Subjective Evaluation Interface

The subjective study uses anonymized system labels and randomized presentation to collect ratings of translation adequacy, naturalness, and speaker similarity. Qualitative figures contrast baseline outputs with DuraS2ST in both translation directions.

E Subjective Evaluation Interface

  • Across 160 samples and eight systems, annotators rate translation adequacy, naturalness, and speaker similarity on separate 1–5 scales using anonymized labels.Sample order is randomized for each annotator.Source: E Subjective Evaluation Interface · p. 16, Figure 11 · p. 20
  • In EN→ZH examples, baseline outputs visibly drift from the source duration, while DuraS2ST tracks duration tightly and remains faithful to the reference.Source: Figure 9 · p. 18
  • In ZH→EN examples, DuraS2ST closely matches source duration and produces reference-consistent translations, while the baseline drifts in duration and content.Source: Figure 10 · p. 19
Loading 2609.33742v1…