PaperlayerView exact revision on arXiv

Source-linked AI summary

SimPOTrans: Self-Adaptation for Offline-to-Simultaneous Speech Translation via Reinforcement Learning

Haotian Tan, Hiroki Ouchi, Sakriani Sakti

arXiv:2610.35042v1cs.CL

TL;DR

Dialogue-style simultaneous speech translation often depends on synthesized training trajectories or explicit joint optimization of translation quality and latency. SimPOTrans uses two-stage reinforcement learning to adapt offline speech translation models from partial speech inputs, and experiments report stronger quality–latency trade-offs than strong LLM-based baselines. At inference, chunk size controls the trade-off without an external decision policy.

  • Problem

    Dialogue-style SimulST approaches rely on synthesized read/write trajectories that may provide suboptimal signals or joint quality–latency optimization whose competing objectives may destabilize training.

    Source: Introduction · p. 1

  • Method

    SimPOTrans applies two-stage reinforcement learning to an offline speech translation model, first improving complete-utterance translation and then learning from partial inputs and self-generated dialogue trajectories.

    Source: Introduction · p. 1, Proposed Method · p. 2

  • Results

    Experiments on ACL 60/60 and MCIF across three language directions show superior quality–latency trade-offs over strong LLM-based baselines.

    Source: Conclusion · pp. 7–8

  • Takeaways & Limitations

    At inference, SimPOTrans requires no external decision policy, and speech-input chunk size provides flexible quality–latency control.

    Source: Introduction · p. 2

    The method uses fixed-duration speech chunks that may split coherent units, and the evaluation covers only speech-to-text translation.

    Source: Limitations · p. 7

Abstract

from arXiv · show

Simultaneous speech translation (SimulST) aims to translate ongoing speech with low latency while preserving translation quality. Recent studies have adopted large language models (LLMs) for SimulST and formulated it as a multi-turn dialogue task for efficient generation, achieving state-of-the-art performance. However, existing dialogue-style SimulST approaches typically rely on specialized training schemes to acquire simultaneous translation behavior, either using synthesized read/write trajectories, which may provide suboptimal training signals, or jointly optimizing translation quality and latency, whose competing objectives may cause optimization instability. In this paper, we introduce SimPOTrans, a reinforcement learning (RL) framework that enables offline speech translation models to self-adapt to dialogue-style SimulST. Through a simple two-stage RL post-training pipeline, SimPOTrans rewards an offline model for exhibiting translation behaviors beneficial to simultaneous scenarios when given partial speech inputs, thereby enabling offline-to-simultaneous self-adaptation without synthesized read/write trajectories or explicit latency optimization. At inference time, SimPOTrans requires no external decision policy and enables flexible quality--latency control by simply adjusting the chunk size of speech input. Experiments on the ACL 60/60 and MCIF benchmarks across three language directions (En$\rightarrow$De, En$\rightarrow$Zh, and En$\rightarrow$Ja) demonstrate that SimPOTrans outperforms several strong baselines, achieving a superior quality--latency trade-off.

Introduction

Earlier dialogue-style SimulST methods rely on synthesized read/write trajectories or joint quality–latency optimization. SimPOTrans instead uses two-stage reinforcement learning to adapt offline speech translation models, with chunk size providing inference-time quality–latency control.

Introduction

  • Existing dialogue-style SimulST approaches use potentially suboptimal synthesized trajectories or jointly optimize quality and latency, which may destabilize optimization.LLM-based methods formulate translation as a multi-turn dialogue and can reuse the KV cache for efficient generation.Source: Introduction · p. 1
  • SimPOTrans uses reinforcement learning to adapt offline speech translation models to dialogue-style SimulST by reinforcing behaviors they exhibit on partial speech inputs.The framework uses Group Relative Policy Optimization (GRPO) and self-generated dialogue-style trajectories.Source: Introduction · pp. 1–2
  • Its two-stage post-training first strengthens translation on complete utterances, then trains on partial speech inputs to learn what and how to translate at each turn.The approach uses standard offline speech translation data without synthesized read/write trajectories or an explicit latency objective.Source: Introduction · pp. 1–2
  • At inference, SimPOTrans needs no external decision policy, and varying speech-input chunk size controls the quality–latency trade-off.The framework is presented as improving translation quality while inducing behavior better suited to simultaneous scenarios.Source: Introduction · p. 2

Related Work

Prior work builds dialogue-style SimulST with synthesized trajectories, applies RL under additional reward or supervision requirements, or uses inference-time policies on offline models. SimPOTrans instead adapts an offline model without synthesized training data, temporal supervision, or explicit latency optimization.

Dialogue-Style SimulST

  • Dialogue-style SimulST uses interleaved speech segments and incremental translations, but synthesized trajectories can contain segmentation or alignment errors.InfiniSST constructs trajectories from word alignments, while EASiST and CLASI prompt LLMs to synthesize them; the dialogue setup also permits KV-cache reuse across turns.Source: Dialogue-Style SimulST · p. 2

Reinforcement Learning for SimulST

  • Prior SimulST reinforcement-learning methods require human interpretation data, temporal supervision, or synthesized trajectories and explicit quality–latency optimization.CLASI is reported to face optimization instability and reward hacking from balancing rewards; Hibiki-Zero uses time-indexed reference-prefix BLEU rewards.Source: Reinforcement Learning for SimulST · p. 2

Offline Models for SimulST

  • Inference-time policies let offline models serve different latency needs, but depend on policy design and do not train the model to handle partial speech.SimPOTrans instead learns from partial speech and supports latency regimes by varying input chunk size.Source: Offline Models for SimulST · p. 2

Proposed Method

SimPOTrans adapts an offline speech translation model to dialogue-style SimulST through two RL stages: optimizing complete-utterance quality, then training on partial-input multi-turn trajectories with quality and monotonicity rewards and a repetition penalty.

Proposed Method

  • SimPOTrans adapts a conventional offline speech translation model to dialogue-style SimulST through a two-stage RL policy optimization procedure.Source: Proposed Method · p. 2

Supervised Fine-Tuning for Offline Translation

Two-Stage Policy Optimization

  • RL-Stage I improves sequence-level translation quality on complete utterances using COMET rewards, while keeping the speech encoder frozen and updating the adaptor and LLM LoRA parameters.Source: Stage I: Offline Translation… · pp. 3–4
  • RL-Stage II divides speech into fixed-duration chunks, generates turn-wise translation segments conditioned on preceding dialogue, and concatenates them into a complete translation.Source: Stage II: Self-Adaptation to SimulST · p. 4
  • Stage II rewards complete-translation quality and source-aligned monotonicity while penalizing excess n-gram overlap between adjacent output segments.Because quality is scored on the complete translation, chunk-level references are not required.Source: Stage II: Self-Adaptation to SimulST · p. 4

SimulST Inference for Unbounded Speech

  • At inference, varying fixed speech-chunk duration controls the quality–latency trade-off, with context resets triggered by the specified 20- and 30-second conditions.After 20 seconds, reset when the latest translation ends with sentence punctuation; if speech would exceed 30 seconds before then, reset before the next chunk.Source: SimulST Inference for Unbounded Speech · p. 4

Experimental Setup

The experiments use CoVoST 2 training data and evaluate three English-to-X directions on long-form scientific speech benchmarks. They compare model variants and SimulST systems using translation-quality metrics and LongYAAL latency.

Data

  • Training uses approximately 430 hours per direction from CoVoST 2; evaluation covers ACL 60/60 talks and the MCIF development set of long-form scientific talks.The directions are En→De, En→Zh, and En→Ja; MCIF serves as development data for the IWSLT 2026 simultaneous speech translation track.Source: Data · pp. 4–5

Model Configuration

  • The backbone combines Whisper-large-v3, a two-layer MLP adaptor, and LoRA-adapted Qwen3-8B, with separate learning rates for SFT and the two RL stages.RL-Stage II uses 2.0-second chunks, λmono = 0.2, and λrep = 0.05.Source: Model Configuration · p. 5

Evaluation Metrics

  • Translation quality is evaluated with XCOMET-XL and MetricX-24, with BLEU also reported; latency uses non-computation-aware LongYAAL for continuous, unbounded speech.MetricX-24 complements XCOMET-XL because XCOMET-XL is also the RL quality reward; LongYAAL is used to mitigate segmentation bias in conventional average-lagging metrics.Source: Evaluation Metrics · p. 5

Baselines

  • Baselines span decision modules, synthesized dialogue-style trajectories, and RL joint quality-latency optimization; the three representative systems are reproduced with SimPOTrans’s training data and backbone.Additional IWSLT 2026 systems provide a low-latency reference at LongYAAL ≤2 seconds, but are not controlled comparisons.Source: Baselines · p. 5

Main Results

On ACL 60/60, SimPOTrans consistently outperforms the baselines on COMET and MetricX-24 across language directions and latency regimes, while BLEU shows a different trend. On MCIF, it is competitive in the low-latency regime, and analyses associate its lower BLEU with more monotonic reformulations that preserve meaning but diverge from reference wording and order.

SimPOTrans Induces Translation Behavior Better Suited to SimulST

  • SimPOTrans performs better on COMET and MetricX-24 but substantially worse on BLEU, which the analysis links to meaning-preserving, more monotonic reformulations that differ from reference wording and order.The altered wording and word order reduce n-gram overlap and can therefore lower BLEU.Source: SimPOTrans Induces Translation… · p. 5

SimPOTrans Achieves Superior Quality–Latency Trade-offs

  • Across three language directions and latency regimes on ACL 60/60, SimPOTrans outperforms all baselines on COMET and MetricX-24, while BLEU follows a different trend.Figure 2 reports the quality–latency curves for the three metrics; MetricX-24 scores are negated so higher indicates better quality.Source: SimPOTrans Achieves Superior… · p. 5, Figure 2 · p. 6

SimPOTrans Delivers Strong Low-Latency Performance without Additional Context

  • On MCIF at LongYAAL ≤2 seconds, En→Zh reaches 0.76 COMET at 1.7 seconds, the highest score; En→De reaches 0.87 at 1.6 seconds, below only NeMo.NeMo is a cascaded system using additional context extracted from PDF documents; SimPOTrans uses no additional contextual information.Source: SimPOTrans Delivers Strong Low-Latency… · p. 5, Table 1 · p. 6

SimPOTrans Induces Translation Behavior Better Suited to SimulST

  • Monotonicity increases across all three language directions; in En→Zh, the offline output scores 54.2 BLEU while the more monotonic SimPOTrans output scores 30.8 despite conveying the core meaning.The example attributes the lower BLEU to reduced n-gram overlap from changes in wording and word order.Source: SimPOTrans Induces Translation… · p. 6

Ablation Study

RL-Stage II enables a smooth quality–latency trade-off through chunk-size changes, while auxiliary rewards and chunk-size choices affect its performance. The ablations identify the combined rewards and 2-second training chunks as strongest settings.

Effect of Simultaneous Adaptation

  • RL-Stage II yields a smooth quality–latency curve as chunk size increases, unlike the irregular, poor partial-speech performance of SFT and RL-Stage I.The authors conclude RL-Stage II is essential for adapting the offline model to simultaneous translation.Source: Effect of Simultaneous Adaptation · pp. 6–7, Figure 3 · p. 7

Effect of Auxiliary Rewards

  • Combining the monotonicity reward and repetition penalty achieves the best overall quality–latency trade-offs.The monotonicity reward yields mixed quality gains, whereas the repetition penalty provides more consistent improvements.Source: Figure 3 · p. 7, Effect of Auxiliary Rewards · p. 7

Effect of Chunk Size in RL-Stage II

  • Training with 2-second chunks achieves the best overall quality–latency trade-offs; 1-second chunks provide insufficient context, and 3-second chunks perform poorly at low latency.The authors therefore use a 2-second chunk size for RL-Stage II.Source: Figure 3 · p. 7, Effect of Chunk Size in RL-Stage II · p. 7

Limitations

SimPOTrans uses fixed-duration speech chunks, which may split semantically or acoustically coherent units; the evaluation also focuses on speech-to-text translation.

Limitations

  • Fixed-duration chunks may split semantically or acoustically coherent units.The authors identify this as a limitation of the current approach.Source: Limitations · p. 7
  • Boundary-aware chunking is proposed as a direction for future work.Source: Limitations · p. 7
  • The evaluation focuses solely on speech-to-text translation; extending SimPOTrans to speech-to-speech translation is left for future work.Source: Limitations · p. 7

Conclusion

SimPOTrans is a two-stage RL post-training framework for adapting offline speech translation models to dialogue-style SimulST. Experiments show superior quality–latency trade-offs over strong LLM-based baselines, with analyses indicating more simultaneous-suitable translation behavior.

Conclusion

  • SimPOTrans first improves translation quality on complete utterances, then learns to handle partial speech inputs through self-generated multi-turn trajectories.The two-stage pipeline eliminates the need for synthesized data and explicit latency optimization.Source: Conclusion · p. 7
  • Experiments demonstrate superior quality–latency trade-offs over strong LLM-based baselines.The evaluation covers ACL 60/60 and MCIF across three language directions.Source: Conclusion · pp. 7–8
  • Further analyses show that SimPOTrans induces translation behavior better suited to simultaneous scenarios.Source: Conclusion · p. 8

Related papers

Loading 2610.35042v1…