PaperlayerView exact revision on arXiv

Source-linked AI summary

From Simultaneous to Streaming Machine Translation by Leveraging Streaming History

Javier Iranzo-Sánchez, Jorge Civera, Alfons Juan

arXiv:2203.02459v3cs.CL

TL;DR

Conventional simultaneous MT limits its history to the sentence being translated, while streaming MT extends translation to a continuous stream. The paper adapts a wait-k system to use streaming history, reporting gains over the sentence-level baseline and competitive quality at lower latency than a segmentation-free system.

  • Problem

    Conventional simultaneous MT limits history to the current sentence, whereas streaming MT can use tokens from earlier sentences.

    Source: §2.1 · p. 2

  • Method

    The system trains on sentence-boundary sliding windows with prior sentence pairs as history and performs streaming inference in a single forward pass.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    The system improves on the sentence-level baseline and reaches similar translation quality to a state-of-the-art segmentation-free streaming system at a fraction of its latency.

    Source: §5 · p. 8

  • Takeaways & Limitations

    A single model uses a flexible inference policy to obtain competitive results across a wide range of latency levels.

    Source: §5 · p. 8

    The segmentation model adds complexity and can greatly affect translation quality.

    Source: §5 · p. 9

Abstract

from arXiv · show

Simultaneous Machine Translation is the task of incrementally translating an input sentence before it is fully available. Currently, simultaneous translation is carried out by translating each sentence independently of the previously translated text. More generally, Streaming MT can be understood as an extension of Simultaneous MT to the incremental translation of a continuous input text stream. In this work, a state-of-the-art simultaneous sentence-level MT system is extended to the streaming setup by leveraging the streaming history. Extensive empirical results are reported on IWSLT Translation Tasks, showing that leveraging the streaming history leads to significant quality gains. In particular, the proposed system proves to compare favorably to the best performing systems.

1 Introduction

The paper extends simultaneous MT toward continuous streaming, where systems must use history across sentence boundaries and meet latency constraints over the full stream. It adapts a wait-k system to streaming and reports quality gains and competitive comparisons.

1 Introduction

  • Streaming MT translates a potentially unbounded, unsegmented text stream, requiring use of history beyond sentence boundaries and latency control across the stream.The motivation includes speech translation from an upstream ASR stream and applications such as person-to-person communication and audiovisual subtitling.Source: §1 · p. 1
  • Prior work on sentence and document context, language modeling, and cross-sentence data augmentation provides related approaches to using context in translation.The paper notes reported quality gains from context in offline MT and more robust ST from cross-sentence context augmentation.Source: §1 · p. 1
  • Most simultaneous MT systems translate sentences independently, while existing sentence-level latency measures do not adequately gauge latency over a stream.The cited exception is Schneider and Waibel (2020), whose segmentation-free streaming model's latency the authors describe as hard to interpret using those measures.Source: §1 · pp. 1–2
  • The streaming-ready system improves BLEU over the sentence-level baseline and compares favorably with state-of-the-art simultaneous MT systems in quality and latency.Reported stream-adapted latency measures are consistent and interpretable.Source: §1 · p. 2

2 Streaming MT

The paper frames streaming MT as incremental translation of an unsegmented stream, with history and latency considered beyond sentence boundaries. It proposes a sliding-window training strategy and a Partial Bidirectional Encoder (PBE) to use longer context while retaining single-pass training efficiency.

2 Streaming MT

  • Streaming MT treats the source as an unsegmented, unbounded token stream and defines delay by the last source position available when each target token is output.Sentence segmentation can mark monotonic source-target sentence boundaries, represented by vectors of source and target sentence start positions.Source: §2 · p. 2

2.1 Simultaneous MT with streaming history

  • Streaming history can span tokens from earlier sentences, replacing sentence-level delay with a global delay function for stream-level timing.The wait-k policy can be reformulated to apply within each sentence; a catch-up factor generalizes its read/write behavior when source and target lengths differ.Source: §2.1 · pp. 2–3
  • Training on arbitrarily long streams risks source-target misalignment, repeated computation, and incompatibility with sentence-level parallel corpora; sliding windows address these constraints.The method concatenates each sentence pair with preceding pairs, shifts the window one sentence at a time, and limits history while maintaining alignment; inference uses a single forward pass.Source: §2.1 · p. 3

2.2 Partial Bidirectional Encoder

  • The unidirectional encoder is efficient but excludes tokens beyond position j from the encoding of source token j, even when more context could be included in one pass.This limitation motivates PBE's access to succeeding positions.Source: §2.2 · p. 4
  • PBE lets initial source positions access succeeding tokens, providing longer context while retaining a single encoder-decoder forward pass at training time.Unlike the bidirectional encoder, which may require recomputation at each timestep and take up to |y| times longer to train, PBE falls back to bidirectional encoding at inference.Source: §2.2 · pp. 3–4

3 Experimental setup

Experiments use IWSLT data to evaluate streaming and simultaneous MT in German→English and English→German, with stream-level latency measures and a text-based segmenter. Training uses sliding windows of previous sentences to supply streaming history.

3 Experimental setup

  • Experiments cover German↔English streaming MT and English→German sentence-based simultaneous MT, with streaming systems tuned on the 2010 dev set and evaluated on its test set.The English→German nonstreaming setting supports comparison with IWSLT 2020 simultaneous-translation systems.Source: §3 · p. 4
  • Training prepends previous sentences until a history-token threshold is reached, using special tokens for sentence boundaries and 40K BPE merge operations.Corpora with available document information use the sliding-window technique; Table 1 summarizes the training-data statistics.Source: §3 · p. 4, Table 1 · p. 4
  • Latency uses stream-adapted versions of AP, AL, and DAL because sentence-level measures do not adequately evaluate streaming systems.The adaptation maps global delay to relative delay; sentence boundaries are obtained by re-segmenting hypotheses to match reference translations.Source: §3 · p. 5
  • Streaming models use Transformer BASE configurations, while the English→German simultaneous system uses BIG; models are also fine-tuned on MuST-C.The paper reports the fine-tuning setup as following Schneider and Waibel (2020).Source: §3 · p. 5
  • Inference uses a text-based sliding-window Direct Segmentation model, whose future-window length trades off latency and translation quality.The segmenter was trained on TED data and was selected for streaming readiness and translation quality relative to other segmenters.Source: §3 · pp. 5–6

4 Evaluation

Across the reported comparisons, the streaming system improves on ON-TRAC and offers adjustable latency at inference, while its streaming test results combine strong BLEU with modest delay. Development results favor using streaming history and the PBE encoder, which the experiments select for further German–English tests.

4 Evaluation

  • On German–English IWSLT 2010 development data, streaming history consistently improves BLEU over h = 0; h = 60 performs best, with a slight decline at h = 80.At k = 32 and h = 60, PBE is 0.7 BLEU points above the unidirectional encoder.Source: Figure 2 · p. 6, §4 · p. 6
  • PBE narrows the gap to offline bidirectional MT at large k, while bidirectional simultaneous MT performs poorly when training and inference expose different source context.The h = 60 PBE system is used for the remaining German–English experiments.Source: §4 · p. 6
  • On the IWSLT 2010 streaming test, the system reaches 29.5 BLEU while lagging 2–10 tokens behind the speaker for nearly maximum BLEU.Reported AP ranges from 0.6 at w = 0 to 1.3 at w = 4; the DS segmenter’s sentence-end events force the MT system to catch up.Source: Figure 3 · p. 7, §4 · p. 7
  • The streaming test integrates segmenter and MT latency by accounting for the segmenter’s future window, with curves comparing w values and points representing wait-k settings.The plotted latency measures are stream-adapted AL and DAL at scale s = 0.85.Source: Figure 3 · p. 7, §4 · p. 7
  • ACT and STR-MT achieve similar translation quality, but ACT lags around 100 tokens behind the speaker and lacks a catch-up mechanism.The comparison is on the IWSLT 2010 German–English test set.Source: Table 2 · p. 7, §4 · pp. 7–8
  • The proposed streaming system consistently exceeds ON-TRAC by 1–2 BLEU, while RWTH and KIT are closer only at AL 5–7 and lack comparable latency flexibility without retraining.The comparison uses MuST-C reference segmentation and sentence-level AL because the other systems translate sentences independently.Source: §4 · p. 8

5 Conclusions

The paper formalizes streaming MT and extends a simultaneous system to use history across sentence boundaries, improving on a sentence-level baseline. It also reports competitive quality across streaming and simultaneous settings while identifying further training and segmentation challenges.

5 Conclusions

  • Streaming MT is formalized as a generalization of simultaneous MT, providing a framework for the paper’s two contributions.Source: §5 · p. 8
  • Using streaming history across sentence boundaries greatly improves the multiple-wait-k system over the sentence-level baseline.Source: §5 · p. 8
  • PBE encodes longer context than its unidirectional counterpart while keeping the same training efficiency.Source: §5 · p. 8
  • In realistic streaming evaluation, the system reaches similar translation quality to a state-of-the-art segmentation-free system at a fraction of its latency.Source: §5 · p. 8
  • A single model is competitive with optimized sentence-level simultaneous MT systems across a wide range of latency levels using a flexible inference policy.Source: §5 · p. 8
  • Further history-aware training and inference remain open research, while the segmentation model adds complexity and can greatly affect translation quality.Source: §5 · pp. 8–9

A Extended Streaming Translation Results

The extended experiments report gains from streaming history in both translation directions and quality improvements from PBE. Under streaming conditions, wait-k and direct segmentation provide adjustable latency, with diminishing quality gains at higher latency.

A Extended Streaming Translation Results

B Efficiency of the proposed models

The encoder variants use attention masks to enforce their token-access constraints during training, with identical computational behavior. At inference, incremental unidirectional encoding is faster on CPU, but GPU parallelization removes the latency advantage.

B Efficiency of the proposed models

C MT System configuration

The configuration describes training and fine-tuning settings, data preprocessing, and construction of streaming training samples. Streaming data is extracted with a sentence-boundary sliding window and upsampled relative to sentence-level data.

C MT System configuration

  • The multi-wait-k models train for 0.5M steps on four 2080Ti GPUs; training takes 40 hours for BASE and 60 hours for BIG models.Source: C MT System configuration · pp. 12–13
  • Fine-tuning changes the learning-rate schedule to fixed and sets the learning rate to 4.47169e-05.Source: C MT System configuration · p. 13
  • Streaming data is lowercased with punctuation removed, while simultaneous-task data is truecased and Moses-tokenized; SentencePiece learns the BPE units.Source: C MT System configuration · p. 13
  • Streaming data is built by sliding a window over whole sentences to extract consistent source-target samples, then upsampled to a 1-to-3 ratio with sentence-level data.Source: C MT System configuration · p. 13

D Segmenter System configuration

The Direct Segmentation system was trained with the official implementation, and the section specifies its model settings and history-based sample construction.

D Segmenter System configuration

  • The segmenter used an ff-text model with two 128-unit feedforward layers, 256-dimensional embeddings and RNN layers, Adam, 0.3 dropout, and 15 epochs.Training used a batch size of 256 and a learning rate of 0.0001.Source: D Segmenter System configuration · p. 13
  • Training configurations paired sample lengths 11–15 with window sizes 0–4, respectively.Source: D Segmenter System configuration · p. 13
  • With history size h=5, boundary and document tokens do not count toward the limit, and a prior sentence pair may be omitted when the source-side limit would be exceeded.Figure 8 illustrates this history-based sample construction.Source: Figure 8 · p. 14
Loading 2203.02459v3…