Source-linked AI summary
Monotonic Infinite Lookback Attention for Simultaneous Machine Translation
Naveen Arivazhagan, Colin Cherry, Wolfgang Macherey, Chung-Cheng Chiu, Semih Yavuz, Ruoming Pang, Wei Li, Colin Raffel
TL;DR
Simultaneous MT must begin translating before the speaker finishes while balancing quality against latency. The paper introduces MILk, which jointly learns an adaptive schedule and NMT model using monotonic scheduling with soft attention over revealed source tokens. MILk achieves favorable latency-quality trade-offs relative to wait-k across many latency values.
Problem
Simultaneous MT needs to schedule source reading and target writing under a quality-latency trade-off, but fixed schedules and separately trained adaptive schedules are limited.
Method
MILk jointly trains an adaptive schedule with an NMT model, using a hard monotonic head for reading and a soft head attending over previously revealed source tokens.
Results
MILk achieves favorable quality-latency trade-offs compared with wait-k strategies at many latency values.
Takeaways & Limitations
MILk extends streaming attention to support long-distance reorderings while retaining an adaptive schedule learned jointly with translation.
Takeaways & Limitations
Without latency-augmented training, MILk can wait for all source tokens before writing, becoming equivalent to standard soft attention.
Abstract
from arXiv · showhide
Simultaneous machine translation begins to translate each source sentence before the source speaker is finished speaking, with applications to live and streaming scenarios. Simultaneous systems must carefully schedule their reading of the source sentence to balance quality against latency. We present the first simultaneous translation system to learn an adaptive schedule jointly with a neural machine translation (NMT) model that attends over all source tokens read thus far. We do so by introducing Monotonic Infinite Lookback (MILk) attention, which maintains both a hard, monotonic attention head to schedule the reading of the source sentence, and a soft attention head that extends from the monotonic head back to the beginning of the source. We show that MILk's adaptive schedule allows it to arrive at latency-quality trade-offs that are favorable to those of a recently proposed wait-k strategy for many latency values.
1 Introduction
Simultaneous MT must schedule source reading and target writing to balance translation quality against latency. MILk jointly learns this adaptive schedule with an NMT model, using monotonic and lookback attention, and achieves favorable trade-offs against wait-k.
- Motivation: Simultaneous MT balances translation quality against latency by scheduling when to read source tokens and write target tokens.Reading too little can degrade quality, while waiting for too much source content can introduce unnecessary delays.
- Motivation: Fixed schedules can delay some sentences excessively or provide too little context for others, while separately trained adaptive schedules use an NMT model trained on complete sentences.The paper therefore jointly trains the adaptive schedule and the underlying NMT system.
- Approach: The method extends monotonic attention to support long-distance reorderings while preserving streaming through latency-augmented training.The system is trained in expectation as a drop-in replacement for familiar soft attention.
- Approach: MILk attention jointly learns an adaptive schedule with an NMT model that attends over all source tokens read thus far.It uses a hard monotonic head to determine available source content and a soft head to attend over revealed tokens.
- Contributions: The paper extends Average Lagging into a differentiable, expectation-calculable latency metric usable as a training objective.This enables latency to guide joint training of the translation model and schedule.
- Results: MILk demonstrates favorable quality-latency trade-offs compared with wait-k strategies at many latency values.The authors also provide evidence that its advantage extends from adapting the schedule to source content.
2 Background
Earlier simultaneous MT work either adapted schedules around fixed full-sentence NMT systems or used simple fixed schedules integrated into training. These approaches trade adaptive context sensitivity against easy training and deployment.
- Adaptive scheduling: Earlier systems often segmented source sentences using prosodic, lexical, or translation-quality cues to decide when partial segments were safe to translate.Segmentation approximates the underlying decision of whether enough source content has been read to write the next target word.
- Adaptive scheduling: Heuristic and reinforcement-learning approaches adapt reading and writing decisions but commonly use an NMT model pretrained on complete sentences.In these systems, the fixed NMT model can limit the adaptive schedule.
- Fixed scheduling: Wait-k uses a deterministic schedule that waits for k source tokens, then alternates reading and writing at a constant rate.Its fixed schedule makes it straightforward to train the NMT system with the schedule in place.
- Alternative approaches: Attention-free adaptive models reduce computation and memory by maintaining a running context vector, but their schedules rely on external word-alignment supervision.MILk instead learns its schedule latently in service of the MT task.
3 Methods
The method combines sequence-to-sequence attention with a learned schedule that limits each output to the source tokens read so far. MILk extends monotonic attention with lookback over all revealed source states, while latency-augmented training encourages streaming behavior.
- Standard soft attention cannot stream because it computes each context vector from the entire input sequence.
- Monotonic Attention: Monotonic attention creates an interleaved read-write schedule by scanning source states left to right and selecting a stopping position for each output.
- Monotonic Infinite Lookback Attention: MILk retains a monotonic head for source availability and adds soft attention over all encoder states from the beginning through the stopping position.This infinite lookback is intended to provide flexibility for long-distance reorderings while preserving streaming.
- Monotonic Infinite Lookback Attention: MILk can otherwise wait until all source tokens are available, making it equivalent to standard soft attention.
- Latency-augmented Training: The training objective interpolates negative log likelihood with a latency cost, using expected delays derived from monotonic attention probabilities.For MILk, the delay before target step i is the monotonic head position t_i, and differentiable latency metrics can guide training toward lower latency.
- Latency-augmented Training: Near-binary stopping probabilities can improve latency estimates but may destabilize MILk training.
4 Measuring Latency
The paper compares latency metrics for simultaneous MT and introduces Differentiable Average Lagging (DAL) to make Average Lagging trainable while preserving intuitive delay behavior. DAL replaces AL’s truncation with a minimum delay for each generated target token.
- 4.1 Previous Latency Metrics: Average Proportion is bounded between 0 and 1 but varies with sentence length and can lack sensitivity to improvements in simultaneous MT.A wait-1 system’s AP ranges from 0.75 for two-token sentences to 0.5 as sequence length approaches infinity.
- 4.1 Previous Latency Metrics: Average Lagging measures how quickly a system falls behind an ideal simultaneous translator and gives wait-k systems lag k when source and target lengths match.The metric also accounts for unequal sequence lengths through γ = |y|/|x|.
- 4.2 Differentiable Average Lagging: DAL removes AL’s argmin-based truncation by enforcing a minimum delay before writing every target token.Each target token takes at least 1/γ units of time, and the recurrence preserves earlier delays.
- 4.2 Differentiable Average Lagging: AL’s truncation prevents tokens generated after the source is exhausted from artificially lowering the measured average lag.Without truncation, those tokens appear instantaneously and reduce the average despite earlier delay.
- 4.2 Differentiable Average Lagging: DAL is differentiable and calculable in expectation, while matching AL for many equal-length wait-k cases.For equal-length wait-k systems, both metrics always return a lag of k.
5 Experiments
Experiments evaluate MILk on WMT14 EnFr and WMT15 DeEn using shared architectures and quality-latency curves. MILk shows favorable trade-offs, including adaptive behavior that preserves quality across relevant latency ranges.
- Experimental setup: Experiments use WMT14 EnFr and WMT15 DeEn, with detokenized, cased BLEU as the translation-quality metric.EnFr contains 36.3M training sentences and DeEn 4.5M.
- Experimental setup: MILk, wait-k, Monotonic Attention, and MoChA are implemented through a common attention interface to isolate their contributions.Streaming models use unidirectional encoders, while offline models use bidirectional encoders.
- Model tuning: Mass preservation is crucial: without it, training destabilizes and produces very poor trade-offs for latency weights as low as λ = 0.2.Even with λ = 0, the unmodified system reduced latency from the maximum value of 27.9.
- Quality-latency comparison: On EnFr, MILk extends the flat safe region to a lag of 7.2 with BLEU=40.5, while wait-k quality drops at k = 8 with DAL=8.4 and BLEU=39.8.At the lowest comparable lag of 4.5 tokens, MILk surpasses wait-k by 2.3 BLEU points.
- Quality-latency comparison: Across AL and AP, MILk’s separation above wait-k grows because these metrics reward consecutive writes that DAL treats more conservatively.The curves report BLEU against DAL, AL, or AP while varying λ for MILk and k for wait-k.
- Schedule behavior: MILk adapts its reading to phrase boundaries and dependencies, waiting beyond noun or verb phrases and pausing before ambiguous words when needed.For John Smith’s lawyer, waiting until the phrase ends enables the correct reordering of avocat; another example selects de la for feminine loi.
6 Conclusion
The conclusion presents MILk as a jointly trained attention mechanism with an adaptive reading schedule. It reports favorable quality-latency trade-offs and a longer region of the curve without major BLEU reduction.
- Contribution: MILk combines a hard monotonic head for source reading with a soft head attending over all source tokens read so far.This supports joint training of the NMT system and its adaptive schedule.
- Results: MILk achieves favorable quality-latency trade-offs compared with wait-k and earlier monotonic attention mechanisms.The paper also contributes latency-augmented training and a differentiable latency metric.
- Implication: MILk is particularly useful for extending the latency-curve region where BLEU has not yet incurred a major reduction.
A Expanded Results
Expanded results provide complete quality-latency curves and tables for DeEn and EnFr under DAL, AL, and AP. The supplementary results parameterize MILk by λ, wait-k by k, and MoChA by chunk size.
- Expanded results: Figure 8 and Tables 5–6 provide complete DeEn and EnFr quality-latency results under DAL, AL, and AP.DAL denotes Differentiable Average Lagging, AL Average Lagging, and AP Average Proportion.
- Parameterization: MILk is parameterized by latency weight λ, wait-k by k, and MoChA by chunk size cs in the expanded comparisons.EnFr MILk with λ = 0.75 is omitted because it failed to converge.