Source-linked AI summary
Learning to Translate in Real-time with Neural Machine Translation
Jiatao Gu, Graham Neubig, Kyunghyun Cho, Victor O. K. Li
TL;DR
Simultaneous translation must emit target words before the source sentence ends while balancing translation quality and delay. The paper combines a pre-trained NMT environment with a recurrent agent that learns READ/WRITE decisions, and reports strong improvements over NMT-based and conventional segmentation baselines on two language pairs. The framework also explores delay targets and simultaneous beam search.
Problem
Simultaneous translation requires incremental output while balancing translation quality against delay, and existing NMT-based and segmentation approaches leave their relative merit unresolved.
Method
A recurrent agent interacts with a pre-trained NMT environment, learning interleaved READ/WRITE decisions with reinforcement learning and using beam search within translation segments.
Results
Strong quantitative improvements over NMT-based and conventional segmentation methods were observed on English-Russian and English-German translation in both directions.
Takeaways & Limitations
The framework finds quality-delay trade-offs across language pairs and directions, with proposed models balancing quality and delay better than the WUE and WOS baselines.
Takeaways & Limitations
The pre-trained NMT environment is trained on complete sentences, which can make verbs unseen in an incomplete source difficult to predict during simultaneous translation.
Abstract
from arXiv · showhide
Translating in real-time, a.k.a. simultaneous translation, outputs translation words before the input sentence ends, which is a challenging problem for conventional machine translation methods. We propose a neural machine translation (NMT) framework for simultaneous translation in which an agent learns to make decisions on when to translate from the interaction with a pre-trained NMT environment. To trade off quality and delay, we extensively explore various targets for delay and design a method for beam-search applicable in the simultaneous MT setting. Experiments against state-of-the-art baselines on two language pairs demonstrate the efficacy of the proposed framework both quantitatively and qualitatively.
1 Introduction
Simultaneous translation must balance translation quality against time delay while processing input incrementally. The paper proposes learning READ/WRITE decisions around a pre-trained NMT system and evaluates the framework across two language pairs.
- Simultaneous translation produces translations in real time, supporting understanding of spoken lectures and conversations.
- Unlike standard MT, simultaneous translation must balance translation quality with the delay before users receive translated content.
- Earlier approaches mainly used segmenters with phrase-based MT, while recent NMT methods used heuristic decoding or an independent segmentation network.
- The proposed framework interleaves READ and WRITE actions, connects an agent to an NMT system, and trains decisions with reinforcement learning using quality and delay rewards.It also introduces beam search within translation segments.
- Experiments on English-Russian and English-German translation in both directions report strong quantitative improvements over NMT-based and conventional segmentation baselines.The study also varies target delay and analyzes qualitative visualizations.
2 Problem Definition
The task models simultaneous translation as sequential READ and WRITE decisions over an input and output buffer. Its objective is to maximize translation quality while minimizing delay caused mainly by waiting for more source words.
- A translator sequentially READs source words as context or WRITEs translated words to the output buffer.The action sequence contains Ts READs and Tt WRITEs, for a total of T = Ts + Tt actions.
- Translation quality is measured by Q(Y), while delay is represented by D(A), which depends mainly on READ actions.READ requires waiting for continued speech, whereas WRITE generates words in milliseconds.
- The optimization objective seeks action policies that trade higher translation quality against lower delay.
- The framework connects READ/WRITE actions to an NMT system and optimizes the resulting simultaneous translation policy.
3 Simultaneous Translation with Neural Machine Translation
The NMT environment incrementally encodes only the source prefix that has been READ and generates candidate target words with an attention-based decoder. A recurrent agent then chooses whether to reject each candidate and READ more input or WRITE it.
- The framework consists of an NMT environment and a trainable recurrent agent that controls translation actions.
- Environment: The encoder uses a left-to-right unidirectional RNN so source representations can be computed without seeing the final source word.Bidirectional encoding is unsuitable for simultaneous processing because reverse-order encoding requires the sentence end.
- Environment: The attention-based decoder generates each target word using only source words already READ from the input.The decoder operates on incomplete input states represented by a prefix of the encoder states.
- Environment: At each step, the decoder supplies a candidate translation corresponding to the current source prefix, and the agent decides whether to accept it or wait.
- Agent: The agent observes the current translation state and uses a recurrent stochastic policy to select actions sequentially.The policy is parameterized by a recurrent neural network and produces an action distribution.
- Agent: READ rejects the current candidate and encodes another source word, whereas WRITE accepts the candidate and emits it.
4 Learning
The framework trains an agent with reinforcement learning to balance translation quality against delay while interacting with a pre-trained NMT environment. Its reward combines quality and delay signals, with target delays and variance-reduction techniques supporting stable learning.
- 4 Learning: The agent learns READ/WRITE behavior through policy-gradient reinforcement learning while the pre-trained NMT parameters remain frozen.Training samples multiple action trajectories, and a baseline network reduces policy-gradient variance while entropy regularization encourages exploration.
- 4 Learning: The pre-trained NMT environment generates translations from incomplete source sentences, although this design is acknowledged as likely sub-optimal.The environment uses a pre-trained encoder-decoder and assumes reasonable translation on partial inputs.
- 4 Learning: The reward evaluates both translation quality and delay, using BLEU-derived quality rewards and delay measures based on waiting for source words.Partial BLEU differences provide intermediate quality rewards, while the brevity penalty is deferred until sentence completion to avoid heavily penalizing partial translations.
- 4 Learning: Consecutive Wait length captures long silences by counting consecutive READ actions between translated words.The metric complements global delay measures by focusing on local waiting intervals.
- 4 Learning: Target delay values d* and c* let the optimization criterion adapt to application-specific latency requirements.The quality–delay trade-off is adjusted through reward coefficients α and β together with the target-delay settings.
5 Simultaneous Beam Search
The paper adapts beam search to simultaneous translation by buffering multiple translation trajectories during consecutive WRITE actions. It outputs the best path when switching to READ, improving path search without increasing delay under the stated assumption.
- 5 Simultaneous Beam Search: Standard beam search improves translation quality by retaining k alternative translation trajectories.Direct application is difficult because standard beam search waits until the final word before writing translation.
- 5 Simultaneous Beam Search: Simultaneous beam search keeps multiple beams in a temporary buffer while the agent consecutively chooses WRITE.When the agent switches to READ, the system outputs the best buffered path.
- 5 Simultaneous Beam Search: Under the assumption that WRITE does not incur delay, the method searches for a relatively better path while keeping delay unchanged.The agent is not retrained; it receives the current best trajectory's observation for its next decision.
6 Experiments
Experiments evaluate the framework across EN-RU and EN-DE in both directions, varying delay targets and comparing decoding strategies with established baselines. The results show that the method learns quality–delay trade-offs, generally improves on baselines, and exposes language-order and beam-search effects.
- Settings: Experiments cover EN-DE and EN-RU translation in both directions, using WUE, WOS, WIW, WID, and SEG baselines.The data use WMT15 parallel corpora, with newstest-2013 for validation and BPE segmentation; evaluation uses the same pre-trained NMT environment.
- Settings: Target delays vary across AP ∈ {0.3, 0.5, 0.7} and CW ∈ {2, 5, 8}, with separate reward-function settings.The experiments adjust α and β for stable learning and retain one target for each delay measure when analyzing learning curves.
- Quantitative Analysis: Learning increases translation quality while pushing delay close to the selected target, and AP and CW decrease together because the two measures are strongly correlated.This pattern is reported for EN-RU validation learning curves across the tested target settings.
- Quantitative Analysis: Smaller target delays produce lower BLEU, while AP-optimized models generally outperform CW-optimized models, especially for “→EN” translation.Lower AP targets are harder to achieve with good quality in “→EN” than in “EN→” translation.
- Quantitative Analysis: The proposed models generally find better quality–delay trade-offs than WUE, WOS, WID, WIW, and SEG, with some models approaching WUE quality at much smaller delay.Compared with SEG, the proposed model produces high-quality translations at much smaller CW; compared with WID and WIW, it performs better in most cases but has some exceptions.
- Quantitative Analysis: Simultaneous beam-search usually raises quality with a slight delay increase, but its gains shrink at lower delays and remain below those seen in standard NMT.The method keeps multiple trajectories during consecutive WRITE actions and outputs the best path when switching to READ.
- Qualitative Analysis: Qualitative behavior depends on word order: larger delay targets improve EN→RU outputs, whereas long-distance German verbs remain a limitation in DE→EN.The DE→EN example mistranslates “gedeckt” as “being paid” when the agent chooses to WRITE rather than incur a large delay.
7 Related Work
Prior simultaneous-translation work includes speech-segmented systems, hand-crafted waiting rules, and reinforcement-learning approaches. The proposed framework differs by learning unconstrained READ/WRITE decisions and enabling broader evaluation and beam search.
- Speech-interpretation systems first recognize and segment incoming speech with ASR, then translate each segment independently.
- Cho and Esipova use manually defined waiting criteria, whereas Satija and Pineau apply reinforcement learning but cannot consecutively decode multiple steps, limiting beam search.
- Unlike earlier efforts, the study performs extensive quantitative and qualitative evaluation against existing state-of-the-art approaches.
- The proposed framework learns READ/WRITE actions that need not align with source words while balancing translation quality and delay.
8 Conclusion
The paper presents a unified neural framework for simultaneous machine translation that explores delay targets and supports beam search. Experiments on two language pairs demonstrate efficacy against state-of-the-art baselines quantitatively and qualitatively.
- The framework unifies neural simultaneous machine translation, explores multiple delay targets, and designs beam search for simultaneous MT.
- Experiments against state-of-the-art baselines on two language pairs demonstrate the framework’s efficacy quantitatively and qualitatively.