Source-linked AI summary
Marian: Fast Neural Machine Translation in C++
Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, Alexandra Birch
TL;DR
Neural Machine Translation frameworks need to support efficient research and execution while accommodating complex model architectures. Marian addresses this with a self-contained C++ toolkit combining dynamic-graph automatic differentiation and extensible encoder-decoder interfaces. It reproduces Nematus quality, improves the WMT2017 test result with a Transformer, and reports substantially faster training and translation.
Problem
The paper addresses the need for a research-friendly Neural Machine Translation toolkit that supports efficient experimentation and execution.
Method
Marian combines a pure-C++ self-contained framework, reverse-mode automatic differentiation with dynamic computation graphs, and composable encoder-decoder interfaces.
Results
Marian matches original Nematus models in BLEU, improves the WMT2017 test result by 1.2 BLEU with a Transformer, and achieves about 4 times faster single-GPU and about 30 times faster 8-GPU training.
Takeaways & Limitations
Marian provides an efficient, research-focused NMT toolkit whose reusable framework supports model development alongside high-speed training and translation.
Abstract
from arXiv · showhide
We present Marian, an efficient and self-contained Neural Machine Translation framework with an integrated automatic differentiation engine based on dynamic computation graphs. Marian is written entirely in C++. We describe the design of the encoder-decoder framework and demonstrate that a research-friendly toolkit can achieve high training and translation speed.
1 Introduction
Marian is a pure-C++11, self-contained Neural Machine Translation framework designed for efficiency, minimal dependencies, and research flexibility. Its implementation supports efficient optimization and broad deployment.
- Marian is an efficient Neural Machine Translation framework written entirely in C++11 with minimal dependencies.
- Its self-contained back end provides reverse-mode automatic differentiation based on dynamic computation graphs.
- Minimal dependencies enable optimization across meta-algorithms, batched beam search, model implementations, custom operators, and GPU kernels.The framework depends only on Boost and CUDA or a BLAS library.
- Marian grew from a C++ reimplementation of Nematus and maintains binary compatibility for common models.Consequently, the paper compares speed mostly against Nematus.
- The toolkit is distributed under the MIT license and is available through its project website and GitHub repository.
2 Design Outline
Marian combines a dynamic-graph automatic-differentiation back end with an extensible encoder-decoder framework and efficient multi-device algorithms. Its interfaces support composable models and reuse across training, scoring, and translation.
- 2.1 Custom Auto-Differentiation Engine: Marian uses reverse-mode automatic differentiation with dynamic computation graphs, optimized for machine translation and related use cases.The back end includes fused RNN cells, attention mechanisms, and an atomic layer-normalization operator.
- 2.2 Extensible Encoder-Decoder Framework: Encoders and decoders are implemented as classes with interfaces for building encoder states, starting decoder states, and stepping through batches.
- 2.2 Extensible Encoder-Decoder Framework: The framework supports Bahdanau-style and multi-source configurations by passing one, multiple, or no encoder states into decoder initialization.
- 2.2 Extensible Encoder-Decoder Framework: A decoder step consumes target batches to produce output logits and can operate over all time steps during training and scoring or step-by-step during translation.
- 2.2 Extensible Encoder-Decoder Framework: Combining different encoders and decoders reduces implementation effort because a new model often requires only one inference-step implementation for training, scoring, and translation.
- Efficient Meta-Algorithms: The toolkit adds multi-device training, scoring, batched beam search, heterogeneous-model ensembling, and multi-node training.
3 Case Studies
Marian serves as a flexible research toolkit across neural machine translation, automatic post-editing, and grammatical error correction. Case studies show replicated or improved translation quality, high training and translation speed, and competitive downstream systems.
- 3.1 Improving over WMT2017 systems: Marian replicates the WMT17 English–German system and improves the recipe by replacing its deep RNN with a Transformer.The architecture follows the reported deep-transition RNN setup, while the Transformer follows Vaswani et al.'s base model.
- 3.1.3 Training Recipe: The training recipe preprocesses data with joint 36,000-unit BPE, back-translates 10M German sentences, and trains ensembles of L2R and R2L models.Evaluation uses sacreBLEU on newstest-2016 and newstest-2017; training runs on eight GPUs with synchronous or asynchronous Adam.
- 3.1.4 Performance and Results: 1.2 BLEU is gained on WMT2017 by replacing the deep-transition RNN with the Transformer, while matching the original Nematus models.This result is reported for the WMT2017 test set.
- 3.1.4 Performance and Results: 4 times faster single-GPU training and 30 times faster eight-GPU training are achieved by Marian for identical models relative to Nematus.Scaling with more GPUs is close to, but not exactly, linear; Nematus lacks multi-GPU training.
- 3.1.4 Performance and Results: 15,850 source tokens per second enables back-translation of 10M sentences in about four hours on eight GPUs with beam size 5.Sentences are translated in parallel across multiple GPUs using batch size 64.
- 3.2 State-of-the-art in Neural Automatic Post-Editing: In automatic post-editing, dual-attention models recover source-dependent missing words, while the final system ranks second by automatic metrics and first by human evaluation.The models combine raw MT output and source input, and the encoder-decoder framework integrates dual encoders without changing beam search or ensembling.
- 3.3 State-of-the-art in Neural Grammatical Error Correction: Model-independent neural GEC methods improve prior neural systems by more than 8% M2 on CoNLL-2014 and more than 4.5% on JFLEG.Non-neural state-of-the-art systems are matched on CoNLL-2014 and exceeded by 2% on JFLEG.
4 Future Work and Conclusions
Marian is introduced as a self-contained C++ neural machine translation toolkit focused on efficiency and research, with future work targeting backend speed and frontend model capabilities.
- Future backend work targets faster CPU computation, auto-batching, and automatic kernel fusion.
- Future frontend work aims to keep Marian aligned with future state-of-the-art models.
- Marian is a self-contained neural machine translation toolkit written in C++ with a focus on efficiency and research.