Source-linked AI summary
NCLS: Neural Cross-Lingual Summarization
Junnan Zhu, Qian Wang, Yining Wang, Yu Zhou, Jiajun Zhang, Shaonan Wang, Chengqing Zong
TL;DR
Large-scale supervised data is lacking for cross-lingual summarization, and traditional pipelines risk error propagation. The paper constructs CLS data with round-trip translation and presents end-to-end NCLS; NCLS outperforms pipeline methods, with further gains from multi-task learning.
Problem
Large-scale supervised CLS data is lacking, and traditional approaches divide the task into summarization and translation, where imperfect machine translation can propagate errors.
Source: §1 · p. 1Method
The paper uses round-trip translation to build CLS datasets and trains end-to-end NCLS with monolingual summarization and machine translation under multi-task learning.
Source: §1 · pp. 1–2, §2 · p. 2Results
NCLS gains +4.87 ROUGE-2 on En2Zh and +5.07 ROUGE-2 on Zh2En over pipeline methods; multi-task NCLS further achieves +3.60 ROUGE-2 on En2Zh and +0.72 ROUGE-2 on Zh2En.
Source: §1 · p. 2Takeaways & Limitations
The authors present their methods as a benchmark for further NCLS study.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Cross-lingual summarization (CLS) is the task to produce a summary in one particular language for a source document in a different language. Existing methods simply divide this task into two steps: summarization and translation, leading to the problem of error propagation. To handle that, we present an end-to-end CLS framework, which we refer to as Neural Cross-Lingual Summarization (NCLS), for the first time. Moreover, we propose to further improve NCLS by incorporating two related tasks, monolingual summarization and machine translation, into the training process of CLS under multi-task learning. Due to the lack of supervised CLS data, we propose a round-trip translation strategy to acquire two high-quality large-scale CLS datasets based on existing monolingual summarization datasets. Experimental results have shown that our NCLS achieves remarkable improvement over traditional pipeline methods on both English-to-Chinese and Chinese-to-English CLS human-corrected test sets. In addition, NCLS with multi-task learning can further significantly improve the quality of generated summaries. We make our dataset and code publicly available here: http://www.nlpr.ia.ac.cn/cip/dataset.htm.
1 Introduction
Cross-lingual summarization lacks large-scale supervised data, while pipeline approaches risk error propagation. The paper builds CLS data with round-trip translation and introduces end-to-end NCLS, whose multi-task version further improves results.
1 Introduction
- Existing pipeline approaches split CLS into summarization and translation, and imperfect machine translation can propagate errors; large-scale supervised CLS data is also lacking.The passages describe both the pipeline problem and the dataset gap motivating the work.Source: §1 · p. 1
- Using round-trip translation from existing large-scale monolingual summarization (MS) datasets, the authors construct large-scale cross-lingual summarization (CLS) corpora: 370K English-to-Chinese (En2Zh) and 1.69M Chinese-to-English (Zh2En) pairs.Source: §1 · pp. 1–2
- NCLS is an end-to-end CLS framework, further trained with monolingual summarization and machine translation under multi-task learning.The authors describe this as the first end-to-end CLS framework trained on parallel corpora.Source: §1 · pp. 1–2
- +4.87 ROUGE-2 on En2Zh and +5.07 ROUGE-2 on Zh2En over traditional pipeline methods; multi-task NCLS further achieves +3.60 ROUGE-2 on En2Zh and +0.72 ROUGE-2 on Zh2En.The authors present these methods as a benchmark for further NCLS study.Source: §1 · p. 2
2 Dataset Construction
The datasets are constructed by translating monolingual summaries across languages and filtering them through round-trip translation. The resulting corpora provide hundreds of thousands of En2Zh pairs and over a million Zh2En pairs, with test references corrected by volunteers.
2 Dataset Construction
- ENSUM combines CNN/Dailymail and MSMO, while LCSTS supplies Chinese short-text summaries as the source monolingual datasets.The cited source datasets contain 626,634 English and 2,400,591 Chinese summarization pairs.Source: §2 · p. 2
- Round-trip translation sends text into another language and back, and the paper uses it to derive CLS document-summary pairs from monolingual summarization datasets.The En2Zh example translates the summary into Chinese and then back into English before pairing the Chinese summary with the original document.Source: §2 · p. 2
- Pairs are filtered by comparing original and round-trip-translated reference text with ROUGE; summaries are processed sentence by sentence, and samples are kept when more than two-thirds of summary sentences remain.The stated ROUGE-1 threshold T2 is 0.2; T1 is 0.45 for English and 0.6 for Chinese.Source: §2 · p. 2
- The construction process yields 370,759 En2Zh pairs from ENSUM and 1,699,713 Zh2En pairs from LCSTS; volunteers corrected test-set references.These counts are reported after round-trip translation.Source: §2 · p. 3
3 Approach
The approach replaces pipeline-based cross-lingual summarization with end-to-end NCLS, then incorporates monolingual summarization and machine translation into CLS training through multi-task learning.
3.1 Baseline Pipeline Methods
- Early Translation translates the document before summarization, while Late Translation summarizes first and translates the summary afterward.Both are pipeline strategies for cross-lingual summarization.Source: §3.1 · p. 3
3.2 Neural Cross-Lingual Summarization
- NCLS uses a Transformer encoder to represent the source document and a decoder to generate a summary in a different language.The encoder and decoder are trained jointly to maximize the target sequence probability given the source.Source: §3.2 · p. 3, Figure 2 · p. 3
- The Transformer encoder uses self-attention followed by a feed-forward block; its decoder adds encoder-decoder attention and masks future positions during training.Residual connections and layer normalization are used around each block.Source: §3.2 · pp. 3–4
3.3 Improving NCLS with MS and MT
- CLS and monolingual summarization share the goal of capturing a document’s core idea, while machine translation is treated as CLS with a 1:1 compression ratio.These relationships motivate using both tasks to improve CLS through multi-task learning.Source: §3.3 · p. 4
- The one-to-many scheme shares an encoder across tasks and uses multiple decoders; the paper studies CLS+MS and CLS+MT.Figure 3 depicts the multi-task setup, with CLS+MT in the lower half.Source: §3.3 · p. 4, Figure 3 · p. 4
- For CLS+MS, one encoded source document feeds two decoders that simultaneously generate the outputs for their respective tasks.The CLS dataset includes bilingual versions of its references.Source: §3.3 · p. 4
- CLS+MT alternates training between the tasks, optimizing each for a fixed number of mini-batches before switching.The MT training uses 2.08M sentence pairs from LDC corpora alongside the CLS dataset.Source: §3.3 · p. 4
4 Experiments
The experiments compare end-to-end NCLS with pipeline baselines, test dataset filtering and multi-task training, and assess generated summaries quantitatively and by human evaluation. NCLS outperforms traditional pipeline methods on both language directions, while multi-task training improves summary quality and case-study outputs are generally more accurate than the baselines.
4.1 Experimental Settings
- The experimental setup varies segmentation granularity and truncates inputs to 200 English words and outputs to 120 words or 150 Chinese characters.The reported training configuration uses Adam, a single NVIDIA TITAN X, and beam search of size 4 at test time.Source: §4.1 · pp. 4–5
4.2 Baselines and Model Variants
- The experiments use Transformer-based models and compare pipeline variants using Transformer MT or Google Translator against TNCLS variants.The paper also reports word, subword, and character segmentation configurations, with training settings including a six-layer encoder and decoder.Source: §4.2 · p. 5
4.3 Experimental Results and Analysis
- NCLS significantly outperforms traditional pipeline methods on both En2Zh and Zh2En CLS tasks.Table 4 reports ROUGE F1 scores on the test sets and their human-corrected counterparts.Source: Table 4 · p. 6, §4.3 · p. 6
- The Filter dataset beats the same-sized Pseudo-Filter dataset on both tasks, supporting the value of back-translation filtering.For En2Zh, however, Non-Filter performs best when reliable news translation and its nearly doubled data scale offset translation noise; for Zh2En, it performs worse than Filter.Source: §4.3 · pp. 6–7, Table 5 · p. 7
- CLS+MS and CLS+MT improve NCLS; CLS+MT significantly beats CLS+MS on En2Zh, while their Zh2En performance is comparable.The authors relate the larger En2Zh benefit to the larger, news-domain MT dataset, which matches the En2ZhSum domain better than Zh2EnSum's social-media domain.Source: §4.3 · p. 7, Table 6 · p. 7
- Human ratings find TNCLS more informative than GLTran, with comparable conciseness and fluency; adding MS or MT significantly improves conciseness and fluency.The evaluation uses 25 random samples from each test set, three graduate raters, and 1–5 ratings for informativeness, conciseness, and fluency.Source: §4.3 · pp. 7–8, Table 7 · p. 8
4.4 Case Study
- The case study shows pipeline summaries with translation or summarization errors, while TNCLS preserves logical accuracy and multi-task outputs generally match the facts.CLS+MS matches the gold summary better in this example, although both multi-task summaries omit the source's scale.Source: §4.4 · p. 8
5 Related Work
Prior CLS work largely used translation-and-summarization pipelines, with some methods drawing on bilingual information. End-to-end approaches and automatic construction of large-scale CLS datasets remained limited or unexplored, motivating this paper's English-Chinese and Chinese-English corpora and models.
5 Related Work
- Earlier CLS methods commonly used early translation before summarization or late translation after summarization, leaving the task in a pipeline paradigm.Examples include Hindi-to-English headline generation and English-to-Chinese sentence selection followed by translation.Source: §5 · pp. 8–9
- Bilingual graph, compression, and concept-based methods used information from both languages but still relied on pipelines and hand-crafted features.These approaches were also described as susceptible to error propagation.Source: §5 · p. 9
- Zero-shot cross-lingual headline and sentence summarization had been explored, but automatic construction of high-quality large-scale CLS datasets remained unexplored.The paper focuses on English-to-Chinese and Chinese-to-English corpora and end-to-end models.Source: §5 · p. 9
6 Conclusion and Future Work
The paper presents NCLS, an end-to-end CLS framework trained on large-scale supervised data acquired from monolingual summarization datasets through round-trip translation. Its models outperform traditional pipelines, and incorporating machine translation and monolingual summarization further improves generated summaries; future work extends the strategy to other language pairs.
6 Conclusion and Future Work
- Round-trip translation of existing monolingual summarization datasets provides large-scale supervised CLS data for end-to-end NCLS.The authors propose applying the strategy to English-Japanese, English-German, Chinese-Japanese, and Chinese-German datasets in future work.Source: §6 · p. 9
- NCLS models significantly outperform traditional pipeline methods, and incorporating MT and MS can significantly improve generated summaries.Source: §6 · p. 9