Source-linked AI summary
Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debates
Javier Iranzo-Sánchez, Joan Albert Silvestre-Cerdà, Javier Jorge, Nahuel Roselló, Adrià Giménez, Albert Sanchis, Jorge Civera, Alfons Juan
TL;DR
SLT research lacks broad, reproducible multilingual corpora, especially beyond English-centered language pairs. The paper introduces Europarl-ST, built from European Parliament debates with a filtering and alignment pipeline, and evaluates it across ASR, MT, and cascade SLT. Fine-tuning on Europarl-ST improves MT by +1.9 to +4.0 BLEU, while the corpus supports evaluation and parliamentary-domain adaptation.
Problem
SLT lacks sufficient multilingual corpora, with many resources centered on English and reported results relying on paid or private datasets.
Method
The paper builds Europarl-ST from European Parliament debates using diarization, forced alignment, CER filtering, sentence alignment, and speaker-independent partitions.
Results
+1.9 to +4.0 BLEU: fine-tuned systems trained on Europarl-ST improve over out-of-domain systems.
Takeaways & Limitations
The released multi-source, multi-target corpus supports evaluation of out-of-domain systems and adaptation to parliamentary debates.
Takeaways & Limitations
Future work is needed to extend the filtering pipeline to additional languages and increase hours available for each language pair.
Abstract
from arXiv · showhide
Current research into spoken language translation (SLT),or speech-to-text translation, is often hampered by the lack of specific data resources for this task, as currently available SLT datasets are restricted to a limited set of language pairs. In this paper we present Europarl-ST, a novel multilingual SLT corpus containing paired audio-text samples for SLT from and into 6 European languages, for a total of 30 different translation directions. This corpus has been compiled using the debates held in the European Parliament in the period between 2008 and 2012. This paper describes the corpus creation process and presents a series of automatic speech recognition, machine translation and spoken language translation experiments that highlight the potential of this new resource. The corpus is released under a Creative Commons license and is freely accessible and downloadable.
1. INTRODUCTION
Recent advances in ASR and MT have increased interest in SLT, but the field remains unresolved and lacks broad, reproducible multilingual data resources.
- 1. INTRODUCTION: Advances in deep learning and greater data availability have driven renewed interest in SLT as the convergence of ASR and MT.The paper frames SLT as benefiting from developments in both component fields.
- 1. INTRODUCTION: SLT combines automatic speech recognition and machine translation, but cascade and end-to-end approaches remain without clear community-wide adoption.The IWSLT 2018 evaluation reported better performance for cascade systems, although more recent developments are narrowing the gap.
- 1. INTRODUCTION: SLT performance, particularly for end-to-end models, is constrained by fewer corpora than the better-resourced ASR and MT fields.Existing resources often pair English speech with translations, limiting research in other language pairs.
- 1. INTRODUCTION: Many reported SLT results rely on paid Fisher/Callhome corpora or private datasets, limiting reproducibility.The paper identifies restricted access as a barrier for the research community.
2. DATA COLLECTION AND PROCESSING
Europarl-ST is constructed from European Parliament speeches through audio alignment, CER-based filtering, sentence alignment, and speaker-independent partitioning, while source-data quality limits coverage.
- 2. DATA COLLECTION AND PROCESSING: Europarl-ST uses publicly available European Parliament debate videos, with each speech defined as one speaker’s intervention.LinkedEP provides access to attributes of individual debates.
- 2. DATA COLLECTION AND PROCESSING: Missing videos, inaccurate timestamps, and missing non-English translations constrain the source material available for corpus construction.Translations after 2012 are unavailable, and additional data is discarded to form audio-transcription-translation triples.
- 2.1. Audio-to-text alignment and data filtering: Speaker diarization, longest same-speaker audio clipping, and forced alignment are used to correct inaccurate speech boundaries and obtain word timestamps.The pipeline uses LIUM SpkDiarization and the TLK decoder for alignment.
- 2.1. Audio-to-text alignment and data filtering: CER-based filtering removes speeches with non-verbatim, non-transcribed, or unuttered content after ASR outputs are compared with reference transcriptions.CER is selected because it better measures phonetic similarity and reduces the effect of ASR out-of-vocabulary words.
- 2.1. Audio-to-text alignment and data filtering: 15% CER thresholds apply to French, German, and Spanish, while English uses 20%, discarding roughly 40–70% of previously selected audio.The retained-data CER figures are reported as an approximate quality-assurance measure.
- 2.1. Audio-to-text alignment and data filtering: Transcriptions and translations are sentence-split, aligned, shortened when longer than 20 seconds, and further paired through word alignments.The remaining corpus statistics exclude speeches without translations in the relevant target language.
- 2.1. Audio-to-text alignment and data filtering: Speaker-independent train, development, and test partitions allocate approximately 3 hours to each of the development and test sets.The training data supports domain adaptation or training in-domain ASR, MT, and SLT models.
3. EXPERIMENTS AND RESULTS
The experiments evaluate ASR, MT, and cascade SLT using Europarl-ST, including general-domain performance and adaptation to parliamentary debates. Fine-tuning on Europarl-ST substantially improves MT, while cascade SLT preserves MT performance rankings despite lower overall BLEU.
- 3. EXPERIMENTS AND RESULTS: The study evaluates ASR, MT, and cascade SLT, testing both general-domain systems and adaptation using Europarl-ST training data.Cascade SLT feeds ASR output into MT; hypotheses are re-segmented before BLEU evaluation.
- 3.1. ASR: ASR WER is below 20% for most language pairs, but pairs with French input perform worse because its system lacks BLSTM acoustic models and has fewer acoustic resources.Table 4 reports WER on the test sets, with rows representing source languages and columns target languages.
- 3.2. MT: MT systems use Transformer models trained mainly on OPUS data, excluding Europarl, then fine-tuned with Europarl-ST data.The training-data tables report the corpora and sentence totals used for each translation direction.
- 3.2. MT: +1.9 to +4.0 BLEU: Europarl-ST fine-tuning substantially improves out-of-domain MT systems across translation directions.The improvement varies with available resources and the intrinsic difficulty of each translation direction.
- 3.3. SLT: Cascade SLT obtains lower overall BLEU than MT, but preserves the ranking across translation directions: stronger MT systems also produce stronger SLT results.The cascade combines the ASR and MT models, using silence-based segmentation of ASR output.
4. CONCLUSIONS
The paper introduces Europarl-ST, a multilingual corpus built from European Parliament proceedings, and reports that its filtered data supports evaluation and domain adaptation. Future work targets additional languages, more available speech hours, and end-to-end SLT comparisons.
- 4. CONCLUSIONS: Europarl-ST is a novel SLT corpus built from European Parliament proceedings, with filtering that produces data useful for evaluation and parliamentary-domain adaptation.The corpus is released as a multi-source and multi-target resource for multilingual SLT research.
- 4. CONCLUSIONS: The filtering pipeline can be extended to additional languages and improved to increase the available hours for each language pair.These extensions are identified as future work.
- 4. CONCLUSIONS: Future experiments will compare end-to-end models with cascade systems adapted to ASR output using real or simulated recognition errors.The proposed adaptation uses real ASR output or source-side noising techniques.