Source-linked AI summary
The Parallel Meaning Bank: Towards a Multilingual Corpus of Translations Annotated with Compositional Meaning Representations
Lasha Abzianidze, Johannes Bjerva, Kilian Evang, Hessel Haagsma, Rik van Noord, Pierre Ludmann, Duc-Duy Nguyen, Johan Bos
TL;DR
The paper addresses the challenge of building multilingual semantic resources without independently annotating every language. It constructs the Parallel Meaning Bank by projecting corrected English semantic annotations onto aligned translations through language-neutral compositional analysis, producing a corpus of over 11.3 million tokens across four languages. The first Dutch results are promising, while the method’s manual effort and cross-lingual accuracy remain to be assessed in other languages.
Problem
Building semantic resources for multiple languages is costly, while shared meaning representations could transfer annotations through meaning-preserving translations.
Method
The PMB projects English annotations onto word-aligned Dutch, German, and Italian translations using CCG-based compositional analysis and language-neutral semantic representations.
Results
Over 11.3 million tokens across 285,154 documents form the corpus, and initial Dutch results are reported as promising.
Takeaways & Limitations
The PMB provides a shared multilingual resource for studying cross-lingual compositional semantics, training non-English semantic parsers, and detecting translation meaning shifts.
Takeaways & Limitations
The method still requires assessment of manual effort in German and Italian and whether projection reduces time and annotation costs relative to building a language resource from scratch.
Abstract
from arXiv · showhide
The Parallel Meaning Bank is a corpus of translations annotated with shared, formal meaning representations comprising over 11 million words divided over four languages (English, German, Italian, and Dutch). Our approach is based on cross-lingual projection: automatically produced (and manually corrected) semantic annotations for English sentences are mapped onto their word-aligned translations, assuming that the translations are meaning-preserving. The semantic annotation consists of five main steps: (i) segmentation of the text in sentences and lexical items; (ii) syntactic parsing with Combinatory Categorial Grammar; (iii) universal semantic tagging; (iv) symbolization; and (v) compositional semantic analysis based on Discourse Representation Theory. These steps are performed using statistical models trained in a semi-supervised manner. The employed annotation models are all language-neutral. Our first results are promising.
1 Introduction
The PMB builds multilingual corpora with shared formal meaning representations by projecting semantic annotations across aligned translations. It uses compositional analyses to study cross-lingual variation and support multilingual semantic parsing and translation-shift detection.
- Motivation: The PMB addresses the premise that meaning representations can be shared across languages by projecting annotations from one language onto aligned translations.The approach assumes meaning-preserving translations have equivalent meaning representations.
- Motivation: Existing semantic resources support semantic parsing but often require expensive manual or semiautomatic annotation.The paper situates the PMB alongside AMR banks and the Groningen Meaning Bank.
- Approach: The paper proposes meaning banks for English, Dutch, German, and Italian using translations and compositional derivations based on CCG.These derivations provide the basis for projecting semantic information between translated sentences.
- Pipeline: Manual corrections can be added at each annotation layer of the PMB pipeline.The figure presents the pipeline as a sequence of annotation layers that remain open to correction.
- Goals: The PMB will test cross-lingual compositional semantics, provide data for non-English semantic parsers, and help detect meaning shifts in translation.Its goals cover loose translations, differing semantic granularities, parser development, and translation analysis.
2 Languages and Corpora
The PMB is built from large, freely distributable parallel-text collections spanning multiple genres and languages. Its resulting corpus exceeds 11.3 million tokens, with English present in every document and uneven coverage across the other languages.
- Corpus construction: The PMB is founded on a large collection of raw parallel texts, preferably freely distributable rather than requiring paid licensing.For the projection method, an English version and one additional language are sufficient.
- Language selection: The corpus includes Dutch and German as English-related Germanic languages and Italian to test projection across greater typological difference.The language selection combines similarity with a more distant Romance-language case.
- Corpus construction: The source material spans twelve corpora and a wide range of genres, including Tatoeba, News-Commentary, textual-entailment data, Sherlock Holmes, and the Bible.The corpus combines resources with substantially different textual domains.
- Gold standard: Two of 100 balanced corpus parts are initially selected as a manually annotated gold standard to represent the full genre range.The selected parts are 00 and 10.
- Corpus statistics: Over 11.3 million tokens are divided across 285,154 documents, all with English; 72% have German, 14% Dutch, and 42% Italian versions.Only 5% of documents exist in all four languages.
3 Automatic Annotation Pipeline
The PMB pipeline richly annotates English text and derives compositional, language-neutral meaning representations through successive linguistic and semantic layers.
- Pipeline overview: The pipeline annotates English text through segmentation, CCG parsing, semantic tagging, symbolization, and semantic interpretation.Figure 2 presents these layers in sequence, culminating in compositional lexical and sentence-level semantics.
- Segmentation: Segmentation identifies word and sentence boundaries while treating multiword constituents as single tokens and decomposing transparent compounds.The character-level IOB scheme supports assigning atomic meanings to tokens.
- Syntactic parsing: CCG derivations support compositional semantics and cross-lingual projection, with empty elements making type-changing rules explicit.The modified grammar retains compositionality while accommodating projection between languages.
- Semantic tagging: Universal semantic tags are language-neutral, generalize over part-of-speech and named-entity classes, and provide semantic information for lexical analysis.The tagset contains 80 fine-grained semtags in 13 coarse-grained classes; a deep residual tagger reports 83.6% accuracy.
- Symbolization: Symbolization combines lemmatization, normalization, and lexical disambiguation to produce symbols that, with CCG categories and semtags, determine token semantics.The approach favors forms that preserve recoverability and alignment, and can be learned with a character-based sequence-to-sequence model.
- Semantic interpretation: DRT supplies the PMB’s semantic formalism, while Boxer converts decorated CCG derivations into lexical and sentence-level DRSs compositionally.Each token receives a lexical DRS, and the complete sentence representation is assembled from these components.
4 Cross-lingual Projection
The PMB projects English annotations to other languages through sentence and word alignment, then improves language-neutral models using manual corrections and retraining.
- Projection: Dutch, German, and Italian receive initial semtags, CCG categories, and symbols from automatically aligned English counterparts.The projection bootstraps non-English annotation while encouraging parallel syntactic and semantic analyses.
- Projection: CCG slashes and 2:1 alignments are adjusted so target derivations produce the same DRS as the English sentence.Functional composition handles 2:1 alignments during projection.
- Alignment: Sentence alignment begins with an ordered one-to-one heuristic, after which experts correct it and GIZA++ automatically aligns words.Projection therefore operates at both sentence and word levels.
- Model improvement: Manual corrections and retraining are used to improve models beyond the initial combination of existing tools and automatic projection.Training only on automatic annotations would not improve performance over that initial combination.
5 Adding Bits of Wisdom
Bits of Wisdom are manual corrections that classify annotation layers by review status and focus expert effort on unstable or difficult cases.
- Bits of Wisdom: Manual corrections can be applied to any annotation layer in any of the four languages, overriding conflicting model annotations.These corrections are called Bits of Wisdom, or BoWs.
- Annotation classes: Gold layers are manually checked, silver layers contain at least one BoW, and bronze layers contain no BoWs.Table 1 distributes these classes across languages and documents.
- Conflict resolution: Annotation conflicts arise when automatic updates alter a manually checked gold layer and introduce possible new errors.Such conflicts are marked for expert review after model retraining.
- Conflict resolution: Conflict-focused review concentrates annotation on difficult cases and can correct human inconsistencies while improving guidelines.BoWs also steer models more effectively when their judgments remain unstable.
6 Conclusion
The paper lays a foundation for language-neutral analysis tools by combining language-neutral schemes with language-specific models, while identifying unresolved evaluation and cost questions.
- Conclusion: The pipeline uses one language-specific model for each task—tokenization, parsing, semantic tagging, symbolization, and semantic interpretation—toward language-neutral analysis tools.The authors also propose language-neutral tagging schemes and extend CCG with lexical categories for empty elements.
- Open evaluation: Dutch results are promising, but the manual effort required for German and Italian remains to be assessed.The authors also plan to evaluate whether projection reduces time and annotation costs relative to building tools from scratch.
- Data access: The PMB’s annotated data are publicly accessible through a web interface, with stable downloadable releases planned periodically.This provides an access route for using the corpus beyond the paper.