Source-linked AI summary
The Multilingual FrameNet Corpus
Beatrice Fiumanò, Nicolas Lazzari, Simone Paolo Ponzetto, Valentina Presutti
TL;DR
Frame Semantic Parsing lacks a unified, large-scale multilingual corpus because existing language-specific FrameNets are scattered and heterogeneous. The paper introduces and harmonizes the mFNC across ten languages, then evaluates FSP architectures on it. Multilingual training consistently improves multilingual and cross-lingual performance, while the corpus still has limited language coverage.
Problem
Frame Semantic Parsing lacks unified multilingual training and evaluation data because existing language-specific FrameNets are scattered and heterogeneous.
Method
The paper constructs the mFNC by collecting and harmonizing ten BFN-annotated language-specific corpora, then trains and evaluates multiple FSP architectures on it.
Results
Multilingual training consistently outperforms English-only training in multilingual and cross-lingual settings.
Takeaways & Limitations
The mFNC provides a unified resource for multilingual FSP and for comparative research on frame-semantic differences across languages.
Takeaways & Limitations
The mFNC has limited coverage, including no representation for Middle-Eastern or African languages.
Abstract
from arXiv · showhide
This paper introduces the Multilingual FrameNet Corpus (mFNC), a novel resource that extends the English Berkeley FrameNet corpus by collecting and harmonizing existing language-specific corpora across nine additional languages: Brazilian Portuguese, Chinese, Dutch, French, German, Italian, Korean, Latvian and Swedish. By training models that rely on different architectures on the mFNC, we consistently outperform existing state-of-the-art Frame Semantic Parsers in both multilingual and cross-lingual settings, underscoring the importance of multilingual training data. The mFNC and our trained FSP models are openly available at https://github.com/beatrice-f/mFNC.
1 Introduction
The paper addresses the underexplored multilingual dimension of Frame Semantic Parsing by introducing the mFNC, a harmonized corpus spanning ten languages. Models trained on multilingual data consistently outperform English-only models in multilingual and cross-lingual settings.
- Motivation: FSP automatically identifies semantic frames in text according to Frame Semantics theory.The task supports applications including information extraction, knowledge graph construction, opinion mining, sentiment analysis, and framing detection.
- Motivation: Multilingual FSP remains underexplored because existing systems and resources are predominantly English-focused, scattered, and heterogeneous.No unified corpus integrating the independent language-specific FrameNets was known to exist.
- Contribution: The mFNC collects and harmonizes ten BFN-annotated language-specific corpora across English and nine additional languages.The additional languages are Brazilian Portuguese, Chinese, Dutch, French, German, Italian, Korean, Latvian, and Swedish.
- Results: Multilingual training consistently outperforms English-only training for FSP models in multilingual and cross-lingual settings.The evaluation compares models trained on BFN with models trained on the mFNC.
2 Related Work
The related work defines FrameNet-based semantic parsing, reviews multilingual FrameNet resources and annotation strategies, and shows that multilingual and cross-lingual FSP research remains limited. Existing systems largely rely on English BFN data, with few multilingual extensions and no previously proposed multilingual train-and-test FSP models.
- Multilingual FrameNets: FrameNet resources extend BFN through projection, culture-specific inventories, or hybrid reuse and expansion of the English frame repertoire.The paper focuses on resources that fully or partially reuse English frames to support cross-lingual alignment.
- Frame Semantic Parsing: FSP annotates sentences with semantic frames by detecting frame-evoking lexical units, assigning frames, and identifying frame-element spans.Systems may solve these sub-tasks separately or jointly.
- Applications: FSP makes predicate-argument structure explicit and supports applications such as semantic matching for question answering and relation-triplet extraction.These applications use frame-semantic structure beyond surface string overlap.
- Multilingual FSP: Most state-of-the-art FSP models are trained and tested on BFN, while multilingual and cross-lingual settings receive less attention.LOME uses multilingual XLM-RoBERTa but is evaluated only on English.
- Multilingual FSP: No FSP models trained and evaluated in a multilingual setting had been proposed, to the authors’ knowledge.Prior work included multilingual embeddings and cross-lingual evaluation, but not the same multilingual train-and-test setting.
3 mFNC: Multilingual FrameNet Corpus
The mFNC harmonizes ten BFN-annotated language-specific corpora into a shared multilingual resource. Its composition reveals substantial variation in annotation density, frame coverage, and similarity across languages.
- Data selection: The mFNC combines ten BFN-annotated corpora in a shared format, covering English plus nine additional languages.The component resources differ in development strategy, document genre, and accessibility.
- Harmonization: The harmonization process standardizes plain and tokenized documents, recovers or creates missing text representations, and collects compatible LU, frame, and FE annotations.Language-specific frames from hybrid resources are removed to maintain compatibility with BFN.
- Corpus composition: The resulting corpus contains 1.5M tokens from approximately 70k sentences, with over 100k annotated frames and 200k annotated FEs.English uses the established BFN splits, and exemplar sentences are excluded.
- Frames coverage: The number of occurrences across frames follows a Zipfian distribution, and 151 of BFN’s 1221 frames never occur in mFNC annotations.CAUSATION and STATEMENT are most frequent, whereas CAUSATION_SCENARIO and EXPLOSION occur only once.
- Annotation density: Annotation density varies substantially: English and Brazilian Portuguese average twice as many annotated frames per document as other languages, while French documents are longer with similar frame counts.The French resource therefore has comparatively lower annotation density.
- Frame similarity: Frame similarity compares language centroids built from FFICF typicality vectors, revealing domain- and construction-related differences across resources.Dutch and French are most dissimilar to BFN and other resources, whereas Korean and Swedish resemble BFN more closely.
4 Experiments
The experiments compare FSP architectures and training on the English BFN corpus versus the multilingual mFNC. Multilingual training maintains competitive English performance, improves multilingual results, and offers stronger cross-lingual generalization, though gains vary by language.
- Experimental setting: The study compares LOME’s multi-stage sequence-labeling architecture with generative mT5-based seq2seq models trained on BFN and mFNC.LOME uses XLM-RoBERTa, a CRF span extractor, and MLP classifiers; the generative approach fine-tunes small and base mT5 models.
- Results on the BFN corpus: Training on the mFNC maintains competitive performance with BFN training on the English target identification and classification task.The evaluation uses traditional micro-averaged F1 and requires exact matching with gold annotations.
- Multilingual performance: LOME trained on the mFNC outperforms existing state-of-the-art FSP models in multilingual evaluation.The comparison aggregates FairEval precision, recall, and F1 over ten languages, with FE metrics measuring end-to-end performance.
- Multilingual performance: Each architecture trained on the mFNC greatly outperforms its English-only counterpart across the evaluated multilingual dimensions.LOME consistently performs best among the evaluated models, supporting sequence labeling over seq2seq modeling for FSP in these experiments.
- Limitations: Performance gains are not equally distributed across the ten languages, and the mFNC’s unbalanced frame coverage may hamper generalization to infrequent frames.The authors leave open whether frame-frequency annealing or hierarchy-based data augmentation could counterbalance this effect.
- Language-wise improvements: English performance is similar across training settings, while improvements are strongest in German, French, Dutch, and Latvian; Swedish shows less pronounced gains.The Swedish corpus is identified as the most challenging, and the figure compares frame and FE F1 scores by language.
- Cross-lingual performance: The mFNC improves cross-lingual generalization over BFN training when Swedish data is removed and LOME is trained from scratch.The cross-lingual comparison evaluates BFN, mFNC without Swedish data, and the full mFNC under the same setting.
5 Discussion
The discussion finds that mFNC training improves multilingual and cross-lingual FSP performance, while task-specific LOME models outperform general seq2seq approaches. It also identifies coverage, annotation balance, and resource integration as boundaries for the corpus and models.
- Findings: BFN-trained state-of-the-art FSP models struggle cross-lingually, but their results greatly improve with the mFNC in multilingual and cross-lingual settings.The discussion connects the performance gap to the corpus used for training rather than to a single evaluation setting.
- Findings: LOME models perform significantly better than more general mT5-based seq2seq approaches.This comparison favors models tailored specifically to the FSP task.
- Limitations: The mFNC is unbalanced in annotated-frame counts, and this directly affects FSP-model effectiveness.The discussion links this imbalance to the language-wise behavior observed in the experiments.
- Integrating the mFNC with other resources: Integrating resources such as PropBank and VerbNet could broaden linguistic coverage, but their different resource nature from BFN complicates direct adoption.The authors specifically note that PropBank frames are more lexically focused than BFN’s semantically oriented frames.
- Extending the mFNC: The mFNC currently lacks representation for Middle-Eastern and African languages and has limited coverage relative to other multilingual datasets.Possible extensions include annotation projection or language-model-generated annotated sentences, although fully automated projection may produce imprecise annotations.
- Frame-based linguistic analyses: By harmonizing ten language-specific datasets, the mFNC supports cross-lingual analyses of how semantically equivalent expressions vary across languages.Its value therefore extends beyond training FSP models to studying linguistic diversity and frame-semantic variation at scale.
6 Conclusion
The mFNC harmonizes ten FrameNet resources to address the lack of multilingual FSP data. Training on multilingual data substantially improves multilingual and cross-lingual systems and supports comparative research across languages.
- The mFNC is a multilingual dataset harmonizing ten language-specific resources annotated using FrameNet.
- The dataset addresses the lack of multilingual training and evaluation data for Frame Semantic Parsing.
- Training on multilingual data substantially improves the performance of multilingual and cross-lingual systems.
- The mFNC enables comparative research on conceptual and frame-semantic differences across languages.
7 Limitations
The paper identifies limitations in mFNC construction and experimentation, including linguistic, domain, annotation, coverage, modeling, and evaluation constraints. These limitations leave open questions about cross-lingual balance, low-resource generalization, and downstream applicability.
- On constructing the mFNC: Combining BFN-annotated corpora assumes that BFN’s conceptual structure transfers correctly to other languages, although this remains an open research question.
- On constructing the mFNC: Translations and projective annotations reveal limits to transferring BFN across languages, because not all BFN frames apply equally to every language.
- On constructing the mFNC: Domain-specific Dutch and French corpora may under-represent frames associated with topics absent from their predominant document domains.
- On constructing the mFNC: Combining resources assumes consistent human annotation, but cross-annotator differences may create uneven frame distributions across corpora.
- On constructing the mFNC: Retaining original tokenization combines potentially incompatible practices, while detokenization or automated retokenization may introduce noise.
- On constructing the mFNC: Because the mFNC mostly covers high-resource languages, the evaluated systems’ generalization to low-resource languages remains uncertain.
- On experimenting with the mFNC: FSP models assume each lexical unit and frame element has a single correct frame, inheriting unknown annotator-perspective biases.
- On experimenting with the mFNC: Current evaluation metrics may over- or underestimate system applicability, and downstream improvements remain an open problem.
A Appendix
The appendix provides visual and tabular summaries of frame distributions and model results across the multilingual resources and languages.
- Figure 6 reports the number of occurrences of all BFN frames for each resource in Table 2 using logarithmic space.
- Table 6 lists the five most common annotated frames for each language-specific corpus in Table 2.
- Table 7 reports per-language model results computed with the FairEval framework.