Source-linked AI summary
Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment
Muhao Chen, Yingtao Tian, Mohan Yang, Carlo Zaniolo
TL;DR
Cross-lingual knowledge alignment is important but remains sparse and difficult because multilingual graphs use separate vocabularies and only partial alignments are available. MTransE jointly learns language-specific knowledge representations and cross-lingual transitions, and its experiments identify linear transformations as the strongest alignment technique while preserving monolingual embedding properties. The authors point to more advanced knowledge-model losses and broader cross-lingual completion tasks as future work.
Problem
Cross-lingual entity and relation alignment remains largely unexplored despite its usefulness for synchronizing multilingual knowledge bases, with Wikipedia inter-lingual links covering less than 15% of entity alignment.
Method
MTransE combines language-specific knowledge models with alignment models that learn transitions across separate embedding spaces using axis calibration, translation vectors, and linear transformations.
Results
Experiments on cross-lingual entity matching and triple alignment verification find linear transformations best among the three techniques, while MTransE preserves key monolingual embedding properties.
Takeaways & Limitations
MTransE provides an automated alignment approach that can learn from partially aligned multilingual graphs while retaining monolingual knowledge-graph functionality.
Takeaways & Limitations
The authors identify replacing MTransE's simple knowledge-model loss with relation-specific entity transformations and extending evaluation to more sophisticated cross-lingual triple-completion tasks as future work.
Abstract
from arXiv · showhide
Many recent works have demonstrated the benefits of knowledge graph embeddings in completing monolingual knowledge graphs. Inasmuch as related knowledge bases are built in several different languages, achieving cross-lingual knowledge alignment will help people in constructing a coherent knowledge base, and assist machines in dealing with different expressions of entity relationships across diverse human languages. Unfortunately, achieving this highly desirable crosslingual alignment by human labor is very costly and errorprone. Thus, we propose MTransE, a translation-based model for multilingual knowledge graph embeddings, to provide a simple and automated solution. By encoding entities and relations of each language in a separated embedding space, MTransE provides transitions for each embedding vector to its cross-lingual counterparts in other spaces, while preserving the functionalities of monolingual embeddings. We deploy three different techniques to represent cross-lingual transitions, namely axis calibration, translation vectors, and linear transformations, and derive five variants for MTransE using different loss functions. Our models can be trained on partially aligned graphs, where just a small portion of triples are aligned with their cross-lingual counterparts. The experiments on cross-lingual entity matching and triple-wise alignment verification show promising results, with some variants consistently outperforming others on different tasks. We also explore how MTransE preserves the key properties of its monolingual counterpart TransE.
1 Introduction
Cross-lingual knowledge alignment remains underexplored despite its value for synchronizing multilingual knowledge bases. MTransE addresses this gap by learning language-specific embeddings together with cross-lingual transitions while preserving monolingual structure.
- Motivation: Cross-lingual knowledge includes entity matches and relation alignments that help synchronize independently evolving language-specific knowledge bases.These alignments support applications including question answering, the semantic Web, and Web search.
- Motivation: Less than 15% of entity alignments are covered by inter-lingual links in Wikipedia.
- Motivation: Cross-lingual transitions are difficult because they span larger domains, cover entities and relations with incoherent vocabularies, and rely on sparse alignment data.Monolingual knowledge-graph structure must also be preserved for correct alignment.
- Approach: MTransE combines a knowledge model with an alignment model to encode each language-specific graph separately and learn cross-lingual transitions.It considers axis calibration, translation vectors, and linear transformations, producing five variants with different loss functions.
- Evaluation: MTransE variants are evaluated on cross-lingual alignment tasks using partially aligned trilingual Wikipedia graphs and compared with TransE on monolingual tasks.The paper reports that MTransE performs as well as TransE on monolingual tasks.
2 Related Work
Related work spans translation-based and non-translation-based knowledge-graph embeddings, multilingual word embeddings, and knowledge-base alignment systems. Existing approaches differ in expressive power, complexity, training strategy, and reliance on human effort or handcrafted features.
- Knowledge Graph Embeddings: Translation-based knowledge-graph models represent relations as transformations between entity embeddings and support knowledge-graph completion.TransE variants specialize entity embeddings for different relations, improving multi-mapping completion at increased model complexity.
- Knowledge Graph Embeddings: Some knowledge-graph embedding methods omit explicit relation embeddings, while neural and random-walk models support structured and text corpora but can be complex.
- Multilingual Word Embeddings: Multilingual word-embedding methods such as LM and CCA induce offline transitions between pretrained monolingual spaces.They do not calibrate inconsistent vector spaces or jointly train with the target knowledge-graph alignment.
- Knowledge Bases Alignment: Knowledge-base alignment projects use crowdsourcing, association-rule mining, or application-specific handcrafted features to create cross-lingual matches.These approaches can require extensive human involvement or fine-tuning.
3 Multilingual Knowledge Graph Embeddings
MTransE combines language-specific TransE knowledge models with alignment models that learn cross-lingual transitions from partially aligned knowledge graphs. It implements axis calibration, translation vectors, and linear transformations, yielding five variants with distinct alignment losses.
- Model Architecture: MTransE combines a knowledge model for language-specific graph structure with an alignment model for cross-lingual transitions.For multilingual knowledge bases, separate language-pair models are defined when alignment data are available.
- Knowledge Model: Each language receives a dedicated k-dimensional embedding space, where TransE encodes entities and relations using translation-based embeddings.The knowledge loss measures triple plausibility and preserves monolingual relations while regularizing alignment.
- Alignment Model: The alignment model scores pairs of aligned triples using three techniques: axis calibration, translation vectors, and linear transformations.Axis calibration brings aligned entity or relation vectors close; translation-vector models add dedicated entity and relation offsets; linear models apply learned matrices between spaces.
- Alignment Model: Linear transformations treat cross-lingual transitions as topological mappings rather than assuming similar spatial organization, with invertibility enabling reverse transitions.Var4 and Var5 use separate matrices for entities and relations because their redundancy differs.
- Variants and Training: MTransE minimizes J = SK + αSA, and its five variants differ in how they compute the alignment score Sa.Cross-lingual counterparts are retrieved by nearest-neighbor search after applying the learned transition.
- Variants and Training: Training alternates stochastic-gradient updates for the knowledge and alignment losses, while unit-norm entity embeddings prevent trivial norm shrinking and imply invertibility for Var4 and Var5.Vectors are initialized on the unit sphere, matrices receive random orthogonal initialization, and negative sampling is not used.
4 Experiments
Experiments evaluate MTransE on cross-lingual entity matching, triple-wise alignment verification, and monolingual prediction tasks. Linear-transformation variants generally lead cross-lingual evaluations while preserving monolingual knowledge characterization.
- Experimental tasks: The experiments cover cross-lingual entity matching, triple-wise alignment verification, and monolingual tail and relation prediction.WK3l datasets are used for the reported experiments, with five MTransE variants compared against related baselines.
- Cross-lingual entity matching: Without jointly adapting monolingual vector spaces, LM and CCA are largely outperformed, while OT performs close to simpler MTransE variants.These comparisons are reported for cross-lingual entity matching and also extend to triple-wise alignment verification.
- Cross-lingual entity matching: Var4 and Var5 outperform the other three MTransE variants across all cross-lingual entity-matching settings.Their close results indicate that adding a relation-dedicated transformation in Var5 causes negligible interference with the entity-dedicated transformation.
- Triple-wise alignment verification: Linear-transformation-based variants lead triple-wise alignment verification, with Var5 remaining within 0.85% of Var4 and Var1 ranking second.Var2 and Var3 trail the optimal solutions by 4.12%–8.44% and 4.52%–10.79%, respectively.
- Monolingual tasks: MTransE preserves monolingual knowledge characterization: Var1, Var4, and Var5 perform at least as well as TransE on each reported setting.The comparison uses Hits@10 for tail prediction and relation prediction in English and French graph versions.
5 Conclusion and Future Work
MTransE generalizes knowledge graph embeddings to multilingual settings by modeling monolingual relations and learning cross-lingual alignment. Experiments identify linear transformations as the strongest alignment technique while preserving monolingual embedding properties, with several directions left for future work.
- MTransE is presented as the first generalization of knowledge graph embeddings to the multilingual scenario.
- The model characterizes monolingual relations and compares axis calibration, translation vectors, and linear transformations for cross-lingual alignment.
- Experiments on cross-lingual entity matching and triple alignment verification identify linear-transformation-based variants as the best among the three techniques.
- MTransE preserves the key properties of monolingual knowledge graph embeddings on monolingual tasks.
- Future Work: Future work includes replacing the knowledge model’s simple loss with relation-specific entity transformations and exploring more sophisticated cross-lingual triple completion.
- Future Work: Combining MTransE with multilingual word embeddings is proposed as a way to extract new relations from multilingual text corpora.
6 Appendix
The appendix illustrates MTransE’s use for cross-lingual entity, relation, and triple completion tasks, and reports additional results on sparse multilingual graphs. Linear-transformation variants perform best in the reported alignment experiments while preserving cross-lingual structural information.
- 6.1 Examples of Knowledge Alignment: Cross-lingual entity and relation candidates are ranked by Euclidean distance to the transformed source vector in the target-language space.Examples are provided for English-to-French and English-to-German matching.
- 6.1 Examples of Knowledge Alignment: Cross-lingual triple completion transfers the known elements of an English triple into another language space to predict the missing element.The appendix presents example query answers and describes basic queries as useful for aided knowledge augmentation.
- 6.1 Examples of Knowledge Alignment: PCA visualization shows transformed English entities retaining their original spatial structure while approaching their corresponding French entities.U.S. cities remain grouped and other countries’ cities remain separated after transformation.
- 6.2 Additional Experimental Results: Var2 incurs a 4%∼9% accuracy drop from Var1 in triple-wise alignment verification because relation-dedicated calibration interferes with optimization.
- 6.2 Additional Experimental Results: On CN3l, linear-transformation variants lead both cross-lingual entity matching and triple-wise alignment verification, while Var1 ranks second in verification.The reported placement is consistent across the five MTransE variants in triple-wise verification.
- 6.2 Additional Experimental Results: CN3l results indicate that MTransE remains promising on very sparse multilingual graphs, with linear transformations the strongest representation technique.