Source-linked AI summary
Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment
Muhao Chen, Yingtao Tian, Kai-Wei Chang, Steven Skiena, Carlo Zaniolo
TL;DR
Sparse inter-lingual links hinder precise cross-lingual inference in multilingual knowledge graphs, despite the availability of entity descriptions. KDCoE co-trains multilingual KG and description embedding models on weakly aligned data, and its performance improves through co-training, surpassing prior models while also supporting zero-shot alignment and cross-lingual KG completion.
Problem
Low coverage of entity alignment hinders precise cross-lingual inference in multilingual knowledge graphs, motivating use of entity descriptions as an additional signal.
Method
KDCoE co-trains a multilingual KG embedding model and a multilingual entity-description embedding model, alternately proposing confident new ILLs for supervision.
Results
KDCoE significantly outperforms previous models on entity alignment and improves zero-shot alignment and cross-lingual KG completion.
Takeaways & Limitations
Co-training strengthens semi-supervised cross-lingual alignment on large, weakly aligned KGs and extends benefits to zero-shot matching and KG completion.
Takeaways & Limitations
The modeling assumes entity pairs have a 1-to-1 mapping, and the chosen knowledge model leaves other techniques for future work because they disable cross-lingual relation inference.
Abstract
from arXiv · showhide
Multilingual knowledge graph (KG) embeddings provide latent semantic representations of entities and structured knowledge with cross-lingual inferences, which benefit various knowledge-driven cross-lingual NLP tasks. However, precisely learning such cross-lingual inferences is usually hindered by the low coverage of entity alignment in many KGs. Since many multilingual KGs also provide literal descriptions of entities, in this paper, we introduce an embedding-based approach which leverages a weakly aligned multilingual KG for semi-supervised cross-lingual learning using entity descriptions. Our approach performs co-training of two embedding models, i.e. a multilingual KG embedding model and a multilingual literal description embedding model. The models are trained on a large Wikipedia-based trilingual dataset where most entity alignment is unknown to training. Experimental results show that the performance of the proposed approach on the entity alignment task improves at each iteration of co-training, and eventually reaches a stage at which it significantly surpasses previous approaches. We also show that our approach has promising abilities for zero-shot entity alignment, and cross-lingual KG completion.
1 Introduction
Multilingual KGs combine relational facts, cross-lingual links, and entity descriptions, but sparse alignment makes cross-lingual inference difficult. KDCoE co-trains KG and description embeddings to improve semi-supervised alignment.
- Multilingual KGs encode monolingual entity facts as triples and synchronize knowledge across languages through inter-lingual links.
- KG embedding models represent entities in low-dimensional spaces and support relational inference through vector algebra.
- ILLs cover less than 20% of entities, limiting the precision of cross-lingual inferences from structured knowledge alone.
- Entity descriptions offer another cross-lingual signal, but matching them requires inadequate labels and handling differences in content, grammar, word order, and multilingual word relatedness.
- KDCoE alternately trains KG and description embedding models, which propose confident new ILLs during co-training to strengthen supervision and improve cross-lingual inference accuracy.
2 Related Work
Prior work extends KG embeddings from monolingual to multilingual settings by aligning language-specific embedding spaces. Existing approaches use transformations, attribute similarity, or self-training, with limitations tied to available attributes and alignment design.
- Monolingual KG Embeddings: Monolingual KG embedding research includes translational models such as TransE, non-translational models such as DistMult and HolE, and neural models such as ConvE.
- Multilingual KG Embeddings: MTransE connects monolingual models through axis calibration, vector translation, or linear transformations across language-specific embedding spaces.
- Multilingual KG Embeddings: MTransE-LT achieves the best performance among MTransE’s alignment techniques, while JAPE strengthens axis calibration using entity-attribute similarity.
- Multilingual KG Embeddings: JAPE performs well on knowledge bases with numerical entity attributes, although such attributes are not generally available in many knowledge bases.
- Multilingual KG Embeddings: ITransE incorporates self-training into a hard-alignment version of MTransE-AC.
3 Modeling
KDCoE models multilingual structured knowledge and entity descriptions with two embedding components trained through iterative co-training. Its KG component combines translational knowledge modeling with linear cross-lingual alignment, while its description component uses attentive recurrent encoders and multilingual word embeddings.
- Co-training: KDCoE iteratively co-trains a multilingual KG embedding model and a multilingual description embedding model, alternately proposing high-confidence ILLs for future training.The procedure reinitializes and retrains both components on the growing ILL training set until neither proposes more alignments.
- Multilingual KG Embeddings: The KG embedding model preserves language-specific entity and relation structure with TransE-style translational modeling and jointly learns cross-lingual alignment.Its objective minimizes SKG = SK + αSA, with a linear transformation mapping entity vectors between language-specific spaces.
- Multilingual KG Embeddings: The KG model constrains entity embedding l2-norms to 1 to avoid trivial collapse toward zero during optimization.The paper contrasts this with constraints requiring norms to be less than 1, which it reports can permit a trivial zero-vector solution.
- Multilingual Entity Description Embeddings: The description embedding model uses attentive GRU layers to encode multilingual entity descriptions and self-attention to emphasize shared semantic information across languages.Attention weights measure the importance of sequence items, and the encoder uses two stacked attentive GRU layers to produce description embeddings.
- Multilingual Entity Description Embeddings: The description model aligns cross-lingual counterpart descriptions by maximizing their embedding-based correspondence, using negative sampling for computationally efficient training.Description embeddings are regularized to unit l2-norm, and negative samples exclude the known cross-lingual counterpart.
4 Experiments
Experiments evaluate KDCoE on cross-lingual entity alignment, zero-shot alignment, and cross-lingual KG completion using the trilingual WK3l60k dataset. Co-training improves alignment across iterations and substantially outperforms prior baselines.
- Evaluation setup: Experiments use the trilingual WK3l60k dataset, extracted from DBpedia to provide ground truth for semi-supervised cross-lingual learning.The evaluation covers cross-lingual entity alignment, zero-shot alignment, and cross-lingual KG completion.
- Cross-lingual entity alignment: Cross-lingual entity alignment is evaluated with Hit@1, Hit@10, and mean reciprocal rank using kNN ranks of target-language entities.KDCoE and baselines are trained on complete KG structures, while its description model uses descriptions covered by the current iteration's ILL set.
- Cross-lingual entity alignment: KDCoE gradually improves MTransE-LT at every co-training iteration, with final stages nearly doubling the best baseline's Hit@1 and substantially improving Hit@10 and MRR.The largest gains occur in the first iterations, when most candidate ILLs are proposed.
- Zero-shot alignment: Zero-shot alignment excludes entities from KG structures during training and evaluates matching using entity descriptions.KDCoE is designed to handle entities absent from KG structure through description representations.
- Zero-shot alignment: 4.04% and 2.63% Hit@1 gains, plus 11.97% and 4.33% Hit@10 gains, occur across the two language settings with co-training.AGRU outperforms other encoders, while self-attention and multilingual word embeddings help capture cross-lingual description semantics.
- Cross-lingual KG completion: KG completion compares KDCoE's KG embedding model with TransE using monolingual and cross-lingual prediction methods on sparse French and German WK3l60k versions.Cross-lingual prediction converts the monolingual completion process into another language's embedding space.
5 Conclusion and Future Work
The paper concludes that KDCoE co-trains multilingual KG and description embeddings for cross-lingual alignment on weakly aligned KGs. It reports strong entity-alignment performance, zero-shot gains, and enhanced KG completion, while proposing broader knowledge models as future work.
- KDCoE co-trains multilingual KG embeddings and entity-description embeddings for cross-lingual knowledge alignment on large, weakly aligned KGs.
- KDCoE significantly outperforms previous models on entity alignment, improves zero-shot description matching through co-training, and enhances KG completion using information from another language.
- Future work includes exploring other knowledge models in KGEM for encoding each language-specific KG structure.