Source-linked AI summary
Aligning Cross-Lingual Entities with Multi-Aspect Information
Hsiu-Wei Yang, Yanyan Zou, Peng Shi, Wei Lu, Jimmy Lin, Xu Sun
TL;DR
Cross-lingual entity alignment must discover equivalent entities across multilingual knowledge graphs despite incomplete inter-lingual links and language differences. The paper combines GCNs over multi-aspect KG information with multilingual BERT for entity descriptions, integrating them through two alignment strategies. Across two benchmark datasets, the approach consistently achieves state-of-the-art performance, while its textual module introduces substantial computational cost and candidate-quality dependence.
Problem
Cross-lingual entity alignment aims to discover missing links between equivalent entities in multilingual knowledge graphs, whose inter-lingual-link coverage can be below 20% in DBpedia.
Method
The method uses MAN and HMAN GCNs to combine topology, relation types, and attributes, then applies multilingual BERT to descriptions and integrates both modules for alignment.
Results
The best model consistently achieves state-of-the-art performance across all datasets and outperforms baselines across evaluated scenarios.
Takeaways & Limitations
Combining multi-aspect graph information with multilingual textual representations provides the paper’s supported approach to cross-lingual entity alignment.
Takeaways & Limitations
POINTWISEBERT is computationally expensive, and reranking performance is bounded by the quality of candidates generated by the GCN models.
Abstract
from arXiv · showhide
Multilingual knowledge graphs (KGs), such as YAGO and DBpedia, represent entities in different languages. The task of cross-lingual entity alignment is to match entities in a source language with their counterparts in target languages. In this work, we investigate embedding-based approaches to encode entities from multilingual KGs into the same vector space, where equivalent entities are close to each other. Specifically, we apply graph convolutional networks (GCNs) to combine multi-aspect information of entities, including topological connections, relations, and attributes of entities, to learn entity embeddings. To exploit the literal descriptions of entities expressed in different languages, we propose two uses of a pretrained multilingual BERT model to bridge cross-lingual gaps. We further propose two strategies to integrate GCN-based and BERT-based modules to boost performance. Extensive experiments on two benchmark datasets demonstrate that our method significantly outperforms existing systems.
1 Introduction
Cross-lingual entity alignment seeks missing links between entities in separate language-specific knowledge graphs. The paper addresses this challenge by combining graph structure, relations, attributes, and multilingual textual descriptions in embedding-based models.
- Multilingual KGs such as DBpedia, YAGO, and BabelNet represent knowledge in separately structured monolingual graphs connected by inter-lingual links.
- Less than 20% of DBpedia entities are covered by inter-lingual links, motivating methods to discover missing links between equivalent entities.
- Existing approaches often use only topological information or combine topology with attributes, leaving broader multi-aspect modeling underused.
- The paper proposes MAN and HMAN, GCN-based models that incorporate topological features, relation types, and attributes into cross-lingual entity embeddings.
- Multilingual BERT is fine-tuned to capture semantic relatedness between literal descriptions written in different languages, and two strategies combine BERT with GCN modules.
- Experiments on two benchmark datasets report new state-of-the-art results for the proposed approach.
2 Problem Definition
The problem definition formalizes multilingual knowledge graphs as language-specific graphs containing entities, relations, attributes, values, and descriptions. Cross-lingual entity alignment uses pre-aligned links as training data to discover additional links between equivalent entities.
- A multilingual KG contains language-specific graphs Gi, each comprising entities, relations, attributes, attribute values, and literal descriptions.
- Each language-specific graph represents relation triples and attribute triples connecting entities with typed relations or attribute values.
- Given two graphs and a set of pre-aligned inter-lingual links, the task is to discover missing links between their cross-lingual entity counterparts.
3 Proposed Approach
The approach combines GCN-based multi-aspect graph embeddings with multilingual BERT representations of entity descriptions for cross-lingual alignment. It uses MAN and HMAN to model graph information, POINTWISEBERT and PAIRWISEBERT for textual information, and integration strategies to balance effectiveness with computational cost.
- Graph-based embeddings: GCNs collect evidence from multilingual KG structures under assumptions about neighboring entities, relation types, and shared attributes.The method assumes equivalent entities tend to have equivalent neighbors connected by the same relation types and similar attributes.
- Graph-based embeddings: The model represents topological connections, relation types, and attributes as three complementary entity-feature aspects.Topological features use identity vectors as initial inputs, while relations and attributes use count-based N-hot vectors restricted to the top-F frequent features.
- MAN: MAN applies separate GCNs to topological, relation, and attribute features, then concatenates the resulting representations for alignment decisions.The concatenated multi-aspect embedding is normalized before use in alignment.
- HMAN: HMAN separates topological propagation from relation and attribute processing, using feedforward networks for the latter to reduce noise from neighboring entities.Its relation and attribute networks include a fully connected layer and a highway network layer, producing a hybrid concatenated embedding.
- Textual embeddings: Multilingual BERT supplies cross-lingual textual embeddings through POINTWISEBERT and PAIRWISEBERT, which respectively score description pairs or encode descriptions separately.POINTWISEBERT ranks candidate pairs but is computationally expensive; PAIRWISEBERT directly measures distances between textual embeddings.
- Integration strategy: A reranking strategy uses MAN or HMAN to generate top-q candidate entities before POINTWISEBERT scores them, reducing computation while making performance depend on candidate quality.Processing every possible description pair is cost prohibitive, motivating a smaller candidate pool.
4 Experiments
Experiments on DBP15K and DBP100K evaluate graph-based and textual approaches for cross-lingual entity alignment. MAN and HMAN outperform prior systems, while ablations and case studies examine the roles of multi-aspect features, propagation, and BERT integration.
- Datasets and Settings: DBP15K and DBP100K contain 15,000 and 100,000 inter-lingual links across Chinese-English, Japanese-English, and French-English subsets.Experiments use the same 30% training and 70% evaluation split, with Hits@k reported in both alignment directions.
- Results on Graph Embeddings: MAN and HMAN consistently outperform all baselines across graph-alignment scenarios, with HMAN generally achieving the strongest performance.The models use topological, relational, and attribute information without the extra pre-aligned relation and attribute supervision required by one baseline.
- Results on Graph Embeddings: Propagation can introduce noise when neighboring entities have mismatched relations or attributes, motivating HMAN's direct feature processing.The case study attributes the performance gap between HMAN and MAN to heterogeneous neighbors whose propagated features lack cross-lingual counterparts.
- Results on Graph Embeddings: Relation and attribute ablations reduce performance across datasets, confirming that both feature types contribute to alignment decisions.HMAN exhibits larger drops than MAN, supporting feedforward networks as a better way to categorize these features in this setting.
- Results on Graph Embeddings: Topological information remains indispensable: removing it hurts HMAN more than MAN because MAN can implicitly retain structure through propagation.The comparison indicates that MAN w/o TE performs better than HMAN w/o TE, while HMAN loses structural knowledge when topological features are excluded.
- Results with Textual Embeddings: BERT-based textual embeddings improve graph-based alignment through reranking or weighted concatenation, although graph embeddings remain necessary when descriptions are unavailable.Reranking reduces the search space and gains depend on candidate-pool quality; weighted combinations produce significant improvements across datasets.
5 Related Work
Related work covers monolingual and multilingual entity alignment, embedding-based cross-lingual methods, and multilingual sentence representations. The paper distinguishes its approach by explicitly incorporating relation features and comparing two ways to model relation and attribute information.
- KG Alignment: Monolingual entity-alignment methods use string or graph similarity, while multilingual approaches learn cross-lingual embeddings or jointly model entity and description embeddings.The related work spans label matching, graph similarity, TransE-based embeddings, and co-training methods.
- KG Alignment: Unlike prior GCN-based work, this approach treats relation features as model inputs and investigates two ways to capture relation and attribute information.The distinction concerns how structured information beyond connectivity is represented and integrated.
- Multilingual Sentence Representations: Multilingual sentence-representation research includes zero-shot classification, multilingual translation encoders, and multilingual BERT representations.The paper adopts multilingual BERT to learn representations for entity descriptions in different languages.
6 Conclusion and Future Work
The paper presents GCN- and multilingual-BERT-based methods for cross-lingual entity alignment, combining topological, relational, attribute, and descriptive information. Its best model achieves state-of-the-art performance across datasets, while future work targets additional multilingual datasets and entities without descriptions.
- Conclusion: The task is to discover mappings between equivalent entities in multilingual knowledge graphs.The paper frames cross-lingual entity alignment as finding missing inter-lingual links.
- Conclusion: Two GCN-based models and two multilingual-BERT uses model multi-aspect entity information and integrate graph and textual modules.The information includes topological connections, relations, attributes, and entity descriptions.
- Conclusion: The best model consistently achieves state-of-the-art performance across all datasets.
- Future Work: Future work will evaluate other multilingual datasets and develop graph-based alternatives for entities lacking literal descriptions.The paper specifically names YAGO and BabelNet as additional datasets of interest.