Source-linked AI summary
A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphs
Zequn Sun, Qingheng Zhang, Wei Hu, Chengming Wang, Muhao Chen, Farahnaz Akrami, Chengkai Li
TL;DR
Entity alignment must overcome symbolic, linguistic, and schematic heterogeneity between independently created knowledge graphs. This paper benchmarks embedding-based approaches using dedicated realistic datasets and an open-source library, finding competitive performance from RDGCN, BootEA, and MultiKE and improvements from CSLS and stable matching.
Problem
Entity alignment seeks to identify entities referring to the same real-world object despite symbolic, linguistic, and schematic heterogeneity between independently created knowledge graphs.
Method
The paper surveys embedding-based entity alignment, proposes degree-aware sampling for realistic benchmark datasets, and develops an open-source library of representative approaches and models.
Results
RDGCN, BootEA, and MultiKE achieve the most competitive performance, while CSLS and stable matching improve performance across approaches.
Takeaways & Limitations
The study provides a benchmark and analysis showing that literal information, bootstrapping, and alignment inference strategies are important considerations for entity alignment.
Takeaways & Limitations
Current approaches require seed alignment supervision, and experiments reveal difficulty aligning long-tail entities, which comprise a large proportion of knowledge graphs.
Abstract
from arXiv · showhide
Entity alignment seeks to find entities in different knowledge graphs (KGs) that refer to the same real-world object. Recent advancement in KG embedding impels the advent of embedding-based entity alignment, which encodes entities in a continuous embedding space and measures entity similarities based on the learned embeddings. In this paper, we conduct a comprehensive experimental study of this emerging field. We survey 23 recent embedding-based entity alignment approaches and categorize them based on their techniques and characteristics. We also propose a new KG sampling algorithm, with which we generate a set of dedicated benchmark datasets with various heterogeneity and distributions for a realistic evaluation. We develop an open-source library including 12 representative embedding-based entity alignment approaches, and extensively evaluate these approaches, to understand their strengths and limitations. Additionally, for several directions that have not been explored in current approaches, we perform exploratory experiments and report our preliminary findings for future studies. The benchmark datasets, open-source library and experimental results are all accessible online and will be duly maintained.
1. INTRODUCTION
Entity alignment matches entities that denote the same real-world object across knowledge graphs, but heterogeneous symbols, languages, schemata, and dataset biases complicate fair evaluation. This paper addresses these gaps through a survey, realistic benchmarks, an open-source library, comprehensive comparisons, and exploratory analyses.
- Knowledge graphs represent facts as entity–relation–entity or entity–attribute–literal triples and support applications including semantic search, question answering, and recommender systems.
- Entity alignment identifies entities referring to the same real-world object across independently created knowledge graphs despite symbolic, linguistic, and schematic heterogeneity.
- Embedding-based approaches encode knowledge-graph symbols as low-dimensional vectors and use geometric structure to capture entity relatedness and align entities.
- The field lacks a prior status-quo survey, widely acknowledged realistic benchmarks, and clear comparisons with conventional entity-alignment approaches.
- The paper surveys 23 approaches, proposes sampling-based benchmark datasets spanning degrees, multilingualism, schemata, and scales, and develops OpenEA with 12 representative approaches.
- It compares representative methods for effectiveness and efficiency, conducts exploratory experiments, and outlines directions including unsupervised, long-tail, large-scale, and non-Euclidean alignment.
- The experiments identify strengths and shortcomings of current methods, while the datasets, library, and results are publicly available to support reproducible research.
2. PRELIMINARIES
The entity-alignment task seeks a one-to-one correspondence between entities in two knowledge graphs, using a known subset of matches as training supervision.
- Entity alignment finds the one-to-one set of equivalent entity pairs between two knowledge graphs, KG1 and KG2.
- A known subset of entity correspondences, called seed alignment, is used as training data.
2.1 Literature Review
Prior work spans knowledge-graph embedding, conventional symbolic alignment, and related network-alignment methods, but their datasets and evaluation practices differ. The reviewed embedding-based approaches use translational models, graph convolutional networks, and attribute or value embeddings.
- Embedding models: Knowledge-graph embedding models include translational, semantic-matching, and deep categories, originally developed mainly for link prediction.
- Datasets and metrics: Link-prediction benchmarks include FB15K-237 and WN18RR, while Hits@1 is emphasized among widely used evaluation metrics because it equals precision.
- Conventional approaches: Conventional entity alignment uses OWL equivalence reasoning, symbolic similarity, statistical machine learning, crowdsourcing, and record-linkage techniques.
- Embedding-based alignment: Embedding-based alignment approaches commonly employ translational models such as TransE, graph convolutional networks, and attribute–value embeddings.
- Datasets and metrics: Previous alignment datasets DBP15K and WK3L differ substantially in degree distributions from DBpedia, whereas EN-FR-15K retains a similar distribution to DBpedia.
2.2 Categorization of Techniques
The paper categorizes embedding-based entity alignment approaches by embedding techniques, interaction modes, learning strategies, and alignment inference. It distinguishes relation, attribute, path, neighborhood, and literal embedding methods, alongside supervised and semi-supervised designs and a degree-based sampling procedure.
- 23 recent approaches are categorized by their embedding and alignment modules and how those modules interact.
- Embedding techniques: Relation embedding captures KG structure from relation triples, while attribute embedding exploits entities’ attribute triples to enhance alignment.Relation embedding is used by all existing approaches surveyed; attribute embedding is used by several approaches.
- Embedding techniques: Triple-based models score relation-triple plausibility, path-based models encode multi-hop relation dependencies, and neighborhood-based models use subgraph structure through GCNs.TransE uses an energy function for triples; IPTransE composes relation paths; GCNs propagate information over adjacency structure.
- Embedding techniques: Attribute methods model either attribute correlations or literal values, with character-level literal embeddings able to represent unseen values but potentially failing cross-lingually.Attribute correlation embedding ignores literal values, whereas literal embedding treats literals as entities for learning from attribute triples.
- Alignment inference: The alignment module compares entity embeddings with distance metrics and can infer alignments using maximum-weight matching or stable marriage algorithms.The Kuhn–Munkres solution takes O(N^3), while stable marriage takes O(N^2) under the stated formulation.
- Combination modes: Four interaction designs reconcile KG embeddings through space transformation, space calibration, parameter sharing, or parameter swapping.Transformation learns a matrix between separate spaces; calibration places aligned entities near one another; sharing and swapping provide special alignment mechanisms.
- Learning strategies: Supervised approaches use seed alignments as labeled data, whereas semi-supervised approaches additionally exploit unlabeled data through self-training or co-training.Seed alignment acquisition is described as costly and error-prone, especially for cross-lingual KGs; self-training augments seed alignment iteratively.
- KG sampling: Iterative degree-based sampling filters KGs by reference alignment, matches degree distributions, deletes entities using PageRank probabilities, and returns sampled KGs with updated alignment.The procedure repeats sampling while both datasets exceed the target entity size and filters the resulting datasets by the reference alignment.
3. DATASET GENERATION
The paper generates realistic entity-alignment benchmarks by sampling paired real-world KGs while preserving degree distributions and varying dataset characteristics. It evaluates IDS against alternative sampling strategies and reports stronger overall dataset properties for IDS.
- Iterative Degree-based Sampling: IDS simultaneously deletes aligned entities from two KGs until the target size while preserving each sample’s degree distribution relative to its source.PageRank weights reduce the probability of deleting high-degree entities, which strongly influence overall degree distributions.
- Iterative Degree-based Sampling: The sampling objective uses Jensen-Shannon divergence, with an expected degree-distribution difference of ϵ ≤5%.A smaller JS divergence indicates more similar degree distributions; PageRank calculation is the main computational cost.
- Dataset construction: 15K and 100K datasets are sampled from DBpedia, Wikidata, and YAGO across entity degrees, languages, schemata, scales, and density.The datasets use five-fold splits and reference alignments from inter-language links and owl:sameAs.
- Dataset construction: V2 datasets first remove low-degree entities to double average degree, then apply IDS, making V2 twice denser than V1 and closer to existing datasets.The paper compares 15K and 100K versions and reports their degree distributions and average degrees against DBpedia.
- Dataset evaluation: IDS shows good comprehensive performance, whereas RAS loses connectivity and degree-distribution fidelity and PRS remains unsatisfactory on several dataset properties.RAS produces sparse datasets with isolated entities, while PRS focuses on high-degree entities but samples only one KG at a time.
4. OPEN-SOURCE LIBRARY
OpenEA is an open-source library that separates embedding and alignment modules within an extensible pipeline. Its architecture supports reusable components and flexible combinations of techniques.
- Architecture: OpenEA independently implements embedding and alignment modules connected through predefined input and output data structures.This loose coupling lets users call and combine techniques from both modules to develop new approaches.
- Architecture: Figure 4 presents the software architecture of OpenEA.The figure corresponds to the library’s modular design.
- Functionality and extensibility: The library provides initialization, loss, negative-sampling, interaction, distance, and alignment-inference functions as reusable components.These components cover core operations across embedding, interaction, and alignment modules.
5. EXPERIMENTS AND RESULTS
The experiments evaluate 12 representative embedding-based entity alignment approaches with five-fold cross-validation across benchmark datasets of differing density, scale, and heterogeneity. Results identify strong overall methods and show how relations, attributes, entity degree, and training strategies affect effectiveness and efficiency.
- Main Results and Analysis: RDGCN, BootEA and MultiKE achieve the top-3 results across the benchmark datasets.Table 5 reports Hits@1, Hits@5 and MRR results for the implemented approaches.
- Sparse versus Dense Datasets: Most approaches perform better on dense than sparse datasets because additional relation triples provide more semantic information, while MultiKE is relatively insensitive to relation changes.MTransE and JAPE can instead decline on dense datasets because TransE handles multi-mapping relations poorly; multi-mapping relations rise from 39.0% to 71.2% between the cited EN-FR-100K variants.
- Alignment Degree: All relation-based approaches perform better on high-degree alignments than on long-tail entities, while literal-based methods alleviate this imbalance; no method handles long-tail entities well.Long-tail entities have few relation triples, limiting the expressiveness of their embeddings.
- Training Strategies: Negative sampling raises MTransE’s Hits@1 on EN-FR-15K (V1) to 0.271, while BootEA’s bootstrapping strategy also contributes substantially to performance.The comparison attributes the gain to negative sampling alongside marginal ranking loss.
- Attributes and Semi-supervised Learning: Literal embedding generally improves alignment more than attribute-correlation methods, whereas attribute heterogeneity limits correlation quality and errors in IMUSE’s augmented alignments can harm performance.JAPE and GCNAlign show no notable improvement from attribute correlations on D-Y, and most approaches do not improve from attribute embedding on D-W.
- Efficiency: MultiKE balances effectiveness and efficiency because its multi-view discriminative features support fast convergence, although auxiliary information or performance-boosting techniques usually increase training time.The broader comparison connects stronger auxiliary signals with additional computational cost.
6. EXPLORATORY EXPERIMENTS
Exploratory experiments examine embedding geometry, hubness, alternative similarity and matching strategies, unexplored KG embedding models, and comparisons with conventional alignment systems. They find that alignment inference choices and complementary systems can materially affect results, while current embedding-based methods remain limited on several settings.
- 6.1.1 Similarity Distribution: High top-1 similarity and large similarity variance characterize more discriminative embedding spaces for entity alignment.BootEA, MultiKE and RDGCN combine high top-1 similarity with strong alignment performance, whereas small variance indicates non-discriminative nearest neighbors.
- 6.1.2 Hubness and Isolation: Hubness and isolated entities distort nearest-neighbor alignment, motivating CSLS and stable matching as alternative inference strategies.CSLS adjusts similarities using local neighbor density, while stable matching considers all entities globally rather than using only greedy nearest neighbors.
- 6.1.2 Hubness and Isolation: Stable matching improves Hits@1 by more than 10% over greedy inference for MTransE, JAPE, KDCoE, GCNAlign, AttrE, IMUSE, SEA and RotatE.CSLS significantly benefits greedy inference, but does not further improve stable matching, suggesting stable matching relies less on the distance metric.
- 6.2 Unexplored KG Embedding Models: TransH and TransD show stable, promising performance, while TransR performs poorly because relation-specific transformations require relation alignment.On 100K datasets, TransH is more robust than MTransE because it handles multi-mapping relations better and uses negative sampling.
- 6.3 Comparison to Conventional Approaches: OpenEA shows no superiority over conventional approaches overall: PARIS performs best on most EN-FR, EN-DE and D-W datasets, while LogMap performs promisingly on D-Y.The comparison identifies alignment inference as an area requiring further improvement in embedding-based approaches.
- 6.3 Comparison to Conventional Approaches: OpenEA, LogMap and PARIS find complementary correct alignments on EN-FR-100K (V1), yet 6.41% of alignments are found by none and 45.56% by all three.All three approaches face symbolic heterogeneity, but their differing outputs indicate potential complementarity.
7. SUMMARYANDFUTUREDIRECTIONS
The experiments identify competitive approaches, useful alignment strategies, and unresolved practical constraints. They also point to supervision, long-tail entities, and scalability as important directions for future work.
- Experimental findings: RDGCN, BootEA and MultiKE achieve the most competitive performance, with literal information and carefully designed bootstrapping suggested as helpful.
- Experimental findings: CSLS distance and stable matching improve performance across all evaluated approaches, while alignment inference has received little attention.
- Experimental findings: Embedding-based and conventional entity alignment approaches are complementary, and Table 9 summarizes their required information for real-world approach selection.
- Future directions: Current approaches require seed alignment as supervision, motivating unsupervised methods that use auxiliary resources or distant supervision.
- Future directions: Sparse and dense datasets reveal difficulty aligning long-tail entities, while larger datasets substantially increase training and testing costs.
8. CONCLUSION
The paper surveys embedding-based entity alignment and benchmarks representative approaches using dedicated datasets and an open-source library. Its experiments analyze the field's current status and identify future directions.
- The study surveys embedding-based entity alignment, creates dedicated datasets for real-world knowledge graphs, and develops an open-source library of approaches and embedding models.
- Experiments analyze the status quo of representative entity alignment approaches and point out future research directions.
- The work acknowledges support from Chinese national research programs and a collaborative innovation center.