Source-linked AI summary
Jointly Learning Entity and Relation Representations for Entity Alignment
Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, Dongyan Zhao
TL;DR
Entity alignment must bridge heterogeneous knowledge graphs, while existing methods often underuse relation representations or require pre-aligned relations. The paper jointly learns entity and relation representations with a GCN, approximating relations from entity embeddings and incorporating them into entities iteratively. Experiments on three real-world datasets show better alignment results than prior methods while reducing the need for seed alignments.
Problem
Existing entity-alignment methods often do not explicitly use relation representations and may require pre-aligned relations or triples, increasing seed-alignment overhead.
Method
A GCN-based framework approximates relation representations from learned entity embeddings, incorporates them into joint entity representations, and iteratively learns both.
Results
Experiments on three real-world datasets report better and more robust entity and relation alignment results than state-of-the-art methods.
Takeaways & Limitations
Joint entity and relation representations can support simultaneous entity and relation alignment without pre-aligned relation training data.
Takeaways & Limitations
Around 20% of inspected Google Translate translations for Japanese and Chinese entity names were wrong, posing further challenges for the model.
Abstract
from arXiv · showhide
Entity alignment is a viable means for integrating heterogeneous knowledge among different knowledge graphs (KGs). Recent developments in the field often take an embedding-based approach to model the structural information of KGs so that entity alignment can be easily performed in the embedding space. However, most existing works do not explicitly utilize useful relation representations to assist in entity alignment, which, as we will show in the paper, is a simple yet effective way for improving entity alignment. This paper presents a novel joint learning framework for entity alignment. At the core of our approach is a Graph Convolutional Network (GCN) based framework for learning both entity and relation representations. Rather than relying on pre-aligned relation seeds to learn relation representations, we first approximate them using entity embeddings learned by the GCN. We then incorporate the relation approximation into entities to iteratively learn better representations for both. Experiments performed on three real-world cross-lingual datasets show that our approach substantially outperforms state-of-the-art entity alignment methods.
1 Introduction
Heterogeneous knowledge graphs make entity alignment necessary, but existing approaches often underuse relation information or require costly pre-aligned relations. The paper proposes jointly learning entity and relation representations from a small set of aligned entities, improving both alignment tasks.
- Different expressions across knowledge graphs make integration difficult, motivating entity alignment between entities with the same real-world identity.
- Embedding-based methods reduce manual feature construction but often emphasize entity embeddings without explicitly using relation embeddings for entity alignment.
- Pre-aligned relations or triples limit scalability because constructing seed alignments for large knowledge graphs incurs overhead.
- The proposed framework jointly learns entity and relation representations using only a small set of pre-aligned entities, avoiding seed relation alignments.
- Relation information is incorporated into joint entity representations, which are iteratively improved to enhance both entity and relation alignment.
- Experiments on three real-world datasets report better and more robust entity and relation alignment results than state-of-the-art methods, while reducing seed-alignment effort.
2 Related Work
Related work includes manual, external-information, translation-based, and GCN-based alignment methods. The paper positions its approach as jointly aligning entities and relations using only aligned entity seeds while deriving relation representations through GCN-based processing.
- 2.1 Entity Alignment: Earlier entity-alignment methods relied on handcrafted features, external information, or embedding models to reduce human involvement.
- 2.1 Entity Alignment: Translation-based methods embed knowledge graphs and commonly use translation models as their backbone.
- 2.1 Entity Alignment: Existing approaches often focus on entity embeddings and overlook the assistance relation representations can provide for entity alignment.
- 2.1 Entity Alignment: Methods that align both entities and relations, such as MTransE and NTAM, require high-quality seed alignments including pre-aligned triples or relations.
- 2.2 Graph Convolutional Networks: GCNs operate on unlabeled graphs and derive node features from neighborhood structure, supporting representation learning for graph-based tasks.
- 2.2 Graph Convolutional Networks: The paper uses GCNs to encode entities and approximate relation representations from entity embeddings for joint entity and relation alignment.
3 Problem Formulation
The task is to find equivalent entities or relations across two knowledge graphs from known alignment seeds. In this framework, only aligned entity pairs train both alignment tasks, while relation alignment is unsupervised.
- A knowledge graph is represented as G = (E, R, T), comprising entities, relations, and triples.
- For two knowledge graphs, known equivalent entity pairs form alignment seeds L used to discover additional correspondences.
- Entity or relation alignment automatically finds further equivalent entities or relations based on known alignment seeds.
- The model uses known aligned entity pairs as training data for both entity and relation alignment, without pre-aligned relation pairs.
4 Our Approach
The approach uses GCNs to align entities, approximate relation representations without relation seeds, and iteratively fuse both representation types for joint alignment.
- The model has three stages: preliminary entity alignment, relation-representation approximation, and joint entity–relation alignment.
- Preliminary Entity Alignment: GCNs embed entities from both KGs in a unified vector space, using stacked layers to incorporate higher-degree neighborhoods.
- Preliminary Entity Alignment: Highway gates control noise propagation across GCN layers while transforming node representations with self-connected graph structure.
- Approximating Relation Representations: Relation representations are approximated from averaged head- and tail-entity embeddings, concatenated, and transformed by a shared learnable matrix.
- Approximating Relation Representations: Relation alignment compares relation-vector distances and increases similarity when the relations connect pre-aligned head or tail entities.
- Joint Entity and Relation Alignment: The method combines relation contexts with entity embeddings, then continues optimizing both through seed entity alignments and backpropagation.
5 Experimental Setup
The experiments use DBP15K cross-lingual datasets, compare entity and relation alignment against established baselines, and evaluate Hits@k under several model variants.
- Datasets: DBP15K comprises three cross-lingual datasets connecting English DBpedia with Chinese, Japanese, and French, each providing 15K pre-aligned entity pairs.The datasets also provide pre-aligned relations, with additional relations manually aligned for relation-alignment test sets.
- Implementation Details: The implementation uses entity-name initialization, translated non-English names, pre-trained English word vectors, and fixed hyperparameters including γ = 1, β = 20, and learning rate 0.001.The paper notes that machine translation errors affect roughly 20% of inspected Japanese and Chinese entity-name translations.
- Baselines: The study compares entity alignment with JE, MTransE, JAPE, IPTransE, BootEA, and GCN, with BootEA identified as the strongest DBP15K baseline.Relation alignment is compared with BootEA-R and MTransE-R, using an MTransE variant without pre-aligned relations.
- Model Variants: HGCN-PE and HGCN-PR are preliminary entity- and relation-alignment models, while HGCN-JE and HGCN-JR add joint representations.Additional variants isolate entity-name initialization, highway gates, and joint learning within GCN-based models.
- Metrics: Performance is measured with Hits@k, the proportion of correctly aligned entities ranked within the top k candidates, where higher scores are better.The same metric framework is used for the reported alignment evaluations.
6 Experiment Results
Across entity and relation alignment experiments, the complete joint models achieve the strongest overall performance and remain effective with limited seed alignments. Analyses attribute gains to combining relation information with entity and neighboring structural representations.
- Entity Alignment: HGCN-JE gives the best entity-alignment performance across all reported metrics and datasets.It improves over preliminary models, including 2.36% and 4.19% Hits@1 gains on DBP15KZH−EN for the HGCN and GCN comparisons, respectively.
- Entity Alignment: 34.31% improvement on Hits@1 on DBP15KFR−EN lets HGCN-PE outperform the strongest baseline BootEA.The result is associated with layer-wise highway gates controlling noisy information propagation.
- Relation Alignment: Relation approximation improves MTransE-PR over MTransE-R, whose Hits@1 performance is below 4% on every dataset.The approximation uses information from head and tail entity embeddings rather than relying only on directly learned relation embeddings.
- Relation Alignment: HGCN-JR delivers the best relation-alignment results across datasets and metrics except Hits@10 on DBP15KFR−EN.Joint learning also improves relation alignment over the corresponding preliminary HGCN-PR and GCN-PR models.
- Seed Alignment Sensitivity: 86.40% Hits@1 on DBP15KFR−EN is achieved by HGCN-JE with only 10% of training data, 17.84% higher than BootEA using 40% of seed alignments.HGCN-JE and HGCN-JR consistently outperform their BootEA counterparts as seed-alignment proportions vary.
- Case Study: Jointly learned representations progressively uncover additional aligned entities and relations after preliminary alignments, including (vFR, vEN), (v1, v4), and (r1, r4).Entities with more neighbors and high-frequency relations are easier to align because they provide richer structural information for the GCN.
7 Conclusions
The framework jointly models entities and relations with gated GCNs, using joint representations to iteratively improve both alignment tasks. Experiments on three real-world datasets report better and more robust performance than state-of-the-art methods.
- The framework jointly models entities and relations of heterogeneous knowledge graphs for simultaneous entity and relation alignment.It does not require pre-aligned relations as training data.
- Gated GCNs learn entity and relation representations, while joint entity representations combine relation information with entity information.The combined representations iteratively improve the learned representations for both entities and relations.
- Experiments on three real-world datasets show better and more robust performance than state-of-the-art methods.
A Training Procedure of Our Approach
Training proceeds through preliminary entity alignment, relation-representation approximation, and joint entity-relation alignment. GCN-based entity embeddings support relation approximation before relation information is incorporated into entity embeddings.
- The framework takes two target knowledge graphs and prior aligned entity pairs as inputs, with tuneable parameters including the number of epochs.
- Training has three stages: preliminary entity alignment, relation-representation approximation, and joint entity-relation alignment.
- GCNs first learn entity representations for preliminary alignment, which are then used to approximate relation representations across knowledge graphs.
- After preliminary alignment stabilizes, relation representations are incorporated into entity embeddings to jointly learn entity and relation alignment.
B Statistical Information of Alignment
The alignment statistics associate successful alignment with neighborhood information and relation frequency. The training algorithm alternates GCN representation learning, negative-sample regeneration, relation approximation, and margin-loss updates before and during joint training.
- Entities with more neighboring entities and relations are more likely to be aligned, and are aligned earlier.The analysis identifies neighbor information as important to entity alignment.
- Relations with abundant objects are embedded accurately, creating a frequency gap between relations aligned preliminarily and those not aligned preliminarily.
- The algorithm alternates GCN entity updates, periodic negative-sample regeneration, relation approximation, margin losses, and parameter updates across preliminary and joint modes.The procedure uses aligned entity pairs, epochs, an interval T for regenerating negative samples, and K negative samples.
- Table 4 reports alignment outcomes for entities and relations predicted in both stages, only the joint stage, or neither stage; its statistics include neighboring entities, neighboring relations, and relation frequency.
- Relations missed during preliminary alignment are more likely to align jointly when they occur frequently, because entity and relation embeddings improve together.