Source-linked AI summary
Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation
Zequn Sun, Chengming Wang, Wei Hu, Muhao Chen, Jian Dai, Wei Zhang, Yuzhong Qu
TL;DR
Entity alignment must handle counterpart entities whose neighborhoods differ across heterogeneous knowledge graphs, limiting conventional GNN-based representations. AliNet expands neighborhoods with selected distant neighbors, gates direct and distant aggregation, and adds relation loss; experiments on five datasets demonstrate its effectiveness.
Problem
Heterogeneous schemas and incomplete data make counterpart entities’ neighborhood structures dissimilar, causing GNNs to learn different representations for entities that should align.
Method
AliNet aggregates direct and distant neighborhoods with attention and gating, combines contrastive alignment and relation losses, and predicts alignments by cross-knowledge-graph nearest-neighbor search.
Results
AliNet outperforms state-of-the-art structure-based embedding models on Hits@1 and MRR, including gains of 0.036 over RSN and 0.057 over MuGNN on DBPFR-EN Hits@1.
Takeaways & Limitations
Experiments on five datasets support AliNet’s effectiveness for mitigating neighborhood non-isomorphism in entity alignment.
Abstract
from arXiv · showhide
Graph neural networks (GNNs) have emerged as a powerful paradigm for embedding-based entity alignment due to their capability of identifying isomorphic subgraphs. However, in real knowledge graphs (KGs), the counterpart entities usually have non-isomorphic neighborhood structures, which easily causes GNNs to yield different representations for them. To tackle this problem, we propose a new KG alignment network, namely AliNet, aiming at mitigating the non-isomorphism of neighborhood structures in an end-to-end manner. As the direct neighbors of counterpart entities are usually dissimilar due to the schema heterogeneity, AliNet introduces distant neighbors to expand the overlap between their neighborhood structures. It employs an attention mechanism to highlight helpful distant neighbors and reduce noises. Then, it controls the aggregation of both direct and distant neighborhood information using a gating mechanism. We further propose a relation loss to refine entity representations. We perform thorough experiments with detailed ablation studies and analyses on five entity alignment datasets, demonstrating the effectiveness of AliNet.
1 Introduction
Embedding-based entity alignment represents knowledge-graph entities as vectors, but heterogeneous schemas and incomplete data often produce dissimilar neighborhoods for counterpart entities. AliNet addresses this non-isomorphism by aggregating selected distant neighborhoods alongside direct neighborhoods.
- Motivation: Embedding-based entity alignment matches entities from different knowledge graphs by comparing distances between their learned vector representations.Representation learning can address symbolic heterogeneity in monolingual and cross-lingual alignment scenarios.
- Motivation: GNNs recursively aggregate neighboring-node representations and can identify isomorphic subgraphs, supporting their use for entity alignment.Their expressiveness provides a theoretical basis when similar entities have similar neighborhoods.
- Results: AliNet achieves promising results in entity alignment, according to the paper’s introduction.The supplied passage reports this outcome without specifying a metric or value.
- Challenge: 89.97%, 86.19%, and 90.71% of aligned pairs have different neighboring entities in Chinese-English, Japanese-English, and French-English DBpedia benchmarks, respectively.These statistics illustrate the prevalence of neighborhood mismatch across language pairs.
- Proposed approach: AliNet aggregates direct and distant neighborhood information, uses attention to select helpful distant neighbors, and applies gating to combine aggregation outputs.The approach is designed to mitigate neighborhood non-isomorphism end to end.
2 Preliminaries
The preliminaries define GNN aggregation, knowledge-graph and entity-alignment settings, and limitations of structure-only or relation-specific approaches. They motivate selectively incorporating distant neighbors because schema heterogeneity mixes direct and distant counterparts.
- GNNs: GNNs learn node representations by recursively aggregating neighboring feature vectors, with aggregation choices defining different GNN variants.The vanilla GCN uses mean pooling over an entity and its one-hop neighbors from the previous layer.
- Definitions: A knowledge graph is represented as G = (E, R, T), with entities E, relations R, and triples T = E × R × E.Entity alignment considers two knowledge graphs and partially pre-aligned entity pairs, then seeks remaining alignments through embeddings.
- Structural limitations: GNNs can propagate alignment information across isomorphic graphs with partially pre-aligned neighborhoods, but non-isomorphic structures give counterpart entities different representations.The latter mismatch follows from schema heterogeneity in cross-knowledge-graph alignment.
- Relation-aware limitations: R-GCN distinguishes neighbors with relation-specific weight matrices, but without pre-aligned relations its aggregation cannot effectively propagate entity-alignment information.This limitation arises when corresponding relations across the two knowledge graphs are not aligned.
- Compensation with distant neighborhood and relations: Schema heterogeneity mixes direct and distant neighbors of counterpart entities, motivating attentive and selective distant-neighborhood aggregation.The toy example shows aligned entities appearing at different hop distances, while some distant neighbors remain unhelpful.
3 Knowledge Graph Alignment Network
AliNet controls entity-representation learning by aggregating neighborhood information within multiple hops through a gating mechanism.
- Architecture: AliNet uses a gating mechanism to control aggregation of neighborhood information within k hops.The paper illustrates k = 2 using one-hop and two-hop neighborhoods, while noting that more hops are possible.
3.1 Gated Multi-hop Neighborhood Aggregation
AliNet separately aggregates direct and distant neighborhoods, using vanilla GCN layers for one-hop information and attention for two-hop information before gating their combination.
- One-hop aggregation: One-hop neighbor representations are aggregated with vanilla GCN layers because direct neighbors are central to characterizing an entity.The one-hop aggregation is computed at each AliNet layer.
- Distant-neighbor aggregation: Two-hop neighbors are aggregated with attention because ordinary layer-by-layer GCN propagation is inefficient and may transmit noise.The two-hop neighborhood is explicitly represented as N2(·).
- Distant-neighbor aggregation: The attention weight for an entity and its distant neighbor is learnable, with its computation specified in the attention mechanism.A separate weight matrix is used in the two-hop aggregation.
- Gated combination: A gating mechanism directly combines the one-hop and two-hop aggregation outputs into the hidden representation at each layer.The gate controls the contribution of both neighborhood ranges using a weight matrix and bias vector.
3.2 Attention for Distant Neighborhood
AliNet uses attention to select useful entities from potentially large two-hop neighborhoods, making distant-neighbor information more informative and comparable across entities.
- Attention for Distant Neighborhood: Two-hop neighborhoods can grow exponentially relative to one-hop neighborhoods, so AliNet uses attention to highlight useful distant neighbors.The attention mechanism is introduced to reduce noise from distant neighbors that do not characterize the central entity.
- Attention for Distant Neighborhood: AliNet’s architecture combines gated two-hop neighborhood aggregation with the broader KG alignment network.
- Attention for Distant Neighborhood: AliNet computes attention between a central entity and each candidate neighbor using separate linear transformations.
- Attention for Distant Neighborhood: The attention weights are normalized with softmax so weights can be compared across different entities.
3.3 Contrastive Alignment Loss
AliNet trains entity representations with a contrastive alignment loss that pulls aligned entities together and pushes negative samples beyond a margin, while incorporating representations from all layers.
- Contrastive Alignment Loss: The contrastive alignment loss minimizes distances between aligned entities and maximizes distances between unaligned entities.
- Contrastive Alignment Loss: Negative samples are formed by randomly substituting one entity in a pre-aligned pair, with their distance expected to exceed margin λ.
- Contrastive Alignment Loss: Rather than using only the final hidden layer, AliNet uses hidden representations from all layers because each layer propagates alignment information.
- Contrastive Alignment Loss: The combined representation concatenates layer outputs and applies L2 normalization to limit trivial increases in vector norm.
3.4 Relation Semantics Modeling
AliNet incorporates relational semantics by retrieving relation representations from related entity embeddings and optimizing a relation loss alongside graph-based entity modeling.
- Relation Semantics Modeling: Relation semantics are incorporated because knowledge graphs provide semantic relations between entities.
- Relation Semantics Modeling: AliNet adopts a TransE-style translational assumption without introducing additional relation-specific embeddings.
- Relation Semantics Modeling: A relation representation is retrieved from the embeddings of its subject-object entity pairs.
- Relation Semantics Modeling: The relation loss refines entity representations using the total relation set across the two knowledge graphs.
3.5 Implementation
AliNet combines contrastive alignment and relation losses, supports recursive k-hop aggregation, augments neighborhoods with balanced edges, and predicts counterparts by cross-graph nearest-neighbor search.
- Implementation: The final objective combines contrastive alignment loss and relation loss to inject relation semantics into preserved graph structures.
- Implementation: The objective is optimized with Adam, while Xavier initialization is used for all learnable parameters, including entity input features.
- Implementation: Sparse adjacency matrices from relational triples enable neighborhood aggregation through sparse matrix multiplication with storage complexity linear in entities and triples.
- Implementation: For k-hop aggregation, AliNet recursively combines neighborhood information using k −1 gating functions.
- Implementation: The method adds balanced edges between counterpart endpoints when an edge exists in one graph but not the other, reducing neighborhood non-isomorphism.
- Implementation: After training, entity alignment is predicted by nearest-neighbor search across knowledge graphs using combined representations and a distance measure such as Euclidean distance.
4 Experiments
AliNet is evaluated on five entity-alignment datasets against embedding-based and GNN baselines, with ablations examining relation loss, neighborhood augmentation, aggregation strategies, layers, and neighborhood overlap.
- Experimental Setup: AliNet is tested on DBP15K and DWY100K using Hits@1, Hits@10, and MRR, where higher scores indicate better alignment performance.DBP15K contains three multilingual DBpedia datasets; DWY100K contains DBpedia–Wikidata and DBpedia–YAGO3 datasets.
- Main Results: AliNet outperforms structure-based embedding models, including gains of 0.036 Hits@1 over RSN and 0.057 over MuGNN on DBPFR-EN.It also achieves the best Hits@10 results on DWY100K.
- Ablation Results: 0.012–0.037 Hits@1 is gained from neighborhood augmentation, while relation loss further improves performance over AliNet without relation loss.These ablations support the roles of neighborhood augmentation and relation loss in the model.
- Comparison with GNN Variants: AliNet outperforms re-tuned GCN, GAT, and R-GCN variants, supporting the integration of multi-hop information for entity alignment.GCN outperforms GAT and R-GCN in the reported comparison; the direct-neighbor aggregation component therefore uses GCN layers.
- Layer and Hop Analysis: Two layers and two-hop neighborhood information achieve the best reported settings, whereas additional layers introduce noise and more non-isomorphic neighborhood structures.For DBPZH-EN, two-hop aggregation reduces unaligned one-hop neighbors from 2.1 to 0.5 for Chinese entities and from 4.1 to 0.9 for English entities.
- Neighborhood-Overlap Analysis: AliNet aligns entities with smaller one-hop-neighbor overlap than GCN, as average overlap coefficients decrease during the first 200 training epochs.The comparison indicates that AliNet can identify correctly aligned pairs whose one-hop neighborhoods overlap less.
5 Related Work
Related work covers GNN-based and embedding-based approaches to entity alignment, alongside analyses of layer-specific AliNet representations on DBP15K.
- Embedding-based methods commonly use TransE to learn entity embeddings and align pre-aligned entities through mappings or distance minimization.
- Related approaches also compare graph-level similarity or capture hierarchical entity structures, whereas this work focuses on node-level alignment.
6 Conclusion and Future Work
AliNet mitigates neighborhood non-isomorphism through multi-hop gated aggregation, attention-based noise reduction, and a relation loss, with experiments demonstrating effectiveness on five datasets.
- AliNet aims to mitigate non-isomorphic neighborhood structures between counterpart entities in an end-to-end manner.
- The model aggregates multi-hop neighborhood information using layer-wise gating and attention to reduce noise.
- A relation loss is introduced to enhance AliNet’s representational expressiveness.
- Experiments on five datasets demonstrate AliNet’s effectiveness for entity alignment.
- Future work will incorporate entity side information from other modalities into preserved graph structures.