Source-linked AI summary
Relational Learning with Gated and Attentive Neighbor Aggregator for Few-Shot Knowledge Graph Completion
Guanglin Niu, Yang Li, Chengguang Tang, Ruiying Geng, Jian Dai, Qiao Liu, Hao Wang, Jian Sun, Fei Huang, Luo Si
TL;DR
Few-shot knowledge graph completion must infer facts for relations with limited training instances, including complex relations and cases with sparse or uninformative neighborhoods. The paper combines a gated and attentive neighbor aggregator with meta-learning over TransH, and reports improved results over MetaR and other baselines on NELL-One and Wiki-One.
Problem
Few-shot relations have limited training instances, while sparse neighborhoods and complex relation patterns make knowledge graph completion challenging.
Source: §1 · p. 2Method
The model combines a gated and attentive neighbor aggregator for relation representations with meta-learning over TransH to model complex relations.
Source: §1 · p. 2Results
Takeaways & Limitations
The results support using valuable neighborhood semantics and MTransH for few-shot relation representation and complex-relation inference on the evaluated datasets.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Aiming at expanding few-shot relations' coverage in knowledge graphs (KGs), few-shot knowledge graph completion (FKGC) has recently gained more research interests. Some existing models employ a few-shot relation's multi-hop neighbor information to enhance its semantic representation. However, noise neighbor information might be amplified when the neighborhood is excessively sparse and no neighbor is available to represent the few-shot relation. Moreover, modeling and inferring complex relations of one-to-many (1-N), many-to-one (N-1), and many-to-many (N-N) by previous knowledge graph completion approaches requires high model complexity and a large amount of training instances. Thus, inferring complex relations in the few-shot scenario is difficult for FKGC models due to limited training instances. In this paper, we propose a few-shot relational learning with global-local framework to address the above issues. At the global stage, a novel gated and attentive neighbor aggregator is built for accurately integrating the semantics of a few-shot relation's neighborhood, which helps filtering the noise neighbors even if a KG contains extremely sparse neighborhoods. For the local stage, a meta-learning based TransH (MTransH) method is designed to model complex relations and train our model in a few-shot learning fashion. Extensive experiments show that our model outperforms the state-of-the-art FKGC approaches on the frequently-used benchmark datasets NELL-One and Wiki-One. Compared with the strong baseline model MetaR, our model achieves 5-shot FKGC performance improvements of 8.0% on NELL-One and 2.8% on Wiki-One by the metric Hits@10.
1 INTRODUCTION
The introduction frames few-shot knowledge graph completion around sparse relation evidence and the difficulty of modeling complex relations with limited instances. It proposes a gated and attentive neighbor aggregator with MTransH, and reports improvements over MetaR.
1 INTRODUCTION
- Existing FKGC approaches use metric learning, adaptive attention, or meta-learning, but some focus on similarity or path finding, and MetaR neglects neighborhood semantics for complex relations.The proposed framework aims to use valuable neighbor information to represent few-shot relations and infer missing facts.Source: §1 · p. 2
- Few-shot relations have limited training instances, while sparse neighborhoods can add noise and complex 1-N, N-1, and N-N relations are difficult to model in the few-shot setting.The introduction notes that complex-relation models may require many training instances or higher complexity.Source: §1 · p. 2
- The proposed global-local model combines a gated and attentive neighbor aggregator with MTransH to represent few-shot relations and model complex relations.The gate uses head and tail entities alongside their neighborhoods to filter noisy neighbor information; MTransH applies meta-learning to TransH.Source: §1 · p. 2
- The paper identifies the aggregator as a way to capture valuable context from head and tail entities and their neighborhoods, and incorporates MAML with TransH for 1-N, N-1, and N-N relations.The authors describe these as first contributions to their knowledge.Source: §1 · p. 2
- 8.0% improvement in 5-shot Hits@10 on NELL-One over MetaR; the proposed model also outperforms other KGC approaches, including FKGC models.The introduction reports that ablations demonstrate the effectiveness of each key module.Source: §1 · p. 3
2 RELATED WORK
Traditional KGC methods learn entity and relation representations but typically require many training instances. Few-shot KGC work spans several modeling approaches, while the paper identifies underused neighborhood information as a limitation.
2.1 Traditional Knowledge Graph Completion
- Traditional KGE methods include translation, relation-specific projection, complex-space, and convolutional approaches, but require many training instances.The section describes TransH as projecting entity embeddings into relation-specific hyperplanes.Source: §2.1 · p. 3
2.2 Few-Shot Knowledge Graph Completion
- Few-shot KGC includes metric-learning, meta-learning, and dual-process approaches, but existing methods do not fully use neighbors to learn few-shot relation representations.GMatching and FSRL use metric learning; MetaR and Meta-KGR use meta-learning; CogKR combines summary and reasoning modules.Source: §2.2 · p. 3
3 PROBLEM FORMULATION
The section formalizes a knowledge graph and its background graph, defines a few-shot relation’s neighborhood, and describes the training and test tasks for few-shot completion.
3 PROBLEM FORMULATION
- A knowledge graph is defined by entity and relation sets and triples; its background graph contains triples associated with high-frequency relations.The relation set includes both high-frequency and few-shot relations.Source: §3 · p. 3
- A few-shot relation’s neighborhood comprises its head and tail entities and their one-hop neighbors drawn from the background graph.Incoming relations to either entity are reversed to outgoing relations in the neighborhood.Source: §3 · p. 3
- At test time, the task is to infer new triples for a relation unseen during training from a few support triples; K denotes the K-shot setting.Training tasks predict new triples for a specific few-shot relation using support and query sets.Source: §3 · p. 3
4 METHODOLOGY
The method first builds a general few-shot relation representation from support-set neighborhoods, then adapts the relation representation and TransH hyper-plane parameters for query-set inference. Its global stage gates and attends to neighborhood information, while its local MTransH stage uses meta-learning to address complex relations.
4.1.1 Gated and Attentive Neighbor Aggregator.
- The gated neighbor aggregator combines head and tail entities with their neighborhoods, using a gate to control neighborhood information when neighborhoods are sparse.Attention weights distinguish neighbors, while the gate determines how much neighborhood representation contributes.Source: §4.1.1 · p. 4
4.1.2 Generating the Few-shot Relation Representation.
- An attentive Bi-LSTM integrates the support set’s neighborhood representations into a general few-shot relation representation.It assigns weights to the final hidden states before combining them across the K support triples.Source: §4.1.2 · pp. 4–5
4 METHODOLOGY
- The two-stage framework learns relation representations from support-set triples and optimizes the whole model with query-set triples.The global stage initializes representations from background knowledge, and MTransH further adapts them to the support set before transfer to queries.Source: §4 · pp. 3–4, Figure 2 · p. 5
4.2 Local Stage: MTransH
- The local stage uses TransH to account for one-to-many, many-to-one, and many-to-many relations in few-shot inference.The method scores triples using projected head and tail entity embeddings and the relation hyper-plane.Source: §4.2 · p. 5
- MTransH adapts the relation representation and relation-specific hyper-plane parameter on the support set, then transfers the updated parameters to query triples.The hyper-plane parameter is adapted with gradient descent following MAML; query-set scores and loss are then calculated.Source: §4.2 · pp. 5–6
5 EXPERIMENTS
Experiments evaluate the model on NELL-One and Wiki-One against traditional and few-shot baselines, then examine complex-relation performance, component ablations, and neighbor weights. The model outperforms the baselines, with results also showing how performance varies by relation category and neighborhood sparsity.
5.1 Datasets and Evaluation Metrics
- NELL-One and Wiki-One select relations with more than 50 but fewer than 500 triples, using 51/5/11 and 133/16/34 training/validation/testing task relations, respectively.The datasets also report MRR and Hits@n, with n set to 1, 5, or 10; few-shot sizes are 1, 3, and 5.Source: §5.1 · p. 6
5.2 Baselines
- Traditional baselines are TransE, TransH, DisMult, and ComplEx; few-shot baselines are GMatching, MetaR, CogKR, and FSRL.Traditional models are adapted by training on background, training, and support-set triples, then tested on query-set triples.Source: §5.2 · p. 6
5.3 Implementation Details
- Traditional KGC baselines use open-source GitHub code, with the paper linking the OpenKE and GMatching repositories.Hyperparameters are selected using validation data; the neighborhood aggregator's maximum size is 50 on both datasets.Source: §5.2 · p. 6, §5.3 · p. 6
5.4 Experimental Results and Analysis
- Compared with MetaR, our model improves Hits@10 by 8.2% on NELL-One and 8.0% on Wiki-One.The gains are 5.1%, 7.3%, and 4.6% on NELL-One and 8.3%, 8.7%, and 8.2% on Wiki-One for MRR, Hits@5, and Hits@1, respectively.Source: §5.4 · p. 7
5.7 Visualization
- In the TeamCoach visualization, team- and coach-related neighbors receive the largest weights, while proxy/location and inverse membership neighbors receive the smallest.The figure selects three highest-weight and two lowest-weight neighbors from each neighborhood.Source: Figure 3 · p. 8, §5.7 · pp. 8–9
5.5 Results on NELL-One of Complex Relations
- The full model outperforms -MTransH, MetaR, and other baselines, including TransH, across 1-N, N-1, and N-N relations.MetaR performs worse than most traditional embedding baselines on 1-N relations; the authors attribute this to its TransE-like score function.Source: §5.5 · p. 8, Table 3 · p. 9
5.6 Ablation Study
- Ablations show substantial drops without GANA, drops on both datasets without MTransH, and a larger gating effect on sparse Wiki-One than NELL-One.The authors explain that gating filters noise when neighborhoods are too sparse; NELL-One has richer neighborhoods.Source: §5.6 · p. 8, Table 4 · p. 9
6 CONCLUSION AND FUTURE WORK
The approach combines a gated and attentive neighbor aggregator with MTransH to learn few-shot relation representations, including in sparse neighborhoods and for complex relation types. Experiments report better performance than state-of-the-art baselines on two few-shot datasets.
6 CONCLUSION AND FUTURE WORK
- The model outperforms other state-of-the-art baselines on the two few-shot datasets.The conclusion also says ablation and complex-relation results illustrate the effects of the aggregator and MTransH.Source: §6 · p. 9
- The gated and attentive neighbor aggregator captures valuable neighbors to learn general few-shot relation representations even with very sparse neighborhoods.Source: §6 · p. 9
- MTransH updates few-shot relation representations to support modeling and inference of 1-N, N-1, and N-N relations.Source: §6 · p. 9
- Future work will consider external knowledge, including text descriptions of relations and entities, to enhance their representations.It will also consider few-shot entities and relations in FKGC.Source: §6 · p. 9