Source-linked AI summary
Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs
Mingyang Chen, Wen Zhang, Wei Zhang, Qiang Chen, Huajun Chen
TL;DR
Few-shot link prediction must infer new triples about relations from only a few observed triples, a setting where embedding methods perform poorly. MetaR transfers relation-specific meta information through relation meta and gradient meta, achieving state-of-the-art results on few-shot link prediction benchmarks. Its performance is constrained when entities appear in only one training triple.
Problem
Embedding-based methods perform poorly for relations with few associative triples, motivating prediction of new triples from only a few observed relation triples.
Method
MetaR transfers relation-specific meta information from support triples to query triples using relation meta and gradient meta.
Results
MetaR achieves state-of-the-art results on few-shot link prediction tasks, and both relation meta and gradient meta contribute significantly.
Takeaways & Limitations
MetaR provides a few-shot link prediction framework that transfers common relation information and accelerates learning within each task.
Takeaways & Limitations
Entities appearing in only one training triple can have biased embeddings, limiting MetaR because entity embeddings rely heavily on related triples.
Abstract
from arXiv · showhide
Link prediction is an important way to complete knowledge graphs (KGs), while embedding-based methods, effective for link prediction in KGs, perform poorly on relations that only have a few associative triples. In this work, we propose a Meta Relational Learning (MetaR) framework to do the common but challenging few-shot link prediction in KGs, namely predicting new triples about a relation by only observing a few associative triples. We solve few-shot link prediction by focusing on transferring relation-specific meta information to make model learn the most important knowledge and learn faster, corresponding to relation meta and gradient meta respectively in MetaR. Empirically, our model achieves state-of-the-art results on few-shot link prediction KG benchmarks.
1 Introduction
MetaR addresses few-shot link prediction by transferring relation-specific meta information from a few observed triples to incomplete triples. It introduces relation meta and gradient meta, and reports state-of-the-art benchmark results.
- Motivation: Few-shot link prediction predicts a tail entity from a head and relation after observing only K triples about that relation.About 10% of Wikidata relations have no more than 10 triples.
- Method: Relation-specific meta information transfers common relation knowledge from observed triples to incomplete triples and accelerates within-task learning.
- Method: MetaR represents relation meta as a high-order relation representation and gradient meta as its loss gradient for rapid task-specific updating.
- Novelty: MetaR does not rely on a background knowledge graph, unlike GMatching, whose approach uses one-hop graph structures and background knowledge.
- Results: MetaR achieves state-of-the-art results on few-shot link prediction tasks, while experiments show that both proposed meta-information types contribute significantly.
- Contribution: MetaR is a meta relational learning framework designed specifically for few-shot link prediction in knowledge graphs.
2 Related Work
Related work covers knowledge graph embedding and meta-learning approaches to learning representations or adapting from few instances. MetaR is positioned as addressing embedding methods’ dependence on rich training instances.
- Knowledge Graph Embedding: Knowledge graph embedding models map entities and relations into continuous vector spaces and score triples for truth.
- Knowledge Graph Embedding: MetaR uses learned relation meta rather than a conventional relation embedding while retaining a knowledge-graph embedding score function.
- Knowledge Graph Embedding: Traditional embedding models depend heavily on rich training instances, limiting their use for few-shot link prediction.
- Meta-Learning: Meta-learning aims to learn quickly from few instances and adapt incrementally across concepts.
- Meta-Learning: Meta-learning methods are grouped into metric-based, model-based, and optimization-based approaches.
- Knowledge-Graph Few-Shot Learning: GMatching is described as the first knowledge-graph few-shot learning model, using a neighbor encoder and LSTM-based multistep matching processor.
3 Task Formulation
Few-shot link prediction is defined per relation: given K support triples, the task predicts tail entities for query heads, including relations unseen during training.
- Task definition: A K-shot task uses a support set of K head–tail pairs associated with one relation and predicts the tail for a new head.The query is formulated as r : (hj, ?).
- Task definition: The query set contains all triples to be predicted for the relation, rather than necessarily one query triple.
- Train/test setting: Training uses tasks with support and query sets, while testing uses analogous tasks involving relations never seen during training.
GTr GTr
Figure 2 presents MetaR around a task Tr containing support and query sets, with relation meta and its updated form represented in the framework.
- Overview: MetaR is organized around a task Tr = {Sr, Qr} containing support and query sets.
- Overview: The framework distinguishes relation meta, updated relation meta, and gradient meta.
4 Method
MetaR transfers relation-specific information from support to query examples through relation meta and gradient meta, using a relation-meta learner and an embedding learner within each task.
- Motivation: MetaR addresses few-shot learning by transferring common relation information and using gradients to accelerate learning from few instances.
- Meta information: Relation meta is a vector extracted from support entity pairs and transferred to query examples because both sets share the same relation.
- Meta information: Gradient meta is the support-set loss gradient of relation meta, which updates relation meta before query-set scoring.
- Model modules: The Relation-Meta Learner generates relation meta from support heads’ and tails’ embeddings, while the Embedding Learner evaluates support and query triples.
- Training procedure: Algorithm 1 samples a training task, obtains relation meta, computes support loss and its gradient, updates relation meta, then computes query loss and updates the model.
- Scoring: The relation-meta learner averages entity-pair-specific relation meta, and the embedding learner scores pairs using an L2-distance function inspired by TransE.
- Training objective: MetaR uses updated relation meta for query scoring, and the query loss is minimized to update the whole model.
- Experimental setting: The experiments use dataset statistics for NELL-One and Wiki-One, with the supplied table caption defining background fitting and train, development, and test relation counts.
5 Experiments
Experiments evaluate MetaR on NELL-One and Wiki-One under multiple dataset settings, comparing it with GMatching and analyzing the roles of meta information, entity sparsity, and training-task count. MetaR performs better across reported metrics, while its performance depends on dataset sparsity and available training tasks.
- Results: MetaR improves over GMatching across all reported metrics on both datasets.For 1-shot prediction, average improvement is 29.53% on NELL-One and 40.08% on Wiki-One; for 5-shot NELL-One, it is 30.13%.
- Results: Transferring relation-specific meta information from support to query sets enables MetaR to accomplish few-shot link prediction and improves performance over GMatching.The reported result supports the framework’s central transfer strategy.
- Ablation study: Removing gradient meta decreases performance by 29.3% and 15%, while additionally removing relation meta decreases it by 55% and 72% versus standard MetaR.The ablation results indicate that both components contribute, with relation meta contributing more than gradient meta.
- Factors affecting performance: Entity sparsity and the number of training tasks affect MetaR’s performance; extremely sparse datasets favor pre-trained entity embeddings, while more training tasks generally help.Wiki-One has 82.8% one-shot entities versus 37.1% in NELL-One, and NELL-One performance rises from 0.279 to 0.401 Hits@10 when training tasks increase from 51 to 321.
6 Conclusion
The paper proposes MetaR for few-shot KG link prediction by transferring relation-specific meta information from support sets to query sets. Relation meta transfers shared information, while gradient meta accelerates learning; performance is affected by training-task count and entity sparsity.
- Conclusion: MetaR transfers relation-specific meta information from support sets to query sets for few-shot link prediction.The framework uses relation meta for shared information and gradient meta to accelerate learning.
- Conclusion: MetaR outperforms GMatching and does not require a background knowledge graph.The paper identifies training-task count and entity sparsity as factors affecting performance.