Source-linked AI summary
Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach
Takuo Hamaguchi, Hidekazu Oiwa, Masashi Shimbo, Yuji Matsumoto
TL;DR
The paper addresses knowledge base completion when test-time entities were absent during training and therefore lack embeddings. It uses Graph-NNs to compute those embeddings from neighborhood information, achieving strong OOKB performance and state-of-the-art WordNet11 performance in standard KBC.
Problem
Embedding-based KBC lacks embeddings for entities first observed after training, creating the OOKB entity problem and making costly retraining the conventional solution.
Method
The paper applies Graph-NNs to compute OOKB entity embeddings from neighborhood vectors and uses TransE as the output model.
Results
The model outperformed baselines considerably in the OOKB setting and achieved state-of-the-art performance on WordNet11 in standard KBC.
Takeaways & Limitations
Graph-NNs provide a KBC model tailored to entities unobserved at training time, while also performing strongly in the standard setting on WordNet11.
Takeaways & Limitations
The standard triplet-classification setting assumes that entities and relations are limited to those appearing in the training knowledge base.
Abstract
from arXiv · showhide
Knowledge base completion (KBC) aims to predict missing information in a knowledge base.In this paper, we address the out-of-knowledge-base (OOKB) entity problem in KBC:how to answer queries concerning test entities not observed at training time. Existing embedding-based KBC models assume that all test entities are available at training time, making it unclear how to obtain embeddings for new entities without costly retraining. To solve the OOKB entity problem without retraining, we use graph neural networks (Graph-NNs) to compute the embeddings of OOKB entities, exploiting the limited auxiliary knowledge provided at test time.The experimental results show the effectiveness of our proposed model in the OOKB setting.Additionally, in the standard KBC setting in which OOKB entities are not involved, our model achieves state-of-the-art performance on the WordNet dataset. The code and dataset are available at https://github.com/takuo-h/GNN-for-OOKB
1 Introduction
Knowledge base completion predicts missing relation triplets, but embedding-based models lack embeddings for entities first seen after training. The paper addresses this OOKB problem with Graph-NNs that transfer information from existing neighbors without retraining.
- Knowledge bases represent facts as relation triplets connecting a head entity, relation, and tail entity.
- OOKB entities appear after training, so embedding-based KBC models cannot directly predict relations involving them without obtaining new embeddings.Retraining with added triplets is possible, but the paper seeks to avoid its cost.
- New entities arise in everyday settings such as events and products, motivating inference from existing knowledge about entities like Blade Runner.The example uses an existing relation for Do-Androids-Dream-of-Electric-Sheep? to estimate whether Blade Runner is science fiction.
- The proposed Graph-NN computes an OOKB entity’s vector from neighborhood vectors at test time and uses an embedding-based KBC objective as its output model.The propagation model pools neighborhood information, while TransE is used as the output model.
- The paper proposes an OOKB formulation and Graph-NN model, then evaluates effectiveness in both standard and OOKB entity settings.
2 OOKB Entity Problem in Knowledge Base Completion
Knowledge completion identifies the complete set of facts from an incomplete knowledge base. The OOKB task extends triplet classification by adding test-time edges to entities absent during training and requiring missing relations involving them to be predicted.
- 2.1 Knowledge Graph: Knowledge completion seeks to identify all gold relation triplets when only an incomplete subset of facts is accessible.
- 2.1 Knowledge Graph: A knowledge graph treats entities as nodes and relation triplets as labeled edges between their endpoint entities.
- 2.2 KBC: Triplet Classification: Triplet classification determines whether each triplet absent from the knowledge base is a missing positive fact or a negative fact.
- 2.2 KBC: Triplet Classification: In standard triplet classification, entities and relations are restricted to those appearing in the training knowledge base.
- 2.3 OOKB Entity Problem: The OOKB entity problem introduces test-time auxiliary triplets containing new entities but no new relations, with each added edge bridging a new entity and a training entity.The task is to identify missing relation triplets involving the new entities.
- 2.3 OOKB Entity Problem: Because OOKB embeddings are unavailable, the model must compute them from embeddings of entities in the original knowledge base using the added auxiliary knowledge.
3 Proposed Model
The proposed model adapts Graph-NNs to knowledge graphs by propagating relation-aware neighborhood information and using TransE for knowledge-base completion.
- Graph-NN architecture: Graph-NNs encode knowledge-graph entities and edges as vectors, using propagation and output models suited to graph-structured KBC.The propagation model controls information flow between nodes, while the output model defines the task objective over vector representations.
- Propagation model: Relation-specific transition functions transform neighbor representations before their information is pooled into the current entity vector.Separate head and tail neighborhoods are represented by Shead(e) and Stail(e), and pooling replaces summation in the propagation update.
- Propagation model: Transition functions can use identity, tanh, or ReLU transformations, with batch normalization, residual connections, and LSTM techniques also available.The paper's experiments use specified transition functions, including batch normalization.
- Stacking and unrolling: Stacking uses different propagation parameters at each time step, whereas unrolling reuses the same parameters across propagations.Both approaches broadcast information from a node to farther nodes by repeatedly applying the propagation model.
- Output model: The output model uses TransE, whose score measures triplet implausibility through the distance between vh + vr and vt.The paper states that other embedding-based KBC models could also be used as the output model.
- Output model: The absolute-margin objective drives positive-triplet scores toward zero and negative-triplet scores to at least the margin τ.Unlike the pairwise-margin objective, it treats positive and negative triplets separately.
4 Experiments
The experiments evaluate the model in standard and OOKB knowledge-base completion settings using WordNet11 and Freebase13, with additional OOKB datasets derived from WordNet11. The proposed Graph-NN outperforms baselines in the OOKB setting and achieves state-of-the-art performance on WordNet11 in the standard setting, while deeper propagation does not improve standard classification.
- Datasets and standard evaluation: WordNet11 and Freebase13 were used for standard evaluation, with positive and negative triplets in validation and test sets.Training sets lacked negative triplets; corrupted triplets were generated using the Bernoulli trick.
- Standard triplet classification: The model showed state-of-the-art performance on WordNet11 but underperformed state-of-the-art KBC methods on Freebase13 while slightly exceeding TransE.These results come from the standard setting, in which OOKB entities are not involved.
- OOKB dataset construction: Nine WordNet11 OOKB datasets varied the number and positions of OOKB entities across Head, Tail, and Both settings.The datasets used 1,000, 3,000, or 5,000 selected test triplets and filtered candidates lacking connections to training entities.
- OOKB entity experiment: Graph-NN models considerably outperformed the simple pooling baselines in the OOKB experiment.Graph-NN with average pooling performed best on all datasets, whereas max pooling was weaker in several settings, including Tail-3000.
- Stacking and unrolling Graph-NNs: Depth 1 yielded 87.8%, and stacking and unrolling did not improve performance in standard WordNet11 triplet classification.Depth indicates how many times the propagation model is iteratively applied; the authors attribute the lack of improvement to embedding models representing distant-node information in continuous space.
5 Conclusion
The paper formulates KBC with entities unobserved during training and proposes a Graph-NN tailored to this OOKB setting. Experiments report considerable improvements over baselines for OOKB entities and state-of-the-art performance on WordNet11 in standard KBC.
- Conclusion: The paper introduces a KBC task involving entities unobserved at training time and proposes a Graph-NN tailored to that task.Two triplet-classification tasks evaluate the proposed model in OOKB and standard settings.
- Conclusion: In the OOKB entity problem, the proposed model outperformed the baselines considerably.
- Conclusion: In standard KBC, the model achieved state-of-the-art performance on WordNet11.