Source-linked AI summary
End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completion
Chao Shang, Yun Tang, Jing Huang, Jinbo Bi, Xiaodong He, Bowen Zhou
TL;DR
Knowledge graph embedding for knowledge base completion lacks structure enforcement in ConvE’s embedding space, despite graph connectivity and node attributes providing useful information. SACN combines a weighted graph convolutional encoder with a Conv-TransE decoder, and reports about 10% relative improvement over ConvE on standard benchmarks, while performance is worse for high-indegree nodes.
Problem
Knowledge base completion must predict missing triplets, while ConvE does not incorporate knowledge-graph connectivity structure into its embedding space.
Method
SACN jointly trains a relation-aware WGCN encoder using graph structure and attributes with a Conv-TransE decoder that preserves es + er ≈ eo.
Results
About 10% relative improvement over ConvE is reported on FB15k-237 and WN18RR for HITS@1, HITS@3, and HITS@10.
Takeaways & Limitations
SACN combines adaptive neighborhood aggregation, attribute integration, and translational convolution for link prediction.
Takeaways & Limitations
SACN performs worse than Conv-TransE for high-indegree nodes because graph aggregation substantially smooths their embeddings among neighbors.
Abstract
from arXiv · showhide
Knowledge graph embedding has been an active research topic for knowledge base completion, with progressive improvement from the initial TransE, TransH, DistMult et al to the current state-of-the-art ConvE. ConvE uses 2D convolution over embeddings and multiple layers of nonlinear features to model knowledge graphs. The model can be efficiently trained and scalable to large knowledge graphs. However, there is no structure enforcement in the embedding space of ConvE. The recent graph convolutional network (GCN) provides another way of learning graph node embedding by successfully utilizing graph connectivity structure. In this work, we propose a novel end-to-end Structure-Aware Convolutional Network (SACN) that takes the benefit of GCN and ConvE together. SACN consists of an encoder of a weighted graph convolutional network (WGCN), and a decoder of a convolutional network called Conv-TransE. WGCN utilizes knowledge graph node structure, node attributes and edge relation types. It has learnable weights that adapt the amount of information from neighbors used in local aggregation, leading to more accurate embeddings of graph nodes. Node attributes in the graph are represented as additional nodes in the WGCN. The decoder Conv-TransE enables the state-of-the-art ConvE to be translational between entities and relations while keeps the same link prediction performance as ConvE. We demonstrate the effectiveness of the proposed SACN on standard FB15k-237 and WN18RR datasets, and it gives about 10% relative improvement over the state-of-the-art ConvE in terms of HITS@1, HITS@3 and HITS@10.
Introduction
Knowledge base completion predicts missing triplets in incomplete multi-relational knowledge graphs. SACN combines graph-structure-aware encoding with convolutional decoding to address limitations in ConvE while improving benchmark performance.
- Motivation: Knowledge base completion predicts new triplets to expand incomplete knowledge bases.Knowledge bases contain many entities and triplets but remain incomplete relative to existing and newly added real-world knowledge.
- Prior limitations: ConvE achieves state-of-the-art link-prediction performance with 2D convolution and nonlinear features but does not preserve TransE’s translational property.ConvE reshapes and concatenates entity and relation embeddings before convolution, unlike the additive operation es + er ≈ eo.
- Design rationale: The framework integrates graph convolutional benefits, including node attributes and shared neighborhood aggregation, with convolutional knowledge-graph embedding.GCNs create node embeddings by aggregating local neighborhood information and can leverage node attributes.
- Proposed model: SACN combines a weighted graph convolutional encoder with a Conv-TransE decoder to use graph structure, node attributes, and relation types.WGCN learns neighbor-aggregation weights, while Conv-TransE retains translational characteristics between entities and relations.
- Results: About 10% relative improvement over ConvE is reported on FB15k-237 and WN18RR for HITS@1, HITS@3, and HITS@10.The reported comparison is against the state-of-the-art ConvE baseline on standard datasets.
Related Work
Knowledge graph embedding methods model relational triplets efficiently, but earlier approaches largely ignore node attributes and large-scale graph connectivity. Related graph-convolution work addresses structure and features, motivating SACN’s combination of these capabilities with convolutional decoding.
- Knowledge graph embeddings: TransE and later embedding models represent entities and relations in continuous vectors for knowledge base completion and relation extraction.TransE imposes the translational constraint es + er ≈ eo, while later models introduce more complex relational representations.
- Convolutional models: ConvE uses 2D convolutions across embedding dimensions, whereas ConvKB uses 1D convolutions that preserve TransE’s translational property.ConvKB can be viewed as a special case of Conv-TransE using filters with width 1.
- Open limitations: Earlier embedding approaches model relational triplets but ignore many node attributes and large-scale connectivity structure.SACN is presented as addressing both omitted sources of information.
- Graph convolutional networks: Graph convolutional networks evolved from computationally intensive spectral formulations toward localized and more efficient spatial operations.Later methods introduced smooth parametric spectral filters, followed by simplified approaches designed to improve computational efficiency.
- Scalability: PinSage demonstrated data-efficient graph embedding at web scale by combining random walks and graph convolutions over structure and node features.The cited Pinterest application involved 3 billion nodes and 18 billion edges.
- Connection to SACN: SACN is illustrated as an end-to-end pipeline that encodes graph structure and decodes entity-relation pairs for candidate-object scoring.The encoder builds node embeddings, Conv-TransE processes es and er, and projected outputs are matched with candidate eo embeddings.
Method
SACN combines a weighted graph convolutional encoder with a Conv-TransE decoder to incorporate graph structure, attributes, and relation types while preserving translational entity–relation modeling.
- SACN architecture: SACN jointly trains a WGCN encoder and Conv-TransE decoder to represent entities from graph neighborhoods and recover knowledge-base triplets.The encoder aggregates connected entities, while the decoder models relations and preserves es + er ≈ eo.
- Weighted graph convolution: WGCN learns relation-specific weights that control how much information each neighboring relation subgraph contributes during aggregation.It treats the multi-relational graph as multiple single-relational subgraphs and combines their GCN embeddings with adaptive weights.
- Weighted graph convolution: Each WGCN layer transforms node representations using first-order neighbors, relation-weighted adjacency matrices, self-connections, and a learned connection matrix.The layer maps F^l input features to F^(l+1) output features, with relation interaction strengths learned during training.
- Node attributes: SACN represents entity attributes as additional attribute nodes, allowing their information to reach related entity embeddings through graph bridges.This avoids sparse attribute vectors and ambiguous zero values in the node representation.
- Conv-TransE: Conv-TransE applies convolution directly to stacked entity and relation embeddings without ConvE’s reshaping step, retaining the translational property of TransE.The decoder uses 2 × k filters, produces feature maps, applies a linear transformation and nonlinear scoring, and matches the result with the object embedding.
- Conv-TransE: SACN uses convolutional feature maps and a transformed embedding to score candidate triplets, with logistic sigmoid scoring during training.The feature map matrix is vectorized, projected into the entity-embedding space, and matched to eo.
Benchmark Datasets
The study evaluates link prediction on three benchmark knowledge graphs: FB15k-237, WN18RR, and FB15k-237-Attr. These datasets derive from Freebase or WordNet and differ in relation and attribute coverage.
- Three benchmark datasets—FB15k-237, WN18RR and FB15k-237-Attr—are used to evaluate link prediction.
- FB15k-237 contains Freebase relation triples and textual mentions, with inverse relations removed.
- Table 2 reports statistics for the benchmark datasets.
- WN18RR contains 93,003 triples, 40,943 entities and 11 relation types, with inverse-relation test leakage removed.
Data Construction
The data construction adds entity attributes to the FB15k-237 evaluation setting, addressing the omission of attributes in many prior methods. The resulting FB15k-237-Attr dataset supports evaluating attribute modeling.
- Many previous methods model entities and relations while ignoring abundant entity attributes.
- FB15k-237-Attr is built by extracting attribute triples for entities in FB15k-237.
- FB24k provides 207,151 attribute triples and 314 attributes, and serves as the source for attribute-triple construction.
Experimental Setup
The experiments use manually specified hyperparameter grids and dataset-specific settings for SACN and Conv-TransE. Training uses two-layer WGCN models, Adam, PyTorch, and NVIDIA Tesla P40 GPUs.
- Hyperparameters are selected by grid search over learning rate, dropout rate, embedding size, number of kernels, and kernel size.
- Table 3 reports link-prediction results for FB15k-237, WN18RR, and FB15k-237-Attr.
- All models use two WGCN layers, with dataset-specific SACN settings for dropout, kernels, learning rate, and embedding size.
- Training uses Adam and PyTorch on NVIDIA Tesla P40 GPUs.
Results
SACN combines structural information and node attributes with convolutional decoding, outperforming ConvE and showing benefits from larger kernel views while revealing an indegree-related limitation.
- Decoder comparison: Conv-TransE improves over ConvE on FB15k-237 by 4.1% for Hits@10 and 5.7% for Hits@3, and on WN18RR by 8.3% and 9.3%.The results support retaining translational characteristics while improving performance.
- Overall performance: SACN improves over ConvE across Hits@10, Hits@3, Hits@1, and MRR on FB15k-237 and WN18RR.Improvements range from 2.2% to 12.5% across the reported dataset-metric pairs.
- Node attributes: 12.2%: SACN with attributes improves ConvE’s Hits@10 on FB15k-237-Attr, while also improving SACN without attributes by 1.9%.The corresponding gains are 14.3% for Hits@3, 12.5% for Hits@1, and 12.5% for MRR versus ConvE; gains over SACN without attributes are 2.6%, 3.8%, and 2.9%.
- Node attributes: Adding attributes to ConvE does not improve its performance on FB15k-237-Attr under the reported test results.The reported values are 0.46 Hits@10, 0.33 Hits@3, 0.22 Hits@1, and 0.30 MRR.
- Kernel size analysis: Increasing kernel size improves all reported metrics on FB15k-237 and FB15k-237-Attr, although the optimal size may depend on the task.Larger kernels collect information across combinations of entity and relation-vector attributes.
- Node indegree analysis: Higher indegree scopes increase average Hits@10 and Hits@3, but high-indegree nodes can receive overly smoothed embeddings and perform worse than with Conv-TransE.Neighbor selection is proposed as one possible solution.
Conclusion and Future Work
The paper introduces SACN, an end-to-end structure-aware convolutional network combining weighted graph convolution with translational convolutional decoding. It reports about 10% overall improvement over ConvE and identifies neighbor selection as future work.
- Conclusion: SACN uses WGCN to incorporate graph connectivity, node attributes, and relation types with learnable neighbor-aggregation weights.Attributes are represented as additional nodes so they can be integrated into the graph structure.
- Conclusion: Conv-TransE models relations as translation while using convolution to capture entity-relation interactions for link prediction.The paper reports that Conv-TransE alone achieves state-of-the-art performance.
- Conclusion: About 10%: SACN improves over ConvE overall, while combining connectivity, attributes, relation types, and translational modeling.The conclusion summarizes the model’s reported performance and design scope.
- Future work: Future work will incorporate neighbor selection, including importance pooling, into the SACN training framework.This addresses the treatment of neighboring nodes in aggregation.