Source-linked AI summary
KGAT: Knowledge Graph Attention Network for Recommendation
Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, Tat-Seng Chua
TL;DR
Recommendation methods often fail to capture relations among interactions and item attributes. KGAT constructs a collaborative knowledge graph and explicitly propagates embeddings with attention over high-order connectivities. Across three public benchmarks, KGAT consistently achieves the best performance, while its attention mechanism supports interpretation of relation importance.
Problem
Existing methods overlook relations among interactions and items, limiting their ability to capture high-order collaborative signals from user behavior and side information.
Method
KGAT unifies user-item interactions and item knowledge, recursively propagates neighbor embeddings, and uses attention to weight high-order connectivities end to end.
Results
KGAT consistently yields the best performance on three datasets, improving recall@20 over the strongest baselines by 8.95% on Amazon-book, 4.93% on Last-FM, and 7.18% on Yelp2018.
Takeaways & Limitations
Explicit embedding propagation captures high-order collaborative signals, while attention weights reveal the importance of compositional semantic relations.
Takeaways & Limitations
The work is an initial attempt focused on knowledge graphs and identifies integrating other structural information, such as social networks and item contexts, as future work.
Abstract
from arXiv · showhide
To provide more accurate, diverse, and explainable recommendation, it is compulsory to go beyond modeling user-item interactions and take side information into account. Traditional methods like factorization machine (FM) cast it as a supervised learning problem, which assumes each interaction as an independent instance with side information encoded. Due to the overlook of the relations among instances or items (e.g., the director of a movie is also an actor of another movie), these methods are insufficient to distill the collaborative signal from the collective behaviors of users. In this work, we investigate the utility of knowledge graph (KG), which breaks down the independent interaction assumption by linking items with their attributes. We argue that in such a hybrid structure of KG and user-item graph, high-order relations --- which connect two items with one or multiple linked attributes --- are an essential factor for successful recommendation. We propose a new method named Knowledge Graph Attention Network (KGAT) which explicitly models the high-order connectivities in KG in an end-to-end fashion. It recursively propagates the embeddings from a node's neighbors (which can be users, items, or attributes) to refine the node's embedding, and employs an attention mechanism to discriminate the importance of the neighbors. Our KGAT is conceptually advantageous to existing KG-based recommendation methods, which either exploit high-order relations by extracting paths or implicitly modeling them with regularization. Empirical results on three public benchmarks show that KGAT significantly outperforms state-of-the-art methods like Neural FM and RippleNet. Further studies verify the efficacy of embedding propagation for high-order relation modeling and the interpretability benefits brought by the attention mechanism.
1 INTRODUCTION
Existing recommendation methods overlook relations among interactions and items, while KGAT explicitly models high-order connectivity in a collaborative knowledge graph using propagation and attention.
- Traditional methods model interactions independently, limiting their ability to distill attribute-based collaborative signals from collective user behavior.
- Collaborative knowledge graphs unify user-item behavior with item side information, making long-range, high-order connectivity central to recommendation.
- Path-based methods extract selected paths or constrained meta-paths, but path selection is separated from recommendation optimization and meta-path design requires domain knowledge.
- KGAT recursively propagates node embeddings to capture high-order relations and uses attention weights to identify the importance of neighbors and connectivities.
- Experiments on three public benchmarks demonstrate KGAT's effectiveness and interpretability for understanding high-order relations.
2 TASK FORMULATION
The task combines user-item interactions and item knowledge into a collaborative knowledge graph, then predicts user adoption by exploiting multi-hop relational connectivity.
- A user-item bipartite graph represents observed interactions, while a knowledge graph represents item-related entities and relations such as directors, actors, and genres.
- A collaborative knowledge graph integrates user behaviors and item knowledge by encoding each interaction as an additional relation in the unified graph.
- The recommendation task takes the collaborative knowledge graph as input and predicts the probability that user u adopts item i.
- High-order connectivity is defined as a multi-hop relational path, extending behavior-based and attribute-based links beyond direct interactions.
- Feature-based methods such as FM and NFM fail to reveal relatedness across fields and instances, leaving compositional high-order relations insufficiently explored.
3 METHODOLOGY
KGAT models high-order connectivity in the collaborative knowledge graph through attentive embedding propagation. Its layers encode relation-aware neighbor importance, aggregate information across propagation orders, and produce user-item matching scores efficiently.
- Model architecture: KGAT combines node embeddings, attentive propagation layers, and a prediction layer to model high-order relations end to end.Neighbor embeddings are recursively propagated, weighted with knowledge-aware attention, and used to update node representations before prediction.
- Information propagation: Entity information propagates through triplet neighbors, allowing attributes and user preferences to enrich item and user representations.The propagation process uses entities as bridges between triplets, transmitting information across the collaborative knowledge graph.
- Knowledge-aware attention: Attention weights control how much information each relation-specific neighbor contributes to a node representation.The coefficient π(h,r,t) determines propagation strength from t to h conditioned on relation r; scores are normalized across connected triplets.
- Knowledge-aware attention: KGAT differs from standard GCN, GraphSage, and graph attention by assigning varying neighbor importance and incorporating relation representations during propagation.Unlike uniform discounting, its attention depends on relational information and entity representations.
- Information aggregation: The Bi-Interaction aggregator models feature interactions between an entity and its ego-network, making propagation sensitive to their affinity.This design can pass more messages from similar entities and is compared empirically with GCN and GraphSage aggregators.
- High-order propagation: Stacked propagation layers capture higher-hop connectivity, while concatenating layer outputs enriches embeddings and controls propagation strength through L.Different layers emphasize different connectivity orders in the user- or item-rooted neighborhood.
- Computational cost: At inference on Amazon-Book, KGAT costs around 560s for all testing instances, comparable to several SL and regularization-based methods.The reported comparison lists FM at around 700s, NFM at 780s, CFKG at 800s, and CKE at 420s.
4 EXPERIMENTS
The experiments evaluate KGAT on three public datasets against knowledge-aware recommendation baselines, then examine sparsity, propagation depth, aggregators, ablations, and explanations. KGAT generally performs best, with high-order connectivity, attentive propagation, and selected architectural components contributing to performance and interpretability.
- Overall Comparison: KGAT consistently yields the best performance on Amazon-book, Last-FM, and Yelp2018, improving recall@20 over the strongest baselines by 8.95%, 4.93%, and 7.18%, respectively.The comparison includes SL, regularization-based, path-based, and graph neural network-based methods.
- Interaction Sparsity: KGAT outperforms other models in most sparsity groups, especially the two sparsest groups in Amazon-Book and Yelp2018.The authors attribute this to recursive propagation enriching inactive-user representations with high-order connectivity.
- Model Depth: Increasing propagation depth from one to two or three layers substantially improves performance, while a fourth layer provides only marginal gains.The authors associate the gains with modeling second- and third-order relations and suggest third-order connectivity may be sufficient.
- Aggregators: The Bi-Interaction aggregator improves representation learning relative to GCN, while GCN consistently outperforms GraphSage.The comparison suggests feature interaction between a node and its ego-network representation matters during aggregation.
- Ablation Study: Removing knowledge graph embedding and attention degrades performance, with removing both components performing worse than removing either one alone.Treating all neighbors equally can introduce noise, whereas triplet-level representation relatedness is explicitly modeled when both components are retained.
- Interpretability: Attention-based high-order connectivity provides recommendation evidence, but general entities can produce low-quality explanations and motivate future hard-attention filtering.One example explains recommending The Last Colony through a shared author connection with Old Man’s War.
5 CONCLUSION AND FUTURE WORK
KGAT explores high-order connectivity in collaborative knowledge graphs through attentive embedding propagation. The work demonstrates its effectiveness while positioning structural knowledge propagation as an initial direction for recommendation research.
- KGAT explicitly models high-order connectivities in collaborative knowledge graphs end to end for knowledge-aware recommendation.Its core attentive embedding propagation layer adaptively propagates neighboring embeddings to update node representations.
- Experiments on three real-world datasets demonstrate the rationality and effectiveness of KGAT.
- The work is an initial attempt to exploit structural knowledge with information propagation in recommendation.Future directions include integrating social networks and decision processes with collaborative knowledge graphs.