Source-linked AI summary
Knowledge-Enhanced Hierarchical Graph Transformer Network for Multi-Behavior Recommendation
Lianghao Xia, Chao Huang, Yong Xu, Peng Dai, Xiyue Zhang, Hongsheng Yang, Jian Pei, Liefeng Bo
TL;DR
Existing recommendation methods largely focus on singular interaction types, despite practical settings involving interdependent behaviors, item knowledge, and dynamic interactions. KHGT combines hierarchical graph transformers, knowledge-aware user-item and item-item relation learning, and temporal encoding to model these patterns. Across three real-world datasets and varied evaluation settings, KHGT consistently outperforms many state-of-the-art recommendation methods.
Problem
Existing methods mainly model singular user-item interaction types, limiting their treatment of multi-behavior dependencies, knowledge-aware item relations, and interaction dynamics.
Method
KHGT jointly learns type-specific user-item and knowledge-aware item-item relations with hierarchical graph transformers, attentive behavior fusion, and temporal encoding.
Results
KHGT consistently outperforms many state-of-the-art recommendation methods across three real-world datasets and various evaluation settings.
Takeaways & Limitations
Jointly modeling multi-type behavior inter-dependencies and knowledge-aware item collaborative signals supports improved target-behavior recommendation performance.
Abstract
from arXiv · showhide
Accurate user and item embedding learning is crucial for modern recommender systems. However, most existing recommendation techniques have thus far focused on modeling users' preferences over singular type of user-item interactions. Many practical recommendation scenarios involve multi-typed user interactive behaviors (e.g., page view, add-to-favorite and purchase), which presents unique challenges that cannot be handled by current recommendation solutions. In particular: i) complex inter-dependencies across different types of user behaviors; ii) the incorporation of knowledge-aware item relations into the multi-behavior recommendation framework; iii) dynamic characteristics of multi-typed user-item interactions. To tackle these challenges, this work proposes a Knowledge-Enhanced Hierarchical Graph Transformer Network (KHGT), to investigate multi-typed interactive patterns between users and items in recommender systems. Specifically, KHGT is built upon a graph-structured neural architecture to i) capture type-specific behavior characteristics; ii) explicitly discriminate which types of user-item interactions are more important in assisting the forecasting task on the target behavior. Additionally, we further integrate the graph attention layer with the temporal encoding strategy, to empower the learned embeddings be reflective of both dedicated multiplex user-item and item-item relations, as well as the underlying interaction dynamics. Extensive experiments conducted on three real-world datasets show that KHGT consistently outperforms many state-of-the-art recommendation methods across various evaluation settings. Our implementation code is available at https://github.com/akaxlh/KHGT.
Introduction
Existing recommender systems often model only one interaction type, while practical settings require learning from interdependent behaviors, item knowledge, and interaction dynamics. KHGT addresses these challenges with hierarchical graph transformers and reports gains across three real-world datasets.
- Motivation: Single-behavior recommendation methods cannot fully distill collaborative signals from users’ multiple interaction types.Auxiliary behaviors such as page views and favorites can assist purchase prediction.
- Challenges: Multi-be behavior recommendation must model complex cross-type dependencies, knowledge-aware item relations, and time-varying interactions.The paper highlights complementary, mutually exclusive, and hierarchical relationships among behaviors.
- Related limitations: Recent multi-behavior models treat interactions locally and independently, leaving high-order multiplex relations, item side knowledge, and interaction dynamics less explored.These limitations motivate a more integrated representation of users and items.
- KHGT framework: KHGT uses a hierarchically structured graph transformer to learn high-order relations in a knowledge-aware multi-behavior collaborative graph.Its first stage captures type-specific user-item patterns in a time-aware environment, while its second stage models cross-type dependencies and contributions.
- Evaluation: KHGT achieves significant gains over many state-of-the-art baselines on movie, venue, and product recommendation datasets.The study also examines model interpretation through qualitative case studies.
Preliminaries
The paper represents recommendation data with a multi-behavior user-item graph and a knowledge-aware item-item relation graph. The task is to infer unseen target-behavior interactions from these two graph inputs.
- User-item graph: The user-item multi-behavior graph Gu contains users, items, and K relation types such as browsing, favoriting, and purchasing.Each edge records a user’s interaction with an item under a specific behavior type.
- Item-item graph: The knowledge-aware item-item graph Gv represents multiplex dependencies between items using external meta-relations.Relations may reflect shared categories, locations, or users under the same behavior type.
- Task formulation: The recommendation task takes Gu and Gv as inputs and predicts the probability of an unseen interaction under target behavior type k.The prediction concerns a user-item pair not observed under that target behavior.
Methodology
KHGT uses hierarchical graph-transformer components to model heterogeneous user-item and item-item relations across behaviors. It adds temporal context, global behavior-aware parameter learning, gated cross-type fusion, and high-order propagation for target-behavior prediction.
- Attentive Heterogeneous Message Aggregation: KHGT aggregates heterogeneous multi-behavior user-item patterns and item-item dependencies in a unified graph-structured neural network.The framework uses attentive heterogeneous message aggregation to preserve behavior-specific relational representations.
- Behavior Dynamics Encoding: Sinusoidal temporal encoding maps each interaction timestamp into behavior-specific context representations before graph attention propagates messages.The temporal representation is projected with behavior-specific weights and added to node embeddings during message propagation.
- Global Context Enhanced Parameter Learning: Global context learning uses M latent projection subspaces and gated weights to represent shared aspects across K behavior types with parameter-efficient aggregation.The channels correspond to latent projection subspaces, and M is typically smaller than K.
- Behavior Hierarchical Dependency Modeling: A transformer-based mutual relation encoder models pairwise dependencies among behavior types and supports collaboration between type-specific behavior representations.The resulting representations are fused using learned importance scores for forecasting the target interaction type.
- Cross-Type Fusion and Prediction: Gated fusion combines type-specific user-item and item-item relation representations, while high-order propagation summarizes embeddings across graph-convolutional orders.The final user and item representations support target-behavior likelihood inference with a marginal pair-wise loss.
- Learning and Complexity: Random-walk-based sub-graph sampling supports large-scale graph processing, while the model's dominant transformations require O(K × (I + J) × d^2) computation.Overall time complexity is reported as comparable to GNN-based multi-behavior recommendation methods, with moderate extra memory.
Evaluation
KHGT is evaluated across datasets, baselines, behavior configurations, sparsity levels, and parameter settings. It achieves the strongest reported performance across the main comparisons, while ablations show benefits from its behavioral, temporal, and knowledge-aware components.
- Experimental Settings: The evaluation uses Yelp, MovieLens, and Online Retail data with HR@k and NDCG@k at k = 10, using time-aware leave-one-out splitting.The test set contains each user’s last interactive item, while earlier interactions are used for training.
- Experimental Settings: The baselines span matrix factorization, autoencoder, neural collaborative filtering, autoregressive, graph neural network, multi-behavior, and knowledge-aware recommendation methods.Examples include BiasMF, AutoRec, CDAE, DMF, NCF, NADE, CF-UIcA, ST-GCN, NGCF, NMTR, DIPN, NGCFM, MATN, MBGCN, and KGAT.
- Performance Validation (RQ1): KHGT achieves remarkable performance improvements across the three datasets and the best performance under different top-N settings on Yelp.The reported gap is attributed to jointly exploring multi-type behavior inter-dependencies and knowledge-aware item collaborative signals.
- Performance Validation (RQ1): Multi-behavior baselines improve over other baselines, while GNN-based approaches outperform autoencoder and autoregressive collaborative filtering models.These comparisons support aggregating multiplex behavioral patterns and exploring high-order user-item collaborative signals.
- Model Ablation Study (RQ2): The full KHGT model achieves the best performance in all ablation cases, supporting explicit type-specific modeling, mutual dependency learning, behavior fusion, temporal context, and item knowledge.The ablation conclusions identify positive effects from temporal context and external item knowledge, alongside the behavioral modules.
- Behavior, Sparsity, and Parameter Studies: Using all interaction behavior types consistently achieves the best performance on Yelp and Online Retail, while parameter studies identify overfitting beyond d = 16 and best results with two graph layers.Weekly temporal resolution and smaller training subgraphs with relatively larger test sampling scales are also reported as favorable settings.
Related Work
Multi-relation recommender systems model different relation types on the user or item side, including social dependencies and external knowledge-graph information. These approaches broaden collaborative filtering beyond a single user-item relation.
- Multi-relation recommendation models characterize users with different relation types from either the user or item side.
- Social recommender systems integrate user-user social influence dependencies with user-item interactions to alleviate data sparsity.
- Knowledge-graph-based recommendation models use external entity or item relations to construct additional structural connections.
Conclusion
KHGT models type-specific user behavior through hierarchical graph-transformer aggregation over knowledge-aware user-item and item-item relations. Experiments on three datasets validate the framework, while future work targets online streaming deployment.
- KHGT explicitly models type-specific user behavioral patterns for recommender systems.
- The framework jointly aggregates user-item and item-item collaborative relations across multiple knowledge-aware behavior modalities.
- KHGT refines type-specific behavior representations and encodes fine-grained interactive preferences over items.
- Evaluation results on three datasets validate the proposed framework.
- Future work will deploy KHGT in an online system to handle streaming data recursively.