Source-linked AI summary
Self-Supervised Hypergraph Transformer for Recommender Systems
Lianghao Xia, Chao Huang, Chuxu Zhang
TL;DR
Existing GNN recommender systems depend on sufficient, high-quality interaction data, although practical behavior data can be noisy and skewed. SHT combines global hypergraph collaborative modeling with cross-view generative self-supervision, and experiments report improved performance over diverse baselines.
Problem
GNN-based collaborative filtering commonly relies on sufficient, high-quality user-item interaction data, whereas practical behavior data can be noisy and skewed.
Method
SHT combines local graph collaborative filtering with global hypergraph dependency learning and cross-view generative self-supervised augmentation.
Results
SHT achieves significant performance improvement over 15 different types of recommendation baselines.
Takeaways & Limitations
SHT improves the robustness of graph-based recommender systems against noise perturbation and addresses representation challenges associated with sparse interactions.
Abstract
from arXiv · showhide
Graph Neural Networks (GNNs) have been shown as promising solutions for collaborative filtering (CF) with the modeling of user-item interaction graphs. The key idea of existing GNN-based recommender systems is to recursively perform the message passing along the user-item interaction edge for refining the encoded embeddings. Despite their effectiveness, however, most of the current recommendation models rely on sufficient and high-quality training data, such that the learned representations can well capture accurate user preference. User behavior data in many practical recommendation scenarios is often noisy and exhibits skewed distribution, which may result in suboptimal representation performance in GNN-based models. In this paper, we propose SHT, a novel Self-Supervised Hypergraph Transformer framework (SHT) which augments user representations by exploring the global collaborative relationships in an explicit way. Specifically, we first empower the graph neural CF paradigm to maintain global collaborative effects among users and items with a hypergraph transformer network. With the distilled global context, a cross-view generative self-supervised learning component is proposed for data augmentation over the user-item interaction graph, so as to enhance the robustness of recommender systems. Extensive experiments demonstrate that SHT can significantly improve the performance over various state-of-the-art baselines. Further ablation studies show the superior representation ability of our SHT recommendation framework in alleviating the data sparsity and noise issues. The source code and evaluation datasets are available at: https://github.com/akaxlh/SHT.
1 INTRODUCTION
SHT addresses noisy, sparse, and skewed user-item interaction data by combining local graph modeling with global hypergraph collaboration and self-supervised augmentation. It is presented as a model-agnostic framework that improves graph-based collaborative filtering.
- Challenges: Noise and sparsity can impair graph-based collaborative filtering by distorting user representations and biasing models toward popular items.Noisy interactions may be propagated across multiple hops, while limited high-quality training signals reduce recommendation performance.
- Challenges: Existing self-supervised recommendation methods use randomly masked interaction views, which may retain noisy interactions.
- SHT Framework: SHT integrates a local collaborative relation encoder with global hypergraph dependency learning under a generative self-supervised framework.
- SHT Framework: SHT provides auxiliary supervision for graph-topology denoising and knowledge distillation, enabling local and global collaborative relations to cooperate.
- SHT Framework: The model uses hypergraph neural networks and topology-aware Transformers to encode global collaborative effects across the user-item representation space.Its hypergraph component uses learnable multi-channel hyperedge-guided message passing.
- Empirical Findings: SHT achieves significant performance improvement over 15 recommendation baselines, with ablation studies supporting its model design.
2 PRELIMINARIES AND RELATED WORK
Graph collaborative filtering models user-item interactions as a graph and refine representations through neighborhood aggregation. Recent self-supervised methods generate additional views through random graph perturbations, while SHT instead uses generative self-supervision.
- Graph Collaborative Filtering: Graph collaborative filtering constructs user-item interaction graphs and aggregates neighboring user or item embeddings to refine representations.
- Graph Collaborative Filtering: GNN-based recommendation architectures include PinSage and NGCF, while LightGCN simplifies message passing with sum-based neighborhood pooling.
- Self-Supervised Recommendation: SGL generates contrastive views through random node and edge dropout, and HCCF uses hypergraphs to generate contrastive signals.
- Self-Supervised Recommendation: SHT differs from these methods by enhancing graph collaborative filtering with a generative self-supervised learning framework.
3 METHODOLOGY
SHT combines topology-aware local graph representations with global hypergraph Transformer propagation, then uses cross-view self-augmentation to improve robustness against noisy and sparse interactions.
- Overall framework: SHT embeds local graph structure into node representations and performs global relation learning with a local-aware hypergraph Transformer.The model augments regular parameter learning with local-global cross-view self-augmentation.
- Local structure encoding: Topology-aware user and item embeddings combine ID embeddings with graph-convolutional representations before hypergraph propagation.The two-layer GCN captures neighboring subgraph structure, while residual connections incorporate nodes at different distances.
- Hypergraph relation learning: SHT uses hyperedges as intermediate hubs to distill global collaborative relations and optimize node-hyperedge connections for implicit dependencies.Multi-head attention performs node-to-hyperedge propagation, and the same framework is applied analogously on user and item sides.
- Iterative propagation: Hierarchical hypergraph layers and iterative propagation model nonlinear hyperedge interactions and long-range user-item dependencies.Layer-specific embeddings are combined by elementwise summation before dot-product prediction scores are computed.
- Local-global self-augmentation: The self-augmented task estimates interaction-edge solidity from global hypergraph representations and transfers denoised knowledge to local topology-aware embeddings.Here, solidity is the probability that an edge is not noisy, enabling local representations to be recalibrated against noisy graph structure.
- Complexity: The hypergraph Transformer reduces propagation complexity from O(K × (I + J) × d) to O((I + J + K) × d^2), achieving efficiency comparable to typical GNN inference.The self-augmented task has the same complexity as the original main task.
4 EVALUATION
SHT is evaluated against diverse recommendation baselines across three datasets, with experiments targeting overall performance, component contributions, robustness to sparsity and noise, and learned implicit relations. Results report strong performance and support the roles of hypergraph modeling, self-augmentation, and global-to-local knowledge transfer.
- Experimental Settings: Experiments compare SHT with 15 baselines across Yelp, Gowalla, and Tmall using Recall@N and NDCG@N under an all-rank protocol.Datasets are split 7:2:1 into training, validation, and testing sets, with N=20 by default.
- Overall Performance Comparison: SHT achieves the best performance under both top-20 and top-40 settings, with t-tests validating significant improvements over the baselines.The evaluation compares performance on three datasets and includes repeated retraining of SHT and SGL for p-value computation.
- Model Ablation Test: Removing topology-aware embedding or the hypergraph transformer severely damages performance, while removing self-augmented learning causes degradation in all cases.These ablations support the necessity of local and global relation learning and the positive effect of global-to-local knowledge transfer.
- Model Robustness Test: SHT maintains superior performance across different sparsity degrees, while item-side sparsity has a larger influence on performance than user-side sparsity.The larger gaps on middle item-side subdatasets suggest stronger anti-sparsity capability through knowledge transfer between dense and sparse samples.
- Case Study: The case study finds dense implicit relations among similarly embedded items and low solidity scores for interactions identified as noise.These observations are used to support SHT’s implicit relation learning and noise discrimination, with global hypergraph information recalibrating topology-aware embeddings.
5 CONCLUSION
SHT is a self-supervised hypergraph-transformer framework designed to improve user-item interaction modeling and robustness in graph-based recommender systems. Experiments report better recommendation results on real-world datasets.
- SHT integrates a hypergraph neural network with a topology-aware Transformer for self-supervised recommendation.
- The framework uses self-augmented supervision signals to improve user-item interaction modeling.
- SHT improves the robustness of graph-based recommender systems against noise perturbation.
- Experiments achieve better recommendation results on real-world datasets.
- Future work will explore disentangled user intents and diverse user-item relations for encoding multidimensional user preferences.
6 SUPPLEMENTAL MATERIAL
The supplemental material presents SHT’s learning procedure, implementation inputs and outputs, and a hyperparameter study. It also describes a neural-network-based visualization method for 32-dimensional item embeddings.
- Learning Process: Algorithm 1 summarizes SHT training from the user-item interaction graph through embedding propagation, transformations, and loss computation.The procedure includes graph topology-aware embeddings, hypergraph propagation, self-augmented learning loss, and pair-wise marginal loss.
- Learning Process: The hypergraph stage propagates information from nodes to hyperedges and back to obtain hypergraph representations.The listed steps include node-to-hyperedge propagation, hierarchical hyperedge feature processing, and hyperedge-to-node propagation.
- Hyperparameter Investigation: Small d causes a 15%-35% performance decrease, while larger d does not always improve results because d=68 marginally increases Yelp performance under overfitting.The study compares relative performance decreases against the best performance under default settings.
- Hyperparameter Investigation: K has at most a −6% effect compared with −35% for d, because its related model capacity is O(K×d) rather than O((I+J)×d).The reported pattern for K generally changes from under-fitting to over-fitting as the latent hyperedge number varies.
- Hyperparameter Investigation: The number of graph iterations L affects high-order neighborhood aggregation, with smaller L hindering nodes from collecting that information.
- Visualization: The case study visualizes 32-dimensional item embeddings with colors using a neural-network-based dimension-reduction algorithm.The algorithm is designed to preserve learned item information while selecting a limited set of colors for readability.