Source-linked AI summary
Automated Self-Supervised Learning for Recommendation
Lianghao Xia, Chao Huang, Chunzhen Huang, Kangyi Lin, Tao Yu, Ben Kao
TL;DR
Existing contrastive recommendation methods rely on manually designed views that may not adapt across datasets and tasks. AutoCF learns graph structure-adaptive augmentation through masking and a masked graph autoencoder, and experiments report superiority over various baselines.
Problem
Contrastive recommendation methods depend on manually generated views, limiting adaptive augmentation across datasets and tasks and potentially introducing noisy information.
Method
AutoCF uses learnable masking and a masked graph autoencoder to distill self-supervised signals while aggregating global collaborative information through masked subgraph reconstruction.
Results
Experiments and ablations on public recommendation datasets demonstrate AutoCF's effectiveness and superiority over existing recommendation methods and strong baselines.
Takeaways & Limitations
Automated self-supervised augmentation provides an effective approach for enhancing graph-based collaborative filtering.
Abstract
from arXiv · showhide
Graph neural networks (GNNs) have emerged as the state-of-the-art paradigm for collaborative filtering (CF). To improve the representation quality over limited labeled data, contrastive learning has attracted attention in recommendation and benefited graph-based CF model recently. However, the success of most contrastive methods heavily relies on manually generating effective contrastive views for heuristic-based data augmentation. This does not generalize across different datasets and downstream recommendation tasks, which is difficult to be adaptive for data augmentation and robust to noise perturbation. To fill this crucial gap, this work proposes a unified Automated Collaborative Filtering (AutoCF) to automatically perform data augmentation for recommendation. Specifically, we focus on the generative self-supervised learning framework with a learnable augmentation paradigm that benefits the automated distillation of important self-supervised signals. To enhance the representation discrimination ability, our masked graph autoencoder is designed to aggregate global information during the augmentation via reconstructing the masked subgraph structures. Experiments and ablation studies are performed on several public datasets for recommending products, venues, and locations. Results demonstrate the superiority of AutoCF against various baseline methods. We release the model implementation at https://github.com/HKUDS/AutoCF.
1 INTRODUCTION
Graph-based collaborative filtering benefits from self-supervision but existing methods depend on manually designed augmentations that may be non-adaptive and noise-sensitive. AutoCF addresses this gap with learnable masking and a masked graph autoencoder for structure-adaptive augmentation.
- Existing contrastive recommenders rely on manually generated views whose effectiveness varies across datasets and tasks and may introduce noisy or irrelevant information.
- Random node or edge dropout can discard important structural information while retaining noisy interactions such as misclicks or popularity bias.
- AutoCF automatically distills structure-adaptive self-supervised signals through a learnable masking function that identifies important centric nodes for reconstruction.
- Its masked graph autoencoder aggregates global collaborative relationships to reconstruct masked user-item subgraphs while preserving graph-based collaborative structure.
- AutoCF improves over state-of-the-art GNN-based and self-supervised recommenders, including NCL and HCCF.
2 GRAPH CONTRASTIVE LEARNING FOR COLLABORATIVE FILTERING
Graph-based collaborative filtering models propagate information over user-item interaction graphs, while contrastive extensions add self-supervised objectives through generated representation views. Their effectiveness nevertheless depends on carefully designed handcrafted augmentations, motivating automated graph augmentation.
- Graph-based CF represents users and items as an interaction graph and uses message construction and aggregation to model high-order connectivity.
- Contrastive recommendation methods align generated representation views and jointly optimize the auxiliary self-supervised loss with the recommendation loss.
- Most graph contrastive recommendation solutions rely on carefully designed contrastive views and handcrafted augmentation strategies, limiting adaptive self-supervision.
3 METHODOLOGY
AutoCF automates self-supervised augmentation by learning which graph structures to mask and reconstruct, while combining local encoding with global attention-based reconstruction. Its training jointly uses reconstruction, mutual-information, and contrastive objectives to improve collaborative-filtering representations.
- 3.1 Learning to Mask Paradigm: AutoCF learns to mask informative user-item edges around selected centric nodes, using k-hop subgraph structure and node-subgraph semantic relatedness.The masking function selects top-ranked centric nodes by learned mask probabilities and removes their sampled-subgraph edges for reconstruction.
- 3.2 Masked Graph Autoencoder: The masked graph autoencoder uses graph convolution for structural encoding and graph self-attention for global, long-range information aggregation over sampled node pairs.The decoder restricts pairwise relation learning to a subset emphasizing the masked subgraph, improving feasibility on large graphs.
- 3.3 Model Training: AutoCF jointly optimizes recommendation with reconstruction, node-subgraph mutual-information maximization, and contrastive learning to strengthen representation discrimination.The framework combines adaptive structure augmentation, masked reconstruction, and self-augmented training objectives.
- 3.2.2 Graph Self-Attention Decoder: Graph self-attention uses multi-head attention with learned query, key, and value projections, while residual connections support propagation in the final graph layer.The attention decoder operates on selected node pairs and uses a node-pair indicator to determine which relations are calculated.
- 3.1.2 Infomax-based Semantic Relatedness: Infomax-based relatedness favors structurally consistent subgraphs and reduces noisy gradients from semantically irrelevant nodes during reconstruction.Subgraphs with larger relatedness scores provide higher lower bounds on node semantic relatedness, lowering the likelihood of noisy reconstruction gradients.
4 EVALUATION
AutoCF is evaluated across recommendation datasets and tasks against conventional, autoencoder-based, GNN-based, disentangled, and self-supervised baselines. The experiments assess overall accuracy, sparsity robustness, component effects, hyperparameters, convergence, efficiency, and representation interpretation.
- Overall Performance Comparison (RQ1): AutoCF consistently achieves the best performance against recommendation methods, including strong self-supervised approaches, across the evaluated datasets.The comparison covers Gowalla, Yelp, and Amazon using Recall and NDCG.
- Performance against Data Sparsity: AutoCF maintains superiority over strong self-supervised baselines across user groups with different interaction frequencies, using Recall@20 and NDCG@20.Figure 3 compares performance curves and improvement ratios across four interaction-frequency groups.
- Module Ablation Study (RQ2): Removing the reconstruction-based self-supervised signals reduces performance, while replacing graph self-attention also lowers results, supporting both modules’ contributions.The ablations compare AutoCF with -M and -GSA variants; the text attributes the self-attention benefit to global aggregation over the augmented graph.
- Convergence and Efficiency (RQ4): AutoCF converges faster, reaching convergence at 20 and 40 training epochs on Gowalla and Yelp, respectively, while retaining superior performance.The authors attribute this efficiency to infomax-based semantic relatedness guiding optimization with better gradients.
- Model Case Study (RQ5): AutoCF’s learned embeddings show stronger uniformity than state-of-the-art contrastive approaches, indicating better preservation of unique user preference.The case study also finds strong homophily in identified subgraphs with high semantic relatedness.
5 RELATED WORK
Related work emphasizes GNN-based recommendation and contrastive self-supervised learning for addressing sparsity and noise challenges. Existing approaches use handcrafted or random contrastive views over graph structures.
- GNN-based recommender systems extend relational modeling across social, sequential, and session-based recommendation scenarios.
- Figure 8 visualizes Yelp user-embedding distributions for different methods using Gaussian KDE.
- Contrastive self-supervised models use handcrafted or random graph-structure views to address recommendation data sparsity and noise.
6 CONCLUSION
The conclusion presents AutoCF as an automated self-supervised recommendation model that distills informative signals through graph augmentation. Experiments on benchmark datasets report effectiveness relative to existing recommendation methods.
- AutoCF addresses limitations in existing self-supervised recommendation models with automated graph augmentation and infomax-based subgraph semantic relatedness.
- The model incorporates self-supervised learning signals to distill informative self-supervision information automatically.
- Extensive benchmark experiments compare AutoCF with other recommender systems and demonstrate its effectiveness.
A APPENDIX
The appendix documents AutoCF’s learning process, baseline implementations, robustness and depth studies, scalability investigation, and theoretical derivations.
- A.1 Learning Algorithm Description: Appendix Section A.1 summarizes the learning process of the AutoCF recommender.
- A.2 Baseline Implementations: Appendix Section A.2 presents implementation details for the baseline methods.
- A.3–A.5 Additional Studies: Appendix Sections A.3–A.5 study robustness to data noise, model depth, and scalability, respectively.
- Theoretical Analysis: The appendix provides derivation details and theoretical analysis for AutoCF.
A.1 Learning Algorithm Description
Algorithm 1 trains AutoCF from a user-item interaction graph by sampling centric nodes, learning infomax-based signals, encoding masked graphs, and reconstructing sampled structures. Recommendation, self-supervised, and regularization losses are combined to update the model.
- The combined objective is used within the stated AutoCF learning process to train the model.
- The algorithm takes a user-item interaction graph and training hyperparameters as input, returning trained model parameters.
- Centric nodes are sampled with Gumbel noise before calculating the infomax-based loss.
- The sampled graph is encoded with a GCN, and edges are sampled for graph self-attention decoding.
- The algorithm calculates reconstruction and self-contrastive SSL losses alongside recommendation loss and weight-decay regularization.
A.2 Baseline Implementation Details
Baseline methods are evaluated with tuned regularization, self-supervised, temperature, and dropout settings. The comparison includes robustness-related evaluation against contaminated interactions.
- Baseline regularization weights are tuned over 1e-k for 3 ≤ k ≤ 8 across methods.
- Self-supervised loss weights are tuned over 1e-k for -1 ≤ k ≤ 6, while contrastive temperatures span 0.01 to 3.
- Noise robustness is evaluated by varying contaminated user-item interactions along the x-axis and comparing method performance on the y-axis.
A.3 Model Robustness Study
AutoCF is assessed for robustness to noisy interactions, efficiency on large-scale data, convergence, and hyperparameter sensitivity. The supplied results state that AutoCF remains superior as noise increases.
- AutoCF consistently outperforms state-of-the-art SSL recommenders as contaminated training interactions increase from 10% to 50%.The test set remains unchanged, and the performance gain becomes larger at higher noise percentages.
- The scalability comparison reports model performance and per-epoch training time for representative methods on the large-scale Tmall dataset.
- Convergence is analyzed across epochs on Gowalla and Yelp using NDCG@20.
- AutoCF hyperparameter changes are studied on Yelp and Gowalla using NDCG@20.
- Model depth and dimensionality are evaluated on Yelp and Gowalla using Recall@20 and NDCG@20.
A.4 Imapct of Depth and Dimensionality
The depth and embedding dimensionality studies characterize how AutoCF’s graph modeling and representation capacity affect performance. Increasing either parameter eventually yields limited additional benefit.
- Too few graph iterations impair connectivity modeling, whereas further increases provide limited improvement because of oversmoothing.
- Embedding dimensionality shows an underfitting-to-overfitting pattern: performance rises sharply from 8 to 16 or 32, then improves marginally at 64.
A.5 Model Scalability Study
The scalability study compares AutoCF with baselines on large-scale recommendation data and reports competitive efficiency alongside better recommendation performance. The supplied passages also include reconstruction-gradient and semantic-relatedness analysis.
- The theoretical analysis examines gradients of the reconstruction loss for masked-subgraph node embeddings.
- The analysis derives lower bounds for semantic relatedness between node pairs using distances and angles between their embedding vectors.