Self-Supervised Hypergraph Transformer for Recommender Systems

• Noise can introduce irrelevant interactions that multi-hop propagation amplifies, while sparse, skewed data biases models toward popular items and weakens training signals.Source: §1 · pp. 1–2
• SGL [31] and SLRec [37] use random masking for self-supervision, which may retain noisy interactions or drop important training signals.Source: §1 · p. 2
• SHT combines local graph convolution with a topology-aware Transformer and learnable multi-channel hyperedge-guided message passing to model global collaborative effects.Its local and global views provide cooperative supervision for interaction-graph denoising and auxiliary knowledge distillation.Source: §1 · p. 2
• SHT uses graph topological denoising and a meta-transformation layer to project global hypergraph representations into local interaction modeling, as a plug-in for existing graph-based recommenders.The framework coordinates local- and global-level collaborative relations to learn user embeddings from noisy, sparse interactions.Source: §1 · p. 2
• SHT achieves significant performance improvement over 15 different types of recommendation baselines, with ablation studies examining its design.Source: §1 · p. 2

Recap Graph Collaborative Filtering Paradigm

• Graph collaborative filtering represents user-item interactions as a graph and aggregates neighboring users’ and items’ embeddings with functions such as mean or sum.Source: §2 · p. 2

Recommendation with Graph Neural Networks

• GNN recommenders propagate embeddings across multi-hop interactions; existing models often rely on observed labels, while SHT proposes self-supervised interaction between local and global collaborative views.Examples include PinSage [38], NGCF [26], LightGCN [6], DGCF [28], DisenHAN [29], KHGT [32], and HMG-CR [35].Source: §2 · p. 2

Hypergraph-based Recommender Systems

• Prior hypergraph recommenders use manually designed structures, whereas SHT automates hypergraph structure learning to model global collaborative relations.HyRec uses users as hyperedges, MHCN [39] models high-order user relations with multi-channel hypergraphs, and DHCF learns hybrid high-order correlations.Source: §2 · p. 2

Self-Supervised Graph Learning

• Graph self-supervision provides auxiliary training signals; unlike contrastive-view methods SGL [31] and HCCF [33], SHT uses a generative self-supervised framework for graph-based collaborative filtering.The reviewed approaches include DGI [23] and GMI [18] generative tasks, SGL’s random node and edge dropout, and HCCF’s hypergraph contrastive signals.Source: §2 · pp. 2–3

3 METHODOLOGY

• SHT encodes local structure in node representations, learns global relations with a local-aware hypergraph transformer, and augments parameter learning through local-global cross-view self-augmentation.The overall architecture is shown in Figure 1.Source: §3 · p. 3

3.1 Local Graph Structure Learning

• SHT first represents each user and item with an embedding vector in a d-dimensional latent space to encode interaction patterns.The user and item vectors are aggregated into separate embedding matrices.Source: §3.1 · p. 3
• A two-layer lightweight GCN encodes each node’s neighboring subgraph as a topology-aware embedding, injecting topology positional information into the graph transformer [6] [30].For users, the stated propagation combines two-hop user information with neighboring item embeddings.Source: §3.1 · p. 3
• SHT uses a normalized graph adjacency matrix and residual connections to consider neighboring nodes at different distances; item topology-aware embeddings are computed analogously.The adjacency matrix is normalized using node degrees.Source: §3.1 · p. 3

3.2 Hypergraph Transformer for Global Relation Learning

• SHT combines ID and topology-aware embeddings, then learns K hyperedges as intermediate hubs to capture implicit global collaborative relations among nodes.Its Transformer-like structure learning is intended to alleviate noise and transfer knowledge from dense user/item nodes to sparse ones.Source: §3.2 · p. 3

3.2.1 Node-to-Hyperedge Propagation.

• Node-to-hyperedge propagation uses multi-head query, key, and value attention to connect user nodes with hyperedges, reducing complexity from O(K×I×d/H) to O((I+K)×d^2/H^2).Two hierarchical hypergraph network layers further model nonlinear interactions among hyperedges using learnable hyperedge-wise relations, an activation, and a residual connection.Source: §3.2.1 · pp. 3–4

3.2.2 Hyperedge-to-Node Propagation.

• Hyperedge-to-node propagation reverses the information flow, sharing most attention parameters with node-to-hyperedge propagation while swapping the former query and key roles.The same propagation process is applied analogously to item nodes.Source: §3.2.2 · p. 4

3.2.3 Iterative Hypergraph Propagation.

• Stacking hypergraph transformer layers iteratively propagates global collaborative relations to characterize long-range user-item dependencies, with layer-specific embeddings combined by elementwise summation.SHT predicts interaction probability with the dot product of the final user and item embeddings.Source: §3.2.3 · p. 4

3.3 Local-Global Self-Augmented Learning

• SHT transfers denoised hypergraph features to noisy topology-aware embeddings through self-augmented learning intended to recalibrate the local graph structure and improve robustness.The augmented task distinguishes the solidity—the probability that an observed edge is not noisy—of sampled user-item edges.Source: §3.3 · p. 4

3.3.1 Solidity Labeling with Meta Networks.

• Hypergraph key vectors and hyperedge embeddings inform edge-solidity labels, with a meta network generating user- and item-specific transformations from global hypergraph features.The model reuses key embeddings for relation modeling and incorporates hyperedge embeddings to introduce global characteristics into solidity labeling.Source: §3.3.1 · p. 4

3.3.2 Pair-wise Solidity Ranking.

• SHT trains topology-aware embeddings on sampled edge pairs, using hypergraph solidity labels in an adaptive ranking task that yields smaller gradients for pairs with similar labels.The pairwise label difference encodes both which edge is more solid and the magnitude of the difference.Source: §3.3.2 · p. 5

3.4 Model Learning

• SHT jointly optimizes implicit-feedback prediction and a self-augmented ranking task using sampled positive/negative edge pairs and weighted loss terms.The pairwise objective uses prediction scores for each positive and negative edge, with λ1 and λ2 weighting different loss terms.Source: §3.4 · p. 5
• The objective includes an L2 regularization term for weight decay.The paper denotes this term as F.Source: §3.4 · p. 5
• O(K × (I+J) × d) is reduced to O((I+J+K) × d^2), yielding complexity close to O((I+J) × d^2) when K is much smaller than node count but larger than d.The paper says this gives inference efficiency comparable to typical graph neural architectures; the self-augmented task has the same complexity as the main task.Source: §3.4.1 · p. 5

4 EVALUATION

• The evaluation asks whether SHT outperforms diverse baselines, how its hypergraph and transformer-like components contribute, and how it handles sparse and noisy data and supports useful interpretations.These are the paper’s four research questions (RQ1–RQ4).Source: §4 · p. 5

4.1.1 Experimental Datasets.

• Experiments use Yelp ratings treated as implicit interactions, Gowalla check-ins from 2016–2019, and Tmall page views collected in December 2017.Yelp rated venues count as interacted items; unrated venues count as non-interacted items.Source: §4.1.1 · pp. 5–6
• Datasets are split 7:2:1 into training, validation, and testing, with all-rank evaluation using Recall@N and NDCG@N at N=20 by default.For each test user, positive test items are ranked together with all non-interacted items.Source: §4.1.2 · p. 6

4.1.2 Evaluation Protocols.

• SHT is compared with 15 baselines from different research lines for a comprehensive evaluation.The listed families include factorization, neural factorization, autoencoder, graph neural, disentangled graph, hypergraph, and self-supervised recommenders.Source: §4.1.3 · p. 6

4.1.3 Compared Baseline Methods.

• Examples include BiasMF, NeuMF-style NCF [7], AutoR, and graph recommenders GCMC [1], PinSage [38], NGCF [26], STGCN [40], LightGCN [6], and GCCF [3].The passage describes BiasMF as adding user and item biases, AutoR as a three-layer behavior-reconstruction autoencoder, and the graph models as convolution-based variants.Source: §4.1.3 · p. 6
• Further baselines include DGCF [28], hypergraph models HyRec and DHCF, and self-supervised recommenders MHCN [39], SLRec [37], and SGL [31].HyRec learns item-wise high-order relations; DHCF uses dual-channel hypergraph networks; SGL augments LightGCN with random-walk and feature-dropout views.Source: §4.1.3 · p. 6
• SHT is implemented in TensorFlow and trained with Adam at learning rate 1e-3, 32-dimensional embeddings, and 128 hyperedges by default.The number of graph layers is searched over {1,2,3}; regularization weights, batch size, and dropout rate are tuned over the specified candidate sets.Source: §4.1.4 · p. 6

4.2 Overall Performance Comparison (RQ1)

• SHT’s hypergraph transformer combines global message passing among semantically related users and items with multi-head attention that refines the hypergraph structure.The paper attributes SHT’s performance superiority partly to this global relation learning design.Source: §4.2 · p. 6
• Self-supervised learning improves MHCN, SLRec, and SGL by adding an augmented learning task that regularizes parameter learning from the input data.SHT’s meta-networks generate global-structure-aware mapping functions to adaptively reduce the gap between local and global feature spaces, without relying on random masking that may drop important information.Source: §4.2 · pp. 6–7
• Hypergraph baselines HyRec and DHCF outperform most conventional GNN baselines, while SHT uses learnable latent hyperedges to model dependencies without precomputing a large high-order relation matrix.The paper argues that conventional GNNs are insufficient for capturing high-order and global graph connectivity.Source: §4.2 · p. 6
• SHT achieves the best performance among the baselines on all three datasets under both top-20 and top-40 settings, with t-tests confirming significant improvements.The comparison is reported in Table 2; the supplied passages provide no specific metric values.Source: §4.2 · p. 6, Table 2 · p. 7

4.3 Model Ablation Test (RQ2)

• Removing either topology-aware embeddings (-Pos) or the hypergraph transformer (-Hyper) severely degrades SHT across all cases, supporting both local and global relation learning.The ablations also support the GCN-based topology-aware embedding and hypergraph transformer modules.Source: §4.3 · p. 7, Table 3 · p. 7
• Removing self-augmented learning (-SAL) causes clear degradation across all cases, while the -Meta variant demonstrates the contribution of meta-network-based domain adaptation.SHT transfers knowledge from high-level global hypergraph embeddings to low-level topology-aware embeddings.Source: §4.3 · p. 7
• SHT’s transformer-like hypergraph attention outperforms independent node-hypergraph matrices (-Trans), and deep hyperedge features and high-order iterations each improve performance.The attention variant uses fewer parameters; deep hyperedge layers contribute through nonlinear feature transformation, while high-order iterations improve iterative hypergraph propagation.Source: §4.3 · p. 7, Table 3 · p. 7

4.4.1 Performance w.r.t. Data Noise Degree.

• SHT shows smaller relative performance degradation than MHCN and LightGCN in most noise settings, while degradation is more obvious on the sparsest Gowalla dataset.Noise levels were 5%, 10%, 15%, 20%, and 25% of edges replaced with randomly generated fake edges.Source: §4.4.1 · p. 7, Figure 3 · p. 8

4.4 Model Robustness Test (RQ3)

• The noise test replaces real interaction edges with fake edges at five noise ratios and compares SHT with MHCN and LightGCN using relative performance degradation.MHCN and LightGCN are described as HGNN- and GNN-based recommenders, respectively.Source: §4.4.1 · p. 7

4.4.2 Performance w.r.t. Data Sparsity.

• SHT’s superior performance is consistent across datasets with different user- and item-sparsity degrees, demonstrating robustness to sparse data.Figure 4 reports Recall@40 and NDCG@40 and performance differences on Gowalla.Source: §4.4.2 · p. 8, Figure 4 · p. 8
• Item interaction sparsity affects all methods more than user sparsity, and SHT’s item-side performance gap is larger on middle-sparsity subsets than on the densest subset.The authors attribute the larger middle-subset gap to hypergraph-transformer knowledge transfer between dense and sparse samples.Source: §4.4.2 · p. 8, Figure 4 · p. 8

4.5 Case Study (RQ4)

• The case study tests whether SHT learns useful implicit node-wise relations and distinguishes noisy edges, visualizing three users with fair interaction counts from Tmall.The interacted items are shown as colored circles representing their trained embeddings.Source: §4.5 · p. 8
• Items with the same embedding color show dense implicit category and co-interaction links, supporting SHT’s ability to identify useful relations absent from training.Only view behavior was used for training; other-behavior interactions were unknown to the trained model.Source: §4.5 · p. 8, Figure 5 · p. 9
• Low solidity scores flag weakly connected items, including items 6508, 10202, and 2042, as potential noisy interactions that self-augmentation can discriminate.SHT recalibrates topology-aware embeddings using global information encoded by the hypergraph transformer.Source: §4.5 · pp. 8–9, Figure 5 · p. 9

• SHT uses a hypergraph transformer and self-augmented supervision signals to improve user-item interaction modeling.Source: §5 · p. 9
• SHT improves the robustness of graph-based recommender systems against noise perturbation.Source: §5 · p. 9
• The experiments achieved better recommendation results on real-world datasets.Source: §5 · p. 9
• Future work will extend SHT to explore disentangled user intents and diverse user-item relations for encoding multidimensional user preferences.Source: §5 · p. 9

6.1 Learning process of SHT

• Algorithm 1 takes the interaction graph, graph-layer and edge-sampling counts, maximum epochs, and learning rate, and outputs trained parameters.The listed process draws a user mini-batch, calculates graph topology-aware embeddings, and generates input embeddings for the hypergraph.Source: §6.1 · p. 10
• The learning procedure includes node-to-hyperedge propagation, hierarchical hyperedge feature processing, and propagation from hyperedges back to nodes.Source: §6.1 · p. 10

6.2 Hyperparameter Investigation

• Small d causes 15%-35% performance decreases, while K has at most a 6% decrease versus 35% for d; increasing d can overfit.The study also reports that L=0 significantly degrades graph neural network performance, while stacking more graph layers may cause over-smoothing; on Yelp, d=68 yields only marginal improvement due to over-fitting. K’s smaller effect reflects its O(K×d) parameters versus O((I+J)×d) for d.Source: Figure 6 · p. 10, §6.2 · p. 10

6.3 Vector Visualization Algorithm

• A multilayer perceptron maps 32-dimensional item embeddings to RGB colors, using item-ID prediction to preserve embedding information and a regularizer to favor preferred colors.Source: §6.3 · p. 10