PaperlayerView exact revision on arXiv

Source-linked AI summary

Socially-Aware Self-Supervised Tri-Training for Recommendation

Junliang Yu, Hongzhi Yin, Min Gao, Xin Xia, Xiangliang Zhang, Nguyen Quoc Viet Hung

arXiv:2106.03569v4cs.IR

TL;DR

Existing self-supervised recommendation methods mainly use augmented views and same-node contrastive signals, overlooking potentially useful supervision from homophilous users. The paper proposes SEPT, which adds social-information views and tri-training encoders to generate cross-user pseudo-labels, and reports consistent gains on multiple real-world datasets. Its scope currently centers on user-generated self-supervision, while item-based pseudo-labels are left for future work.

  • Problem

    Existing SSL recommendation methods use same-node self-discrimination and neglect other-user signals, although homophily may make those users useful supervision.

  • Method

    SEPT augments user views with social information and uses three graph encoders, with two generating pseudo-labels for the recommendation encoder and one another.

  • Results

    Experiments on multiple real-world datasets consistently validate SEPT's effectiveness for improving recommendation.

  • Takeaways & Limitations

    Self-supervised tri-training can exploit positive supervision from other users through complementary views within a recommendation framework.

  • Takeaways & Limitations

    The framework exploits self-supervision signals only from users; informative item pseudo-labels are left for future work.

Abstract

from arXiv · show

Self-supervised learning (SSL), which can automatically generate ground-truth samples from raw data, holds vast potential to improve recommender systems. Most existing SSL-based methods perturb the raw data graph with uniform node/edge dropout to generate new data views and then conduct the self-discrimination based contrastive learning over different views to learn generalizable representations. Under this scheme, only a bijective mapping is built between nodes in two different views, which means that the self-supervision signals from other nodes are being neglected. Due to the widely observed homophily in recommender systems, we argue that the supervisory signals from other nodes are also highly likely to benefit the representation learning for recommendation. To capture these signals, a general socially-aware SSL framework that integrates tri-training is proposed in this paper. Technically, our framework first augments the user data views with the user social information. And then under the regime of tri-training for multi-view encoding, the framework builds three graph encoders (one for recommendation) upon the augmented views and iteratively improves each encoder with self-supervision signals from other users, generated by the other two encoders. Since the tri-training operates on the augmented views of the same data sources for self-supervision signals, we name it self-supervised tri-training. Extensive experiments on multiple real-world datasets consistently validate the effectiveness of the self-supervised tri-training framework for improving recommendation. The code is released at https://github.com/Coder-Yu/QRec.

1 INTRODUCTION

Existing self-supervised recommendation methods learn from augmented graph views but neglect supervisory signals from other users, which may be valuable under user homophily. The paper proposes socially-aware self-supervised tri-training that uses complementary social views and multiple encoders to improve recommendation.

  • Motivation: SSL recommendation methods commonly perturb graphs to create supplementary views and maximize agreement between representations of the same node.Typical augmentations include node or edge dropout and feature shuffling or masking.
  • Motivation: Self-discrimination treats other nodes as negatives, although homophilous users may be similar enough to provide useful positive supervision.Roughly classifying such users as negatives can reduce recommendation performance.
  • Approach: The proposed framework augments user views with social relations to capture homophily from expanding social circles and shared items.Together with the user-item preference view, these relations provide three perspectives on user preferences.
  • Approach: Three graph encoders iteratively generate pseudo-labels for one another, while the preference-view encoder also performs recommendation.The framework unifies recommendation and self-supervised learning and uses positive signals from other users.
  • Evidence: Experiments on multiple real-world datasets report significant recommendation gains and include ablations of the framework's modules.The paper also investigates the effectiveness of each module through a comprehensive ablation study.

2 RELATED WORK

Related work establishes graph neural and self-supervised learning as important tools for recommendation. Existing self-supervised recommendation methods use varied augmentation strategies but are often designed for specific situations and lack broad generalizability.

  • Graph-based recommendation: Graph neural recommendation models exploit high-order user-item neighbors by aggregating neighbor embeddings to refine node representations.Examples include GCMC, NGCF, and LightGCN.
  • Self-supervised learning: Graph contrastive learning creates multiple graph views through stochastic augmentation and contrasts representations across those views.Common augmentations include node or edge dropout, feature shuffling, and random-walk subgraph sampling.
  • Self-supervised recommendation: Self-supervised recommendation studies apply masking, dropout, future-group signals, or sequence perturbations to specific recommendation settings.The cited methods target sequential recommendation, item recommendation, or particular group-based scenarios.
  • Self-supervised recommendation: These existing self-supervised recommendation methods cannot be easily generalized to other scenarios.The limitation is stated for methods designed around specific situations.

3 PROPOSED FRAMEWORK

The paper introduces SEPT, its self-supervised tri-training framework for recommendation, and presents Figure 1 as an overview of the approach.

  • 3 PROPOSED FRAMEWORK: SEPT is the paper's proposed SElf-suPervised Tri-training framework for recommendation.Its overview is illustrated in Figure 1.

3.1 Preliminaries

The preliminaries define the user-item interaction and social graph setting and describe tri-training as a three-classifier method for labeling unlabeled examples through agreement.

  • Data setting: The recommendation setting uses a user-item interaction graph, a user social network, binary interaction matrix R, m users, and n items.An entry r_ui equals 1 when user u has consumed or clicked item i, and 0 otherwise.
  • Tri-training: Tri-training initializes three diverse classifiers and labels an unlabeled example when the other two classifiers agree.The resulting pseudo-label trains the corresponding classifier in the next labeling round.

3.2 Data Augmentation

SEPT augments recommendation data with socially informed views that capture homophily through reliable triangular relations, then perturbs the joint graph to create unlabeled examples for tri-training.

  • Motivation: SEPT uses user social relations to augment recommendation data because social networks can reflect homophily among users.The framework aligns the user-item interaction graph with the social network to obtain supplementary views.
  • Triangle-based augmentation: It retains two triangle types: three mutually connected users and two socially connected users sharing an interacted item.These structures represent friendship expansion and shared-item interests, respectively.
  • Triangle-based augmentation: The two augmented adjacency matrices are A_f=(SS)⊙S and A_s=(RR^⊤)⊙S, computed from social and interaction matrices.Matrix multiplication accumulates shared-friend or shared-item paths, while the Hadamard product restricts relations to existing social links.
  • Views: The resulting preference, friend, and sharing views characterize user preferences from different perspectives.A_f and A_s augment the social and interaction information to support tri-training with SSL.
  • Unlabeled examples: Edge dropout perturbs both the recommendation and social graphs because integrated self-supervision requires perturbing the joint graph.The corrupted graph supplies unlabeled examples through the augmented social information.

3.3 SEPT: Self-Supervised Tri-Training

SEPT performs asymmetric self-supervised tri-training across three graph views: auxiliary encoders generate mutually agreed pseudo-labels, while a recommendation encoder learns from them through neighbor discrimination and joint optimization.

  • Architecture: SEPT builds three graph encoders over the preference, sharing, and friend views, with two auxiliary encoders producing user pseudo-labels.The preference-view encoder also generates recommendations and learns user and item representations.
  • Constructing self-supervision signals: The two auxiliary encoders must agree that a user is positive before that user is labeled for the recommendation encoder.Their predicted probabilities are combined to select high-confidence positive samples.
  • Constructing self-supervision signals: SEPT dynamically reconstructs the perturbed graph and regenerates positive pseudo-labels for each user during every iteration.The evolving labels supervise refinement of the shared bottom representations.
  • Contrastive learning: Neighbor-discrimination pulls each user representation toward pseudo-labeled users and away from unlabeled users in the current view.These positives represent semantic neighbors or potential neighbors identified from the other views.
  • Contrastive learning: Neighbor-discrimination extends self-discrimination by using supervisory signals from other users, while self-discrimination is its one-positive special case.With sufficient positives, both objectives can be adopted simultaneously, and iterative encoder updates refine the pseudo-labels.
  • Optimization: Instead of inserting pseudo-labels into adjacency matrices, SEPT guides representations through a soft mutual-information-based mechanism.This differs from vanilla tri-training, which adds pseudo-labels to the training set for the next round.
  • Optimization: SEPT combines recommendation training with neighbor-discrimination contrastive learning after warming up shared embeddings with the recommendation loss.The self-supervised tri-training then acts as an auxiliary task in joint learning, with β controlling its magnitude.

3.4 Discussions

SEPT differs from social regularization by dynamically refining supervisory signals and separating users that are not semantically positive across views. Its framework combines augmented social and interaction views with iterative pseudo-labeling and joint optimization, while its LightGCN-based implementation adds graph-convolution and labeling costs.

  • Running Process: SEPT augments views, constructs an unlabeled example set, obtains top-K positive examples, and jointly optimizes the overall objective.The listed procedure includes view augmentation, graph construction, positive-example selection, and joint optimization.
  • Connection with Social Regularization: SEPT dynamically improves supervisory signals for uncertain users rather than statically regularizing socially connected users.Social regularization minimizes representation distance between connected users, whereas SEPT iteratively updates neighbor-based supervision.
  • Connection with Social Regularization: Unlike social regularization, SEPT can push users who are not semantically positive in the three views apart.The method therefore uses both positive and negative relational information inferred from the views.
  • Complexity: SEPT is model-agnostic, but the paper implements it with LightGCN-based encoders whose graph-convolution cost is less than 4O(|R|d).The cited complexity discussion also identifies Top-K labeling and neighbor-discrimination contrastive learning as additional costs.

4 EXPERIMENTAL RESULTS

Experiments on Last.fm, Douban-Book, and Yelp evaluate SEPT against general and social recommendation baselines, isolate its components, and test parameter sensitivity. SEPT generally improves recommendation, while view contributions and hyperparameter effects vary across datasets.

  • Overall Performance Comparison: SEPT significantly boosts LightGCN across different layer settings, with maximum improvement reaching 11%.The gains are higher on the sparser Douban-Book and Yelp datasets, and SEPT still outperforms LightGCN under three-layer over-smoothing.
  • Overall Performance Comparison: SEPT achieves more performance gains than MHCN, while outperforming S^2-MHCN on LastFM and Douban-Book but slightly losing on Yelp.SEPT also uses fewer learned parameters and runs much faster than S^2-MHCN in the reported experiments.
  • Self-Discrimination v.s. Neighbor-Discrimination: Both self-discrimination and neighbor-discrimination outperform LightGCN, but their relative importance varies by dataset.They contribute almost equally on LastFM; self-discrimination is more important on Douban-Book, while neighbor-discrimination is more effective on Yelp.
  • View Study: All views contribute on LastFM and Yelp, whereas self-supervised co-training performs best on Douban-Book.Using fewer views slightly reduces performance on LastFM, causes an obvious drop on Yelp, and produces a non-monotonic pattern on Douban-Book; a single view remains better than LightGCN.
  • Parameter Sensitivity Analysis: SEPT is sensitive to β but not to the edge dropout rate ρ, with small β generally desirable and ρ=0.8 still producing informative self-supervision signals.A highly sparse perturbed graph cannot provide useful information for self-supervised learning.

5 CONCLUSION AND FUTURE WORK

SEPT addresses the limits of self-discrimination in recommendation by using socially aware self-supervised tri-training and neighbor-based pseudo-labels. The paper reports effective recommendation improvements, while limiting self-supervision to users and leaving item-based signals for future work.

  • SEPT combines social information, self-supervised tri-training, and neighbor-discrimination contrastive learning to refine user representations.The framework discovers self-supervision from two complementary views of the raw data.
  • Extensive experiments and ablation studies demonstrate SEPT’s effectiveness and examine the rationale of self-supervised tri-training.
  • The framework exploits only users’ self-supervision signals, leaving informative item pseudo-labels as future work.The paper suggests leveraging item multimodality to implement item-based self-supervision.
Loading 2106.03569v4…