PaperlayerView exact revision on arXiv

Source-linked AI summary

Bootstrapping User and Item Representations for One-Class Collaborative Filtering

Dongha Lee, SeongKu Kang, Hyunjun Ju, Chanyoung Park, Hwanjo Yu

arXiv:2105.06323v1cs.LGcs.IR

TL;DR

OCCF aims to find positive but unobserved user-item interactions from sparse implicit feedback, while negative sampling can mislabel such pairs. BUIR learns user and item representations with mutually learning encoders and neighborhood-based stochastic augmentation, without negative sampling. It performs best among the compared OCCF methods, with stronger improvements on sparse datasets, and its representations also support item-category classification.

  • Problem

    OCCF must identify positive but unobserved user-item pairs from sparse implicit feedback, while negative sampling can mistakenly treat such pairs as negative.

    Source: §1 · pp. 1–2

  • Method

    BUIR trains an online encoder to predict target-encoder representations, updates the target encoder by a slow moving average, and stochastically augments interactions using neighborhood information.

    Source: §1 · p. 2, §3.2 · p. 4, §3.4 · pp. 4–5

  • Results

    BUIR consistently performs best among the compared OCCF methods for top-K recommendation, with larger improvements on sparse datasets, and its representations outperform baselines on item-category classification.

    Source: §1 · p. 2, §4.2 · p. 6

  • Takeaways & Limitations

    BUIR's learned representations also support the downstream task of classifying items into categories.

    Source: §1 · p. 2

    Generative OCCF methods implicitly assume all unobserved user-item pairs are negative, an assumption the paper says is not strictly valid and leads to limited performance.

    Source: §2.1 · p. 3

Abstract

from arXiv · show

The goal of one-class collaborative filtering (OCCF) is to identify the user-item pairs that are positively-related but have not been interacted yet, where only a small portion of positive user-item interactions (e.g., users' implicit feedback) are observed. For discriminative modeling between positive and negative interactions, most previous work relied on negative sampling to some extent, which refers to considering unobserved user-item pairs as negative, as actual negative ones are unknown. However, the negative sampling scheme has critical limitations because it may choose "positive but unobserved" pairs as negative. This paper proposes a novel OCCF framework, named as BUIR, which does not require negative sampling. To make the representations of positively-related users and items similar to each other while avoiding a collapsed solution, BUIR adopts two distinct encoder networks that learn from each other; the first encoder is trained to predict the output of the second encoder as its target, while the second encoder provides the consistent targets by slowly approximating the first encoder. In addition, BUIR effectively alleviates the data sparsity issue of OCCF, by applying stochastic data augmentation to encoder inputs. Based on the neighborhood information of users and items, BUIR randomly generates the augmented views of each positive interaction each time it encodes, then further trains the model by this self-supervision. Our extensive experiments demonstrate that BUIR consistently and significantly outperforms all baseline methods by a large margin especially for much sparse datasets in which any assumptions about negative interactions are less valid.

1 INTRODUCTION

OCCF must identify unobserved positive interactions from sparse implicit feedback, while negative-sampling methods risk treating positive-but-unobserved pairs as negative. BUIR avoids negative sampling by bootstrapping user and item representations with two encoders and uses neighborhood-based augmentation to expose varied views of positive interactions; evaluations report strong recommendation results, especially on sparse datasets.

1 INTRODUCTION

  • OCCF seeks likely positive user-item interactions among many unobserved pairs, but sparse implicit feedback and missing negative labels make this difficult.Negative-sampling methods treat unobserved pairs as negative, although some are positive but unobserved; this uncertainty grows with sparsity and can degrade top-K recommendation.Source: §1 · pp. 1–2
  • BUIR bootstraps representations with online and target encoders: the online encoder predicts target outputs, while the target encoder slowly tracks the online encoder.The target encoder's moving-average update supplies consistent targets and is intended to prevent collapse without negative interactions.Source: §1 · p. 2
  • BUIR stochastically augments positive interactions using user and item neighborhoods, so encoders learn varied views without manually adding interactions.Using random subsets of neighbors as input features produces an effect similar to increasing the number of positive pairs.Source: §1 · p. 2
  • BUIR consistently performs best for top-K recommendation, with larger improvements on sparser datasets, and its representations also perform better on item-category classification.The reported gains are associated with augmented positive-interaction views and avoiding uncertain negative interactions.Source: §1 · p. 2

2 RELATED WORK

OCCF methods learn from positive-only interaction data, but many approaches use unobserved pairs as negatives. Related work also covers self-supervised contrastive learning, which uses negative instances, and bootstrapping methods designed to avoid them.

2.1 One-Class Collaborative Filtering

  • Discriminative OCCF methods [11] [32] learn to separate positive and negative interaction scores, often using pairwise ranking and unobserved pairs as negatives.Their convergence and final performance can depend on the negative-sampling distribution.Source: §2.1 · p. 2
  • Generative OCCF methods [18] [31] use VAE or GAN architectures to reconstruct or generate user interaction vectors, but implicitly treat all unobserved pairs as negative.The paper notes this assumption is not strictly valid and can limit performance.Source: §2.1 · pp. 2–3

2.2 Self-supervised Contrastive Learning

  • Contrastive self-supervised learning creates positive pairs through data augmentation and uses other instances as negatives in its NCE loss [23].Negative pairs are used to prevent representations from collapsing to the same values.Source: §2.2 · p. 3
  • Bootstrapping-based self-supervised methods [3] [6] use two networks that learn from each other to avoid collapse without negative instances.The cited approach reports strong downstream-task performance and robustness to data-augmentation choices in computer vision.Source: §2.2 · p. 3

3 BUIR: PROPOSED FRAMEWORK

BUIR learns user and item representations without assumptions about negative interactions, using online–target bootstrapping and neighborhood-based stochastic augmentation. Its framework defines interaction scores for top-K recommendation and augments observed interactions with perturbed views.

3.1 Problem Formulation

  • OCCF estimates preference scores from observed interactions and uses those scores to rank each user’s top-K items.The framework learns low-dimensional user and item representations to capture preferences.Source: §3.1 · p. 3

3.2 Bootstrapping the Representations

  • BUIR cross-predicts user and item vectors through an online encoder against a momentum-updated target, avoiding direct same-encoder matching that the paper says risks collapse.The target encoder slowly approximates the online encoder, providing enhanced but consistent targets.Source: §3.2 · pp. 3–4

3.3 Top-K Preferred Item Prediction

  • For top-K inference, BUIR sums symmetric cross-prediction scores; its predictor models higher-level user-item relationships, and scoring uses the online encoder alone.The target encoder is discarded after training because the online and target encoders converge toward equilibrium.Source: §3.3 · p. 4

3.4 Neighbor-based Data Augmentation

  • Neighbor-based encoders combine user or item identities with sets of interacted neighbors, whose shared parameters also receive supervision as neighbors.During training, BUIR stochastically perturbs identity-neighbor inputs into augmented views, extending self-supervision for sparse data.Source: §3.4 · pp. 4–5

4 EXPERIMENTS

The experiments assess BUIR against OCCF methods for top-K recommendation and examine its components, learned representations, and hyperparameters.

4 EXPERIMENTS

  • The evaluation compares BUIR with other OCCF methods for top-K recommendation and validates components through ablation studies.Source: §4 · p. 5
  • The experiments evaluate the quality of the learned representations on a downstream task.Source: §4 · p. 5
  • The experiments also analyze the model’s hyperparameters.Source: §4 · p. 5

4.1 Experimental Settings

The experiments compare two BUIR encoder variants with OCCF baselines on three real-world datasets, using user-level splits and ranking metrics.

4.1 Experimental Settings

  • Experiments use CiteULike, Ciao, and FourSquare, with user and item interaction-count filters applied during preprocessing.The minimum user and item counts vary by dataset.Source: §4.1 · p. 5
  • Baselines included BPR, M-VAE [18], and CFGAN, among OCCF methods grouped by whether they used only user/item IDs or additionally used neighborhood information.M-VAE and CFGAN are generative methods.Source: §4.1 · p. 5
  • Each user’s interactions are split into training, validation, and test sets, with training proportions β∈{10%, 20%, 50%} and results averaged over five random-seed runs.The remaining interactions are divided equally between validation and test sets.Source: §4.1 · p. 6
  • Performance is measured with P@K and N@K, while models are trained with Adam and tuned by validation-set grid search with early stopping.Early stopping uses P@10, terminating after 50 successive epochs without improvement.Source: §4.1 · p. 6

4.2 Comparison with OCCF Methods

BUIR performs best among the evaluated OCCF methods, with larger gains in sparser settings. Its ID-only and neighborhood-based variants also outperform relevant competitors, while avoiding dependence on negative-sampling choices.

4.2.3 Comparison of different negative sampling strategies.

  • BUIRid consistently performs best across negative-sampling strategies, while discriminative methods’ performance depends substantially on the strategy used.The strategies show different tendencies and optimal settings by dataset and method; Figure 3 presents the comparison.Source: Figure 3 · p. 6, §4.2.3 · p. 8

4.2.1 Effectiveness of BUIRid.

  • BUIRid outperforms ID-only discriminative baselines on every dataset, and its advantage over the best baseline grows as training data becomes sparser.Its use of only positive interactions avoids inconsistent supervision from uncertain negative interactions.Source: §4.2.1 · p. 6
  • BUIR shows larger gains on P@10 and N@10 than on P@50 and N@50, indicating stronger improvement for top-ranked recommendations.The paper describes this as practically advantageous for systems aiming to provide users their most preferred items.Source: §4.2.1 · p. 6

4.2 Comparison with OCCF Methods

  • BUIR achieves the best top-K recommendation performance among the baselines, with especially significant improvements on highly sparse datasets.Table 2 reports results across dataset sparsity levels and defines improvements over the best ID-based and neighbor-based baselines.Source: §4.2 · p. 6, Table 2 · p. 7

4.2.2 Effectiveness of BUIRnb.

  • BUIRnb significantly outperforms neighborhood-based competitors; BUIR’s framework is effective across information sources without assuming unobserved interactions are negative.The passage also notes that most neighborhood baselines, except LGCN, perform worse than BUIRid; BUIR avoids the advanced stability techniques discussed for generative methods.Source: §4.2.2 · p. 8

4.3 Ablation Study

The ablation study distinguishes the contribution of BUIR’s learning framework from its score function, neighborhood encoder, and stochastic augmentation.

4.3 Ablation Study

  • BUIR’s cross-prediction score is less effective than the full approach and even conventional BPR’s inner-product score, pointing to the learning framework rather than predictor-based score modeling as the main source of improvement.The compared scores are q(f(u))⊤f(v) + f(u)⊤q(f(v)) and f(u)⊤f(v), respectively.Source: §4.3 · p. 8
  • Without stochastic augmentation, an LGCN encoder trained within BUIR still beats LGCN trained with BPR, supporting the benefit of BUIR’s framework under uncertain negative sampling.Source: §4.3 · p. 8
  • Stochastic neighbor augmentation further improves BUIR by supplying varied views of positive user-item interactions for optimization.Source: §4.3 · p. 8

4.4 Effect of Neighbor Augmentation

Stochastic neighbor augmentation improves recommendation over fixed encoder neighborhoods and helps BUIR address OCCF sparsity. Its benefit grows with perturbation on the less-sparse Ciao dataset.

4.4 Effect of Neighbor Augmentation

  • The analysis varies the maximum drop probability P from 0.0 to 1.0, drawing each drop probability uniformly from [0, P].It evaluates BUIRnb on CiteULike and Ciao at β = 20%.Source: §4.4 · p. 8
  • Stochastic augmentation significantly improves BUIR over fixed neighborhood inputs and helps it learn preferences even on much sparser datasets.The comparison is between P > 0 and P = 0.Source: §4.4 · pp. 8–9
  • On Ciao, augmentation's benefit increases with maximum drop probability because more neighbors allow more varied views that boost recommendation performance.Ciao is less sparse than CiteULike.Source: §4.4 · p. 9

4.5 Evaluation on Representation Quality

BUIR's learned item representations perform strongly on item classification using linear and nonlinear classifiers. Their method rankings align with top-K recommendation rankings.

4.5 Evaluation on Representation Quality

  • The evaluation uses fixed item representations to predict item labels with linear and nonlinear classifiers, reporting 10-fold cross-validation averages and standard deviations.Ciao supplies 28 product categories; FourSquare item coordinates are clustered into 100 labels.Source: §4.5 · p. 9
  • BUIRid and BUIRnb achieve significantly higher item-classification accuracy than other methods in each category.The paper interprets this as evidence that BUIR representations capture item characteristics or relationships more accurately than baseline representations.Source: §4.5 · p. 9
  • Method rankings on item classification match those on top-K recommendation, which the paper links to positive correlations between observed interactions and item semantics.The paper states that effectively learning users' implicit feedback also results in good downstream-task performance.Source: §4.5 · p. 9

4.6 Sensitivity Analysis

BUIR's performance is largely insensitive to momentum coefficients above 0.9, but a fixed target encoder fails to learn effective representations. A single-layer predictor performs best.

4.6 Sensitivity Analysis

  • The sensitivity analysis examines representation dimension, momentum coefficient τ, and predictor depth on FourSquare at β = 50%.The paper concludes that BUIR is powerful with fewer hyperparameters than OCCF methods using varied regularization terms or modeling components.Source: §4.6 · p. 9
  • For τ in [0.9, 1.0), BUIR performance is hardly affected; however, when the target encoder is fixed at τ = 1, BUIR cannot learn effective representations at all.Source: §4.6 · p. 9
  • BUIR performs best with a single-layer predictor; multiple layers make the relationship between the two encoder outputs harder to optimize.Source: §4.6 · p. 9

5 CONCLUSION

BUIR learns user and item representations from positive interactions without negative sampling, using cross-prediction and augmented neighborhood views. It outperforms OCCF baselines on top-K recommendation, especially on sparse datasets.

5 CONCLUSION

  • BUIR consistently outperforms other OCCF baselines on top-K recommendation, with larger gains on much sparser datasets.The conclusion attributes this scope to sparse positives and less-valid assumptions about negative interactions.Source: §5 · p. 9
  • BUIR bootstraps user and item representations by minimizing cross-prediction error using only partially observed positive interactions, eliminating negative sampling.Source: §5 · p. 9
  • BUIR learns augmented views of positive interactions from neighborhood information to further relieve OCCF data sparsity.Source: §5 · p. 9
  • The authors expect BUIR's compatibility with existing user and item encoders to make it a major OCCF solution replacing conventional BPR.Source: §5 · p. 9
Loading 2105.06323v1…