Contrastive Learning for Cold-Start Recommendation

Yinwei Wei; Xiang Wang; Qi Li; Liqiang Nie; Yan Li; Xuanping Li; Tat-Seng Chua

Source-linked AI summary

Contrastive Learning for Cold-Start Recommendation

Yinwei Wei, Xiang Wang, Qi Li, Liqiang Nie, Yan Li, Xuanping Li, Tat-Seng Chua

arXiv:2107.05315v3Jul 12, 2021cs.IRcs.MM

TL;DR

Cold-start items lack interaction histories, so collaborative filtering cannot learn their collaborative embeddings reliably. The paper reformulates representation learning using mutual information, implements the objective with CLCRec’s contrastive framework, and reports significant improvements in warm- and cold-start scenarios.

Problem
Cold-start recommendation concerns predicting preferences for new items without interaction records, which makes collaborative filtering unable to optimize their collaborative embeddings.
Method
CLCRec maximizes a lower bound combining user-item and representation-embedding mutual information through contrastive pair organization, embedding networks, and optimization.
Results
CLCRec achieves significant improvements over state-of-the-art approaches in both warm- and cold-start scenarios across four datasets.
Takeaways & Limitations
The framework preserves informative collaborative signals in content representations for both warm- and cold-start items.
Takeaways & Limitations
Performance can drop when negative pairs exceed 256, suggesting that overusing negative sampling may overfit representation-embedding mutual information.

Abstract

from arXiv · show

Recommending cold-start items is a long-standing and fundamental challenge in recommender systems. Without any historical interaction on cold-start items, CF scheme fails to use collaborative signals to infer user preference on these items. To solve this problem, extensive studies have been conducted to incorporate side information into the CF scheme. Specifically, they employ modern neural network techniques (e.g., dropout, consistency constraint) to discover and exploit the coalition effect of content features and collaborative representations. However, we argue that these works less explore the mutual dependencies between content features and collaborative representations and lack sufficient theoretical supports, thus resulting in unsatisfactory performance. In this work, we reformulate the cold-start item representation learning from an information-theoretic standpoint. It aims to maximize the mutual dependencies between item content and collaborative signals. Specifically, the representation learning is theoretically lower-bounded by the integration of two terms: mutual information between collaborative embeddings of users and items, and mutual information between collaborative embeddings and feature representations of items. To model such a learning process, we devise a new objective function founded upon contrastive learning and develop a simple yet effective Contrastive Learning-based Cold-start Recommendation framework(CLCRec). In particular, CLCRec consists of three components: contrastive pair organization, contrastive embedding, and contrastive optimization modules. It allows us to preserve collaborative signals in the content representations for both warm and cold-start items. Through extensive experiments on four publicly accessible datasets, we observe that CLCRec achieves significant improvements over state-of-the-art approaches in both warm- and cold-start scenarios.

1 INTRODUCTION

Cold-start recommendation is difficult because unseen items lack interaction-derived collaborative embeddings. CLCRec addresses this gap by maximizing mutual information between collaborative and content representations through contrastive learning.

Cold-start items lack historical interactions, preventing collaborative filtering from learning their collaborative embeddings and accurately inferring user preferences.
Existing robustness- and constraint-based methods incorporate item content but insufficiently model mutual dependencies between content features and collaborative embeddings.
CLCRec reformulates feature representation learning around a lower bound combining user-item and representation-embedding mutual information.
CLCRec organizes contrastive pairs, builds contrastive embedding networks, and optimizes them by identifying positive pairs among multiple negatives.Historical interactions form positive user-item pairs, while item self-discrimination supplies representation-feature pairs.
Experiments on four public datasets show significant improvements over state-of-the-art approaches in both warm- and cold-start scenarios.

2 METHODOLOGY

CLCRec reformulates complete cold-start recommendation as preserving collaborative signals in content-based item representations. Its objective maximizes U-I and R-E mutual information through contrastive pair organization, embedding networks, and optimization.

2.1 Problem Formalization: Complete cold-start items have no interaction records, making collaborative filtering unable to optimize their item embeddings directly.The framework targets recommending items without any historical interactions.
2.1 Problem Formalization: Feature representations are learned from item content, but prior methods either separate them from collaborative embeddings or encourage only dimension-wise similarity.CLCRec instead seeks to encode collaborative signals and preserve affinities with interacting users.
2.2 Objective Function: The posterior objective is lower-bounded by U-I mutual information between user and item collaborative embeddings plus R-E mutual information between item collaborative and feature representations.This casts representation learning as mutual-information maximization.
2.2 Objective Function: Contrastive learning estimates these dependencies by identifying positive pairs among negatives, using density-ratio functions within the objective.The two losses are L_RE and L_UI; L_UI has the same optimal objective as BPR, while λ balances collaborative and feature-representation learning.
2.3 CLCRec: CLCRec organizes observed user-item interactions as positive U-I pairs and unobserved interactions as negatives, while R-E pairs match each item with itself positively and other items negatively.These pair constructions support contrastive embedding and mutual-information optimization.
2.3 CLCRec: The shared CF encoder learns user and item collaborative embeddings, while the R-E network aligns item collaborative signals with content-derived feature representations for cold-start inference.At inference, user–cold-start-item affinity is measured from their representations.

3 EXPERIMENTS

Experiments on four real-world datasets compare CLCRec with cold-start, content-based, and BPR loss-based baselines across warm-start, cold-start, and all-item settings. CLCRec improves recommendation performance broadly, while ablations show benefits from contrastive components, negative-pair sampling, and mutual-information weighting, with some trade-offs from excessive sampling.

Experimental Settings: Experiments use Movielens, Tiktok, Kwai, and Amazon, evaluating warm-start, cold-start, and all-item recommendation with recall@10 and NDCG@10.The datasets include item content information and historical user-item interactions.
Performance Comparison: CLCRec outperforms competing methods across warm, cold, and all-item scenarios on both recall@10 and NDCG@10.For cold-start recall@10, improvements over the strongest baseline are 5.64%, 103.95%, 44.71%, and 96.24% across the four datasets.
Performance Comparison: CLCRec’s objective implementations significantly outperform MF-BPR and LightGCN in both cold-start and warm-start conditions.The comparison uses MF-based and GCN-based contrastive embedding networks against BPR loss-based models.
Ablation Study: Using 128 negative pairs substantially improves performance in most cases, while hybrid contrastive training helps cold-start representation learning even at ρ=0.2.On Movielens, increasing sampling and hybrid-training intensity can instead introduce noise and slightly reduce performance.
Ablation Study: Increasing R-E negative pairs from 32 to 256 improves performance, reaching recall@10 of 0.0810 on Movielens, but performance drops beyond 256.The authors attribute the decline to overfitting R-E mutual information and reduced collaborative-embedding expressiveness.
Ablation Study: Increasing λ raises cold-start recall@10 on Movielens from 0.0463 to 0.0730, while warm-start performance changes little in most cases.This supports the role of R-E mutual information in extracting collaborative information from content for cold-start recommendation.

4 RELATED WORK

Prior work incorporates side information into collaborative filtering for cold-start recommendation, while CLCRec treats collaborative embeddings and content features as distinct views and maximizes their mutual information.

Cold-start recommendation methods commonly incorporate item content as a bridge for transferring collaborative signals to items without interaction histories.Examples include images, videos, and attributes integrated into CF-based models.
Contrastive learning identifies positive pairs among negatives and maximizes mutual information between different representations to discover shared semantic information.It has been widely used in computer vision and natural language processing, with InfoNCE providing a probabilistic contrastive loss.
Existing recommender contrastive methods generally optimize representations within either collaborative or content space rather than connecting the two.CLCRec instead treats collaborative embeddings and feature representations as different views and maximizes their mutual information.

5 CONCLUSION AND FUTURE WORK

The paper reformulates complete cold-start representation learning around two mutual-information objectives and develops CLCRec to optimize them with contrastive learning. Experiments across four datasets show large-margin improvements over state-of-the-art baselines in both warm- and cold-start scenarios, while future work extends mutual-information methods to explainable and cross-modal recommendation tasks.

CLCRec reformulates complete cold-start representation learning using lower-bounded user-item and representation-item mutual information objectives.Its framework comprises contrastive pair organization, contrastive embedding network, and contrastive optimization.
CLCRec outperforms state-of-the-art baselines by a large margin in both warm- and cold-start scenarios across four datasets.The experiments verify the method’s effectiveness and efficiency.
The authors identify mutual-information maximization between item feature representations and collaborative embeddings as a first approach to solving the cold-start problem.They propose future applications in explainable recommendation and cross-modal tasks such as video localization and fashion matching.