PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-view Multi-behavior Contrastive Learning in Recommendation

Yiqing Wu, Ruobing Xie, Yongchun Zhu, Xiang Ao, Xin Chen, Xu Zhang, Fuzhen Zhuang, Leyu Lin, Qing He

arXiv:2203.10576v1cs.IRcs.AIcs.LG

TL;DR

Multi-behavior recommendation seeks to use multiple behaviors to improve target-behavior prediction, while modeling behavior commonalities and differences across sequence and graph views. MMCLR introduces three contrastive-learning tasks for these relationships. The paper reports that it outperforms SOTA baselines on all datasets and metrics in evaluations on two real-world datasets.

  • Problem

    Multi-behavior recommendation must capture commonalities and fine-grained differences across behaviors while considering individual sequence and global graph views.

    Source: §1 · p. 2

  • Method

    MMCLR combines sequence and graph modules with multi-behavior, multi-view, and behavior-distinction contrastive-learning tasks.

    Source: §1 · p. 3

  • Results

    MMCLR outperforms SOTA baselines on all datasets and metrics, with evaluations and ablation tests on two real-world datasets.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The paper concludes that its framework and contrastive-learning tasks are effective, with cold-start-user performance further demonstrating its cold-start capability.

    Source: §5 · p. 14

Abstract

from arXiv · show

Multi-behavior recommendation (MBR) aims to jointly consider multiple behaviors to improve the target behavior's performance. We argue that MBR models should: (1) model the coarse-grained commonalities between different behaviors of a user, (2) consider both individual sequence view and global graph view in multi-behavior modeling, and (3) capture the fine-grained differences between multiple behaviors of a user. In this work, we propose a novel Multi-behavior Multi-view Contrastive Learning Recommendation (MMCLR) framework, including three new CL tasks to solve the above challenges, respectively. The multi-behavior CL aims to make different user single-behavior representations of the same user in each view to be similar. The multi-view CL attempts to bridge the gap between a user's sequence-view and graph-view representations. The behavior distinction CL focuses on modeling fine-grained differences of different behaviors. In experiments, we conduct extensive evaluations and ablation tests to verify the effectiveness of MMCLR and various CL tasks on two real-world datasets, achieving SOTA performance over existing baselines. Our code will be available on \url{https://github.com/wyqing20/MMCLR}

1 Introduction

Multi-behavior recommendation uses auxiliary behaviors to improve target-behavior prediction, but must capture behavior commonalities and differences while combining individual sequence and global graph views. MMCLR addresses these challenges with three contrastive-learning tasks and outperforms SOTA baselines on all datasets and metrics.

1 Introduction

  • Single-behavior recommendation can suffer from data sparsity and cold-start problems, especially for high-cost, low-frequency target behaviors.Other behaviors such as clicks and add-to-cart interactions can provide additional information about users.Source: §1 · pp. 1–2
  • Existing MBR models face challenges in extracting behavior commonalities, jointly modeling sequence and graph views, and distinguishing behavior-specific preferences.The paper notes that click interactions can substantially outnumber purchases, while purchase may have higher priority than click.Source: §1 · p. 2
  • MMCLR combines sequence and graph modules with multi-behavior, multi-view, and behavior-distinction contrastive-learning tasks.The tasks target coarse-grained commonalities across behaviors and views, and fine-grained behavioral differences.Source: §1 · p. 3
  • MMCLR outperforms SOTA baselines on all datasets and metrics, and its contrastive-learning tasks and cold-start capability are verified.The paper reports significant improvements over competitive baselines and ablation versions on real-world MBR datasets.Source: §1 · p. 3

2 Related Work

Prior recommendation work models sequential behavior or graph interactions, while MBR uses auxiliary behaviors to support target-behavior prediction. The paper identifies limited use of contrastive learning to model relations across behaviors as a gap.

Sequence-based & Graph-based Recommendation

  • Sequence-based methods focus on users’ sequential interests, while graph-based methods model global, high-order interaction information.The paper cites RNNs, memory networks, attention, and mixed models as sequence-based approaches, and GNNs for graph-based recommendation.Source: §2 · pp. 3–4

Multi-behavior Recommendation

  • MBR uses users’ other behaviors to help predict a target behavior, with prior methods using sequence, graph, meta-learning, or external-knowledge approaches.Examples include collective matrix factorization, sequential graphs, and user-item or co-behavior graphs.Source: §2 · p. 4

Self-supervised Learning

  • The paper frames contrastive learning as a way to model commonalities and differences across users’ behaviors and views, unlike much prior self-supervised recommendation work focused on single behavior.The related-work discussion also situates self-supervised learning across computer vision, NLP, graphs, and recommender systems.Source: §2 · p. 4

3 Methodology

MMCLR predicts a user’s target-behavior interaction from multi-behavior sequences and a global multi-relation user-item graph. Its encoders and fusion modules form sequence- and graph-view representations and apply contrastive-learning tasks across behaviors and views.

3.1 Preliminaries

  • The task is to predict the item a user will interact with under the target behavior from the user’s multi-behavior sequences and global multi-relation graph.The preliminaries define multi-behavior modeling around a target behavior and construct sequence and graph views.Source: §3.1 · pp. 4–5

Overview.

  • Sequence and graph encoders produce behavior-specific user representations, which are fused within views and then across views alongside three contrastive-learning tasks.The final fused user-item similarity serves as the ranking score.Source: Overview · p. 5, Figure 1 · p. 5

Multi-view Encoder

  • The multi-view encoder learns separate single-behavior user and item representations from individual behavior sequences and the global user-item graph.The paper uses Bert4rec and LightGCN as its sequence and graph encoders, respectively, while noting that encoder choices are flexible.Source: Multi-view Encoder · pp. 5–6

Multi-behavior Fusion.

MMCLR fuses single-behavior representations within each view, then combines sequence and graph views to form final user and item representations.

Multi-behavior Fusion.

  • Auxiliary behaviors can supply information for sparse target behaviors, so MMCLR fuses single-behavior representations into integrated sequence-view and graph-view representations.The sequence and graph fusion modules produce separate integrated representations.Source: Multi-behavior Fusion · p. 6

Multi-view Fusion.

  • Multi-view fusion combines individual sequence and global graph information in the final user and item representations.The integrated representations are used to calculate user-item ranking scores.Source: Multi-view Fusion · p. 6
  • The ranking objective treats target-behavior interactions as positives and randomly sampled items as negatives.The paper specifies these positive and negative sets for the ranking loss.Source: Multi-view Fusion · p. 6

3.3 Multi-behavior Contrastive Learning

Sequential multi-behavior contrastive learning brings same-user representations from different behaviors closer while separating representations of different users.

3.3 Multi-behavior Contrastive Learning

  • Multi-behavior CL uses separate sequence- and graph-view auxiliary losses to exploit commonalities among a user's behavior representations.The approach assumes the user's single-behavior representations share information about that user.Source: §3.3 · p. 7

Sequential Multi-behavior CL.

  • Sequential multi-behavior CL treats two different behavior representations of one user as a positive pair and another user's representation as a negative.It minimizes differences within the positive pair and maximizes differences between users.Source: Sequential Multi-behavior CL · p. 7
  • A projector MLP maps single-behavior representations into a shared sequential semantic space for the contrastive loss.The loss uses a pair-wise distance function and sigmoid activation.Source: Sequential Multi-behavior CL · p. 7

Graphic Multi-behavior CL.

Graphic multi-behavior CL applies the multi-behavior contrastive objective to graph-view representations, supporting more robust representations, particularly for sparse target behaviors.

Graphic Multi-behavior CL.

  • Graphic multi-behavior CL applies the same contrastive-learning design to graph-view representations.The loss is defined for graph-view representations.Source: Graphic Multi-behavior CL · p. 7
  • The graph-view task pairs different behavior representations of the same user positively and uses another user's representation as a negative.The positive and negative samples follow the sequential task's contrastive setup.Source: Graphic Multi-behavior CL · p. 7
  • Sequential and graphic multi-behavior CL learn more robust single-behavior representations, especially when target behaviors are sparse.The paper identifies these representations as a foundation for users' diverse preferences.Source: Graphic Multi-behavior CL · p. 7

3.4 Multi-view Contrastive Learning

Multi-view CL brings a user's sequence-view and graph-view representations together, aiming to let the two views cooperate in multi-behavior recommendation.

3.4 Multi-view Contrastive Learning

  • Multi-view CL treats the same user's sequence-view and graph-view representations as a positive pair and other users' representations as negatives.The representations are treated as view-level augmentations of the user.Source: §3.4 · pp. 7–8
  • The task aims to connect individual sequence and global graph views so they can cooperate in multi-behavior recommendation.The two views reflect the same user's preferences while learning from different information.Source: §3.4 · pp. 7–8

3.5 Behavior Distinction Contrastive Learning

Behavior distinction contrastive learning preserves the priority of target behaviors over auxiliary and random items, capturing fine-grained behavioral differences alongside shared preferences.

3.5 Behavior Distinction Contrastive Learning

  • MMCLR orders target-behavior items above auxiliary-behavior hard negatives and random in-batch items in the integrated user representation.The representation should be closest to target items, then auxiliary-behavior items, and most distinct from random negatives.Source: §3.5 · p. 8
  • The item-aspect distinction loss uses a projector MLP to form user and item representations for the target, auxiliary, and random-item behavior levels.The loss includes a β-weighted term; β is a loss weight, and vi and vj are target or auxiliary behaviors of user u.Source: §3.5 · p. 8
  • The distinction task models fine-grained behavior differences and priorities to better represent target-behavior preferences.Auxiliary behaviors such as clicks can act as hard negatives for high-cost target behaviors such as purchases.Source: §3.5 · p. 8

3.6 Optimization

MMCLR combines the supervised objective with four contrastive-learning losses. Its contrastive training adds no encoding time, and the losses are computed offline, leaving online serving complexity equal to other models.

3.6 Optimization

  • The total objective weights the supervised loss and four contrastive losses: sequential, graph, multi-view, and behavior-distinction.The loss is defined using hyperparameters λ.Source: §3.6 · p. 8
  • Parallel graph and sequential encoders make encoding complexity depend on the more complex encoder, so MMCLR adds no encoding time.Contrastive MLP training is O(|U|d^2), and contrastive learning is O(|U|Nd), where |U| is the number of users and N is batch size.Source: §3.6 · p. 9
  • Contrastive losses are computed offline, so online serving complexity is equal to that of other models.The stated contrastive-learning complexity is equal to existing CL models and can run in parallel with fusion operations.Source: §3.6 · p. 9

4 Experiments

The experiments evaluate MMCLR on two e-commerce multi-behavior datasets using leave-one-out ranking metrics and comparisons with state-of-the-art baselines and ablations. MMCLR leads on the reported benchmarks, while ablations, cold-start evaluation, and parameter analyses examine its components and settings.

4.1 Datasets

  • The evaluation uses Tmall and CIKM2019 EComm AI, each containing click, add-to-cart, and purchase behaviors.The datasets contain 22,014 and 23,032 users, respectively; evaluation uses top-K HIT, NDCG, MRR, and AUC under leave-one-out with 99 sampled negative items per ground truth.Source: §4.1 · p. 9, §4.3 · p. 10

4.2 Competitors

  • Competitors include sequence-based, graph-based, and multiplex-graph MBR models, plus single-view CL variants and multi-view fusion ablations.For non-MBR baselines, MMCLR's fusion function is used to jointly consider multiple behaviors.Source: §4.2 · pp. 9–10

4.3 Experimental Settings

  • All methods use 64-dimensional user and item embeddings, and the evaluation follows leave-one-out with HIT, NDCG, MRR, and AUC.For each ground truth, 99 un-interacted items are randomly sampled as negatives; MMCLR's learning rate is 1e−3 and its L2 coefficient is 1e−4.Source: §4.3 · p. 10

4.4 Results of Multi-behavior Recommendation (RQ1)

  • MMCLR ranks best among baselines and its ablations on every metric across both datasets, improving 4%–11.8% over the best baselines on most metrics with p < 0.05.The comparison is reported in Table 1; significance is based on paired t-tests against baselines.Source: §4.4 · p. 10, Table 1 · p. 11

4.5 Ablation Study (RQ2)

  • The ablation study reports benefits from sequential and graph multi-behavior CL, multi-view CL, and behavior-distinction CL.BCL improves most metrics by over 2%, VCL by nearly 1% on most metrics, and DCL by 0.6–1.4%; Table 2 reports the ablations.Source: §4.5 · p. 12, Table 2 · p. 12

4.6 Results on Cold-start Scenarios (RQ3)

  • On Tmall, MMCLR is compared with baselines and ablations for overall users and cold-start users defined as having fewer than three target behaviors in training.Fig. 2(a) presents NDCG@10 for both groups; MMCLR improves consistently from graph+BCL to the full model.Source: §4.6 · pp. 12–13, Figure 2 · p. 13

4.7 Parameter Analyses (RQ4)

  • For the main-task loss weight on Tmall, HIT@10 and NDCG@10 peak at λo = 1.0, and MMCLR outperforms baselines across tested weights.The tested supervised-loss weights are 0.2, 1, 2, 4, and 8, with contrastive-loss weights held at 0.2, 0.2, 0.2, and 0.05.Source: §4.7 · p. 13, Figure 3 · p. 14
  • Embedding dimensions from 16 to 128 improve performance as dimension increases, while 256 performs worse than 128.The authors suggest the larger dimension may suffer from overfitting and state that dimensions that large are unnecessary.Source: §4.7 · p. 14, Figure 3 · p. 14

5 Conclusion

The paper presents MMCLR as a framework for multi-behavior recommendation that jointly models behavior and view commonalities and differences through three contrastive-learning tasks.

5 Conclusion

  • MMCLR jointly models commonalities and differences between behaviors and views through three contrastive-learning tasks.The framework is proposed to address sparsity in target behaviors.Source: §5 · p. 14
  • Experiments support the effectiveness of MMCLR and its contrastive-learning tasks.Source: §5 · p. 14
  • Results on cold-start users further support MMCLR's performance on the cold-start problem.Source: §5 · p. 14

Related papers

Loading 2203.10576v1…