PaperlayerView exact revision on arXiv

Source-linked AI summary

Contrastive Meta Learning with Behavior Multiplicity for Recommendation

Wei Wei, Chao Huang, Lianghao Xia, Yong Xu, Jiashu Zhao, Dawei Yin

arXiv:2202.08523v1cs.IRcs.AIcs.LG

TL;DR

Multi-behavior recommendation must address sparse target-behavior supervision and personalized dependencies across heterogeneous user actions. CML combines cross-type contrastive learning with a contrastive meta network, and experiments on three real-world datasets show consistent improvements over state-of-the-art methods. The study concludes that contrastive meta learning has potential for capturing behavior multiplicity in recommendation.

  • Problem

    Existing recommendation models inadequately capture multiplex user-item relationships, while target-behavior labels are sparse and personalized cross-behavior dependencies remain underexplored.

  • Method

    CML distills transferable cross-type behavior knowledge through contrastive learning and uses meta contrastive encoding to customize behavior dependencies for different users.

  • Results

    CML consistently outperforms various state-of-the-art recommendation methods across three real-world datasets and various settings.

  • Takeaways & Limitations

    Contrastive meta learning offers potential for capturing diverse and personalized multi-behavior patterns in recommendation.

Abstract

from arXiv · show

A well-informed recommendation framework could not only help users identify their interested items, but also benefit the revenue of various online platforms (e.g., e-commerce, social media). Traditional recommendation models usually assume that only a single type of interaction exists between user and item, and fail to model the multiplex user-item relationships from multi-typed user behavior data, such as page view, add-to-favourite and purchase. While some recent studies propose to capture the dependencies across different types of behaviors, two important challenges have been less explored: i) Dealing with the sparse supervision signal under target behaviors (e.g., purchase). ii) Capturing the personalized multi-behavior patterns with customized dependency modeling. To tackle the above challenges, we devise a new model CML, Contrastive Meta Learning (CML), to maintain dedicated cross-type behavior dependency for different users. In particular, we propose a multi-behavior contrastive learning framework to distill transferable knowledge across different types of behaviors via the constructed contrastive loss. In addition, to capture the diverse multi-behavior patterns, we design a contrastive meta network to encode the customized behavior heterogeneity for different users. Extensive experiments on three real-world datasets indicate that our method consistently outperforms various state-of-the-art recommendation methods. Our empirical studies further suggest that the contrastive meta learning paradigm offers great potential for capturing the behavior multiplicity in recommendation. We release our model implementation at: https://github.com/weiwei1206/CML.git.

1 INTRODUCTION

Recommendation systems must model multiplex user-item behaviors, but target-behavior labels are sparse and users exhibit personalized behavior dependencies. CML addresses these challenges through contrastive learning and meta-based customization, with consistent improvements across datasets and settings.

  • Recommendation models often assume a single interaction type, despite complementary behaviors such as page views, favourites, and purchases.
  • Sparse target-behavior labels, especially for purchases, limit supervised training of multi-behavior recommender systems.
  • CML uses multi-behavior contrastive learning to distill transferable cross-type interaction knowledge and augment sparse target-behavior supervision.
  • CML uses meta contrastive encoding to preserve personalized multi-behavior characteristics and customized behavior dependencies.
  • Across three real-world datasets and 12 baselines, CML consistently improves performance under various settings, with ablations supporting its sub-modules.

2 PRELIMINARY

The paper represents recommendation data as multiple user-item interaction matrices, distinguishes target from auxiliary behaviors, and formulates target-behavior prediction over a multi-behavior interaction graph.

  • Multi-behavior data consists of K user-item interaction matrices, one for each behavior type.
  • Target behaviors define the prediction objective, while other behaviors provide auxiliary interaction information.
  • Purchases are commonly treated as target behaviors because they are directly related to Gross Merchandise Value in e-commerce.
  • The task is to estimate the likelihood that a user will interact with an item under the target behavior type.

3 METHODOLOGY

CML is a self-supervised neural architecture that integrates customized meta learning for personalized multi-behavior dependency modeling.

  • CML encapsulates customized meta learning into a self-supervised neural architecture.
  • The framework is designed to model personalized dependencies across multiple user behaviors.
  • Figure 1 presents the overall model flow, with the framework's key components described in subsequent subsections.

3.1 Behavior-aware Graph Neural Network

The behavior-aware graph neural network propagates information over a multi-behavior interaction graph and aggregates behavior-specific representations across users and items.

  • CML performs behavior-aware message passing over a multi-behavior user-item interaction graph.
  • The graph architecture injects high-order connectivity into multiplex relation learning across users and items.
  • The model constructs contrastive views between auxiliary and target behavior embeddings after graph-based representation learning.
  • The meta contrastive encoder captures customized cross-type behavior dependencies using a meta weight network.
  • Behavior-specific interaction patterns are aggregated across behavior types for user representations, with analogous aggregation on the item side.

3.2 Multi-Behavior Contrastive learning

CML treats each behavior type as a separate view and uses contrastive learning to transfer supervision across target and auxiliary behaviors. It contrasts same-user behavior views as positives and different users as negatives, supporting target-behavior prediction under sparse supervision.

  • Contrastive View Generation: The framework supplies auxiliary supervision signals to the supervised target-behavior prediction task.It is designed to capture dependencies across different interaction types while supplementing sparse target-behavior supervision.
  • Contrastive View Generation: CML treats each behavior type as an individual view for contrastive learning between user embeddings.This design constructs representations from different behavior views for comparison.
  • Behavior-Wise Contrastive Learning: Different behavior views of the same user form positive pairs, while views of different users form negative pairs.The resulting contrastive objective recognizes a user across behavior views and discriminates behavior embeddings across users.
  • Behavior-Wise Contrastive Learning: CML uses InfoNCE loss to maximize mutual information between user representations from contrasting behavior views.The loss compares positive pairs with sampled negative counterparts using an embedding similarity function and temperature parameter.
  • Behavior-Wise Contrastive Learning: The model generates contrastive loss functions for each target-behavior and auxiliary-behavior pair.These pair-specific losses are collected into a multi-behavior contrastive objective.

3.3 Meta Contrastive Encoding

CML uses meta contrastive encoding to model user-specific dependencies among behavior types and customize the integration of contrastive losses. A meta-knowledge encoder feeds a weighting function that produces personalized loss weights for self-supervised and recommendation objectives.

  • Motivation: Users exhibit diverse behavior patterns and item interaction preferences, motivating personalized cross-behavior dependency modeling.For example, users may differ in how strongly adding items to favourites relates to later purchases.
  • Meta Contrastive Encoding: The meta contrastive encoding scheme learns an explicit weighting function for integrating multi-behavior contrastive losses.Its two phases first extract personalized meta-knowledge and then incorporate it into a meta weight network.
  • Meta-Knowledge Encoder: The meta-knowledge encoder preserves user-specific behavior dependencies and interaction context from learned behavior representations.Its integration design retains both auxiliary-target behavior dependency and user-specific interaction context.
  • Meta Weight Network: The weighting function maps meta-knowledge to contrastive loss weights through a transformation layer.The function is designed to customize multi-behavior relationships according to personalized behavior intentions.
  • Meta Weight Network: The resulting meta weights represent customized dependence between target and auxiliary behavior types and weight both InfoNCE and BPR objectives.CML therefore produces separate loss-weight lists for self-supervised contrastive learning and recommendation optimization.

3.4 The Learning Process of CML Framework

CML combines behavior-specific ranking with alternating optimization of a graph architecture and a contrastive meta network. Its training process also analyzes graph, aggregation, meta-encoder, and contrastive-learning costs.

  • Optimization Objective: CML uses behavior-specific BPR loss to rank observed user interactions above unobserved counterparts.The objective also applies L2 regularization to alleviate overfitting.
  • Training Strategy: Training alternates updates to the behavior-aware graph neural architecture and multi-behavior contrastive meta network.The graph uses behavior-aware interaction adjacency information, while the meta network processes cross-type and behavior-specific embeddings.
  • Training Strategy: The optimization loop first learns initial encoder parameters, then refines the meta network on meta data, and finally updates the graph network using personalized contrastive-loss weights.These three stages are designed to improve training efficiency.
  • Model Complexity Analysis: Graph message passing costs O(L × K × |R_k+| × d), while multi-behavior aggregation costs O(L × (N + M) × d × (K + d)).The graph cost depends on propagation layers, behavior types, nonzero behavior-specific relations, and embedding dimensionality.
  • Model Complexity Analysis: The meta contrastive encoder costs O(K × |R_k+| × d^2), and InfoNCE calculation costs O(B × d) for the numerator and O(B × S × d) for the denominator.Sampling size S reduces time complexity and increases randomness for robustness.

4 EVALUATION

The evaluation tests CML on three public recommendation datasets against single-behavior, multi-behavior, and heterogeneous graph baselines using ranking metrics and ablations. CML consistently achieves significant improvements, while ablations support both contrastive learning and customized meta weighting.

  • Evaluation Setup: Experiments on three public datasets address CML’s effectiveness, module contributions, sparsity handling, hyperparameter sensitivity, and interpretability.The datasets include Tmall, with page-view, favorite, cart, and purchase behaviors, and use purchase or transaction as the target behavior.
  • Baselines: CML is compared with single-behavior, multi-behavior, and heterogeneous graph recommendation methods.The comparison includes BPR, PinSage, NGCF, NMTR, MATN, MBGCN, HGT, and HeCo among the described baselines.
  • Evaluation Setup: NDCG@10 and HR@10 evaluate recommendations under a leave-one-out test using users’ last target-behavior interactions.CML and the best baseline are each run 10 times for significance testing.
  • Performance Comparison: CML consistently outperforms all baseline types across the three datasets, with p-values much less than 0.05.The paper attributes the gains to customized multi-behavior dependency modeling and auxiliary self-supervised signals.
  • Performance Comparison: Multi-behavior methods generally outperform single-behavior methods, while CML also exceeds HGT and HeCo by a large margin in all cases.The comparisons support using multi-behavior information and encoding relation heterogeneity for preference modeling.
  • Ablation and Effectiveness Analyses: Removing the multi-behavior contrastive framework, meta contrastive network, or meta knowledge encoder degrades performance.The ablations associate these components with cross-view knowledge transfer, pair-specific influence weighting, and customized contrastive learning.

4.4 Model Performance on Alleviating Interaction Data Sparsity (RQ3)

CML is evaluated for robustness to interaction sparsity, hyperparameter sensitivity, and interpretability. Results indicate that multi-behavior contrastive learning improves sparse-data recommendation and that personalized weights reflect user-specific behavior patterns.

  • Interaction sparsity: CML consistently outperforms other multi-behavior recommendation methods across different interaction degrees on Tmall data.Performance is evaluated with HR and NDCG averaged within user groups defined by interaction counts.
  • Interaction sparsity: Recommendation accuracy improves for all compared methods as users have more interactions.The study attributes this trend to better learning of behavior embeddings from sufficient user behaviors.
  • Hyperparameter analysis: More graph propagation layers improve performance when L≤3, whereas additional layers may introduce noise through oversmoothing.The analysis links shallower gains to capturing high-order-neighbor dependencies and deeper degradation to noisy user representations.
  • Hyperparameter analysis: CML performs well with representation dimensionality 16≤d≤32 and with a meta-network batch smaller than the graph-network batch.The smaller meta batch is reported to improve cooperation between the augmented self-supervised task and the BPR-based ranking objective.
  • Qualitative evaluation: Personalized contrastive weights vary across users and quantify the importance of individual target–auxiliary behavior losses.For user 27310, favorite-buy receives weight 0.595 versus 0.243 for view-buy.
  • Qualitative evaluation: CML produces agreement among projected embeddings for page view, add-to-favorite, add-to-cart, and purchase behaviors.The visualization compares CML with the model without the contrastive learning framework on IJCAI-Contest data.

5 RELATED WORK

Related work covers graph neural recommendation, multi-behavior recommendation, and self-supervised representation learning. Existing methods model high-order connectivity or behavior relationships, while contrastive learning supplies a representation-learning paradigm for multiple views.

  • Graph neural recommendation: GNN-based recommendation models aggregate embeddings over neighbors, with NGCF using stacked propagation layers to capture high-order connectivity.LightGCN is described as simplifying NGCF-style message passing by omitting the weight matrix and using sum-based aggregation.
  • Multi-behavior recommendation: Multi-behavior recommendation methods model behavior multiplicity through attention mechanisms or graph convolutional networks.The cited approaches include behavior-wise attention, MBGCN’s discriminative behavior representations, and MATN’s attentive pattern aggregation.
  • Self-supervised learning: Self-supervised learning learns discriminative representations by contrasting positive and negative samples from different views.Prior applications include image and text representation learning, with graph methods adapting the paradigm to topological structures.

6 CONCLUSION

The paper presents CML as a multi-behavior contrastive meta learning framework that preserves heterogeneous behavioral context through agreement between behavior views. Experiments on real-world datasets support its effectiveness, while future work targets pre-training for online user modeling and disentangled user representations.

  • Conclusion: CML learns user representations by preserving heterogeneous behavior context through agreement between contrastively constructed behavior views.Its behavior-aware graph neural architecture combines multi-behavior self-supervision for heterogeneous relational learning.
  • Conclusion: Experiments on several real-world datasets compare CML with state-of-the-art methods to demonstrate its effectiveness.
  • Future work: Future directions include pre-training CML for online user modeling and learning disentangled representations reflecting multidimensional user interests.The paper gives user profiling as an example application for pre-training.
Loading 2202.08523v1…