PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Behavior Recommendation with Cascading Graph Convolution Networks

Zhiyong Cheng, Sai Han, Fan Liu, Lei Zhu, Zan Gao, Yuxin Peng

arXiv:2303.15720v1cs.IR

TL;DR

Existing multi-behavior models often fail to capture behavior-chain dependencies in embedding learning, while auxiliary interactions can help address sparse target-behavior data. MB-CGCN uses behavior-specific GCN blocks, passing embeddings from each behavior to the next after feature transformation and aggregating them for prediction. Across two benchmark datasets, MB-CGCN achieves average relative gains of 33.7% in Recall@10 and 35.9% in NDCG@10 over state-of-the-art multi-behavior recommendation methods.

  • Problem

    Existing multi-behavior models often fail to capture behavior-chain dependencies in embedding learning, while auxiliary interactions can help address sparse target-behavior data.

    Source: §1 · pp. 1–2

  • Method

    MB-CGCN uses behavior-specific GCN blocks, passing embeddings from each behavior to the next after feature transformation and aggregating them for prediction.

    Source: §1 · p. 2

  • Results

    Across two benchmark datasets, MB-CGCN achieves average relative gains of 33.7% in Recall@10 and 35.9% in NDCG@10 over state-of-the-art multi-behavior recommendation methods.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Experiments on two real-world datasets show MB-CGCN outperforming state-of-the-art multi-behavior models, and ablations support its feature transformation and embedding aggregation designs.

    Source: §5 · p. 9

Abstract

from arXiv · show

Multi-behavior recommendation, which exploits auxiliary behaviors (e.g., click and cart) to help predict users' potential interactions on the target behavior (e.g., buy), is regarded as an effective way to alleviate the data sparsity or cold-start issues in recommendation. Multi-behaviors are often taken in certain orders in real-world applications (e.g., click>cart>buy). In a behavior chain, a latter behavior usually exhibits a stronger signal of user preference than the former one does. Most existing multi-behavior models fail to capture such dependencies in a behavior chain for embedding learning. In this work, we propose a novel multi-behavior recommendation model with cascading graph convolution networks (named MB-CGCN). In MB-CGCN, the embeddings learned from one behavior are used as the input features for the next behavior's embedding learning after a feature transformation operation. In this way, our model explicitly utilizes the behavior dependencies in embedding learning. Experiments on two benchmark datasets demonstrate the effectiveness of our model on exploiting multi-behavior data. It outperforms the best baseline by 33.7% and 35.9% on average over the two datasets in terms of Recall@10 and NDCG@10, respectively.

1 INTRODUCTION

Auxiliary behaviors offer richer interaction data that can help address sparsity in target-behavior recommendation. MB-CGCN models behavior order by passing transformed embeddings between behavior-specific GCN blocks and reports gains on two benchmark datasets.

1 INTRODUCTION

  • Auxiliary behaviors such as clicks and carts provide richer interactions than purchases and can help capture preferences and alleviate data sparsity.Clicks can provide item information, while carts can help users compare candidates.Source: §1 · p. 1
  • Existing methods often fail to use behavior-chain dependencies directly in embedding learning, despite later behaviors offering stronger preference signals than earlier ones.The paper notes that some methods model dependencies for aggregation and only limited methods consider behavior chains.Source: §1 · p. 2
  • MB-CGCN passes behavior embeddings through a feature transformation into the next GCN block, then aggregates embeddings for target prediction using only target-behavior supervision.The transformation is designed to reduce noise or misleading information during delivery between blocks.Source: §1 · p. 2
  • Users may skip one or two intermediate behaviors and proceed directly to the final behavior.This is noted as a possibility in real scenarios.Source: §1 · p. 2
  • Across two benchmark datasets, MB-CGCN achieves average relative gains of 33.7% in Recall@10 and 35.9% in NDCG@10 over state-of-the-art multi-behavior recommendation methods.Further empirical studies examine the validity of different model designs and the effects of different numbers and orders of behaviors.Source: §1 · p. 2

2 RELATED WORK

Prior multi-behavior recommendation work extends traditional methods or uses DNNs and GCNs to combine behavior information. The paper distinguishes MB-CGCN by explicitly passing transformed features along a behavior chain without multi-task optimization.

2 RELATED WORK

  • Traditional approaches extend matrix factorization across behavior matrices or adapt BPR sampling to use multiple behaviors.Sampling variants use behavior preference levels, item similarity, or adaptive balancing.Source: §2 · p. 2
  • DNN approaches learn behavior embeddings and aggregate them for target prediction, while NMTR reuses earlier behavior prediction scores in a multi-task framework.DIPN, MATN, and KHGT use attention or transformers to model behavior relations.Source: §2 · pp. 2–3
  • GCN approaches commonly build a unified graph from multiple behaviors, with methods modeling behavior contributions, semantics, shared or unique embeddings, and diverse patterns.Some approaches also model multiple interests or learn dependencies through graph contrastive learning.Source: §2 · p. 3
  • Earlier methods generally did not model behavior order or explicitly exploit behavior dependencies in embedding learning; CRGCN introduced cascading GCNs with multi-task learning.The paper says CRGCN’s residual design restricts auxiliary behaviors to single-layer GCNs; MB-CGCN instead uses feature transformation and omits multi-task optimization.Source: §2 · p. 3

3.1 Problem Formulation

The task is to use ordered interaction data across multiple behaviors to predict whether a user will interact with an item under the target behavior. The interaction matrices encode observed user-item behavior as binary entries.

3.1 Problem Formulation

  • The input comprises ordered interaction matrices for multiple behaviors, with the final matrix representing the target behavior.The number of behavior types is denoted by B.Source: §3.1 · p. 3
  • Each matrix entry is 1 when a user interacted with an item under that behavior and 0 otherwise.Source: §3.1 · p. 3
  • The model’s output is the probability that a user will interact with an item under the target behavior.The target behavior is the B-th behavior in the ordered set.Source: §3.1 · p. 3

3.2 Model Description

MB-CGCN uses a behavior-ordered sequence of LightGCN blocks to learn embeddings, passing features from one behavior to the next after transformation. It aggregates embeddings from all behaviors for target-behavior prediction.

3.2.2 Embedding Initialization.

  • User and item embeddings are initialized from one-hot ID vectors, then used as the input features for the first behavior’s LightGCN.The user and item embeddings have dimension d.Source: §3.2.2 · pp. 3–4

3.2.3 Cascading GCN blocks.

  • MB-CGCN chains LightGCN blocks in behavior order, transforming embeddings learned for one behavior before using them as the next behavior’s input features.Each block learns user and item embeddings from its corresponding behavior.Source: §3.2.3 · p. 4
  • LightGCN is the model’s backbone for learning user and item embeddings from each behavior.The paper describes LightGCN as omitting the transformation matrix and nonlinear activation of vanilla GCN.Source: §3.2.3 · p. 4
  • Feature transformation processes embeddings from a former behavior before delivery, to distill useful features while retaining information from different behaviors.The design addresses concerns that direct delivery may lose diverse behavior information or pass noise into later learning.Source: §3.2.3 · pp. 4–5

3.2.4 Embedding Aggregation.

  • The model aggregates embeddings learned from all behaviors and predicts target-behavior interactions using the inner product of user and item embeddings.The inner product serves as a ranking score for target-behavior recommendation.Source: §3.2.4 · p. 5

3.3 Model Training

The model is optimized with pairwise learning using the standard BPR loss. The objective compares observed and unobserved target-behavior items and includes L2 regularization.

3.3 Model Training

  • The standard BPR loss assumes a higher score for an observed item than for an unobserved item and is used for pairwise model optimization.The objective applies to positive and negative samples defined by the target behavior.Source: §3.3 · p. 5
  • The objective applies L2 regularization to all trainable parameters, with λ controlling its strength.The paper states that L2 regularization is adopted to prevent over-fitting.Source: §3.3 · p. 5
  • Positive and negative sample pairs are formed from items observed and unobserved, respectively, in the target behavior.The objective uses the sigmoid function on the difference between the corresponding scores.Source: §3.3 · p. 5

3.4 Discussion

The discussion distinguishes MB-CGCN from CRGCN by how embeddings pass between behaviors. It motivates transforming embeddings to address noise from earlier behaviors and support higher-order propagation.

3.4 Discussion

  • Unlike CRGCN’s residual connection, MB-CGCN transforms embeddings to distill useful information for the next behavior’s learning.CRGCN refines embeddings through the behavior chain and uses the last behavior’s embeddings for prediction, making performance sensitive to earlier embeddings.Source: §3.4 · p. 5
  • The authors argue that noisy earlier behaviors can make higher-order propagation harmful, while transformation helps MB-CGCN benefit from it.The paper links this concern to CRGCN’s use of only one propagation layer in auxiliary behaviors.Source: §3.4 · p. 5
  • Table 1 summarizes statistics for the datasets used in the experiments.Source: Table 1 · p. 5

4 EXPERIMENT

The experiments evaluate MB-CGCN on two datasets using leave-one-out testing and compare it with single- and multi-behavior baselines. MB-CGCN achieves the best performance in the reported comparisons, with additional analyses of model design and implementation settings.

4.1 Experiment Settings

  • Beibei has 21,716 users and 7,977 items, while Tmall has 15,449 users and 11,953 items; both datasets include view, cart, and buy behaviors.Beibei follows the view>cart>buy order; duplicate interactions were removed by retaining the earliest one.Source: §4.1 · p. 5

4.1.2 Evaluation Protocols.

  • Evaluation uses leave-one-out testing, with each user’s last interacted item in the test set and second-last item in validation.The remaining positive items are used for training; Recall@K and NDCG@K measure top-n recommendation performance.Source: §4.1.2 · p. 5

4.1.3 Baselines.

  • Baselines include MF-BPR, NeuMF, LightGCN, and multi-behavior models NMTR, RGCN, GNMR, MBGCN, and CRGCN.LightGCN models high-order user-item graph connectivity, while the multi-behavior baselines use distinct strategies for behavior relations, contributions, and cascading effects.Source: §4.1.3 · pp. 5–6

4.2 Overall Performance

  • 9.0% and 16.1% on Beibei, and 46.6% and 30.1% on Tmall, are MB-CGCN’s improvements over the best baseline for Recall@20 and NDCG@20, respectively.MB-CGCN achieves the best performance among the baselines on both metrics across the two datasets.Source: Table 2 · p. 6, §4.2 · p. 6

4.1.4 Hyper-parameter Settings.

  • The experiments use Adam, 64-dimensional embeddings, batch size 1024, and tuned learning rates and GCN layer counts.For view>cart>buy, the layer counts are {3, 4, 3} on Beibei and {3, 4, 2} on Tmall; other baselines use official code with tuned parameters.Source: §4.1.4 · p. 6

4.2 Overall Performance

  • MB-CGCN passes behavior embeddings through feature transformations between GCN blocks and outperforms CRGCN, which uses residual delivery and multi-task learning.The paper attributes MB-CGCN’s improvement over CRGCN to its design; multi-behavior models generally outperform single-behavior models, and LightGCN beats MF-BPR and NeuMF.Source: §4.2 · pp. 6–7

4.3 Ablation Study

The ablations examine feature transformation and feature aggregation in MB-CGCN, including performance differences between alternative aggregation variants.

4.3.1 Effect of feature transformation.

  • The default ablation order is view>cart>buy, while other behavior orders are studied in Section 4.4.2.Without feature transformation, embeddings from one behavior directly initialize embedding learning for the next behavior.Source: §4.3.1 · p. 7
  • With feature transformation, MB-CGCN improves by 9.0% and 5.8% on Beibei, and 0.7% and 6.7% on Tmall, for Recall@20 and NDCG@20, respectively.The authors interpret the results as evidence that transformation distills useful information from earlier behaviors for later embedding learning.Source: §4.3.1 · p. 7

4.3.2 Effects of feature aggregation.

  • Linear feature aggregation performs better than concatenation, and the authors report that considering embeddings from all behaviors is necessary.The compared variants use the last GCN block alone, linear aggregation, or concatenation of behavior embeddings; more sophisticated fusion methods are left to future work.Source: §4.3.2 · p. 7

4.4 Impact of multi-behaviors

The section examines how behavior count and order, as well as GCN layer number, affect MB-CGCN performance. More behaviors and increasing layer numbers improve performance, while ordering behaviors consistently with their preference signals matters.

4.4.1 Behavior number.

  • Adding behavior types improves performance; cart yields a large gain, while adding view yields a smaller further improvement.The comparison uses buy alone, cart>buy, and view>cart>buy; Table 5 reports top-20 results.Source: §4.4.1 · p. 8, Table 5 · p. 8

4.5 Effects of layer number

  • The reported layer-number experiments use the same layer count for every behavior; separate per-behavior tuning results are not reported.The authors say the shared-layer experiments provide a global view of layer-number effects.Source: §4.5 · p. 8

4.4.1 Behavior number.

  • On both datasets, O4 (view>cart>buy) outperforms O3 (cart>view>buy), supporting behavior orders consistent with real-world chains.Despite richer view interactions, O2 (view>cart) performs better than O1 (cart>buy); the authors attribute this to cart revealing preferences more accurately.Source: Figure 2 · p. 8, Figure 3 · p. 8, §4.4.2 · p. 8

4.5 Effects of layer number

  • MB-CGCN performs better on both datasets as the layer number increases.The authors attribute this to LightGCN exploiting higher-order graph information; feature transformation lets MB-CGCN benefit from higher-order operations on auxiliary behaviors, unlike CRGCN’s residual delivery design.Source: §4.5 · p. 8

5 CONCLUSION

MB-CGCN passes transformed behavior features through cascading GCN blocks and aggregates embeddings for prediction. On two real-world datasets, it outperforms state-of-the-art multi-behavior models, with ablations supporting feature transformation and embedding aggregation.

5 CONCLUSION

  • MB-CGCN uses cascading GCN blocks, passing transformed features learned from one behavior to the next and aggregating embeddings for final prediction.The behavior features are learned with LightGCN.Source: §5 · p. 8
  • On two real-world datasets, MB-CGCN outperforms state-of-the-art multi-behavior models with substantial performance gains.The conclusion also reports that ablations verify the effectiveness of feature transformation and embedding aggregation.Source: §5 · p. 9
  • The paper studies the impact of multi-behavior number and order on final performance.Table 6 reports layer-number effects using the same layer count across behaviors.Source: §5 · p. 9, Table 6 · p. 9

Related papers

Loading 2303.15720v1…