PaperlayerView exact revision on arXiv

Source-linked AI summary

Graph Meta Network for Multi-Behavior Recommendation

Lianghao Xia, Yong Xu, Chao Huang, Peng Dai, Liefeng Bo

arXiv:2110.03969v1cs.IRcs.AI

TL;DR

Multi-behavior recommendation must account for complex cross-type dependencies and user-specific behavior diversity that single-behavior methods overlook. MB-GMN combines personalized meta-knowledge learning, graph neural propagation, and cross-type prediction transfer; experiments on three real-world datasets report improved performance over varied baselines. Its evaluation treats purchases as target behaviors in the experimented datasets.

  • Problem

    Existing recommendation methods often overlook heterogeneous, interdependent, and user-specific multi-behavior patterns, despite their relevance to representing user preferences.

  • Method

    MB-GMN uses a meta-knowledge learner, meta graph neural network, and meta prediction network to model personalized behavior heterogeneity, high-order connectivity, and cross-type dependency.

  • Results

    Experiments on three real-world datasets show that MB-GMN advances recommendation performance compared with baselines from various research lines.

  • Takeaways & Limitations

    The framework jointly incorporates inter-correlations among multiple user behaviors into collaborative filtering through customized multi-behavior pattern modeling.

  • Takeaways & Limitations

    The experiments designate purchases as target behaviors because they are treated as indicators of Gross Merchandise Volume in online retailing.

Abstract

from arXiv · show

Modern recommender systems often embed users and items into low-dimensional latent representations, based on their observed interactions. In practical recommendation scenarios, users often exhibit various intents which drive them to interact with items with multiple behavior types (e.g., click, tag-as-favorite, purchase). However, the diversity of user behaviors is ignored in most of the existing approaches, which makes them difficult to capture heterogeneous relational structures across different types of interactive behaviors. Exploring multi-typed behavior patterns is of great importance to recommendation systems, yet is very challenging because of two aspects: i) The complex dependencies across different types of user-item interactions; ii) Diversity of such multi-behavior patterns may vary by users due to their personalized preference. To tackle the above challenges, we propose a Multi-Behavior recommendation framework with Graph Meta Network to incorporate the multi-behavior pattern modeling into a meta-learning paradigm. Our developed MB-GMN empowers the user-item interaction learning with the capability of uncovering type-dependent behavior representations, which automatically distills the behavior heterogeneity and interaction diversity for recommendations. Extensive experiments on three real-world datasets show the effectiveness of MB-GMN by significantly boosting the recommendation performance as compared to various state-of-the-art baselines. The source code is available athttps://github.com/akaxlh/MB-GMN.

1 INTRODUCTION

Existing recommender methods often model only one interaction type, while real users generate heterogeneous and personalized multi-behavior patterns. MB-GMN addresses these challenges with meta-learning and graph-based modeling, and experiments on three real-world datasets report improved recommendation performance.

  • Motivation: Single-behavior methods may not comprehensively represent users’ multidimensional preferences in real-world multi-behavior scenarios.The same user-item pair can involve clicks, favorites, add-to-cart actions, and purchases.
  • Challenges: Behavior heterogeneity requires modeling complementary type-specific information and complex pairwise or high-level cross-type correlations.Different behavior views can interweave while contributing distinct knowledge about user interests.
  • Challenges: Interaction diversity varies across users because their type-specific behaviors can exhibit different dependencies and personal patterns.For example, some users favorite items mainly when likely to purchase, whereas others favorite many items but purchase sporadically.
  • Approach: MB-GMN combines a personalized meta-knowledge learner, meta graph neural network, and meta prediction network to model heterogeneous and cross-type behavior patterns.The framework uses high-order graph connectivity and transfers behavior knowledge into prediction parameters.
  • Evaluation: Experiments on three real-world datasets report that MB-GMN advances recommendation performance compared with baselines from various research lines.The study also includes ablation experiments examining the model design.

2 PRELIMINARY

The paper represents multiplex user-item interactions as a multi-behavior tensor and distinguishes one target behavior from context behaviors. The task is to estimate target-behavior adoption from observed interactions across behavior types.

  • Notation: Users and items are represented as sets U and V, with I users and J items indexed by i and j.The formulation uses these sets to organize multiplex user-item interactions.
  • Interaction Tensor: The multi-behavior interaction tensor X ∈R^I×J×K records typed interactions between users and items, where K is the number of behavior types.Examples include click, tag-as-favorite, and purchase.
  • Task Definition: One interaction type is designated as the target behavior, while other types serve as context behaviors that assist target prediction.Purchase can be the target, with click, favorite, or add-to-cart as context behaviors.
  • Task Definition: The input is the observed multi-behavior interaction tensor across users, items, and K behavior types.The tensor encodes the available interaction observations.
  • Task Definition: The output is a predictive function estimating the likelihood that user u_i adopts item v_j with the target behavior type.Prediction is made for the designated target interaction.

3 METHODOLOGY

MB-GMN has two core components: personalized multi-behavior pattern encoding and cross-type behavior dependency modeling. Together, they use meta-learning to customize representations and the prediction network across behavior types.

  • Multi-Behavior Pattern Encoding: Multi-behavior pattern encoding uses a meta-knowledge learner to capture personalized multi-behavior characteristics.This component models interaction patterns in a user-specific manner.
  • Cross-Type Dependency Modeling: Cross-type behavior dependency modeling uses transfer learning to customize the prediction network by transferring knowledge across behavior types.The component targets dependencies among different interaction behaviors.

3.1 Multi-Behavior Pattern Modeling

The multi-behavior pattern modeling module learns user-specific behavior semantics and uses them to customize graph propagation. A low-rank meta-knowledge transformation supports this personalized modeling while preserving behavior-specific and high-order relational information.

  • Design Goal: MB-GMN targets personalized multi-behavior characteristics rather than placing different users’ interaction patterns in one shared representation space.The architecture jointly models user-item relation heterogeneity and relationships among behaviors.
  • Architecture: Figure 1 organizes the architecture into behavior-heterogeneity modeling, behavior-semantic graph propagation with high-order connectivity, and prediction-parameter customization for cross-type dependency.These correspond to the meta-knowledge learner, meta graph neural network, and meta-knowledge transfer networks.
  • Meta-Knowledge Learner: The meta-knowledge learner uses user-specific multi-behavior patterns to generate transformation weights that inject type-specific semantics into initial embeddings.It learns a personalized context projection from initialized user and item embeddings and interaction context.
  • Meta-Knowledge Learner: The learned matrix V_i,k is used to generate graph-neural-network weights for handling behavior heterogeneity.V_i,k is a customized parametric matrix for user u_i and behavior type k.
  • Low-Rank Transformation Decomposition: Low-rank transformation decomposition restricts the learned transformation to a lower-dimensional state d′ that is smaller than the embedding dimension d.The transformation is decomposed into two low-rank projections with trainable parameters.

3.2 Meta Graph Neural Network

The meta graph neural network models personalized multi-behavior patterns by encoding behavior semantics, learning cross-behavior relations, and aggregating high-order graph context.

  • MB-GMN uses three sub-modules: behavior semantic encoding, behavior mutual dependency learning, and high-order multi-behavioral context aggregation.
  • Behavior Semantic Encoding: Behavior-aware message passing constructs behavior-specific user-item graphs and preserves personalized semantic signals for each interaction type.
  • Behavior Relation Learning: An attention-based relation encoder refines propagated embeddings by injecting mutual relational information across different behavior types.
  • High-order Multi-Behavioral Context Aggregation: The model produces a global user embedding that incorporates all behavior types alongside behavior-specific representations.
  • High-order Multi-Behavioral Context Aggregation: High-order embedding propagation integrates semantic and relational information to capture multi-hop connectivity in the interaction graph.

3.3 Meta-Knowledge Transfer Networks

The meta-knowledge transfer networks customize behavior-dependent prediction using user-item-specific knowledge and transfer information from context behaviors to a target behavior.

  • Knowledge Transfer Learning Framework: For target-behavior prediction, context behavior information provides knowledge about the user-item pair and is used to generate customized prediction sub-networks.
  • MB-GMN learns meta knowledge customized to each user-item pair and the relation between source context behaviors and a target behavior.
  • Meta-knowledge Learner for Behavior Dependency: The meta-knowledge representation captures dependent interactive relations between user and item embeddings through element-wise multiplication and concatenation.
  • Knowledge Transfer Learning Framework: The meta network generates prediction-network parameters, including weights and biases, from the learned behavior-dependent meta knowledge.

3.4 Multi-task Learning Framework

MB-GMN trains predictions for source-target behavior pairs with sampled positive and negative interactions, Adam optimization, regularization, and stated computational costs.

  • The training framework optimizes prediction objectives for every pair of source and target behaviors.
  • For each user and target behavior, the model samples S positive interactions and S non-interacted items as negative samples.
  • The objective combines a marginal pair-wise loss with weight-decay regularization weighted by λ, optimized using Adam.
  • Computational Complexity: The multi-behavior pattern modeling costs O(L × K × d × |E|), while behavior relation learning adds O(L × K × d × (K + d) × (N + M)).

4 EVALUATION

The evaluation compares MB-GMN with diverse recommendation baselines on three real-world datasets using top-N recommendation metrics, ablations, behavior analyses, sparsity tests, and hyperparameter studies. Results report improved and consistently strong recommendation performance, alongside analyses of multi-behavior modeling and model explainability.

  • Experimental Setup: Experiments evaluate MB-GMN on three real-world datasets, treating purchases as the target behavior because they indicate Gross Merchandise Volume.The datasets contain multiple user-item relation types, including page view, add-to-cart, tag-as-favorite, and purchase in Taobao.
  • Experimental Setup: Performance is measured with NDCG@N and HR@N under leave-one-out evaluation, using the last interactive item for testing.Each test instance pairs the held-out item with other items for efficient evaluation.
  • Performance Validation (RQ1): MB-GMN improves top-N recommendation performance over the evaluated baselines, producing customized user and item representations through personalized multi-behavior modeling.The comparison includes conventional matrix factorization, neural collaborative filtering, autoencoder, autoregressive, graph neural, and multi-behavior methods.
  • Performance Validation (RQ1): MB-GMN consistently outperforms multi-behavior baselines, while the authors attribute competing methods’ gaps to limited modeling of complex cross-behavior dependencies or shared message passing.The discussed limitations concern cascade modeling, weighted aggregation, and parameter sharing that overlooks unique behavior characteristics.
  • Performance Validation (RQ1): Adding multi-behavior information improves recommendation over methods that do not differentiate interactions, and graph neural models outperform other baselines in the reported comparisons.The authors associate the latter observation with deriving high-order user-item relations through stacked message-passing layers.
  • Performance Validation (RQ1): On the Beibei dataset, MB-GMN achieves the best HR and NDCG performance across different top-N positions.The authors present this as consistent superiority in assigning higher probability to correctly interacted items.

4.3 Model Ablation Study (RQ2)

The ablation and sensitivity studies evaluate MB-GMN’s components, context behaviors, sparsity robustness, hyperparameters, and interpretability. Results support the value of low-rank transformations, behavior-relation learning, cross-type prediction, and jointly modeled context behaviors.

  • Ablation Study: The ablation study evaluates MB-GMN variants that remove low-rank decomposition, meta-knowledge encoding, or meta-knowledge-based prediction parameters.These variants are reported alongside the key-component ablation results in Table 4.
  • Ablation Study: The low-rank transformation decomposition helps alleviate overfitting during heavy transformation operations by acting as an efficient regularization term.
  • Ablation Study: The attention layer captures pair-wise correlations among behavior types through multiple representation subspaces during customized message passing.
  • Ablation Study: Context behavior relations provide auxiliary knowledge during multi-behavior aggregation and supervising signals for optimization under a multi-task learning paradigm.
  • Context Behavior Study: Using all context behaviors achieves the best performance among variants that omit page view, tag-as-favorite, or add-to-cart behavior, or use purchase alone.
  • Sparsity Study: MB-GMN maintains consistent performance gains across different interaction sparsity degrees, with accuracy measured by NDCG@10 and HR@10.The sparsity study varies the number of user interactions on the x-axis and recommendation accuracy on the y-axis.
  • Hyperparameter and Interpretation Studies: Two graph layers outperform one, whereas greater depth introduces noise and over-smoothing; case studies show page views usually contribute most to predicting other behaviors.

5 RELATED WORK

Related work extends collaborative filtering with neural networks, social relations, knowledge graphs, and graph neural networks. These approaches enhance representation learning by modeling nonlinear interactions or additional relational information.

  • Neural Collaborative Filtering: Neural collaborative-filtering methods use architectures such as multilayer perceptrons and autoencoders to learn nonlinear or dense user-item representations.
  • Side-Information Recommendation: Social-relation recommendation models use connections among users to enhance user representations.
  • Side-Information Recommendation: Knowledge-aware recommendation models supplement user-item interaction learning with connections between entities and relational paths in knowledge graphs.
  • Graph Neural Networks: Graph neural networks perform graph-structured learning through message passing that aggregates feature information from neighboring nodes.

6 CONCLUSION

The conclusion presents MB-GMN as a graph-meta-network framework for modeling heterogeneous and diverse multi-behavior interactions in a customized meta-learning paradigm. Experiments on three large-scale e-commerce datasets verify its effectiveness, while future work targets side information and time-sensitive recommendation.

  • Conclusion: MB-GMN explicitly models inter-correlations among multiple user behaviors within collaborative filtering and customizes multi-behavior pattern modeling through meta-learning.
  • Conclusion: Experiments on three large-scale e-commerce datasets verify the effectiveness of MB-GMN.
  • Future Work: Future work will incorporate user profiles and item text descriptions, and investigate time-sensitive modeling for newly arriving behavior data and real-time recommendation.
Loading 2110.03969v1…