PaperlayerView exact revision on arXiv

Source-linked AI summary

Dynamic Graph Neural Networks for Sequential Recommendation

Mengqi Zhang, Shu Wu, Xueli Yu, Qiang Liu, Liang Wang

arXiv:2104.07368v2cs.IR

TL;DR

Sequential recommendation needs to model changing preferences from historical sequences, while many methods overlook dynamic collaborative signals across users. DGSR represents sequences and their time- and order-aware collaborations in a dynamic graph, uses DGRN to extract preferences, and reports best performance on three datasets across most metrics.

  • Problem

    Sequential recommendation must predict users’ next items from evolving interaction histories, but existing methods often ignore dynamic collaborative signals among different user sequences.

  • Method

    DGSR converts user sequences into a time- and order-aware dynamic graph, applies a Dynamic Graph Recommendation Network, and formulates next-item prediction as user–item link prediction.

  • Results

    DGSR achieves the best performance on three datasets with most evaluation metrics, including NDCG@10 improvements of 10.56%, 3.92%, and 3.81% over the strongest baselines on Beauty, Games, and CDs.

  • Takeaways & Limitations

    Modeling dynamic collaborative information among user sequences is reported as effective and rational for sequential recommendation.

Abstract

from arXiv · show

Modeling user preference from his historical sequences is one of the core problems of sequential recommendation. Existing methods in this field are widely distributed from conventional methods to deep learning methods. However, most of them only model users' interests within their own sequences and ignore the dynamic collaborative signals among different user sequences, making it insufficient to explore users' preferences. We take inspiration from dynamic graph neural networks to cope with this challenge, modeling the user sequence and dynamic collaborative signals into one framework. We propose a new method named Dynamic Graph Neural Network for Sequential Recommendation (DGSR), which connects different user sequences through a dynamic graph structure, exploring the interactive behavior of users and items with time and order information. Furthermore, we design a Dynamic Graph Recommendation Network to extract user's preferences from the dynamic graph. Consequently, the next-item prediction task in sequential recommendation is converted into a link prediction between the user node and the item node in a dynamic graph. Extensive experiments on three public benchmarks show that DGSR outperforms several state-of-the-art methods. Further studies demonstrate the rationality and effectiveness of modeling user sequences through a dynamic graph.

1 INTRODUCTION

Sequential recommendation must capture changing preferences from ordered interaction histories, but existing approaches largely encode users independently and omit dynamic collaboration across sequences. DGSR addresses this gap with a dynamic graph that links sequences through time- and order-aware interactions and uses DGRN for preference extraction.

  • Sequential recommendation uses users’ historical interaction sequences because preferences change over time with previously interacted items.
  • Existing methods largely encode each user’s own sequence while ignoring high-order collaborative connections among users and items in different sequences.
  • Dynamic collaborative information also changes over time, altering the high-order context that influences user interests and item semantics.
  • DGSR converts user sequences into a dynamic graph with time- and order-annotated edges, connecting sequences that share common items.
  • Its Dynamic Graph Recommendation Network samples associated subgraphs, models long- and short-term preferences, and uses stacked layers to capture dynamic high-order connectivity.
  • DGSR converts next-item prediction into user–item link prediction and achieves the best performance on three datasets across most evaluation metrics.

2 RELATED WORK

Sequential recommendation research has progressed from conventional sequence models to deep learning and graph neural networks. Existing approaches largely model individual sequences, while dynamic-graph methods have not been adapted to sequential recommendation scenarios.

  • Sequential recommendation predicts a user’s next item from historical interaction sequences, using methods such as Markov chains and translation-based models.Markov methods use k-order interactions, while TransRec models third-order interactions.
  • Deep learning approaches apply RNNs and CNNs to sequential recommendation, with GRU4Rec introducing RNNs to session-based recommendation and Caser investigating interaction patterns.
  • GNN-based methods capture item-transition relationships, personalized attention, or long- and short-term interests in session-aware recommendation.Examples include SR-GNN, A-PGNN, and MA-GNN.
  • These GNN models focus on intra-sequence preferences and generally ignore item relationships across sequences, motivating methods such as HyperRec and CSRM.
  • Dynamic-graph research models evolving relationships in applications such as academic, social, and recommender networks, but existing methods have not been adapted to sequential recommendation.The paper identifies no prior study illustrating sequential recommendation from the perspective of dynamic graphs.

3 PRELIMINARIES

The preliminaries define sequential recommendation using user action and timestamp sequences and describe dynamic graphs as time- or order-indexed interactions. The paper focuses on continuous-time dynamic graphs for representing evolving node relationships.

  • Sequential recommendation predicts the next item in a user sequence from interactions occurring before the current time.Users, items, action sequences, and corresponding timestamp sequences are formally defined.
  • Users and items are represented by d-dimensional embedding vectors collected into user and item embedding matrices.
  • Dynamic graphs may be discrete-time or continuous-time, and this work primarily considers continuous-time dynamic graphs.
  • A dynamic network records interactions as time- or order-indexed edges, allowing it to capture how relationships between nodes evolve.An edge is represented as a triplet containing two nodes and an interaction time.

4 METHODOLOGY

DGSR converts users’ timestamped and ordered interaction sequences into dynamic graphs, samples sequence-centered sub-graphs, and uses Dynamic Graph Recommendation Networks to propagate information across related users and items. Its prediction layer aggregates refined user representations for next-item prediction.

  • 4.1 Dynamic Graph Construction: DGSR organizes user-item interactions into a dynamic graph whose edges record timestamps and interaction order.The graph at time t contains interactions occurring at or before t, while shared items connect different user sequences.
  • 4.1 Dynamic Graph Construction: Shared items connect user sequences through user-to-item and item-to-user relationships, exposing collaborative information across sequences.The resulting graph includes relationships such as users sharing interacted items.
  • 4.2 Sub-Graph Sampling: The sampling strategy selects recent first-order item neighbors and recursively samples multi-hop neighbors to form an m-order sub-graph around the target user.The sub-graph contains the target sequence and associated sequences; m controls its size.
  • 4.3 Dynamic Graph Recommendation Networks: DGRN propagates messages in both directions and updates user and item nodes to encode preferences and item characteristics from dynamic contextual information.Its design addresses sequential information in neighbors from both user and item perspectives.
  • 4.3 Dynamic Graph Recommendation Networks: For user nodes, DGRN separates long-term preference from short-term preference when aggregating chronological item neighbors.Long-term preference uses historical neighbors, while short-term preference reflects the user’s latest interest.
  • 4.3 Dynamic Graph Recommendation Networks: DGRN combines graph attention with sequence encoding in a dynamic attention module to differentiate neighbors using interaction order.The model also uses relative-order embeddings and recurrent processing for sequential neighbor information.

2 Convert all user sequences into a dynamic graph G

The section outlines DGSR’s training and prediction pipeline, then positions it as a flexible framework that can incorporate single-sequence models while modeling cross-sequence interactions.

  • Dynamic graph pipeline: DGSR generates dynamic sub-graphs, initializes node representations, updates users and items with DGRN, and predicts the next item.The supplied algorithm fragments identify these stages in sequence.
  • Training objective: The objective uses one-hot ground-truth next-item labels, all model parameters, L2 regularization, and a regularization-strength coefficient.The objective-function display itself is not supplied, but its accompanying text defines these components.
  • Model flexibility: DGSR can view TiSASRec, SR-GNN, and HGN as special cases by modifying message propagation and disabling cross-sequence propagation.This flexibility allows the framework to fuse nearly all single-sequence models through its message-propagation component.
  • Cross-sequence modeling: Unlike approaches based on direct sequence similarity, hypergraphs, or social attributes, DGSR represents cross-sequence interactions with finer-grained order information without requiring auxiliary social data.The comparison discusses interaction order, richer dynamic-graph representation, and independence from social relationship attributes.

5 EXPERIMENTS

The experiments evaluate DGSR on comparative performance, the effectiveness of its dynamic graph recommendation network, and sensitivity to key hyperparameters.

  • Experimental goals: The study uses experiments on three real-world datasets to evaluate DGSR.The experiments are designed around three stated research questions.
  • Experimental goals: RQ1 asks how DGSR performs against state-of-the-art sequential recommendation methods.This is the primary comparative evaluation question.
  • Experimental goals: RQ2 examines the effectiveness of the dynamic graph recommendation networks component in DGSR.The question isolates the contribution of the DGRN component.
  • Experimental goals: RQ3 studies the effects of DGRN layer number, sub-graph sampling size, maximum sequence length, and embedding size.The question targets architectural and representation hyperparameters.

5.1 Datasets

The dataset study uses three Amazon domains with timestamped interactions, filters sparse users and items, and constructs chronological training, validation, and test splits with incremental sequence-label pairs.

  • Datasets: The evaluation uses Amazon-CDs, Amazon-Games, and Amazon-Beauty, which vary in domain, size, and sparsity.These are three public real-world datasets from Amazon platforms.
  • Preprocessing: All datasets contain timestamps or specific interaction dates, and users and items with fewer than five related actions are discarded.Reviews or ratings are treated as implicit feedback before filtering.
  • Data splits: Each user’s most recent item is used for testing, the second most recent for validation, and earlier items for training.The split follows interaction recency within each user sequence.
  • Data construction: Sequences are expanded into incremental prefixes paired with the next item, such as [i1] →i2, [i1, i2] →i3, and [i1, i2, i3] →i4.Each prefix-label pair is associated with a corresponding dynamic sub-graph and linked item.

5.2 Experiment Settings

DGSR is compared with matrix-factorization, Markov, RNN, CNN, self-attention, hypergraph, and GNN-based recommenders using Hit@10 and NDCG@10 under fixed evaluation and implementation settings.

  • Compared methods: SR-GNN, HGN, and TiSASRec represent graph-based, gated, and interval-aware self-attention approaches among the compared sequence models.The supplied descriptions specify their modeling mechanisms.
  • Compared methods: The comparison includes HyperRec, BPR-MF, FPMC, GRU4Rec+, Caser, SASRec, SR-GNN, HGN, and TiSASRec.These baselines cover hypergraph, matrix-factorization, Markov-chain, recurrent, convolutional, self-attention, and graph-based approaches.
  • Evaluation metrics: Hit@K measures the proportion of ground-truth items appearing in the top K, while NDCG@K is position-aware and rewards higher-ranked targets.Higher NDCG indicates that target items tend to receive better ranks.
  • Evaluation protocol: Evaluation ranks the ground-truth item with 100 randomly sampled negative items and uses K=10 by default.Both Hit@10 and NDCG@10 are computed from these 101 candidates.
  • Implementation settings: The implementation fixes embedding size at 50, maximum sequence length at 50, learning rate at 0.01, batch size at 50, λ at 1e-4, and sub-graph sampling order at 4.The DGRN layer number is set to 3 for Beauty and CDs and 2 for Games; results average four random seeds.

5.3 Performance Comparison (RQ1)

DGSR achieves the strongest overall performance across three datasets, with particularly large NDCG@10 gains on Beauty, Games, and CDs. The results also indicate advantages over both sequence-only and alternative collaborative approaches.

  • DGSR achieves the best performance on three datasets with most evaluation metrics.
  • 10.56%, 3.92%, and 3.81% are DGSR’s NDCG@10 improvements over the strongest baselines on Beauty, Games, and CDs, respectively.
  • DGSR’s high-order dynamic-graph connectivity is associated with a significant improvement on the sparse and short Beauty dataset.The paper links this result to auxiliary information from cross-sequence connections.
  • SASRec, HGN, SR-GNN, and TiSASRec outperform GRU4Rec+ and Caser.The paper suggests attention or hierarchical gating can capture item-item relations explicitly.
  • BPR-MF performs poorly on all three datasets because it captures general interests rather than users’ behavior sequences.

5.4 Study of Dynamic Graph Recommendation Networks (RQ2)

The ablation study evaluates how DGRN’s long-term and short-term components contribute to DGSR. Combining these components and using attention with relative-order information generally gives the strongest results.

  • DGSR-D outperforms DGSR-R and DGSR-G on the Games and CDs datasets.
  • The paper attributes DGSR-D’s improvement to combining attention with relative-order embedding for distilling long-term neighbor information.
  • GCN-based variants perform poorly on all three datasets, which the paper associates with treating all neighbor nodes as equally important.Equal weighting may introduce more noise during message propagation.
  • DGSR-A performs better than DGSR-L, indicating that using only the last interaction embedding is insufficient for short-term information.
  • Variants combining long-term and short-term modules consistently outperform variants using only one module.The results support combining both information types in DGSR.

5.5 The Sensitivity of Hyper-parameters (RQ3)

DGSR benefits from additional dynamic collaborative context, but its sensitivity studies show diminishing returns and deterioration when propagation, sequence length, or sampling becomes excessive. Performance improves with embedding size before stabilizing.

  • Effect of DGRN Layer numbers: Further stacking causes DGSR-3 and DGSR-4 performance to deteriorate, possibly because excessive propagation leads to over-smoothing.
  • Effect of DGRN Layer numbers: DGSR-1 consistently outperforms DGSR-0, while DGSR-2 and DGSR-3 achieve the best performance on Games and Beauty, respectively.DGSR-1 uses intra-sequence information, whereas higher layers incorporate higher-order user-sequence information.
  • Effect of sub-graph sampling size: Increasing sub-graph sampling order from 1 to 3 improves performance, after which performance tends to stabilize.The paper attributes stabilization to the limited number of DGRN layers.
  • Effect of the maximum sequence length: Increasing maximum sequence length from 10 to 50 consistently improves Games performance, while larger values do not necessarily improve DGSR or DGSR-1.The paper suggests excessive sequence length may introduce noise and attenuate performance.
  • Effect of the maximum sequence length: DGSR outperforms DGSR-1 at every tested sequence length, including when n is 10.The authors interpret this as evidence that high-order contextual information helps when user history is insufficient.
  • Effect of the embedding size: Model performance gradually improves as embedding size increases from 16 to 80, then tends to stabilize.

6 CONCLUSION

This work introduces DGSR, a dynamic-graph framework that explicitly models collaborative information across user sequences. It encodes chronological interactions and converts next-item recommendation into dynamic-graph node-link prediction, with experiments on three real-world datasets supporting its effectiveness and rationality.

  • DGSR explicitly models dynamic collaborative information among different user sequences in sequential recommendation.
  • DGSR converts all user sequences into a dynamic graph containing chronological order and timestamps of user-item interactions.
  • Its Dynamic Graph Recommendation Network explicitly encodes dynamic collaborative information across user sequences.
  • The next-item prediction task becomes node-link prediction in the dynamic graph, enabling end-to-end model training.
  • Experiments on three real-world datasets verify DGSR's effectiveness and rationality.
Loading 2104.07368v2…