Source-linked AI summary
Memory Augmented Graph Neural Networks for Sequential Recommendation
Chen Ma, Liheng Ma, Yingxue Zhang, Jianing Sun, Xue Liu, Mark Coates
TL;DR
Sequential recommendation must use chronological interaction histories while modeling short-term interests, long-term dependencies, and item co-occurrence patterns. MA-GNN combines a GNN, shared memory network, and bilinear function for these factors, and experiments show significant improvements over state-of-the-art methods on five real-world datasets.
Problem
Sequential recommendation must model short-term interests, long-term interests, and item co-occurrence patterns from chronological interactions.
Method
MA-GNN uses a GNN for short-term item context, a shared memory network for long-range dependencies, and a bilinear function for item co-occurrence patterns.
Results
MA-GNN significantly outperforms state-of-the-art methods for sequential recommendation on five real-world datasets.
Takeaways & Limitations
The results validate MA-GNN’s performance advantages and demonstrate the effectiveness of its proposed modules for Top-K sequential recommendation.
Abstract
from arXiv · showhide
The chronological order of user-item interactions can reveal time-evolving and sequential user behaviors in many recommender systems. The items that users will interact with may depend on the items accessed in the past. However, the substantial increase of users and items makes sequential recommender systems still face non-trivial challenges: (1) the hardness of modeling the short-term user interests; (2) the difficulty of capturing the long-term user interests; (3) the effective modeling of item co-occurrence patterns. To tackle these challenges, we propose a memory augmented graph neural network (MA-GNN) to capture both the long- and short-term user interests. Specifically, we apply a graph neural network to model the item contextual information within a short-term period and utilize a shared memory network to capture the long-range dependencies between items. In addition to the modeling of user interests, we employ a bilinear function to capture the co-occurrence patterns of related items. We extensively evaluate our model on five real-world datasets, comparing with several state-of-the-art methods and using a variety of performance metrics. The experimental results demonstrate the effectiveness of our model for the task of Top-K sequential recommendation.
Introduction
Sequential recommendation exploits chronological user-item behavior but must capture short-term interests, long-term dependencies, and item co-occurrence patterns. MA-GNN addresses these factors with graph neural networks, memory, gating, and bilinear modeling.
- Sequential recommendation predicts future interactions from users’ chronological item-access histories.
- Short-term interests reflect recent preferences, long-term interests capture dependencies from earlier items, and co-occurrence patterns represent jointly appearing related items.
- Existing methods may model only short-term interests or omit explicit short-term-interest modeling, leaving important sequential factors insufficiently captured.
- MA-GNN combines general matrix-factorization interest modeling, GNN-based short-term aggregation, memory-based long-term modeling, and bilinear item co-occurrence modeling.
- A gating mechanism controls the contributions of long-term and short-term representations, while experiments evaluate MA-GNN on five real-world datasets against state-of-the-art methods.
Related Work
Sequential recommendation methods have progressed from Markov and similarity-based models to neural sequence models. MA-GNN differs by combining graph-based short-term context, external-memory long-term dependencies, and item co-occurrence modeling.
- Sequential recommendation uses chronological item sequences to model users’ future interactions.
- Classical approaches include Markov-chain models, while Fossil combines similarity-based modeling with high-order Markov chains.
- Neural approaches include Caser’s CNN processing of item-embedding sequences and later RNN-based methods.
- MA-GNN applies a graph neural network with external memories for short-term item context and long-term dependencies, adding an item co-occurrence module.
Problem Formulation
The paper formulates sequential recommendation from chronological implicit-feedback sequences. Given an earlier user-item subsequence, the task is to recommend K items from N candidates and assess whether future interactions appear in the recommendation list.
- Training data consist of each user’s chronological sequence of implicitly interacted items.
- Given an earlier subsequence for M users, the model recommends K items from N total items, with K < N.
- Recommendations are evaluated by whether the user’s subsequent sequence items appear in the recommended list.
Methodology
MA-GNN combines general, short-term, long-term, and item co-occurrence modeling for sequential recommendation. It uses GNN aggregation for recent item context, shared memory for historical interests, and a learnable gate to fuse interest representations.
- Model Overview: MA-GNN introduces four components: general interest, short-term interest, long-term interest, and item co-occurrence modeling.The general-interest component uses matrix factorization, while the other modules model sequential dynamics and related-item structure.
- Short-term Interest Modeling: A sliding window divides each user sequence into successive short-term subsequences, using recent items to predict the next target items.Each input contains |L| successive items and predicts the next |T| items; GNNs aggregate neighborhood information from the input subsequence.
- Short-term Interest Modeling: The item graph connects each item to several subsequent items, counts pairwise edges across users, and row-normalizes the resulting adjacency matrix.The experiments extract three subsequent items, allowing nearby sequence items to provide neighborhood information.
- Short-term Interest Modeling: A two-layer GNN aggregates neighboring item embeddings to form a short-term user-interest representation that summarizes locally relevant items.The resulting summary supports inference about items the user may access next.
- Long-term Interest Modeling: A shared memory network stores latent interest representations for all users, avoiding per-user memory overhead while modeling long-range dependencies.Multi-dimensional attention converts historical item embeddings into a query that selects an appropriate combination of shared memory units.
- Interest Fusion: The model uses a learnable gate to control the contributions of recent and long-term interest representations to the combined representation for item prediction.The gate follows the LSTM idea of balancing current inputs with historical hidden states; ⊙ denotes element-wise multiplication.
- Item Co-occurrence Modeling: A bilinear function explicitly models pairwise relations between items in the short-term sequence and other items through learnable feature-correlation parameters.The matrix W_r captures correlations between item latent features.
Evaluation
The study evaluates MA-GNN using five real-world datasets spanning multiple domains and sparsity levels, with chronological train, validation, and test splits.
- Datasets: Five datasets cover movies, books, CDs, children’s books, and comics across domains with different sparsities.The datasets are MovieLens-20M, Amazon-Books, Amazon-CDs, Goodreads-Children, and Goodreads-Comics.
- Preprocessing: Positive implicit feedback retains ratings of at least four out of five, while lower ratings are treated as missing entries.
- Data Splits: Each user’s earliest 70% of interactions forms training data, the next 10% validation data, and the remaining 20% test data.Testing includes interactions from both the training and validation sets in the input sequences.
- Evaluation Procedure: All models are trained five times, and average results are reported.
Evaluation Metrics
Performance is assessed with Recall@10 and NDCG@10, using top-10 recommendation quality and ranking position as complementary criteria.
- Metrics: Recall@10 measures the percentage of a user’s rated items appearing among the top 10 recommendations.
- Metrics: NDCG@10 measures normalized discounted cumulative gain at 10, accounting for the positions of correctly recommended items.
- Methods: The comparison includes matrix-factorization, recurrent, graph-contextualized, and other sequential recommendation methods.The listed baselines include BPRMF, GRU4Rec, GRU4Rec+, and GC-SAN.
- Reporting: Table 2 marks the best method in bold, the second-best value with underlining, and statistical significance with * for p <= 0.01 versus the best baseline.
Experiment Settings
The experiments compare MA-GNN with established recommenders under fixed latent dimensions and report broad performance comparisons across datasets and metrics.
- Experiment Settings: All models use a latent dimension of 50, while session-based methods treat items in a short-term window as one session.
- Performance Comparison: MA-GNN achieves the best performance on all five datasets and all evaluation metrics.
- Performance Comparison: MA-GNN outperforms SASRec, Caser, GC-SAN, and MARank, which the authors relate to modeling co-occurrence patterns or long-term item dependencies.
- Performance Comparison: MA-GNN obtains better results than GRU4Rec and GRU4Rec+, while outperforming BPRMF by incorporating sequential interaction patterns and short-term interests.
- Other Observations: Performance is better on MovieLens-20M and both Goodreads datasets than on the sparser Amazon datasets.The authors state that data sparsity negatively impacts recommendation performance.
- Ablation Analysis: The ablation analysis identifies short-term, long-term, and item co-occurrence modules as components evaluated within MA-GNN.
Ablation Analysis
The ablation study evaluates the contributions of short-term interest, long-term interest, fusion, and item co-occurrence modeling within MA-GNN.
- Short-Term Interest: Adding a conventional GNN short-term interest module to BPR matrix factorization slightly improves performance.
- Ablation Design: The ablation design begins with BPR matrix factorization and progressively incorporates short-term, long-term, fusion, and co-occurrence components.
- Interest Fusion: The proposed gating mechanism performs considerably better than concatenation or GRU for combining short-term and long-term interest representations.The authors interpret this as evidence that gating adaptively combines the two hidden representations.
- Item Co-occurrence: Adding item co-occurrence modeling further improves performance over the configuration combining short-term and long-term interests.
Influence of Hyper-parameters
The study examines how the multi-dimensional attention dimension h and memory-unit count m affect MA-GNN on the CDs and Comics datasets. Both components contribute to capturing long-term user interests, with larger performance gains on CDs.
- The multi-dimensional attention dimension h and memory-unit count m are investigated as important MA-GNN hyper-parameters on CDs and Comics.Their effects are examined in Figure 3.
- Both multi-dimensional attention and the memory network contribute to capturing long-term user interests.
- The two components produce larger performance improvements on CDs than on Comics.The authors suggest this difference may indicate help with data sparsity.
Memory Visualization
The memory visualization examines how memory units attend to different movies. Different attention patterns across movie types suggest that memory units may represent distinct types of user interest.
- The case study uses selected MovieLens movies as queries to visualize attention weights computed by Eq. 4.The number of memory units is set to 10.
- Memory units perform differently for different types of movies, suggesting that each unit may represent one type of user interest.
- The Three Colors trilogy receives similar memory-network attention-weight patterns across its three movies.The authors relate this similarity to the trilogy’s shared political-ideal basis.
- Die Hard has a distinct attention-weight pattern because it is an action thriller unlike the other case-study movies.
Conclusion
The paper concludes that MA-GNN combines graph-based short-term context modeling, memory-based long-range dependency modeling, and bilinear item-correlation modeling for sequential recommendation. Experiments on five real-world datasets report performance advantages over many state-of-the-art methods and support the effectiveness of the proposed modules.
- MA-GNN models items’ short-term contextual information with a GNN and long-range item dependencies with a memory network.
- A bilinear function models feature correlations between items alongside user-interest modeling.
- Experiments on five real-world datasets validate MA-GNN’s performance advantages over many state-of-the-art methods.
- The experimental results demonstrate the effectiveness of the proposed modules.