Source-linked AI summary
Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding
Jiaxi Tang, Ke Wang
TL;DR
Top-N sequential recommendation must capture ordered, recent-item dependencies alongside general preferences, because existing methods can miss joint and skipped-item patterns. Caser embeds recent items into a time-by-latent-space image and learns sequential patterns with convolutional filters. Experiments on public datasets reported consistent outperformance over state-of-the-art sequential recommendation methods across common evaluation metrics.
Problem
Existing sequential recommendation approaches fail to adequately model union-level influences and skip behaviors, while general-preference systems overlook recent sequential dependencies.
Method
Caser represents the previous L items as an L × d embedding matrix and applies convolutional filters to learn sequential patterns while modeling user-specific features.
Results
Caser consistently outperformed state-of-the-art methods for top-N sequential recommendation on public real-life datasets.
Takeaways & Limitations
A unified network can capture general preferences together with point-level, union-level, and skip sequential patterns.
Takeaways & Limitations
Existing Markov-based approaches are limited because they fail to model union-level sequential patterns and skip behaviors.
Abstract
from arXiv · showhide
Top-$N$ sequential recommendation models each user as a sequence of items interacted in the past and aims to predict top-$N$ ranked items that a user will likely interact in a `near future'. The order of interaction implies that sequential patterns play an important role where more recent items in a sequence have a larger impact on the next item. In this paper, we propose a Convolutional Sequence Embedding Recommendation Model (\emph{Caser}) as a solution to address this requirement. The idea is to embed a sequence of recent items into an `image' in the time and latent spaces and learn sequential patterns as local features of the image using convolutional filters. This approach provides a unified and flexible network structure for capturing both general preferences and sequential patterns. The experiments on public datasets demonstrated that Caser consistently outperforms state-of-the-art sequential recommendation methods on a variety of common evaluation metrics.
1 INTRODUCTION
Top-N sequential recommendation must account for both users’ long-term preferences and short-term sequential patterns, including joint influences and skipped steps. Caser addresses these limitations by embedding recent items as an image and applying convolutional filters.
- General-preference recommenders can miss short-term dependencies, such as buying phone accessories soon after an iPhone.
- Top-N sequential recommendation models users as ordered item sequences to predict items likely to be interacted with in the near future.
- Existing Markov-based methods primarily aggregate point-level influences and do not adequately capture items that jointly influence a later action.
- Sequential association rules show that many patterns have orders L = 2 or L = 3, with a sizable number involving one- or two-step skips.The rules use minimum support count = 5 and minimum confidence = 50%.
- Caser represents the previous L items as an L × d embedding matrix and learns sequential patterns as local features using convolutional filters.
- Caser combines horizontal and vertical filters with personalization to capture point-level, union-level, skip, and general-preference signals in one framework.
2 FURTHER RELATED WORK
Prior recommendation methods include explicit pattern mining, neural models, and recurrent approaches, but they have limitations for sequential recommendation. The paper distinguishes this problem from temporal recommendation and illustrates Caser’s architecture.
- Conventional collaborative filtering, matrix factorization, and top-N methods do not model action order and therefore are not suited to capturing sequential patterns.
- Caser’s architecture comprises embedding lookup, convolutional layers, and fully connected layers, using item and user embeddings to learn sequential and personalized features.
- Explicit sequential association-rule mining can miss patterns in unobserved states and suffers from large search spaces, threshold sensitivity, and redundant rules.
- Earlier neural recommendation models using RBMs, auto-encoders, or CNNs were not designed for sequential recommendation.
- RNN-based session recommendation may be mismatched to sequential recommendation because adjacent actions do not necessarily have dependency relationships.
- Temporal recommendation uses time-of-day patterns, whereas top-N sequential recommendation models item-order dependencies independently of absolute time.
3 PROPOSED METHODOLOGY
Caser combines item and user embeddings with horizontal and vertical convolutional layers to model sequential patterns alongside general preferences. Its training and recommendation procedures use cross-entropy-based prediction over target items.
- Architecture: Caser combines embedding lookup, convolutional layers, and fully connected layers to learn user-specific and sequential features.The network uses item embeddings for recent history and a user embedding for user features.
- Embedding and convolution: The recent-item embeddings form an L × d latent-space image, where convolution filters detect sequential patterns as local features.Filters are learned jointly with the item embeddings rather than applied to a fixed input image.
- Horizontal convolution: Horizontal filters slide across successive items to capture union-level patterns, including ordered combinations of multiple items.Different filter heights support union-level patterns with different numbers of items.
- Vertical convolution: Vertical filters aggregate the previous items’ latent representations with learned weighted sums, producing multiple point-level sequential patterns.Each vertical filter acts as a different aggregator, unlike Fossil’s single weighted sum for each user.
- Prediction layer: The convolutional sequence embedding captures short-term sequential features, while the user embedding captures long-term general preferences before item scores are produced.The output layer has one node per item, and each output value represents the likelihood of interaction.
- Training and inference: Training transforms output values into probabilities with a sigmoid-based likelihood objective and minimizes the resulting binary cross-entropy loss.The model uses sampled negative instances, Adam optimization, L2 regularization, and dropout during training.
4 EXPERIMENTS
Caser is evaluated against sequential and non-sequential baselines on four public datasets using ranking metrics and controlled analyses of its hyperparameters and components. It generally improves over competing methods while demonstrating sensitivity to data sparsity, latent dimensions, recency weighting, and component combinations.
- Experimental Setup: The experiments compare Caser with six baselines on four public datasets using Precision@N, Recall@N, and MAP for N ∈{1, 5, 10}.The data include MovieLens, Gowalla, Foursquare, and Tmall; sequences are split into 70% training, 10% validation, and 20% test actions.
- Performance Comparison: Except for MovieLens, Caser improves the best baseline across all tested N values and metrics by a large margin.Sequential recommenders generally outperform BPR, while FPMC and Fossil outperform FMC, supporting the value of sequential information and personalization.
- Performance Comparison: On MovieLens, GRU4Rec performs close to Caser, but its performance is much worse on the other three datasets.The paper relates this difference to stronger sequential signals in MovieLens and GRU4Rec’s session-based, rather than personalized, recommendation setting.
- Hyperparameter Analysis: Caser’s performance depends on latent dimensionality: larger d can overfit on dense MovieLens, whereas sparse datasets require more dimensions.Across all datasets, Caser beats the strongest baseline with a relatively small number of latent dimensions.
- Hyperparameter Analysis: Figure 6 compares MAP across Markov orders L for Caser-1, Caser-2, and Caser-3, which use target counts T of 1, 2, and 3.The supplied figure description specifies the axes and the meaning of the three Caser variants, but not the observed performance trend.
- Component Analysis: Jointly enabling personalization, horizontal convolution, and vertical convolution gives the best component-ablation performance, denoted Caser-pvh.The reported results are shown for MovieLens and Gowalla, with similar results on the other two datasets.
- Network Visualization: Vertical filters assign greater weight to recent actions, while horizontal filters capture union-level sequential features from related prior items.On MovieLens with L = 9, vertical-filter weights show an ascending trend toward recent positions; masking related items lowers the target movie’s rank.
5 CONCLUSION
Caser addresses top-N sequential recommendation by representing recent actions as an image across time and latent dimensions and learning sequential patterns with convolutional filters. The resulting network captures both sequential patterns and long-term user preferences, and experiments suggested improved performance over state-of-the-art methods.
- Caser represents recent user actions as an image across time and latent dimensions, then learns sequential patterns with convolutional filters.
- The unified network captures point-level and union-level sequential patterns, skip behaviors, and long-term user preferences.
- Experiments and case studies on public real-life datasets suggested that Caser outperforms state-of-the-art methods for top-N sequential recommendation.