PaperlayerView exact revision on arXiv

Source-linked AI summary

A Simple Convolutional Generative Network for Next Item Recommendation

Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M Jose, Xiangnan He

arXiv:1808.05163v4cs.IRcs.LGstat.ML

TL;DR

The paper addresses limitations of session-based CNN recommenders in modeling long-range dependencies and using sequence-wide conditional information. It introduces a generative model built from masked, dilated 1D convolutions and residual learning, and reports state-of-the-art accuracy with less training time.

  • Problem

    Typical session-based CNN recommenders can lose positional information through max pooling, use shallow networks, and estimate only the desired item, limiting long-range sequence modeling.

  • Method

    The model uses masked 1D dilated convolutional layers, predicts distributions for all sequence items, removes pooling, and applies residual learning for deeper networks.

  • Results

    NextItNet largely outperforms Caser and GRURec, achieves a best MUSIC_M5 MRR@20 of 0.3223, and trains faster than both comparison models.

  • Takeaways & Limitations

    The model provides a generic approach for representing both short- and long-range dependencies in session-based recommendation sequences.

  • Takeaways & Limitations

    Caser’s max pooling and shallow architecture are identified as limitations for modeling complex relations and long-range dependencies.

Abstract

from arXiv · show

Convolutional Neural Networks (CNNs) have been recently introduced in the domain of session-based next item recommendation. An ordered collection of past items the user has interacted with in a session (or sequence) are embedded into a 2-dimensional latent matrix, and treated as an image. The convolution and pooling operations are then applied to the mapped item embeddings. In this paper, we first examine the typical session-based CNN recommender and show that both the generative model and network architecture are suboptimal when modeling long-range dependencies in the item sequence. To address the issues, we introduce a simple, but very effective generative model that is capable of learning high-level representation from both short- and long-range item dependencies. The network architecture of the proposed model is formed of a stack of \emph{holed} convolutional layers, which can efficiently increase the receptive fields without relying on the pooling operation. Another contribution is the effective use of residual block structure in recommender systems, which can ease the optimization for much deeper networks. The proposed generative model attains state-of-the-art accuracy with less training time in the next item recommendation task. It accordingly can be used as a powerful recommendation baseline to beat in future, especially when there are long sequences of user feedback.

1 INTRODUCTION

Session-based recommendation exploits correlations in ordered user-item interactions, but RNNs limit parallel computation and typical CNN designs lose important long-range sequence information. The paper addresses these issues with a generative CNN model that explicitly uses the complete sequence of dependencies.

  • Motivation: User-item interaction sessions contain sequential patterns that can inform next-item recommendations.Examples include songs or videos sharing an album, writer, or genre.
  • Motivation: RNNs depend on the hidden state of the entire past, limiting parallel computation during training and evaluation.
  • CNN-based recommendation: Caser replaces recurrent processing with CNN-based sequence embeddings and achieves comparable or superior top-N sequential recommendation performance.
  • Limitations: Typical Caser-style architectures can discard positional and recurrent signals through max pooling and underuse dependency information when sequences become long.The paper identifies these drawbacks as increasingly severe for longer sessions and sequences.

2 PRELIMINARIES

The preliminaries define top-N session recommendation and review sequence-modeling approaches, highlighting limitations of Caser and the relatively limited exploration of CNN architectures for sequential patterns.

  • Top-N Session-based Recommendation: Top-N session recommendation ranks candidate items from a distribution generated from a prefix of the user-item interaction sequence.The top-N items are selected from scores, probabilities, or ranks assigned to candidate next items.
  • Caser: Caser embeds previous items as rows of a t × k latent matrix and treats the matrix as an image for CNN processing.
  • Caser: Caser uses filters spanning the embedding width, pools variable-length feature maps, and reduces each map to a 1 × 1 map.
  • Limitations of Caser: Max pooling can ignore feature position and multiplicity, making it potentially harmful for long-range sequence modeling.
  • Limitations of Caser: Caser’s shallow structure is likely to fail on complex relations or long-range dependencies, while its next-item generation process has an additional disadvantage.
  • Related Work: CNN-based sequential recommendation remains less explored than recurrent approaches because convolutions are not a natural way to capture sequential patterns.

3 MODEL DESIGN

The proposed model is a deep 1D convolutional generative architecture that models all item distributions, uses dilated convolutions, and removes pooling to address Caser’s limitations.

  • Model Design: The model estimates distributions for all individual items rather than only the final item in a sequence.
  • Model Design: Its deep architecture uses efficient 1D dilated convolutions instead of standard 2D convolutions.
  • Model Design: Pooling layers are removed from the proposed architecture.

3.1 A Simple Generative Model

The paper models the full interaction sequence by factorizing its joint distribution into conditional item probabilities, estimated with a stack of 1D convolutional networks. Unlike methods that predict only one next item or separately optimized sub-sessions, the proposed model estimates all item-wise conditionals and captures internal sequence relations.

  • Generative formulation: The model factorizes the joint distribution of an item sequence into conditional probabilities using the chain rule.Each conditional probability represents an item given all preceding items.
  • Generative formulation: A stack of 1D convolutional networks receives previous items and models the conditional distributions of user-item interactions.
  • Comparison with prior methods: Unlike Caser and GRURec, the proposed method estimates distributions for all individual items in the sequence rather than only the next item.For a given sequence ending at x15, baseline methods estimate x15, whereas the proposed method estimates x1 through x15.
  • Comparison with prior methods: Modeling all sequence relations explicitly captures internal sequence features that Caser and GRURec do not explicitly model.Those baselines typically create sub-sequences through padding, splitting, or shifting for training.
  • Limitations of sub-session training: Separately optimizing each generated sub-session cannot guarantee optimal results and introduces additional computational costs.

3.2 Network Architecture

The architecture uses stacked one-dimensional dilated convolutions over reshaped item embeddings to model both local and long-range dependencies efficiently. Dilation expands receptive fields exponentially without pooling, larger filters, or excessive depth.

  • Input representation: The model embeds the first t items into a t × 2k matrix and applies one-dimensional convolutional filters rather than treating it as a 2D image.The 2D matrix is reshaped into a 1 × t × 2k tensor, with 2k treated as the channel dimension.
  • Dilated layer: Dilated convolution enlarges the filter’s receptive field by inserting zeros, requiring fewer parameters than an equivalently wide standard filter.The dilated filter is therefore also called a holed or sparse filter.
  • Dilated layer: Dilated convolutions provide exponential receptive-field growth, r = 2^j+1 − 1, compared with linear growth, r = 2j + 1, for standard convolutions.This structure is intended to handle long-range sequences without larger filters or substantially deeper networks.
  • Dilated layer: Repeating dilation factors such as 1, 2, 4, 8 increases the model’s capacity and receptive fields through stacked architectural blocks.The architecture can repeat this dilation pattern multiple times.
  • One-dimensional transformation: Each dilated residual block reshapes the embedding tensor before its 1 × 1 and masked 1 × 3 convolutions, then reshapes it back afterward.The default stride for dilated convolution is 1.

3.3 Masked Convolutional Residual Network

The network combines residual blocks with dilated convolutions to support deeper recommendation models while preserving information and preventing future-item leakage. Skip connections fit residual mappings, and masking ensures predictions use only prior items.

  • Residual learning: Residual learning addresses degradation and vanishing-gradient difficulties that can arise when increasing network depth.Residual blocks were introduced to make deep-network optimization easier.
  • Residual learning: A residual block learns F(E) = H(E) − E and reconstructs the desired mapping as F(E) + E through a skip connection.The skip connection passes earlier feature information to later layers and enlarges propagated gradients.
  • Residual block design: The architecture wraps either each dilated layer or every two dilated layers in residual blocks containing normalization, activation, convolutional layers, and skip connections.The second design connects the input layer to the second convolutional layer.
  • Residual block design: Using 1 × 1 filters changes the channel size from 2k to k before the 1 × 3 filter and restores it to 2k afterward.This reduces the parameters learned by the 1 × 3 kernel while preserving dimensions for subsequent stacking.
  • Residual block design: The residual mapping uses ReLU, layer normalization, standard 1 × 1 convolutions, and l-dilated 1 × 3 filters.Bias terms are omitted from the formulation for simplicity.
  • Masked convolution: Masking prevents the convolution from using xi:t when predicting p(xi | x0:i−1), avoiding leakage from future items.The masking can be implemented through input padding or by shifting the output sequence.

3.4 Final Layer, Network Training and Generating

The final layer converts convolutional representations into item distributions at every sequence position, while training maximizes sequence likelihood and generation proceeds one next item at a time. Padding-based masking enforces the past-only prediction constraint.

  • Final layer: A 1 × 1 × 2k × n convolution converts the final representation into Ep ∈ R^t×n, whose softmax rows give categorical distributions over sequence items.The distribution for xt supplies the top-N recommendation probabilities.
  • Network training: Training maximizes the log-likelihood of the sequence, equivalently minimizing the sum of binary cross-entropy losses across x1:t.For very large item sets, negative sampling can replace full-softmax generation.
  • Masked generation: The future item is predicted only from past items, with padding used to mask unavailable future information during convolution.For example, items {1, 2, 3, 4} are masked when predicting item 1.
  • Network training: Sampled softmax and kernel-based sampling achieve nearly identical recommendation accuracy to full softmax when the sampling size is properly tuned.These strategies reduce the cost of generating distributions over tens of millions of items.
  • Generating: Evaluation predicts one next item, whereas generation can continue sequentially by feeding each predicted item back into the network.This matches scenarios in which the next action follows observation of the current item.

4 EXPERIMENTS

Experiments compare NextItNet with GRURec and Caser across short- and long-range session datasets using top-N next-item metrics. NextItNet achieves strong accuracy and faster training, with evaluations covering preprocessing, session length, residual blocks, embedding size, and runtime.

  • Datasets and baselines: The experiments compare NextItNet against GRURec and Caser on short- and long-range sequential recommendation datasets.The evaluation omits content- and context-based models because the paper focuses on sequential modeling.
  • Evaluation protocols: Evaluation predicts each test sequence’s next item using MRR@N, HR@N, and NDCG@N at N=5 and 20.The reported metrics evaluate the ranking of the final item in each testing sequence.
  • Accuracy results: On MUSIC_M5, the three neural models perform more than 120 times better than MostPop on MRR@5, while NextItNet reaches MRR@20 of 0.3223.The 0.3223 result is reported for NextItNet among 20,000 candidate items.
  • Accuracy results: NextItNet outperforms Caser and GRURec because it leverages complete sequential information and avoids pooling, which preserves the embedding matrix’s spatial resolution.Caser and GRURec remain worse even with manually created sub-sessions, whereas their performance degrades especially on long sessions without them.
  • Efficiency results: NextItNet and Caser require less training time than GRURec, and NextItNet further improves on Caser by converging in fewer training epochs.The paper attributes CNN speed to parallel convolution and NextItNet’s additional advantage to using complete session information during training.

5 CONCLUSION

The paper presents a convolutional generative model for session-based top-N recommendation that combines masked filters, dilated convolutions, and residual learning. It reports strong performance while leaving the evaluation of additional contextual information for future work.

  • The model combines masked filters with 1D dilated convolutions to capture both short- and long-range session dependencies.Dilated convolutions increase receptive fields, while masked filters preserve the sequential generation structure.
  • Residual learning enables optimization of much deeper recommendation networks.
  • The model greatly outperforms state-of-the-art methods on real-world session-based recommendation tasks.
  • Additional contexts such as user identity and location were excluded from the comparisons, although the model can incorporate them.The paper leaves evaluation with these contexts for future work.
Loading 1808.05163v4…