PaperlayerView exact revision on arXiv

Source-linked AI summary

No Attention, No Problem: Rethinking Session-based Recommendation with Pure Convolution

Tao Huang, Wei Zhou

arXiv:2609.34802v1cs.IR

TL;DR

Traditional convolutional models offer efficiency but have weaker global modeling than Transformers in session-based recommendation. NextConvRec combines graph- and position-aware preprocessing with modern convolutional modules, and achieves Transformer-comparable recommendation performance with improved inference efficiency on four datasets.

Abstract

from arXiv · show

Session-based recommendation (SBR) predicts the next choice in a session by analyzing recent interactions. Transformer-based models are widely used because of their ability to capture long-range dependencies through self-attention mechanisms. In contrast, traditional convolutional models, although more efficient, are often limited by their weak global modeling capabilities and are losing ground in SBR tasks. In this work, we propose a Next-generation Pure Convolutional Framework (NextConvRec) for SBR tasks, aiming to balance efficiency and performance. NextConvRec uses a Structural and Positional Convolutional Encoder (SPCE) for preprocessing, combining learnable convolutional positional biases with session-level structural signals extracted through GCN layers. Its backbone convolutional module effectively expands the effective receptive field through depthwise convolutions and pointwise convolutions, enabling robust long-range preference modeling without attention mechanisms. Extensive experiments on 4 benchmark datasets show that NextConvRec outperforms several state-of-the-art baselines by around 1.73% on average, and reduces the average inference time per session by 16.7%. The convolutional architectures remain a promising direction for efficient and accurate session-based recommendations.

Introduction

Transformer models have displaced traditional convolutional models in session-based recommendation (SBR), despite convolution’s efficiency and local-pattern strengths. NextConvRec adapts modern convolution and adds SPCE preprocessing to balance recommendation performance with inference efficiency.

Introduction

  • Modern convolution enlarges the effective receptive field to improve context modeling in SBR.Modern convolution adopts design elements from Transformers while replacing multi-head attention blocks with convolutional blocks.Source: Figure 1 · p. 1, Introduction · p. 2
  • Traditional convolutional models have lost prominence in SBR because Transformers’ global effective receptive field better captures global session patterns.Convolutional models retain efficient parallelization and local-pattern modeling, but have been limited by their effective receptive field.Source: Introduction · pp. 1–2
  • Earlier SBR convolutional work includes Caser, NextItNet, and 3D-CNN, but CNN application in SBR remains limited.NextItNet uses deep one-dimensional convolutional stacks and dilated convolutions to expand the receptive field while retaining parallel computation.Source: Introduction · p. 1
  • NextConvRec rivals state-of-the-art Transformer models in recommendation performance while significantly outperforming them in inference efficiency across four benchmark datasets.The framework is designed to balance performance and efficient inference.Source: Introduction · p. 2
  • NextConvRec combines modern convolutional techniques with SPCE preprocessing that integrates graph structure and positional awareness for convolutional modeling.SPCE is designed to help modern convolutional structures capture long-range preferences.Source: Introduction · p. 2

Related Work

Related work spans session-based recommendation models, Transformer advances, and modern convolution techniques. The paper positions its contribution at the intersection of modern convolution and the comparatively limited use of updated convolutional models in SBR.

Session-based Recommendation

  • SBR methods include Markov-chain, temporal, CNN, MLP, large-language, and diffusion approaches for next-item recommendation from anonymous session behavior.Examples include RNN-based GRURec, NARM, and RepeatNet; CNN-based Caser, NextItNet, and 3D-CNN; and MLP-based FMLP4Rec and TriMLP.Source: Session-based Recommendation · p. 2

Transformer-based Method

  • Transformer-based models such as SASRec and BERT4Rec established a paradigm based on self-attention and are reported to outperform convolutional models through stronger global modeling.Later variants add dual sparse attention, filters, causal sequence recommendation, or item image information.Source: Transformer-based Method · p. 2

Modern Convolution

  • Modern convolution techniques improve modeling through structural modifications and have spread across vision, time-series prediction, and GANs, while SBR convolution has lacked updates.The paper aims to address this research gap in session-based recommendation.Source: Modern Convolution · pp. 2–3

Preliminaries

The preliminaries define SBR as predicting a session’s next item from its interaction sequence, then describe session and relationship graphs and convolutional sequence processing. They establish graph structure and convolutional receptive fields as relevant representations for session recommendation.

Problem Definition

  • SBR predicts the most probable next item from the ordered items in the current session, using only that session’s interaction sequence.The session is an ordered sequence of interacted items, and recommendations are presented as a ranked list of candidate items.Source: Problem Definition · p. 3

Modeling Session and Cross-session Graphs

  • A session graph represents unique session items as nodes and adjacent-item click order as directed edges, while a relationship graph links sessions sharing at least one item.Relationship-graph edge weights represent the closeness of potential interest patterns between sessions.Source: Modeling Session and Cross-session… · p. 3

Convolution for Session-based Recommendation

  • Convolution processes a session matrix along its sequence dimension to encode adjacent interactions; depthwise convolution captures temporal dependencies and pointwise convolution fuses channels.Stacking n convolutional modules gives an approximate effective receptive field of 1 + n · (k − 1), allowing longer dependencies to be captured with low computational complexity.Source: Convolution for Session-based… · p. 3

Proposed Method

NextConvRec is a pure-convolution SBR framework with an SPCE embedding preprocessor and a backbone convolutional module. It combines graph structure and position-aware convolution to build session representations for item scoring.

Embedding Layer

  • NextConvRec embeds a zero-padded session sequence with item vectors from E to form Xu.Short sessions are padded to the maximum length L; padded actions use zero vectors and are excluded from gradient updates.Source: Embedding Layer · p. 3

Structural and Positional Convolutional Encoder

  • SPCE combines GCN-derived session structure with learnable position encodings and depthwise-convolution local patterns into the representation passed downstream.The GCN extracts higher-level user-item co-occurrence patterns, and the resulting representation feeds the NextConvRec Block.Source: Structural and Positional… · pp. 3–4

Next-generation Convolutional Block

  • The NextConvRec Block replaces multi-head attention with depthwise and pointwise convolutions, using a large kernel to expand ERF and model temporal, feature, and session dependencies.Its ConvFFN(B) and ConvFFN(D) layers learn dependencies across channels and sessions, respectively.Source: Next-generation Convolutional Block · p. 4

Overall Structure

  • Stacked residual NextConvRec blocks transform the SPCE embedding across time, variables, and channels into the final representation Y.The number of stacked layers is set to K.Source: Overall Structure · p. 5

Prediction Layer

  • The prediction layer scores items by dot-product similarity to the user's final representation and trains with cross-entropy loss.The ground-truth next item is used for training.Source: Prediction Layer · p. 5

Experiments and Analysis

Experiments across four public datasets compare NextConvRec with established recommendation baselines, test its components and hyperparameters, and assess inference and convergence efficiency.

Experimental Setup

  • Evaluation spans Beauty, Sports, Toys, and Yelp, using the last session item for testing, the second-to-last for validation, and HR/NDCG at K={5,20}.The datasets are used to evaluate performance across differing sparsity levels and domains.Source: Experimental Setup · p. 5

Overall Performance Analysis (RQ1)

  • Across four datasets, NextConvRec surpasses BSARec in three; on Yelp, HR@20 and NDCG@5 improve by 1.2% and 6.7%, respectively.In Sports, NextConvRec's HR@20 is slightly below BSARec, while other evaluation metrics improve.Source: Table 2 · p. 6, Overall Performance Analysis (RQ1) · p. 6

Ablation Studies (RQ2)

  • 2.3% lower Beauty HR@20 follows PosRes removal; graph-convolution preprocessing is crucial, and replacing the convolutional backbone with MHSA reduces performance and inference speed.The ablation comparison covers Beauty, Sports, and Yelp.Source: Ablation Studies (RQ2) · pp. 6–7, Table 3 · p. 6

Hyperparameter Sensitivity Analysis (RQ3)

  • 7 and 25 are optimal small/large kernel sizes on Beauty, versus 5 and 31 on Toys; increasing the large kernel can compensate for global-modeling limits.The two kernel sizes are selected jointly to balance local feature extraction and global receptive-field control.Source: Hyperparameter Sensitivity Analysis… · p. 7, Figure 3 · p. 7
  • Across tested FFN ratios {1,2,4,8}, NDCG@20 peaked at {2,4} and HR@20 at 8; too-low ratios limit expressive power, while too-high ratios may cause overfitting or gradient instability and affect inference speed.The FFN ratio controls nonlinear mapping capacity in each convolutional block.Source: Hyperparameter Sensitivity Analysis… · p. 7

Efficiency and Complexity Analysis (RQ4)

  • 0.22 ms/session yields NDCG@20 0.0683; NextConvRec improves inference efficiency over BSARec by approximately 16.7% and reaches its performance limit in about 22 epochs.Figure 5 analyzes inference speed and NDCG@20 on Beauty; its convergence curves track NDCG@20 over training epochs.Source: Figure 5 · p. 7, Efficiency and Complexity Analysis (RQ4) · p. 7

Conclusion and Future Work

NextConvRec combines graph structure and position-aware preprocessing with modern convolutions to address convolutional models’ weaker global modeling in session-based recommendation. Across four datasets, it achieves performance comparable to Transformers while maintaining efficiency, and the authors identify lighter variants for future work.

Conclusion and Future Work

  • Across four datasets, NextConvRec achieves performance comparable to Transformer models while maintaining efficiency and significantly improving inference speed.The authors propose exploring lighter, more efficient convolutional variants to further reduce computational complexity and adapt to large-scale session data.Source: Conclusion and Future Work · p. 7
  • NextConvRec combines graph structures and position-aware mechanisms in feature preprocessing, then uses modern convolutional techniques to expand the effective receptive field and enhance global modeling.Source: Conclusion and Future Work · p. 7
  • The authors identify lighter, more efficient convolutional variants as future work to further reduce computational complexity and adapt to large-scale session data.Source: Conclusion and Future Work · p. 7
Loading 2609.34802v1…