PaperlayerView exact revision on arXiv

Source-linked AI summary

Future Data Helps Training: Modeling Future Contexts for Session-based Recommendation

Fajie Yuan, Xiangnan He, Haochuan Jiang, Guibing Guo, Jian Xiong, Zhezhao Xu, Yilin Xiong

arXiv:1906.04473v4cs.IRcs.LG

TL;DR

Past-only session recommendation can miss future interactions that also signal preference, while directly using future data risks leakage. GRec trains an encoder-decoder through gap-filling to model such contexts; the paper reports stronger performance than traditional unidirectional methods in controlled experiments and ablations.

  • Problem

    Left-to-right session recommenders use only past behaviors, although future interactions can also provide evidence about user preference, and using them directly risks data leakage.

    Source: §1 · p. 2

  • Method

    GRec masks items in a session with gaps, encodes the incomplete sequence, and trains a decoder to predict the masked items from the representation.

    Source: §1 · p. 2

  • Results

    GRec is more powerful than traditional unidirectional recommendation methods in the paper’s ablations and controlled experiments.

    Source: §5 · p. 10

  • Takeaways & Limitations

    The paper concludes that GRec jointly models future contexts while maintaining an autoregressive generation mechanism.

    Source: §5 · p. 10

    Two-way NextItNets cannot use future contexts during inference, and the training–prediction discrepancy may hurt recommendation performance.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

Session-based recommender systems have attracted much attention recently. To capture the sequential dependencies, existing methods resort either to data augmentation techniques or left-to-right style autoregressive training.Since these methods are aimed to model the sequential nature of user behaviors, they ignore the future data of a target interaction when constructing the prediction model for it. However, we argue that the future interactions after a target interaction, which are also available during training, provide valuable signal on user preference and can be used to enhance the recommendation quality. Properly integrating future data into model training, however, is non-trivial to achieve, since it disobeys machine learning principles and can easily cause data leakage. To this end, we propose a new encoder-decoder framework named Gap-filling based Recommender (GRec), which trains the encoder and decoder by a gap-filling mechanism. Specifically, the encoder takes a partially-complete session sequence (where some items are masked by purpose) as input, and the decoder predicts these masked items conditioned on the encoded representation. We instantiate the general GRec framework using convolutional neural network with sparse kernels, giving consideration to both accuracy and efficiency. We conduct experiments on two real-world datasets covering short-, medium-, and long-range user sessions, showing that GRec significantly outperforms the state-of-the-art sequential recommendation methods. More empirical studies verify the high utility of modeling future contexts under our GRec framework.

1 INTRODUCTION

Existing session-based recommenders use past interactions to predict a target, although later interactions can also signal user preference. GRec uses gap-filling to model future context without data leakage.

1 INTRODUCTION

  • The introduction frames long-range session dependencies as a challenge, noting that such sessions occur in micro-video and news recommendation scenarios.It gives TikTok viewing as an example of many short interactions within a session.Source: §1 · p. 1
  • Future interactions can evidence user preference, but incorporating them into training risks data leakage when targets also appear in encoder inputs.The paper notes that session behaviors need not have a strict sequential dependency, using phone accessories as an example.Source: §1 · p. 2
  • GRec jointly trains an encoder and decoder by masking session items with gaps, encoding the incomplete sequence, and predicting the masked items.The decoder conditions on the encoded representation, which includes unmasked actions representing general user preference.Source: §1 · p. 2
  • The proposed framework is instantiated with convolutional neural networks and sparse kernels, and its decoder uses an inverted-bottleneck projector to increase representational bandwidth.The projector is described as a contribution alongside experiments on two real-world datasets.Source: §1 · p. 2

2 PRELIMINARIES

This section defines top-N session-based recommendation and reviews left-to-right training approaches alongside related neural recommendation models. It positions GRec as a partial-sequence encoder-decoder approach that combines two-sided context encoding with autoregressive generation.

2.1 Top-N Session-based Recommendation

  • Top-N session-based recommendation predicts the next item from a prefix of chronologically ordered session items and returns the highest-scoring items.A session may contain items such as videos, songs, or queries consumed within a period.Source: §2.1 · p. 2

2.2 The Left-to-Right-style Algorithms

  • Data augmentation trains on session prefixes as separate sequences, while autoregressive models predict each item from its preceding items.Augmentation can reduce overfitting for longer, sparse sessions, but may break session integrity and increase training time; autoregressive training avoids augmentation.Source: §2.2.1 · p. 3, §2.2.2 · p. 3
  • Left-to-right methods omit future context, which may matter because recommendation behaviors need not follow rigid order and additional context may help with sparsity.The section also reviews RNN, CNN, and attention-based approaches, noting parallelization advantages of CNNs and the quadratic complexity of self-attention.Source: §2.2.2 · p. 3, §2.3 · pp. 3–4

2.3 Related Work

  • GRec is presented as a partial-sequence-to-partial-sequence encoder-decoder model, unlike NARM’s sequence-to-one decoder for the final item.Its encoder and decoder use incomplete sequences to combine two-sided contexts with autoregressive sequence generation.Source: §2.3 · p. 4

3 METHODOLOGIES

The methodology examines ways to incorporate future session context, then introduces GRec, which trains an encoder and decoder to fill masked items without data leakage. Its convolutional design combines bidirectional encoding with autoregressive generation.

3.1 Two-way Data Augmentation

  • Two-way data augmentation reverses the sequence to use future context, but shares parameters across directions and trains them separately, which can yield suboptimal performance.The proposed alternative is one objective with separate parameter sets for left and right contexts.Source: §3.1 · p. 4

3 METHODOLOGIES

  • Reversing the input and training with reversed outputs is a straightforward way to add future context; the methodology then develops GRec using NextItNet’s dilated CNNs for accuracy and efficiency.The paper first analyzes simpler approaches to future context before presenting its gap-filling encoder-decoder framework.Source: §3.1 · p. 4, §3 · p. 4

3.2 Two-way NextItNets (tNextItNets)

  • Two-way NextItNets combine forward and backward predictions, but their future-context branch is unavailable at inference and the training–prediction mismatch may hurt recommendation performance.The two directions are also a shallow concatenation of independently trained models, limiting their ability to represent complex contextual representations.Source: §3.2 · pp. 4–5

3.3.1 Seq2seq for SRS.

  • For seq2seq learning in SRS, the input session and output sequence have equal lengths, and the output items correspond to the input session items.The model learns the conditional probability of the output sequence given the input session.Source: §3.3.1 · p. 5

3.3.2 General Framework of Pseq2pseq.

  • GRec uses gap-filling to train an encoder and decoder jointly, predicting randomly masked session items from a partially observed sequence without data leakage.The encoder is bidirectional, while the decoder is causal and predicts the masked items.Source: §3.3.2 · pp. 5–6

3.3.3 GRec Architecture.

  • GRec’s bidirectional CNN encoder sees left and right contexts around gaps, while its causal CNN decoder attends only leftward and estimates masked items.Gap positions are randomly selected anew across training batches; predicting only masked items requires more training steps to converge than NextItNet-style models, but fewer than left-to-right data-augmentation models such as Caser.Source: §3.3.3 · pp. 6–7
  • The inverted-bottleneck projector expands and then restores channel dimensions to connect encoder representations with decoder embeddings.Direct element-wise addition may not yield the best results; the projector uses a skip connection and expands to f = 2d in this paper.Source: §3.3.3 · p. 6

3.3.4 Connection to Existing Models.

  • GRec jointly models past and future contexts, retains autoregressive generation unlike BERT-style models, and avoids the data leakage of standard seq2seq models on the same input.With one masked position it becomes a sequence-to-one model; with all but the first position masked, its encoder is almost ineffective and the model is likely to perform worse than NextItNet.Source: §3.3.4 · p. 7

4 EXPERIMENTS

The experiments evaluate GRec against sequential recommendation baselines across short-, medium-, and long-range sessions, then test gap-filling choices, key modules, and recurrent variants. GRec outperforms NextItNet, with its best reported gap-filling percentage between 30% and 50%.

4.1 Experimental Settings

  • On two real-world datasets with three different session lengths, experiments evaluate each test session’s next item using MRR@N, HR@N, and NDCG@N, with N=5 and 20.The datasets are TW10, ML30, and ML100. User sequences are randomly split into 80% training, 10% validation, and 10% testing.Source: §4.1.1 · p. 7, §4.1.2 · p. 8

4.2 Performance Comparison (RQ1)

  • GRec with the pseq2pseq structure significantly exceeds NextItNet, while NextItNet+ and tNextItNet perform worse than NextItNet.NextItNet also outperforms Caser and GRU4Rec across all three datasets.Source: §4.2 · pp. 8–9, Table 2 · p. 8
  • GRec converges more slowly at first, but performs better after more training epochs, and its gains outweigh the marginally increased training cost.On TW10, NextItNet's converged results are plotted after epoch 20.Source: §4.2 · p. 9, Figure 5 · p. 9

4.3 Impact of the Gap-filling Strategies (RQ2)

  • The highest recommendation accuracy occurs with 30%–50% of session items masked; larger or smaller gap-filling percentages typically yield suboptimal performance.Masking too much can discard future context and add noise, while masking too little weakens the autoregressive property.Source: §4.3 · pp. 9–10

4.4 Ablation Studies (RQ3)

  • The projector improves GRec, but GRec still exceeds NextItNet with or without the projector; full GRec also largely exceeds its encoder-only variant.The encoder-only model uses bidirectional context but does not explicitly model previous interactions' sequential patterns over time.Source: Table 3 · p. 9, §4.4 · p. 10, Table 4 · p. 10
  • With embedding dimension d=512, GRec still significantly improves over NextItNet, alongside much longer training time and larger memory consumption.The reported comparison uses MRR@5.Source: §4.4 · p. 10, Table 5 · p. 10

4.5 GRec Variants (RQ4)

  • Convolutional GRec generally outperforms variants using a recurrent encoder or decoder, while those recurrent variants still exceed their corresponding NextItNet or GRU baselines.The tested recurrent variants use a Bi-GRU encoder or a GRU decoder.Source: §4.5 · p. 10, Table 6 · p. 10, Figure 7 · p. 10

5 CONCLUSION

The paper studies incorporating future contexts into left-to-right session recommendation and presents GRec, a gap-filling encoder-decoder framework that jointly trains on the same action sequence without data leakage.

5 CONCLUSION

  • GRec uses a gap-filling optimization objective in a pseq2pseq encoder-decoder framework to incorporate future contexts into session recommendation.The paper describes the framework as general and flexible, jointly training encoder and decoder on the same user action sequence without data leakage.Source: §5 · p. 10
  • The study targets left-to-right recommendation architectures that fail to model past and future contexts simultaneously.Source: §5 · p. 10
  • Ablations and controlled experiments show that GRec is more powerful than traditional unidirectional models.Source: §5 · p. 10
  • Future work will study whether right contexts or GRec can improve recommendation diversity for session-based recommendation.Source: §5 · p. 10

Related papers

Loading 1906.04473v4…