Source-linked AI summary
Recurrent Neural Networks with Top-k Gains for Session-based Recommendations
Balázs Hidasi, Alexandros Karatzoglou
TL;DR
Session-based recommenders often lack persistent user histories, so accurate next-item ranking must rely on current-session actions. This paper introduces RNN-tailored ranking losses and sampling refinements, achieving up to 35% gains over prior session-based RNN solutions and up to 53% over classical collaborative filtering.
Problem
Session-based recommendation must rank likely next items using current-session behavior when persistent user histories are unavailable.
Method
The paper analyzes and introduces RNN-tailored ranking losses, ranking-max methods, and additional negative sampling for session-based recommendations.
Results
Up to 35% improvement in MRR and Recall@20 over previous session-based RNN solutions and up to 53% over classical collaborative filtering were reported.
Takeaways & Limitations
The proposed losses and sampling strategy provide top-k gains for RNN session-based recommendations without significantly increasing training time.
Takeaways & Limitations
Mini-batch sampling can miss high-scoring negatives, couples accuracy and GPU speed to batch size, and popularity sampling may not suit every dataset.
Abstract
from arXiv · showhide
RNNs have been shown to be excellent models for sequential data and in particular for data that is generated by users in an session-based manner. The use of RNNs provides impressive performance benefits over classical methods in session-based recommendations. In this work we introduce novel ranking loss functions tailored to RNNs in the recommendation setting. The improved performance of these losses over alternatives, along with further tricks and refinements described in this work, allow for an overall improvement of up to 35% in terms of MRR and Recall@20 over previous session-based RNN solutions and up to 53% over classical collaborative filtering approaches. Unlike data augmentation-based improvements, our method does not increase training times significantly. We further demonstrate the performance gain of the RNN over baselines in an online A/B test.
1 INTRODUCTION
Session-based recommendation must work without persistent user histories, making current-session behavior central to accurate prediction. The paper studies RNN ranking losses to improve top-ranked recommendations over conventional methods.
- Session-based recommendation uses only actions within the current session when user history is unavailable.
- RNNs model the whole interaction session and can improve recommendation accuracy by 20%-30% over traditional methods.
- Top-ranked items matter most because users are unlikely to select items far down the recommendation list.
- Up to 35% improvement over previous session-based RNN losses and 53% over conventional memory-based collaborative filtering were reported for MRR and Recall@20.
- The work combines ranking-loss analysis with a new class of losses tailored to RNN-based session recommendations.
- The authors argue that the results demonstrate the potential of deep learning methods for recommender systems.
2 SAMPLING THE OUTPUT
GRU4Rec predicts the next session item from the current event while avoiding scores for the entire item catalog through mini-batch negative sampling. The paper extends sampling with additional negatives, balancing coverage, ranking relevance, and computational cost.
- GRU4Rec converts the current session event into a one-hot input and outputs scores representing each item's likelihood of being next.
- Computing scores for every catalog item is impractical, motivating negative sampling during training.
- Mini-batch sampling uses other examples in the same batch as negative items while benefiting from hardware parallelization.
- Ranking losses require high-scoring negative items because low-scoring negatives contribute little learning signal.
- Mini-batch sampling is restrictive because small batches may miss high-scoring negatives, batch size affects speed and accuracy, and popularity sampling may not suit every dataset.
- Additional samples supplement the batch-derived negatives, and their distribution can be chosen independently of the mini-batch.
- Popularity-based sampling uses α to interpolate between uniform sampling at α = 0 and popularity-based sampling at α = 1.
- Additional samples increase nominal complexity, but parallelization can avoid training-time increases up to a certain sample size on modern GPUs.
3 LOSS FUNCTION DESIGN
The paper analyzes ranking losses for RNN-based session recommendations, identifying vanishing-gradient problems as negative samples increase and introducing ranking-max losses that focus learning on the most relevant negatives. These designs preserve useful gradients with many samples while addressing numerical instability and TOP1’s sensitivity to high-scoring negatives.
- 3.1 Categorical cross-entropy: Cross-entropy was reported as slightly better than other losses but unstable across a large fraction of the hyperparameter space because limited precision can produce log 0.The paper stabilizes it by adding ϵ or computing the logarithm in an alternative form; the two variants showed no observed result differences.
- 3.2 Ranking losses: TOP1 & BPR: TOP1 combines pushing targets above negatives with penalizing high negative scores, while BPR maximizes the probability that target scores exceed negative-sample scores.Both are implemented as listwise losses formed from multiple pairwise comparisons.
- 3.2 Ranking losses: TOP1 & BPR: Ranking losses can stop learning when averaging over negative samples causes gradients to vanish as the sample count increases.Irrelevant negatives increasingly dominate the average, especially with nonpopularity-based sampling and large sample sets.
- 3.2 Ranking losses: TOP1 & BPR: TOP1 can respond undesirably when a popular negative scores far above a niche target, particularly early in training.This sensitivity follows from the TOP1 pairwise loss rather than its aggregation.
- 3.3 Ranking-max loss function family: Ranking-max losses compare the target with the highest-scoring negative, weighting individual pairwise losses toward relevant samples and ignoring low-scoring irrelevant samples.When all negatives are irrelevant, the gradient can become near zero because the target already exceeds every sample score.
- 3.3 Ranking-max loss function family: BPR-max retains slightly less gradient than BPR when many negatives remain relevant, but BPR’s gradient vanishes more quickly as the target reaches lower ranks.With small sample sets, BPR begins vanishing around rank 5, whereas BPR-max does not vanish until rank 0; with more samples, BPR’s gradient becomes very low at even lower ranks.
4 EXPERIMENTS
Experiments evaluate additional negative samples, ranking losses, unified item representations, and online performance across session-based recommendation datasets. The proposed changes improve accuracy over GRU4Rec and item-kNN while keeping sampling costs manageable and improving online recommendation KPIs.
- Experimental setup: Experiments evaluate fixed cross-entropy, ranking-max losses, and additional samples on four datasets, using next-item prediction with time-based session splits.VIDXL compares each target against the 50,000 most popular testing items, which overestimates performance but preserves fair algorithm comparisons.
- Using additional samples: Additional samples improve the alternative losses, while TOP1 degrades as irrelevant samples accumulate; BPR-max improves until using all items.Cross-entropy reaches diminishing returns after a few thousand extra samples, whereas TOP1-max begins to lose accuracy beyond that point.
- Using additional samples: 6–7 minutes is the full CLASS training time without extra samples, remaining 7–8 minutes at 2048 extra samples before increasing rapidly thereafter.The reported training-time trend reflects logarithmically plotted measurements and depends on dataset, model parameters, and framework support.
- Using additional samples: The sampling parameter α should be higher with small cross-entropy sample sizes and lower with large ones, while ranking-max losses favor intermediate values.The authors associate these preferences with popular-sample usefulness, pairwise-loss behavior, and score regularization.
- Loss-functions: 18–37.5% higher accuracy than original GRU4Rec and up to 55% higher than item-kNN are achieved by combining additional samples with improved losses.BPR-max matches or exceeds cross-entropy on four datasets, with a reported 2–6% improvement when extra samples are used.
- Unified item representations: Unified embeddings significantly improve CLASS recall@20 and MRR@20 by 18.74% and 29.44%, respectively, while reducing model size and memory footprint by approximately fourfold.Most other datasets show slight recall gains and slight MRR decreases.
5 CONCLUSION
The paper introduces new loss functions and an improved sampling strategy for top-k gains in session-based recommendation, with GRU4Rec outperforming previous solutions in a live online setting.
- New loss functions combined with improved sampling provide top-k gains for RNNs in session-based recommendations.
- The techniques may also apply to other recommendation algorithms and potentially to natural language processing.