Source-linked AI summary
Efficiently Leveraging Multi-level User Intent for Session-based Recommendation via Atten-Mixer Network
Peiyan Zhang, Jiayan Guo, Chaozhuo Li, Yueqi Xie, Jaeboum Kim, Yan Zhang, Xing Xie, Haohan Wang, Sunghun Kim
TL;DR
Session-based recommendation must predict the next action from short, dynamic sessions, but increasingly complex GNN propagations often add marginal gains. The paper proposes Atten-Mixer, which shifts reasoning toward multi-level concept- and instance-view readouts, and reports improved benchmark and live-system performance with lower complexity.
Problem
Complex GNN-based SBR models achieve relatively marginal improvements despite exponential growth in model complexity.
Method
Atten-Mixer removes GNN propagation and combines concept-view and instance-view readouts, using local invariance and inherent priority to prune the reasoning search space.
Results
Atten-Mixer achieves better empirical performance than attention-based and other state-of-the-art SBR models, with shorter processing time and deployment-scale improvements.
Takeaways & Limitations
Multi-level user intent can support efficient reasoning over item transitions without enhancing item embeddings through GNNs.
Abstract
from arXiv · showhide
Session-based recommendation (SBR) aims to predict the user's next action based on short and dynamic sessions. Recently, there has been an increasing interest in utilizing various elaborately designed graph neural networks (GNNs) to capture the pair-wise relationships among items, seemingly suggesting the design of more complicated models is the panacea for improving the empirical performance. However, these models achieve relatively marginal improvements with exponential growth in model complexity. In this paper, we dissect the classical GNN-based SBR models and empirically find that some sophisticated GNN propagations are redundant, given the readout module plays a significant role in GNN-based models. Based on this observation, we intuitively propose to remove the GNN propagation part, while the readout module will take on more responsibility in the model reasoning process. To this end, we propose the Multi-Level Attention Mixture Network (Atten-Mixer), which leverages both concept-view and instance-view readouts to achieve multi-level reasoning over item transitions. As simply enumerating all possible high-level concepts is infeasible for large real-world recommender systems, we further incorporate SBR-related inductive biases, i.e., local invariance and inherent priority to prune the search space. Experiments on three benchmarks demonstrate the effectiveness and efficiency of our proposal. We also have already launched the proposed techniques to a large-scale e-commercial online service since April 2021, with significant improvements of top-tier business metrics demonstrated in the online experiments on live traffic.
1 INTRODUCTION
Session-based recommendation addresses next-action prediction when user profiles and long-term histories are unavailable or limited. The paper argues that complex GNN propagation is often redundant and proposes Atten-Mixer, which shifts reasoning to multi-level attention-based readouts.
- Session-based recommendation predicts users’ next actions from short sessions when conventional profiles and long-term histories are unavailable or limited.
- Existing GNN-based SBR models can grow exponentially in complexity while producing only marginal benchmark improvements.
- Sparse session graphs can make elaborate GNN designs burdensome because readout modules may learn most of the overall preference.
- The method uses local invariance and inherent priority to prune high-level concept search, balancing recommendation efficiency and accuracy.
- Atten-Mixer combines concept-view and instance-view readouts to perform multi-level reasoning over item transitions.
- The proposed techniques were deployed in a live e-commerce system and reportedly outperformed previous state-of-the-art baselines while serving millions of page views daily.
2 RELATED WORKS
Related work progresses from recurrent and convolutional models toward attention mechanisms and GNNs for modeling session preferences and item transitions. Recent GNN-based methods add increasingly sophisticated structures to capture relational, positional, contextual, popularity, and information-loss effects.
- RNN-based, attention-based, and CNN-based systems model user preferences through sequential, long- and short-term, or matrix-convolution representations.
- Recent SBR research primarily uses GNNs to model complex item transitions within sessions.
- GNN-based extensions incorporate highway networks, position information, target information, and global context to refine session modeling.
- Other recent methods address popularity bias and information loss in session-based recommendation.
3 ANALYSIS ON GNN-BASED SBR MODELS
The analysis decomposes GNN-based SBR into propagation and readout components, then uses SparseVD to assess their parameter importance. Results indicate that propagation weights become sparse while attention readout weights remain denser, motivating simpler models centered on readout reasoning.
- Typical GNN-based SBR models contain GNN layers that propagate pairwise transitions and attention readouts that aggregate session items.
- The analysis separates GNN propagation and readout parameters before applying SparseVD and measuring their density ratios.
- SparseVD regularizes weights toward zero for unimportant parameters while retaining important weights during training.
- Graph-propagation density approaches zero during training, whereas attention-pooling weights retain a relatively higher density ratio across SBR models.
- These findings motivate removing GNN propagation, preserving initial embeddings, and strengthening the attention-based readout module.
- Atten-Mixer’s multi-level reasoning is motivated by human information processing that first considers high-level concepts before specific related items.
4 METHODOLOGY
Atten-Mixer replaces complex propagation with a multi-level attention readout that generates, mixes, and pools attention maps to form session representations for next-item prediction. It uses SBR-specific inductive biases to reduce complexity while supporting flexible item encoders.
- 4.1 Overall Workflow: The workflow normalizes item embeddings, applies the Atten-Mixer readout, and combines the session representation with local preference to score candidate items.Candidate-item scores use normalized item and session representations together with a learnable transformation of their concatenation.
- 4.1 Overall Workflow: Atten-Mixer generates multi-level attention maps from session item embeddings, mixes them with L_p pooling, and uses the resulting session representation for recommendation.The framework separates attention-map generation from attention mixing and supports different item encoders before the Atten-Mixer component.
- 4.2.1 Multi-level User Intent Generation: Deep sets over the last l hidden states produce L queries with different receptive fields, combining instance-view and higher-level attention information.The first query is instance-view, while later queries represent higher-level attention; permutation invariance matches the local-invariance bias.
- 4.2.1 Multi-level User Intent Generation: Multi-head attention applies the generated queries to session item states, while L_p pooling balances salient and comprehensive multi-level user intent.For sessions shorter than L, all available items are used to generate queries.
- 4.3 Complexity Analysis: SBR-related inductive biases prune the search space, reducing the pruned model’s complexity to O(HL^2nd) when L is much smaller than session length n.The analysis presents this reduction as evidence that the inductive biases substantially improve efficiency.
5 OFFLINE EXPERIMENTS
Offline experiments on three benchmark datasets show that Atten-Mixer improves recommendation accuracy while retaining efficiency, and that its multi-level reasoning, inductive biases, and attention mixing each contribute to performance.
- 5.2 Overall Comparison: GNN-based methods generally outperform RNN- and CNN-based methods, while Atten-Mixer surpasses sophisticated GNN models without GNN-enhanced embeddings.The main comparison uses HR@K and MRR@K, with results reported in Table 2.
- 5.2 Overall Comparison: Atten-Mixer outperforms previous sophisticated GNN-based models even without GNN-enhanced item embeddings.This supports the effectiveness of the attention-based readout operation itself.
- 5.2 Overall Comparison: Atten-Mixer maintains high efficiency and accurate prediction, whereas DHCN requires about 10 times longer training than other methods.The comparison positions Atten-Mixer as suitable for real-world applications.
- 5.3 Atten-Mixer Enhancement Study: Atten-Mixer significantly improves all metrics across all datasets when integrated into SR-GNN and SGNN-HN.The improvements are reported as statistically significant in the corresponding comparison table.
- 5.4 Ablation Study: Atten-Mixer outperforms its simplified variants, showing benefits from multi-level reasoning, both SBR inductive biases, and attention-map mixing.Removing mixing substantially worsens performance, although the variant remains better than most baseline models.
- 5.5 Analysis of Session Length: As session length increases, all models’ Diginetica performance decreases, but GNN with Atten-Mixer achieves the best performance.The authors associate longer sessions with more unrelated items and harder preference identification.
6 ONLINE PERFORMANCE ANALYSIS
Atten-Mixer was deployed in heavy-traffic production scenarios and consistently outperformed the previous baseline during a seven-day online experiment.
- 6 ONLINE PERFORMANCE ANALYSIS: +1.5% in top business metrics was achieved over seven days across scenarios receiving millions of page views each day.The Atten-Mixer-enabled model was subsequently launched in production.
7 CONCLUSION
The paper concludes that classical GNN-based SBR models are over-parameterized and presents Atten-Mixer as an efficient SBR model with multi-level reasoning, supported by online and offline analyses.
- 7 CONCLUSION: Atten-Mixer is proposed after finding that classical GNN-based SBR models are over-parameterized.The model adds a multi-level reasoning component while targeting efficiency.
- 7 CONCLUSION: Extensive online and offline analyses validate the proposal.