PaperlayerView exact revision on arXiv

Source-linked AI summary

RouteRec: Behavior-Guided Sparse Routing for Sequential Recommendation

Junyeong Song, Jaemin Yoo

arXiv:2609.39007v1cs.IRcs.LG

TL;DR

Existing recommenders process sessions with different behavioral patterns through shared blocks, while MoE leaves open what should guide expert allocation. RouteRec routes using cues derived from observed session behavior, and ranks first in 12 of 18 dataset–metric cells across six datasets.

  • Problem

    Existing models apply shared blocks to sessions with different behavioral patterns, and MoE routing leaves open how to allocate sessions when predefined labels are unavailable.

    Source: §1 · p. 1

  • Method

    RouteRec uses Tempo, Focus, Memory, and Popularity cues to select expert groups across macro, mid, and micro scopes, then uses cues and current hidden state to select experts within groups.

    Source: §1 · p. 2, §3.3 · p. 4

  • Results

    RouteRec ranks first in 12 of 18 dataset–metric cells and achieves an overall average rank of 1.61, compared with 4.11 for next-best baseline FDSA.

    Source: §4.2 · p. 6, §5 · p. 8

  • Takeaways & Limitations

    Routing profiles and cue perturbations suggest expert allocation shifts with observed behavior and depends on cue–session alignment, while controls indicate gains are not explained by shared capacity alone.

    Source: §5 · p. 8

    RouteRec is intended for heterogeneous sessionized histories rather than as a uniform improvement across sequential datasets; its dense expert execution costs more than SASRec-wide, and some cues require timestamps or item-group metadata.

    Source: §5 · p. 8

Abstract

from arXiv · show

Sessionized interaction histories contain behavioral patterns that can improve sequential recommendation. However, existing models process all sessions through the same parameterized blocks, regardless of their behavioral differences. Mixture of Experts (MoE) enables conditional computation, but it leaves open what should guide expert allocation. We propose RouteRec, a sequential recommender that uses observed session behavior as the routing criterion. RouteRec summarizes four types of behavioral evidence from sessionized histories: interaction tempo, item-group focus, repetition and carryover, and popularity tendency. It uses these cues to route computation at macro, mid, and micro scopes. Cue-derived scores first select expert groups; within each selected group, the current backbone state then refines expert selection. Across six public datasets and 18 dataset-metric combinations, RouteRec ranks first in 12 and second in three, yielding the best overall average rank of 1.61 compared with 4.11 for the next-best baseline. Additional analyses suggest that the behavioral cues guide expert allocation beyond added capacity and produce routing patterns aligned with observed behavior. Our code is available at https://github.com/jy1559/RouteRec

1 Introduction

Sessions can exhibit different behaviors that call for different predictive evidence, yet existing models process them through shared blocks. RouteRec uses observed session behavior to guide sparse expert allocation and performs strongly across six datasets.

1 Introduction

  • Different sessions can make different evidence useful, but existing models apply the same shared blocks to every session.Rapid transitions emphasize recent item changes, whereas repetitive sessions emphasize repeated items and cross-session carryover.Source: §1 · p. 1
  • Because session-aware recommendation lacks task or domain labels, routing must infer its signal from observed sessionized history.The routing signal is intended to be available before prediction and derivable from standard logs and metadata without routing labels.Source: §1 · p. 1
  • RouteRec derives Tempo, Focus, Memory, and Popularity cues from histories and uses them to select expert groups before hidden-state refinement within each group.This hierarchical routing operates at macro, mid, and micro scopes; different sessions from one user can activate different paths as behavior changes.Source: §1 · p. 2
  • RouteRec ranks first in 12 of 18 dataset–metric cells and achieves the best overall average rank across six datasets and nine baselines.Its routing profiles and cue perturbations also suggest that allocation shifts with observed behavior and depends on cue–session alignment.Source: §1 · p. 2

2 Problem Definition and Related Work

The section defines session-aware next-item prediction and reviews sequential, session-based, session-aware, and MoE recommendation. It frames RouteRec’s routing signal as observed session behavior rather than a predefined label.

2.1 Problem Definition

  • At prediction time, the task is to predict the next item in an active session from its observed prefix and the user’s earlier sessions.RouteRec’s inputs use only information available before the target; popularity comes from training data, unavailable item-group cues are zero-filled, and the held-out target is never used.Source: §2.1 · p. 2

2.2 Sequential, Session-Based, and Session-Aware Recommendation

  • Session-based methods model current-session intent, while session-aware methods also use the same user’s earlier sessions as distinct evidence.Prior work also enriches sequence representations with side information such as item attributes or interaction intervals.Source: §2.2 · p. 2

2.3 MoE in Recommendation

  • Prior recommendation work uses MoE for multimodal fusion, facet preferences, temporal modules, or heterogeneous sequential encoders, but routing evidence remains a separate design choice.Examples include M3SRec, FAME [17], HM4SR [38], HM2Rec, and FamouSRec [37].Source: §2.3 · p. 2

2.4 MoE Routing Design

  • RouteRec derives Tempo, item-group Focus, Memory and carryover, and Popularity cues from sessionized histories at macro, mid, and micro scopes.Unlike predefined task, dataset, or language labels, these cues arise from interaction history; cue-family signals select expert groups, while the corresponding cues and current hidden state select experts within groups.Source: §2.4 · pp. 2–3, Figure 3 · p. 3

3 Proposed Method

RouteRec uses observed session behavior to guide sparse expert allocation in a sequential recommender. Its behavioral cues summarize different windows and guide routing through a hierarchy of scopes, groups, and experts.

3 Proposed Method

  • RouteRec uses observed session behavior to guide expert allocation while retaining a standard sequential backbone.Source: §3 · p. 3

3.1 Main Ideas and Architecture

  • The architecture replaces SASRec feed-forward stages with routed expert blocks while retaining self-attention and residual connections.Source: §3.1 · p. 3

3.2 Behavioral Cue Construction

  • Four cue families capture interaction tempo, item-group focus, repetition and carryover, and popularity tendency from sessionized histories.Source: §3.2 · p. 3

3.3 Coarse-to-Fine Multi-Scope Routing

  • Macro cues summarize earlier sessions, mid cues the active-session prefix, and micro cues the last five interactions; macro and mid cues are shared across positions, while micro cues are re-evaluated at each position.Source: §3.3 · p. 4

3.4 Hierarchical Sparse Expert Allocation

  • Cue projections first select expert groups, then cues and the current backbone state select experts within each group, yielding sparse, behavior-dependent paths.Source: §3.4 · p. 4

3.5 Training Objective

  • Training combines next-item cross-entropy with route-consistency and z-loss regularizers to stabilize routing while preserving behavior-dependent expert concentration.Source: §3.5 · pp. 4–5

4 Experiments

RouteRec is evaluated across six datasets, then analyzed through routing controls and architectural ablations. It achieves the strongest aggregate benchmark ranking, while the analyses test whether behavior-aligned routing and its multiscale design account for the gains.

4.1 Experimental Setup

  • Six public datasets use chronological held-out-session prediction, shared top-10 metrics, a closed candidate catalog, and nine complementary baselines.Hyperparameters and checkpoints are selected using validation HR@10, NDCG@10, and MRR@10, with test results averaged over seeds 42–44.Source: §4.1 · p. 5

4.2 Overall Performance

  • 1.61 average rank across 18 dataset–metric cells beats FDSA’s 4.11; RouteRec ranks first in 12 cells and second in three.It leads all three metrics on KuaiRec, LastFM, and Retail Rocket; on LastFM, NDCG@10 rises from .2703 to .2866 over SASRec, a 6.0% relative gain.Source: §4.2 · p. 6

4.3 Evidence for Behavior-Guided Routing

  • On KuaiRec, each dominant behavioral family raises selection of its matched expert group by 3.8–5.5 percentage points over the overall routing profile.Rows partition test sessions by dominant family and sum to 100%; the highlighted diagonal marks matched groups.Source: §4.3 · p. 6, Figure 4 · p. 6
  • Cue corruption lowers MRR@10 on both KuaiRec and LastFM; on KuaiRec, role swapping reduces it by about 35%, versus about 28% when cues are zeroed.Training-time shuffling sharply degrades KuaiRec, and removing cues brings performance to approximately SASRec level on both datasets.Source: §4.3 · pp. 6–7, Figure 5 · p. 6
  • On KuaiRec, behavior-guided routing beats hidden-state routing with SASRec and DuoRec, while widening SASRec yields only a small gain.Hidden MoE remains a strong control for SASRec but does not improve vanilla DuoRec; the benefit extends beyond SASRec, with magnitude dependent on backbone.Source: §4.3 · p. 7, Figure 6 · p. 7

4.4 Core Architectural Choices

  • Reducing routing scopes progressively lowers performance, and hierarchical sparse routing outperforms hierarchical dense, flat sparse, and flat dense alternatives.The comparisons support multiple behavioral time scales and cue-family organization beyond sparse expert selection alone.Source: §4.4 · p. 7, Figure 8 · p. 8
  • Removing any scope lowers MRR@10 on both datasets; removing micro hurts most on KuaiRec, while no-mid and no-micro are nearly tied on LastFM.A dense feed-forward baseline performs worst on both datasets, and the relative scope effects differ by dataset.Source: §4.4 · p. 7, Figure 9 · p. 8
  • Removing both routing regularizers causes the largest drop on both datasets, while load balancing also degrades performance.Route consistency matters more than z-loss on KuaiRec and about as much on LastFM; the authors omit load balancing because uniform usage conflicts with behavior-dependent allocation.Source: §4.4 · p. 7, Figure 10 · p. 8

5 Conclusion

RouteRec routes sessionized histories using four behavioral cue families across three scopes. It performs strongly across six datasets, while the authors qualify its usefulness by dataset-level variation, execution cost, and metadata availability.

5 Conclusion

  • Tempo, Focus, Memory, and Popularity cues guide sparse expert routing across macro, mid, and micro scopes.The conclusion describes RouteRec as a hierarchical sparse MoE recommender using prefix-derived cues.Source: §5 · p. 8
  • RouteRec achieves the best overall average rank across six datasets and nine baselines, ranking first in 12 of 18 dataset–metric cells.Routing profiles and cue perturbations indicate allocation shifts with observed behavior and depends on cue–session alignment; controls suggest gains are not explained by shared capacity alone and transfer beyond SASRec.Source: §5 · p. 8
  • RouteRec is best viewed for heterogeneous sessionized histories; its dataset-level variation does not establish when routing is most useful.Dense expert execution costs more than SASRec-wide, and some cues require timestamps or item-group metadata.Source: §5 · p. 8

A Cue Availability under Limited Metadata

Reduced-cue retraining tests RouteRec when timestamp or item-group metadata is unavailable. The impact varies by dataset, but the tested variants remain competitive with the strongest non-RouteRec baseline.

A Cue Availability under Limited Metadata

  • No category cues removes label-dependent Focus, No time cues removes timestamp-dependent Tempo, and Sequence-only cues removes both families.The conditions are retrained and compared with the full model and the best non-RouteRec baseline.Source: A Cue Availability under Limited… · p. 8
  • The reported configuration fixes router layout and activation across datasets, while capacity and optimization values are tuned per dataset.Table 3 records the hyperparameter configuration used in the main experiments.Source: Table 3 · p. 9
  • On KuaiRec, all reduced-cue variants, including sequence-only cues, remain close to the full model; on LastFM, removing Tempo causes a larger decline than removing Focus alone.Removing Tempo together with Focus also causes a larger LastFM decline; all variants remain competitive with the strongest non-RouteRec baseline.Source: A Cue Availability under Limited… · pp. 8–9

B Implementation Details

The implementation fixes router layout and activation across datasets while allowing capacity and optimization settings to vary. Cue computation and routing weights follow scope-specific rules and use only observed history and training statistics.

B.1 RouteRec Instantiation

  • Router layout and activation are fixed across datasets; capacity and optimization values are selected per dataset.Table 3 records the main-experiment configuration.Source: §B.1 · p. 9
  • Macro and mid cues are computed once per session, while micro cues are recomputed at each position from the last w items.Group- and expert-level top-k weights are renormalized over active support.Source: §B.1 · p. 9
  • All cues use observed history and training-split statistics, with no held-out target information entering the router.Source: §B.1 · p. 9

B.2 Cue Bank by Scope and Family

RouteRec’s cue bank is organized by behavioral family and routing scope. The scopes capture cross-session tendencies, active-session behavior, and immediate local changes.

B.2 Cue Bank by Scope and Family

  • The cue bank groups cues by behavioral family and routing scope.Table 4 lists the complete bank.Source: §B.2 · p. 9
  • Macro cues use five recent completed sessions, mid cues use the observed active-session prefix, and micro cues use the recent local suffix.These scopes capture cross-session tendencies, behavior unfolding in the active session, and immediate local changes without using the prediction target.Source: §B.2 · p. 9
  • Figure 12 projects session-level cue vectors for all four families onto the first two principal components, with one panel per family.Diamonds mark per-dataset medians.Source: Figure 12 · p. 9

C Experimental Details

All experiments in the appendix use one NVIDIA RTX 4080 SUPER GPU per run.

C Experimental Details

C.1 Datasets and Preprocessing

The experiments use chronologically split, processed session data and tune models on validation metrics before a single test evaluation. The appendix also describes cue-space diversity across datasets.

C.1 Datasets and Preprocessing

  • Table 5 reports processed dataset statistics for data spanning e-commerce sessions, location visits, ratings, videos, and long listening histories.Source: §C.1 · p. 9, Table 5 · p. 10
  • Sessions are split chronologically 70/15/15, and chunks from an original session remain within the same partition.Sessions shorter than five interactions and items with fewer than three occurrences are removed; validation/test targets unseen in training are excluded, and cues use observed history and training-split statistics.Source: §C.1 · p. 9

C.2 Behavioral Cue Diversity

  • Figure 12 shows cue structure varying by dataset and family, with especially pronounced separation for KuaiRec and LastFM, not explained by average session length alone.The projection is descriptive and does not train the router or define dataset labels.Source: §C.2 · p. 10

C.3 Model Selection and Tuning

  • Models are selected by the validation mean of HR@10, NDCG@10, and MRR@10, then evaluated once on test data; results average seeds 42–44.Table 7 lists the model-selection search spaces.Source: §C.3 · p. 10, Table 7 · p. 10

C.4 Computational Efficiency

RouteRec incurs runtime costs relative to SASRec-wide, while its throughput is close to execution-matched Hidden MoE. The paper reports that its AI-assisted work was limited to support tasks and reviewed by the authors.

C.4 Computational Efficiency

  • The efficiency benchmark uses one RTX 4080 SUPER, batch size 256, and matched data split, sequence length, and candidate set.Training throughput is measured over one representative epoch; full-sort latency uses a fixed validation batch, warm-up, timed batches, and three repetitions.Source: §C.4 · p. 10
  • RouteRec retains 60.6% and 55.5% of SASRec-wide training throughput and incurs 6.06× and 4.30× full-sort latency on Foursquare and KuaiRec, respectively.Against execution-matched Hidden MoE, RouteRec retains 92.5% and 95.5% of training throughput; the benchmark supports a routing-quality contribution, not a realized sparse-efficiency advantage.Source: §C.4 · p. 10

GenAI Usage Disclosure

  • The authors report AI assistance for editing, routine code implementation/debugging under specified settings, output organization, and plot checks, with all outputs reviewed and verified.They state that AI tools did not determine research ideas, experimental design, parameter choices, substantive analyses, interpretations, or conclusions.Source: GenAI Usage Disclosure · p. 11
Loading 2609.39007v1…