PaperlayerView exact revision on arXiv

Source-linked AI summary

Think Harder Where It Matters: Item-Level Adaptive Recursion for Sequential Recommendation

Zhanshuo Cao, Jiakai Tang, Yuqi Zhou, Sunhao Dai, Jun Xu

arXiv:2610.30128v1cs.IRcs.AIcs.CLcs.LG

TL;DR

Sequential recommenders give every history item the same computational depth, although items differ in how much processing they need. LoopRec uses a shared recursive block and item-wise exit gates to adapt depth. Across benchmarks and backbones, it consistently outperforms matched fixed-depth counterparts.

  • Problem

    Sequential recommenders uniformly allocate depth across history items, despite heterogeneous computational needs and evidence that flexible allocation can improve rankings.

    Source: §1 · p. 1

  • Method

    LoopRec reuses a weight-shared block and makes a learned item-level exit-or-continue decision at each recursion boundary.

    Source: §1 · p. 2

  • Results

    Across three Amazon benchmarks and four backbone families, LoopRec consistently improves ranking over fixed-depth counterparts.

    Source: §6 · p. 9

  • Takeaways & Limitations

    Learned routes allocate greater depth to recent and less frequent items, consistent with the diagnostic patterns.

    Source: §6 · p. 9

Abstract

from arXiv · show

Existing sequential recommenders usually allocate a fixed computational budget to every item in a user's interaction history, regardless of the item's contribution to the final prediction. We challenge this de facto design with two diagnostic findings. (1) Non-uniform computational need: items within the same sequence vary drastically in how much computation they require. Most converge after shallow processing while a critical minority only stabilizes at full depth. (2) Suboptimality of fixed allocation: given oracle access to the target item, a non-uniform depth assignment consistently outranks the fixed one, indicating that existing uniform computation both wastes capacity on already-converged items and sacrifices overall recommendation quality. These observations motivate a shift from fixed-depth computation to item-level adaptive reasoning. To this end, we propose LoopRec. LoopRec recursively applies a single weight-shared block and introduces a lightweight gate at each recursion boundary that makes a binary exit-or-continue decision for every item independently. Items whose representations have sufficiently matured exit early, while those requiring deeper reasoning continue to subsequent recursions. This design converts the fixed global depth into a learned, per-item property without auxiliary depth labels, and is readily adaptable to diverse backbone architectures. Extensive experiments on public benchmarks with four backbone families demonstrate that LoopRec consistently outperforms matched fixed-depth counterparts. Further analysis reveals that LoopRec's learned allocation exhibits interpretable patterns aligned with our post-hoc oracle analysis: lower-frequency items and more recent interactions naturally receive deeper computation. Our code is available at https://github.com/Elpmis117/LoopRec.

1 Introduction

Sequential recommenders apply the same computational depth to every history item, despite heterogeneous user interactions and item-level computational needs. LoopRec replaces this uniform allocation with learned item-level recursion and consistently improves ranking across benchmarks and backbones.

1 Introduction

  • Uniform depth mismatches heterogeneous histories, where routine interactions carry minimal predictive signal while rare, exploratory, or pivotal ones can shape intent.Existing models give every history item the same depth.Source: §1 · p. 1
  • On a three-layer SASRec, 76.14% of items stabilized after one layer, while 6.72% remained sensitive through all three.The oracle preferred fixed allocation in only 4.6% of sequences and achieved up to 8.8% relative Hit Rate improvement through flexible reassignment.Source: §1 · p. 1
  • LoopRec shares one recursive block and uses a lightweight gate at each recursion boundary to decide independently whether each item exits or continues.Shared parameters keep representations in a common feature space; the method needs no auxiliary depth labels and can adapt to diverse backbones.Source: §1 · p. 2
  • Across three benchmarks and four backbone families, LoopRec consistently outperforms matched fixed-depth models, routing lower-frequency and more recent items to greater depth.The learned allocation patterns align with the post-hoc oracle analysis.Source: §1 · p. 2

2 Diagnostic Analysis

Diagnostics on Amazon Office Products with a three-layer SASRec find both heterogeneous item-level saturation depths and better rankings under flexible per-item allocation. Most items saturate shallowly, but shallow- and deep-saturating items often coexist in one history.

2.1 Probe 1: Item-Level Saturation Depth

  • Saturation depth is the earliest layer where an item's representation can be frozen without changing the final Top-K recommendation set.For the diagnostic, the frozen item is excluded from later layer computations while other positions continue normally.Source: §2.1 · p. 2
  • 76.14% of items reach saturation depth ℓ★_i ≤ 2, while 6.72% saturate only at full depth ℓ★_i = 3.Many sequences contain items spanning all three saturation depths.Source: §2.1 · p. 2

2.2 Probe 2: Item-Level Depth Allocation

  • The oracle searches depths 1, 2, or 3 for the five most recent items, keeps earlier positions at full depth, and selects the assignment with the best target rank.The fixed-depth assignment is included as a candidate, and the oracle has access to the groundtruth target.Source: §2.2 · p. 3
  • 7.1% NDCG@20, 6.5% MRR@20, and 8.8% HR@20 gains; fixed allocation is preferred in only 4.6% of sequences.The oracle-selected assignments often improve rank while consuming less tail-side computation.Source: §2.2 · p. 3, Table 1 · p. 3

3 The Proposed LoopRec

LoopRec adapts sequential recommendation by assigning different recursion depths to history positions while retaining the standard next-item prediction objective. It combines a shared recursive backbone with position-wise routing and regularizers designed to promote depth diversity and stable training.

3.1 Task Formulation

  • LoopRec keeps the standard next-item prediction objective: given an interaction prefix, the model predicts the next item from a distribution over candidates.Training instances are time-ordered item sequences, and the objective uses prefix–target pairs.Source: §3.1 · p. 3

3.2 LoopRec Architecture

  • A shared backbone block is reused across recursions, while gates at each boundary let active positions stop or continue independently.Only continuing positions are updated; stopped positions retain their current states, and all remaining positions stop at the final recursion.Source: §3.2 · pp. 3–4, Figure 3 · p. 4
  • Depth-specific readout uses each position’s selected-depth state while retaining a residual connection to its input token.The readout weight preserves differentiability with respect to router probabilities despite hard depth selection in the forward pass.Source: §3.2 · p. 4

3.3 Training Objective

  • Router training addresses depth collapse and premature gate saturation with depth-usage balancing and a gate-stabilization penalty.The recommendation loss remains the only item-prediction supervision; the additional terms regularize routing, and routing warm-up initially forces positions to maximum depth.Source: §3.3 · pp. 4–5
  • Soft depth usage is based on the continuation mass reaching each boundary and is matched against empirical hard assignment frequencies.The continuation mass represents route mass that reaches a boundary without exiting earlier.Source: §3.3 · pp. 4–5

4 Experiments

Across three Amazon benchmarks and four backbone families, LoopRec consistently improves fixed-depth ranking, while its learned routes vary with position and item frequency. The analyses also compare allocation strategies, routing components, and compute savings against measured latency.

4.1 Experimental Setup

  • The evaluation uses three Amazon Reviews 2023 categories and compares four backbone families with their LoopRec counterparts.The categories are Software, Office, and Patio; the backbones are SASRec, BSARec, NextItNet, and LRURec.Source: §4.1 · p. 5

4.2 Main Results

  • LoopRec improves all 48 backbone–dataset–metric comparisons, with 44 statistically significant gains and relative gains of 0.69%–49.07%.The reported gains span backbone families and datasets, with improvements aligned across MRR and NDCG within each backbone block.Source: Table 3 · p. 6, §4.2 · p. 6

4.3.1 Depth by Sequence Position.

  • Later history deciles receive larger selected recursion rounds on average across four SASRec cases, though position is not a per-item rule.The cases cover Office and Patio under two LxRy configurations.Source: §4.3.1 · pp. 6–7, Figure 4 · p. 7

4.3.2 Depth by Item-Level Factors.

  • On Office, routed depth generally decreases with training frequency: mean depth falls from 2.160 for single-occurrence items to 1.330 for 100+ items.The depth-1 share rises from 50.3% to 88.6%, while the depth-4 share falls from 29.1% to 10.7%; LoopRec also exceeds position- and frequency-based routing heuristics in mean NDCG@20 over five seeds.Source: §4.3.2 · p. 7

4.3.3 Alignment with Saturation Depth.

  • The saturation-depth probe mirrors the routing trends: recent and rare items show greater depth sensitivity than earlier and more frequent items.Mean saturation depth rises to 1.76 in the most recent quartile, while it falls from 1.873 in the single-occurrence bucket to 1.197 in the 100+ bucket.Source: §4.3.3 · p. 7

4.4.1 Ablation Studies.

  • Full LoopRec has the highest mean MRR@20 and NDCG@20 on all three datasets, exceeding sequence-level looping by 0.0025 and 0.0048 on Software.Sequence-level looping improves both metrics over Vanilla on Software and Office, but not Patio; on Patio, LoopRec raises MRR@20 from 0.0390 to 0.0416 versus sequence-level looping.Source: §4.4.1 · p. 8, Table 5 · p. 8
  • Removing warm-up lowers both metrics across all datasets, while removing both router regularizers reduces average MRR@20 by 5.36% and NDCG@20 by 4.25%.The warm-up ablation drops MRR@20 by 4.03% and NDCG@20 by 3.11% on average.Source: §4.4.1 · p. 8, Table 6 · p. 8

4.4.2 Computation and Latency.

  • Across three measured settings, LoopRec uses 0.407–0.755× the fixed-depth compute, while encoder latency remains 0.869–0.995×.Compute savings translate only partly into wall-clock gains.Source: §4.4.2 · p. 8

5 Related Work

Prior work studies stronger fixed-depth sequential encoders, shared-weight or adaptive-depth Transformers, and token-level adaptive computation. LoopRec applies a recurrent stopping gate to item-level computation in recommendation histories.

Sequential Recommendation

  • Sequential recommenders have developed from transition and recurrent or convolutional models to attention-based and inductive-bias-enhanced encoders, generally retaining uniform depth across history positions.LoopRec poses item-level computation allocation as a distinct question while keeping the recommendation objective and backbone family intact.Source: §5 · p. 8

Looped and Adaptive-Depth Transformers

  • Universal Transformers support adaptive per-position halting, while later work studies looped computation and adaptive-depth routing across Transformer and language-model settings.Mixture-of-Recursions combines shared recursion with token-level adaptive computation for language modeling.Source: §5 · p. 8
  • Unlike MoR’s upfront token-choice depth or capacity-constrained expert-choice routing, LoopRec updates a shared binary stop-or-continue decision at each recursion.The paper studies this design for recommendation histories with item-level diagnostics and multiple backbone families.Source: §5 · pp. 8–9

6 Conclusion

LoopRec replaces uniform fixed-depth processing with weight-shared recursion and learned item-wise routing. Across three Amazon benchmarks and four backbone families it improves ranking, routes more depth to recent and less frequent items, and reduces compute in measured configurations.

Conclusion

  • Diagnostics reveal heterogeneous depth sensitivity among history items, motivating adaptive rather than uniform computation.The conclusion also notes that latency gains vary despite reduced encoder computation.Source: §6 · p. 9
  • LoopRec uses weight-shared recursion and learned item-wise routing to allocate computation adaptively.Its routes give greater depth to recent and less frequent items, consistent with the diagnostic patterns.Source: §6 · p. 9
  • Across three Amazon benchmarks and four backbone families, LoopRec consistently improves ranking over fixed-depth counterparts.The maximum recursion budget is fixed per configuration before training; learning that budget is left for future work.Source: §6 · p. 9

A.1 Routing and Readout Formulation

The routing formulation distinguishes hard selected depth from soft route mass used in readout and router training. It also contrasts readout weights with balancing mass and specifies a full-depth warm-up phase.

Routing and Readout Formulation

  • The formulation defines d_i as the hard selected depth and describes readout weights as retaining soft route mass so recommendation loss can update the router.The readout weight at the selected depth contributes to the final representation.Source: §A.1 · p. 10
  • Continuation mass is carried to the next boundary for positions that have not reached their selected depth.The route mass reaching the first boundary is initialized to 1.Source: §A.1 · p. 10
  • Balancing mass distributes soft route mass over evaluated depths, whereas readout weight assigns mass only to the selected depth and contributes to the final representation..Source: §A.1 · p. 10
  • During routing warm-up, valid positions run to maximum depth, routing decisions and router regularizers are disabled, and each readout weight is set to 1.The forward procedure keeps exited states and preserves chronological order among active positions.Source: §A.1 · p. 10

A.2 Backbone-Specific Implementation

LoopRec’s implementation applies recursion around each backbone’s native block and uses backbone-specific configurations and search spaces. Training and evaluation use a shared protocol, with additional tuning for each architecture.

Shared training protocol

  • All models use Adam for up to 100 epochs, validation NDCG@20 early stopping, and five-seed averages; full LoopRec fixes the z-loss coefficient at 10^-3.The main table marks paired-t-test significance at p<0.05 against the corresponding fixed-depth backbone.Source: §A.2 · p. 10

SASRec

  • SASRec uses a Transformer block per recursion round and searches the number of blocks, recursion configuration, router warm-up, and balancing coefficient.The batch size is 1024; configurations include L1R4, L2R2, L1R6, L2R3, L1R8, and L2R4.Source: §A.2 · p. 10

BSARec

  • BSARec applies recursion around its native block, shares SASRec-family search ranges, and additionally searches frequency parameters α and c.The batch size is 512.Source: §A.2 · pp. 10–11

NextItNet

  • NextItNet applies recursion at its native residual convolution block, with six residual blocks and a (1, 2, 4) dilation cycle.It searches batch size over 256, 512, or 1024 and LoopRec configurations L3R2, L3R3, or L6R2.Source: §A.2 · p. 11

LRURec

  • LRURec applies recursion at its native block and searches one to three blocks, learning rate, and dropout while sharing several LoopRec search ranges.Its batch size is fixed at 512.Source: §A.2 · p. 11

A.3 Fixed-Depth Diagnostic Details

The diagnostic details define a counterfactual truncation test of recommendation-set stability and a restricted oracle over the five most recent history items.

Top-30 Exact-Set Saturation

  • The Top-30 saturation diagnostic uses a trained three-layer SASRec on Amazon Office Products and counterfactually stops updates for one selected history position after a chosen layer.All other positions continue through the normal forward pass.Source: §A.3 · p. 11
  • The saturation depth ℓ★ᵢ serves only as a stability statistic for the trained model’s full-depth recommendation set.It is not presented as a supervision target for the model.Source: §A.3 · p. 11

Restricted Tail-5 Oracle

  • The Tail-5 oracle varies only the five most recent items’ update depths, keeps model parameters fixed, and selects the candidate giving the target item the best rank.Each tail position receives depth 1, 2, or 3; earlier positions retain full-depth updates, and candidates exceeding baseline tail computation are excluded.Source: §A.3 · p. 11

A.4 Depth Distribution Analysis

The router uses multiple effective depths across positions, with substantial shares at both shallow and deep depths across the examined configurations.

Depth distribution

  • All four examined LoopRec configurations use multiple effective depths, with substantial shares at shallow and deep depths.The distributions are reported for LxRy configurations on Office.Source: §A.4 · p. 11
  • The analysis calculates each test sequence’s fraction of valid positions assigned to each effective depth and summarizes those fractions across the test set.Figure 6 reports the distributions for configurations where each round applies x shared blocks and at most y rounds are available.Source: §A.4 · p. 11
  • The non-uniform allocations indicate variation across positions and are consistent with the coexistence of shallow- and deep-saturating items found in the diagnostic analysis.The subsection examines whether routing uses multiple depths rather than collapsing to a near-uniform strategy.Source: §A.4 · p. 11

A.5 Item-Level Routing Examples

The routing examples show that item depth varies with popularity, recency, and context, while heuristic routing and runtime analyses assess performance and execution costs.

A.5 Item-Level Routing Examples

  • In the Office sequence example, frequent mailer and label items exit at depth 1, while several rarer items receive depths 2 or 3.The examples also show that routing is not determined by frequency or recency alone: PACKZON shipping labels and TECKNET mouse pad reach depth 4, while a later Springhill cardstock item remains shallow.Source: §A.5 · p. 12
  • Across category leaves, larger median training frequency tends to correspond to lower mean routed depth, though decisions still vary by position and context within a category.Figure 7 plots category median frequency against mean routed depth.Source: §A.5 · p. 12, Figure 7 · p. 12

A.6 Comparison with Heuristic Routing

  • The heuristic comparison replaces the learned router with position- or frequency-based rules while retaining LoopRec’s shared recursive backbone.Position routing assigns greater depth to more recent positions; frequency routing assigns greater depth to less frequent items.Source: §A.6 · p. 12
  • LoopRec has the highest mean on all four metrics, with NDCG@20 exceeding Position by 1.64% and Frequency by 1.88%.Both heuristic strategies yield modest NDCG@20 gains over vanilla SASRec, but those gains do not extend to every metric.Source: §A.6 · p. 12, Table 8 · p. 12

A.7 Runtime and Latency Measurements

  • Compute savings occur in all three settings, but encoder latency falls for L2R4 and L2R3 and remains nearly unchanged for L2R2.The results are measured against fixed-depth SASRec references at the same maximum effective depth on one NVIDIA A6000 GPU; savings only partly translate into wall-clock gains.Source: §A.7 · p. 13, Table 9 · p. 13

Related papers

Loading 2610.30128v1…