Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation

Jiakai Tang; Sunhao Dai; Teng Shi; Jun Xu; Xu Chen; Wen Chen; Jian Wu; Yuning Jiang

Source-linked AI summary

Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation

Jiakai Tang, Sunhao Dai, Teng Shi, Jun Xu, Xu Chen, Wen Chen, Jian Wu, Yuning Jiang

arXiv:2503.22675v3Mar 28, 2025cs.IRcs.AIcs.CL

TL;DR

Existing sequential recommenders rely on direct forward inference, which limits computational depth for modeling evolving preferences and long-tail users and items. ReaRec adds inference-time implicit multi-step reasoning with reasoning position embeddings, alongside ERL and PRL for optimization. Experiments across five real-world datasets and varied backbones report broad effectiveness, while post-hoc analysis shows performance-ceiling gains of approximately 30%-50%.

Problem
Direct forward inference in sequential recommendation has limited computational depth for modeling complex evolving preferences and nuanced long-tail users and items.
Method
ReaRec autoregressively reuses the sequence's last hidden state for implicit multi-step reasoning, separates reasoning inputs with RPE, and optimizes reasoning with ERL and PRL.
Results
Experiments across five real-world datasets and varied SeqRec models validate ReaRec's effectiveness and generality, with post-hoc analyses showing performance-ceiling improvements of up to 50%.
Takeaways & Limitations
ReaRec establishes inference-time computational scaling as a promising direction for sequential recommendation.
Takeaways & Limitations
The authors identify theoretical understanding and efficient inference as open challenges for autoregressive reasoning-based recommendation.

Abstract

from arXiv · show

Sequential Recommendation (SeqRec) aims to predict the next item by capturing sequential patterns from users' historical interactions, playing a crucial role in many real-world recommender systems. However, existing approaches predominantly adopt a direct forward computation paradigm, where the final hidden state of the sequence encoder serves as the user representation. We argue that this inference paradigm, due to its limited computational depth, struggles to model the complex evolving nature of user preferences and lacks a nuanced understanding of long-tail items, leading to suboptimal performance. To address this issue, we propose \textbf{ReaRec}, the first inference-time computing framework for recommender systems, which enhances user representations through implicit multi-step reasoning. Specifically, ReaRec autoregressively feeds the sequence's last hidden state into the sequential recommender while incorporating special reasoning position embeddings to decouple the original item encoding space from the multi-step reasoning space. Moreover, we introduce two lightweight reasoning-based learning methods, Ensemble Reasoning Learning (ERL) and Progressive Reasoning Learning (PRL), to further effectively exploit ReaRec's reasoning potential. Extensive experiments on five public real-world datasets and different SeqRec architectures demonstrate the generality and effectiveness of our proposed ReaRec. Remarkably, post-hoc analyses reveal that ReaRec significantly elevates the performance ceiling of multiple sequential recommendation backbones by approximately 30\%-50\%. Thus, we believe this work can open a new and promising avenue for future research in inference-time computing for sequential recommendation.

1 Introduction

Sequential recommendation models typically use direct forward inference, limiting their ability to represent evolving preferences and challenging long-tail cases. ReaRec adds implicit multi-step inference with specialized learning strategies, and experiments report consistent gains across datasets and models.

Motivation: Transformer-based sequential recommenders typically use the final encoded position as the user representation after weighting historical items.This direct-forward pipeline is illustrated with models such as SASRec and UniSRec.
Proposed framework: ReaRec enables SeqRec models to perform implicit multi-step reasoning over latent representations before producing the final user embedding.It autoregressively feeds the last hidden state back into the encoder and uses specialized positional encoding to distinguish sequence encoding from reasoning.
Reasoning learning: ERL and PRL are introduced to optimize implicit reasoning and mitigate degradation during deeper reasoning processes.ERL uses ensemble-style multi-order representations and multi-step supervision; PRL draws on curriculum learning.
Empirical evaluation: 7.49% average performance gain is reported for ReaRec across five benchmark datasets.The experiments evaluate the framework on five real-world datasets and representative SeqRec models.
Empirical evaluation: Up to 50% improvement in performance ceilings is reported in post-hoc analysis across different backbone models.The analysis is presented for optimal reasoning steps on the Yelp dataset across different SeqRec models.

2 Preliminary

Sequential recommendation predicts a user's next item from chronological interaction history. Its standard pipeline encodes item representations, processes them with a sequential model, and scores candidate items against the resulting sequence representation.

2.1 Problem Definition: Users and items are represented by sets U and V, with M users and N items, while each user has a chronological sequence S_u.Each item has a unique ID and may also include textual attributes stored in a dictionary.
2.1 Problem Definition: The task is to predict each user's next interaction from the user's historical interaction sequence.User sequences are represented chronologically, and the learning objective maximizes next-item prediction probability.
2.2 Sequential Recommendation Pipeline: Historical interactions are first converted into item embeddings and then encoded by a sequential model to produce a sequence representation.The conventional representation is typically taken from the final sequence position.
2.2 Sequential Recommendation Pipeline: Candidate-item similarity scores, computed with measures such as dot product or cosine similarity, determine next-item probabilities.The sequence representation is compared with candidate item embeddings at the final prediction stage.
2.2 Sequential Recommendation Pipeline: Sequential recommenders mainly differ in whether they use ID-based or text-based item encoding.SASRec and BERT4Rec exemplify ID-based encoding, whereas UniSRec and MoRec use text-based representations.
2.2 Sequential Recommendation Pipeline: ReaRec treats item representations generically as e_v because its reasoning framework is model-agnostic.The paper omits representation-acquisition details and uses e_v for item v's representation.

3 Methodology

ReaRec extends sequential recommendation with multi-step implicit reasoning to refine user representations and better approximate user-interest distributions. The framework is designed to be simple, scalable, and applicable across sequential recommender architectures.

3 Methodology: ReaRec replaces traditional direct recommendation with multi-step implicit reasoning that refines user representations.The framework is intended to exploit additional inference-time computation to approximate the distribution of user interests.
3 Methodology: ReaRec is described as model-agnostic and highly scalable for integration with sequential recommenders.The framework is introduced as a general approach rather than one tied to a single backbone architecture.
3 Methodology: The methodology combines a foundational inference-time computation extension with ERL and PRL for reasoning-process optimization.The paper presents these components as lightweight methods addressing challenges in multi-step reasoning.

3.1 ReaRec Backbone

ReaRec extends sequential recommenders with autoregressive multi-step inference, feeding the latest hidden state back into the encoder and separating reasoning positions from item encoding positions. This added computation is designed to refine user-interest representations while addressing reasoning-pattern degradation.

3.1 ReaRec Backbone: ReaRec is model-agnostic and can extend computational capacity during inference for various sequential recommendation backbones.The framework is illustrated with a Transformer-based recommender but is designed for broader integration.
3.1 ReaRec Backbone: The conventional paradigm directly uses the final layer’s last-position output as the user representation, limiting computation for modeling nuanced user interest.ReaRec introduces implicit reasoning to refine interest modeling toward real preference distributions.
3.1 ReaRec Backbone: Unlike conventional inference, ReaRec autoregressively feeds the last hidden state back into the encoder for K-pass forward computations.This increases inference-time computation to capture more intricate sequential dependencies.
3.1 ReaRec Backbone: Reasoning Position Embeddings distinguish the original sequence-encoding phase from the multi-step reasoning phase.At each reasoning step, the original item positions remain unchanged while a reasoning position embedding is added to the recurrent latent representation.
3.1 ReaRec Backbone: The reasoning hidden states are represented as R = [r0, r1, ..., rK], with each ri denoting the reasoning state at step i.The final user representation can use the last reasoning output rK, while ERL later aggregates states across steps.
3.1 ReaRec Backbone: A naive reasoning objective lacks supervision for intermediate states, leaving the reasoning process vulnerable to pattern degradation.This motivates the ERL and PRL learning strategies.

3.2 Ensemble Reasoning Learning (ERL)

Ensemble Reasoning Learning (ERL) supervises and combines multiple reasoning states as multi-view user representations. Its training objective uses recommendation learning and KL regularization, while inference averages reasoning states before ranking candidate items.

3.2 Ensemble Reasoning Learning (ERL): ERL treats hidden states from different reasoning steps as multi-view representations of evolving user interests.Aggregating diverse reasoning results reduces reliance on a potentially suboptimal final output alone.
3.2 Ensemble Reasoning Learning (ERL): ERL applies cross-entropy supervision to the ensembled sequence representation and averages reasoning states across steps for the final user representation.The resulting representation is h_u = 1/K sum r_i.
3.2 Ensemble Reasoning Learning (ERL): Recommendation-only training can encourage shortcut copying between consecutive reasoning states, causing pattern collapse and weakening inference-time computation scaling.The paper introduces KL divergence regularization to address this issue.
3.2 Ensemble Reasoning Learning (ERL): ERL’s objective combines the recommendation loss with a KL regularization term controlled by the hyperparameter λ.The KL term balances the strength of the constraint during optimization.
3.2 Ensemble Reasoning Learning (ERL): During inference, the pooled user representation is compared with candidate item representations using inner product or cosine similarity, and top-scoring items are selected.The similarity function depends on the underlying sequential recommendation algorithm.

3.3 Progressive Reasoning Learning (PRL)

Progressive Reasoning Learning (PRL) guides reasoning toward increasingly sharp preference distributions through temperature annealing and process supervision. It additionally addresses reasoning bias with noise-based denoising and InfoNCE contrastive learning.

3.3 Progressive Reasoning Learning (PRL): PRL progressively sharpens predicted distributions so intermediate reasoning chains increasingly approximate the user’s true preference distribution.Its motivation is that greater inference-time computation should capture finer-grained sequential features.
3.3.1 Progressive Temperature Annealing (PTA): Progressive Temperature Annealing (PTA) uses a step-specific temperature τ_k to control prediction sharpness during reasoning.The base temperature τ and decay-rate hyperparameter α determine the annealing schedule.
3.3.2 Reasoning-aware Contrastive Learning (RCL): PRL applies separate recommendation losses to each reasoning hidden state, injecting process supervision into the reasoning process.This contrasts with ERL’s supervision of an aggregated representation.
3.3.1 Progressive Temperature Annealing (PTA): PTA encourages broader exploration early and narrows the search space as reasoning progresses by gradually reducing τ_k.The method is intended to guide predictions toward the user’s true preference distribution.
3.3.2 Reasoning-aware Contrastive Learning (RCL): PRL addresses reasoning bias by injecting normally distributed noise into reasoning states to simulate accumulated reasoning error.The noised inputs are passed through the Transformer encoder to produce alternative hidden-state views.
3.3.2 Reasoning-aware Contrastive Learning (RCL): The reasoning contrastive objective maximizes mutual information between original and denoised hidden states through an InfoNCE-based lower-bound optimization.Negative samples use corresponding reasoning states from other item sequences in the same batch.
3.3.2 Reasoning-aware Contrastive Learning (RCL): PRL combines recommendation loss with reasoning contrastive loss to improve robustness during reasoning denoising.The contrastive component provides a tractable alternative because direct high-dimensional mutual-information maximization is infeasible.

3.4 Discussion

ReaRec increases inference-time computation through autoregressive reasoning, deepening feature interactions while using lightweight mechanisms to exploit this added reasoning capacity.

ReaRec autoregressively feeds reasoning hidden states into the sequence encoder, increasing feature-crossing depth and capturing finer-grained sequence characteristics.
ERL aggregates multi-level deep-crossing features into the final user representation to capture diverse latent interest distributions.
PRL uses curriculum learning to progressively uncover increasingly complex intent-evolution patterns during reasoning.
With KV caching, K reasoning steps add O(L(K(C+K)d+Kd^2)) time and only Kd-dimensional reasoning position embeddings.When K is much smaller than sequence length C, the overhead simplifies to O(L(KCd+Kd^2)).

4 Experiments

The experiments benchmark ReaRec across five real-world datasets, multiple sequential-recommendation backbones, and NDCG and Recall metrics, with improvements reported for both ID-based and text-based models.

Datasets: Experiments use five Yelp and Amazon recommendation datasets, with chronological preprocessing and positive interactions defined by ratings greater than 3.
Evaluation: Recommendation quality is measured using NDCG@10, NDCG@20, Recall@10, and Recall@20.NDCG evaluates relevance and ranking quality, while Recall measures recovery of ground-truth items in the top-k list.
Baselines: The benchmark covers ID-based SASRec and BERT4Rec alongside text-based UniSRec and MoRec backbones.
Overall Performance: Text-based methods consistently outperform ID-based models across all datasets, with UniSRec improving Yelp NDCG@20 by 9.51%.
Overall Performance: ERL and PRL consistently surpass baseline models, improving SASRec by 6.76% and 8.21% and UniSRec by 12.29% and 10.43%, respectively, across five datasets.

4.3 Further Analysis

Further analyses show that reasoning benefits depend on user and item characteristics, reasoning depth, optimization components, and inference cost, with excessive steps causing degradation.

4.3.1 Robustness Analysis Across User and Item Subgroups: User groups are partitioned by sequence length and item groups by interaction frequency, with higher group numbers indicating longer sequences and greater popularity.
4.3.1 Robustness Analysis Across User and Item Subgroups: For short-sequence users and unpopular items, NDCG@20 improves with additional reasoning, including gains of 12.08%, 16.35%, and 18.69% for IG-1.Performance declines for long-sequence users and popular items, where overthinking can reduce gains.
4.3.2 Impact of Reasoning Steps on Recommendation Performance: Naive reasoning can underperform the base model, whereas reasoning positional embeddings improve performance by separating sequence encoding from reasoning phases.
4.3.2 Impact of Reasoning Steps on Recommendation Performance: ERL and PRL use stepwise supervision to mitigate degradation of intermediate reasoning states, while performance declines as inference steps become excessive.
4.3.3 Impact of Reasoning Steps on Inference Latency: At two reasoning steps, ReaRec typically reaches optimal performance, raising average performance by 7.49% with a 3.51% latency overhead.
4.3.3 Impact of Reasoning Steps on Inference Latency: KV caching reduces attention complexity from O(N^2) to O(N), keeping the additional inference latency manageable.
4.3.4 Ablation Study: Removing KL regularization from ERL or RCL from PRL worsens NDCG@20, indicating both auxiliary losses support reasoning quality.
4.3.5 Hyperparameter Sensitivity Analysis: PRL performance generally improves as base temperature increases, whereas ERL is usually insensitive to λ but declines beyond a threshold.

4.4 Case Studies

Case studies examine target-item ranking trajectories and show how PRL progressively refines recommendations during multi-step inference.

The analysis tracks target-item rank changes on Yelp across PRL settings and an ablated version without RCL.
The full PRL method progressively improves target-item ranking as reasoning depth increases.
For an Amazon user with FPS-game and Xbox-related purchase history, successive outputs illustrate stepwise preference refinement.

5 Related Work

Sequential recommendation models user behavior from interaction sequences to predict the next item, while related work has increasingly examined inference-time reasoning as computation shifts beyond training.

Sequential recommendation predicts a user’s next interaction by modeling behavior patterns and evolving interests from interaction sequences.
Inference-time scaling has emerged as a research direction as large language models encounter training-stage scaling bottlenecks.
Recent reasoning-oriented language models use long Chain-of-Thought mechanisms and exhibit test-time scaling phenomena.

6 Conclusion and Future Work

The paper presents ReaRec as an exploratory inference-time computing framework for sequential recommendation and identifies adaptive depth, theoretical analysis, and efficient inference as future directions.

6.1 Conclusion: ReaRec integrates multi-step implicit reasoning into sequential recommendation and raises existing backbone performance ceilings by up to 50% with post-hoc optimal step selection.
6.2 Future Work: The work is positioned as an initial exploratory effort using simple inference-time computational strategies.
6.2 Future Work: Adaptive inference depth selection should balance computational depth with sequence complexity because reasoning can degrade performance for high-activity users and popular items.
6.2 Future Work: Future theoretical work should explain how multi-step reasoning contributes to improved recommendation performance.
6.2 Future Work: Future efficiency research may explore linear attention, model quantization, and long-to-short reasoning distillation for industrial-scale deployment.