PaperlayerView exact revision on arXiv

Source-linked AI summary

Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation

Jiakai Tang, Sunhao Dai, Teng Shi, Jun Xu, Xu Chen, Wen Chen, Jian Wu, Yuning Jiang

arXiv:2503.22675v3cs.IRcs.AIcs.CL

TL;DR

Sequential recommenders commonly use a single final sequence state as the user representation, limiting inference-time computation for capturing nuanced user interests. ReaRec adds implicit multi-step reasoning during inference, and experiments report an average performance gain of 7.49% across five benchmark datasets.

  • Problem

    Sequential recommenders commonly use the final encoded position as the user representation, while non-reasoning inference has limited computation for capturing nuanced user interests.

    Source: §1 · p. 1, §3.1.2 · p. 4

  • Method

    ReaRec autoregressively feeds the last hidden state back into the encoder for multi-step reasoning, using reasoning position embeddings to distinguish reasoning from sequence encoding.

    Source: §1 · p. 2, §3.1.2 · p. 4

  • Results

    Across five benchmark datasets, ReaRec achieves an average performance gain of 7.49%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    ReaRec provides an initial framework for exploring inference-time computation in sequential recommendation.

    Source: §1 · p. 2, §6.1 · p. 13

    The authors characterize this work as an initial exploratory effort.

    Source: §6.2 · p. 13

Abstract

from arXiv · show

Sequential Recommendation (SeqRec) aims to predict the next item by capturing sequential patterns from users' historical interactions, playing a crucial role in many real-world recommender systems. However, existing approaches predominantly adopt a direct forward computation paradigm, where the final hidden state of the sequence encoder serves as the user representation. We argue that this inference paradigm, due to its limited computational depth, struggles to model the complex evolving nature of user preferences and lacks a nuanced understanding of long-tail items, leading to suboptimal performance. To address this issue, we propose \textbf{ReaRec}, the first inference-time computing framework for recommender systems, which enhances user representations through implicit multi-step reasoning. Specifically, ReaRec autoregressively feeds the sequence's last hidden state into the sequential recommender while incorporating special reasoning position embeddings to decouple the original item encoding space from the multi-step reasoning space. Moreover, we introduce two lightweight reasoning-based learning methods, Ensemble Reasoning Learning (ERL) and Progressive Reasoning Learning (PRL), to further effectively exploit ReaRec's reasoning potential. Extensive experiments on five public real-world datasets and different SeqRec architectures demonstrate the generality and effectiveness of our proposed ReaRec. Remarkably, post-hoc analyses reveal that ReaRec significantly elevates the performance ceiling of multiple sequential recommendation backbones by approximately 30\%-50\%. Thus, we believe this work can open a new and promising avenue for future research in inference-time computing for sequential recommendation.

1 Introduction

ReaRec extends sequential recommendation with implicit multi-step inference-time reasoning to address limitations of direct sequence encoding, and pairs this framework with learning strategies intended to preserve reasoning quality.

1 Introduction

  • Direct forward inference uses the final encoded sequence position as the user representation, which may inadequately capture evolving preferences, long-tail interests, and unpopular items.The introduction particularly identifies users with fewer interactions and less popular items as cases requiring deeper representation learning.Source: §1 · p. 1
  • ReaRec autoregressively reasons over latent sequence representations before producing a user embedding, using specialized positional encodings to distinguish item representations from reasoning inputs.The framework is presented as the first systematic exploration of inference-time computation in recommender systems.Source: §1 · p. 2
  • Implicit reasoning lacks the intermediate process supervision available to explicit reasoning chains, risking repeated reasoning patterns or drift from users’ true interest distributions.The introduction motivates ERL and PRL as strategies to address these risks.Source: §1 · p. 2
  • ERL combines multi-step supervision and representation-diversity regularization to form multi-order user representations and mitigate reasoning-pattern degradation.PRL uses progressive temperature annealing and reasoning-aware contrastive learning to refine interest modeling and improve robustness and generalization.Source: §1 · p. 2
  • Across five benchmarks, ReaRec gains 7.49% on average across all metrics with 3.51% additional inference latency, while raising backbone performance ceilings by approximately 30%-50%.Post-hoc analysis also finds steady quality improvements for users with sparse interactions and for long-tail items.Source: Figure 2 · p. 2, §1 · p. 2

2 Preliminary

Sequential recommendation predicts a user’s next interaction from historical sequences, encoding items and using a sequence representation to score candidate items. The section defines this task and outlines common pipeline and item-encoding choices.

2.1 Problem Definition

  • Each item has a unique ID and textual attributes that can be concatenated into an unordered-list description.The description represents each attribute as a key-value pair.Source: §2.1 · p. 3
  • The task is to predict each user’s next item from historical interactions by maximizing its prediction probability.For each user, the history is a chronological interaction sequence; the model encodes the observed subsequence as item embeddings before predicting the next item.Source: §2.1 · p. 3

2.2 Sequential Recommendation Pipeline

  • The typical pipeline encodes historical interactions as item embeddings, feeds them to a sequential model, then scores candidate items against the final-position sequence representation.Similarity scores, such as dot product or cosine similarity, are used to predict the probability of the next interaction.Source: §2.2 · p. 3
  • Item representations may use item IDs or text encoded by a pretrained language model, while this paper’s model-agnostic framework denotes them uniformly as item embeddings.Text-based encodings can use average pooling or hidden states from special positions such as [CLS] or the last position.Source: §2.2 · p. 3

3 Methodology

ReaRec extends sequential recommendation with multi-step implicit reasoning to refine user representations. The methodology also introduces two lightweight reasoning-learning methods and analyzes the framework.

3.1 ReaRec

  • ReaRec uses multi-step implicit reasoning to refine user representations and exploit sequential models’ computational potential to approximate users’ true interest distributions.It is presented as a simple, highly scalable alternative to direct recommendation without reasoning.Source: §3 · p. 3

3.2–3.4

  • The methodology introduces Ensemble Reasoning Learning and Progressive Reasoning Learning, then provides a deeper analysis of ReaRec, illustrated in Fig. 3.The two lightweight methods are presented as addressing challenges raised earlier in the paper.Source: §3 · p. 3

3.1 ReaRec Backbone

ReaRec adds autoregressive multi-pass inference to a sequential recommender and uses reasoning position embeddings to distinguish reasoning from item-sequence encoding.

3.1 ReaRec Backbone

  • ReaRec is model-agnostic and integrates with varied sequential recommenders; the paper illustrates the framework using the widely adopted transformer [57].Source: §3.1 · p. 3
  • The illustrated encoder sums item and absolute position embeddings, stacks self-attention and feed-forward layers, and conventionally uses the final-position output as the user representation.Source: §3.1 · p. 3

3.1.2 Extended Inference-Time Reasoning.

  • ReaRec autoregressively feeds the last-position hidden state back into the encoder for K passes, increasing inference-time computation to model intricate sequential dependencies.The method is intended to refine user-interest modeling beyond non-reasoning forward inference.Source: §3.1.2 · p. 4
  • At each reasoning step, a Reasoning Position Embedding distinguishes reasoning from sequence encoding and is added to the previous step’s last output.The original n item positions remain unchanged while the reasoning representation is appended.Source: §3.1.2 · p. 4
  • The straightforward reasoning-based representation uses the last reasoning output, rK, as the user representation and optimizes the recommendation objective with cross-entropy loss.The predicted probability is defined for the ground-truth next item.Source: §3.1.2 · p. 4
  • Without supervision for intermediate reasoning states, the naive objective risks reasoning-pattern degradation, motivating Ensemble Reasoning Learning and Progressive Reasoning Learning.Source: §3.1.2 · p. 4, Figure 3 · p. 4

3.2 Ensemble Reasoning Learning (ERL)

ERL trains on reasoning states from multiple steps as diverse views of evolving user interests, rather than relying only on the final reasoning output. It combines recommendation supervision with a KL-divergence constraint and pools all reasoning states for inference.

3.2 Ensemble Reasoning Learning (ERL)

  • ERL treats hidden states from different reasoning steps as multi-view representations of evolving user interests, aggregating them to avoid relying on the final output alone.This applies ensemble learning to combine diverse reasoning results.Source: §3.2 · p. 4

3.2.1 Multi-Step Reasoning Supervision.

  • For multi-step reasoning supervision, ERL applies cross-entropy to an ensembled sequence representation formed by average-pooling hidden states across all reasoning steps.This provides process guidance using multiple reasoning states rather than only the last-step state.Source: §3.2.1 · p. 4

3.2.2 KL Divergence Regularization.

  • To mitigate shortcut copying and pattern collapse, ERL encourages diversity by maximizing pairwise KL divergence between predictive distributions from different reasoning states.The KL constraint is combined with recommendation loss, with λ balancing the constraint strength.Source: §3.2.2 · pp. 4–5

3.2.3 Inference Phase.

  • At inference, ERL average-pools reasoning hidden states into a user representation, then ranks candidate items by inner product or cosine similarity.The similarity measure depends on the sequential recommendation algorithm; top-scoring items form the recommendation list.Source: §3.2.3 · p. 5

3.3 Progressive Reasoning Learning (PRL)

Progressive Reasoning Learning (PRL) sharpens predicted user-interest distributions across reasoning steps and adds contrastive learning to improve robustness against reasoning errors.

3.3 Progressive Reasoning Learning (PRL)

  • PRL progressively sharpens predicted distributions as inference computation increases, aiming to approximate users’ true preference distributions more accurately.The method is introduced as an alternative to Ensemble Reasoning Learning.Source: §3.3 · p. 5

3.3.1 Progressive Temperature Annealing (PTA).

  • Progressive Temperature Annealing (PTA) lowers the reasoning-step temperature, encouraging broad early exploration and a narrower later search while applying recommendation losses to each reasoning state.The temperature coefficient τ_k is controlled by a base temperature and a decay-rate hyperparameter; early exploration is intended to avoid local optima.Source: §3.3.1 · p. 5

3.3.2 Reasoning-aware Contrastive Learning (RCL).

  • Reasoning-aware Contrastive Learning (RCL) adds normally distributed noise to reasoning states and uses an InfoNCE objective to optimize a lower bound on mutual information between original and denoised states.This addresses accumulated reasoning bias by encouraging the model to retain essential sequential information across the two views.Source: §3.3.2 · p. 5

3.3.3 Inference Phase.

  • At inference, PRL uses the final reasoning state r_K as the user representation and scores candidate items against their embeddings to produce a recommendation list.Source: §3.3.3 · pp. 5–6

3.4 Discussion

ReaRec deepens sequential recommenders’ feature crossing through autoregressive reasoning, while ERL and PRL exploit that reasoning through multi-level feature integration and curriculum learning. With KV caching and few reasoning steps, its additional time cost is limited and its added embeddings are negligible in size.

3.4.1 Principle Analysis.

  • Autoregressive reasoning deepens feature crossing, while ERL integrates multi-level features and PRL progressively learns more complex intent evolution patterns.The framework aims to capture finer-grained sequence characteristics and move closer to the real user-interest distribution.Source: §3.4.1 · p. 6

3.4.2 Time and Space Complexity.

  • With K << C, reasoning adds O(L(KCd + Kd^2)) time and only Kd-dimensional reasoning-position embeddings, which are characterized as negligible in size.KV caching avoids redundant computations; the discussion gives K = 2 as an example of a typically small number of reasoning steps.Source: §3.4.2 · p. 6

4 Experiments · 4.1 Experimental Setup · 4.2 Overall Performance

Experiments on five real-world recommendation datasets benchmark ReaRec across ID-based and text-based sequential recommenders. ERL and PRL improve their backbones, while text-based baselines outperform ID-based models.

4.1.1 Datasets.

  • The evaluation uses five Yelp and Amazon datasets, with positive interactions defined by ratings above 3 and dataset-specific filtering and chronological splits.Yelp1 uses 20-core filtering and retains business name, location, and categories; Amazon 2023 covers four domains, retains product title, description, and price, filters by interaction count, and uses official absolute timestamps for splits.Source: §4.1.1 · p. 6

4.1.2 Evaluation Metrics.

  • Recommendation quality is measured with NDCG@{10,20} and Recall@{10,20}, assessing ranking quality and ground-truth retrieval in the top-k list.The metrics are described as widely used in sequential recommendation research [50] [70].Source: §4.1.2 · p. 6

4.1.3 Baselines.

  • The benchmark covers ID-based SASRec [31] and BERT4Rec [49], and text-based UniSRec [26] and MoRec backbones.The text-based methods use textual features, with MoRec incorporating text and visual encoders [38] [12].Source: §4.1.3 · pp. 6–7

4.2 Overall Performance

  • 6.76% and 8.21% are the average improvements of ERL and PRL on SASRec, while UniSRec gains 12.29% and 10.43%, respectively.These averages cover all metrics across five datasets; ReaRec adds latent-space inference-time computation to deepen feature crossing.Source: Table 2 · p. 7, Table 3 · p. 8, §4.2 · p. 8

4.1.4 Implementation Details.

  • Experiments use 8 NVIDIA A100 GPUs, 256-dimensional embeddings, batch size 2048, and sequences truncated to 50 items.For BERT4Rec, prefix masking keeps bidirectional attention for items and unidirectional attention for reasoning; item text is encoded with LLaMA-3.1-8B and reduced to 768 dimensions with PCA. ERL tunes λ over {0.001, 0.005, 0.01, 0.05, 0.1}; PRL sets γ=0.01 and tunes τ and α.Source: §4.1.4 · p. 7

4.2 Overall Performance

  • Across all datasets, text-based UniSRec and MoRec consistently outperform ID-based models; on Yelp, UniSRec improves NDCG@20 by 9.51% and Recall@20 by 14.14% over SASRec.The passage attributes this improvement to text-based models leveraging language models to encode item information, effectively mitigating data sparsity.Source: §4.2 · p. 7

4.3 Further Analysis

The analysis probes ReaRec’s subgroup robustness, reasoning-depth behavior and latency, component contributions, hyperparameter sensitivity, and hidden-state dynamics.

4.3.1 Robustness Analysis Across User and Item Subgroups.

  • 12.08%, 16.35%, and 18.69% NDCG@20 gains are reported for item group IG-1 as reasoning steps increase; gains favor short-sequence users and unpopular items, while longer sequences and popular items can decline.The authors attribute this to sparse signals needing more reasoning, whereas richer histories and well-trained popular-item representations may make further computation redundant or harmful.Source: §4.3.1 · pp. 8–9, Figure 4 · p. 9

4.3.3 Impact of Reasoning Steps on Inference Latency.

  • 7.49% average performance improvement across all metrics comes with only 3.51% latency overhead, and performance is generally optimal at two reasoning steps.KV caching reuses past key and value vectors, reducing attention computation complexity from O(N^2) to O(N).Source: Table 4 · p. 9, §4.3.3 · p. 10

4.3.2 Impact of Reasoning Steps on Recommendation Performance.

  • Naive multi-step reasoning can underperform the base model, RPE improves performance, and ERL and PRL stepwise supervision helps, but additional inference steps consistently reduce performance.The authors suggest excessive reasoning can cause overthinking and motivate adaptive inference-depth selection based on sequence complexity.Source: Figure 5 · p. 9, §4.3.2 · pp. 9–10

4.3.5 Sensitivity Analysis.

  • Moderate base temperature τ and decay rate α perform best, while ERL is usually insensitive to λ but degrades when λ exceeds a threshold.The tested α extremes of 1.0 and 10.0 are suboptimal, and performance drops significantly when λ exceeds 0.05; overly sharp or rapidly changing distributions and excessive regularization are cited as causes.Source: Figure 6 · p. 10, §4.3.5 · pp. 10–11

4.3.4 Ablation Study.

  • Removing KL regularization from ERL or RCL from PRL lowers NDCG@20, indicating both auxiliary components contribute to recommendation performance.Without KL, ERL may develop homogeneous reasoning outputs; without RCL, PRL lacks a robust mechanism to self-correct intermediate-state deviations.Source: Figure 7 · p. 10, §4.3.4 · p. 10

4.3.6 Embedding Visualization Analysis.

  • 1.00 and 0.98 similarity scores between RPE’s final output and the previous two steps indicate highly homogeneous reasoning states.PRL leverages reasoning-enhanced computation, while ERL without KL shows more overlapping patterns across steps; the visualizations support the proposed methods’ role in addressing reasoning-state degradation.Source: Figure 9 · p. 11, Figure 10 · p. 11, §4.3.6 · pp. 11–12

4.4 Case Studies

The case studies illustrate how multi-step reasoning can refine target-item rankings and progressively resolve ambiguity in a real-world recommendation scenario.

4.4 Case Studies

  • On Yelp, full PRL progressively improves the target item’s ranking as reasoning depth increases; smaller α produces smoother score-distribution changes, while larger α changes them more aggressively.Without RCL, reasoning errors can push the target down: its rank falls from #12 at step 1 to #22 at step 2.Source: §4.4.1 · p. 12

4.4.2 Case Study in Real-world Recommendation Scenario.

  • In the Amazon XBox gaming case, recommendations progress from an older shooter to an accessory and finally Resident Evil 2, matching the target and integrating temporal context, collaborative relevance, and diversity.The sequence moves from FPS preference at R0, to recent accessory habits at R1, to a newly released shooter at R2.Source: §4.4.2 · p. 12

5 Related Work

Sequential recommendation has evolved from Markov-based transitions to neural and attribute-aware models, but existing methods remain constrained by reasoning-free forward computation. This work brings implicit inference-time reasoning to sequential recommendation.

Sequential Recommendation

  • Sequential recommendation progressed from Markov methods to RNN, CNN, and Transformer models, alongside attribute-based approaches for sparsity and cold-start, yet remains reasoning-free at inference.The paper positions its contribution as extending sequential recommenders’ multi-step computational depth at inference time.Source: §5 · p. 12

Inference-time Reasoning

  • LLM inference-time scaling uses longer Chain-of-Thought reasoning to improve problem-solving, as illustrated by OpenAI O1 [28], Qwen QwQ, and DeepSeek R1 [18].The passage describes increased inference computation through token generation and extended reasoning space.Source: §5 · p. 13
  • Latent-space implicit reasoning has been explored in language and multimodal models, including Coconut [20] and Heima [46]; this work adapts that paradigm to sequential recommendation.The paper proposes two lightweight reasoning learning methods to push sequential recommenders’ performance ceiling.Source: §5 · p. 13

6 Conclusion and Future Work

ReaRec adds implicit multi-step inference and lightweight reasoning-based learning to sequential recommendation, with experiments supporting its effectiveness and a conclusion that frames the work as an initial step toward further research. The future-work agenda addresses adaptive reasoning depth, separating encoding from reasoning, inference-time scaling, theory, and efficiency.

6.1 Conclusion

  • By up to 50%, post-hoc optimal step selection raises existing SeqRec backbones’ performance ceiling; experiments across five real-world datasets validate ReaRec’s effectiveness and generalizability and report gains for long-tail users and items.ReaRec expands computational depth through multi-step implicit reasoning; Ensemble Reasoning Learning (ERL) and Progressive Reasoning Learning (PRL) are proposed to enhance reasoning robustness and effectiveness.Source: §6.1 · p. 13

6.2 Future Work

  • Adaptive inference depth is an open direction: extra steps can hurt high-activity users and popular items, while a gap remains between current performance and the post-hoc optimal-step upper bound.The proposed goal is a policy that balances computational depth with sequence complexity, addressing overthinking when shallow inference already captures well-learned preferences.Source: §6.2 · p. 13
  • Parameter disentanglement could separate item-sequence encoding from multi-step reasoning to reduce task interference and improve representations and recommendation quality.The current shared-parameter design is efficient but creates task ambiguity; reasoning position embeddings may not resolve it, given the observed improvement-then-decline trajectory as reasoning steps increase.Source: §6.2 · p. 13
  • Whether inference-time scaling laws exist for recommendation, and how to realize them, remain open questions because ReaRec does not consistently show the expected gains as reasoning steps increase.Further exploration could clarify model reasoning capabilities and advance reasoning-enhanced recommendation.Source: §6.2 · p. 13
  • Theoretical analysis of how multi-step reasoning captures higher-order sequential feature crossings could ground more principled model design and optimization.The paper presents this connection as an intuition to be established through future work.Source: §6.2 · pp. 13–14
  • Future efficiency work could use linear attention, model quantization, and long-to-short reasoning distillation to support lighter, faster inference at industrial scale.Although experiments report only marginal latency overhead, autoregressive generation may raise efficiency concerns as inference-time scaling advances.Source: §6.2 · p. 14

Related papers

Loading 2503.22675v3…