PaperlayerView exact revision on arXiv

Source-linked AI summary

Reason-to-Recommend: Using Interaction-of-Thought Reasoning to Enhance LLM Recommendation

Keyu Zhao, Fengli Xu, Yong Li

arXiv:2506.05069v2cs.IRcs.AI

TL;DR

Recommendation uses implicit user behavior, while LLM reasoning methods have largely developed in domains with explicit supervision. R2Rec turns user–item interaction chains into structured reasoning traces and trains the model with supervised fine-tuning and reinforcement learning. Across three datasets, it outperforms baseline models, with a 10.48% average improvement in HitRatio@1 over the second-best baseline.

  • Problem

    Recommendation relies on implicit user behavior and lacks explicit reasoning data or natural-language supervision, making direct application of reasoning methods developed for mathematics and code suboptimal.

    Source: §1 · p. 2

  • Method

    R2Rec converts sampled user–item interaction chains into structured reasoning traces and uses supervised fine-tuning followed by reinforcement learning to train LLMs to reason over them.

    Source: §1 · p. 2

  • Results

    R2Rec outperforms baseline models across three datasets, achieving a 10.48% average improvement in HitRatio@1 over the second-best baseline.

    Source: §4.2.2 · p. 8

  • Takeaways & Limitations

    The structured reasoning sequences reveal the decision process and are reported to enhance interpretability in recommendation.

    Source: §5 · p. 9

    LLM context-length constraints limit how many interaction-of-thought sequences can be processed simultaneously, potentially limiting effectiveness.

    Source: §5 · p. 9

Abstract

from arXiv · show

Driven by advances in Large Language Models (LLMs), integrating them into recommendation tasks has gained interest due to their strong semantic understanding and prompt flexibility. Prior work encoded user-item interactions or metadata into prompts for recommendations. In parallel, LLM reasoning, boosted by test-time scaling and reinforcement learning, has excelled in fields like mathematics and code, where reasoning traces and correctness signals are clear, enabling high performance and interpretability. However, directly applying these reasoning methods to recommendation is ineffective because user feedback is implicit and lacks reasoning supervision. To address this, we propose $\textbf{R2Rec}$, a reasoning-enhanced recommendation framework that samples interaction chains from the user-item graph and converts them into structured interaction-of-thoughts via a progressive masked prompting strategy, with each thought representing stepwise reasoning grounded in interaction context. This allows LLMs to simulate step-by-step decision-making based on implicit patterns. We design a two-stage training pipeline: supervised fine-tuning teaches basic reasoning from high-quality traces, and reinforcement learning refines reasoning via reward signals, alleviating sparse explicit supervision. Experiments on three real-world datasets show R2Rec outperforms classical and LLM-based baselines with an average $\textbf{10.48%}$ improvement in HitRatio@1 and $\textbf{131.81%}$ gain over the original LLM. Furthermore, the explicit reasoning chains enhance interpretability by revealing the decision process. Our code is available at: https://anonymous.4open.science/r/R2Rec-7C5D.

1 Introduction

LLMs are increasingly used in recommendation, but reasoning methods developed for explicitly supervised domains do not transfer readily to recommendation’s implicit user behavior. The paper introduces interaction-based reasoning traces and a two-stage training pipeline to address this gap.

1 Introduction

  • LLM recommendation draws on semantic understanding and world knowledge, while LLM reasoning research has pursued structured multi-step inference, especially in domains with explicit supervision.Prior recommendation approaches adapt LLMs or use them to add semantic features to traditional recommenders.Source: §1 · pp. 1–2
  • Recommendation relies on implicit user behaviors and lacks explicit reasoning data or natural-language supervision, limiting direct application of reasoning models developed for mathematics and code.The authors identify this mismatch as a gap in integrating robust reasoning into recommendation.Source: §1 · p. 2
  • R2Rec samples user-neighborhood interaction chains and uses progressive masked prompting to turn them into stepwise, interpretable reasoning traces.The traces represent interaction patterns between sampled users and items.Source: §1 · p. 2
  • A two-stage post-training pipeline uses supervised fine-tuning on annotated traces to teach reasoning, then reinforcement learning with reward signals to refine it despite scarce supervised data.The authors describe supervised fine-tuning as a warm start and reinforcement learning as scalable supervision.Source: §1 · p. 2
  • 10.48% average improvement in HitRatio@1 over state-of-the-art baselines on three real-world recommendation datasets.The framework also outperforms classical and LLM-based methods.Source: §1 · p. 2

2 Related Works

Related work covers graph-based recommendation, LLM reasoning, and LLM-based recommendation. The paper positions interaction chains as a way to bring structured reasoning to recommendation, where prior approaches often focus on semantic modeling or single-step prediction.

2.1 Graph-based recommendation

  • Graph-based recommenders use GNNs to aggregate neighborhood information and capture direct or high-order user–item relationships, but lack inherent reasoning and interpretability.The paper instead extracts interaction chains as structured reasoning paths for explainable recommendations.Source: §2.1 · pp. 2–3

2.2 LLM reasoning

  • LLM reasoning research uses reinforcement learning and test-time scaling for multi-step inference, but predominantly targets mathematics and programming rather than recommendation.Recommendation is characterized as involving implicit behavior and scarce explicit supervision.Source: §2.2 · p. 3

2.3 LLM-based recommendation

  • LLM-based recommenders either adapt LLMs as standalone systems or use them to generate semantic features for traditional recommenders, often neglecting structured reasoning over interactions.Examples include ChatRec, TALLRec, and LLaRA; some approaches rely on domain-specific pretraining or single-step prediction.Source: §2.3 · p. 3

3 Methods

R2Rec samples local user-item interaction chains, turns them into stepwise reasoning with progressive masked prompts, and trains LLMs through supervised fine-tuning followed by reinforcement learning. Its method is designed to preserve behavioral context while internalizing reasoning for recommendation.

3.2 Transforming Interaction Chain into Interaction-of-Thought

  • The method samples a target user's neighborhood to form short interaction chains, then transforms them into step-by-step reasoning chains with progressive, masked prompting.The chains retain behavioral context while remaining suitable for prompt-based input.Source: §3.2 · pp. 3–4

3.2.1 Interaction Chain

  • Each interaction chain forms a behavioral loop that starts and ends with the target user, using a sampled intermediate user and two shared items.The closure constraint is intended to avoid chains whose final item is unrelated to the target user.Source: §3.2.1 · p. 4

3.2.2 Interaction-of-thoughts

  • Progressive masked prompting injects the correct answer but instructs the LLM to ignore it, generating each reasoning step from prior steps as if the answer were unknown.The process represents reasoning generation as r = LLM(S, Q, A), where S is the prompting strategy, Q the predicted rating, and A the correct rating.Source: §3.2.2 · pp. 4–5, Figure 2 · p. 5

3.3.1 Supervised Fine-tuning

  • Supervised fine-tuning uses high-quality (Q, R, A) examples to train the LLM to generate reasoning chains from recommendation queries and correct answers.Although the answer is embedded in the prompt, the LLM is instructed to ignore it and simulate solving the problem from scratch.Source: §3.3.1 · pp. 5–6, Figure 3 · p. 6

3.3.2 Reinforcement Learning

  • R2Rec uses supervised fine-tuning to teach reasoning over interaction chains, then reinforcement learning to refine reasoning through reward-driven optimization.The supervised stage trains on (Q, R, A) examples, while the reinforcement-learning stage is intended to generalize beyond limited annotated examples without additional labeled reasoning chains.Source: §3.3.1 · p. 5, §3.3.2 · p. 6
  • The reinforcement-learning reward combines a reasoning-step format signal and a ranking-correctness signal, with weights of 1 and 2, respectively.GRPO computes group-normalized advantages over sampled completions without a separate value network.Source: §3.3.2 · p. 6

4 Experiments

Experiments evaluate R2Rec on three recommendation datasets against classical and LLM-based methods, then test its components, transferability, and interpretability. R2Rec leads the baselines on HitRatio@1, while ablations and cross-domain tests examine the contributions and reach of its reasoning approach.

4.1.1 Datasets / 4.2 Performance Comparsion

  • Evaluation uses MovieLens-1M, Amazon Book, and Amazon Electronics, ranking one recent positive item against 19 sampled negatives for 1,000 users per dataset.Users have at least six ratings above 3; results use HitRatio@1, HitRatio@3, and NDCG@3.Source: §4.1.1 · p. 7, §4.2 · p. 7

4.1.2 Implementation Details / 4.2.1 Baselines

  • The comparison includes graph and sequential classical recommenders, prompt- or fine-tuning-based LLM recommenders, and DeepSeek-R1-Distill-Llama-8B as the backbone.The SFT and RL stages train LoRA adapters for 10 and 1 epochs, respectively, using 500 samples each.Source: §4.2.1 · p. 7, §4.1.2 · p. 7

4.2.2 Results

  • 10.48% average HitRatio@1 gain over the second-best baseline; R2Rec leads on MovieLens, Amazon-Book, and Amazon-Electronics.The reported relative gains on those datasets are 6.32%, 10.99%, and 14.14%, respectively.Source: Table 1 · p. 7, §4.2.2 · p. 8
  • Classical methods consistently underperform R2Rec; among LLMs, o4-mini beats GPT-4o-mini, while task-aligned recommenders sometimes trail o4-mini.The authors relate the comparisons to semantic understanding and reasoning over user–item interactions.Source: §4.2.2 · p. 8

4.3.1 Ablation Study of Interaction Chain

  • Removing reasoning chains lowers performance across datasets, while removing interaction chains causes the largest prompt-component drop; Amazon Electronics HitRatio@1 falls from 0.192 to 0.072.The authors describe interaction chains as the behavioral context from which reasoning chains are derived.Source: §4.3.1 · p. 8

4.3.2 Ablation Study of Training Stage

  • Without SFT, MovieLens HitRatio@1 falls from 0.404 to 0.156; removing RL also degrades results, but less severely.The authors report higher, more stable RL rewards with SFT initialization than training from scratch.Source: §4.3.2 · p. 8

4.4 Transferability

  • A MovieLens-trained LLM reaches 0.502 HitRatio@1 on Amazon Book, a result described as competitive and better than several in-domain baselines.The authors attribute cross-domain generalization to structural patterns in user–item interactions rather than memorized domain-specific behavior.Source: §4.4 · p. 9, Table 3 · p. 9

4.5 Case Study

  • In the case study, shared ratings of Antz and Aladdin refine a young user’s inferred preference toward animation and lighthearted themes over musicals, leading to Pocahontas.The inference also draws on the user’s demographic profile and high rating for A Bug’s Life.Source: §4.5 · p. 9, Figure 4 · p. 9

5 Conclusions

The conclusion presents interaction-of-thoughts and a two-stage SFT-plus-RL pipeline for preference modeling, reporting performance gains, generalization, and interpretability. It identifies LLM context length as a constraint on how many reasoning sequences can be processed at once.

5 Conclusions

  • R2Rec derives structured reasoning sequences from user–item interaction chains to guide LLM preference modeling, then combines supervised fine-tuning and reinforcement learning to internalize the reasoning.The conclusion reports performance gains, improved generalization, and enhanced interpretability across diverse recommendation domains.Source: §5 · p. 9
  • LLM context length limits how many interaction-of-thought sequences can be processed simultaneously, potentially limiting effectiveness.The conclusion suggests retrieval-augmented prompting and more compact encoding as future directions.Source: §5 · p. 9

A.1 The progressive, masked prompting strategy to generate Interaction-of-Thoughts

The prompt turns user–movie interaction chains into progressive, masked reasoning traces. Each step uses only information available before that hop, and the reasoning is meant to reach the correct answer without seeing it in advance.

A.1 The progressive, masked prompting strategy to generate Interaction-of-Thoughts

  • The example chain records users’ ratings and basic user or movie information as interactions connect the target user, movies, and another user.The prompt illustrates the chain using shared ratings of two movies and the target user’s ratings of those movies.Source: §A.1 · p. 14
  • Reasoning proceeds hop by hop: each step relies on prior reasoning and may use only information before the current hop.The prompt also instructs the model not to include information about the final answer in its reasoning process.Source: §A.1 · p. 14
  • The prompt asks the model to output both its reasoning process and the answer, with the answer consistent with the correct one supplied for the task.The supplied answer is framed as guidance for generating the reasoning trace, not information to use in advance.Source: §A.1 · p. 14

A.2 Reward Function Design

The RL reward combines reasoning-step quality with ranking correctness. The two components target appropriately sized reasoning traces and higher placement of the ground-truth item.

A.2 Reward Function Design

  • The final RL reward combines reasoning-step and ranking-correctness rewards.The step reward discourages reasoning that is too short or unnecessarily long, while ranking correctness rewards placing the ground-truth item higher.Source: §A.2 · pp. 14–15

A.3 Reward Curve Analysis During RL Training

SFT initialization is associated with stronger and steadier reward trajectories during RL training than training directly with RL from scratch. The scratch-trained model remains lower and more volatile, converges more slowly, and does not reach the SFT-initialized model's performance.

A.3 Reward Curve Analysis During RL Training

  • SFT-initialized RL training starts at a higher reward and grows steadily, while RL from scratch has lower, more volatile rewards and fails to reach its performance.The authors interpret this as SFT providing a reasoning foundation that helps the model benefit from RL reward signals.Source: §A.3 · p. 15, Figure 5 · p. 15

A.4 Compute Resources

The paper reports GPU resources and approximate runtimes for SFT, RL, and evaluation.

A.4 Compute Resources

  • SFT used one A100 GPU for about 2.5 hours per run; GRPO-based RL used two A100 GPUs for about 8 hours per run.Evaluation and inference used one A100 GPU, with average inference time of about 2 seconds per user.Source: §A.4 · p. 15

A.5 Ethical Considerations

The study used publicly available, anonymized datasets and reports that it collected no private user data or involved human subjects.

A.5 Ethical Considerations

  • MovieLens, Amazon Books, and Amazon Electronics are publicly available, anonymized datasets with no personal or identifiable information.The paper describes them as widely used in recommendation research.Source: §A.5 · p. 15
  • The study involved no human subjects and collected or processed no private user data.The authors report no foreseeable privacy, fairness, or safety risks and compliance with the NeurIPS Code of Ethics.Source: §A.5 · p. 16
Loading 2506.05069v2…