PaperlayerView exact revision on arXiv

Source-linked AI summary

ThinkRec: Thinking-based recommendation via LLM

Qihang Yu, Kairui Fu, Zheqi Lv, Shengyu Zhang, Xinhui Wu, Chen Lin, Feng Wei, Bo Zheng, Fei Wu

arXiv:2505.15091v4cs.IRcs.AI

TL;DR

Many LLM-based recommendation methods match items from interaction histories without explicit reasoning through deeper behavioral logic. ThinkRec adds reasoning-augmented training and user-specific expert fusion; experiments on three real-world datasets report higher average AUC and METEOR than state-of-the-art baselines, alongside traceable reasons.

  • Problem

    Many LLM4Rec methods match items from click histories but do not explicitly reason through deeper behavioral logic, limiting their treatment of user preferences.

    Source: §1 · pp. 1–2

  • Method

    ThinkRec augments item metadata with keywords, trains on synthesized reasoning traces alongside recommendation data, and fuses recommendation experts using latent user features.

    Source: §1 · pp. 2–3

  • Results

    ThinkRec average outperforms state-of-the-art baselines by 7.96% in AUC and 56.54% in METEOR across three real-world recommendation datasets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    ThinkRec produces traceable reasons and the authors report improved recommendation accuracy and transparency within their evaluated setting.

    Source: §5 · p. 8

Abstract

from arXiv · show

Recent advances in large language models (LLMs) have enabled more semantic-aware recommendations through natural language generation. Existing LLM for recommendation (LLM4Rec) methods mostly operate in a System 1-like manner, relying on superficial features to match similar items based on click history, rather than reasoning through deeper behavioral logic. This often leads to superficial and erroneous recommendations. Motivated by this, we propose ThinkRec, a thinking-based framework that shifts LLM4Rec from System 1 to System 2 (rational system). Technically, ThinkRec introduces a thinking activation mechanism that augments item metadata with keyword summarization and injects synthetic reasoning traces, guiding the model to form interpretable reasoning chains that consist of analyzing interaction histories, identifying user preferences, and making decisions based on target items. On top of this, we propose an instance-wise expert fusion mechanism to reduce the reasoning difficulty. By dynamically assigning weights to expert models based on users' latent features, ThinkRec adapts its reasoning path to individual users, thereby enhancing precision and personalization. Extensive experiments on real-world datasets demonstrate that ThinkRec significantly improves the accuracy and interpretability of recommendations. Our implementations are available at https://github.com/Yu-Qi-hang/ThinkRec.

1 Introduction

Prior LLM4Rec methods often match items from click histories without reasoning through deeper behavioral logic. ThinkRec combines reasoning activation and personalized expert fusion, and outperforms state-of-the-art baselines on three real-world datasets.

1 Introduction

  • Prior LLM4Rec methods tend to match similar items from click histories while overlooking deeper behavioral logic, as illustrated by a genre-based but unsuitable Hyperion recommendation.The example user likes several science-fiction works but might dislike Hyperion’s philosophical or metaphysical themes.Source: §1 · pp. 1–2
  • ThinkRec activates reasoning with item metadata keywords and synthesized reasoning traces, then uses latent user features to personalize recommendation experts and their fusion.Its reasoning chain analyzes interaction histories, identifies preferences, and gives reasons for recommendations.Source: §1 · pp. 2–3
  • ThinkRec average outperforms state-of-the-art baselines by 7.96% in AUC and 56.54% in METEOR across three real-world recommendation datasets.The authors report these averages as evidence of the framework’s effectiveness.Source: §1 · p. 2

2 Related Work

Prior LLM recommendation research spans scoring, item generation, hybrid modeling, and newer multi-step reasoning approaches. ThinkRec addresses the stated gaps in explicit reasoning supervision and fine-grained user modeling through reasoning-augmented training and personalized expert fusion.

2.1 LLM-based Recommendation

  • LLM recommendation methods include prompt-based scoring, item-ID generation, and hybrid models that combine prediction, ranking, or list generation.These methods differ in output formats and representation strategies, but often treat LLMs as static scorers or selectors with limited interpretability.Source: §2.1 · p. 3
  • Recent approaches use modular reasoning, interactive-agent pipelines, instruction tuning, or chain-of-thought prompting to support multistep inference and personalized decisions.Chain-of-thought, ReAct, and Tree-of-Thoughts generate intermediate steps, while process rewards and reasoning distillation further reinforce reasoning.Source: §2.1 · p. 3
  • Explicit reasoning supervision and fine-grained user modeling remain underexplored; ThinkRec combines reasoning-augmented training with personalized expert fusion.The related-work discussion positions this approach alongside research on structured reasoning processes.Source: §2.1 · p. 3

3 Method

ThinkRec frames sequential recommendation as predicting an item preference from a user’s interaction history and item text. It enriches item information, synthesizes reasoning examples, jointly trains recommendation and reasoning objectives, and organizes the method around four stages.

3.1 Preliminary

  • Sequential recommendation represents each user’s prior interactions as a chronological history and predicts whether the user will enjoy a target item, with an accompanying reason.The paper also describes aligning collaborative user and item embeddings with the LLM’s language space.Source: §3.1 · p. 3

3.2 Thinking Enhanced Recommendation

  • ThinkRec’s thinking-enhanced framework combines data construction with joint training to align recommendation and language-modeling objectives.The overview also shows global and user-grouped base LoRAs, with the selected or fused adapter determined by similarity to user features.Source: §3.2 · p. 3, Figure 2 · p. 4

3.2.1 Reasoning data construction.

  • Item augmentation retains up to 10 representative metadata keywords, includes low-scoring interactions, and adds explanations synthesized from sampled training examples.The reasoning model is repeatedly queried until it predicts correctly, and its latest explanation is recorded as the reason.Source: §3.2.1 · pp. 3–4

3.2.2 Thinking activation.

  • Thinking activation mixes binary-labeled recommendation instances with synthesized reasoning examples, training the model to predict outcomes and generate explanations in a shared representation space.Thinking instances use token-level cross-entropy, while recommendation instances use binary cross-entropy; instance type determines the loss weights.Source: §3.2.2 · p. 4

3.3 Recommendation Experts Fusion

ThinkRec forms user-specialized experts from clustered collaborative representations, then uses user-feature matches to weight or route recommendations across experts. Its staged training combines text-based and collaborative models, grouped expert adaptation, and a projector.

3.3.1 Base Expert fine-tuning.

  • Unsupervised clustering of pretrained collaborative user embeddings partitions users, and grouped data fine-tunes the global LoRA’s last eight layers into base experts.The global expert is first fine-tuned on all data to preserve general thinking capability while adapting to finer-grained preferences.Source: §3.3.1 · pp. 4–5

3.3.2 Instance Wise Expert Fusion.

  • Cosine similarity and softmax convert matches between user features and group-mean expert representations into user-specific participation weights.These weights estimate each base expert’s involvement in modeling a user's preferences.Source: §3.3.2 · p. 5
  • Users with highly averaged or concentrated preference profiles are routed directly to a global or base expert; remaining users receive instance-wise fusion.The stated thresholds are H(w_u) > 0.95 log N or max(w_u) > 0.5 + 0.6/N.Source: §3.3.2 · p. 5

3.4 Staged Training

  • Training fits full-data text-only and ID-based models in parallel, then adapts the global LoRA’s last eight layers on grouped data and trains a projector with that LoRA fixed.The projector maps collaborative embeddings into the LLM language space using a global dataset with injected collaborative signals.Source: §3.4 · p. 5

4 Experiments

Experiments compare ThinkRec with traditional and LLM-based recommenders across three real-world datasets, then examine recommendation accuracy, generated reasons, expert fusion, cross-domain transfer, and inference efficiency. ThinkRec performs strongly across recommendation and reason-quality metrics, while ablations and expert analyses show that reasoning supervision and personalization matter.

4 Experiments

  • Experiments use timestamp-based train/validation/test splits on ML1M, Yelp, and Book, and compare traditional and LLM-based methods using ranking and generated-reason metrics.Ratings are binarized with thresholds of 3 for ML1M and Yelp and 4 for Book; the evaluation includes AUC, UAUC, NDCG, MAP, METEOR, and BLEURT.Source: §4.1.1 · p. 5, §4.1.2 · pp. 5–6, §4.1.3 · p. 6

4.2.1 Accuracy of Recommendation.

  • +0.0582 (9.13%) on AUC in Yelp and +0.0623 (8.72%) on AUC in ML1M over CoLLM; ThinkRec leads almost all metrics on ML1M and Yelp, and AUC and UAUC on Book.The comparison table reports prediction performance across the three datasets.Source: Table 2 · p. 6, §4.2.1 · p. 6

4.2.2 Quality of Generated Reasons.

  • +56.54% on METEOR and +23.35% on BLEURT are ThinkRec’s average relative improvements over three LLM-based baselines for generated reasons.The case study reports structured, step-by-step reasoning aligned with user history and target-item semantics, unlike disorganized or unrelated outputs from other LLM-based methods.Source: §4.2.2 · p. 6, Table 3 · p. 7, §4.2.3 · p. 7

4.3.2 Study on the Fusion of Experts (RQ3).

  • Performance across expert counts rises and then falls, with 2–3 groups particularly improving UAUC; performance also improves as user-group differences increase.Further partitioning reduces subgroup training data and can increase overfitting; AUC consistently declines as expert count grows. Figure 3 covers Book and Yelp, while Figure 4 relates performance to Cohen’s d.Source: Figure 3 · p. 7, §4.3.2 · pp. 7–8, Figure 4 · p. 8

4.3.1 Ablation Studies (RQ2).

  • Removing thinking supervision lowers performance across datasets; on Book, UAUC falls from 0.5705 to 0.4692, while removing experts also reduces performance.The authors report that multi-expert modeling brings substantial benefits only when thinking is enabled, with reasoning supervision supporting specialization to user groups.Source: Table 4 · p. 7, §4.3.1 · p. 7

4.3.3 Cross-Domain Generalization.

  • Reasoning paths learned with the global expert on ML1M retain more than 90% of domain-specific performance when tested on Book and Yelp.The cross-domain evaluation is summarized in Table 6.Source: Table 6 · p. 8, §4.3.3 · p. 8

4.3.4 Inference Efficiency.

  • Inference uses one LLM forward pass with a fused LoRA adapter and lightweight gating; latency increases by less than 10% relative to a standard LoRA-tuned LLM.LoRA adapters add less than 1.5% extra parameters.Source: §4.3.4 · p. 8

5 Conclusion

ThinkRec shifts LLM-based recommendation toward explicit reasoning through thinking activation and instance-wise expert fusion. The paper reports improved accuracy, transparency, and user trust, and points to deeper reasoning and personalized selection as future directions.

5 Conclusion

  • ThinkRec advances LLM-based recommendation from intuitive System 1 matching toward explicit System 2 reasoning.The framework combines thinking activation with instance-wise expert fusion.Source: §5 · p. 8
  • ThinkRec generates traceable reasons alongside recommendations, with reported gains in accuracy, transparency, and user trust.Source: §5 · p. 8
  • The paper identifies deeper cognitive insights, activation steering, and multi-agent collective intelligence as possible future directions.Source: §5 · p. 8

A Ethical Considerations

The study uses publicly available recommendation benchmarks and reports that they contain no personally identifiable information or sensitive attributes. It involves no human participants and states that data use complies with ACM policy.

A Ethical Considerations

B Prompt Templates

The appendix supplies prompt templates for metadata summarization, reasoning-data synthesis, and recommendation. The templates request item keywords, iterative reflection, and a preference persona informed by user and item features.

B.1 Prompt for Summarizing Metadata

  • The metadata summarization prompt requests up to ten keywords from a book profile, separated by commas.Source: §B.1 · p. 10

B.2 Prompt for Reasoning Data Synthesis

  • The first reasoning-data synthesis turn asks whether a user who rated listed books highly would enjoy a target book.Source: §B.2 · p. 10
  • The reflect-turn templates ask the model to reconsider its prediction using historical information and explain an oversight from earlier analysis.Source: §B.2 · p. 10
  • The reflect-turn prompt variants use different wording to request reanalysis of the target-book prediction.Source: §B.2 · p. 10

B.3 Prompt for Recommendation

  • The recommendation prompt asks the model to construct a preference persona from historical ratings and descriptions, using UserID and TargetItemID features.Source: §B.3 · p. 11
  • The recommendation prompt directs the analysis to begin with “Yes” or “No” before evaluating enjoyment of the target book.Source: §B.3 · p. 11

C Case Study

The case study presents a ThinkRec prediction that a user would enjoy a target book, supported by matching themes in the user's history. The appendix also includes baseline prompt and code excerpts, including an API description.

C Case Study

  • The TALLRec excerpt contains a Python script that groups some rated books by matching titles and genre-related conditions.Source: C Case Study · p. 11
  • The accompanying API description says to send a JSON object with a books array of titles to the books-rating endpoint.Source: C Case Study · p. 11
  • ThinkRec predicts “Yes” for “If I Run,” linking the user’s history of thrillers, suspense, family themes, and investigations to the target’s themes.The explanation also notes that marriage and divorce are not explicitly present in the user’s history.Source: C Case Study · p. 11

D Implementation Details

The implementation uses standard binary cross-entropy and optimizer choices, with separate tuning and sequence-length settings for traditional and LLM-based methods. LoRA, data-sampling, loss-weighting, and temperature settings follow specified configurations.

D Implementation Details

  • LLM-based methods use AdamW and other models use Adam, with binary cross-entropy adopted unless otherwise specified.Source: D Implementation Details · p. 11
  • Traditional models tune learning rate, embedding size, and weight decay; LLM-based methods use a 1e-4 learning rate and 1e-3 weight decay.Source: D Implementation Details · p. 11
  • LoRA uses r=8, alpha=16, dropout=0.05, and q/v projection targets; reasoning and recommendation data rates are 0.2 and 0.8, respectively.Loss weights are alpha=0.1, beta=0.9, eta=0.9, gamma=0.1, and temperature tau=0.1.Source: D Implementation Details · p. 11

E Notation

The notation table defines symbols for user behaviors, items, recommendation explanations, embeddings, expert weights, model functions, and losses.

E Notation

  • The notation distinguishes a user's history and current behavior, item identifiers and text fields, recommendation explanations, and binary predicted and ground-truth labels.Source: Table 7 · p. 12, E Notation · p. 12
  • Embedding notation covers user/item representations, text-token and first-token embeddings, projected language-space embeddings, and question and answer embeddings.Source: Table 7 · p. 12, E Notation · p. 12
  • The table also defines expert participation weights, embedding dimensions, token positions and IDs, temperature and loss coefficients, and functions used for encoding, projection, language modeling, similarity, and loss.Source: Table 7 · p. 12, E Notation · p. 12
Loading 2505.15091v4…