PaperlayerView exact revision on arXiv

Source-linked AI summary

Review-driven Personalized Preference Reasoning with Large Language Models for Recommendation

Jieyong Kim, Hyunseo Kim, Hyunjin Cho, SeongKu Kang, Buru Chang, Jinyoung Yeo, Dongha Lee

arXiv:2408.06276v5cs.CL

TL;DR

Existing LLM recommenders may have limited user and item context or lack rich reasoning, while review-derived explanations can be poorly grounded. EXP3RT distills a model to extract review preferences, build profiles, and reason step by step; experiments report improved rating prediction and reranking, with explainable recommendations.

  • Problem

    Existing methods often use limited user and item information or short outputs, and review-derived explanation targets may lack grounding and generalize poorly.

    Source: §1 · p. 2

  • Method

    EXP3RT distills a student LLM to extract review preferences, construct user and item profiles, and predict ratings through step-by-step reasoning.

    Source: §1 · p. 2

  • Results

    Experiments report improved rating prediction and reranking accuracy, alongside logical and understandable explanations for recommendation.

    Source: §1 · p. 2

  • Takeaways & Limitations

    EXP3RT can operate independently or rerank conventional collaborative-filtering candidates in a multi-stage top-k recommendation pipeline.

    Source: §1 · p. 2

Abstract

from arXiv · show

Recent advancements in Large Language Models (LLMs) have demonstrated exceptional performance across a wide range of tasks, generating significant interest in their application to recommendation systems. However, existing methods have not fully capitalized on the potential of LLMs, often constrained by limited input information or failing to fully utilize their advanced reasoning capabilities. To address these limitations, we introduce EXP3RT, a novel LLM-based recommender designed to leverage rich preference information contained in user and item reviews. EXP3RT is basically fine-tuned through distillation from a teacher LLM to perform three key tasks in order: EXP3RT first extracts and encapsulates essential subjective preferences from raw reviews, aggregates and summarizes them according to specific criteria to create user and item profiles. It then generates detailed step-by-step reasoning followed by predicted rating, i.e., reasoning-enhanced rating prediction, by considering both subjective and objective information from user/item profiles and item descriptions. This personalized preference reasoning from EXP3RT enhances rating prediction accuracy and also provides faithful and reasonable explanations for recommendation. Extensive experiments show that EXP3RT outperforms existing methods on both rating prediction and candidate item reranking for top-k recommendation, while significantly enhancing the explainability of recommendation systems.

1 INTRODUCTION

Existing LLM recommenders often use limited user and item information and short outputs, while review-derived explanations may lack grounding and generalize poorly. EXP3RT distills a student LLM to build preference profiles and reason step by step about ratings; experiments report stronger rating prediction, reranking, and explainability.

1 INTRODUCTION

  • Existing methods often rely on limited user and item context, which restricts preference understanding, and short outputs provide little context for accurate predictions or logical explanations.The cited examples include IDs, titles, and interaction history as inputs, and yes/no answers or rating scores as outputs.Source: §1 · p. 2
  • Explanation targets taken directly from reviews can fail to generalize to unobserved interactions when they lack evidence grounding or logical reasoning.Some prior methods use a review’s first sentence or a brief summary as the target explanation.Source: §1 · p. 2
  • A teacher LLM annotates rating-aligned reasoning for observed interactions, and a student LLM learns to generate reasoning before predicting ratings.The paper describes this distillation as enabling domain-specific adaptation while reducing computational costs.Source: §1 · p. 2
  • EXP3RT extracts preferences from reviews, aggregates them into user and item profiles, then uses step-by-step reasoning to predict ratings from profile and item-description information.The profiles are lists of key preference descriptions that help mitigate noise in raw reviews.Source: §1 · p. 2
  • Experiments report improved rating prediction and reranking accuracy, with logical, understandable explanations and effective reranking of conventional CF candidates for top-k recommendation.EXP3RT can also function independently as a recommender.Source: §1 · p. 2
  • EXP3RT uses rich review-derived preferences to produce detailed reasoning intended to provide faithful and logical recommendation explanations.The paper presents this reasoning as supporting both rating accuracy and explainability.Source: §1 · p. 2

2 RELATED WORK

Prior LLM recommenders cover prediction, item generation, reranking, and review-based explanation tasks, but the paper identifies gaps in preference context, textual reasoning, and explanation generalization. It positions EXP3RT’s review-derived profiles and reasoning as responses to those gaps.

2.1 LLM-based Recommenders for Prediction

  • Table 1 compares LLM recommenders by target task; its caption identifies P5 as supporting both reranking and explainability through multi-task learning.The caption defines rerank-ability as functioning as a candidate-item reranker in a multi-stage top-k pipeline.Source: Table 1 · p. 3
  • LLM recommenders address binary and rating prediction, item generation, sequential recommendation, candidate reranking, and profile-based top-k recommendation.Examples include TALLRec, P5, GenRec, LlamaRec, and Zero-Shot Ranker.Source: §2.1 · pp. 2–3
  • The paper argues that prior prediction methods often use limited user and item context and short-form outputs, leaving advanced textual reasoning unexplored.It motivates using reviews and distillation to support rating prediction through enhanced reasoning.Source: §2.1 · p. 3

2.2 LLM-based Recommenders for Explanation

  • Prior explanation approaches use reviews, features, or collaborative signals, while review-based targets may generalize poorly when they lack evidence grounding or logical reasoning.The paper responds by aggregating preference insights across multiple reviews into structured collective knowledge.Source: §2.2 · p. 3

3 PROPOSED METHOD: EXP3RT

Exp3rt distills a teacher LLM into a student that extracts review preferences, builds user and item profiles, and predicts ratings through textual reasoning. It also supports continuous rating prediction and reranks candidate items supplied by collaborative-filtering models.

3.1.2 Knowledge distillation from teacher to student LLM.

  • GPT-3.5-generated training outputs distill reasoning capabilities into a Llama3-8B student fine-tuned for Exp3rt’s recommendation tasks.The paper describes teacher-generated training data followed by supervised fine-tuning of the student.Source: §3 · p. 3, §3.1.2 · p. 4

3.2 Preference Extraction from Reviews

  • Exp3rt extracts key preferences from raw reviews into structured descriptions that categorize inferred user tastes as likes or dislikes.The preference descriptions are organized as lists.Source: §3.2 · p. 4

3.3 User and Item Profile Construction

  • User and item profiles aggregate preferences extracted from their respective review sets into structured summaries of subjective preferences.The paper states that these profiles reduce review noise and capture key preferences and characteristics.Source: §3.3 · p. 4

3.4 Reasoning-enhanced Rating Prediction

  • For rating prediction, Exp3rt reasons step by step by matching user-profile preferences against item-profile characteristics and objective item descriptions.A teacher LLM generates detailed reasoning for observed interactions using both profiles, item descriptions, and the actual rating.Source: §3.4 · pp. 4–5

3.5 Optimization

  • Instruction tuning combines the three task datasets, and the final negative log-likelihood objective updates only QLoRA parameters.The student model’s original parameters remain unchanged during training.Source: §3.5 · p. 5

3.6 Inference

  • At inference, Exp3rt builds profiles from user and item review preferences; it uses all available user reviews and, by default, all item preferences.Item profiles can instead select helpful-review preferences or preferences from users similar to the target user; the rating prompt also includes average user and item ratings.Source: §3.6 · p. 5

3.6.1 Rating Score Prediction.

  • Expected ratings from the model’s integer-rating probabilities provide continuous predictions despite training on discrete rating tokens.The probabilities are computed from the rating-token logits using softmax.Source: §3.6.1 · p. 5

3.6.2 Top-k Recommendation with Exp3rt.

  • For top-k recommendation, collaborative-filtering models generate candidates efficiently and Exp3rt reranks them by predicted rating.The pipeline combines CF candidate-generation efficiency with LLM-based reranking accuracy.Source: §3.6.2 · pp. 5–6

4 EXPERIMENTS

The experiments evaluate rating prediction, candidate reranking, explanation generation, and component configurations on IMDB and Amazon-Book. The setup specifies task-specific baselines and metrics alongside teacher–student models and training procedures.

4 EXPERIMENTS

  • The experiments ask how Exp3rt performs in rating prediction, top-k reranking, and personalized reasoning, and how its components contribute to performance.The study evaluates these questions on two datasets.Source: §4 · p. 6

4.1.1 Datasets.

  • IMDB provides movie ratings on a 1–10 scale, while Amazon-Book uses a 1–5 scale and both datasets include item metadata and rated user reviews.After 5-core filtering, the timestamp-based train/validation/test split is 8:1:1; the datasets contain approximately 19,124 IMDB and 118,705 Amazon-Book interactions.Source: §4.1.1 · p. 6

4.1.2 Baselines.

  • Baselines cover conventional and review-enhanced collaborative filtering, LLM rating prediction, candidate reranking, and explanation generation.The compared methods include MF and WDL, RGCL and P5, prompting-based LLMRec, TALLRec and ZS-Ranker, and XRec.Source: §4.1.2 · p. 6

4.1.3 Evaluation Metrics.

  • Rating prediction is evaluated with MAE and RMSE, while top-k recommendation uses Recall and NDCG.Source: §4.1.3 · p. 6

4.1.4 Implementation Details.

  • GPT-3.5 supplies teacher training data and Llama3-8B-Instruct is the student; experiments average three random-seed runs with AdamW training.The setup uses early stopping and dataset-specific QLoRA hyperparameters.Source: §4.1.4 · pp. 6–7

4.2 Rating Prediction Performance (RQ1)

Exp3rt outperforms the compared methods on both rating-prediction datasets. It also improves results in warm- and cold-start settings, with a larger advantage over baselines in cold-start cases.

4.2 Rating Prediction Performance (RQ1)

  • Exp3rt consistently outperforms all compared methods on both datasets, including strong CF and LLM-based baselines.The authors attribute the gains to leveraging LLM reasoning for rating prediction.Source: §4.2 · p. 7
  • The cold-start evaluation defines warm-start users as having more than three training interactions and uses Amazon-Book to identify cold-start users and items.The unseen set contains interactions from users absent from training; Amazon-Book is described as sparse.Source: §4.2 · p. 7
  • Exp3rt improves performance in both warm- and cold-start settings, with a larger baseline gap in cold-start than in the original test set.The authors report that it can predict ratings for unseen users or items when their reviews are available, unlike transductive CF methods.Source: §4.2 · p. 7

4.3 Top-k Recommendation Performance (RQ2)

On IMDB, Exp3rt reranks CF-generated candidate items and improves recommendation accuracy. The evaluation focuses on prioritizing highly preferred items among retrieved candidates.

4.3 Top-k Recommendation Performance (RQ2)

  • The IMDB test set contains only highly preferred items rated 9 or 10, selected from interactions rated 6 or higher.Those highly preferred items are evenly split between training and testing to assess whether the reranker places them near the top.Source: §4.3 · p. 8
  • BPR-MF and LightGCN retrieve the top 20 candidates per user, which Exp3rt and baseline rerankers reorder using predicted scores or relevance.Source: §4.3 · p. 8
  • Exp3rt considerably improves recommendation accuracy by reranking candidate items, while other LLM rerankers sometimes reduce accuracy.The comparison focuses on distinguishing highly preferred from less preferred items.Source: §4.3 · p. 8

4.4 Explanation Quality (RQ3)

Human evaluation and qualitative comparisons favor Exp3rt’s recommendation explanations. The evaluation considers multiple dimensions of explanation quality, while the reported comparison is bounded by the described datasets and examples.

4.4 Explanation Quality (RQ3)

  • The human evaluation samples 100 examples from each dataset and asks three judges per example to compare explanation quality.The passage introduces five evaluation criteria for those comparisons.Source: §4.4 · p. 8
  • Amazon-Book is excluded from the reranking evaluation because its rating distribution is skewed toward ratings near 5.Source: §4.4 · p. 8
  • The five criteria are persuasiveness, reasonability, scrutability, faithfulness, and overall explanation quality.The criteria assess convincingness, logical rationale, preference understanding, accuracy about item attributes, and general explanation quality.Source: §4.4 · p. 8
  • Exp3rt beats all baselines by a large margin across the human-rated explanation criteria, with particularly strong persuasiveness.The authors describe its reasoning as combining user/item profile preferences with objective item-description information.Source: §4.4 · p. 8
  • The qualitative comparison describes Exp3rt’s explanation as richer and more generalized than P5’s brief, vague statement or XRec’s abstract explanation.The authors characterize Exp3rt’s rationale as capturing varied item aspects and aligning with user preferences.Source: §4.4 · p. 8

4.5 Comprehensive Analysis of Exp3rt (RQ4)

Exp3rt’s analysis finds benefits from rating bias, textual reasoning, and distillation, alongside gains from comprehensive item profiles and a CF-assisted ranking pipeline. The reported efficiency comparison shows a substantial reduction in computation time when CF first selects candidates.

4.5 Comprehensive Analysis of Exp3rt (RQ4)

  • Using all available reviews for item profiles is more effective than helpfulness-based or neighbor-based selection strategies.The authors retain all reviews as the default profile-construction setting.Source: §4.5 · p. 9
  • Both rating bias and textual reasoning significantly reduce prediction errors, and the distilled, target-dataset-fine-tuned model outperforms GPT-3.5 prompting with the same inputs.Source: §4.5 · p. 9
  • With GPT-4 used in place of GPT-3.5 under the same prompts, the reported result supports domain fine-tuning and distillation’s improvements to LLaMA3-8B’s cost-efficiency and recommendation accuracy.Source: §4.5 · p. 9, Figure 4 · p. 9
  • CF candidate selection reduces computation time by over 98% compared with Exp3rt directly ranking all items.The IMDB comparison uses LightGCN to select 20 candidates before reranking and is run on one RTX 3090 GPU.Source: §4.5 · p. 9

5 CONCLUSION

EXP3RT uses review-derived profiles and detailed reasoning for rating prediction, and is evaluated for rating prediction and top-k reranking. Its reasoning is also presented as an explanation of user and item preferences.

5 CONCLUSION

  • EXP3RT is an LLM recommender for reasoning-enhanced rating prediction, optimized through distillation for preference extraction, profile construction, and textual reasoning.Source: §5 · p. 9
  • On two datasets, EXP3RT is reported effective for rating prediction and as a reranker in a multi-stage top-k recommendation pipeline.Source: §5 · p. 9
  • The authors report that its textual reasoning provides faithful, reasonable explanations that capture subjective user and item preferences.Source: §5 · p. 9

A PROMPTS

The prompts guide EXP3RT through preference extraction, profile construction, and rating prediction with textual reasoning. Together, these steps turn review preferences into profiles used alongside item descriptions.

A PROMPTS

  • The prompts systematically guide the model across its three primary process steps.Source: A PROMPTS · p. 10
  • Step 1 extracts key preferences from raw reviews and organizes them into structured, interpretable sets.The extraction prompt is listed in Table 7.Source: A PROMPTS · p. 10, Table 7 · p. 11
  • Step 2 aggregates extracted preferences into user and item profiles that summarize information for later tasks.The user- and item-profile prompts are listed in Tables 8 and 9.Source: A PROMPTS · p. 10, Table 8 · p. 11, Table 9 · p. 11
  • Step 3 generates reasoning and predicts user ratings by integrating the profiles with supplementary item descriptions.Teacher reasoning prompts and model training and inference prompts are listed in Tables 10 and 11.Source: A PROMPTS · p. 11

B USER SIMILARITY CALCULATION

For neighbor-based item-profile construction, the method uses user-user cosine similarity to find users with similar preferences. Reviews from the three most similar users who interacted with the same item form a preference set.

B USER SIMILARITY CALCULATION

C IMPLEMENTATION DETAILS

The implementation uses publicly available baseline code when available and follows its specified preprocessing and implementation. For P5, the authors fine-tune T5 on each dataset using task-aligned prompts.

C IMPLEMENTATION DETAILS

  • The authors follow baseline code instructions for preprocessing and implementation, and implement MF and WDL from their papers when official code is unavailable.Source: C IMPLEMENTATION DETAILS · p. 12
  • For P5, T5 is fine-tuned separately on each dataset with prompts aligned to the evaluation tasks.Source: C IMPLEMENTATION DETAILS · p. 12
  • For rating prediction, P5 uses all prompts with the star_rating target template and reports their average score; for explanation generation, it uses prompt ID 3-1.The Amazon-Book prompt matches the IMDB prompt except for its rating scale.Source: C IMPLEMENTATION DETAILS · p. 12, Table 11 · p. 13

Related papers

Loading 2408.06276v5…