Source-linked AI summary
PALR: Personalization Aware LLMs for Recommendation
Fan Yang, Zheng Chen, Ziyan Jiang, Eunah Cho, Xiaojiang Huang, Yanbin Lu
TL;DR
Recommendation research has explored LLMs less extensively than general NLP, while direct use of general-purpose LLM knowledge faces grounding, coverage, and efficiency challenges. PALR combines user behavior with retrieval and fine-tuned LLM ranking, and reports competitive performance against state-of-the-art methods across sequential recommendation tasks.
Problem
Recommendation research has less extensively explored LLMs, while direct use of general-purpose LLM knowledge faces item knowledge gaps, hallucinations, and input-length limits.
Method
PALR generates user profiles, retrieves candidates, and fine-tunes a LLaMa 7B model to rank recommendation candidates from natural-language prompts.
Results
PALR outperforms multiple baselines by a large margin on two benchmark datasets, and reranking improves on SASRec recommendations.
Takeaways & Limitations
PALR provides a flexible personalized recommendation framework that combines user behaviors with LLM-based candidate selection across retrieval methods.
Takeaways & Limitations
The paper identifies computational intensity and latency as limitations requiring future optimization, and notes that fine-tuning candidate construction is retrieval-layer agnostic rather than tied to a specific retriever.
Abstract
from arXiv · showhide
Large language models (LLMs) have recently received significant attention for their exceptional capabilities. Despite extensive efforts in developing general-purpose LLMs that can be utilized in various natural language processing (NLP) tasks, there has been less research exploring their potential in recommender systems. In this paper, we propose a novel framework, named PALR, which aiming to combine user history behaviors (such as clicks, purchases, ratings, etc.) with LLMs to generate user preferred items. Specifically, we first use user/item interactions as guidance for candidate retrieval. Then we adopt a LLM-based ranking model to generate recommended items. Unlike existing approaches that typically adopt general-purpose LLMs for zero/few-shot recommendation testing or training on small-sized language models (with less than 1 billion parameters), which cannot fully elicit LLMs' reasoning abilities and leverage rich item side parametric knowledge, we fine-tune a 7 billion parameters LLM for the ranking purpose. This model takes retrieval candidates in natural language format as input, with instruction which explicitly asking to select results from input candidates during inference. Our experimental results demonstrate that our solution outperforms state-of-the-art models on various sequential recommendation tasks.
1 INTRODUCTION
PALR addresses challenges in applying LLMs to recommendation by combining user behaviors with retrieval and fine-tuned LLM ranking. The framework reports competitive performance against state-of-the-art methods on two public datasets.
- LLMs support inductive item representation, integration of metadata and context, and knowledge transfer across domains for recommendation.
- Directly using general-purpose LLM knowledge is difficult because of item knowledge gaps, hallucinated outputs, and input-length or efficiency limits.
- PALR combines user behaviors with LLMs through user-profile generation, candidate retrieval, and item ranking.
- PALR fine-tunes a recommendation-oriented LLaMa 7B model and evaluates it on two public datasets.
- PALR demonstrates strong potential for LLM-based recommendation in comparison to state-of-the-art methods on Amazon Beauty.
2 METHODOLOGY
PALR uses a multi-step recommendation pipeline that summarizes user preferences, retrieves candidates, and prompts an LLM to select recommendations. Instruction-based fine-tuning teaches the model to recommend future items and retrieve targets from candidate lists.
- 2.1 PALR Framework: The user profile summarizes preferences from interactions, item attributes, or user information, helping represent users with mixed affinities.
- 2.1 PALR Framework: PALR constructs recommendations from interaction history, a natural-language user profile, and retrieved candidates.
- 2.1 PALR Framework: The recommendation prompt asks the LLM to select items from a candidate pool that best align with the user profile.
- 2.1 PALR Framework: Dedicated prompt design and instruction-based fine-tuning are used to elicit the LLM’s reasoning ability and improve recommendation behavior.
- 2.2 Fine-Tuning: The Recommend task maps up to 20 interacted items to future items the user may interact with.
- 2.2 Fine-Tuning: The Recommend_Retrieval task trains the model to retrieve target future items from candidates containing targets and similar negative items.
- 2.2 Fine-Tuning: Fine-tuning is retrieval-layer agnostic, so candidate-list construction is not tied to a particular retrieval algorithm.
- 2.2 Fine-Tuning: The experiments fine-tune on 20% of users to demonstrate inductive learning capabilities.
3 EXPERIMENTS
The experiments evaluate PALR on two public sequential-recommendation datasets using standard preprocessing and leave-one-out metrics. PALRv2 outperforms multiple baselines, while retrieval and fine-tuning contribute substantially to performance.
- Datasets: The evaluation uses Amazon Beauty and MovieLens-1M datasets, with preprocessing that binarizes interactions, removes duplicates, orders histories chronologically, and applies 5-core filtering.MovieLens-1M contains one million movie ratings.
- Evaluation: Performance is evaluated with leave-one-out testing on the full item set using Hit Ratio and NDCG.The last interaction is held out for testing, the preceding interaction for validation, and earlier interactions for training.
- Overall performance: PALRv2 outperforms multiple baselines by a large margin on both benchmark datasets.Table 2 reports the best results of the compared models on the two datasets.
- Overall performance: Candidate retrieval plays a crucial role in improving performance, as shown by the comparison between PALRv1 and PALRv2.The framework is not dependent on any particular retrieval algorithm.
- Retrieval robustness: PALR improves the top-10 recommendations from SASRec, with similar trends observed when using BERT4Rec and LightGCN as retrieval algorithms.The reported retrieval layer uses SASRec's top 50 recommendations.
- Fine-tuning: Fine-tuning with an additional instruction enables PALRv2 to select retrieval targets more effectively than PALRv1.Before fine-tuning, PALRv1 tended to recommend popular movies or select candidates randomly.
4 CONCLUSION
The paper presents PALR as a multi-step framework for personalized sequential recommendation that uses LLM knowledge and reasoning abilities. It reports competitive results while identifying computational latency as an area for future optimization.
- Conclusion: PALR uses a multi-step paradigm to leverage LLM parametric knowledge and reasoning abilities for sequential recommendation.The framework is presented as a generative approach to personalized recommendations.
- Conclusion: LLMs offer pathways for incorporating external knowledge and supporting complex recommendation scenarios such as explainable and conversational recommendations.These benefits are discussed as additional possibilities beyond the reported experiments.
- Future work: Reducing latency without sacrificing accuracy or personalization remains a future challenge because LLMs can be computationally intensive.The paper identifies performance optimization and latency reduction as future work.