PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Large Language Models for Recommendation

Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, Hui Xiong, Enhong Chen

arXiv:2305.19860v5cs.IRcs.AI

TL;DR

Recommendation systems need effective ways to use LLM representations and external knowledge to improve personalized recommendations. This survey organizes LLM-based recommendation research into discriminative and generative paradigms, reviews their methods and findings, and identifies challenges, while noting unresolved evaluation and bias concerns.

  • Problem

    Recommendation systems need ways to harness LLMs' textual representations and external knowledge to improve relevant and personalized recommendations.

  • Method

    The survey classifies LLM-based recommendation systems into discriminative and generative paradigms, reviews their adaptation methods and techniques, and distinguishes fine-tuning, prompting, prompt tuning, and instruction tuning.

  • Results

    The survey provides a systematic review of LLM-based recommendation systems, including the first systematic review dedicated to generative LLMs for recommendation, and summarizes common findings and challenges.

  • Takeaways & Limitations

    LLMs show potential for zero- and few-shot recommendation and generated explanations, but representative demonstrations, broader domain evidence, and generative-recommendation evaluation remain open needs.

  • Takeaways & Limitations

    Evaluating generative recommendation remains an open question, especially when LLMs generate items absent from historical data.

Abstract

from arXiv · show

Large Language Models (LLMs) have emerged as powerful tools in the field of Natural Language Processing (NLP) and have recently gained significant attention in the domain of Recommendation Systems (RS). These models, trained on massive amounts of data using self-supervised learning, have demonstrated remarkable success in learning universal representations and have the potential to enhance various aspects of recommendation systems by some effective transfer techniques such as fine-tuning and prompt tuning, and so on. The crucial aspect of harnessing the power of language models in enhancing recommendation quality is the utilization of their high-quality representations of textual features and their extensive coverage of external knowledge to establish correlations between items and users. To provide a comprehensive understanding of the existing LLM-based recommendation systems, this survey presents a taxonomy that categorizes these models into two major paradigms, respectively Discriminative LLM for Recommendation (DLLM4Rec) and Generative LLM for Recommendation (GLLM4Rec), with the latter being systematically sorted out for the first time. Furthermore, we systematically review and analyze existing LLM-based recommendation systems within each paradigm, providing insights into their methodologies, techniques, and performance. Additionally, we identify key challenges and several valuable findings to provide researchers and practitioners with inspiration. We have also created a GitHub repository to index relevant papers on LLMs for recommendation, https://github.com/WLiK/LLM4Rec.

1 Introduction

LLMs are being integrated into recommendation systems because their textual representations and external knowledge can support personalization. This survey organizes the field, reviews its methods, and identifies challenges.

  • Motivation: LLMs attract recommendation research because they extract textual representations and leverage external knowledge.These capabilities can help establish correlations between users and items.
  • Generative models: Generative LLMs may improve recommendation explanations and support more personalized, context-aware interactions.Their language generation can explain recommendation factors, while customizable prompts support chat-based recommendations.
  • Research gap: Prior surveys emphasized pretraining transfer strategies rather than the broader capabilities of LLM-based recommendation systems.The authors identify this as a gap motivating a more comprehensive review.
  • Contributions: The survey systematically reviews LLM-based recommendation systems and analyzes their methods and applications.It focuses on expanding language-model capacity and organizing existing advancements.
  • Contributions: The survey categorizes recommendation research by modeling paradigms and critically examines methods, limitations, and challenges.The supplied material describes three modeling schools and a systematic analysis of their advantages and disadvantages.

2 Modeling Paradigms and Taxonomy

The survey describes three ways to connect LLMs with recommendation systems and separately organizes studies into discriminative and generative categories. Training manner further refines this taxonomy.

  • Modeling paradigms: The three modeling paradigms are LLM embeddings plus recommendation systems, LLM tokens plus recommendation systems, and LLMs acting as recommendation systems.They respectively use LLM-derived embeddings, generated semantic tokens, or the language model itself in recommendation.
  • LLM Embeddings + RS: LLM embeddings provide knowledge-aware features that traditional recommendation models can use for recommendation tasks.The language model functions as a feature extractor for user and item information.
  • LLM Tokens + RS: LLM-generated tokens capture potential user preferences through semantic mining and can enter the recommendation decision process.This paradigm retains a recommendation model while incorporating token-level language-model outputs.
  • Taxonomy: The survey’s main taxonomy divides existing work into discriminative and generative LLMs for recommendation.The distinction reflects the choice of language-model family and its role in the recommendation system.
  • Taxonomy: Discriminative LLMs generally support embedding-based paradigms, whereas generative LLMs support token-based or end-to-end recommendation paradigms.The survey further subdivides these categories according to training manner.

3 Discriminative LLMs for Recommendation

Discriminative LLM recommendation methods mainly use BERT-family models as embedding backbones, adapting them through fine-tuning or prompt tuning. The survey reports gains in representation and recommendation capabilities while noting scope limitations.

  • Overview: Discriminative recommendation models mainly use BERT-family LLMs as embedding backbones for downstream tasks.Their natural-language understanding capability supports alignment between pretrained representations and recommendation data.
  • Fine-tuning: Fine-tuning adapts pretrained language-model parameters to recommendation-specific data, including interactions, item descriptions, profiles, and context.The model is initialized with pretrained parameters and updated for the target recommendation task.
  • Scope: Existing fine-tuning-based models often exploit training strategies without expanding large language models into the recommendation field.The survey therefore excludes many such models from its main discussion.
  • Fine-tuning: BERT fine-tuning combines external knowledge with personalized user preference to promote recommendation accuracy and provide limited cold-start handling.The stated cold-start benefit concerns new items with limited historical data.
  • Prompt tuning: Prompt tuning aligns recommendation objectives with pretrained losses using hard or soft prompts and label-word verbalizers.For masked-language models, the verbalizer maps predicted words at [MASK] to task labels.
  • Prompt tuning: Multi-prompt ensembling noticeably improves recommendation performance over single discrete or continuous prompts.Prompt4NR reframes candidate-news click prediction as a cloze-style mask-prediction task.

4 Generative LLMs for Recommendation

Generative LLM recommendation methods translate recommendation tasks into language-generation problems and organize them by whether model parameters are tuned. They span prompting, in-context learning, feature construction, system control, and fine-tuning, while facing challenges in output grounding and demonstration selection.

  • Prompting and tuning: Generative recommendation approaches translate recommendation tasks into natural-language tasks and adapt LLMs through in-context, prompt, or instruction tuning.These approaches directly generate recommendation results rather than only aligning learned representations with recommendation domains.
  • Taxonomy: Generative LLM methods are divided into non-tuning and tuning paradigms according to whether the LLM module's parameters are updated.The survey presents representative approaches and datasets for both paradigms.
  • Prompting: Prompting studies design task descriptions, behavior injection, format indicators, and task-specific prompts to improve recommendation understanding and output control.Prompt templates may use item attributes, language-based preferences, or both.
  • System integration: LLMs can augment item and user features, control interactive recommendation systems, or integrate recall, ranking, and re-ranking in an end-to-end framework.Controller-based systems use dialogue and tools such as databases, retrievers, and memory.
  • In-context Learning: In-context learning adds demonstration input-label pairs to prompts, and reported experiments show improved recommendation abilities on most evaluated tasks.Demonstrations can also control output format and content, supporting more stable evaluation.
  • Fine-tuning: Fine-tuned generative models include GPTRec, which uses SVD Tokenisation and Next-K generation, while other methods format interactions as prompts for classification or regression.In one comparison, fine-tuned FLAN-T5-XXL (11B) achieved the best result among evaluated model sizes and settings.
  • Challenges: Generative recommendation remains constrained by out-of-corpus outputs, motivating item indexing and grounding methods that align generated identifiers with the item database.The survey also identifies demonstration selection and demonstration-count effects as open in-context-learning questions.

5 Findings

The survey identifies promising capabilities of LLM-based recommendation alongside unresolved technical challenges. These include bias, representation and context limitations, output control, evaluation, datasets, knowledge retention, and computational cost.

  • Findings: The field remains at an early exploration stage, with knowledge forgetting and the computational costs of excessively large models posing additional challenges.The survey presents these limitations as opportunities for further research and advancement.
  • Model Bias: Candidate order and item popularity can bias generative recommendation rankings, while pretrained data can also contribute to fairness concerns.The survey notes that models often prioritize early candidates and popular items, and may produce recommendations associated with gender or race biases.
  • Recommendation Prompt Designing: LLM-based systems often represent items by names and users by item-name lists, which inadequately captures heterogeneous behavior and ID-like features.The survey also identifies limited context length as a constraint on behavioral sequences and candidate lists.
  • Promising Ability: LLMs show zero- and few-shot recommendation abilities across multiple domain datasets, suggesting potential for recommendation with limited data.Few-shot learning does not change model parameters, and the survey connects this capability to mitigating cold-start problems.
  • Evaluation Issues: Generative recommendation still faces output-format control and evaluation challenges, especially when models generate items absent from historical data.Standard metrics apply to rating prediction and ranking, but evaluating generative recommendation remains an open question.

6 Conclusion

The survey positions LLM-based recommendation as a rapidly advancing field with substantial opportunities for more sophisticated, personalized, multimodal, and ethically grounded applications. It presents the review as a foundation for future innovation and refinement.

  • The survey identifies continued opportunities to develop more sophisticated and diverse LLM applications in recommendation systems.It anticipates that advancing computational capabilities and artificial intelligence will support these applications.
  • Future systems may produce real-time, personalized recommendations that incorporate multimodal inputs.
  • Future LLM-based recommendation systems might integrate fairness, accountability, and transparency more intrinsically as ethical considerations gain prominence.
  • The survey aims to serve as a foundational stepping stone for subsequent discoveries in the evolving field of LLM-based recommendation.The conclusion characterizes the field as offering opportunities for continued innovation and refinement.
Loading 2305.19860v5…