PaperlayerView exact revision on arXiv

Source-linked AI summary

How Can Recommender Systems Benefit from Large Language Models: A Survey

Jianghao Lin, Xinyi Dai, Yunjia Xi, Weiwen Liu, Bo Chen, Hao Zhang, Yong Liu, Chuhan Wu, Xiangyang Li, Chenxu Zhu, Huifeng Guo, Yong Yu, Ruiming Tang, Weinan Zhang

arXiv:2306.05817v6cs.IRcs.AI

TL;DR

Conventional recommender systems lack open-world knowledge and struggle to capture users’ underlying preferences, motivating the integration of LLM capabilities. This survey maps LLM adaptation across the recommendation pipeline and organizes training and inference strategies, while identifying challenges and future directions. It concludes that LLM use is expanding across pipeline stages, although evidence remains limited on whether larger models are always better and long inputs remain costly for recommendation settings.

  • Problem

    Conventional recommender systems have limited open-world knowledge and often implicitly model users’ preferences and motivations from behavior.

  • Method

    The survey systematically reviews LLM-enhanced recommendation from the whole pipeline using WHERE and HOW taxonomies.

  • Results

    LLMs are being used not only for feature encoding and scoring/ranking but also across additional recommendation pipeline stages.

  • Takeaways & Limitations

    The survey provides a systematic view and feasible suggestions for the evolution of LLM-enhanced recommender systems.

  • Takeaways & Limitations

    Evidence remains insufficient to conclude that larger LLM size definitely improves recommendation because a larger-model benchmark is absent.

Abstract

from arXiv · show

With the rapid development of online services, recommender systems (RS) have become increasingly indispensable for mitigating information overload. Despite remarkable progress, conventional recommendation models (CRM) still have some limitations, e.g., lacking open-world knowledge, and difficulties in comprehending users' underlying preferences and motivations. Meanwhile, large language models (LLM) have shown impressive general intelligence and human-like capabilities, which mainly stem from their extensive open-world knowledge, reasoning ability, as well as their comprehension of human culture and society. Consequently, the emergence of LLM is inspiring the design of recommender systems and pointing out a promising research direction, i.e., whether we can incorporate LLM and benefit from their knowledge and capabilities to compensate for the limitations of CRM. In this paper, we conduct a comprehensive survey on this research direction from the perspective of the whole pipeline in real-world recommender systems. Specifically, we summarize existing works from two orthogonal aspects: where and how to adapt LLM to RS. For the WHERE question, we discuss the roles that LLM could play in different stages of the recommendation pipeline, i.e., feature engineering, feature encoder, scoring/ranking function, user interaction, and pipeline controller. For the HOW question, we investigate the training and inference strategies, resulting in two fine-grained taxonomy criteria, i.e., whether to tune LLM or not, and whether to involve conventional recommendation models for inference. Then, we highlight key challenges in adapting LLM to RS from three aspects, i.e., efficiency, effectiveness, and ethics. Finally, we summarize the survey and discuss the future prospects. We actively maintain a GitHub repository for papers and other related resources: https://github.com/CHIANGEL/Awesome-LLM-for-RecSys/.

1 INTRODUCTION

Conventional recommender systems remain limited by narrow domain knowledge and implicit preference modeling, while LLMs offer knowledge and reasoning capabilities for broader integration. The survey organizes this emerging area around where and how LLMs can be adapted across the recommendation pipeline.

  • Conventional recommender systems are domain-oriented and built on discrete ID features, limiting open-domain knowledge and cross-domain transfer.
  • Behavior-driven optimization can weaken explainability and obscure users’ complicated, context-dependent intentions and motivations.
  • LLMs contribute open-world knowledge, logical and commonsense reasoning, and awareness of human society and culture through natural language.
  • The survey addresses where to adapt LLMs across feature engineering, feature encoding, scoring/ranking, user interaction, and pipeline control.
  • It addresses how to adapt LLMs by classifying whether their parameters are frozen during training and whether conventional recommendation models participate during inference.
  • The survey highlights efficiency, effectiveness, and ethics as key challenges and proposes suggestions for evolving online recommendation platforms.

2 BACKGROUND AND PRELIMINARY

Modern recommender systems form a six-stage information cycle from data collection through pipeline control, while LLMs are scaled pretrained language models with broad modeling capabilities. The pipeline stages can be skipped, linked, or merged depending on the adaptation design.

  • The recommendation pipeline comprises data collection, feature engineering, feature encoding, scoring/ranking, user interaction, and pipeline control.
  • Data collection gathers explicit and implicit feedback, raw item and user features, demographics, and contextual information from online services.
  • Feature engineering transforms collected raw data into structured inputs and may include feature manipulation, sample synthesis, and augmentation.
  • Feature encoders convert processed multimodal features into neural embeddings, while scoring and ranking select and order relevant items.
  • User interaction represents recommendations and receives feedback, including conversational and multimodal interaction scenarios.
  • Adapting an LLM as the scoring/ranking function can skip feature encoding, whereas other designs merge CRM embeddings with engineered features.
  • LLMs are scaled pretrained language models trained on larger model sizes and data volumes, with capabilities including in-context learning, instruction following, and reasoning.

3 WHERE TO ADAPT LARGE LANGUAGE MODELS

The survey answers where LLMs can enter recommender systems by mapping their roles to five pipeline stages. Because LLMs support multiple tasks, one work may involve several stages.

  • LLMs can be adapted for feature engineering, feature encoding, scoring/ranking, user interaction, and recommendation pipeline control.
  • A single research work may use LLMs at multiple recommendation stages because of their multi-task nature.

3.1 LLM for Feature Engineering

In feature engineering, LLMs transform original recommendation features into auxiliary textual information and synthetic data. The survey groups these uses into feature augmentation and instance-level sample generation.

  • LLMs take item descriptions, user profiles, and behaviors as inputs to generate auxiliary textual features for data augmentation.
  • LLM-generated features can enrich training data, alleviate long-tail problems, and support preference modeling and item-content understanding.
  • Feature augmentation mainly operates at the user or item level, while another category generates training instances synthetically.
  • Some methods generate user-side preference knowledge and item-side factual knowledge as plug-in features for conventional recommendation models.
  • ChatGPT can summarize user interests into keywords, condensing profiling data within 128 tokens for smaller language-model encoders.
  • Other approaches fine-tune language models or use prompting, query rewriting, privacy-aware synthesis, dataset alignment, and simulated interactions for recommendation data enhancement.

3.2 LLM as Feature Encoder

LLMs serve as auxiliary feature encoders that enrich user and item representations with semantic information and support cross-domain recommendation through natural language.

  • LLMs encode textual user and item features to enrich representations used by downstream neural recommendation models.This extends conventional one-hot feature encoding with semantic information.
  • Natural language can bridge domains for cross-domain recommendation even when ID feature fields are not shared.Shared or universal textual representations support transfer across different data distributions and domains.
  • User-interest modeling uses sequential review and behavior information because user preferences evolve over time.U-BERT encodes review texts into dense vectors, while LLM4ARec extracts personalized aspects and latent vectors from profiles and reviews.
  • Vector quantization converts LLM-generated semantic item representations into discrete tokens for sequential or generative recommendation.TIGER compresses items into tuples of discrete semantic tokens, while LMIndexer learns semantic representations and tokens end to end.
  • BERT-based methods produce universal item representations for zero-shot or cross-domain sequential recommendation.Examples include ZESRec and UniSRec, with UniSRec adding a lightweight MoE-enhanced network after fixed BERT encoding.
  • Layerwise adapter tuning and unified textual embeddings address representation transfer across domains and cold-start users or items.Other methods use fixed LLMs or shared LLM representations to capture cross-domain commonalities.

3.3 LLM as Scoring/Ranking Function

LLMs can act as scoring or ranking functions through pointwise scoring, dual-tower matching, or token-based prediction, and can generate items in open- or closed-set settings.

  • The scoring/ranking stage uses LLMs for item scoring, item generation, or hybrid tasks, with textual or encoded user-item inputs.The goal is to estimate utility or produce a ranked item list.
  • Item scoring task: Pointwise scoring requires continuous preference values, whereas language-model decoders naturally produce discrete tokens.Typical targets include CTR values in [0, 1] and ratings in [0, 5].
  • Item scoring task: Single-tower methods replace the language-modeling head with a projection layer that maps the final representation to a classification or regression score.The projection layer can be an MLP operating on the final LLM representation h.
  • Item scoring task: Two-tower methods separately encode users and items with LLM-backed towers and calculate preference using a distance metric.Cosine similarity and L2 distance are examples of the metric used between the two representations.
  • Item scoring task: Token-based scoring converts preference estimation into binary question answering and derives scores from the probabilities of label tokens such as Yes and No.Other approaches score prompts using perplexity, log-likelihood, or joint probability.
  • Item generation task: Generation methods produce ranked item lists directly, but closed-set approaches provide candidates while open-set approaches require item grounding and matching.Pre-filtering reduces the universal item set to a limited candidate set, often up to 20 items because of context-window limits.
  • Item generation task: Shuffling candidate-item order in prompts can affect the resulting ranking output.
  • Item generation task: Open-set generation may hallucinate items outside the universal pool, increasing inference overhead through post-generation matching.The survey contrasts open-set and closed-set generation as having complementary strengths and weaknesses.

3.4 LLM for User Interaction

LLMs broaden recommender-system interaction from passive one-turn behavior monitoring to adaptive dialogue that elicits preferences, explains suggestions, and processes feedback.

  • Conventional one-turn recommendation has limited ability to acquire interests or detect current needs in complex scenarios.
  • LLMs support more active interaction by gathering nuanced natural-language feedback about user preferences in real time.
  • LLM-based user interaction commonly uses multi-turn dialogue and can provide tailored recommendations, explanations, and feedback processing.
  • Task-oriented user interaction: Task-oriented interaction assumes clear user intent and uses LLM components to analyze intentions, encode dialogue information, recommend items, and generate responses.TG-ReDial separately encodes profiles, dialogue history, and conversation topics before recommendation and response generation.
  • Task-oriented user interaction: Recent billion-level LLMs are incorporated for conversational recommendation, and one study reports outperforming finetuned traditional conversational recommenders in zero-shot settings.
  • Open-ended user interaction: Open-ended interaction assumes vague intent and uses dialogue to gradually acquire interests or guide users toward recommendations.Here LLMs can drive preference elicitation and recommendation rather than only encode dialogue or generate responses.
  • Open-ended user interaction: End-to-end open-ended systems can combine context understanding, preference elicitation, item recommendation, and response generation.A special token may trigger recommended-item generation during response generation.

3.5 LLM for Pipeline Controller

LLMs can move beyond component-level recommendation roles to control multi-stage pipelines through dialogue management, API calls, tools, ranking decisions, and user simulation.

  • Emergent LLM abilities enable active participation in pipeline control, potentially supporting more interactive and explainable recommendation.
  • ChatGPT-based systems can infer preferences, decide whether to call a backend recommendation API, and filter or rerank returned candidates.These operations extend LLM control across a multi-stage recommendation pipeline.
  • Integrated conversational recommenders can let LLMs manage dialogue, understand preferences, arrange ranking, and generate synthetic conversations.
  • Tool-enabled interactive agents can provide intention detection, information queries, item retrieval, and item ranking.
  • RecMind applies task deconstruction, self-inspiring prompting, and multi-step reasoning with tool usage for finer-grained recommendation control.

3.6 Discussion

LLM adaptation in recommender systems has expanded from small language-model encoders and scoring functions to broader pipeline roles. This progression leverages open-world knowledge, synthetic data, natural-language interaction, reasoning, and potential pipeline control.

  • Evolution of LLM roles: Earlier works used relatively small language models as textual feature encoders or scoring/ranking functions in straightforward predictive recommendation tasks.These models were commonly fine-tuned to fit recommender-system data distributions.
  • Evolution of LLM roles: As model size increased, LLMs added emergent reasoning, instruction-following, open-world knowledge, and text-generation capabilities to recommendation.
  • Expanded pipeline roles: LLMs now serve beyond feature encoding and ranking, including feature engineering, user interaction, and pipeline control.They can generate auxiliary features and synthetic data, provide natural-language dialogue, and potentially use reasoning and tools for pipeline control.
  • Expanded pipeline roles: Feature engineering can inject LLM open-world knowledge into closed-domain recommendation models through auxiliary features and synthetic samples.
  • Future direction: The survey anticipates deeper coupling between LLMs and multiple recommendation-pipeline stages, including models customized for recommender-system requirements.

4 HOW TO ADAPT LARGE LANGUAGE MODELS

The survey classifies LLM adaptation along two orthogonal axes: whether LLM parameters are tuned and whether conventional recommendation models participate during inference. Across the resulting quadrants, the reported development path increasingly combines LLM capabilities with in-domain collaborative knowledge.

  • Taxonomy: The HOW taxonomy classifies methods by tuning versus not tuning LLMs and inference with versus without conventional recommendation models.Tuning includes full and parameter-efficient methods such as LoRA and prompt tuning; CRM may also act only as an independent pre-filter.
  • Quadrant 1: Quadrant 1 works align better with recommendation data and achieve satisfying performance even when the involved language model is relatively small.
  • Development trend: The adaptation trend moves from small-scale models toward billion-scale LLMs to exploit their open-world knowledge and capabilities.
  • Quadrant 3: Frozen LLMs without CRM in Quadrant 3 remain inferior to lightweight CRMs tuned on recommendation data, even with behavior retrieval and tool usage.

4.4 Tune LLM & Infer w/o CRM (Quadrant 4)

Quadrant 4 tunes LLMs for recommendation without using CRM during inference, but scaling this approach creates efficiency challenges. The survey highlights data downsampling and the broader need to inject collaborative knowledge for strong recommendation performance.

  • Quadrant 4: Quadrant 4 fine-tunes LLMs on recommendation data to serve as scoring or ranking functions without CRM inference.
  • Modeling strategies: Quadrant 4 methods formulate recommendation as classification, sequence-to-sequence generation, or next-item prediction using fine-tuned language models.
  • Efficiency: Full fine-tuning of small language models becomes expensive and difficult to scale as model size grows to tens or hundreds of billions of parameters.
  • Efficiency: Parameter-efficient methods such as LoRA reduce adaptation cost and are commonly used for billion-level LLMs in recommendation tasks.
  • Data efficiency: Less than 10% of training instances enabled ReLLa to outperform conventional recommendation baselines trained on the full dataset.
  • Discussion: LLM fine-tuning shows promising recommendation performance with suitable task formulations, but scalability and efficiency remain major challenges.
  • Industrial implication: LLMs can be useful rerankers for hard samples, allowing lightweight models to handle other samples and reduce computational costs.

5 CHALLENGES FROM REAL-WORLD APPLICATIONS

Real-world LLM-enhanced recommendation faces efficiency, effectiveness, and ethics challenges. The survey discusses training and inference costs, long-text and ID modeling, fairness, privacy, hallucination, and explainability.

  • Efficiency: Recommendation systems require massive training data and frequent model updates, making training efficiency a central real-world challenge.
  • Training efficiency: PEFT reduces memory usage but can remain time-consuming at large scale, motivating asynchronous LLM updates alongside frequently updated CRMs.
  • Inference efficiency: LLM inference can violate recommendation latency requirements because model execution and prompt construction add substantial time costs.
  • Inference efficiency: Caching and pre-computation can reduce inference burden, but rapidly changing user behavior makes user-side caching frequency a performance–cost trade-off.
  • In-domain long text modeling: Long user histories, candidate sets, and feature prompts create memory costs and can cause information loss or inferior outputs.Classical transformer space complexity is O(L^2), where L is the token count.
  • ID indexing and modeling: ID features can improve recommendation performance, but LLM-based ID indexing and modeling remain an active design problem.
  • Ethics: LLM-enhanced recommendation raises user-side and item-side fairness concerns, including implicit demographic inference and popularity bias.
  • Trustworthiness: Trustworthy LLM-enhanced recommenders require attention to hallucination, privacy, explainability, and improved explanation prompting and interpretability.

6 CONCLUSION AND FUTURE PROSPECTS

The survey organizes LLM-enhanced recommendation research around where LLMs enter the recommendation pipeline and how they are trained and used for inference. It identifies evaluation needs and outlines future directions including personalized content generation and recommendation-specific foundation models.

  • The survey systematically examines LLM adaptation to recommender systems from the perspective of the whole industrial recommendation pipeline.
  • WHERE: LLMs can support feature engineering, feature encoding, scoring or ranking, user interaction, and pipeline control.
  • HOW: The HOW taxonomy classifies methods by whether they tune LLM parameters during training and whether they involve conventional recommendation models during inference.
  • Future prospects: Future systems may generate personalized presentations of the same item, such as revised text or thumbnails, for different users.
  • Challenges and prospects: A unified public benchmark is needed because fine-grained cross-comparison remains unavailable and reproducing LLM-enhanced recommendation experiments is expensive and difficult.
  • Future prospects: The survey envisions recommendation-specific foundation models controlling the entire pipeline and enabling greater automation in recommender systems.

A LOOK-UP TABLE FOR MENTIONED WORKS

The survey provides a lookup table of works adapting language models to recommender systems, organized by recommendation-pipeline stage and detailed by model, tuning strategy, task, and scenario.

  • Table organization: Table 1 catalogs mentioned works according to the recommendation-pipeline stage where their adapted language model is involved.Different stages are separated by colors, and each work is represented as a table row.
  • Table organization: Each table row records the model name, language-model backbone, tuning strategy, recommendation task, and scenario.The table also includes model-size information for each work.
  • Resources: The authors maintain a GitHub repository for more up-to-date information as this research area continues to evolve.The lookup table is intended to facilitate research-community reference to related works and resources.
  • Adaptation strategies: The table includes both frozen and tuned language-model approaches, with tuning strategies such as full finetuning, prompt tuning, layerwise adapter tuning, and parameter-efficient methods.The caption defines FFT as full finetuning, PT as prompt tuning, LAT as layerwise adapter tuning, and T-FEW as few-shot parameter efficient tuning.
  • Application scenarios: Covered applications include e-commerce, movies, news, books, music, jobs, web search, social networks, fashion, video, and other domains.The table entries pair recommendation tasks with one or more application scenarios.
  • Pipeline coverage: The listed works span feature engineering, feature encoding, and scoring or ranking functions across tasks including retrieval, sequential recommendation, top-N recommendation, and prediction.Examples include instance-level sample generation, representation enhancement, item scoring, and item generation.
Loading 2306.05817v6…