PaperlayerView exact revision on arXiv

Source-linked AI summary

Leveraging Large Language Models in Conversational Recommender Systems

Luke Friedman, Sameer Ahuja, David Allen, Zhenning Tan, Hakim Sidahmed, Changbo Long, Jun Xie, Gabriel Schubiner, Ajay Patel, Harsh Lara, Brian Chu, Zexi Chen, Manoj Tiwari

arXiv:2305.07961v2cs.IRcs.CLcs.LG

TL;DR

Conversational recommenders offer users multi-turn control, but LLM-based systems face challenges in managing dialogue, retrieving from large item corpora, and operating without conversational training data. The paper proposes an LLM-based CRS roadmap and introduces RecLLM for YouTube videos, illustrating its fluency and functions through example conversations.

  • Problem

    Large-scale conversational recommenders face technical challenges in controlling complex dialogue and retrieving from evolving item corpora, while new systems may lack conversational training data.

    Source: §4 · p. 7

  • Method

    The paper proposes an LLM-based CRS architecture spanning dialogue, retrieval, ranking, explanations, user profiles, and controllable user simulation.

    Source: §1 · p. 2

  • Results

    RecLLM, a LaMDA-powered CRS for YouTube videos, is presented with example conversations illustrating fluency and diverse functionality.

    Source: §1 · p. 2, §8 · p. 11

  • Takeaways & Limitations

    The paper outlines an end-to-end CRS approach that uses synthetic conversations to address the absence of logs from an existing product.

    Source: §4 · p. 7, §8 · p. 11

    Generating synthetic sessions for recommendation-module training requires complex conditioning on a target item and turn.

    Source: §4.1 · p. 9

Abstract

from arXiv · show

A Conversational Recommender System (CRS) offers increased transparency and control to users by enabling them to engage with the system through a real-time multi-turn dialogue. Recently, Large Language Models (LLMs) have exhibited an unprecedented ability to converse naturally and incorporate world knowledge and common-sense reasoning into language understanding, unlocking the potential of this paradigm. However, effectively leveraging LLMs within a CRS introduces new technical challenges, including properly understanding and controlling a complex conversation and retrieving from external sources of information. These issues are exacerbated by a large, evolving item corpus and a lack of conversational data for training. In this paper, we provide a roadmap for building an end-to-end large-scale CRS using LLMs. In particular, we propose new implementations for user preference understanding, flexible dialogue management and explainable recommendations as part of an integrated architecture powered by LLMs. For improved personalization, we describe how an LLM can consume interpretable natural language user profiles and use them to modulate session-level context. To overcome conversational data limitations in the absence of an existing production CRS, we propose techniques for building a controllable LLM-based user simulator to generate synthetic conversations. As a proof of concept we introduce RecLLM, a large-scale CRS for YouTube videos built on LaMDA, and demonstrate its fluency and diverse functionality through some illustrative example conversations.

1 INTRODUCTION

Conversational recommender systems give users a multi-turn channel to express and refine preferences, while LLMs offer natural dialogue and world knowledge. The paper outlines ways to use LLMs across a large-scale CRS and introduces RecLLM as a proof of concept.

1 INTRODUCTION

  • LLMs can draw on world knowledge and common-sense reasoning to match conversational context to items and generate explanations, while also supporting profiles and user simulation.These are described as possible uses of LLMs within a CRS.Source: §1 · p. 1
  • CRSs let users refine recommendations through multi-turn dialogue, but large evolving item corpora, hallucinations, and difficult control complicate effective LLM use.Unlike a point-and-click interface, dialogue lets the system query users and receive feedback; large-scale systems also need to connect LLMs to external recommendation engines or databases.Source: §1 · pp. 1–2
  • The proposed roadmap combines unified LLM dialogue management, large-corpus retrieval, joint ranking and explanations, interpretable user profiles, and controllable user simulation.The system uses persistent profiles alongside session context and proposes synthetic conversations for tuning system modules.Source: §1 · p. 2
  • RecLLM is a proof-of-concept LLM-based CRS for YouTube videos, powered by LaMDA [86], with example conversations illustrating fluency and diverse functionality.The paper presents it as a first step toward realizing the broader vision.Source: §1 · p. 2

2 PROBLEM SCOPE

RecLLM separates natural-language conversation from a recommendation slate and uses public YouTube videos as its item corpus. The section describes its interaction scope and highlights conversational capabilities and broader agent problems it leaves unaddressed.

2 PROBLEM SCOPE

  • RecLLM pairs an ongoing natural-language conversation with a recommendation slate, which may be updated in response to the dialogue.The authors separate the two components to better reflect a possible production setting.Source: §2 · pp. 2–3
  • RecLLM currently excludes traditional interface interactions such as viewing or rating recommendations, aiming instead to augment them with natural-language expression.The authors do not intend natural language to replace those interactions.Source: §2 · p. 3
  • The system recommends from all public YouTube videos because the corpus is large-scale and cannot feasibly be recommended directly from LLM memory.The paper notes that RecLLM has no interaction logs to jumpstart training, while presenting its approach as intended to extend to other domains.Source: §2 · p. 3
  • Initial testers expected context retention, topic shifts, and references to slate items; Figure 3 illustrates some supported capabilities in a mock conversation.Figure 2 is a screenshot of an LLM-based user simulator talking with RecLLM.Source: Figure 2 · p. 2, §2 · p. 3, Figure 3 · p. 3
  • The paper does not directly address broader conversational-agent problems such as dialogue safety, debiasing, or consistent personality.Its focus is on problems specific to conversational recommenders.Source: §2 · p. 3

3 SYSTEM OVERVIEW

RecLLM uses LLMs across dialogue management, retrieval, ranking and explanations, and natural-language user profiles. Its design combines conversational context with item-corpus retrieval and profile information.

3.1 Dialogue Management

  • A unified LLM turns session context into dialogue outputs and system actions, replacing fixed dialogue states with a flexible, example-driven approach.The system can function with limited manually generated examples, but teaching a sophisticated recommender-specific policy remains difficult.Source: Figure 4 · p. 3, §3.1 · p. 4

3.2 Recommendations and Refinement

  • RecLLM uses two stages: retrieval selects candidate items from the corpus, then a ranker orders them and jointly generates explanations.Retrieval reduces potentially hundreds of millions of items to a small candidate set, such as 100, for downstream ranking.Source: §3.2 · p. 5, §3.2.1 · p. 5

3.2.1 Retrieval.

  • A conversational dual encoder can use an LLM to encode context and retrieve item embeddings, but aligning context and item representations can require substantial training data.Using internal LLM embeddings also sacrifices some transfer from language pretraining.Source: §3.2.1 · p. 5
  • Other retrieval options include generating item IDs or titles, extracting concepts for approximate search, or sending a query to an existing search API.Direct item generation requires corpus information in model parameters; concept lists can lose language nuance, while search-API quality depends on how well a query represents the conversation.Source: §3.2.1 · pp. 5–6

3.2.2 Ranking / Explanations.

  • The LLM ranker scores each candidate from item and context summaries, using sequential reasoning whose steps can also serve as explanations.Item summaries can draw on video titles, entities, descriptions, transcripts, and comments; explanations can support debugging or be shown to users.Source: Figure 6 · p. 6, §3.2.2 · p. 6

3.3 User Profile

  • RecLLM represents users with editable natural-language profiles built from repeated interactions, using salient facts rather than learned user embeddings.Profiles can preserve preferences across sessions, and representing facts in language can make them more transparent and attributable to sources.Source: §3.3 · pp. 6–7
  • The profile module extracts enduring user facts and retrieves one fact per turn by comparing the last utterance’s embedding with profile-fact embeddings.Retrieval uses cosine distance, with profile triggering implemented by thresholding the minimum distance.Source: §3.3 · p. 7
  • RecLLM injects retrieved profile facts into the dialogue manager’s input, leaving the LLM to decide how they should modulate current-session preferences.The paper illustrates that a current request may override an apparently conflicting stored preference.Source: §3.3 · p. 7

4 SIMULATION AND LARGE-SCALE TUNING

Because conversational recommenders lack the training and evaluation data available to established products, the paper proposes controlled LLM user simulation and synthetic-data methods for tuning CRS modules. The methods address realism and controllability as well as module-specific training constraints.

4 SIMULATION AND LARGE-SCALE TUNING

  • Conversational recommenders lack existing-product interaction data, motivating simulated conversations and larger-scale tuning of system components.The paper notes that existing systems can bootstrap from product logs, whereas conversational systems are nascent and generally lack such products.Source: §4 · p. 7

4.1 User Simulation

  • A user simulator maps a partial session to a distribution over possible next user utterances, allowing it to interact with a fixed CRS to generate sessions.The proposed realism measures include human or model discrimination between simulated and real sessions, or comparing distributions over session properties; diversity is a necessary condition.Source: §4.1 · p. 8
  • Controlled simulation conditions an LLM on latent variables, using a session-level profile or turn-level intents to guide its utterances.Variables can be translated into input text; the paper suggests sampling them from real-session distributions or grounding them in related-product trajectories to increase realism.Source: Figure 8 · p. 7, §4.1 · p. 8, Figure 9 · p. 8
  • A sampled session-level label can serve as a training label when the simulator is successfully conditioned to produce the corresponding sentiment.The paper also proposes conditioning on an item and turn index to generate a session where that item would be a good recommendation; this is more complex, and simple zero-shot priming will not work.Source: §4.1 · pp. 8–9

4.2 Tuning System Modules

  • For dual-encoder retrieval, supervised gradient descent can tune the model on partial sessions, positive items, and sampled negatives.The loss remains differentiable whether tuning is limited to adapter layers or also updates LLM parameters.Source: §4.2 · p. 9
  • For Search API Lookup, the LLM can be tuned as a contextual-bandit policy because the black-box search algorithm makes the loss non-differentiable.The LLM produces a query for the search environment; tuning may target task-specific protected parameters or parameters shared with other modules.Source: §4.2 · p. 9
  • Ranking can be trained as regression on slate relevancy scores, while explanation tuning requires label-based bootstrapping rather than direct supervision.The proposed joint ranking/explanation module generates scores and explanations; Figure 10 depicts the indirect tuning route for explanations.Source: §4.2 · p. 9
  • Dialogue-manager tuning extends a moderate set of example sequences with simulated sessions, crowdworker ratings, reward models, and reinforcement learning.Workers rate turn-level fluency, interestingness, and groundedness, and session-level effectiveness in helping users explore the corpus.Source: §4.2 · p. 9

5 RELATED WORK

The related work spans LLM capabilities and tuning, CRS dialogue and recommendation architectures, user simulation, and datasets. It situates RecLLM's large-corpus retrieval and joint recommendation-explanation focus among gaps in prior CRS work.

Large Language Models

  • LLMs offer zero- or few-shot adaptation, parameter-efficient tuning, and chain-of-thought reasoning, but CRS work must contend with scarce training data.The paper also discusses self-consistency and boosting as ways to amplify reasoning.Source: §5 · p. 9

CRS Datasets

  • Existing CRS datasets either focus on small, conversation-only domains or adapt large-domain data into rigid or synthetic conversations, unlike the large-corpus slate setting targeted here.The authors identify human evaluation and a public dataset as future work for quantitatively evaluating RecLLM design alternatives.Source: §5 · pp. 9–10

Dialogue Management

  • The unified LLM dialogue manager extends prior work by integrating large-scale recommendation, internal language representations, and natural-language inputs such as profiles and item-slate descriptions.The paper also discusses LLM tuning through reinforcement learning and human feedback [1] [27] [65].Source: §5 · pp. 10–11

Recommendations / Explanations

  • The paper frames recommendation retrieval as an integration of LLMs with external resources, while prior retrieval-enhanced work also reports improved question answering and reduced hallucinations.The cited discussion distinguishes existing recommendation-domain retrieval approaches from work on external memory and retrieval-augmented models.Source: §5 · p. 10
  • RecLLM applies LLM ranking to CRS items with heterogeneous metadata, an application the authors say they are unaware of in prior work.Its explanations are post hoc natural-language justifications, leaving their correctness open to verification.Source: §5 · pp. 10–11

User Profile

  • Recent works explore extracting transparent natural language user profiles to personalize open-ended chatbots [56] [101] [109] and recommender systems [70] [87]; BlenderBot [80] also separates the problem into extraction, triggering, retrieval, and generation phases.Source: §5 · p. 11

Simulation / Large scale training

  • Prior user-simulation and synthetic-data work informs controlled simulation, but generating full conversations partly outside the simulator's control may require more than basic few-shot prompting.The paper relates its approach to label-conditioned synthetic data generation [93].Source: §5 · p. 11

6 RECLLM PROTOTYPE

The initial RecLLM prototype uses Search API Lookup with few-shot prompting and a public YouTube search API. LaMDA powers its dialogue, recommendation, explanation, profile-integration, and simulation functions.

6 RECLLM PROTOTYPE

  • RecLLM implements retrieval through Search API Lookup, in-context few-shot learning, and a public YouTube search API.Source: §6 · p. 11
  • LaMDA powers dialogue management, recommendations, explanations, user-profile integration, and user simulation in the prototype.Source: §6 · p. 11
  • The appendix presents sample RecLLM sessions to demonstrate core competencies.Source: §6 · p. 11

7 ETHICAL CONSIDERATIONS

The paper describes potential benefits of LLM-based recommendation, including item-attribute reasoning, natural-language justifications, user control, and editable profiles. It also notes that inherited LLM problems and resource costs remain unresolved constraints on production deployment.

7 ETHICAL CONSIDERATIONS

  • The proposed system offers item-attribute reasoning, natural-language justifications users can validate, nuanced language-based control, and editable profiles.Source: §7 · p. 11
  • LLM reliance brings societal biases learned through pretraining, hallucinations, and expensive resource use into the proposed system.Source: §7 · p. 11
  • Task controls are unlikely to fully remove inherited LLM issues, and the paper calls for progress in debiasing, factual grounding, and efficient serving before safe production deployment.Source: §7 · p. 11

8 CONCLUSIONS AND FUTURE WORK

The conclusion presents RecLLM as a proof-of-concept roadmap for an end-to-end CRS that uses LLM generalization and synthetic conversations without existing product logs. It identifies evaluation, interaction scope, and scaling as future work.

8 CONCLUSIONS AND FUTURE WORK

  • RecLLM is presented as a proof of concept for an end-to-end CRS that uses LLM generalization and synthetic training conversations without existing product logs.The roadmap aims to support a healthier recommender system ecosystem with controllable and explainable CRSs.Source: §8 · p. 11
  • The authors plan human evaluations and a public dataset to support quantitative evaluation of RecLLM design alternatives in multimodal, large-scale CRSs.These resources are intended to help the community study CRSs in that setting.Source: §8 · p. 11
  • The current setting assumes users interact only through conversation, while future work would add feedback channels such as clicks or likes and hierarchical item shelves.The authors describe the conversation-only assumption as simplified.Source: §8 · p. 11
  • Large-scale tuning remains proposed rather than established: RecLLM currently relies on in-context few-shot learning or tuning on small amounts of crowdsourced data.The authors say proving out synthetic-data tuning ideas is critical for handling huge item corpora and the full space of possible conversations.Source: §8 · pp. 11–12
  • The authors identify question answering over corpus items as a mixed-initiative conversational use case for future support.Source: §8 · p. 12

A RECLLM SAMPLES

The appendix presents a small set of RecLLM samples grouped by competency. The samples visualize only the final recommendation slate, although the slate evolves during conversation in the interface.

A RECLLM SAMPLES

Related papers

Loading 2305.07961v2…