PaperlayerView exact revision on arXiv

Source-linked AI summary

Advances and Challenges in Conversational Recommender Systems: A Survey

Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke, Tat-Seng Chua

arXiv:2101.09459v7cs.IRcs.CL

TL;DR

Static recommenders struggle to identify users’ exact preferences and motivations from offline historical behavior. This survey reviews CRSs through a three-component framework, synthesizes five research directions, and discusses their challenges and future opportunities. It concludes that CRS models, technologies, datasets, and evaluation remain insufficiently mature for complex real-world use.

  • Problem

    Static models infer preference from historical behavior without explicit instructions or active feedback, making users’ exact preferences and motivations difficult to determine.

  • Method

    The survey defines a CRS framework with a user interface, conversation strategy module, and recommendation engine, then reviews work across five primary research directions.

  • Results

    The survey synthesizes existing CRS techniques, identifies limitations across five directions, and outlines future challenges and opportunities.

  • Takeaways & Limitations

    CRS research offers a roadmap spanning preference elicitation, conversational strategy, dialogue, exploration, and evaluation for researchers across multiple communities.

  • Takeaways & Limitations

    Current CRS datasets are too small and insufficiently natural or diverse to generalize reliably to complex real-world conversations.

Abstract

from arXiv · show

Recommender systems exploit interaction history to estimate user preference, having been heavily used in a wide range of industry applications. However, static recommendation models are difficult to answer two important questions well due to inherent shortcomings: (a) What exactly does a user like? (b) Why does a user like an item? The shortcomings are due to the way that static models learn user preference, i.e., without explicit instructions and active feedback from users. The recent rise of conversational recommender systems (CRSs) changes this situation fundamentally. In a CRS, users and the system can dynamically communicate through natural language interactions, which provide unprecedented opportunities to explicitly obtain the exact preference of users. Considerable efforts, spread across disparate settings and applications, have been put into developing CRSs. Existing models, technologies, and evaluation methods for CRSs are far from mature. In this paper, we provide a systematic review of the techniques used in current CRSs. We summarize the key challenges of developing CRSs in five directions: (1) Question-based user preference elicitation. (2) Multi-turn conversational recommendation strategies. (3) Dialogue understanding and generation. (4) Exploitation-exploration trade-offs. (5) Evaluation and user simulation. These research directions involve multiple research fields like information retrieval (IR), natural language processing (NLP), and human-computer interaction (HCI). Based on these research directions, we discuss some future challenges and opportunities. We provide a road map for researchers from multiple communities to get started in this area. We hope this survey can help to identify and address challenges in CRSs and inspire future research.

1. Introduction

Static recommenders infer preferences from offline historical behavior, but CRSs use real-time multi-turn interaction to elicit dynamic preferences and act on current needs. This survey organizes CRS research around a three-component framework and five primary challenges.

  • Limitations of Static Recommendations: Static recommendation models analyze past behaviors offline, but historical interactions can be sparse, noisy, mistaken, or outdated representations of preference.These limitations make it difficult to determine exactly what users like and why they like an item.
  • Introduction of CRSs: CRSs elicit users’ dynamic preferences and act on current needs through real-time multi-turn interactions.Conversational interaction can elicit current preferences and motivations behind consumption behavior.
  • Connections with Interactive Recommendations: Existing interactive and critiquing recommenders are less flexible because they use predefined options and recommend after each feedback signal, whereas CRSs support richer interaction and selective recommendation timing.CRSs can use tags, template utterances, or free natural language, and some strategies determine when to ask versus recommend.
  • General CRS Framework: A CRS comprises a user interface, conversation strategy module, and recommendation engine that translate user input, coordinate interaction, and model or retrieve recommendations.The conversation strategy coordinates the other components, while the recommendation engine models entities and records preferences.
  • Five Primary Challenges: The survey identifies five challenges: question-based preference elicitation, multi-turn strategies, dialogue understanding and generation, exploitation-exploration trade-offs, and evaluation with user simulation.These challenges span the CRS components and organize the survey’s review of existing efforts and limitations.

2. Question-based User Preference Elicitation

Question-based preference elicitation asks users about items or attributes so CRSs can construct and update user profiles during multi-turn interaction. Methods range from choice and probabilistic approaches to sequential and reinforcement-learning strategies for selecting questions and recommendations.

  • Overview: CRSs proactively ask users questions and use their feedback to understand needs, attitudes, and current preferences before recommending items.Recommendations can also be adjusted when users are dissatisfied.
  • Question-driven methods: Question-driven methods ask either about items or about their attributes, topics, or categories.Attribute-centric questions are generally more practical when candidate item sets are large and users may become bored by longer conversations.
  • Choice-based methods: Choice-based methods repeatedly let users select preferred items or item sets from options, including pairs, lists, or pairs of lists.Users may ignore a question when they cannot distinguish between the options or do not care about the difference.
  • Bayesian preference elicitation: Bayesian preference elicitation represents a user’s preference as a probability distribution, computes expected item utility, and updates the belief distribution from feedback.The system can use the utility function to select items for querying and recommendation.
  • Query strategies: Query strategies include POMDP and EVOI approaches, while query types include pairwise comparisons and other item-selection schemes.These strategies determine which items to ask about during preference elicitation.
  • Sequential and interactive methods: Sequential neural models predict the next attribute to ask or item to recommend, whereas MAB, meta-learning, and deep reinforcement-learning methods support interactive recommendation.MAB and meta-learning methods balance exploration and exploitation, while deep reinforcement learning can model dynamic preference and long-term utility.

3. Multi-turn Conversational Strategies for CRSs

CRS conversation strategies decide when to continue eliciting preferences and when to recommend, while adapting to feedback, topics, and scenarios. Recent approaches use learned state representations, reinforcement learning, and knowledge graphs to improve this timing and efficiency.

  • Conversation Strategies for Determining When to Ask and Recommend: A good strategy chooses between asking further questions to reduce preference uncertainty and recommending when confidence is sufficiently high.Strategies may be rule-based, random, or learned; fixed heuristics can determine recommendation timing but do not capture rich conversational semantics.
  • Conversation Strategies for Determining When to Ask and Recommend: Reinforcement learning models conversational state and selects whether to ask about an attribute or yield a recommendation.One policy uses l + 1 actions: l facet-question actions and one recommendation action, with rewards from user feedback and automatic recommendation evaluation.
  • Conversation Strategies for Determining When to Ask and Recommend: The EAR framework defines the right time to recommend using candidate-space size, expected usefulness of further questions, user patience, and recommendation confidence.Its workflow estimates the current state, takes an ask-or-recommend action, and reflects on available information; CRM’s latent state is noted as difficult to interpret.
  • Conversation Strategies for Determining When to Ask and Recommend: Knowledge-graph integration in CPR restricts attribute search and reduces the policy action space to asking an attribute or recommending an item.The survey reports that this makes CPR more efficient than EAR and improves multi-turn conversational reasoning.
  • Section Summary: Beyond ask-or-recommend timing, CRSs can lead multi-topic conversations and support suggesting, negotiating, persuading, and proactive transitions from other dialogue types.These capabilities address scenarios in which users may not initially have a clear idea of what they want and require datasets supporting diverse interactions.

4. Dialogue Understanding and Generation in CRSs

Dialogue understanding and generation are central to CRSs because recommendation depends on extracting user intentions from natural language and producing responses that are proper, useful, and recommendation-relevant. Existing approaches range from structured slot filling and end-to-end neural models to retrieval and generation methods, but interpretability and response quality remain challenges.

  • Dialogue Understanding: Most CRSs rely on preprocessed ratings, binary answers, or attribute values instead of extracting intentions from users’ raw utterances.This constrains user expression and motivates explicit or implicit semantic extraction from natural language.
  • Slot Filling: Slot-filling approaches define aspects of interest and infer their values from user input, including facet-value pairs encoded for recommendation and policy modules.One belief tracker uses n-gram utterance vectors and an LSTM to represent each facet’s possible values.
  • Intentions and Sentiment Learning: Explicit aspect-value representations can be inadequate for intent and sentiment, so some CRSs learn these representations implicitly with end-to-end neural models.Deep neural approaches can model natural-language semantics but are criticized as non-transparent and difficult to interpret.
  • Intentions and Sentiment Learning: End-to-end neural frameworks can map raw utterances directly to responses while preserving mentioned items, attributes, attitudes, and interests.The REDIAL dataset supports natural-language movie recommendation dialogues in which movie mentions are explicitly tagged.
  • Intentions and Sentiment Learning: Probes of BERT indicate that it can learn and retain both content-based and collaborative-based item knowledge relevant to conversational search and recommendation.The survey presents this as evidence that end-to-end language models may support direct interaction in complex contexts.
  • Response Generation: Template-based responses require substantial manual work and can be repetitive or inflexible, while generated responses may lack well-formedness, commonsense, personality, emotion, or user-profile awareness.Recommendation dialogue also requires responses to mention recommended entities and provide meaningful, useful information rather than merely grammatical text.
  • Section Summary: CRSs differ in interface mechanisms: some use pre-annotated inputs with rule- or template-based outputs, whereas dialogue-oriented systems more often process raw language and generate responses automatically.The survey identifies user understanding and response generation as continuing critical research areas because they directly affect user experience.

5. Exploration-Exploitation Trade-offs

CRSs use exploration–exploitation strategies to balance learning about uncertain options with selecting options already estimated to be effective. Multi-armed bandits support this balance through online feedback, while contextual variants incorporate user and item features.

  • Exploration–Exploitation Trade-offs: Exploration–exploitation balances exploiting the best known option against exploring unknown options, sometimes sacrificing short-term performance for long-term optimization.The trade-off is especially relevant to cold-start users but can also improve recommendations for warm users.
  • 5.1.1. Introduction to Multi-Armed Bandits: Multi-armed bandits model recommendation as repeated arm selection, where the system uses observed rewards to choose among uncertain alternatives.The objective can be expressed as maximizing cumulative reward or minimizing regret relative to the theoretically optimal arm.
  • 5.1.1. Introduction to Multi-Armed Bandits: Common bandit strategies include greedy, random, ε-greedy, Upper Confidence Bound, and Thompson Sampling.Greedy always exploits the currently highest estimated reward, random always explores, and ε-greedy mixes them with probability ε.
  • 5.1.2. Recommendation via MAB-based Methods: Contextual bandits address the inefficiency of treating items as independent arms by using user and item features to predict rewards.LinUCB models expected reward from a joint user–item feature vector and derives recommendations from an upper confidence bound.
  • 5.1.2. Recommendation via MAB-based Methods: Recommendation-oriented bandit extensions include contextual, cascading, pseudo-reward, and item-dependency models for richer feedback and candidate relationships.These approaches respectively use features, ranked-list examination, estimates for unselected actions, and dependencies among items.
  • 5.1.2. Recommendation via MAB-based Methods: Bandit methods enable online learning, allowing systems to learn cold-user preferences and adjust recommendation strategies after several trials.They can be integrated into online recommendation, interactive recommendation, and conversational recommender systems, with arms corresponding to items.

6. Evaluation and User Simulation

CRS evaluation remains underexplored and spans both individual system turns and the performance of multi-turn conversation strategies. User simulation is important for evaluating the latter.

  • 6. Evaluation and User Simulation: CRS evaluation covers turn-level outputs and conversation-level multi-turn strategies.Turn-level evaluation treats recommendation and response generation as supervised prediction tasks, while conversation-level evaluation concerns sequential decision making.

6.1. Datasets and Tools

CRS research uses crowdsourced and other human–human or human–machine conversation datasets, but existing resources remain insufficient for industrially applicable systems. Tools such as CRSLab organize CRS tasks and evaluation components.

  • 6.1. Datasets and Tools: Common CRS datasets include human–human and human–machine conversations collected under specified rules, often through crowdsourcing platforms such as Amazon Mechanical Turk.Participants may be rewarded for providing qualified data.
  • 6.1. Datasets and Tools: Existing CRS datasets lack sufficient scale and conversational naturalness to support systems intended for industrial applications.Many datasets do not cover real-world entities and concepts, or derive conversations from non-conversational data and rigorous constraints.
  • 6.1. Datasets and Tools: The survey calls for large-scale, generalizable, diverse, and natural CRS datasets.This need follows from the limited real-world coverage and constrained construction of existing resources.
  • 6.1. Datasets and Tools: CRSLab unifies CRS work into recommendation, conversation, and policy subtasks corresponding to the recommendation engine, user interface, and conversation strategy module.The toolkit implements selected models and metrics for building and evaluating CRSs.

6.2. Turn-level Evaluation

Turn-level CRS evaluation measures recommendation and language-generation outputs, but lexical metrics and sparse interaction records provide incomplete or biased evidence. Human assessment and online feedback therefore remain important for judging CRS quality.

  • 6.2. Turn-level Evaluation: Turn-level evaluation examines two tasks: language generation and recommendation.Fine-grained assessment focuses on the output produced at each individual conversation turn.
  • 6.2.1. Evaluation of Language Generation: BLEU and ROUGE evaluate generated responses through lexical precision and recall against reference utterances.BLEU measures generated-word or n-gram precision, whereas ROUGE measures reference-word or n-gram recall.
  • 6.2.1. Evaluation of Language Generation: Lexical metrics may not capture semantic or syntactic variation and do not directly measure long-term dialogue success or user satisfaction.The survey therefore identifies user-satisfaction-oriented metrics as more suitable for some CRS evaluations.
  • 6.2.1. Evaluation of Language Generation: Human evaluation found that about one-third of system utterances were not meaningful in context and fewer than two-thirds of recommendations were meaningful.The evaluated end-to-end systems also largely reproduced utterances already present in training data, motivating human and expert assessment.
  • 6.2.2. Evaluation of Recommendation: Recommendation evaluation compares predicted results with test-set records using rating-based or ranking-based metrics.Rating-based metrics assess score divergence, while ranking-based metrics assess whether predicted item order matches test-set order and can use implicit interactions.
  • 6.2.2. Evaluation of Recommendation: Sparse real-world interactions make consumed test-set items an incomplete proxy for user preferences and can introduce evaluation bias.CRSs can collect online feedback about item satisfaction, enabling conversation-level assessment that may avoid this bias.

6.3. Conversation-level Evaluation

Conversation-level evaluation assesses how a CRS policy performs over sequential, feedback-dependent interactions, using online tests, off-policy evaluation, and user simulation. Each approach trades realism, cost, variance, or bias, while reliable user simulation remains an open challenge.

  • Conversation-level evaluation measures policy performance across sequential interactions rather than isolated supervised predictions.Each system action can influence future observations, and evaluation depends heavily on user feedback.
  • Online user tests directly evaluate conversation policies with real feedback using metrics such as average turn and recommendation success rate.Average turn captures how many turns are needed to complete a successful recommendation, while recommendation success rate measures success by turn.
  • Off-policy evaluation estimates a target policy from behavior-policy logs using importance sampling or inverse propensity scores to correct distribution mismatch.The estimator can have high variance when the target and behavior policies differ substantially; weight clipping and TRPO reduce this variance.
  • User simulation efficiently supplies feedback without online testing, but simulated preferences may diverge from real users and introduce bias.Simulation strategies use interaction histories, item-preference estimates, reviews, or human conversational corpora.
  • Reliable user simulation remains immature, despite agenda-based approaches that update simulated-user actions through push and pull operations.The user agenda is a stack-like structure updated according to dialogue state and the CRS action.

6.4. Section Summary

CRS evaluation spans turn-level prediction and conversation-level strategy assessment, but the field still lacks large-scale dense data and uniform procedures for comparing both aspects together.

  • Turn-level evaluation measures single-round recommendation and language-generation performance, whereas conversation-level evaluation measures multi-turn strategy performance.
  • Online user tests are costly, so researchers commonly use off-policy evaluation and user simulation as alternatives.
  • CRS evaluation still needs large-scale dense conversational recommendation data and uniform methods integrating recommendation and conversation quality.
  • Figure 8 illustrates an agenda sequence and state transition in which one agenda update uses push and the others use pull operations.

7. Future Directions and Opportunities

Future CRS research should jointly optimize system components, address persistent bias, enrich interactions with multimodal knowledge, and improve multi-turn strategies and evaluation. These directions reflect limitations in current modular, naive, and simulation-dependent approaches.

  • 7.1. Jointly Optimizing Three Tasks: The user interface, conversation strategy, and recommendation engine share objectives and data, motivating joint learning under an explicit conversation strategy.The strategy could plan dialogue acts using item relationships such as complementarity and substitutability.
  • 7.2. Bias and Debiasing: CRSs can query users about popular items to reduce some popularity or conformity biases, but exposure and selection biases persist and may be amplified by feedback loops.
  • 7.3. Multi-turn Conversational Recommendation Strategies: Current multi-turn strategies are often hand-crafted or lack explicit control, while reinforcement-learning designs still have room to improve their states, actions, and rewards.More sophisticated actions could include answering open-domain questions and chatting about non-task-oriented topics.
  • 7.4. Multi-modal Knowledge Enrichment: Knowledge graphs, item attributes, and multimodal data offer additional information for modeling preferences and enriching CRS interaction.Visual dialogue systems can use images to improve entity representations and convey information to users.
  • 7.5. Evaluation and User Simulation: Evaluation needs more faithful user simulators, dense interaction data, and comprehensive procedures that compare CRS models across differing datasets and assumptions.Simulated users cannot fully replace humans, and comprehensive evaluation metrics remain an open problem.

8. Conclusion

Static recommenders cannot directly communicate with users about preferences, whereas CRSs use real-time interaction to address this information asymmetry. The survey organizes CRS research into a common framework, synthesizes five directions, and identifies further challenges for the field.

  • Traditional recommenders estimate preferences from historical behavior without opportunities to communicate with users, creating a fundamental information asymmetry.
  • The survey introduces a CRS framework with three components: user interface, conversation strategy module, and recommendation engine.
  • The survey synthesizes five directions: preference elicitation, multi-turn strategies, dialogue understanding and generation, exploitation-exploration, and evaluation with user simulation.
  • Future challenges include joint component optimization, bias debiasing, multi-turn strategies, multimodal knowledge enrichment, and evaluation and user simulation.
  • The survey’s retrospective is intended to provide a basis for further CRS development and inspire researchers and practitioners across academia and industry.
Loading 2101.09459v7…