Source-linked AI summary
Towards Topic-Guided Conversational Recommender System
Kun Zhou, Yuanhang Zhou, Wayne Xin Zhao, Xiaoke Wang, Ji-Rong Wen
TL;DR
Existing CRS datasets provide limited proactive guidance from ordinary conversation into recommendation. The paper introduces TG-ReDial, a real-data, semi-automatic dataset organized by topic threads, defines a three-subtask topic-guided recommendation task, and reports effective performance across those subtasks.
Problem
Existing CRS datasets mainly assume immediate user requests and do not adequately study explicit semantic transitions into recommendation.
Method
The paper constructs TG-ReDial from real user data with topic threads and presents a solution using multiple data signals for item recommendation, topic prediction, and response generation.
Results
Extensive experiments demonstrate the proposed approach’s effectiveness on item recommendation, topic prediction, and response generation.
Takeaways & Limitations
TG-ReDial provides a dataset and testbed for conversational recommendation with natural topic transitions and historical user interaction data.
Takeaways & Limitations
The potential of TG-ReDial has not been fully explored, and further work is needed on additional tasks and more effective approaches.
Abstract
from arXiv · showhide
Conversational recommender systems (CRS) aim to recommend high-quality items to users through interactive conversations. To develop an effective CRS, the support of high-quality datasets is essential. Existing CRS datasets mainly focus on immediate requests from users, while lack proactive guidance to the recommendation scenario. In this paper, we contribute a new CRS dataset named \textbf{TG-ReDial} (\textbf{Re}commendation through \textbf{T}opic-\textbf{G}uided \textbf{Dial}og). Our dataset has two major features. First, it incorporates topic threads to enforce natural semantic transitions towards the recommendation scenario. Second, it is created in a semi-automatic way, hence human annotation is more reasonable and controllable. Based on TG-ReDial, we present the task of topic-guided conversational recommendation, and propose an effective approach to this task. Extensive experiments have demonstrated the effectiveness of our approach on three sub-tasks, namely topic prediction, item recommendation and response generation. TG-ReDial is available at https://github.com/RUCAIBox/TG-ReDial.
1 Introduction
Existing CRS datasets often assume immediate user requests and do not adequately model proactive semantic transitions into recommendation. TG-ReDial addresses this gap with a semi-automatically constructed, topic-guided dataset and a three-subtask recommendation framework.
- Existing CRS datasets largely assume clear, immediate requests and underrepresent proactive transitions from non-recommendation conversation to recommendation.
- TG-ReDial contains 10,000 two-party movie-domain dialogues between seekers and recommenders.
- Topic threads guide each conversation from an initial non-recommendation topic toward recommendation through evolving topics and chit-chat.
- Real user data supports recommended movies, topic threads, recommendation reasons, and user identities, while annotators revise, polish, or rewrite conversation data.
- The paper releases TG-ReDial and proposes a Transformer-based solution that leverages multiple data signals across the three subtasks.
- Topic-guided conversational recommendation comprises item recommendation, topic prediction, and response generation.
2 Related Work
Prior conversational systems span task-oriented and chit-chat settings, while CRS research has produced datasets with varying degrees of synthetic construction and human annotation. TG-ReDial differs from DuRecDial by modeling content evolution with topic threads and using real user-related data.
- Conversation systems include task-oriented systems for specific goals and chit-chat systems for general-purpose dialogue.
- Existing CRS datasets include synthetic template-based resources and human-annotated datasets such as ReDial, GoReDial, and DuRecDial.
- DuRecDial uses goal sequences spanning multiple task types, whereas TG-ReDial uses topic threads to characterize evolving content flow.
- TG-ReDial is designed to be easier to integrate with open-domain dialogue because its topic threads describe content evolution rather than task switching.
- Unlike DuRecDial, TG-ReDial derives user-related data from real users rather than relying on annotators to generate it.
3 Dataset Construction
TG-ReDial is built from real movie-watching and user data, topic-linked recommendations, and semi-automatic human annotation. The resulting dataset combines guided dialogue construction with user profiles, interaction histories, and privacy protections.
- Dataset setting: The two-party setting begins with chit-chat, proactively moves toward a target topic, and then recommends a movie using the seeker’s interests.
- Collection process: Three movies sharing the “family” tag illustrate how film records, ConceptNet topic threads, and candidate sentences are combined during collection.
- Real-world data: Conversations are associated with real Douban users, enabling watching records, user profiles, and review-derived recommendation reasons to support recommendation.
- Movie and tag preparation: Movies are grouped into coherent watching subsequences and annotated with meaningful tags drawn from original metadata, reviews, and manual selection.
- Creating topic threads: Topic threads connect an initial greeting to a recommended movie’s target tag by traversing ConceptNet and using shortest paths identified with depth-first search.
- Human annotation: Annotators generate utterances from retrieved topic-relevant candidates and revise them for dialogue consistency, while recommendation reasons come from retrieved review sentences.
- Dataset statistics: TG-ReDial contains 129,392 utterances from 1,482 users; dialogues average 7.9 topics and 19.0 words per utterance.
- Privacy: Privacy protection restricts sampling to users with many watching records and anonymizes or modifies derived user data before excluding identifiable users.
4 Our Approach
The approach formulates topic-guided conversational recommendation as coordinated topic prediction, item recommendation, and response generation. It combines conversation, topic, profile, interaction, and pretrained language-model signals across specialized modules.
- Task formulation: The task uses user profile, interaction history, prior utterances, and topics to predict the next topic, recommend an item, and generate a response.
- Recommendation module: The recommendation module combines BERT representations of historical utterances with SASRec representations of the user interaction sequence.These signals form the user representation used for recommendation.
- Recommendation module: The recommendation module ranks items using their learned embeddings and the user representation, selecting the item with the largest probability.
- Dialog module: The dialog module generates responses for topic guidance or item recommendation using specific models, including GPT-2 for response generation.
- Dialog module: The topic prediction model uses conversation-BERT, topic-BERT, and profile-BERT to encode historical utterances, topic history, and the user profile.The model ranks candidate topics and selects the one with the largest probability.
5 Experiments
Experiments evaluate the approach on item recommendation, topic prediction, and response generation using TG-ReDial and task-specific baselines and metrics. The proposed model outperforms the baselines across the reported tasks, with its gains attributed to combining relevant conversational, interaction, topic, and item signals.
- Experimental Setup: TG-ReDial is split into training, validation, and test sets with an 8:1:1 ratio, and evaluation covers item recommendation, topic prediction, and response generation.The model generates replies or recommendations turn by turn from each conversation's first utterance.
- Item Recommendation: Item recommendation uses NDCG@k and MRR@k with k = 10 and 50 to rank all possible items.
- Item Recommendation: The proposed model significantly outperforms all item-recommendation baselines by combining historical utterances with interaction sequences.The comparison includes CRS, interaction-history, and text-based baselines; BERT outperforms TextCNN among the text-based models.
- Topic Prediction: Topic prediction uses Hit@k with k = 1, 3, and 5, comparing heuristic, recurrent, BERT-based, and target-topic ablation baselines.The task predicts the next topic leading toward the recommendation target.
- Topic Prediction: The proposed topic-prediction model outperforms all baselines, while removing the target topic significantly decreases performance.The model jointly encodes historical utterances, topics, and user profiles with different BERT models.
- Response Generation: For response generation, the proposed model outperforms all baselines in most cases, especially for perplexity, Distinct, and human evaluation.Transformer, GPT-2, and the proposed model have similar BLEU scores, while predicted topics or items enhance generated text quality in the proposed approach.
6 Conclusion
The paper introduces TG-ReDial, a human-annotated dataset based on real-world user data, and a topic-guided conversational recommendation task with an effective solution. Experiments demonstrate effectiveness across three sub-tasks, while the dataset's broader potential remains underexplored.
- TG-ReDial is a high-quality conversational-recommender dataset constructed through human annotation based on real-world user data.
- The paper presents topic-guided conversational recommendation and a solution evaluated on item recommendation, topic prediction, and response generation.
- The proposed approach demonstrates effectiveness on all three sub-tasks in extensive experiments.
- TG-ReDial's potential has not been fully explored, including possible applications to personalized chit-chat, target-guided conversation, and sequential recommendation.The authors identify broader task exploration and more effective approaches as future work.