PaperlayerView exact revision on arXiv

Source-linked AI summary

A Unified Multi-task Learning Framework for Multi-goal Conversational Recommender Systems

Yang Deng, Wenxuan Zhang, Weiwen Xu, Wenqiang Lei, Tat-Seng Chua, Wai Lam

arXiv:2204.06923v1cs.IRcs.CL

TL;DR

MG-CRS research often addresses only some tasks or treats them independently, despite their interdependencies. UniMIND unifies four tasks through Seq2Seq formulations and prompt-based multi-task learning, and achieves state-of-the-art performance on each task across two MG-CRS datasets.

  • Problem

    Existing MG-CRS systems often address only some tasks or learn them independently, although the tasks are closely related and expected to be learned automatically.

    Source: §1 · p. 2

  • Method

    UniMIND reformulates goal planning, topic prediction, item recommendation, and response generation as Seq2Seq tasks and uses prompt-based learning for multi-task learning.

    Source: §1 · pp. 2, 4

  • Results

    UniMIND achieves state-of-the-art performance on each MG-CRS task with a unified model on two datasets.

    Source: §1 · p. 4, §8 · p. 21

  • Takeaways & Limitations

    Analyses highlight the importance of each task and the difficulty of handling different dialogue types in MG-CRS.

    Source: §8 · p. 21

    On TG-ReDial, UniMIND and the baselines have low recommendation success amid sparse interactions, and UniMIND does not use historical interaction data.

    Source: §7.4 · p. 20

Abstract

from arXiv · show

Recent years witnessed several advances in developing multi-goal conversational recommender systems (MG-CRS) that can proactively attract users' interests and naturally lead user-engaged dialogues with multiple conversational goals and diverse topics. Four tasks are often involved in MG-CRS, including Goal Planning, Topic Prediction, Item Recommendation, and Response Generation. Most existing studies address only some of these tasks. To handle the whole problem of MG-CRS, modularized frameworks are adopted where each task is tackled independently without considering their interdependencies. In this work, we propose a novel Unified MultI-goal conversational recommeNDer system, namely UniMIND. In specific, we unify these four tasks with different formulations into the same sequence-to-sequence (Seq2Seq) paradigm. Prompt-based learning strategies are investigated to endow the unified model with the capability of multi-task learning. Finally, the overall learning and inference procedure consists of three stages, including multi-task learning, prompt-based tuning, and inference. Experimental results on two MG-CRS benchmarks (DuRecDial and TG-ReDial) show that UniMIND achieves state-of-the-art performance on all tasks with a unified model. Extensive analyses and discussions are provided for shedding some new perspectives for MG-CRS.

1 INTRODUCTION

MG-CRS aims to guide conversations through multiple goals and topics, but existing approaches often handle only some tasks or train them independently. UniMIND brings the four tasks into a shared Seq2Seq framework with prompt-based multi-task learning.

1 INTRODUCTION

  • MG-CRS comprises goal planning, topic prediction, item recommendation, and response generation, but existing systems often omit tasks, learn them independently, or assume goals are known.The tasks are expected to be learned and inferred automatically despite differing paradigms and information needs.Source: §1 · p. 2
  • UniMIND reformulates the four MG-CRS tasks as Seq2Seq tasks and uses prompt-based learning to support multi-task learning.The paper also investigates special-token prompting to bridge relationships within each type of information.Source: §1 · pp. 2, 4

2.1 Conversational Recommendation

  • CRS work spans attribute-based and open-ended systems, while MG-CRS extends recommendation to conversations led through multiple goals and topics.Prior MG-CRS studies either assume a complete goal sequence or jointly address only some tasks.Source: §2.1 · p. 4

2.2 Pre-trained Seq2Seq Models for Dialogue

  • Pre-trained Seq2Seq models support dialogue generation, and UniMIND seeks to use them across all MG-CRS tasks.Existing CRS typically use pre-trained Seq2Seq models for response generation.Source: §2.2 · pp. 4–5

2.3 Paradigm Shift & Prompt-based Learning

  • Paradigm-shift work has applied Seq2Seq to dialogue-state tracking and action prediction, while UniMIND extends the approach to MG-CRS subtasks spanning classification, ranking, and generation.The framework also designs prompt-based approaches for using pre-trained Seq2Seq models in MG-CRS.Source: §2.3 · p. 5

3 PROBLEM DEFINITION

MG-CRS coordinates goal planning, topic prediction, item recommendation, and response generation across a conversation. UniMIND reformulates these tasks as Seq2Seq tasks, adds task-specific prompts, and uses multi-task training, prompt-based learning, and inference.

3 PROBLEM DEFINITION

  • MG-CRS plans the next goal, predicts topics, recommends items, and generates a response using dialogue history and, where available, user profiles.The generated response concerns the predicted topics and selected goal; recommendation responses are also expected to give persuasive reasons for the item.Source: §3 · pp. 5–6

4 METHOD

  • UniMIND reformulates MG-CRS tasks in a unified Seq2Seq paradigm and uses a three-stage procedure: multi-task learning, prompt-based learning, and inference.The method section introduces the paradigm shift and prompt strategies for multi-task learning.Source: §4 · p. 6

4.1.1 Goal Planning.

  • Goal Planning directly generates the next-goal label as natural language, unifying goal completion estimation and goal prediction and potentially alleviating two-stage error propagation.The objective maximizes the sequential log-likelihood of the generated goal label.Source: §4.1.1 · p. 7

4.1.2 Topic Prediction.

  • Topic Prediction treats labels as a sequence-to-generate, separating individual labels with a special token so predictions can vary in number.The approach is intended to handle large, variable topic sets and use semantic relationships between context and labels.Source: §4.1.2 · p. 7

4.1.3 Item Recommendation.

  • Item Recommendation expands the vocabulary with candidate items and generates an item expression containing both its index and its text.At inference, the model decodes one step and ranks items by probability in the expanded vocabulary.Source: §4.1.3 · p. 8

4.1.4 Response Generation.

  • Response Generation remains standard Seq2Seq text generation, with the response target length denoted by L_d.The paper states this task requires no paradigm shift.Source: §4.1.4 · p. 8

4.2 Prompt-based Learning

  • Task-specific prompts identify which task to perform; UniMIND investigates natural-language prompts and special-token prompts.Special segment tokens can distinguish information types, including groups of goal and topic labels and system versus user utterances.Source: §4.2 · pp. 8–9

4.3 Multi-task Training and Inference

  • UniMIND initializes from a pretrained encoder-decoder language model, trains jointly on all tasks, then tunes prompts for each task before ordered inference.Inference produces the next goal, topic, item, and response using corresponding tuned models.Source: §4.3 · p. 9

6 end

Algorithm 1 marks the beginning of prompt-based learning at step 7.

6 end

  • Algorithm 1 marks step 7 as the start of prompt-based learning.Source: §6 · p. 10

15 end

The procedure obtains the next goal, topic, item, and response in order using task-tuned models.

15 end

  • Inference obtains the next goal, topic, item, and response in order using the corresponding tuned models.Source: §15 · p. 10

5 EXPERIMENTAL SETUPS

The experiments evaluate UniMIND on DuRecDial and TG-ReDial, using task-specific baselines and metrics. Implementation uses a pretrained BARTbase model and a shared set of hyperparameters for both datasets.

5.1 Research Questions

  • The experiments ask how UniMIND compares on response generation and three subtasks, how training stages and subtasks contribute, and how MG-CRS differs from traditional CRS.Source: §5.1 · p. 10

5.2 Datasets

  • DuRecDial spans multiple domains and has 21 goal types; TG-ReDial is movie-domain and has 8 goal types, with user profiles in both datasets.DuRecDial provides goal and knowledge-entity labels; TG-ReDial labels actions as conversational goals.Source: §5.2 · p. 11

5.3.1 Response Generation.

  • Response generation is compared with CRS and general text-generation baselines, including knowledge-grounded and goal-conditioned variants, using F1, BLEU, Distinct, and perplexity.The compared baseline groups differ between TG-ReDial and DuRecDial.Source: §5.3.1 · pp. 11–12

5.3.2 Goal Planning.

  • Goal Planning is compared with MGCG, BERT, and BERT+CNN using macro-averaged precision, recall, and F1.Source: §5.3.2 · p. 12

5.3.3 Topic Prediction.

  • Topic Prediction is compared with MGCG, BERT-based topic and profile rankers, and Union using Hit@k and micro-averaged precision, recall, and F1.For empty gold-label responses, empty predictions receive precision, recall, and F1 of 1; non-empty predictions receive 0.Source: §5.3.3 · p. 12

5.3.4 Item Recommendation.

  • Item Recommendation is compared with interaction-history, dialogue-context, and combined baselines using NDCG@k and MRR@k.Source: §5.3.4 · p. 12

5.4 Implementation Details

  • Implementation uses BARTbase initialized with Chinese BART weights, shared hyperparameters across datasets, 15 epochs of multi-task training, and 5 epochs of prompt-based learning.BARTbase was selected to have a parameter count similar to BERTbase and GPT-2.Source: §5.4 · pp. 12–13

6 EXPERIMENTAL RESULTS

UniMIND significantly outperforms strong baselines across MG-CRS subtasks, including response generation, goal planning, topic prediction, and item recommendation. Human evaluation also favors its responses, while leaving room for improvement in informativeness and appropriateness.

6.1.1 Automatic Evaluation.

  • UniMIND achieves state-of-the-art F1 and BLEU and promising Dist and PPL performance on both datasets.The reported automatic evaluation covers content preservation, diversity, and fluency; pre-defined goals improve baseline performance.Source: §6.1.1 · p. 13, Table 3 · p. 14

6.1.2 Human Evaluation.

  • Human evaluation favors UniMIND across four aspects, with especially large gains in Appropriateness and Proactivity.The study sampled 100 TG-ReDial dialogues for evaluation; informativeness and appropriateness still trail human reference responses.Source: §6.1.2 · pp. 14–15, Table 4 · p. 14

6.2.1 Evaluation on Goal Planning.

  • Both UniMIND variants significantly outperform the goal-planning baselines on DuRecDial and TG-ReDial.BERT and BERT+CNN outperform MGCG among the baselines.Source: §6.2.1 · p. 15, Table 5 · p. 15

6.2.2 Evaluation on Topic Prediction.

  • UniMIND significantly outperforms topic-prediction baselines on both datasets in Hit and F1 scores; its Hit@1 exceeds Union’s Hit@5.Without target-topic guidance, prediction must draw on dialogue coherence, the historical topic thread, and the user profile.Source: Table 6 · p. 15, §6.2.2 · p. 16, Table 7 · p. 16

6.2.3 Evaluation on Item Recommendation.

  • UniMIND_S significantly and consistently outperforms Union on both datasets, while UniMIND_N outperforms Union on DuRecDial.UniMIND_S performs much better than UniMIND_N on item recommendation; the authors attribute this contrast to natural-language prompts not fully exploiting expanded item-vocabulary relationships.Source: §6.2.3 · pp. 16–17, Table 8 · p. 16, Table 9 · p. 17

6 EXPERIMENTAL RESULTS

  • UniMIND significantly outperforms strong baselines on each MG-CRS subtask, including systems with pre-defined goals.The authors report this overall pattern across both datasets.Source: §6.2.3 · p. 17

7 DETAILED ANALYSES & DISCUSSIONS

Ablations indicate that combining multi-task learning with task-specific prompt-based learning yields the best performance. Replacing dialogue-history-only response inputs with ground-truth goals, topics, and items improves performance by 15%–25%.

7.1.1 Effect of Training Procedure.

  • The best performance requires combining multi-task learning with task-specific prompt-based learning.The comparison includes independent Seq2Seq models for each task and one unified model without task-specific prompt tuning.Source: §7.1.1 · p. 17, §7.1 · p. 17, Table 10 · p. 18

7.1.2 Effect of Each Sub-task.

  • 15%–25%: response-generation performance improves from DirectGen to OracleGen.DirectGen uses dialogue history alone; OracleGen adds ground-truth goals, topics, and items to the input.Source: §7.1.2 · p. 17, Table 10 · p. 18

Response Generation

The Response Generation results report separate values for OracleGen and DirectGen.

Response Generation

  • OracleGen reports 39.43, 0.350/0.104, 55.07, and 0.525/0.444 across the listed columns.The passage does not identify the column labels.Source: Response Generation · p. 18
  • DirectGen reports 32.80, 0.291/0.070, 48.41, and 0.418/0.328 across the listed columns.The passage does not identify the column labels.Source: Response Generation · p. 18
  • The two rows list distinct values across each reported column.No metric names or dataset labels are provided in the passage.Source: Response Generation · p. 18

Goal Planning

The Goal Planning results list values for Oracle, BERT+CNN, and the variant without goal information.

Goal Planning

  • Oracle reports 36.78, 0.328/0.097, 53.28, and 0.498/0.417 across the listed columns.The passage does not identify the column labels.Source: Goal Planning · p. 18
  • BERT+CNN reports 35.51, 0.314/0.090, 51.67, and 0.475/0.396 across the listed columns.The passage does not identify the column labels.Source: Goal Planning · p. 18
  • The w/o goal variant reports 35.13, 0.309/0.086, 50.97, and 0.467/0.380 across the listed columns.The passage does not identify the column labels.Source: Goal Planning · p. 18

Topic Prediction

The Topic Prediction results list values for Oracle, Union, and the variant without topic information.

Topic Prediction

  • Oracle reports 38.22, 0.336/0.100, 52.68, and 0.492/0.413 across the listed columns.The passage does not identify the column labels.Source: Topic Prediction · p. 18
  • Union reports 34.32, 0.301/0.084, 51.09, and 0.469/0.383 across the listed columns.The passage does not identify the column labels.Source: Topic Prediction · p. 18
  • The w/o topic variant reports 35.01, 0.307/0.085, 48.73, and 0.426/0.340 across the listed columns.The passage does not identify the column labels.Source: Topic Prediction · p. 18

Item Recommendation

The analyses report differing contributions of MG-CRS tasks, goal-type-specific behavior, a dialogue case study, and limitations involving recommendation, informativeness, and error propagation.

Item Recommendation

  • Goal Planning and Topic Prediction influence final Response Generation more than Item Recommendation, though all subtasks contribute.The analysis reports that precise topic prediction can bring the most prominent improvement in final response generation.Source: Item Recommendation · p. 18

7.2 Performance w.r.t. Goal Type (RQ4)

  • Goal-type performance differs: Recommendation is easiest to plan in TG-ReDial, while chit-chat has lower content preservation but higher diversity in both datasets.TG-ReDial uses abstractive conversational topics and DuRecDial uses knowledge entities; the differing annotations yield different topic-prediction patterns. Recommendation responses have lower diversity scores.Source: §7.2 · pp. 18–19, Table 11 · p. 19

7.3 Case Study

  • In the case study, UniMIND responds to a Love chit-chat turn and later addresses both Love and Starry Sky in a movie recommendation.Union predicts Starry Sky prematurely in the earlier turn, while systems without goal planning tend to recommend frequently.Source: §7.3 · p. 19, Figure 3 · p. 20

7.4 Error Analysis and Limitations

  • On TG-ReDial, all systems have low recommendation success amid sparse user-item interactions, and UniMIND does not use historical interaction data.The analysis also reports a gap in Informativeness and error propagation as task inputs are generated sequentially; contrastive objectives and noisy-channel models are noted as possible mitigations.Source: §7.4 · pp. 20–21, Table 12 · p. 21

8 CONCLUSIONS

UniMIND unifies four MG-CRS tasks in a Seq2Seq framework using prompt-based learning. The authors report state-of-the-art performance on each task with one model and identify several directions for further improvement.

8 CONCLUSIONS

  • UniMIND unifies four MG-CRS tasks in a Seq2Seq paradigm and uses prompt-based learning to support multi-task learning.Source: §8 · p. 21
  • On two MG-CRS datasets, UniMIND achieves state-of-the-art performance on each task with a unified model.Source: §8 · p. 21
  • The authors identify incorporating historical user-item interaction data as a potential way to improve recommendations.Source: §8 · p. 21
  • They suggest open-domain dialogue post-training to generate more informative and diverse responses, and investigating ways to alleviate training error propagation.Source: §8 · p. 21
  • They also propose extending the prompt-based method to low-resource or few-shot MG-CRS settings.Source: §8 · p. 21
Loading 2204.06923v1…