PaperlayerView exact revision on arXiv

Source-linked AI summary

PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning

Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhen Guo, Zhibin Liu, Xinchao Xu

arXiv:2006.16779v4cs.CL

TL;DR

Open-domain dialogue requires modeling multiple appropriate responses while maintaining response quality and coherence. PLATO-2 addresses this with curriculum learning: a coarse one-to-one generator is followed by latent-variable generation and evaluation models. It achieves substantial improvements and state-of-the-art results across Chinese and English evaluations.

  • Problem

    Open-domain dialogue involves a one-to-many mapping in which one context can have multiple appropriate responses, while directly scaling such modeling causes training instability and efficiency issues.

    Source: §1 · p. 1

  • Method

    PLATO-2 trains a coarse one-to-one generation model first, then trains a latent-variable fine-grained generator and an evaluation model for diverse responses and coherence estimation.

    Source: §2.1.3 · p. 3, §5 · p. 9

  • Results

    PLATO-2 achieves substantial improvements over state-of-the-art methods in both Chinese and English evaluations, including new state-of-the-art results.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Curriculum learning supports scaling PLATO-2 to billions of parameters and adapting the two-stage models to multiple conversational tasks.

    Source: §1 · p. 2, §3.5.3 · p. 8

Abstract

from arXiv · show

To build a high-quality open-domain chatbot, we introduce the effective training process of PLATO-2 via curriculum learning. There are two stages involved in the learning process. In the first stage, a coarse-grained generation model is trained to learn response generation under the simplified framework of one-to-one mapping. In the second stage, a fine-grained generative model augmented with latent variables and an evaluation model are further trained to generate diverse responses and to select the best response, respectively. PLATO-2 was trained on both Chinese and English data, whose effectiveness and superiority are verified through comprehensive evaluations, achieving new state-of-the-art results.

1 Introduction

PLATO-2 scales PLATO through curriculum learning, progressing from one-to-one response generation to latent-variable modeling and response evaluation. Trained in Chinese and English, it achieves state-of-the-art performance across comprehensive evaluations.

1 Introduction

  • PLATO-2 addresses the instability and efficiency difficulties encountered when directly scaling PLATO's one-to-many dialogue modeling.PLATO models one-to-many relationships with discrete latent variables, but learning this relationship from scratch complicates direct scaling.Source: §1 · p. 1
  • PLATO-2 uses curriculum learning to progress from coarse-grained one-to-one response generation to fine-grained one-to-many modeling.The second stage adds a latent-variable generation model for diverse responses and an evaluation model for response coherence.Source: §1 · pp. 1–2
  • PLATO-2 achieves new state-of-the-art results and adapts to task-oriented and knowledge-grounded dialogue.The two-stage framework enabled scaling to billions of parameters and benefited multiple conversational tasks.Source: §1 · p. 2
  • Comprehensive experiments show that PLATO-2 outperforms Meena, Blender, and other state-of-the-art models on Chinese and English evaluations.The released models include English and Chinese variants trained with different model sizes and large-scale conversation data.Source: §1 · p. 2

2 Methodology

PLATO-2 uses curriculum learning to progress from coarse one-to-one response generation to fine-grained one-to-many generation and response evaluation. Its unified transformer architecture supports diverse candidate generation and coherence-based response selection.

2 Methodology

  • PLATO-2 trains a coarse-grained baseline first, then fine-grained generation and evaluation models in a two-stage curriculum.Stage 1 learns general response generation under one-to-one mapping; stage 2 models one-to-many generation and response coherence.Source: §2.1 · p. 2, §2.1.1 · p. 3, §2.1.2 · p. 3
  • The fine-grained generator introduces a K-way discrete latent variable representing response speech acts and jointly learns latent-act recognition and response generation.The model estimates p(z|c,r) and generates responses with p(r|c,z); Gumbel-Softmax approximates nondifferentiable sampling.Source: §2.1.2 · p. 3
  • The fine-grained generation objective combines autoregressive response modeling with a bag-of-words loss that encourages latent variables to capture global response information.The bag-of-words loss discards word order while predicting target-response words non-autoregressively.Source: §2.1.2 · p. 3
  • The evaluation model estimates bidirectional context-response coherence and selects the candidate with the highest coherence value.Candidates are generated under each latent value, then ranked by p(l_r=1|c,r).Source: §2.1.3 · pp. 3–4, §2.2 · p. 4
  • The evaluation network additionally trains a masked language model, randomly masking 15% of input tokens and recovering them to maintain distributed representations.The inputs include dialogue context and responses, with masked tokens predicted from the remaining tokens.Source: §2.1.3 · p. 4

3 Experiments

The experiments compare PLATO-2 with established dialogue systems using English and Chinese data, across automatic, human, interactive, and static evaluations. Results show strong performance for PLATO-2, including advantages over Blender and XiaoIce in the reported evaluations.

Evaluation Metrics

  • Automatic evaluation measures lexical diversity with distinct-1/2, while human evaluation scores coherence, informativeness, engagingness, and humanness on a 0–2 scale.The experiments include both static and interactive evaluations, with self-chat logs assessed by crowd workers.Source: §3.3.2 · p. 5, §3.4 · p. 5, §3.4.1 · p. 5

Experimental Setup

  • The experiments compare models across three groups matched by parameter scale and training data, including PLATO, DialoGPT, Blender, and multiple PLATO-2 sizes.The groups compare similar-scale systems, while the largest group contrasts Blender 2.7B with PLATO-2 1.6B.Source: §3.3.1 · p. 5, §3.4.1 · p. 6

Self-Chat Evaluation

  • PLATO-2 achieves the best performance across human and automatic self-chat evaluations, outperforming Blender in the largest comparison group.The comparison also reports stronger lexical diversity than Blender according to distinct-1/2.Source: §3.4.1 · p. 6

Chinese Interactive Evaluation

  • PLATO-2 achieves significant improvements over XiaoIce across all human evaluation metrics in Chinese interactive evaluation.XiaoIce obtains higher distinct values, which may reflect retrieval-based response generation.Source: §3.4.2 · p. 7

3.5 Discussions

The discussion contrasts Blender’s rapid topic switching with PLATO-2’s sustained, in-depth discussions, and attributes the difference to diverse generation plus coherent response selection. Further evaluations associate the two-stage curriculum with strong performance across conversational tasks.

3.5.1 Case Analysis

  • Blender tends to switch topics quickly, whereas PLATO-2 can maintain the original topic for deeper discussion.The differing discourse styles are illustrated through self-chat examples; Blender’s style may relate to BST fine-tuning data.Source: §3.5.1 · pp. 7–8
  • PLATO-2 sustains the start topic and conducts more in-depth discussions than Blender.The comparison is reported in the context of Table 4’s start-topic evaluation.Source: §3.5.1 · p. 8, Table 4 · p. 8
  • PLATO-2’s diverse responses and evaluation model are identified as reasons it generates coherent, topic-focused discussions.The discrete latent variable models one-to-many responses, while the evaluation model selects a coherent response.Source: §3.5.1 · p. 8, §3.5.2 · p. 8

3.5.2 Ablation Study

  • Table 5 evaluates the separate effects of PLATO-2’s two-stage models and supports the effectiveness of curriculum learning.The analysis separates the fine-grained generation and evaluation components within the curriculum framework.Source: §3.5.2 · p. 8, Table 5 · p. 8

3.5.3 Further Exploration of PLATO-2

  • PLATO-2 achieved first place in DSTC9 tasks covering open-domain, knowledge-grounded, and task-oriented conversation.The authors present this as evidence that the two-stage framework adapts across conversational tasks.Source: §3.5.3 · p. 8

4 Related Work

Related work covers large-scale pretrained language models and their application to open-domain dialogue generation. GPT-2 scaled transformer language modeling substantially, while Megatron-LM demonstrated even larger model training through model parallelism.

Large-scale Language Models

  • The related-work discussion situates open-domain dialogue generation within broader progress in large-scale language models.GPT and BERT are presented as representative unidirectional and bidirectional pretrained models.Source: §4 · pp. 8–9
  • GPT-2 extended pretrained language-model scale from 117M to 1.5B parameters.The work modified pre-normalization and weight initialization to support this scale increase.Source: §4 · p. 9
  • Megatron-LM trained an 8.3B-parameter model on 512 GPUs to address memory constraints.It used model parallelism for large-scale training.Source: §4 · p. 9

5 Conclusion

PLATO-2 uses curriculum learning to train coarse-grained generation first, then fine-grained generation and evaluation models. Experiments report substantial improvements over state-of-the-art methods in both Chinese and English evaluations.

5 Conclusion

  • PLATO-2’s two-stage curriculum trains general response generation before diverse generation and response-coherence estimation.The first stage uses one-to-one mapping, while the second trains fine-grained generation and evaluation models.Source: §5 · p. 9
  • PLATO-2 achieved substantial improvements over state-of-the-art methods in both Chinese and English evaluations.The conclusion reports this outcome across the paper’s evaluations.Source: §5 · p. 9

A Data Cleaning Process

PLATO-2 constructs training examples from social-media message trees and removes messages or subtrees that fail length, formatting, duplication, content, or source-quality checks.

A Data Cleaning Process

  • Each conversation path from a message-tree root to a node becomes a training sample, with prior turns as context and the node as response.The data consist of English and Chinese open-domain social-media conversations.Source: A Data Cleaning Process · p. 11
  • Messages are removed when token length is outside the 2–128 BPE-token range.This is one of the explicit cleaning conditions.Source: A Data Cleaning Process · p. 11
  • Cleaning removes messages with long words or texts, low alphabetic content, URLs, special strings, parent-text overlap, or excessive repetition.These conditions target malformed, redundant, or unsuitable messages.Source: A Data Cleaning Process · p. 11
  • Cleaning also removes offensive messages, quarantined subreddits, and messages authored by known bots.These filters constrain source and content quality in the English training data.Source: A Data Cleaning Process · p. 11

B Training Configurations

PLATO-2 is provided in three model sizes, with configurations differing in parameter count, transformer depth, attention heads, and embedding dimension.

C Chinese Case Analysis

The Chinese case analysis compares human-bot interactions from Microsoft XiaoIce and PLATO-2 using chat examples shown in Figure 4.

D Scoring Criteria in Human Evaluation

The paper reports the criteria and score details used for human evaluation, with the criteria provided in Table 7.

E Response Selection Comparison

The response-selection experiments construct an annotated Chinese dataset and compare distinct scoring functions using ranking metrics.

Related papers

Loading 2006.16779v4…