Source-linked AI summary
GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection
Wanwei He, Yinpei Dai, Yinhe Zheng, Yuchuan Wu, Zheng Cao, Dermot Liu, Peng Jiang, Min Yang, Fei Huang, Luo Si, Jian Sun, Yongbin Li
TL;DR
Existing pre-training methods improve dialog understanding and generation but largely neglect explicit dialog policy. GALAXY learns policy from labeled and unlabeled dialogs through semi-supervised pre-training, achieving new state-of-the-art benchmark results and stronger few-shot ability.
Problem
Existing pre-training methods mainly enhance dialog understanding and generation while neglecting explicit dialog policy.
Method
GALAXY introduces dialog-act prediction, consistency regularization, and a gating mechanism for semi-supervised pre-training on labeled and unlabeled dialogs.
Results
GALAXY achieves new state-of-the-art results on several task-oriented dialog benchmarks and outperforms existing models in various low-resource settings.
Takeaways & Limitations
GALAXY explicitly learns dialog policy while maintaining dialog understanding and generation ability.
Takeaways & Limitations
The work focuses on text-in-text-out task-oriented dialog systems and leaves spoken dialog acts for future research.
Abstract
from arXiv · showhide
Pre-trained models have proved to be powerful in enhancing task-oriented dialog systems. However, current pre-training methods mainly focus on enhancing dialog understanding and generation tasks while neglecting the exploitation of dialog policy. In this paper, we propose GALAXY, a novel pre-trained dialog model that explicitly learns dialog policy from limited labeled dialogs and large-scale unlabeled dialog corpora via semi-supervised learning. Specifically, we introduce a dialog act prediction task for policy optimization during pre-training and employ a consistency regularization term to refine the learned representation with the help of unlabeled dialogs. We also implement a gating mechanism to weigh suitable unlabeled dialog samples. Empirical results show that GALAXY substantially improves the performance of task-oriented dialog systems, and achieves new state-of-the-art results on benchmark datasets: In-Car, MultiWOZ2.0 and MultiWOZ2.1, improving their end-to-end combined scores by 2.5, 5.3 and 5.5 points, respectively. We also show that GALAXY has a stronger few-shot ability than existing models under various low-resource settings.
1 Introduction
Task-oriented dialog systems must understand users, plan dialog acts, and generate responses, but existing pre-training largely neglects explicit dialog policy. GALAXY addresses this gap through semi-supervised policy-aware pre-training using labeled and unlabeled dialogs.
- Task-oriented dialog systems successively perform understanding, policy planning, and generation to complete replies.
- Plain-text pre-training can hinder conversational knowledge, while dialog-corpus pre-training improves understanding and generation.
- Existing conversational pre-training methods largely ignore explicit dialog policy or use latent variables without external policy information.
- Large-scale policy-aware pre-training is difficult because dialog-act schemas differ, most dialogs lack labels, and naive joint training risks overfitting.
- GALAXY combines dialog-act prediction, consistency regularization, and a learnable gate to use suitable unlabeled dialogs during pre-training.
- GALAXY introduces UniDA and UnDial while achieving state-of-the-art results on several task-oriented dialog benchmarks.
2 Related Work
Related work adapts pre-trained language models to conversational and task-oriented dialog generation, while semi-supervised learning reduces dependence on dialog labels. GALAXY extends these directions by targeting explicit policy learning during dialog pre-training.
- Pre-trained language models improve dialog systems by modeling task-oriented sub-tasks in unified text sequences.
- Pre-trained conversation models adapt language models to dialog corpora or conversational objectives to improve conversational generation.
- Semi-supervised learning combines labeled and unlabeled data through approaches including generative models, pseudo-labeling, and latent-variable methods.
- GALAXY focuses on semi-supervised dialog pre-training that explicitly models dialog policy rather than relying only on general conversational objectives.
3 Pre-training Dialog Datasets
The paper develops two pre-training dialog resources: UniDA, a labeled dataset built around a unified dialog-act taxonomy, and UnDial, a large unlabeled corpus assembled from diverse sources.
- UniDA addresses limited and inconsistent dialog-act annotations by organizing labels under a unified taxonomy.
- Dialog policy is formulated as predicting dialog acts from dialog context.
- UnDial contains 35M utterances collected from 14 existing dialog corpora, including online forums and customer-service conversations.
4 Method
GALAXY combines multiple pre-training objectives with semi-supervised learning over labeled and unlabeled dialog data, then preserves dialog-act prediction during fine-tuning. Its architecture supports understanding, generation, policy prediction, and consistency regularization.
- Model architecture: GALAXY uses UniLM with shared-weight bidirectional encoding for understanding and unidirectional decoding for generation, using token, role, turn, and position embeddings.
- Pre-training objectives: The pre-training process employs response selection, response generation, dialog-act prediction, and consistency regularization.
- Pre-training objectives: Response selection models context–response coherence as binary classification, while response generation predicts the response autoregressively from dialog context.
- Pre-training objectives: Dialog-act prediction uses multi-label classification to predict the response acts from context, accommodating responses associated with multiple dialog acts.
- Pre-training objectives: Consistency regularization runs the same unlabeled context through dropout twice and aligns the resulting dialog-policy distributions with KL divergence.
- Semi-supervised pre-training: A soft entropy-based gate selects higher-quality UnDial examples for consistency regularization, while UniDA and UnDial are mixed and shuffled during pre-training.
- Fine-tuning: During fine-tuning, semantic labels can be generated with responses, and dialog-act prediction is retained to reduce the discrepancy between pre-training and fine-tuning.
5 Experimental Settings
GALAXY is evaluated on In-Car and MultiWOZ benchmarks spanning multiple task-oriented dialog domains. Evaluation uses end-to-end generation metrics and task-completion measures, including dataset-specific combined scores.
- Evaluation Datasets: In-Car covers calendar scheduling, weather information retrieval, and point-of-interest navigation in dialogs with an in-car assistant.The dataset is split into 2425 training, 302 validation, and 304 testing dialogs.
- Evaluation Datasets: MultiWOZ spans seven domains and is evaluated in both its original MultiWOZ2.0 and revised MultiWOZ2.1 versions.The processed splits contain 8438 training, 1000 validation, and 1000 testing dialogs.
- Evaluation Metrics: BLEU measures response generation, while Inform and Success assess MultiWOZ task completion and Match and SuccF1 assess In-Car performance.Combined scores are computed by combining task-completion metrics with BLEU.
6 Experimental Results
GALAXY is evaluated in end-to-end settings against published systems, pretrained conversation models, semi-supervised methods, and low-resource baselines. It achieves state-of-the-art combined scores across three benchmarks and significantly outperforms other models in low-resource settings.
- 6.1 Benchmark Performance: GALAXY achieves state-of-the-art combined scores, improving In-Car by 2.5 points, MultiWOZ2.0 by 5.3 points, and MultiWOZ2.1 by 5.5 points.The scores increase from 104.95 to 107.45, 105.05 to 110.35, and 105.25 to 110.76, respectively.
- 6.1 Benchmark Performance: GALAXY is the only model in the reported tables to obtain the best Success while maintaining BLEU at a very high level.The authors associate this result with stronger dialog policy and better response generation.
- 6.2 Comparison with Other PCMs: On MultiWOZ2.0, GALAXY outperforms UniLM and PLATO variants trained on the same pre-training datasets and fine-tuning process.The comparison attributes the weaker baseline results to their lack of dialog-policy information.
- 6.3 Comparison with Other Semi-supervised Pre-training Methods: Multi-task learning is the strongest of the compared semi-supervised baselines, while GALAXY further benefits from inferring dialog-act information on unlabeled dialogs.The VAE baseline performs worst, and GALAXY without unlabeled-dialog inference cannot fully explore stored dialog-policy knowledge.
- 6.4 Low Resource Evaluation: GALAXY significantly outperforms DAMD, SOLOIST, MinTL, PPTOD, and UBAR at 5%, 10%, 20%, and 50% training-data settings.The evaluation removes the unused MultiWOZ training data from UniDA under each setting to avoid external-data influence.
7 Analysis and Discussion
The analysis examines learning dynamics, ablations, gating, and a case study of GALAXY's dialog-policy behavior. Results indicate that consistency regularization and gating improve semi-supervised pre-training, while explicit policy learning supports successful task completion.
- Learning Curve: 10% UniDA and 100% UnDial are used to monitor testing DA F1 and LKL during simulated semi-supervised pre-training.The experiment holds out the remaining UniDA data for testing.
- Ablation Results: Table 8 reports ablation results on MultiWOZ2.0, and Table 9 lists predicted gating scores with manually annotated dialog acts.These tables support analysis of the losses and gating mechanism.
- Learning Curve: Testing DA F1 keeps increasing without degradation during pre-training, while LDA addresses the initial collapsing mode.LKL on labeled data also helps avoid over-fitting.
- Ablation Results: Comb rises from 108.11 to 110.35 when the gating mechanism is added, indicating that it filters inappropriate unlabeled data.Removing LDA causes the worst performance because of collapse; removing LKL reduces the benefits of unlabeled data.
- Case Study: GALAXY chooses correct dialog acts in the first two turns, whereas UBAR selects notify-failure and a redundant request, leading to interaction failure.The case study links early dialog-act choices with the eventual conversation outcome.
8 Conclusion
The conclusion presents GALAXY as a pre-trained conversation model that explicitly learns dialog policy through semi-supervised learning. It also highlights the unified dialog-act taxonomy, supporting corpora, and broad taxonomy coverage used by the study.
- Conclusion: GALAXY learns dialog policy explicitly during pre-training through semi-supervised learning.The paper introduces dialog-act prediction, consistency regularization, and gating of suitable unlabeled samples.
- Dialog-Act Taxonomy: The unified dialog-act taxonomy contains 20 labels spanning social convention, directive, information seeking, and information providing.The taxonomy also includes information checking and notification-related acts.
- Training Resources: UniDA and UnDial provide labeled and large-scale unlabeled dialog resources for policy-oriented pre-training.UnDial aggregates 14 Internet dialog corpora after filtering URLs, repetitions, non-English text, markup, offensive language, and non-Unicode characters.
- Downstream Representation: Downstream input sequences incorporate belief states, database results, dialog acts, and delexicalized responses as semantic labels.Special tokens concatenate these components for In-Car and MultiWOZ fine-tuning.
B.2. Implementation
The implementation uses a 12-block GALAXY model with dataset-specific fine-tuning settings and a 20-act taxonomy. The appendix also reports UnDial corpus statistics and the taxonomy figure.
- Model and Pre-training: GALAXY uses 12 transformer blocks, hidden dimension 768, and 20 dialog acts.Pre-training is initialized with UniLM and uses AdamW with an initial learning rate of 1e-5.
- Fine-tuning: Fine-tuning uses maximum sequence lengths of 1024 for context and 100 for responses.These longer limits accommodate responses containing semantic labels.
- Resources and Taxonomy: Figure 6 illustrates the unified dialog-act taxonomy, while Table 10 reports statistics for each UnDial corpus.These provide the taxonomy structure and corpus-level implementation references.
- Fine-tuning: MultiWOZ fine-tuning uses batch size 32 and dropout 0.1, while In-Car uses batch size 64 and dropout 0.35.Both settings use AdamW with an initial learning rate of 1e-4.
C.1. Other Semi-supervised Pre-training Methods.
The paper contrasts GALAXY's semi-supervised pre-training with pseudo-labeling and VAE alternatives. These alternatives either rely on self-predicted labels or model dialog acts through latent variables in labeled and unlabeled settings.
- Pseudo-Labelling: Pseudo-labeling first trains on labeled data, predicts dialog-act labels for UnDial, then retrains on labeled and pseudo-labeled data.Its initial loss combines LDA, LRS, and LRG.
- Variational Autoencoder: The VAE alternative represents dialog acts with a latent variable z shared in size with dialog act a.For unlabeled data, z is inferred from context and response during training and from context during testing.
- Variational Autoencoder: For unlabeled VAE data, the model generates response r from dialog context c and latent variable z.The generative process is expressed as pθ(r|z, c).
- Variational Autoencoder: For labeled VAE data, the model predicts dialog act a and generates response r using context c, latent variable z, and act a.The labeled process samples z using c, r, and a during training, then uses context alone at testing.
- GALAXY Pre-training: GALAXY's pre-training uses response selection, response generation, dialog-act prediction, and consistency regularization.The final semi-supervised pre-training loss is introduced after the component computations.
Appendix D
Appendix D reports total end-to-end performance under oracle belief states on MultiWOZ2.0 and MultiWOZ2.1, including results with and without pre-training.
- Table 11 reports total end-to-end performances given oracle belief states on MultiWOZ2.0 and MultiWOZ2.1.
- The table presents results from the original papers.
- “w/o pre-train” denotes initialization with UniLM’s original weights.