Source-linked AI summary
Multi-domain Neural Network Language Generation for Spoken Dialogue Systems
Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Lina M. Rojas-Barahona, Pei-Hao Su, David Vandyke, Steve Young
TL;DR
Multi-domain generation must adapt to limited target-domain data, especially when unseen slot-value pairs cannot benefit from ordinary parameter sharing. The paper combines data counterfeiting with discriminative training for RNN-based generators, achieving good performance with much less training data and confirming effectiveness through human assessment.
Problem
Unseen target-domain slot-value pairs must otherwise be learned from scratch, limiting the benefit of model fine-tuning when target data is scarce.
Method
The paper trains multi-domain RNN language generators using data counterfeiting and discriminative training, with semantic input features and reading gates in the SC-LSTM.
Results
Good performance was achieved when adapting models across four dialogue domains with much less training data, and human judges confirmed the approach's effectiveness.
Takeaways & Limitations
Sharing realisations among functionally similar slot-value pairs facilitates transfer learning during domain adaptation.
Takeaways & Limitations
The method still requires a small amount of annotated data to be collected offline.
Abstract
from arXiv · showhide
Moving from limited-domain natural language generation (NLG) to open domain is difficult because the number of semantic input combinations grows exponentially with the number of domains. Therefore, it is important to leverage existing resources and exploit similarities between domains to facilitate domain adaptation. In this paper, we propose a procedure to train multi-domain, Recurrent Neural Network-based (RNN) language generators via multiple adaptation steps. In this procedure, a model is first trained on counterfeited data synthesised from an out-of-domain dataset, and then fine tuned on a small set of in-domain utterances with a discriminative objective function. Corpus-based evaluation results show that the proposed procedure can achieve competitive performance in terms of BLEU score and slot error rate while significantly reducing the data needed to train generators in new, unseen domains. In subjective testing, human judges confirm that the procedure greatly improves generator performance when only a small amount of data is available in the domain.
1 Introduction
Open-domain NLG is difficult because semantic input combinations grow rapidly across domains, motivating data-efficient adaptation that reuses similarities between domains. The paper proposes an RNN-based multi-domain training procedure using counterfeited data and discriminative training.
- Semantic input combinations grow exponentially as NLG expands across domains, making open-domain generation difficult.
- Statistical spoken dialogue systems aim to provide domain-scalable development with less human intervention than heavily handcrafted approaches.
- Prior adaptation work improved belief tracking and policy learning by training general models and adapting them to specific domains.
- Earlier adaptive NLG addressed preferences, personality, conversational alignment, and domain adaptation, but linguistically rich approaches were not readily adaptable to data.
- The proposed procedure trains multi-domain RNN generators through counterfeited out-of-domain data and discriminative training to improve efficiency with scarce in-domain data.
2 Related Work
Related work covers linguistic personalization and domain adaptation, feature- and model-based adaptation, neural language-model adaptation, and discriminative training. These studies motivate adapting data-driven NLG models while distinguishing the paper's direct generator-training objective.
- Domain Adaptation: Domain adaptation transfers knowledge from a data-rich source domain to a related target domain with little or no labeled data.It is especially relevant to speech and language applications because language usage and topics continually evolve.
- NLG Adaptation: Earlier NLG adaptation used LFG f-structure generators, while other work explored reinforcement learning, active learning, preferences, and personality traits.
- Adaptation Methods: Feature-based adaptation exploits correlations between data points, whereas model-based adaptation is particularly useful for language modeling.
- Neural LM Adaptation: Neural language-model adaptation has used feature augmentation, parameter fine-tuning, regularisation, and curriculum learning.
- Discriminative Training: Discriminative training separates correct annotations from competing incorrect annotations and has been applied to speech recognition, machine translation, and RNN language modeling.Prior RNNLM probabilities served as features for a phrase-based translation system.
3 The Neural Language Generator
The paper uses a semantically conditioned RNN language generator that consumes dialogue acts and tokens, updates recurrent states, predicts subsequent tokens, and lexicalises the generated sequence. Its SC-LSTM adds reading gates that control semantic inputs and learn their alignments with realisations.
- Model Inputs and Generation: The generator is an RNNLM augmented with semantic input features, including a dialogue act specifying the required output semantics.A dialogue act combines an action type with slot-value pairs.
- Model Inputs and Generation: At each time step, the model consumes dialogue-act and token representations, updates its state, and predicts a distribution over the next token.
- Model Inputs and Generation: The model repeatedly samples output tokens until an end-of-sentence symbol, then lexicalises slot tokens into their values to form the target utterance.Delexicalisation replaces slot values with slot tokens during model operation; lexicalisation reverses that replacement.
- SC-LSTM Architecture: SC-LSTM is a specialised LSTM extension for language generation that learns generation decisions end-to-end from paired dialogue acts and utterances.
- SC-LSTM Architecture: SC-LSTM reading gates control semantic input features and act as keyword or key-phrase detectors for aligning them with their realisations.
- Training Objective: The network is trained end-to-end with cross entropy between predicted and actual word distributions, plus regularisation on dialogue-act transition dynamics.The regularisation constants η and ξ are set to 10^-4 and 100, respectively.
4 Training Multi-domain Models
The paper adapts an SC-LSTM generator from a data-rich source domain to a data-limited target domain through fine-tuning and data counterfeiting. Counterfeiting replaces delexicalised source slots with target-domain slots of matching functional classes, enabling transfer while preserving distinctions between slot-value pairs.
- Model adaptation: The adaptation goal is to learn target-domain SC-LSTM parameters from rich source-domain and limited target-domain DA–sentence pairs.
- Model fine-tuning: Fine-tuning a source generator on target data shares language-model parameters but cannot transfer realisations for unseen target slot-value pairs.The authors report that unseen slot-value realisations must otherwise be learned from scratch.
- Data counterfeiting: Data counterfeiting categorises source and target slots by functional class, delexicalises slots and values, and randomly substitutes compatible target slots to create pseudo-target instances.The classes are informable, requestable, and binary.
- Data counterfeiting: The counterfeited generator is trained on pseudo-target data and then refined on real in-domain data to produce the final target model.
- Transfer mechanism: Sharing realisations among functionally similar slot-value pairs facilitates transfer learning while preserving distinctions between similar but distinct pairs.The approach also preserves slot-value co-occurrence statistics, allowing the model to learn gating before adaptation data is introduced.
5 Discriminative Training
The discriminative-training procedure generates candidate utterances, scores them against reference outputs, and updates the RNN generator to separate better candidates from competing alternatives. Its differentiable cost permits direct gradient-based parameter updates while combining multiple scoring functions.
- Training objective: Discriminative training generates candidate sentences for each dialogue act using the current model and separates correct outputs from competing incorrect hypotheses.
- Candidate scoring: Candidate quality is evaluated by a scoring function that compares each candidate utterance with the ground-truth utterance.
- Candidate scoring: Candidate probabilities are normalised from RNN token likelihoods, with γ controlling whether the distribution is flattened or sharpened.γ < 1 flattens the distribution, whereas γ > 1 sharpens it.
- Objective construction: The scoring function can combine several component scores, each weighted by its corresponding βj.
- Parameter updates: Because the resulting cost function is differentiable everywhere, backpropagation directly computes gradients for parameter updates.
6 Datasets
The experiments cover restaurant, hotel, laptop, and television dialogue domains, with larger laptop and TV input spaces designed to test recombination and generalisation to unseen dialogue acts. The TV adaptation evaluation compares scratch training, fine-tuning, and data counterfeiting from laptop models.
- Experimental domains: Experiments use four domains: restaurant, hotel, laptop, and television.
- Dataset construction: Laptop and TV datasets were constructed by enumerating ontology-based combinations of dialogue-act types and slots before collecting one human realisation per dialogue act.
- Adaptation evaluation: Figure 2 evaluates TV-domain adaptation from laptop models by comparing scratch training, fine-tuning, and data counterfeiting, with 10% representing approximately 700 examples.
- Dataset scale: The laptop and TV datasets contain about 13K and 7K distinct dialogue acts, respectively, creating large input spaces with sparse examples.
- Dataset purpose: The resulting datasets test whether generators can learn partial realisations, recombine concepts, and apply them to unseen dialogue acts.
7 Corpus-based Evaluation
Corpus-based evaluation compares training from scratch, model fine-tuning, data counterfeiting, and discriminative training across domain-adaptation settings using BLEU-4 and slot error rate. Data counterfeiting improves adaptation, while discriminative training further improves both evaluation metrics.
- Evaluation Metrics: BLEU-4 and slot error rate were plotted against varying amounts of adaptation data, with the graphs using a logarithmic x-axis.Slot error rate was averaged over the top five realisations, and multiple references were used for BLEU when available.
- Data Counterfeiting: 10% adaptation data corresponds to approximately 2K examples in restaurant-and-hotel-to-laptop-and-TV adaptation.Figure 3 compares the same methods used for the laptop-to-TV adaptation scenario on more disjoint domains.
- Data Counterfeiting: Data counterfeiting outperforms fine-tuning and training from scratch, improving BLEU and substantially reducing slot error rate.The reported explanation is better realisation of unseen slot-value pairs and preservation of co-occurrence statistics supporting semantic alignments.
- Discriminative Training: 10% adaptation data corresponds to approximately 700 examples in the laptop-to-TV discriminative-training evaluation.Figure 4 measures the effect of applying DT after ML adaptation.
- Discriminative Training: Discriminative training consistently improves generator performance on BLEU and slot error rate after maximum-likelihood adaptation.Slot error rate reaches zero after DT, whereas sentence-level BLEU optimisation is less direct because evaluation uses corpus BLEU.
8 Human Evaluation
Human evaluation compares generators across laptop-to-TV and TV-to-laptop adaptation scenarios using informativeness, naturalness, and pairwise preference judgments. With limited target-domain data, adaptation outperforms training from scratch, and DT is generally preferred to ML adaptation.
- Evaluation Design: Human judges rated five realisations per dialogue act for informativeness and naturalness on a 3-point scale and indicated pairwise preferences.The study covered laptop-to-TV and TV-to-laptop adaptation and compared four training conditions.
- Results: DT-10% and ML-10% outperform scr-10% when only 10% of target-domain data is available.The comparison uses adaptation with DT or ML training against training from scratch with the same limited data.
- Results: DT-10% is preferred to ML-10%, especially for informativeness, although the preference is not statistically significant in the reported laptop-to-TV case.In that scenario, DT-10% has informativeness considered indistinguishable from the full-data scrALL system.
9 Conclusion and Future Work
The paper proposes a general procedure for training multi-domain, RNN-based language generators through data counterfeiting and discriminative training. Evaluation across dialogue domains shows effective adaptation with less training data, while the method still requires offline annotated data.
- The proposed procedure combines data counterfeiting and discriminative training to adapt multi-domain, RNN-based language generators.The authors describe the procedure as general and applicable to any data-driven language generator.
- Good performance was achieved when adapting models across four dialogue domains with much less training data.
- Human judges confirmed the effectiveness of the proposed adaptation approach.
- The method requires a small amount of annotated data to be collected offline.Future work targets training with real user feedback during conversation.