Source-linked AI summary
TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
Thomas Wolf, Victor Sanh, Julien Chaumond, Clement Delangue
TL;DR
Building intelligent conversational agents remains unsolved, with earlier neural systems struggling with inconsistent personality, limited dialogue memory, and generic responses. TransferTransfo applies transfer learning with a Transformer and multi-task fine-tuning to generative dialogue, significantly improving baseline systems, including a 45% absolute perplexity improvement on the private test set.
Problem
Building intelligent conversational agents remains an unsolved problem, while earlier models showed inconsistent personality, limited long-term memory, and generic responses.
Method
TransferTransfo extends transfer learning to generative open-domain dialogue using a multilayer Transformer encoder and multi-task fine-tuning.
Results
45% absolute improvement in perplexity (PPL), 46% absolute improvement in Hits@1 and 20% improvement in F1 were obtained on the private test set.
Takeaways & Limitations
Transfer learning can yield strong empirical improvements for generative open-domain dialogue, although optimal fine-tuning settings and models remain unresolved.
Takeaways & Limitations
Earlier conversational models were limited by inconsistent outputs, lack of long-term memory, and difficulty incorporating more than the last dialogue utterance.
Abstract
from arXiv · showhide
We introduce a new approach to generative data-driven dialogue systems (e.g. chatbots) called TransferTransfo which is a combination of a Transfer learning based training scheme and a high-capacity Transformer model. Fine-tuning is performed by using a multi-task objective which combines several unsupervised prediction tasks. The resulting fine-tuned model shows strong improvements over the current state-of-the-art end-to-end conversational models like memory augmented seq2seq and information-retrieval models. On the privately held PERSONA-CHAT dataset of the Conversational Intelligence Challenge 2, this approach obtains a new state-of-the-art, with respective perplexity, Hits@1 and F1 metrics of 16.28 (45 % absolute improvement), 80.7 (46 % absolute improvement) and 19.5 (20 % absolute improvement).
Introduction
Building intelligent conversational agents remains unsolved, with recurrent neural systems limited by inconsistency, weak long-term context, and generic responses. TransferTransfo targets more relevant, coherent, fluent, and personality-consistent dialogue.
- Conversational agents remain an unsolved artificial-intelligence problem despite neural models generating meaningful responses in some chit-chat settings.
- Inconsistent outputs and the lack of a consistent personality are identified as major architectural issues.
- Difficulty incorporating more than the last utterance limits these models’ long-term memory.
- Generic responses such as “I dont know” are vague and unengaging for humans.
- TransferTransfo proposes architecture, training, and generation algorithms to improve relevance, personality and dialog-history coherence, grammaticality, and fluency over traditional baselines.
Tasks and evaluation
ConvAI2 evaluates persona-conditioned open-domain dialogue using PERSONA-CHAT, combining automated task metrics with human judgments on conversational quality and persona recognition.
- ConvAI2 uses PERSONA-CHAT, where speakers condition utterances on predefined personality profiles while chatting naturally to get to know each other.
- Example conversations contain rapid topic turns as participants exchange information while learning about each other.
- Automatic evaluation remains an open question, so PERSONA-CHAT provides three automated metrics and ConvAI2 adds private-test and human evaluation.
- PPL measures gold-token likelihood, Hits@1 measures retrieving the gold response among 19 distractors, and F1 measures content-word precision and recall.
- Human evaluation scores fluency, consistency, and engagingness from 1 to 5, plus whether evaluators identify the bot’s persona.
Model
TransferTransfo uses a multilayer, decoder-only Transformer based on GPT, with masked self-attention and learned positional embeddings for autoregressive language modeling.
- The generative model is a multilayer Transformer encoder based on Radford et al.’s Generative Pre-trained Transformer.
- The model uses a 12-layer decoder-only Transformer with masked self-attention, 768-dimensional states, and 12 attention heads.
- Masked attention restricts each token to attending only to its left context.
- The implementation follows a PyTorch adaptation of the large Transformer model used in downstream NLP work.
- Learned positional embeddings support sequences up to 512 tokens, while byte-pair encoding uses a vocabulary with 40,000 merges.
Training
TransferTransfo adapts a pretrained Transformer to persona-conditioned dialogue by encoding speaker and persona states, fine-tuning with language modeling and next-utterance classification, and controlling generation through ranked beam search.
- Training: Pretraining uses BooksCorpus, a document-level collection of over 7,000 unpublished books containing about 800M words to support long-range conditioning.
- Training: Fine-tuning on PERSONA-CHAT combines an augmented input representation with a multi-task learning scheme.
- Input representation: Each input concatenates the current speaker’s 4 to 6 persona sentences with typically 3 to 5 previous dialogue utterances.
- Input representation: Dialog-state embeddings identify personality sentences and PERSON1 or PERSON2 utterances alongside pretrained word and positional embeddings.
- Multi-task learning: Next-utterance classification distinguishes the correct continuation from 2 to 6 random distractors using a classifier over the final hidden state.
- Multi-task learning: Fine-tuning combines next-utterance classification with language modeling, where the latter predicts gold next tokens through vocabulary softmax probabilities.
- Generation: Generation uses sampling with beam size 4, n-gram filtering, and a score combining length-normalized probability with next-utterance classification.
Results
TransferTransfo significantly outperforms existing systems on the public PERSONA-CHAT validation set, and these gains carry over to the privately held test set.
- 51% absolute improvement in perplexity, 35% in Hits@1, and 13% in F1 are reported on the public validation set.
- 45% absolute improvement in perplexity, 46% in Hits@1, and 20% in F1 are reported on the private test set.The private test results were scored by the ConvAI2 evaluation server after hyper-parameter tuning on validation.
Conclusion
The paper extends transfer learning from discriminative language understanding to generative open-domain dialogue and identifies effective multi-task fine-tuning setups. It also notes that further work is needed to determine optimal settings and models.
- Transfer learning from language models is extended to generative open-domain dialogue involving co-reference, common-sense knowledge, and long-range dependency modeling.
- The paper provides guidance on effective multi-task fine-tuning setups for generative dialogue tasks.
- Further work is needed to understand the most optimal settings and models.