Source-linked AI summary
Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization
Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, Bill Dolan
TL;DR
Neural conversational models struggle to produce responses that are both informative and diverse. AIM combines adversarial distribution matching with variational mutual-information optimization, and experiments show more informative and diverse responses than existing methods. The paper extends AIM to DAIM with dual forward-backward objectives, while noting limitations in backward-model learning and evaluation metrics.
Problem
Neural conversational models tend to generate safe but bland responses, and informativeness and diversity remain distinct goals that must be balanced.
Method
AIM combines adversarial training for diversity with explicit optimization of a variational lower bound on mutual information, using a backward proposal network.
Results
The methods produced more informative and diverse responses than existing methods across two real-world datasets.
Takeaways & Limitations
DAIM extends AIM with dual forward-backward learning, and the reported results support jointly targeting informativeness and diversity.
Takeaways & Limitations
Learning an appropriate backward proposal network is difficult because it may itself generate bland sources, and distributional discrepancy was left for future evaluation.
Abstract
from arXiv · showhide
Responses generated by neural conversational models tend to lack informativeness and diversity. We present Adversarial Information Maximization (AIM), an adversarial learning strategy that addresses these two related but distinct problems. To foster response diversity, we leverage adversarial training that allows distributional matching of synthetic and real responses. To improve informativeness, our framework explicitly optimizes a variational lower bound on pairwise mutual information between query and response. Empirical results from automatic and human evaluations demonstrate that our methods significantly boost informativeness and diversity.
1 Introduction
Neural conversational models often generate safe but bland responses, reflecting distinct challenges of informativeness and diversity. AIM addresses these challenges by combining adversarial training for diversity with variational mutual-information maximization for informativeness.
- Maximum-likelihood training can average over training responses, producing safe but bland conversational outputs.
- Informativeness and diversity are distinct goals: responses can be varied but uninformative or informative but repetitive.
- Earlier mutual-information approaches conflated informativeness and diversity, while GAN objectives mainly discourage responses with less variation than human responses.
- AIM combines adversarial training to encourage diversity with an explicitly maximized variational information objective to produce informative responses.
- AIM uses a backward model that generates the source from the target and guides the forward model toward relevant responses during training.
- The method also uses an embedding-based discriminator, deterministic policy gradients, discrete approximation, and dual forward-backward adversarial objectives.
2 Method
AIM combines adversarial distribution matching with variational mutual-information maximization to generate responses that are both diverse and specific to their queries. Its architecture uses conditional CNN-LSTM generation, embedding-based discrimination, and a dual forward-backward objective.
- Model overview: AIM learns pθ(T|S) with adversarial training for diversity and variational information maximization for informativeness.The adversarial component matches real and synthetic query-response distributions, while the mutual-information term favors responses specific to the source.
- Information-promoting objective: The backward proposal network qφ(S|T) estimates a variational mutual-information lower bound, but learning it can be difficult because it may also generate bland sources.The dual objective addresses this issue by treating source and target symmetrically through decoder joint-distribution matching.
- Model overview: The AIM objective combines the adversarial loss LGAN and mutual-information regularizer LMI, weighted by hyperparameter λ.The objective is LAIM(θ, φ, ψ) = LGAN(θ, ψ) + λ · LMI(θ, φ).
- Diversity-encouraging objective: The conditional generator encodes the source with a CNN, adds Gaussian noise, and decodes responses with LSTM units using deterministic soft-argmax generation.Noise is injected to promote diversity, while clamping randomness into Z enables deterministic gradient propagation through the generator.
- Diversity-encouraging objective: The discriminator projects source, synthetic response, and human response into embeddings and compares source-response cosine similarities.Synthetic and human responses share an embedding network, while the source uses a separate CNN encoder.
- Diversity-encouraging objective: AIM uses deterministic policy gradients and discrete approximation to backpropagate discriminator signals while reducing gradient-estimation variance.Because randomness is separated into the global noise vector Z, gradients propagate through deterministic nodes rather than stochastic sampling nodes.
3 Related Work
The paper distinguishes informativeness from diversity and combines mutual-information optimization with adversarial training to target both properties explicitly. AIM extends prior adversarial and information-based approaches, while DAIM adds a dual-learning component with VIMO.
- Unlike prior work that used an approximate MI objective only during decoding, AIM optimizes a principled variational MI lower bound during training.
- AIM differs from earlier adversarial response generation in its embedding-based structured discriminator and generator design.
- Earlier information-promoting work conflated informative and diverse responses, whereas AIM treats them as distinct objectives.
- AIM combines GAN-based adversarial learning for response diversity with mutual-information optimization for informativeness.
- DAIM extends AIM with a VIMO-based dual objective that supports communication between forward and backward models.
4 Experiments
Experiments on Reddit and Twitter evaluate relevance and diversity using automatic metrics, alongside human judgments of informativeness and relevance on Reddit. Adversarial and mutual-information objectives improve diversity and can offset relevance trade-offs, while DAIM achieves stronger diversity than AIM at higher computational cost.
- Setups: Relevance is measured with BLEU, ROUGE, and embedding-based metrics, while diversity uses Dist-n and the proposed Ent-n metrics.Ent-n accounts for how evenly n-gram frequencies are distributed, unlike Dist-n alone.
- Setups: The study evaluates cGAN, AIM, DAIM, seq2seq, and MMI on Reddit and Twitter datasets.Reddit contains 2 million source-target pairs, while Twitter contains 7 million single-turn conversations.
- Reddit evaluation: Adversarial loss improves generated-response diversity over seq2seq, while most relevance metrics increase slightly except BLEU.
- Reddit evaluation: DAIM outperforms AIM in diversity, better approximating human responses, but requires around 3.5 times the seq2seq computation time.cGAN and AIM require around 1.7 and 2.5 times the baseline computation time, respectively.
- Limitations: The authors identify distributional discrepancy between generated and ground-truth responses as a more reasonable future evaluation metric than single-response judgment.
- Reddit evaluation: Human judges significantly prefer DAIM over MMI for informativeness (p < 0.00001), while relevance judgments are on par.The evaluation used 600 sampled sources, with seven judges ranking each paired comparison.
- Twitter evaluation: Adversarial training increases diversity at the cost of slightly lower relevance scores, which may reflect utterance-pair discrepancy metrics favoring bland outputs.
- Twitter evaluation: The additional MI term appears to compensate for relevance decreases and improves diversity, especially for Dist-n and Ent-n at larger n.
5 Conclusion
AIM and its dual-objective extension promote informative and diverse dialogue, outperforming existing methods on two real-world datasets.
- AIM trains response generation models to promote informative and diverse conversations between humans and dialogue agents.
- AIM directly optimizes a lower bound of the MMI objective during training, unlike MMI reranking during decoding.
- Evaluations on two real-world datasets show that AIM methods produce more informative and diverse responses than existing methods.