Source-linked AI summary
Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
Jiasen Lu, Anitha Kannan, Jianwei Yang, Devi Parikh, Dhruv Batra
TL;DR
MLE-trained generative dialog models tend to give generic responses, while discriminative models rank candidate answers but cannot directly serve conversations without candidate lists. This paper transfers discriminator feedback to a generative visual-dialog model using differentiable sampling. On VisDial, the resulting generator improves over state-of-the-art by 2.43% on recall@5 and 2.67% on recall@10.
Problem
Method
The model transfers a discriminator's perceptual feedback to a generator using Gumbel-Softmax sequence sampling and a straight-through gradient estimator.
Source: §1 · p. 2, §4 · p. 4, §4.3 · p. 5Results
2.43% on recall@5 and 2.67% on recall@10: the model improves over state-of-the-art on VisDial.
Source: §1 · p. 2Takeaways & Limitations
Abstract
from arXiv · showhide
We present a novel training framework for neural sequence models, particularly for grounded dialog generation. The standard training paradigm for these models is maximum likelihood estimation (MLE), or minimizing the cross-entropy of the human responses. Across a variety of domains, a recurring problem with MLE trained generative neural dialog models (G) is that they tend to produce 'safe' and generic responses ("I don't know", "I can't tell"). In contrast, discriminative dialog models (D) that are trained to rank a list of candidate human responses outperform their generative counterparts; in terms of automatic metrics, diversity, and informativeness of the responses. However, D is not useful in practice since it cannot be deployed to have real conversations with users. Our work aims to achieve the best of both worlds -- the practical usefulness of G and the strong performance of D -- via knowledge transfer from D to G. Our primary contribution is an end-to-end trainable generative visual dialog model, where G receives gradients from D as a perceptual (not adversarial) loss of the sequence sampled from G. We leverage the recently proposed Gumbel-Softmax (GS) approximation to the discrete distribution -- specifically, an RNN augmented with a sequence of GS samplers, coupled with the straight-through gradient estimator to enable end-to-end differentiability. We also introduce a stronger encoder for visual dialog, and employ a self-attention mechanism for answer encoding along with a metric learning loss to aid D in better capturing semantic similarities in answer responses. Overall, our proposed model outperforms state-of-the-art on the VisDial dataset by a significant margin (2.67% on recall@10). The source code can be downloaded from https://github.com/jiasenlu/visDial.pytorch.
1 Introduction
MLE-trained generative dialog models tend toward generic responses, while discriminative models perform strongly but require candidate lists. The paper transfers discriminative feedback to a generative visual-dialog model and reports improved VisDial results.
1 Introduction
- MLE-trained models tend to produce safe, generic responses in dialog, while multimodal next-utterance distributions encourage models to favor frequent responses.The paper says these responses can disrupt conversational flow and disengage users.Source: §1 · pp. 1–2
- Reinforcement learning with sequence-level metrics such as BLEU, ROUGE, or CIDEr is presented as an alternative, but those metrics correlate poorly with human judgment in dialog [36] [27].This poor correlation renders that alternative infeasible for dialog models.Source: §1 · p. 2
- The discriminator ranks candidate responses and learns task-dependent perceptual similarities, allowing the generator to use feedback beyond maximum-likelihood training.Metric learning and self-attentive answer encoding encourage meaningful similarities, including among multiple valid responses.Source: §1 · p. 2
- The generator receives discriminator-loss gradients through a sequence of Gumbel-Softmax samplers and a straight-through estimator, enabling end-to-end training over discrete tokens.The estimator addresses the lack of useful gradients from naïvely sampled discrete symbols.Source: §1 · p. 2
- 2.43% on recall@5 and 2.67% on recall@10: the discriminator-trained generator improves over state-of-the-art on VisDial.It also outperforms the MLE-trained generator by 1.7% on recall@5 and produces more diverse and informative responses.Source: §1 · p. 2
- The proposed visual-dialog encoder maintains separate visual and textual memory banks and outperforms encoders used in prior work.The banks represent where to look in the image and what is known from dialog history.Source: §1 · p. 2
2 Related Work
The paper distinguishes its knowledge-transfer setup from prior sequence-generation GANs and situates its visual-dialog encoder among attention-based vision-language models.
2 Related Work
- Figure 1 depicts D sorting candidate responses while G aims to produce a sequence D ranks highest; multiple plausible responses can receive high scores.The figure uses an image from the COCO dataset [25].Source: Figure 1 · p. 3
- Unlike prior GAN approaches trained jointly from scratch to distinguish real from fake responses, this method starts with pretrained G and D, transfers D's knowledge to G, and keeps D fixed.The related-work discussion also connects the scoring mechanism to Energy Based GAN [54], without adversarial training.Source: §2 · p. 3
- Prior GAN work used policy-gradient rewards or Gumbel-Softmax relaxation for discrete generation [53] [6] [41] [23] [21], whereas this paper uses discriminator feedback for knowledge transfer.The paper describes its discriminator's energy as deep metric-learning-based scoring.Source: §2 · pp. 2–3
- Unlike prior visual-dialog work, which attends to dialog-history utterances but uses the whole image embedding, HCIAE localizes image regions using the history and question.This lets the encoder implicitly resolve textual coreferences and ground them in the image.Source: §2 · p. 3
3 Preliminaries: Visual Dialog
Visual dialog asks a model to answer a question using an image and the dialog so far. Generative models produce answers, whereas discriminative models rank a supplied candidate list.
3 Preliminaries: Visual Dialog
- The task input comprises an image, its caption, dialog history through round t−1, and the follow-up question at round t.The response should be valid for that question.Source: §3 · p. 3
- A generative model maximizes the log-likelihood of the ground-truth answer sequence given the encoded image, history, and question.The answer is represented as a sequence drawn from the round-t answer set.Source: §3 · p. 3
- A discriminative model receives 100 candidate answers alongside the encoded input and learns to sort them, so it cannot be tested without candidates.The candidates form the list available at round t.Source: §3 · p. 4
4 Approach: Backprop Through Discriminative Losses for Generative Training
The approach transfers discriminator feedback to a visual-dialog generator by scoring its sampled answers in D’s learned metric space. It combines question-conditioned attention over history and image features, a metric-learning discriminator, and differentiable Gumbel-Softmax sampling.
4 Approach: Backprop Through Discriminative Losses for Generative Training
- G samples answer tokens from an LSTM conditioned on a joint image-dialog embedding, and D scores those samples in its learned metric space to train G.Source: §4 · p. 4
4.1 History-Conditioned Image Attentive Encoder (HCIAE)
- HCIAE first uses the question to attend over dialog history, then uses the question and attended history to attend to image regions for a joint encoding.Source: §4.1 · p. 4
4.2 Discriminator Loss
- The discriminator uses an attention-based LSTM to encode answers and N-pair loss to rank the ground truth above alternatives while favoring answers similar to it.Source: §4.2 · p. 5
4.3 Discriminant Perceptual Loss and Knowledge Transfer from D to G
- The perceptual loss updates G so its sampled answer scores higher than the human answer under D’s learned embedding and scoring function.Source: §4.3 · p. 5
- A sequence of Gumbel-Softmax samplers with a straight-through estimator enables end-to-end gradients through generated answer tokens.The forward pass discretizes samples, while the backward pass uses the continuous relaxation.Source: §4.3 · p. 5
5 Experiments
On VisDial, HCIAE-G-DIS outperforms the prior generative state of the art and HCIAE-G-MLE, while the experiments compare encoder and discriminator variants and examine discriminator updating and response quality.
5 Experiments
- Experiments use VisDial v0.9: 83k dialogs on COCO-train and 40k COCO-val images (82k train, 1k validation, 40k test), with each question evaluated by ranking 100 candidate answers at each dialog round.G is pretrained with standard MLE for 20 epochs and D with supervised training for 30; subsequently G is trained with L_G + 0.5 L_MLE.Source: §5 · p. 6
5.1 Results and Analysis
- HCIAE-G-DIS uniformly performs best across metrics, outperforming MN-G by 2.43% on R@5.The authors attribute the result to knowledge transfer and their encoder architecture.Source: §5.1 · p. 7, Table 1 · p. 7
- HCIAE-G-DIS exceeds HCIAE-G-MLE by 1.7% on R@5, while HCIAE-G-MLE is 0.7% above MN-D, showing the encoder alone does not match transfer.HCIAE-G-MLE uses the proposed encoder with MLE; HCIAE-G-DIS adds discriminator loss to MLE.Source: §5.1 · pp. 6–7
- In the discriminative setting, HCIAE-D-NP-ATT beats prior state of the art by 2.53% on R@5; n-pair loss adds 0.85% and answer attention adds 0.4%.The compared variants use the proposed encoder, with n-pair loss and self-attentive answer encoding added in turn.Source: §5.1 · pp. 6–7, Table 2 · p. 7
5.2 Does updating discriminator help?
- Adversarial HCIAE-GAN1 reaches 32.97% R@5; updating D on only the ground-truth and generated answers degrades its previously learned structure.The authors attribute the structure loss to omitting additional incorrect options used in the pretrained discriminator.Source: §5.2 · p. 7, Table 4 · p. 8
- Structure-preserving D updates produce a small improvement in G, but their computational overhead exceeds that gain and D itself performs worse.This variant trains D with generated samples as fake answers while retaining incorrect options as negatives.Source: §5.2 · p. 7, Table 4 · p. 8
- The authors report GAN instability when short or cryptic answers match and when descriptive truths face short generated answers; they argue D needs similarity scoring, not only real/fake discrimination.Their experiments also characterize a pretrained discriminator that captures answer structure as key to transferring knowledge to G.Source: §5.2 · pp. 7–8
5.3 Qualitative Comparison
- Qualitative examples show G-DIS less often gives safe answers such as “Yes” or “I can’t tell,” instead producing more diverse, informative responses.Table 3 shows G-DIS samples with different Gumbel noise.Source: §5.3 · p. 8, Table 3 · p. 8
6 Conclusion
The paper transfers knowledge from a discriminative visual dialog model to a generative model, aiming to combine the former’s performance with the latter’s ability to converse with users. It reports a generative model that significantly outperforms state of the art.
6 Conclusion
- The proposed model transfers knowledge from a discriminative visual dialog model to a generative model, which can converse without canned candidate responses.The generative model uses a sequence of Gumbel-Softmax samplers with a straight-through gradient estimator for end-to-end differentiability.Source: §6 · p. 8
- A history-informed visual dialog encoder reasons about image attention, while self-attentive answer encoding and metric learning help the discriminator learn meaningful response structure.Source: §6 · p. 8
- The resulting generative visual dialog model significantly outperforms state of the art.Source: §6 · p. 8