Source-linked AI summary
Analyzing the Behavior of Visual Question Answering Models
Aishwarya Agrawal, Dhruv Batra, Devi Parikh
TL;DR
VQA models often perform similarly despite substantial room relative to human accuracy, creating a need to understand how they behave. The paper develops systematic analyses across representative attention-based, non-attention-based, and challenge-winning models, finding myopic, conclusion-jumping, and stubborn behaviors. These findings are presented as behavior observations shaped by model architectures and dataset statistics, not claims of human-like models.
Problem
Most VQA models cluster around 60-70% performance, motivating techniques to understand their strengths, weaknesses, and failure modes.
Method
The paper develops behavioral analysis techniques and applies them to without-attention, with-attention, and multimodal compact bilinear VQA models.
Results
VQA models are myopic on sufficiently novel instances, often converge after half the question, and do not change answers across images; attention models are less stubborn than non-attention models.
Takeaways & Limitations
Behavior analysis provides a first step toward comparing model strengths and weaknesses, diagnosing failure modes, and guiding future choices about model classes and dataset biases.
Takeaways & Limitations
The observed pathological behaviors are described as consistent with the trained architectures and dataset statistics, and the anthropomorphic labels are used only pedagogically.
Abstract
from arXiv · showhide
Recently, a number of deep-learning based models have been proposed for the task of Visual Question Answering (VQA). The performance of most models is clustered around 60-70%. In this paper we propose systematic methods to analyze the behavior of these models as a first step towards recognizing their strengths and weaknesses, and identifying the most fruitful directions for progress. We analyze two models, one each from two major classes of VQA models -- with-attention and without-attention and show the similarities and differences in the behavior of these models. We also analyze the winning entry of the VQA Challenge 2016. Our behavior analysis reveals that despite recent progress, today's VQA models are "myopic" (tend to fail on sufficiently novel instances), often "jump to conclusions" (converge on a predicted answer after 'listening' to just half the question), and are "stubborn" (do not change their answers across images).
1 Introduction
VQA models achieve clustered performance, motivating systematic behavioral analysis. The paper develops techniques to compare model strengths, weaknesses, and failure modes across representative architectures.
- VQA asks models to produce natural-language answers from an image and a natural-language question.Examples include identifying a store type or counting people in a queue.
- Most VQA methods achieve 60-70% performance, while the top-nine entries in the 2016 challenge differ by only 5%.Human performance is reported as 83% on open-ended and 91% on multiple-choice tasks.
- The paper develops behavioral analysis techniques and applies them to with-attention, without-attention, and challenge-winning VQA models.These analyses are intended to support meaningful comparisons and identify failure modes and promising directions for progress.
2 Related Work
Prior work diagnosed failures in language models, object detectors, and VQA systems. This paper extends that direction by examining VQA behavior along more specific dimensions.
- Earlier studies constructed diagnostic oracles for character-level language models and analysis tools for object detectors.
- Previous VQA analysis categorized errors involving attention regions, incorrect answers, acceptable answer differences, and incorrect labels.
- The paper instead investigates whether VQA models generalize to novel instances, process the entire question, and use image information.
3 Behavior Analyses
The paper evaluates VQA models along generalization, question understanding, and image understanding, using three representative models on the VQA validation set. The analyses show weaknesses involving novel instances, partial-question reliance, and invariant answers across images.
- Behavior Analyses: The analyses examine generalization to novel instances, complete question understanding, and complete image understanding.The study analyzes two model classes and the 2016 challenge winner on the VQA validation set.
- Generalization to novel instances: Negative accuracy-distance correlations are significant for CNN+LSTM (-0.41 at k=504) and ATT (-0.42 at k=155), but not for MCB (-0.14 at k=16).The result suggests weaker generalization to novel test QI pairs for CNN+LSTM and ATT, while MCB generalizes better in this analysis.
- Generalization to novel instances: Answer-distance correlations are negative for CNN+LSTM and ATT (-0.62) and significant for MCB (-0.47), indicating a tendency to reproduce training answers.The corresponding distance features predict 74.19% of CNN+LSTM failures, 75.41% of ATT failures, and 70.17% of MCB failures, although ground-truth labels are required.
- Complete question understanding: At half the question, responses already match the full-question answer for 40% of CNN+LSTM cases, 49% of ATT cases, and 45% of MCB cases.Half-question accuracy reaches 68% of final accuracy for CNN+LSTM, 74% for ATT, and 67% for MCB.
- Complete question understanding: Dropping wh-words changes model decisions most, whereas dropping pronouns changes them least.The analysis measures response stability after removing words grouped by part-of-speech tags.
4 Conclusion
The paper characterizes VQA models through behavioral analyses, finding myopia, premature answer commitment, and stubbornness across images. It also notes that these behaviors can be correct consequences of the architectures and training dataset.
- Behavior analysis identifies three recurring patterns: models fail on novel instances, settle after half the question, and retain answers across images.
- The authors present behavioral characterization as a first step toward understanding models, comparing strengths and weaknesses, and diagnosing failure modes.
- Attention-based models are less stubborn than non-attention-based models.
- The exposed behaviors may be correct given the model architectures and the dataset used for training.
Appendix I: Behavioral analysis for question-only and image-only VQA models
Question-only and image-only evaluations test whether VQA responses change when the complementary modality is added. Responses changed about 40% more when the question was added than when the image was added, suggesting stronger question dependence.
- The evaluation compares feeding only the question or only the image against adding the complementary modality.
- About 40% more responses changed when the question was added than when the image was added.
- The results suggest that VQA models are more heavily driven by the question than by the image.
Appendix II: Scatter plot of average distance of test instances from nearest neighbor training instances w.r.t. VQA accuracy
For CNN+LSTM, the scatter plot relates test accuracy to average distance from k-NN training points. Lower-accuracy test points tend to be farther from training points.
- Figure 7 plots CNN+LSTM test accuracy against average distance from k-NN training points.Each plotted point represents average accuracy and distance for a random subset of 25 test points.
- Lower-accuracy test points have higher average distance from k-NN training points than higher-accuracy test points.
- The accuracy-distance correlation is significant at -0.41 for k = 50.7.
Appendix III: Additional qualitative examples for “generalization to novel test instances”
Qualitative nearest-neighbor examples show that correct CNN+LSTM predictions typically have similar training QI pairs and ground-truth labels. Incorrect predictions arise when either QI pairs or training labels differ from the test case.
- Correctly answered test QI pairs have similar nearest-neighbor training QI pairs and similar ground-truth labels.
- Some errors likely occur because the test QI pair has no similar training QI pairs.The cited examples are rows 2, 4, and 5.
- Other errors likely occur because nearest-neighbor training labels differ from the test ground-truth label.The cited examples are rows 1 and 3.
Appendix IV: Analyses on “complete question understanding” for different question types
The appendix examines whether VQA models use complete questions across yes/no, number, and other categories. The analyses compare answer stability as question content is progressively revealed or POS tags are removed.
- Scope: The analyses cover question completeness and POS-tag sensitivity across yes/no, number, and other questions.The three categories are defined by answer type: binary answers, numeric answers, and all remaining questions.
- Figure measures: The figures measure answer agreement with the full question and partial-question VQA accuracy as partial question length varies.
- Partial-question analysis: ATT answers yes/no questions after only the first few words, while CNN+LSTM converges later after at least 35% of the question.ATT reaches final accuracy from the first few words; CNN+LSTM reaches as much as its final accuracy after convergence.
- Partial-question analysis: For number and other questions, ATT and CNN+LSTM show similar convergence trends as question length increases.
- Figure measures: Figures 13–15 measure how often answers remain unchanged as POS tags are dropped for yes/no, number, and other questions.
- POS-tag analysis: VQA models are most sensitive to adjectives in yes/no questions, while sensitivity differs by model for number questions and centers on nouns for other questions.For number questions, CNN+LSTM is most sensitive to adjectives and ATT to wh-words; both models are most sensitive to nouns for other questions.
Appendix V: Additional qualitative examples for “complete question understanding”
Qualitative examples show that CNN+LSTM can settle on an answer before processing the entire question, with both correct and incorrect outcomes.
- Qualitative examples: CNN+LSTM converges without the entire question and is correct for some image-question pairs but incorrect for others.
Appendix VI: Analyses on “complete image understanding” for different question types
The appendix evaluates complete image understanding by testing whether models change answers across images for the same question. It compares this answer stability across question categories and models.
- Cross-image stability: Figures 17–19 show answer consistency across images for yes/no, number, and other questions.The histograms measure the percentage of images receiving the same answer for a question, while cumulative plots summarize questions stable across at least a given image percentage.
- Model comparison: ATT is more stubborn than CNN+LSTM for yes/no questions but less stubborn for number questions.Here, stubbornness means producing the same answer across images for a given question.
Appendix VII: Additional qualitative examples for “complete image understanding”
Additional examples examine cases where CNN+LSTM gives the same answer for at least half of the images associated with a question.
- Qualitative examples: The examples pair cross-image answer repetition with the accuracy achieved on those image-question pairs.