Source-linked AI summary
Grounding Visual Explanations
Lisa Anne Hendricks, Ronghang Hu, Trevor Darrell, Zeynep Akata
TL;DR
Existing explanation agents may mention class-associated attributes absent from the image, undermining trust. This paper adds a phrase-critic that scores grounded candidate explanations, yielding improved fine-grained explanations, counterfactuals, and FOIL phrase correction.
Problem
Existing visual explanation agents can justify predictions with class-associated attributes that are not actually present in the image, threatening user trust.
Method
The phrase-critic grounds attribute phrases in image regions, assigns normalized relevance scores, ranks candidate explanations, and trains with flipped mismatching attributes as negatives.
Results
On CUB, the model produces more image-relevant fine-grained explanations and accurate counterfactuals; on FOIL, it detects, grounds, and corrects erroneous phrases significantly better than competing models.
Takeaways & Limitations
Grounding candidate phrases supports explanations that justify the correct class, contrast alternative classes, and correct visual-language mistakes.
Takeaways & Limitations
Failure cases include incorrect attributes despite correct noun grounding and inaccurate localization of semantically correct phrases.
Abstract
from arXiv · showhide
Existing visual explanation generating agents learn to fluently justify a class prediction. However, they may mention visual attributes which reflect a strong class prior, although the evidence may not actually be in the image. This is particularly concerning as ultimately such agents fail in building trust with human users. To overcome this limitation, we propose a phrase-critic model to refine generated candidate explanations augmented with flipped phrases which we use as negative examples while training. At inference time, our phrase-critic model takes an image and a candidate explanation as input and outputs a score indicating how well the candidate explanation is grounded in the image. Our explainable AI agent is capable of providing counter arguments for an alternative prediction, i.e. counterfactuals, along with explanations that justify the correct classification decisions. Our model improves the textual explanation quality of fine-grained classification decisions on the CUB dataset by mentioning phrases that are grounded in the image. Moreover, on the FOIL tasks, our agent detects when there is a mistake in the sentence, grounds the incorrect phrase and corrects it significantly better than other models.
1 Introduction
The paper targets explanations that are class-discriminative yet genuinely supported by image evidence. It introduces a phrase-critic to ground candidate explanations and extends the framework to counterfactual reasoning and phrase correction.
- Neural networks can generate natural-language explanations, but their decision reasoning is often hidden from users.
- Effective explanations must mention class-discriminative attributes that are also present in the image, while contrasting class-specific attributes can support counterfactual explanations.
- Class relevance and image relevance can conflict when agents mention attributes associated with a class without checking whether the image contains them.
- The phrase-critic explicitly tests image relevance by visually grounding discriminative object parts mentioned in an explanatory sentence.
- Training uses a ranking loss with flipped, mutually exclusive attributes as mismatching examples, encouraging correct attributes to receive higher scores.
- The framework produces image-relevant explanations more accurately than mean-pooled grounding scores and performs strongly on FOIL phrase-error detection and correction.
2 Related Work
Prior work motivates explanations as tools for understanding and trust, but existing textual systems do not consistently ground generated attributes in images. This paper ranks grounded explanatory phrases rather than merely visualizing important regions.
- Explainability: Psychology research describes explanations as aids for learning and understanding that depend on users’ prior knowledge and goals.
- Textual and Visual Explanation: Trust is a primary motivation for explainable intelligent systems, and explanations that state class features without image relevance may lose users’ trust.
- Textual and Visual Explanation: Earlier textual explanation systems were mainly template-based, while later models generated fine-grained justifications without ensuring that mentioned parts actually appeared in the image.
- Textual and Visual Explanation: Visual explanation studies identify important image regions, whereas this model ranks generated explanatory phrases by how well they are grounded in the image.
- Pragmatics-Oriented Language Generation: Pragmatics-oriented generation separates sentence production from ranking, selecting descriptions that fulfill goals such as distinguishing similar images.
- The phrase-critic pipeline samples generated explanations, grounds constituent nouns, and selects the sentence with the highest cumulative phrase-grounding score.
3 Visual Explanation Critic
The visual explanation critic combines discriminative explanation generation, phrase grounding, and normalized sentence scoring. It ranks candidate explanations using grounded attribute evidence while filtering out insufficiently fluent sentences.
- The system generates candidate explanations, grounds their constituent phrases, scores phrase–region pairs, and selects the highest-scoring image- and class-relevant sentence.
- Phrase-Critic: The critic maps attribute phrases, grounded regions, and localization scores to one normalized image-relevance score.
- Phrase-Critic: Positive explanations are paired with negative explanations containing mismatched attributes, and a margin-based ranking loss trains the critic to score positives higher.
- Ranking Explanations: At inference, low-fluency sentences are discarded before choosing the candidate with the highest relevance score; the fluency threshold is T = -5.
- Grounding Visual Features: The explanation generator is conditioned on both the image and predicted class, using relevance and discriminative losses to produce class-specific attributes.
- Grounding Visual Features: Attribute phrases are extracted with a rule-based chunker and localized using a Visual Genome-trained grounding model without task-specific ground-truth annotations.
- Phrase-Critic: The grounding model and critic are complementary: generation emphasizes class-discriminative attributes, while criticism checks whether those phrases are image-relevant.
4 Experiments
Experiments on CUB show that phrase-critic explanations are more accurate, class-specific, and visually grounded than baseline systems, while also supporting counterfactual explanations and strong FOIL performance.
- Baseline Comparison: On CUB, phrase-critic explanations improve both class-specific accuracy and visual grounding over the baseline explanation and grounding models.Human evaluations compare the phrase critic with the explanation generator and grounding model.
- Failure Cases: Failure cases include incorrect attributes despite correct noun grounding and semantically accurate phrases whose bounding boxes are incorrectly localized.A red fruit can cause a black beak to be predicted as red, and accurate feet descriptions can still receive off-target boxes.
- Baseline Comparison: 59.67% of phrase-critic sentences were fully correct, versus 51.49% for the baseline and 47.74% for the grounding model.A correct sentence requires all noun phrases to match the image semantically.
- Grounding Accuracy: Grounding accuracy varies by bird part: beak, head, and belly reach 95.88%, 74.06%, and 66.65%, while eye detection reaches 56.72%.Head localization has a Euclidean distance of 20.26 for the phrase critic, compared with 46.31 for and 57.56 for.
- Counterfactual Explanations: The phrase critic can produce factual explanations for the correct class and counterfactual explanations based on discriminative properties absent from the image.Examples contrast bird classes using properties such as a missing long flat bill, long pointy bill, or gray wing.
- FOIL Experiments: On FOIL tasks, the phrase critic outperforms all baselines by over 20 points on classification, over 30 points on word identification, and close to 30 points on word correction.The model adapts grounding outputs to detect image relevance, identify incorrect phrases, and correct them.
5 Conclusions
The phrase-critic measures image relevance in generated explanations by grounding visual attributes, ranking candidate sentences, and selecting explanations that are both image- and class-relevant. Experiments on CUB and FOIL show accurate grounded and counterfactual explanations, including improved detection and correction of erroneous phrases.
- The phrase-critic assigns normalized grounding scores to noun phrases, ranks candidate explanations cumulatively, and selects the best image- and class-relevant sentence.The framework first factorizes alternative explanations into visual attributes and grounds them in the image.
- On CUB, the grounding approach makes explanations refer to image-present, class-specific elements and supports accurate counterfactual explanations.Counterfactual explanations address why an image was not classified as another class.
- On FOIL tasks, the framework detects inaccurate phrases, identifies their image evidence, and corrects them significantly better than competing models.