Source-linked AI summary
Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP
Vedant Palit, Rohan Pandey, Aryaman Arora, Paul Pu Liang
TL;DR
Causal mediation analysis has been applied to unimodal language models, leaving its use for understanding vision-language mechanisms limited. This paper adapts causal tracing to BLIP’s image-conditioned visual question answering and finds that relevance is concentrated in later question-encoder and answer-decoder layers.
Problem
Causal mediation analysis has been implemented for unimodal language models, limiting its application to understanding multimodal models.
Source: §1 · p. 1Method
The authors adapt causal tracing to BLIP by corrupting image embeddings and patching clean intermediate states into corrupted runs while measuring answer-probability changes.
Source: §3.1 · pp. 2–3Results
Causal relevance is concentrated in question-encoder layer 11 for all tokens and answer-decoder layers 9–11.
Source: §4 · p. 3Takeaways & Limitations
The authors release the BLIP causal-tracing tool and code to support further vision-language mechanistic-interpretability experiments.
Source: §5 · p. 4The role of the noise factor ν remains unclear, and the authors suggest studying multi-point restoration to investigate cross-module coordination.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Mechanistic interpretability seeks to understand the neural mechanisms that enable specific behaviors in Large Language Models (LLMs) by leveraging causality-based methods. While these approaches have identified neural circuits that copy spans of text, capture factual knowledge, and more, they remain unusable for multimodal models since adapting these tools to the vision-language domain requires considerable architectural changes. In this work, we adapt a unimodal causal tracing tool to BLIP to enable the study of the neural mechanisms underlying image-conditioned text generation. We demonstrate our approach on a visual question answering dataset, highlighting the causal relevance of later layer representations for all tokens. Furthermore, we release our BLIP causal tracing tool as open source to enable further experimentation in vision-language mechanistic interpretability by the community. Our code is available at https://github.com/vedantpalit/Towards-Vision-Language-Mechanistic-Interpretability.
1. Introduction
Causal mediation analysis has been used in unimodal language models, leaving vision-language mechanisms less understood. This work adapts causal mediation analysis to BLIP to study how it performs visual question answering.
1. Introduction
- Causal mediation analysis studies how an intervention on a mediator affects a system’s outcome, but prior implementations were limited to unimodal language models.The paper frames this as a limit on understanding multimodal models.Source: §1 · p. 1
- A clean intermediate state can be patched into a corrupted-image run to measure its effect on answer probabilities.Figure 1 illustrates the intervention with a cow image and the answer “brown.”Source: Figure 1 · p. 1
- The authors adapt causal mediation analysis to BLIP to investigate how it performs visual question answering.BLIP combines image and text processing to condition answer generation.Source: §1 · p. 1, Figure 2 · p. 2
2. Related Work
Prior mechanistic-interpretability work developed causal interventions and other tools mainly for language models. Multimodal work includes probing and interaction analyses, but the cited literature had not focused on identifying causal mechanisms in vision-language models.
2. Related Work
- Causal mediation analysis measures how intervening on a mediator changes an outcome; Vig et al. applied it to study grammatical-gender bias in GPT-2 models.The cited work contrasts causal approaches with probing, which can reveal information without showing whether a model uses it [9].Source: §2 · p. 2
- Mechanistic-interpretability studies have examined transformer-module interactions, training dynamics, activation interventions, and neuron features, largely in unimodal language models.The passage cites examples across these research directions [32], [10] [29] [3] [7] [19], [12] [41] [13] [42] [8] [15], and [43] [16].Source: §2 · p. 2
- Meng et al. [27] corrupted subject-token embeddings and patched clean-run layer outputs into corrupted GPT-2 XL and GPT-NeoX runs to trace factual knowledge.The paper says its implementation follows this intervention approach most closely.Source: §2 · p. 2
- Vision-language interpretability research has used probing approaches [6], examined unimodal interactions and methods to understand their nature and degree [39] [24], and tested reasoning ability in VQA-trained transformers.A comprehensive survey covered multimodal interpretability through early 2021 [18].Source: §2 · p. 2
3. Method
The method adapts causal tracing to BLIP by corrupting image embeddings and patching clean intermediate states into corrupted runs. It measures answer-probability recovery across states and applies the approach to COCO-QA color identification, where BLIP performs best among the described task categories.
3. Method
3.1. Causal Tracing for BLIP
- All 577 image-patch embeddings are corrupted with sampled multiplicative noise; clean-run states are then patched individually into the corrupted question-encoder and answer-decoder runs.The patched states are assessed by their effect on the answer logits.Source: §3.1 · pp. 2–3
- The intervention score compares the correct answer’s probability after patching with its probability in the corrupted run, normalized against the clean-to-corrupted probability difference.The score is expected to range from 0 (no recovery) to 1 (perfect recovery); the method also averages probability differences across layer-token patches as a function of noise.Source: §3.1 · p. 3
3.2. COCO-QA Dataset
- COCO-QA contains 123,287 images and one-word answers across object, counting, color, and location categories; the study uses color identification because BLIP performs best there.The authors divide the training subset into color, location, and counting splits before assessing zero-shot performance.Source: §3.2 · p. 3
4. Results
Causal tracing locates the strongest measured relevance in the final question-encoder layer and final answer-decoder layers. The study also selects an intermediate noise level after observing how the average intervention effect varies with corruption strength.
4. Results
- As noise factor ν rises from 0.1 to 30, average Γ decays, varies little at extremely high noise, and is negative for some values; the authors choose ν = 5.The choice avoids noise so low that patching is trivial or so high that restoration is impossible.Source: §4 · p. 3, Figure 5 · p. 4
- The heatmaps show two examples averaged over 10 runs and an average across 200 COCO-QA samples, with layers indexed 0–11 and question tokens plotted vertically.Source: §4 · p. 3
- Causal relevance is concentrated at question-encoder layer 11 for all tokens and answer-decoder layers 9–11.The authors note that clean image information appears to affect the answer mainly in the final layers; their explanations for this pattern remain possibilities.Source: §4 · p. 3, Figure 4 · p. 4
5. Conclusion
The paper introduces a causal tracing tool for BLIP and applies it to a COCO-QA subset. Its conclusion highlights unresolved methodological questions and the need to identify larger vision-language mechanisms.
5. Conclusion
- The study introduces the first causal tracing tool for a vision-language model and uses it to localize BLIP performance on a COCO-QA subset.The authors open-source the code and plan to release a visualizer and adaptations to other vision-language models.Source: §5 · p. 4
- The role of noise factor ν remains unclear, including why performance does not decrease monotonically as ν increases.The authors suggest studying component-specific noise sensitivity and restoring clean image embeddings at multiple points to investigate cross-module coordination.Source: §5 · p. 4
- Identifying larger mechanisms could help explain vision-language task performance and test whether models use good algorithms or poor shortcuts on compositional-understanding benchmarks.The authors point to Winoground as an example benchmark.Source: §5 · p. 4