PaperlayerView exact revision on arXiv

Source-linked AI summary

AnswerMap: Faithful Spatial Interpretability of VLMs from Answer Posteriors

Mohamed Eltahir, Fardows Adam, Duaa M. Tahir, Lama Alamoudi, Sana Ammar, Atheer A. Alboloshi, Jory Albluey, Tanveer Hussain, Naeemullah Khan

arXiv:2609.35247v1cs.CVcs.LG

TL;DR

VLM explanations can lack explicit spatial grounding, while internal read-outs and perturbation have limitations in output alignment, access, or cost. AnswerMap constructs a map from answer posteriors and applies fixed read-outs to derive answers. It passes two faithfulness tests across models and supports downstream visual tasks through multiple read-outs.

  • Problem

    Existing VLM interpretability approaches lack a spatial rationale whose cell scores are direct answers read from the output head at linear cost.

    Source: §2 · p. 4

  • Method

    AnswerMap combines yes/no posteriors for isolated row and column bands into a query-conditioned map, then applies a fixed read-out to derive an answer.

    Source: §1 · p. 2

  • Results

    AnswerMap passes two ground-truth-free faithfulness tests across four models, while attention fails both and the strongest white-box baseline trails on each.

    Source: §1 · p. 2, §7 · p. 10

  • Takeaways & Limitations

    Read-outs on the same map flag hallucinated objects, localize when the model’s own pointing fails, and fix half of its wrong answers through a crop.

    Source: §1 · p. 3, §7 · p. 10

    The operator requires a model that accepts non-square inputs, and row–column factorization can light empty crossings when a query matches two separate objects.

    Source: §7 · p. 10

Abstract

from arXiv · show

When a VLM answers a visual query, current interpretability tools rely on text rationales, which use a mismatched modality, or on internal read-outs, which originate too early to reflect the final output and require white-box access to the model. We introduce AnswerMap, a training-free, task-agnostic, black-box visual rationale constructed from the output head. The image is cut into K row and K column bands, each shown alone to the frozen model along with the query in the format of a yes/no relevance question. The outer product of the row and column ``yes'' posteriors gives the query-conditioned spatial map. Crucially, by defining a fixed read-out R (e.g., expectation, maximum) on top of AnswerMap, we can derive continuous outputs like location natively. This bypasses the reliance on discrete text tokens for continuous-output tasks and guarantees an image-dependent answer by construction. However, a rationale can be confabulated, so we validate AnswerMap across four models and three query distributions with two tests: (a) agreement with the model's own generated point and (b) deletion of the map's region. The map lands where the model points (AUC 0.85 against 0.38 for attention), and deleting its region flips 53% of correct answers (against 19% for attention's). Beyond establishing faithfulness, we demonstrate the map's task-agnostic utility through three distinct read-outs: its maximum flags hallucinated objects without generation, its expectation localizes correctly when the model's own pointing fails, and its top-mass region, fed back as a crop, fixes half of the model's wrong answers. AnswerMap thus offers a new lens on VLM interpretability and, through its read-outs, a new output interface for visual tasks beyond text tokens.

1 INTRODUCTION

AnswerMap replaces text-based or internal spatial explanations with a black-box map built from answer posteriors for isolated image bands. Its fixed read-outs support continuous, image-dependent answers, and the map passes two faithfulness tests while serving several downstream tasks.

1 INTRODUCTION

  • Text rationales name no explicit image region, while internal read-outs originate before answer generation and standard occlusion scales quadratically.Occlusion can also score a cell as unimportant when the rest of the image supplies the same evidence.Source: §1 · p. 2
  • AnswerMap asks yes/no relevance questions about isolated row and column bands, then combines their posteriors into a query-conditioned spatial map using O(K) queries.This scores each region on its own rather than measuring its removal from the full image.Source: §1 · p. 2
  • A fixed read-out of the map can derive continuous answers such as locations, with image dependence guaranteed by construction.The map’s expectation can yield a location, while its maximum can indicate grounding strength.Source: §1 · p. 2
  • Across four models, AnswerMap passes both ground-truth-free faithfulness tests, while attention fails both and the strongest white-box baseline trails on each.The tests compare the map with the model’s generated point and measure whether deleting its peak region breaks the answer.Source: §1 · p. 2
  • Three read-outs use the same map to flag hallucinated objects, localize when the model’s pointing fails, and correct prior mistakes through a targeted crop.The crop-based read-out fixes half of the model’s wrong answers, as stated in the paper context.Source: §1 · p. 3
  • The contributions include a spatial rationale from answer posteriors, map-based read-outs as an output interface, and a two-test faithfulness protocol.The protocol is described as ground-truth-free and applicable to spatial explanations of VLMs.Source: §1 · p. 3

2 RELATED WORK

Prior VLM explanation approaches differ in modality, access requirements, and computational cost, but leave a need for spatial cell scores tied directly to answers. The related work positions AnswerMap against text rationales, internal read-outs, perturbation, and answer-space probing.

Text rationales

Attention and attribution

  • Attention weights and related internal attributions do not directly tie cell scores to final answers, and attribution methods require white-box access (Jain & Wallace, 2019; Mann et al., 2026).VLM attention can emphasize sink tokens (Kang et al., 2025b), while multi-encoder architectures make gradient methods difficult to apply (Xing et al., 2025).Source: §2 · p. 3

Black-box perturbation

  • Black-box occlusion and RISE can be tested by intervention, but require one forward pass per occluder and score each cell against a context-rich full image (Zeiler & Fergus, 2014; Petsiuk et al., 2018).RISE may require 4,000 to 8,000 masked passes per map.Source: §2 · p. 4

Low-rank factorization and answer-space probing

  • GridProbe uses 2K row and column questions and an outer product to form video-frame importance maps, motivating a related factorization for images (Eltahir et al., 2026).For images, the passage frames the resulting map as the explanation itself rather than a means to select inputs.Source: §2 · p. 4

Positioning

  • The stated gap is a spatial rationale whose cell scores are direct answers, read from the output head at linear cost.The passage contrasts this need with text explanations, internal read-outs, and costly perturbation.Source: §2 · p. 4

3 ANSWERMAP

AnswerMap builds a spatial map from the model’s yes posteriors for image strips, then uses fixed read-outs to derive answers from that map. Its multigrid product combines views at different band widths, and two tests assess whether spatial explanations reflect the model’s belief and answer dependence.

3.1 WHAT THE PROBE READS

  • Each row and column band is shown alone with a yes/no relevance question, and the outer product combines their yes posteriors into a spatial map.The read-out can be the expectation, a continuous sub-cell location, or the maximum, a scalar grounding strength.Source: §3.1 · pp. 4–5

3.2 MULTIGRID PRODUCT

  • Coprime grids add distinct boundaries, and multiplying their normalized maps preserves regions endorsed across both broad-context and fine-detail views.Nested grid sizes repeat boundaries, while the product remains separable and sharpens one region rather than adding a second.Source: §3.2 · p. 5, Figure 4 · p. 5

3.3 THE TWO TESTS

  • Agreement scores the map at the model’s generated point, while deletion compares answer changes after removing the map’s region versus a same-size random region.These ground-truth-free tests assess whether the map aligns with the model’s stated spatial belief and whether the answer depends on its highlighted region.Source: §3.3 · p. 6, Figure 5 · p. 6

4 EXPERIMENTS

The experiments test AnswerMap’s agreement with model-generated points and the effect of deleting its highlighted region, across models, query types, and settings. Results report stronger deletion effects than random regions and baselines, consistent performance across model scale and family, and operation through a closed model’s API.

4.1 SETUP

  • Experiments use frozen open-weight models without training, and compare the default K=8 probe with an equal-query-cost coprime pair.Test 1 scores maps at generated points; Test 2 deletes eight high-valued cells and compares answer changes with same-size random-region controls.Source: §4.1 · pp. 6–7

4.2 TEST 1: THE MAP LANDS WHERE THE MODEL POINTS

  • The probe and equal-cost multigrid lead Test 1; occlusion trails on both metrics despite four times the queries, while attention is at or below chance.The same ordering holds across RefCOCOg, RefCOCO+, and CAVE; blank-image controls land at chance, while swapped-image maps follow the substituted image.Source: §4.2 · p. 7, Table 2 · p. 7

4.3 TEST 2: DELETING THE MAP’S REGION BREAKS THE ANSWER

  • Six times on TextVQA and five times on GQA: deleting the probe’s region breaks answers at those rates relative to uniform random regions.The gap holds under blur and on non-binary questions; occlusion trails despite four times the queries, and T-MM is below the probe.Source: Table 3 · p. 7, §4.3 · p. 8

4.4 THE RESULT HOLDS ACROSS SCALE AND FAMILY

  • The probe is stable across model scale and family without per-model tuning; attention’s best layer varies by model.On InternVL, occlusion matches the probe’s NSS but trails its AUC at four times the queries.Source: §4.4 · p. 8, Table 4 · p. 8

4.5 THE OPERATOR RUNS ON A CLOSED MODEL

  • AnswerMap runs unchanged on GPT-6-sol through its API using first-token logprobs, and multigrid narrows the map onto the query-named truck.The 8 × 8 map spreads over both trucks in the same rows, while the multigrid product narrows it; the figure reports 16 versus 58 calls.Source: §4.5 · p. 8, Figure 6 · p. 8

5 READ-OUTS AS THE OUTPUT INTERFACE

The same map supports different fixed read-outs, each evaluated against ground truth. Expectation localizes when generated pointing fails, maximum flags hallucinated objects, and a top-mass crop can correct wrong answers.

Read-out 1: expectation

  • 11 points: on Lingshu-7B, the map’s expectation outperforms the model’s generated point on RefCOCOg.The model’s pointing fails to parse on 30% of queries; on the strong pointer Qwen3-VL-4B, generation is the richer channel and the map recovers most of it.Source: §5 · p. 9

Read-out 2: maximum

  • ROC-AUC 0.833: the map’s maximum distinguishes grounded from hallucinated yes claims on POPE without labels and with 16 queries.The model answered yes 4,006 times, including 223 hallucinated claims; hallucinated yes answers also broke more often under deletion of their map region.Source: §5 · p. 9

Read-out 3: top-mass region

  • 50%: the map’s top-mass crop turns half of the model’s 42 wrong TextVQA answers right.A random crop turns 38% right; the map crop disturbs 4% of previously correct answers, versus 6% for the random crop.Source: §5 · p. 9

6 ABLATION STUDY

The ablation study finds that composing coprime grids makes agreement robust across query budgets, while product fusion outperforms mean and max.

Query budget and fusion

  • K=6 is the narrow peak for one grid, while coprime grids {3, 5} reach the same peak and stay flat across a five-fold budget range.On deletion, {3, 5} ties K=8 (52.3 against 53.4); nested {2, 4, 8} spends more queries for less, and product beats mean beats max.Source: §6 · p. 10

7 CONCLUSION

AnswerMap reads a spatial rationale from a frozen VLM’s output head, and fixed read-outs derive image-dependent answers. Its faithfulness tests and task-specific read-outs show promise, while its input and row–column construction impose limitations.

Conclusion

  • AnswerMap passes two ground-truth-free faithfulness tests across three query distributions, four models, and two families, while attention fails both and the strongest white-box baseline trails.Its three read-outs flag hallucinated objects, localize where the model’s own pointing fails, and fix half of the model’s wrong answers through a crop.Source: §7 · p. 10

Limitations and refinements

  • The operator requires models that accept non-square inputs, and row–column maps can light empty cross-cells when a query matches two separate objects.The paper proposes richer read-outs, training on the map as label-free supervision, and probe-guided generation as possible refinements.Source: §7 · p. 10

A.1 ADDITIONAL FIGURES AND TABLES

The appendix explains why attention underperforms, compares query-budget patterns, and reports the probe’s runtime relative to other methods.

Additional figures

  • The probe’s expectation tracks the model’s generated point, while best-layer attention forms a flat band because its mass sits on query-irrelevant sink tokens.The flat band barely moves with the point, corresponding to below-chance AUC in coordinates.Source: §A.1 · p. 14, Figure 7 · p. 14

Query budget curves

  • Composing coprime grids reaches the one-grid peak and stays flat across a five-fold query-budget range, whereas refining one grid peaks narrowly and collapses as bands thin.Figure 8 shows the budget curves and grey markers for fusion and independence controls.Source: §A.1 · p. 14, Figure 8 · p. 14

Cost

  • 0.4 to 1.3 seconds per map: the probe is four to five times cheaper than occlusion, though internal read-outs are cheaper when available.The probe’s case is access and faithfulness at modest cost, not raw speed.Source: §A.1 · p. 14

A.2 PROMPT TEMPLATES

The templates define band probes using a yes/no relevance question and first-token posteriors, alongside prompts for native pointing, open-ended answers, existence claims, and crop-based self-conditioning.

A.2 PROMPT TEMPLATES

  • Each of the 2K band questions asks whether the query is present in any shown tile, and the yes posterior is computed from first-answer-token logits rather than generated text.Surface forms for yes and no are aggregated by logsumexp.Source: §A.2 · p. 15
  • Native pointing prompts vary by model family: Qwen3-VL uses JSON point output, InternVL uses its documented bounding-box prompt, and other models use a generic pixel-space prompt.Generated coordinates are interpreted on a 0-to-1000 scale; InternVL box centers serve as points.Source: §A.2 · p. 15
  • Open-ended and existence queries use phrase and yes/no responses, respectively, while self-conditioning supplies the map’s top-mass crop as a second image.The close-up prompt identifies that second image as the region most relevant to the question.Source: §A.2 · pp. 15–16

A.3 QUALITATIVE EXAMPLES

The appendix shows qualitative maps across query types and reports diagnostics on deletion, agreement, map strength, attention layers, and localization heads, followed by protocol and configuration details.

A.3 QUALITATIVE EXAMPLES

  • At K=8, Qwen3-VL-4B maps showed a single mode on a landmark, a referring-expression target with a distractor, and a pathology-slide query.The examples illustrate use with a specific-domain medical query as well as general visual queries.Source: §A.3 · p. 16, Figure 9 · p. 16

A.4 ADDITIONAL DIAGNOSTICS

  • Among Qwen3-VL-4B decoder attention layers, a mid-stack band peaked at layer 15, while the last layer scored negatively against the model’s generated point.Layer 15 is the layer used for attention rows in the main tables.Source: §A.4 · p. 16, Figure 10 · p. 17
  • Probe-region deletion flipped 57.4% of image-needed questions and 22.0% of questions not needing the image, with this split holding across scales and both model families.The questions were labeled by whole-image deletion; Table 10 reports the probe-region flip rates.Source: §A.4 · p. 16, Table 10 · p. 17
  • On RefCOCOg, blank-image expectations land at the center, while swapped-image expectations lie farther from the generated point than the center prior and show no correlation.Table 11 adds expectation-to-point distance and coordinate Pearson correlation to the agreement metrics.Source: §A.4 · p. 16, Table 11 · p. 18
  • Across five configurations, higher map-maximum terciles had higher probe-region flip rates; on the 30B model, the maximum predicted flips at AUC 0.677 but not whole-image reliance (AUC 0.49).The maximum measures spatial commitment, not whether the answer required the image.Source: §A.4 · p. 17, Table 12 · p. 18
  • Without calibration, per-example localization heads performed at noise level: RefCOCOg distance was 0.375, coordinate correlation was −0.051, and deletion matched the random control.The passage relates this to the authors’ finding that uncalibrated attention maps are sparse and noisy (Kang et al., 2025a).Source: §A.4 · p. 17

A.5 CONFIGURATION DETAILS

  • Runs used one NVIDIA A100 80GB with HuggingFace Transformers; open-ended answers used normalized string match, and point parse failures counted as misses.On a 300-example RefCOCOg split, K=8 NSS was 1.14 at both tested resolutions; AUC was 0.801 at 512 pixels and 0.807 at 1024 pixels.Source: §A.5 · p. 17, Table 13 · p. 18
Loading 2609.35247v1…