PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Visual Attribution for Chest X-ray Reasoning in Large Vision Language Models

Guangzhi Xiong, Qiao Jin, Sanchit Sinha, Zhiyong Lu, Aidong Zhang

arXiv:2605.20158v1cs.CVcs.AIcs.CL

TL;DR

The paper addresses the lack of objective evidence for evaluating whether LVLM attributions reflect the visual regions used in medical predictions. It builds a causally filtered CXR benchmark and introduces MedFocus, finding that prior methods often fail while MedFocus substantially outperforms them across evaluated settings. The work takes a step toward more trustworthy medical LVLM reasoning.

  • Problem

    Objective ground truth for the visual evidence underlying LVLM decisions is generally unavailable, making attribution faithfulness difficult to verify in high-stakes medical applications.

  • Method

    The paper builds MedGround-Bench by retaining CXR-VQA samples whose expert-annotated regions are causally validated through counterfactual editing, then proposes MedFocus for concept-based causal attribution.

  • Results

    Across 11 attribution methods, six LVLMs, and two output modes, existing methods often fail to identify prediction-driving evidence, whereas MedFocus substantially outperforms prior methods.

  • Takeaways & Limitations

    Clinically named, causally measured attributions can provide more faithful and interpretable explanations for medical LVLM reasoning.

  • Takeaways & Limitations

    The evaluation focuses on CXR because other modalities currently lack both expert spatial annotations and publicly available region-localized counterfactual editing models.

Abstract

from arXiv · show

Large Vision Language Models (LVLMs) show promise in medical applications, but their inability to faithfully ground responses in visual evidence raises serious concerns about clinical trustworthiness. While visual attribution methods are widely used to explain LVLM predictions, whether these explanations actually reflect the visual evidence underlying the model's decision is largely unverified, since ground-truth annotations for internal model reasoning are typically unavailable. We address this question for chest X-ray (CXR) reasoning by developing a causal evaluation framework that retains only CXR-VQA samples for which the expert-annotated region is verified, via counterfactual editing, to be causally responsible for the model's prediction. Using this framework across 11 attribution methods, six open-source LVLMs, and two output modes (direct answer and step-by-step reasoning), we find that existing attribution methods often fail to identify the evidence used by LVLMs. To address this failure, we propose MedFocus, a concept-based attribution method that localizes clinically meaningful anatomical regions via unbalanced optimal transport and measures their causal effect on model outputs through targeted interventions. MedFocus produces spatial, concept-level, and token-level attributions and substantially outperforms prior methods, taking a step toward more trustworthy attribution for medical LVLMs. Our data and code are available at https://github.com/gzxiong/medfocus/.

1 Introduction

The paper argues that faithful visual attribution is essential for trustworthy medical LVLM deployment but remains difficult to evaluate because objective ground truth for model-used evidence is lacking. It introduces a causal CXR evaluation framework and MedFocus to address this gap.

  • Reliable attribution is needed for clinician trust, error detection, and patient safety as LVLMs enter high-stakes medical applications.
  • Existing attribution methods lack reliable ground truth for determining whether identified regions truly support black-box LVLM outputs.Human annotations may be subjective or misaligned with the model’s internal reasoning, hindering rigorous method comparison.
  • The causal framework filters CXR-VQA samples using counterfactual editing to retain cases where expert-annotated regions are verified as causally responsible for predictions.The resulting MedGround-Bench contains 3940 samples across six LVLMs and two output modes.
  • Across 11 attribution methods, six LVLMs, three CXR datasets, and two reasoning modes, existing methods consistently fail to faithfully identify prediction-driving visual evidence.
  • MedFocus grounds attribution in clinically meaningful anatomical regions, measures their influence through targeted interventions, and produces spatial, concept-level, and token-level outputs.It substantially outperforms prior methods and is designed to make explanations more faithful and clinically interpretable.
  • The authors release MedGround-Bench as a causally validated CXR-VQA suite for future attribution evaluation.

2 Related Work

Related work spans several attribution paradigms and grounding benchmarks, but existing resources generally assess localization against annotations rather than whether highlighted evidence drives a model’s prediction. Causal and concept-based interpretability provide the methodological foundations for addressing this distinction.

  • LVLM attribution methods include gradient-based saliency, attention-based aggregation, perturbation-based occlusion, and prompting-based grounding.
  • General and medical grounding benchmarks evaluate localization against natural-language or expert spatial annotations, but not necessarily faithfulness to the evidence driving model predictions.
  • A correct model answer can rely on spurious cues outside the expert-annotated region, separating localization accuracy from attribution faithfulness.
  • Causal interpretability identifies prediction-driving features through counterfactual interventions, while concept-based methods connect low-level representations to human-understandable concepts.

3 A Causal Framework for Evaluating CXR Attribution Faithfulness

The framework constructs a causally validated CXR attribution benchmark by filtering VQA samples with correctness and foreground/background counterfactual tests. It supports evaluation across multiple LVLMs and output modes while retaining only samples with verified causal links between annotated regions and predictions.

  • MedGround-Bench starts from CXR VQA data with expert-annotated regions and retains samples whose annotated regions are verified through counterfactual editing to drive predictions.
  • The benchmark reformulates annotations from three spatially grounded CXR datasets into binary VQA questions about clinical attributes.The source datasets are ImaGenome, VinDR-CXR, and PadChest-GR.
  • The three-step filter first keeps correctly answered questions, then retains cases where foreground editing flips the answer, and finally requires background editing to leave it unchanged.
  • The resulting evaluation set provides reliable attribution ground truth by verifying a causal link between each annotated region and the model prediction.
  • The framework evaluates direct-answer and step-by-step reasoning modes across six open-source LVLMs spanning generalist and medical model families.

4 MedFocus: Concept-based Causal Attribution for Medical Reasoning

MedFocus attributes medical LVLM reasoning by mapping clinically meaningful anatomical concepts from a normal reference CXR to the target image, then measuring each concept’s causal effect through targeted interventions. It reports spatial, concept-level, and token-level explanations while handling jointly relevant regions and predictions not driven by predefined concepts.

  • Concept-based attribution outputs: MedFocus segments clinically meaningful anatomical regions and uses their causal influence on model outputs to produce three complementary attribution forms.Spatial attribution uses the most causally important region’s bounding box, concept-level attribution names the anatomy, and token-level attribution reports contributions over reasoning outputs.
  • Concept Segmentation via Unbalanced Optimal Transport: Unbalanced optimal transport maps annotated concepts from a normal reference CXR to a target image while relaxing marginal constraints for pathological distribution changes.The method uses UOT because abnormalities such as pleural effusion and cardiomegaly make the mapping inherently unbalanced.
  • Concept Segmentation via Unbalanced Optimal Transport: MedFocus refines transferred concept regions with MedSAM after computing each concept’s tight bounding box.The refinement addresses noisy UOT boundaries before causal intervention.
  • Causal Attribution via Concept Intervention: Bounding-box masking removes a concept and its surrounding context, providing a stronger attribution signal than pixel-level masking or generative counterfactual editing.The ablations reported in the method support using bounding boxes for cleaner causal measurement.
  • Causal Attribution via Concept Intervention: For each concept, MedFocus measures cumulative drops in token log-probabilities under the counterfactual, with larger drops indicating stronger causal contributions.Conditioning on the original output avoids regeneration and sampling noise, while max(0, ·) retains only probability decreases.
  • Causal Attribution via Concept Intervention: The method evaluates clinically meaningful composite groups and defaults to the entire image when no predefined concept exceeds the relevance threshold.Composite groups capture jointly used regions, while thresholding handles predictions not driven by a single anatomical concept.

5 Experiments

Experiments show that existing attribution methods often fail to faithfully localize evidence in direct and reasoning settings, while MedFocus provides stronger localization and concept-level explanations. Ablations support its two-stage segmentation, UOT localization, MedSAM refinement, and bounding-box zero-masking intervention choices.

  • 5.1 Attribution Evaluation with MedGround-Bench: No existing attribution method achieves consistently faithful attribution on MedGround-Bench-Direct.The evaluation averages IoU, F1, Precision, and Recall across models.
  • 5.1 Attribution Evaluation with MedGround-Bench: GradCAM and Integrated Gradients perform poorly, while Gradient-weighted Attention combines near-perfect recall with very low precision.Baselines either produce diffuse maps with low precision or focused maps that miss the true evidence.
  • 5.1 Attribution Evaluation with MedGround-Bench: 52.95% IoU on ImaGenome: MedFocus maintains strong attribution quality in the reasoning setting.GradCAM++ drops from 30.54% to 23.70% IoU on ImaGenome, whereas MedFocus remains robust to multi-step reasoning.
  • 5.2 Qualitative Comparison: MedFocus produces tighter localization than baselines across examples involving lobar or segmental collapse, interstitial lung disease, and cardiomegaly.Baselines often generate diffuse or misplaced regions with weak overlap with annotated evidence.
  • 5.2 Qualitative Comparison: MedFocus tracks different anatomical concepts across reasoning tokens, progressing from broad lung context to cardiac-silhouette evidence for the final diagnosis.This provides token-level attribution rather than a single diffuse heatmap for the entire reasoning chain.
  • 5.3 LVLM Attribution across Models and Sample Groups: IoU improves from G1 to G3 while failure rates decrease as samples become more causally grounded.The filtering pipeline removes cases relying on irrelevant cues or producing correct answers for the wrong reasons; reasoning-mode failure rates are often near zero on G3.
  • 5.3 LVLM Attribution across Models and Sample Groups: MedGemma1.5-4B and MedGemma-4B show stronger G3 attribution than generalist Qwen2.5-VL and Gemma3 models, especially in reasoning mode.The results also report better attribution with larger models within the same family.
  • 5.4 Ablation Studies: UOT localization with MedSAM refinement achieves a better precision-recall balance and higher IoU and F1 than alternative localization designs.End-to-end segmentation variants yield lower IoU and F1, while Grounding-DINO provides high recall but low precision.

6 Conclusion

The paper introduces MedGround-Bench as a causal framework for evaluating visual attribution faithfulness in chest X-ray reasoning. It finds that existing methods often miss prediction-driving evidence and presents MedFocus as a more faithful and interpretable alternative across models, datasets, and output modes.

  • MedGround-Bench provides a causally validated benchmark for evaluating visual attribution faithfulness in chest X-ray reasoning.
  • Existing attention-, gradient-, prompting-, and perturbation-based methods often fail to identify the visual evidence driving medical LVLM predictions.
  • MedFocus grounds explanations in clinically meaningful anatomical regions and measures their influence through targeted interventions.It produces spatial, concept-level, and token-level attributions across multiple LVLMs, datasets, and output modes.

A.1 Limitations

The study’s scope is bounded by CXR data, binary VQA, and attribution evaluation on correctly answered samples. Broader modalities, richer clinical tasks, incorrect predictions, and subgroup generalization remain open directions or cautions.

  • Scope: The evaluation focuses on CXR because expert spatial annotations and region-localized counterfactual editing are publicly available for this modality.The construction recipe is modality-agnostic but depends on analogous annotations and editing tools for extension.
  • Task formulation: Binary visual questions provide a controlled correctness test but do not cover full report generation or multi-step diagnostic reasoning.These richer tasks require validation tools beyond binary correctness checking.
  • Evaluation boundary: Attribution faithfulness is evaluated only on correctly answered samples because reliable ground-truth attribution is ambiguous for incorrect predictions.Attribution analysis of incorrect predictions is identified as complementary future work.
  • Responsible use: Dataset collection biases may affect how attribution-faithfulness conclusions generalize across demographic groups and clinical contexts.The authors encourage evaluation across more diverse demographic and clinical subgroups.

B.3 Benchmark Statistics

The benchmark partitions samples by correctness and causal grounding, retaining only predictions validated by targeted foreground and background counterfactual edits. Its strict filtering produces model-specific subsets for direct and reasoning modes.

  • Filtering: The filtering procedure separates samples into incorrect, correct-but-ungrounded, and correct-and-grounded groups.Correct-and-grounded samples answer correctly, flip after foreground editing, and remain unchanged after both background edits.
  • Filtering: Many correct predictions are not causally grounded in expert-annotated regions, demonstrating why expert boxes alone are insufficient attribution ground truth.The benchmark therefore relies on causal filtering rather than localization annotations alone.
  • Post-hoc analysis: MedFocus alignment generally increases from incorrect samples to correct-but-ungrounded samples and then to correct-and-grounded samples.This post-hoc pattern is consistent with the intended filtering effect, although MedFocus is not used to construct the benchmark.
  • Benchmark size: 1,880 direct-answer samples and 2,060 reasoning samples remain after filtering across models and datasets.Per-model retention rates range from approximately 1.5% to 20%, reflecting the strictness of the three causal checks.
  • Benchmark interpretation: The benchmark targets causally relevant annotated regions without requiring them to be the model’s sole visual evidence source.Evaluated attribution methods do not receive the expert boxes or RadEdit edits.

C Implementation Details of Baseline Methods

The baseline suite covers attention-, gradient-, prompting-, and perturbation-based attribution methods, with outputs standardized for comparison against bounding-box ground truth.

  • Standardization: Pixel-level saliency maps are min-max normalized, thresholded at the 90th percentile, filtered by connected-component size, and converted to tight bounding boxes.This standardized conversion enables comparison with expert bounding-box annotations.
  • Attention-based methods: Attention-based baselines include Attention Head, Attention Rollout, and LRP.They use attention weights or propagate attention-based relevance through the network.
  • Gradient-based methods: Gradient-based baselines include GradCAM, GradCAM++, Gradient-weighted Attention, and Integrated Gradients.These methods derive spatial attribution from gradients and hidden-state or attention representations.
  • Prompting-based methods: Prompting baselines ask the LVLM for relevant bounding boxes, optionally refining the identified regions with MedSAM.The prompting pipeline outputs box coordinates, while Prompting + MedSAM uses region descriptions for segmentation refinement.
  • Perturbation-based methods: Perturbation baselines estimate importance from output changes caused by sliding patches or randomly masked image regions.Occlusion and RISE use 8 × 8 pixel patches, with RISE probing randomly masked inputs.

D Implementation Details of MedFocus

MedFocus transfers clinically defined anatomical concepts from a selected normal CXR to target images using unbalanced optimal transport, then refines transferred regions for causal attribution. Its vocabulary is interpretable but limits granularity for findings not represented by the concepts.

  • Concept vocabulary: MedFocus uses 11 predefined anatomical concepts and four bilateral composite groups to define clinically meaningful attribution regions.The concepts include lungs, mediastinal structures, clavicles, hilar structures, and costophrenic angles.
  • Vocabulary limitation: The current anatomical vocabulary limits granularity for findings such as small nodules, diffuse bilateral disease, and cardiothoracic-ratio-based cardiomegaly.The framework can support finer concepts when reliable masks or proposals become available.
  • Concept transfer: Unbalanced optimal transport with Sinkhorn iterations transfers concept regions from a reference normal CXR to the target image.The method uses entropic regularization and relaxed marginal constraints to compute the transport plan.
  • Region extraction: For each concept, transported mass is aggregated over target pixels and a dense core covering 75% of total mass is selected.This avoids using the numerically dense full support produced by entropic regularization.
  • Reference selection: The reference normal CXR is selected from 16 fully annotated candidates using the lowest total transport cost, with selection at 14 × 14 and transfer at 56 × 56 resolution.Mapped masks are later upsampled to 224 × 224 for MedSAM refinement and causal attribution.
  • Performance: MedFocus performance varies with model scale and training domain, with larger models and medically trained models generally achieving higher attribution scores.Table 5 summarizes performance across datasets and evaluation modes.

E.2 Comparison of Method Efficiency

MedFocus trades some speed for stronger attribution analysis, while remaining faster than perturbation-based alternatives. Its outputs are reasonably stable across UOT settings and are dominated by anatomically plausible pulmonary concepts.

  • Inference Efficiency: 1.65 seconds per sample makes MedFocus slower than lightweight gradient- and attention-based baselines but faster than expensive perturbation-based alternatives.Attention-based methods are fastest overall, while prompting-based methods take approximately one second per sample.
  • Hyperparameter Sensitivity: MedFocus exhibits reasonable stability across the tested UOT hyperparameter settings.The analysis varies candidate reference images, marginal relaxation coefficients, and entropic regularization.
  • Hyperparameter Sensitivity: Increasing the marginal relaxation coefficient λ to 1.0 enforces stricter adherence to the original marginal distributions and degrades performance.This identifies a sensitivity to the balance between marginal adherence and attribution performance.
  • Anatomical Concept Distribution: The left and right lungs account for the vast majority of identified important concepts across the three datasets.This pattern matches the benchmark’s emphasis on pulmonary findings, while the cardiac silhouette appears more often on PadChest-GR.

F Qualitative Model Comparison and Error Analysis

Qualitative comparisons show that attribution quality varies across model families and reasoning outputs, even when models answer correctly. MedGemma often localizes evidence more precisely, but all models can produce broad or incomplete attributions for small lesions.

  • Spatial Attribution Comparison: MedGemma variants localize bilateral aspiration abnormalities more precisely and overlap more strongly with expert annotations than Gemma3 variants.This comparison uses representative ImaGenome examples.
  • Spatial Attribution Comparison: All models produce overly broad lung-level attributions for the nodule/mass example instead of tightly focusing on the small focal lesion.The example is drawn from VinDR-CXR.
  • Error Analysis: For abnormal foreign body or metal, all models partially collapse to coarse thoracic-level attributions despite the evidence occupying a very small region.This example is drawn from PadChest-GR and illustrates partial coverage and overly broad localization.
  • Reasoning-Grounding Alignment: In the osteosynthesis-material reasoning example, MedGemma concentrates on the left shoulder/clavicular region, whereas Gemma3 relies on diffuse, partially relevant evidence.All four models answer correctly, but their reasoning-grounding alignment differs substantially.
  • Reasoning-Grounding Alignment: Correct answers do not guarantee that spatial attributions and intermediate reasoning are tied to the true supporting evidence.The qualitative comparisons reveal marked differences in grounding alignment across models.
Loading 2605.20158v1…