PaperlayerView exact revision on arXiv

Source-linked AI summary

AlphaRAD: Grounded Zero-Shot Classification in Chest Radiology via $α$-Corrected Binary Cross Entropy and Factorized Latent Supervision

Jianzhong You, Yuan Gao, Chris McIntosh

arXiv:2609.01757v1cs.CV

TL;DR

AlphaRAD addresses underexplored problems in structured clinical-semantic supervision and faithful spatial grounding for chest-radiology VLPMs. It combines LLM-derived concept discrimination with α-BCE and parameter-neutral FLaS factorized fusion, achieving state-of-the-art average classification and best reported grounding, phrase-grounding, and segmentation performance across the evaluated benchmarks.

  • Problem

    Medical VLPMs need better use of structured clinical semantics to reduce in-batch false negatives and better cross-modal fusion for faithful spatial grounding.

  • Method

    AlphaRAD trains on LLM-parsed positive medical concepts with α-BCE and uses FLaS to factorize VLPM representations into independently supervised subspaces without additional parameters.

  • Results

    AlphaRAD achieves state-of-the-art average classification performance and best performance across the evaluated grounding, phrase-grounding, and segmentation benchmarks.

  • Takeaways & Limitations

    The unified model supports spatially grounded open-vocabulary zero-shot inference across diverse chest-radiology tasks.

  • Takeaways & Limitations

    FLaS subspaces can redundantly converge on the same diagnostic evidence, and incorrect localizations by several subspaces can pull the averaged map away from the target.

Abstract

from arXiv · show

Vision-Language Pretrained Models (VLPMs) offer a scalable path to open-vocabulary chest radiology understanding, yet two aspects remain underexplored: how structured clinical semantics extracted from medical reports can reduce in-batch noise during contrastive learning, and how cross-modal fusion can be designed to produce more faithful spatial grounding without added complexity. We introduce AlphaRAD, addressing these opportunities through two contributions. First, we construct a large-scale structured medical concept space from medical reports parsed by a Large Language Model for training, thereby mitigating in-batch learning noise and removing heuristic pair matching in contrastive learning, and thus naturally positioning AlphaRAD as a medical concept discriminator trained via $α$-Corrected Binary Cross-Entropy. Second, we propose FLaS (Factorized Latent Supervision), an extremely simple yet effective cross-modal feature fusion module that factorizes VLPM representations into independent subspaces, using dedicated alignment supervision to enhance the expressiveness of spatial grounding without introducing additional model parameters. Through extensive empirical validation, AlphaRAD shows strong zero-shot generalization across diverse chest radiology tasks. Notably, it establishes state-of-the-art average performance across 16 classification benchmarks, while achieving individual state-of-the-art results via distinct gains on 7 grounding/phrase grounding and 3 segmentation datasets.

1 Introduction

Medical VLPMs support open-vocabulary chest-radiograph inference, but clinical concepts remain underused during training and spatial fusion remains limited for faithful grounding. AlphaRad addresses these gaps with concept discrimination via α-BCE and parameter-neutral factorized fusion, achieving broad zero-shot performance across classification, grounding, and segmentation tasks.

  • Motivation: VLPMs align chest radiographs with reports to support recognition of unseen concepts without task-specific retraining or exhaustive labels.This is useful where fine-grained CXR annotations require expensive expert interpretation.
  • Motivation: Spatially grounded predictions are essential for clinical integration because accuracy alone does not establish that decisions rely on relevant visual evidence.
  • Motivation: LLM-parsed clinical concepts encode disease type, presence, and anatomical location, enabling more precise vision-language alignment for classification, grounding, and segmentation.
  • Limitations: Existing training can misclassify semantically related image-concept pairs as false negatives, while fusion mechanisms still leave spatial localization and semantic discriminability jointly under-addressed.For example, an image showing pleural effusion may be treated as negative for that concept when it appears elsewhere in the batch.
  • AlphaRad: AlphaRad pairs each study with all extracted positive findings and treats only absent concepts as negatives, using α-BCE to reduce in-batch noise without pairwise matching heuristics.
  • AlphaRad: FLaS decomposes VLPM representations into independently supervised subspaces with separate attention and alignment signals, enhancing fine-grained spatial-semantic alignment without added parameters.
  • Results: AlphaRad supports spatially grounded open-vocabulary zero-shot inference and reports state-of-the-art average classification across 16 datasets, with best results across 7 grounding/phrase-grounding and 3 segmentation datasets.

3 Methods

AlphaRad extends medical vision-language alignment into concept discrimination and factorized spatial supervision. Its training combines α-corrected BCE for structured clinical concepts with independently supervised latent subspaces for coarse and fine-grained tasks.

  • Medical Concept Discrimination: LLM-extracted clinical concepts encode findings, polarity, and anatomical context, replacing global report representations with structured medical semantics.This addresses the risk that multi-finding reports entangle heterogeneous semantics and weaken fine-grained image–text correspondence.
  • Medical Concept Discrimination: α-corrected BCE treats extracted positive findings as positives and unmentioned concepts as negatives, while reweighting the negative signal by β.The objective is motivated by label sparsity, which can cause negatives to dominate gradients.
  • Medical Concept Discrimination: Negative-concept sampling replaces the full clinical-concept negative sum with an unbiased Monte Carlo estimate, reducing computation while preserving the objective in expectation.The method samples negatives instead of computing logits for all clinical concepts at every step.
  • Factorized Latent Space Supervision: FLaS partitions the D-dimensional latent space into F disjoint subspaces and applies separate attention and alignment supervision to each.Each subspace receives an independent MP-NCE loss rather than averaging alignment scores before contrastive supervision.
  • Factorized Latent Space Supervision: Unlike transformer-decoder MHCA, FLaS operates directly on the existing latent space without additional projection matrices, but it omits cross-subspace interactions.The design bypasses concatenation and feed-forward processing to preserve the integrity of each factorized latent.
  • Final Training Objective: The unified AlphaRad objective combines α-BCE on averaged subspace logits with FLaS supervision to support coarse-grained and fine-grained downstream tasks.An ablation compares independent subspace alignment against averaging scores across factorized subspaces before MP-NCE.

4 Experiments

AlphaRad is trained on MIMIC-CXR with LLM-extracted medical concepts and evaluated using zero-shot classification, grounding, and segmentation metrics across multiple datasets.

  • Training Data: AlphaRad is trained on the official MIMIC-CXR training set, comprising 377,110 images from 227,835 studies.
  • Training Data: LLM-extracted unique medical concepts are derived from the MIMIC-CXR training reports.
  • Evaluation Setup: Evaluation covers 22 testing datasets, including shared test sets and full-dataset testing for several limited-size benchmarks.
  • Evaluation Metrics: Zero-shot classification uses AUROC, grounding uses Pointing Game accuracy, and segmentation uses Dice on positive samples.
  • Model Configuration: Experiments compare XrayDINOv2 encoders at 224 × 224 and 518 × 518 resolutions with BioBERT or MPNet text encoders.

5 Empirical Results and Analysis

AlphaRad achieves strong zero-shot performance across classification, grounding, phrase grounding, and segmentation tasks. Its factorized subspaces provide both effective localization and interpretable diagnostic evidence, while qualitative analysis exposes redundancy-related failures.

  • Zero-Shot Classification: 83.3% average AUROC across 16 classification tasks surpasses RadZero at 80.2%, BioViL-T at 72.8%, and KAD at 76.8%.
  • Grounding and Phrase Grounding: Table 4 reports disease-level Pointing Game accuracy across ChestXDet10, CheXlocalize, and VinDr-CXR.
  • Grounding and Phrase Grounding: AlphaRad establishes state-of-the-art localization across seven grounding and phrase-grounding benchmarks.
  • Grounding and Phrase Grounding: At 224 resolution, AlphaRad exceeds RadZero-518 on CXL with 66.9 versus 65.4 Pointing Game accuracy.
  • Zero-Shot Segmentation: AlphaRad-224 surpasses RadZero-518 on CheXlocalize with 54.2 and QaTa-COV19 with 54.7 Dice.
  • Qualitative Analysis of Factorized Subspaces: FLaS factorized subspaces capture overlapping Pleural Effusion evidence, non-redundant Pulmonary Fibrosis cues, and incorrect Atelectasis localizations.

6 Ablation Studies

Ablations show complementary roles for α-BCE and MP-NCE, and demonstrate that independent supervision of every FLaS subspace is important for localization performance.

  • Loss Ablation: 54.8 on ChestXDet10 and 12.8 on SIIM-ACR are achieved when α-BCE and MP-NCE are combined.
  • Subspace Supervision: Removing independent subspace supervision reduces ChestXDet10 performance from 56.0 to 53.9 and SIIM-ACR performance from 16.2 to 12.3.
  • Loss Ablation: The ablations characterize α-BCE as promoting concept-level discrimination and MP-NCE as improving spatial alignment.

7 Conclusion

AlphaRad combines LLM-parsed medical concept discrimination with parameter-neutral FLaS subspace alignment to support grounded open-vocabulary zero-shot inference. The paper reports state-of-the-art average classification and benchmark-level grounding, phrase-grounding, and segmentation performance.

  • AlphaRad uses α-BCE over an LLM-parsed medical concept space to reduce in-batch false negatives.
  • FLaS aligns disjoint latent subspaces without additional parameters to enhance spatial faithfulness.
  • AlphaRad achieves state-of-the-art average classification performance and best performance on each grounding, phrase-grounding, and segmentation benchmark.

8 Supplementary Material

The supplementary material details LLM-based concept extraction, model and dataset configurations, and ablations showing how FLaS heads behave across classification and spatial tasks.

  • Supplementary Methods: LLM prompts extract diseases, devices, and abnormalities with confirmed, uncertain, or ruled-out status and anatomical modifiers.Uncertain concepts are treated as negatives during training.
  • Datasets: 377,110 MIMIC-CXR images from 227,835 studies provide paired radiology reports and labels for 14 thoracic conditions.The dataset contains data from more than 65,000 patients.
  • Additional Ablations: AlphaRad reaches an average AUROC of 83.27 across 16 classification benchmarks, exceeding every individual FLaS head and FLaS-1* at 83.11.Individual heads span only 83.01–83.17 because classification supervision is applied to averaged logits.
  • Additional Ablations: VinDr-CXR grounding varies by 3.4 points (46.5–49.9) and MS-CXR V1 by 4.2 points (85.6–89.8) across FLaS heads.FLaS-1* reaches 59.7 on CheXlocalize grounding and 11.9 on SIIM-ACR segmentation, below individual-head ranges of 65.0–66.2 and 15.3–16.6.
  • Additional Ablations: Averaging FLaS-8 heads aggregates complementary spatial representations, improving grounding and segmentation over individual heads.Shared classification supervision produces consensus, while independent alignment supervision permits distinct localization subspaces.

9 Segmentation Performance in ChestXlocalize

AlphaRad performs strongly on zero-shot ChestXlocalize segmentation, with lower-resolution results leading most pathology classes and higher resolution helping selected findings.

  • ChestXlocalize Segmentation: At 224 resolution, AlphaRad achieves the best Dice performance on 8 of 10 ChestXlocalize pathology classes.It particularly scores 61.5 for Atelectasis, 56.9 for Opacity, and 53.7 for Lesion, outperforming RadZero at 518 resolution.
  • ChestXlocalize Segmentation: Increasing AlphaRad from 224 to 518 resolution raises Device Dice from 37.9 to 43.9 and Pneumothorax Dice from 37.0 to 45.1.The passage also reports regressions on other classes.

10 Grounding Performance for each Disease

AlphaRad consistently outperforms established baselines across three chest-radiography grounding datasets, with especially strong localization for several disease categories. Higher input resolution further improves detection of localized or subtle findings.

  • Grounding results: 85.4% for Atelectasis and 87.3% for Pleural Effusion on ChestXDet10, plus 97.7% for Cardiomegaly on CheXlocalize, are reported as state-of-the-art localization results.These results are compared against BioViL-T, KAD, CARZero, and RadZero.
  • Grounding results: AlphaRad consistently outperforms established baselines across ChestXDet10, CheXlocalize, and VinDr-CXR.The comparison covers a wide array of thoracic pathologies.
  • Resolution effects: 70.1% versus 39.0%: AlphaRad’s ChestXDet10 Nodule accuracy rises when input resolution increases from 224 to 518.The passage attributes particularly substantial gains to higher resolution for highly localized or subtle findings.
  • Overall conclusion: The quantitative results across diverse clinical datasets support AlphaRad’s effectiveness for zero-shot disease localization in chest radiography.The conclusion emphasizes the framework’s spatial understanding.

11 LLM Parsing Noise Analysis

The noise analysis finds that AlphaRad remains robust to simulated false-negative in-batch noise, even at extreme perturbation levels. The authors attribute this behavior to the small α used in the α-BCE objective.

  • Experimental setup: The analysis simulates false-negative noise by treating each positive medical concept as a Bernoulli random variable and flipping positives to negatives.The flip probabilities range from 0.05 to 0.99.
  • False-negative noise: Performance remains stable across classification, grounding, and segmentation when false-negative noise reaches 0.99.False-negative noise is simulated by randomly flipping ground-truth positive medical concepts to negatives.
  • Interpretation: The authors attribute robustness to α = 1 in α-BCE, which approximates a very small β in β-BCE.The passage links this setting to damping the learning signal derived from noise.

12 Sensitivity to α in α-BCE

AlphaRad is reported to be robust across a wide range of α values between the two theoretical β-BCE bounds. Classification, grounding, phrase grounding, and segmentation remain broadly stable despite the associated gradient-weight changes.

  • Evaluation protocol: Macro-averaged evaluation measures AUROC for classification, PG accuracy for grounding and phrase grounding, and Dice score for segmentation.The metrics are reported across all evaluated datasets.
  • α sweep: The strongest AUC performance occurs at the lower bound 1 512, which balances positive and negative gradients of the β-BCE loss.The passage describes this as the lower theoretical bound.
  • α sweep: The upper bound 215698 512 provides an unbiased estimator of unweighted β-BCE but creates severe positive-negative gradient weight imbalance.Despite this imbalance, the reported degradation is minor for zero-shot segmentation and grounding.
  • Task-specific effects: The upper-bound setting increases average performance for classification and phrase grounding while slightly degrading zero-shot segmentation and grounding.These effects are reported relative to the broader α-sensitivity analysis.
  • Overall conclusion: Across classification, grounding, and segmentation, performance remains remarkably stable within the tested theoretical α limits.The conclusion states that this stability is independent of specific α tuning within those limits.

13 Factorized Subspace Interaction

The factorized-subspace analysis compares FLaS’s disjoint formulation with learned weighted fusion and examines performance stability across random seeds. Learned subspace weights provide only minimal gains, while results remain tightly bounded across downstream tasks.

  • Factorized Subspace Interaction: Learned per-subspace weights yield only minimal gains over the standard unweighted vanilla average.The comparison uses eight factorized latent spaces.
  • Factorized Subspace Interaction: The minimal-gain trend holds across classification, spatial grounding, and pixel-level segmentation.This suggests that FLaS heads self-regulate their feature contributions without parameterized weighting.
  • Stability analysis: Performance metrics are averaged across 10 distinct random-seed training runs to assess optimization stability and reproducibility.The analysis covers zero-shot classification, grounding, phrase grounding, and semantic segmentation.
  • Stability analysis: The reported variance bounds are tight across all evaluated downstream tasks.Classification AUROC is described as steady across benchmarks including CheXpert.
  • Reported results: Tables 13 and 14 report average zero-shot classification, grounding, phrase-grounding, and segmentation results using specified pretrained-backbone settings and RadZero as a baseline reference.The captions identify the dino.txt learning style and the relevant backbone configurations.

15 Expansion on the Controlled Experiments

Controlled experiments show that FLaS improves zero-shot classification across 16 datasets and strengthens grounding, phrase grounding, and segmentation performance. Additional analyses examine perturbations, factorized fusion, random-seed robustness, and image-resolution effects.

  • Zero-shot classification: 82.0% and 82.2% average accuracy are achieved by FLaS-1 and FLaS-8 at 224 resolution across 16 classification datasets, respectively.Both configurations outperform the RadZero comparison in the controlled evaluation.
  • Grounding and phrase grounding: 54.0% grounding and 76.1% phrase-grounding average scores are obtained by FLaS-8 at 518 resolution, exceeding RadZero’s 53.4% and 75.3%.The gains are reported for dense prediction tasks under the same controlled comparison.
  • Zero-shot segmentation: 40.1% average zero-shot segmentation is reached by FLaS-8, compared with 38.8% for the baseline.The authors present this as parallel improvement alongside grounding and phrase grounding.
  • Qualitative analysis: The qualitative grounding illustrations compare success and failure cases using ground-truth annotations with heatmap peaks for each disease class.Green marks ground truth; yellow marks the success-case peak, while cyan marks the failure-case peak.
  • Qualitative analysis: The qualitative segmentation illustrations compare ground-truth annotations with predicted masks across CheXlocalize disease classes 1–10.Classes 1–5 and 6–10 are shown separately, with green ground truth and cyan predicted masks.
  • Additional controlled analyses: The controlled studies assess false-negative and false-positive concept perturbations, α sensitivity, weighted-average versus disjoint factorization, ten random seeds, and image resolution.These analyses evaluate training perturbations, fusion design, initialization robustness, and resolution effects across the reported tasks.
Loading 2609.01757v1…