PaperlayerView exact revision on arXiv

Source-linked AI summary

CoRE: Relating Fine-Grained Chest X-ray Findings in Anatomical Context

Minjun Kim, Jong Hak Moon

arXiv:2610.30584v1eess.IV

TL;DR

Global finding relations do not distinguish findings across anatomical locations, although both location and co-occurring abnormalities matter in chest X-ray interpretation. CoRE groups region–finding labels by co-occurrence and combines group-wise representations with region-specific visual evidence. It improves region–finding classification and finding-specific spatial alignment, with effective finding-level predictions on internal and external datasets without target-domain adaptation.

  • Problem

    Global finding-label relations do not distinguish the same finding across anatomical locations, despite the importance of both location and co-occurring abnormalities in CXR interpretation.

    Source: §1 · p. 1

  • Method

    CoRE groups anatomically specified region–finding labels by co-occurrence and combines group-wise label representation learning with region-specific visual evidence.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    CoRE improves region–finding classification and spatial alignment of finding-specific evidence, and remains effective for finding-level predictions on internal and external datasets without target-domain adaptation.

    Source: §5 · pp. 8–9, §4.6 · p. 8

  • Takeaways & Limitations

    The results support combining structured label sharing with anatomically grounded visual evidence for fine-grained CXR analysis.

    Source: §5 · p. 9

    Fine-grained region–finding supervision and validation are largely limited to one anatomically annotated dataset, and spatial alignment is assessed with bounding-box criteria rather than dense localization.

    Source: §5 · p. 9

Abstract

from arXiv · show

Chest X-ray interpretation depends on both the anatomical location of findings and their relationships with other abnormalities. However, relations defined over global finding labels do not explicitly distinguish the same finding across anatomical locations. We propose CoRE(Co-occurrence-guided Region Evidence), a framework that incorporates relationships among findings and anatomical context into fine-grained chest X-ray prediction. It groups anatomical region-finding labels by co-occurrence and uses these groups to guide shared representation learning through visual prompt tuning on a frozen vision transformer. The predictions are supplemented by an anatomically supervised region-evidence stream through region-matched logit residuals. Experiments on MIMIC-CXR with Chest ImaGenome annotations show that CoRE improves region-finding classification over standard and relation-aware prompt-based baselines. Quantitative evaluations on MS-CXR and the Chest ImaGenome gold standard show improved finding localization, supported by qualitative comparisons. CoRE also improves finding-level classification on mapped finding labels in external VinDr-CXR and CheXpert datasets without target-domain adaptation. These results demonstrate the value of jointly modeling finding relationships and anatomical context for fine-grained chest X-ray analysis.

1. Introduction

Chest X-ray interpretation depends on both the anatomical location of findings and their relationships with other abnormalities. Existing label relations often use global findings, while localized methods do not typically use region–finding pairs to define those relations; CoRE addresses this gap.

1. Introduction

  • Finding combinations and anatomical location can each shape diagnostic interpretation.The introduction gives congestive heart failure and upper-lobe versus dependent opacity as examples.Source: §1 · p. 1
  • Existing label-correlation methods model relations over global findings, so they do not distinguish the same finding across anatomical locations.The cited work includes multi-label image-recognition and CXR methods (Chen et al., 2019; Ma et al., 2025; Wang et al., 2023).Source: §1 · p. 1
  • Localized CXR methods associate findings with regions, but typically do not use anatomically specified findings as units for label relations or co-occurrence.CoRE instead uses region–finding pairs as both prediction targets and units for co-occurrence-guided label representation.Source: §1 · p. 2

2. Related Work

Prior work models label dependencies globally or uses anatomical regions and localized evidence for CXR prediction. CoRE defines co-occurrence over individual region–finding labels to organize label representations.

2. Related Work

  • Multi-label methods model label dependencies using co-occurrence statistics or graph relations, including methods for CXR classification.Examples include ML-GCN and ML-VPT for multi-label prediction, and CheXGCN, GL-MLL, and BB-GCN for CXR classification.Source: §2 · p. 2
  • These approaches generally relate global finding labels without explicitly incorporating anatomical specificity into the relation unit.Source: §2 · p. 2
  • Anatomy-aware CXR methods use region boxes, localized representations, or inter-region relations, whereas CoRE organizes label representations using co-occurrence among individual region–finding labels.Examples include ADPD, CheXDetector, and AnaXNet.Source: §2 · p. 2

3. Method

CoRE predicts 252 region–finding labels with two streams built on a frozen vision transformer. It groups labels by co-occurrence, combines group-wise label representations with region-specific evidence, and trains the resulting predictions with auxiliary supervision.

3. Method

  • CoRE predicts 252 valid combinations of 28 findings and 22 Chest ImaGenome regions using label-relation and region-evidence streams on a frozen ViT.Visual prompt tuning is used in the Label-Relation Stream.Source: §3 · p. 2

3.1. Label-Relation Stream

  • NPMI affinities from training-split co-occurrence statistics are spectrally clustered into groups that serve as a structural prior for representation sharing.Positive NPMI is retained; non-positive and never-observed pairs receive zero affinity.Source: §3.1 · pp. 2–3
  • Within each co-occurrence group, K prompt–expert pairs are shared, and image-conditioned routing produces label-side representations.The prompts are inserted into every ViT block; each label routes among the expert outputs using its group’s prompt features.Source: §3.1 · pp. 3–4

3.2. Region-Evidence Stream

  • The Region-Evidence Stream uses a query for each anatomical region to weight frozen-ViT patch features and form a region representation with a spatial attention map.Independent sigmoid weighting allows multiple patches to receive high relevance for the same region.Source: §3.2 · p. 4

3.3. Region-Matched Logit Residual

  • Each label receives an additive residual from its matched region representation, adapted and scored using the label-specific classifier direction.The adapter’s output projection is zero-initialized, so the region residual starts at zero and does not perturb the base logit.Source: §3.3 · p. 4

3.4. Training Objective

  • Training uses asymmetric loss on final region–finding logits and adds routing, patch-attention, and region-level supervision.The auxiliary gating objective is used only during training; patch attention is supervised from anatomical-box masks, and region predictions indicate whether any associated finding is present.Source: §3.4 · pp. 4–5

4. Experiments

Experiments evaluate CoRE on 252 anatomical region–finding labels in MIMIC-CXR, with localization and external finding-level evaluations also included. CoRE outperforms relation-aware prompt tuning at a nearly matched parameter budget, while ablations support the contributions of its streams, grouping, and auxiliary objectives.

4.1. Experimental Setup

  • The MIMIC-CXR experiments use 252 region–finding targets, with localization benchmarks and external VinDr-CXR and CheXpert evaluation without target-domain adaptation.The official MIMIC-CXR split has 236,089/1,959/3,403 training/validation/testing images; classification reports macro AUROC and AUPRC, alongside F1 and trainable parameters.Source: §4.1 · p. 5

4.2. Overall Comparison

  • With EVA-X, CoRE beats ML-VPT at nearly matched trainable parameters: AUROC 0.8430 vs 0.8283 and AUPRC 0.2024 vs 0.1831, with improved F1 and P. Hit.The comparison uses the same 252-label prediction setting; CoRE also shows better spatial correspondence of target-specific evidence.Source: §4.2 · pp. 5–6, Table 1 · p. 6

4.3. Ablation Studies

  • Removing either stream lowers performance, with a larger drop when the Label-Relation Stream is removed; the full two-stream model performs best.The ablation assesses the complementary roles of label-side representation learning and region-matched visual evidence.Source: §4.3 · p. 6, Table 2 · p. 6
  • All multi-group variants outperform the single-group baseline, while Random, Conditional, and NPMI grouping perform similarly and NPMI has the highest point estimates.The single-group baseline retains label-specific prompt–expert routing but shares candidates across labels.Source: §4.3 · p. 6
  • The full auxiliary objective achieves the highest AUROC and AUPRC; removing L_attn causes the largest P. Hit reduction among leave-one-out variants.Removing L_gate slightly increases P. Hit but reduces AUROC and AUPRC.Source: §4.3 · pp. 6–7, Table 3 · p. 7

MS-CXR CI

CoRE improves finding-level classification after region–finding outputs are aggregated, and its region-evidence stream improves spatial alignment. Qualitative examples show more concentrated saliency within annotated regions than comparison methods.

4.4. Finding-Specific Spatial Alignment

  • CoRE achieves the highest P. Hit on MS-CXR (0.7853) and CI (0.9045); adding the Region-Evidence Stream raises scores from 0.7655 to 0.7853 and 0.8861 to 0.9045, respectively.Finding-level saliency uses the maximum logit across region–finding outputs, while phrase-grounding models use predicted-box centers.Source: §4.4 · p. 7, MS-CXR CI · p. 7

4.5. Finding-Level Transfer and Generalization

  • CoRE achieves the highest AUROC and AUPRC on CI, VinDr-CXR, and CheXpert after aggregating region–finding logits.Compared with ML-VPT, AUROC/AUPRC rises from 0.8319/0.3564 to 0.8442/0.3748 on CI, 0.9140/0.4777 to 0.9313/0.4978 on VinDr-CXR, and 0.8983/0.6540 to 0.9125/0.6653 on CheXpert; performance is also higher than evaluated zero-shot CXR baselines on both external datasets.Source: §4.5 · pp. 7–8

4.6. Visualization of Finding-Specific Evidence

  • For single- and multi-box examples, CoRE’s saliency is more concentrated within annotated regions with less spillover than ML-VPT, while grounding boxes can shift, oversize, or incompletely cover targets.In a right lower lung consolidation example, both grounding models extend beyond the ground-truth box, while ML-VPT has an isolated off-target activation.Source: §4.6 · p. 8, Figure 2 · p. 8

5. Conclusion

CoRE combines group-wise label representation learning with region-specific visual evidence for fine-grained region–finding prediction. It improves classification and spatial alignment, and remains effective after aggregation on internal and external datasets without target-domain adaptation.

5. Conclusion

  • CoRE combines group-wise label representation learning with region-specific visual evidence, improving region–finding classification and spatial alignment while using a small fraction of full fine-tuning’s trainable parameters.Aggregated finding-level predictions remain effective on internal and external datasets without target-domain adaptation.Source: §5 · p. 8
  • NPMI partitions anatomically specified labels into relatively balanced groups that reflect observed co-occurrence, while grouping strategy has relatively little effect on predictive performance.The paper concludes that structured label sharing and anatomically grounded visual evidence benefit fine-grained CXR analysis.Source: §5 · p. 9
  • CoRE relies on static co-occurrence statistics, and fine-grained region–finding supervision and validation are largely limited to one anatomically annotated dataset.Spatial alignment is evaluated with bounding-box criteria rather than dense localization.Source: §5 · p. 9

Appendix A. Label Space and Grouping Analysis

The appendix defines the retained region–finding label space, compares grouping strategies, and reports sensitivity and implementation settings. NPMI yields relatively balanced groups, and region–finding-level NPMI supports the choice to preserve anatomical specificity.

A.1. Region–finding label space

  • The 252 retained labels span 22 regions and are drawn from a 28-finding vocabulary, with each pair requiring at least 80 positive training examples.Broader-region targets are masked during training when a more specific label is positive; evaluation applies upward closure to scores and ground-truth labels.Source: §A.1 · p. 13

A.2. Grouping balance and composition

  • At G = 8, NPMI group sizes range from 16 to 51 labels, compared with 11 to 76 for Conditional grouping; NPMI has a narrower range across evaluated G values.Table 7 compares the finding composition of the two grouping strategies at G = 8.Source: §A.2 · p. 13, Table 8 · p. 14, Table 7 · p. 14

A.3. Grouping strategy and relation granularity

  • NPMI has the highest mean AUROC and AUPRC among the grouping strategies, while region–finding-level NPMI yields higher AUROC than finding-level NPMI across all three seeds with comparable AUPRC.Results are relatively close across grouping criteria, indicating limited sensitivity to the partition within the evaluated setting.Source: §A.3 · pp. 14–15, Table 9 · p. 15

A.4. Sensitivity to G and K

  • With K = 5 fixed, G = 8 performs best among evaluated multi-group configurations; with G = 8 fixed, performance is similar across nearby K values.The study selects G = 8 and K = 5 as the default balance between predictive performance and model complexity.Source: §A.4 · p. 15, Table 10 · p. 15

B.1. Training and baseline implementation

  • Prompt-based baselines and CoRE use a frozen EVA-X backbone with a shared optimization protocol; linear probing freezes its backbone, whereas full fine-tuning updates it.Phrase-grounding and zero-shot baselines use released checkpoints and model-specific preprocessing pipelines.Source: §B.1 · p. 15

B.2. Evaluation-set exclusion and patient overlap

Evaluation images are excluded from the training split by image identifier, but many evaluation patients have another training study. On patient-disjoint subsets, CoRE has higher point estimates than ML-VPT for localization on both datasets and CI Gold classification.

B.2. Evaluation-set exclusion and patient overlap

  • Although evaluation images are excluded by image identifier, 712 of 851 MS-CXR patients and 335 of 500 CI Gold patients have another study in the training split.Mean within-patient Jaccard similarity of positive report-derived finding sets is 0.247 for MS-CXR and 0.145 for CI Gold.Source: §B.2 · p. 15
  • Among patients with no study in the 252-label training split, CoRE has higher point estimates than ML-VPT for localization on MS-CXR and CI Gold, and for CI Gold region–finding classification.The subsets include 206 MS-CXR localization images from 136 patients, 98 CI Gold localization images from 98 patients, and 149 CI Gold classification images from 149 patients over 58 evaluable labels.Source: §B.2 · pp. 15–16, Table 11 · p. 16

C.1. Label support and rare-label analysis

CoRE's advantage persists on a restricted label set, transfers to CI Gold region–finding classification, and supports external finding-level evaluation, though neither supervision strategy dominates every external metric.

C.1. Label support and rare-label analysis

  • +0.0128 AUROC (95% CI: [0.0083, 0.0177]) and +0.0120 AUPRC (95% CI: [0.0020, 0.0203]) for CoRE–ML-VPT on labels with at least ten positives.The paired patient-level bootstrap intervals exclude zero; absolute metrics for the restricted and full label sets are not directly comparable.Source: §C.1 · p. 16, Table 12 · p. 16

C.2. Region–finding classification on CI Gold

  • On CI Gold, CoRE exceeds ML-VPT for macro AUROC, macro AUPRC, and macro F1 across 83 evaluable labels, but its absolute metrics are not comparable to CI Silver results.The Gold evaluation uses a different annotation mask and excludes unannotated pairs rather than treating them as negative.Source: §C.2 · p. 16, Table 13 · p. 16

C.3. Finding-level transfer and external label mappings

  • VinDr-CXR and CheXpert results show neither EVA-X FFT-28 nor CoRE is consistently superior across both datasets and metrics; CoRE supports finding-level evaluation without target-domain adaptation.CoRE has higher VinDr-CXR AUPRC and CheXpert AUROC, while EVA-X FFT-28 has higher VinDr-CXR AUROC and CheXpert AUPRC.Source: §C.3 · p. 17, Table 14 · p. 17

Appendix D. Spatial Attribution and Localization Metrics

The appendix evaluates spatial alignment by attributing each region–finding target to image patches and checking whether peak-attribution locations fall within annotated boxes.

Appendix D. Spatial Attribution and Localization Metrics

  • Target-specific patch attribution uses patch representations after the first layer-normalization module in the final ViT block, with the same protocol across classification methods.For each region–finding target and patch, the method computes a gradient–activation attribution score.Source: Appendix D · p. 17
  • P. Hit counts a positive instance when the center of its highest-attribution patch lies inside the union of the corresponding region's ground-truth boxes.P. Hit is averaged over evaluated instances.Source: Appendix D · p. 17
  • Under hierarchy propagation, a parent's attribution comes from whichever parent or descendant output attains the propagated maximum, and its peak is checked against the parent box.P. Hit is not separately maximized across hierarchical outputs.Source: Appendix D · p. 17
  • Finding-level localization selects the highest-logit regional output for each finding and checks its peak patch against that finding's annotated boxes.After resizing and center-cropping images to 224 × 224, two empty apical boxes are excluded, leaving 1,160 of 1,162 MS-CXR annotations; cropping may reduce coverage near image boundaries.Source: Appendix D · p. 17

Appendix E. Qualitative Error Case Analysis

Qualitative examples show that localization errors can reflect partial coverage of extended findings or strict mismatch with annotated target boxes.

Appendix E. Qualitative Error Case Analysis

  • For extended lung opacity and edema, saliency can stay in anatomically relevant areas while covering only part of the annotated finding.CoRE is optimized for region–finding classification, not dense lesion delineation, and does not explicitly enforce full-abnormality coverage.Source: Appendix E · pp. 17–18
  • In costophrenic-angle examples, P. Hit marks responses outside the target box as errors even when activation falls on an adjacent area showing the same abnormality.The examples suggest some errors reflect spatial discrepancy with the annotated target rather than entirely irrelevant model evidence.Source: Appendix E · p. 18

Related papers

Loading 2610.30584v1…