Source-linked AI summary
From Adults to Children: Benchmarking Foundation Models for Chest X-Ray Classification under Domain Shift
Sara Inácio, Hugo Proença, João C. Neves
TL;DR
The robustness of foundation models for adult-to-pediatric chest X-ray classification remains unclear. This benchmark harmonizes adult and pediatric datasets and compares foundation models and CNN baselines under adult-only adaptation at three depths. All models show substantial degradation on pediatric data, and greater adaptation depth does not consistently resolve the transfer gap.
Problem
How foundation models with different pretraining and adaptation settings compare under a common adult-to-pediatric chest X-ray protocol remains unclear.
Source: §1 · pp. 1–2Method
The benchmark harmonizes VinDr-CXR and PediCXR into five labels and evaluates six foundation models and two CNN baselines after adult-only adaptation using linear probing, partial fine-tuning, or full fine-tuning.
Source: §1 · p. 2Results
Takeaways & Limitations
In-domain performance alone gives an incomplete view of transfer across populations, so shifted-population evaluation is important when assessing robustness and transferability.
Source: §6 · p. 8The evaluation covers a single adult-to-pediatric cross-dataset shift, and differences across most models prevent attributing robustness to any one factor.
Source: §7 · p. 8
Abstract
from arXiv · showhide
Foundation models have shown significant potential across diverse fields and have been applied to medical imaging, but their robustness across patient populations remains unclear. In this work, we investigate foundation models' ability to generalize from adult to pediatric chest X-ray classification. To the best of our knowledge, this is the first benchmark to compare multiple foundation models under a common adult-to-pediatric evaluation setting while also analyzing the effect of adaptation depth. We harmonize VinDr-CXR and PediCXR into a five-label classification task and evaluate six foundation models and two CNN baselines. All models are adapted using adult data only and tested in both the in-domain and pediatric domains using linear probing, partial fine-tuning, and full fine-tuning. The benchmark reveals a clear and consistent trend: all models suffer a substantial performance degradation when transferred from adult to pediatric data, with drops exceeding 10 percentage points. While Macro-AUROC values reach approximately 92% on the adult test set, performance decreases to around 78% on the pediatric domain. Another important finding is that increasing the depth of adaptation does not significantly improve cross-population generalization. Moreover, strong in-domain performance does not translate into better performance on pediatric data.
1. Introduction
Adult-to-pediatric chest X-ray transfer is a poorly characterized domain-shift problem, despite the clinical use of pediatric radiography and the promise of foundation models. This benchmark compares models and adaptation depths under a common adult-only training protocol.
1. Introduction
- Prior studies report substantial performance degradation when models trained on adult chest radiographs are evaluated on pediatric populations.Age-related differences include anatomy, disease prevalence, and the presentation of radiographic findings.Source: §1 · p. 1
- The adult-to-pediatric behavior of foundation models remains poorly characterized, including how pretraining and downstream adaptation relate to cross-domain performance.Existing work has mainly examined individual models, specific tasks, or single adaptation settings.Source: §1 · pp. 1–2
- The benchmark compares two CNN baselines and six foundation models trained or adapted on adult VinDr-CXR and evaluated on adult VinDr-CXR and pediatric PediCXR.It evaluates linear probing, partial fine-tuning, and full fine-tuning; pediatric data is not used for downstream adaptation.Source: §1 · p. 2
2. Related Work
Prior work explores medical and radiology foundation models, while domain shifts from adult to pediatric data remain an important transfer concern. This benchmark builds on those strands by comparing multiple foundation models and adaptation depths under one adult-to-pediatric protocol.
2.1. Foundation Models for Medical Imaging
- Medical foundation models use image-text or image-only pretraining, with methods such as MedCLIP [29], BioMedCLIP [35], and DINOv2 [17] representing distinct approaches.Image-text pretraining can use reports as semantic supervision, while image-only methods learn transferable visual representations without text.Source: §2.1 · p. 2
2.2. Foundation Models for Chest X-rays
- Chest X-ray foundation models include report-paired approaches such as BioViL-T [1] and image-only RAD-DINO.Other work includes contrastive models focused on zero-shot or transferable recognition and methods connecting radiograph representations with language models.Source: §2.2 · p. 2
2.3. Domain Shift in Medical Imaging
- Differences in scanners, protocols, institutions, patient populations, and collection periods can shift medical-image distributions and reduce cross-domain performance.Population shifts can affect image appearance and the distribution of findings.Source: §2.3 · pp. 2–3
2.4. Adult-to-Pediatric Domain Shift
- Prior adult-to-pediatric studies examine CNN transfer, unsupervised adaptation, segmentation, and zero-shot detection, but multiple foundation models and adaptation depths were not systematically compared under one protocol.The literature review identifies limited pediatric data availability as a relevant concern.Source: §2.4 · p. 3
3. Adult-to-Pediatric CXR Benchmark
The benchmark evaluates multi-label chest X-ray classification after adult-only adaptation, then tests transfer to pediatric data without target-domain training. It harmonizes the datasets and compares varied model pretraining strategies and adaptation depths.
3.1. Problem Formulation
- Models are optimized on adult source data and evaluated on adult and pediatric test sets, with no pediatric samples used for adaptation or model selection.The setup treats adult and pediatric data as source and target domains, respectively.Source: §3.1 · p. 3
3.2. Datasets
- VinDr-CXR and PediCXR labels are harmonized into five shared targets: No Finding, Other Diseases, Lung Tumor, Pneumonia, and Tuberculosis.COPD and pediatric labels without a VinDr-CXR counterpart are excluded; pneumonia subtypes in PediCXR are merged.Source: §3.2 · p. 3
3.3. Models
- The benchmark spans CNN baselines and foundation models pretrained on natural, biomedical, or radiological data using supervised, image-only self-supervised, or image-text objectives.All models use the visual encoder and the same downstream classification framework; multimodal text encoders are not used for downstream training or inference.Source: §3.3 · pp. 3–4
3.4. Adaptation Protocols
- Linear probing trains only the classification head, partial fine-tuning updates the final encoder block or stage and head, and full fine-tuning updates the entire encoder and head.Partial fine-tuning is structurally defined and is not parameter-matched across architectures.Source: §3.4 · p. 4
4. Evaluation
The evaluation compares adult in-domain performance with pediatric cross-domain performance after adult-only training. It reports complementary classification metrics and uses repeated experiments with source-domain model selection.
4.1. In-Domain & Cross-Domain
- Models are trained on VinDr-CXR adult training data and evaluated on the held-out adult test set and PediCXR without pediatric adaptation.The same evaluation procedure is applied across CNN baselines, foundation models, and all three adaptation strategies.Source: §4.1 · pp. 4–5
4.2. Metrics
- Macro-AUROC is the primary domain-shift metric, while mAP and Macro-F1 provide complementary multi-label classification measures.Metrics are computed over five labels; Macro-F1 uses a fixed 0.5 threshold.Source: §4.2 · p. 5
4.3. Implementation Details
- The study repeats each experiment with three random seeds and selects checkpoints by validation Macro-AUROC on adult source data.Training uses a fixed adult-data split, and the official VinDr-CXR test set is reserved for in-domain evaluation.Source: §4.3 · p. 5
- The two conventional CNN architectures serve as baselines, providing a reference to assess whether foundation models offer an advantage over established CNN-based approaches under the same training and evaluation settings.The baselines are DenseNet-121 [9] and EfficientNet [25].Source: §4.3 · p. 5
5. Results
The results compare CNNs and foundation models across pretraining settings, adaptation strategies, and adult-to-pediatric robustness. Across these comparisons, transfer varies by model, and adult performance alone does not reliably indicate pediatric performance.
5.1. Foundation Models vs. CNNs
- All models degrade on pediatric data, but the strongest foundation models outperform the CNN baselines under linear probing; DenseNet-121 becomes competitive with several foundation models under full fine-tuning.BiomedCLIP, MedCLIP, and RAD-DINO reach approximately 75% or higher pediatric AUROC under linear probing, versus 56.11% for DenseNet-121 and 46.84% for EfficientNet-B0.Source: §5.1 · p. 6
5.2. Influence of Pretraining on Domain-Shift
- RAD-DINO maintains pediatric AUROC near 80% across adaptation protocols, while other models vary; BioViL-T improves from 39.23% with linear probing to 68.16% with full fine-tuning.MedCLIP and BiomedCLIP perform strongly with linear probing but decline with more fine-tuning; DINOv2 performs best with partial fine-tuning and CLIP with full fine-tuning.Source: §5.2 · p. 6
5.3. Adaptation on Domain-shift Robustness
- Average pediatric AUROC rises from 60.72% with linear probing to 65.84% with partial fine-tuning and 69.00% with full fine-tuning, while the domain-shift gap falls from 21.65 to 18.39 and 17.21.Most average gap reduction occurs with partial fine-tuning, but effects vary by model; RAD-DINO changes little, while MedCLIP and BiomedCLIP perform best with linear probing.Source: §5.3 · p. 6, Figure 3 · p. 6
5.4. In-Domain vs. Cross-Domain Robustness
- All configurations show lower pediatric than adult AUROC, and BioViL-T drops from 80.66% in-domain to 39.23% pediatric AUROC under linear probing.RAD-DINO and BiomedCLIP maintain smaller gaps while achieving strong pediatric performance; similar adult scores can correspond to different pediatric outcomes.Source: §5.4 · pp. 6, 8, Figure 4 · p. 7
6. Discussion
The discussion finds that transfer robustness varies across foundation models and adaptation strategies. It emphasizes that in-domain scores alone are insufficient for evaluating adult-to-pediatric transfer.
6. Discussion
- RAD-DINO, MedCLIP, and BiomedCLIP preserve performance relatively well under limited adaptation, but BioViL-T has the largest linear-probing gap despite medical image-text pretraining.The experiments do not isolate architecture, pretraining data, or objective as causes of these differences.Source: §6 · p. 8
- RAD-DINO remains near 0.80 pediatric AUROC across protocols, while MedCLIP and BiomedCLIP decline as more parameters are updated.Partial fine-tuning captures most of the average gap reduction observed with full fine-tuning, though model-specific effects vary.Source: §6 · p. 8
- Adult and pediatric performance generally trend together, but similar adult scores can yield different transfer outcomes, so in-domain performance alone gives an incomplete robustness assessment.The discussion identifies BioViL-T as an example of strong adult performance alongside a substantial pediatric decline.Source: §6 · p. 8
7. Limitations
The evaluation covers one adult-to-pediatric cross-dataset shift, limiting what can be concluded about generalization to other settings.
7. Limitations
- Because age is confounded with dataset-specific differences such as image acquisition, the study cannot isolate age effects or establish that its trends hold across other hospitals, devices, pediatric populations, or shifts.The full model set also varies in architecture, data, scale, supervision, and objective, so observed differences are associations rather than evidence that one factor causes robustness.Source: §7 · p. 8
8. Conclusion
The benchmark finds substantial adult-to-pediatric degradation across models, with transfer depending on both pretrained representation and adaptation. Adult performance does not reliably predict pediatric performance.
8. Conclusion
- All models are affected by the adult-to-pediatric shift, but degradation varies substantially across pretrained representations.RAD-DINO, MedCLIP, and BiomedCLIP retain comparatively strong pediatric performance with limited adaptation, while other models are more shift-sensitive.Source: §8 · p. 8
- More adaptation is not necessarily better: fine-tuning more of the encoder can improve transfer, have little effect, or reduce pediatric performance.The outcome depends on the model.Source: §8 · p. 8
- Strong adult test-set performance does not guarantee that a model will transfer best to pediatric data.Source: §8 · p. 8
- Cross-population robustness depends on both the pretrained representation and its adaptation to the downstream task.The findings support evaluating foundation models directly across patient populations rather than using in-domain performance as a proxy for cross-domain robustness.Source: §8 · p. 8