Source-linked AI summary
Efficient Chest X-ray Representation Learning via Semantic-Partitioned Contrastive Learning
Wangyu Feng, Shawn Young, Lijian Xu
TL;DR
CXR self-supervised learning must avoid low-value reconstruction and augmentation-induced distortion while capturing globally organized diagnostic structure. S-PCL contrasts non-overlapping patch partitions from one image, and experiments report competitive downstream performance with favorable computational efficiency.
Problem
Existing CXR SSL methods can overemphasize high-frequency reconstruction details, rely on risky augmentations, or depend on auxiliary textual supervision.
Method
S-PCL randomly partitions masked CXR patch tokens into two non-overlapping views and maximizes agreement between their shared-encoder representations.
Results
Across large-scale CXR benchmarks, S-PCL achieves competitive downstream performance and favorable scaling efficiency while using a streamlined architecture.
Takeaways & Limitations
S-PCL provides an efficient CXR pre-training framework that avoids pixel-level reconstruction, hand-crafted augmentations, momentum encoders, and decoders.
Abstract
from arXiv · showhide
Self-supervised learning (SSL) has emerged as a powerful paradigm for Chest X-ray (CXR) analysis under limited annotations. Yet, existing SSL strategies remain suboptimal for medical imaging. Masked image modeling allocates substantial computation to reconstructing high-frequency background details with limited diagnostic value. Contrastive learning, on the other hand, often depends on aggressive augmentations that risk altering clinically meaningful structures. We introduce Semantic-Partitioned Contrastive Learning (S-PCL), an efficient pre-training framework tailored for CXR representation learning. Instead of reconstructing pixels or relying on heavy augmentations, S-PCL randomly partitions patch tokens from a single CXR into two non-overlapping semantic subsets. Each subset provides a complementary but incomplete view. The encoder must maximize agreement between these partitions, implicitly inferring global anatomical layout and local pathological cues from partial evidence. This semantic partitioning forms an internal bottleneck that enforces long-range dependency modeling and structural coherence. S-PCL eliminates the need for hand-crafted augmentations, auxiliary decoders, and momentum encoders. The resulting architecture is streamlined, computationally efficient, and easy to scale. Extensive experiments on large-scale CXR benchmarks, including ChestX-ray14, CheXpert, RSNA Pneumonia and SIIM-ACR Pneumothorax, show that S-PCL achieves competitive performance while attaining the lowest GFLOPs and superior accuracy among existing SSL approaches.
1 Introduction
Existing CXR self-supervised methods can overemphasize low-level reconstruction, distort anatomical cues through aggressive augmentation, or depend on auxiliary supervision. S-PCL instead contrasts complementary non-overlapping patch subsets to learn coherent diagnostic representations efficiently.
- Motivation: Existing CXR SSL strategies emphasize pixel fidelity, potentially distort subtle anatomical cues, or depend on auxiliary textual supervision.The paper identifies these limitations as failing to explicitly exploit spatially sparse but globally organized diagnostic information.
- S-PCL: S-PCL randomly partitions image patches into two non-overlapping subsets and maximizes agreement between their representations.The complementary views encourage inference of missing context and coherent global thoracic structure.
- S-PCL: S-PCL avoids reconstruction overhead and augmentation-induced distortion while using contrastive learning to obtain high-level diagnostic representations.The framework also avoids momentum encoders and complex decoders.
- Evidence: Extensive experiments on large-scale CXR datasets demonstrate state-of-the-art downstream performance and favorable scaling efficiency.The contribution statement summarizes the evaluation across large-scale chest X-ray datasets.
2 Methodology
S-PCL creates two disjoint views from masked visible patch tokens and encodes them with a shared Vision Transformer. Contrastive alignment between the resulting embeddings encourages long-range anatomical reasoning while avoiding reconstruction and auxiliary-network overhead.
- Pipeline: S-PCL tokenizes a CXR, randomly masks tokens, partitions visible tokens into disjoint subsets, and encodes both views with a shared Vision Transformer.The two [CLS] representations provide the paired embeddings used for contrastive learning.
- Semantic partitioning: Random partitioning preserves statistically consistent patch positions while exposing each branch to a substantially higher effective masking ratio.This dual-ratio design creates complementary views with fine-grained semantic differences.
- Representation learning: The shared encoder must infer global thoracic structure and localized pathology from severely partial visual evidence.The internal information bottleneck constrains self-attention and promotes long-range dependency modeling.
- Contrastive objective: The T-SP contrastive objective aligns same-image partition embeddings and separates representations from different images in the batch.It uses positive pairs from the same radiograph and the remaining batch representations as negatives.
- Efficiency: S-PCL defines its objective in latent space and omits momentum encoders, projection MLP heads, and decoders.This streamlined design reduces computation and memory and is reported to achieve the lowest GFLOPs among comparable SSL methods.
3 Experiments
Experiments evaluate S-PCL across classification, dense prediction, scaling, and representation quality on multiple chest X-ray benchmarks. S-PCL delivers competitive accuracy with substantially lower pre-training cost and strong performance across model scales and supervision levels.
- Experimental setup: S-PCL uses MIMIC-CXR-JPG for pre-training and evaluates on ChestX-ray14, CheXpert, RSNA Pneumonia, and SIIM-ACR Pneumothorax.The evaluation includes multi-label classification, semantic segmentation, and object detection settings.
- Efficiency and scaling: 540 GPU hours yields 89.1% mAUC for S-PCL, compared with 1200 GPU hours and 89.2% mAUC for Medical MAE.MRM uses 800 GPU hours and reaches 88.7% mAUC.
- Efficiency and scaling: 82.8% mAUC with ViT-S/16 and 78.2% mAUC with ViT-T/16 demonstrate favorable efficiency-performance trade-offs at smaller scales.The corresponding pre-training costs are 140 and 65 GPU hours, respectively.
- Downstream performance: Across ChestX-ray14, CheXpert, and RSNA Pneumonia, S-PCL remains competitive under 1%, 10%, and 100% fine-tuning ratios.CheXpert reaches 86.7%, 88.4%, and 89.1% AUC; RSNA Pneumonia reaches 86.6%, 89.2%, and 91.2% AUC.
- Downstream performance: S-PCL achieves 89.1% mean AUC on CheXpert and the highest AUC for Cardiomegaly, Edema, and Effusion.The reported disease-specific scores are 95.4%, 94.1%, and 95.6%, respectively.
- Downstream performance: S-PCL obtains 91.4% for Effusion and 92.5% for Pneumothorax on ChestX-ray14, while also showing competitive dense prediction performance.The paper reports improvements in semantic segmentation under 10% and 100% supervision and across object-detection settings.
- Representation analysis: t-SNE visualization shows clear separation between pathological and normal radiographs in the learned global representations.The visualization uses 10,000 test images, including 8,851 diseased and 1,149 normal scans.
4 Conclusion
S-PCL learns CXR representations by contrasting semantic-partitioned views rather than reconstructing pixels or applying risky augmentations. The approach uses medical-image redundancy to learn robust representations with reduced computational overhead and outperforms state-of-the-art baselines.
- Conclusion: S-PCL contrasts semantic-partitioned views, eliminating pixel-level reconstruction and risky augmentations.The method exploits partial observations to learn CXR representations.
- Conclusion: S-PCL reduces computational overhead while outperforming state-of-the-art baselines.The conclusion positions the framework for scalable learning in high-resolution medical foundation models.