Source-linked AI summary
Which Pretext Task Transfers? Self-Supervised Pretraining Objectives for Lung Ultrasound
Moein Heidari, Junbo Rao, Jai Choraria, Wenjin Chen, David J. Foran, Ilker Hacihaliloglu
TL;DR
The paper asks which self-supervised pretext objective transfers best for lung ultrasound, addressing confounded prior comparisons. It controls the backbone, corpus, schedule, and frozen evaluation across contrastive, masked-reconstruction, and latent-prediction methods. VideoMAE and V-JEPA lead on POCUS, but MoCo leads on independently acquired Mendeley-Uganda, showing that in-domain probe accuracy alone is insufficient for selecting cross-dataset transfer.
Problem
Prior ultrasound studies compare pretext objectives using different corpora, architectures, and evaluation protocols, while LUS gaps include limited temporal-video and independent-dataset testing.
Method
The study pretrains contrastive, masked-reconstruction, and latent-prediction encoders on COVID-BLUeS using a shared backbone and training setup, then freezes them for probe evaluation on POCUS and Mendeley-Uganda.
Results
VideoMAE and V-JEPA perform best on POCUS, whereas MoCo performs best on Mendeley-Uganda and V-JEPA falls near chance.
Takeaways & Limitations
POCUS probe accuracy alone does not identify the objective that transfers best across datasets.
Takeaways & Limitations
The study evaluates only one ViT-S backbone scale and a three-class classification task that does not directly test the temporal information motivating video pretraining.
Abstract
from arXiv · showhide
Self-supervised learning (SSL) can reduce the need for labelled medical images, but the choice of pretext objective remains unclear for lung ultrasound (LUS). Contrastive learning, masked reconstruction, and joint-embedding predictive architectures (JEPA) differ in the space in which their targets are defined, yet existing ultrasound studies compare them under different corpora, backbones, and evaluation protocols. We compare these three objective families using the same encoder backbone, pretraining corpus, optimisation schedule, and frozen-evaluation protocol. Encoders are pretrained on COVID-BLUeS LUS videos and evaluated with linear, $k$NN, and attentive probes at 5\%, 10\%, 50\%, and 100\% label budgets. Evaluation is performed on POCUS using patient-level five-fold cross-validation and on the independently acquired Mendeley-Uganda dataset, which is excluded from both pretraining and probe fitting. At the full label budget under linear probing, VideoMAE and V-JEPA achieve $66.5 \pm 13.1$ and $65.4 \pm 11.7$ balanced accuracy on POCUS, while MoCo achieves $42.1 \pm 1.2$. On Mendeley-Uganda, the ranking reverses: MoCo performs best at $62.7 \pm 1.0$, followed by VideoMAE at $53.8 \pm 2.8$, while V-JEPA falls near chance at $35.1 \pm 4.9$. These results show that POCUS probe accuracy alone does not identify the objective that transfers best across datasets. We also outline planned representation-level analyses to examine this reversal. Code is publicly available at https://github.com/moeinheidari7829/LUSVideoSSL.
1. INTRODUCTION
The paper addresses the unclear choice of self-supervised objective for lung ultrasound by comparing contrastive, masked-reconstruction, and latent-prediction methods under controlled conditions. It tests whether performance transfers from POCUS to an independently acquired dataset.
- LUS interpretation requires both fine spatial detail and temporal behaviour, while expert annotation is costly and labelled collections remain small.
- Contrastive learning aligns augmented views, masked autoencoders reconstruct withheld pixels, and JEPA predicts withheld representations from visible context.
- Existing ultrasound comparisons confound pretext objectives with differences in corpus, architecture, and evaluation, preventing direct inference about objective merit.
- The study closes three LUS gaps by comparing objectives on temporal video, using a modest single-application corpus, and testing an independently acquired dataset.
- At full-label linear probing, VideoMAE and V-JEPA lead on POCUS, whereas MoCo leads on Mendeley-Uganda and V-JEPA approaches chance.
2. METHOD
The method compares three pretext objectives while holding the encoder, training setup, corpus, and frozen evaluation protocol fixed. Probes are evaluated across label budgets on POCUS and an external dataset.
- 2.1 Pretext objectives: The three paradigms differ only in the pretext task and the space in which its loss is evaluated.Contrastive learning uses whole-clip embeddings, masked reconstruction uses images, and latent prediction uses feature tokens.
- 2.1 Pretext objectives: Contrastive pretraining aligns temporally offset augmented clips from the same video and separates clips from distinct videos with InfoNCE.
- 2.1 Pretext objectives: Masked reconstruction removes 90% of spatio-temporal tubelets and trains a decoder to regress withheld pixels using mean squared error.
- 2.1 Pretext objectives: Latent prediction estimates masked target representations from visible context using an EMA teacher and an L1 feature-space loss.
- 2.2 Pretraining: All methods use the same randomly initialized ViT-S/16 backbone, 16-frame clips, COVID-BLUeS videos, optimizer, schedule, and epoch budget.
- 2.3 Frozen evaluation: Frozen encoders are assessed with linear, kNN, and attentive probes at 5%, 10%, 50%, and 100% labels using patient-level five-fold POCUS validation and external Mendeley-Uganda evaluation.Mendeley-Uganda is excluded from pretraining and probe fitting; balanced accuracy is averaged across five evaluations.
3. PRELIMINARY RESULTS
Objective rankings differ sharply between in-distribution POCUS evaluation and external Mendeley-Uganda transfer. VideoMAE and V-JEPA lead on POCUS, but MoCo leads externally while V-JEPA falls near chance.
- 66.5 ± 13.1 and 65.4 ± 11.7 balanced accuracy put VideoMAE and V-JEPA ahead of MoCo’s 42.1 ± 1.2 on POCUS under full-label linear probing.
- 62.7 ± 1.0 makes MoCo best on Mendeley-Uganda, ahead of VideoMAE at 53.8 ± 2.8 and V-JEPA at 35.1 ± 4.9.
- 20.6 points is MoCo’s increase from POCUS to Mendeley-Uganda, while VideoMAE and V-JEPA decrease by 12.7 and 30.3 points.
- MoCo performs best on Mendeley-Uganda across every probe and label budget, whereas its POCUS disadvantage appears at higher label budgets.
- Interpretation: The reversal may reflect different acquisition-setting trade-offs, with augmentation-based invariance potentially aiding MoCo transfer and JEPA sample-complexity requirements potentially harming V-JEPA.The authors state that these explanations are possible and that results may depend on pretraining scale and evaluation distribution.
4. SCOPE OF THE PRESENT STUDY AND PLANNED ANALYSIS
The study is preliminary because it evaluates only one backbone scale and a three-class task that does not directly test the temporal information motivating video pretraining. Planned representation-level analyses will compare attention, patch-token projections, and anchor-based similarity maps against annotated diagnostic regions to examine the cross-dataset ranking reversal.
- Scope: Only ViT-S is evaluated, and the three-class downstream task does not directly test the temporal information motivating video pretraining.
- Planned analysis: The planned analysis compares attention maps, patch-token principal components, and cosine-similarity maps anchored on pleural lines and B-line artefacts.
- Planned analysis: Similarity maps will be quantitatively compared with annotated A-lines, B-lines, consolidation, and pleural effusion regions to assess whether representations capture diagnostic structure or acquisition artefacts.
5. CONCLUSION
Using a controlled protocol, the study finds that the best objective depends on the evaluation dataset: VideoMAE and V-JEPA lead on POCUS, whereas MoCo leads on Mendeley-Uganda. This reversal means POCUS probe accuracy alone cannot identify the objective that transfers best to a new acquisition setting.
- Conclusion: VideoMAE and V-JEPA perform best on POCUS, while MoCo performs best on Mendeley-Uganda and V-JEPA drops near chance.
- Conclusion: POCUS probe accuracy alone is insufficient for selecting the objective that transfers best to a new acquisition setting.