Source-linked AI summary
When Adaptation Hurts: Connecting Representational Drift to OOD Failures in MedSAM Fine-Tuning
Marko Haralović, Sounic Akkaraju, Carlo Baretta, Vasil Zapryanov, Alexia Briassouli
TL;DR
MedSAM adaptation can improve in-domain and close-OOD segmentation while harming far-OOD robustness, but the relationships among adaptation, prompt noise, and representation preservation remain unclear. The paper evaluates six adaptation strategies under controlled prompt perturbations and domain shifts, using CKA to relate performance to internal drift. Full fine-tuning offers the strongest overall tradeoff, encoder-only LoRA is the strongest parameter-efficient alternative, and variable 0–100-pixel jitter generally produces the most reliable adapted models.
Problem
Prior work lacks a systematic account of which MedSAM fine-tuning strategies preserve generalization under both noisy ROI prompts and distribution shift.
Method
The study compares six MedSAM adaptation strategies across clean and perturbed prompts on in-domain, close-OOD, and far-OOD datasets, relating performance to layerwise CKA similarity.
Results
Full fine-tuning provides the strongest overall tradeoff, while encoder-only LoRA is the strongest parameter-efficient alternative for preserving far-OOD performance.
Takeaways & Limitations
Robust MedSAM adaptation requires jointly considering prompt-noise exposure, domain shift, and representation preservation, with random 0–100-pixel jitter producing the most reliable adapted models.
Takeaways & Limitations
The CKA observations are not independent, so reported p-values should be interpreted descriptively rather than as strict hypothesis tests.
Abstract
from arXiv · showhide
Foundation models for medical image segmentation, like prompt-based MedSAM, generalize well across domains and modalities, often in zero or few-shot setups. However, their performance depends on the quality of prompts and the adaptation of the models to custom datasets. This work systematically examines how MedSAM generalizes across diverse medical imaging benchmarks, with six adaptation strategies: full-model and encoder-only LoRA, shallow and deep visual prompt tuning (VPT), and decoder-only and full fine-tuning. Models are trained on the International Skin Imaging Collaboration Challenge (ISIC 2018) dataset and evaluated under clean and increasingly noisy prompts on IN and Out-of-Distribution (OOD) datasets: close-OOD PH2 (dermoscopy), far-OOD BUSI (Breast Ultrasound Images Dataset) and CBIS-DDSM (Curated Breast Imaging Subset of the Digital Database for Screening Mammography). We show that adaptation improves performance on IN and close-OOD data but often reduces performance on far-OOD data. Full fine-tuning provides the best tradeoff, while encoder-only LoRA is the strongest parameter-efficient alternative, outperforming standard LoRA and VPT under far-OOD shifts. Using Centered Kernel Alignment (CKA), we show that far-OOD degradation is strongly associated with drift in decoder representations, whereas encoder similarity alone does not explain robustness. This suggests encoder-only LoRA provides stronger robustness than standard LoRA by adapting the encoder to distribution shift in visual features, while preserving the decoder pathway. We further show that random 0-100 pixel jitter on prompts produces more robust and better performing models. We thus conclude that robust MedSAM adaptation requires the combined consideration of prompt noise exposure, domain shift, and representation preservation. We release our code: https://github.com/ImSounic/medsam-vpt
1 Introduction
MedSAM adaptation must be evaluated jointly under prompt noise and domain shift because target-domain improvements can come at the expense of OOD robustness. This work compares adaptation strategies and connects their performance to internal representation changes.
- Challenges: Medical segmentation foundation models face domain shift, weak boundaries, low contrast, and modality-specific appearance changes.MedSAM addresses medical imaging transfer but assumes ROI prompts at inference time, creating additional prompt-sensitivity concerns.
- Research gap: Prior studies have not systematically connected fine-tuning strategy, prompt robustness, and domain generalization in medical segmentation foundation models.The paper examines which strategies improve target-domain performance while preserving generalization and robustness to noisy ROI prompts.
- Study design: The study jointly evaluates adaptation strategy, prompt robustness, and domain generalization rather than treating these factors separately.The comparison includes zero-shot MedSAM, full and decoder-only fine-tuning, full-model and encoder-only LoRA, and shallow and deep VPT.
- Study design: Models are trained and evaluated with controlled bounding-box perturbations to identify robustness under imperfect ROI prompts and shifted test domains.The experiments use clean, fixed-noise, and variable-noise training boxes and multiple evaluation perturbation levels.
2 Related Work
Medical segmentation foundation models offer transfer benefits but remain vulnerable to domain shift and prompt quality, motivating efficient adaptation and layerwise representation comparison. The paper uses CKA to examine how fine-tuning changes MedSAM internally.
- Foundation models: MedSAM transfers across medical imaging modalities, but target-domain fine-tuning can be expensive, require annotations, and reduce generalization ability.These constraints motivate lightweight adaptation methods for medical segmentation foundation models.
- Efficient fine-tuning methods: Prompt tuning, visual prompt tuning, and LoRA reduce training cost while remaining sensitive to task design and prompt quality.These methods seek to preserve the benefits of large pretrained models while adapting fewer parameters.
- Efficient fine-tuning methods: MedSAM combines an image encoder, prompt encoder, and mask decoder, while VPT learns prompts and LoRA updates low-rank trainable weight matrices with the backbone fixed.The mask decoder fuses prompt and image embeddings to produce segmentation masks.
- Domain and prompt robustness: Medical segmentation models often degrade across scanners, hospitals, or modalities, and promptable models can lose accuracy when bounding boxes are perturbed.Prior work addresses these vulnerabilities through prompt refinement or augmentation and by studying which model components govern robustness.
- Feature representation comparison: Layerwise CKA compares internal representations of adapted models with zero-shot MedSAM using the same input images.This approach captures changes that comparisons based only on final embeddings may miss.
3 Methodology
The methodology evaluates MedSAM across controlled prompt perturbations, domain-shifted datasets, and internal representation similarity. Training and evaluation noise regimes are varied to test robustness beyond clean ROI prompts.
- Prompt perturbation: Evaluation uses bounding-box perturbations of 0, 20, 50, 100, and 200 pixels, corresponding to increasing box-size changes relative to the image.Mean side-length increases range from 0% to 19.53%, while mean area increases range from 0% to 280.14%.
- Prompt perturbation: Training uses clean boxes, fixed 20-pixel perturbations, or random perturbations between 0 and 100 pixels.These regimes test whether imperfect-prompt exposure improves localization robustness and whether fixed noise causes overfitting.
- Prompt perturbation: Each trained model is evaluated across five perturbation levels to test generalization from training noise up to more severe 200-pixel inference noise.This directly evaluates robustness under progressively noisier ROI prompts.
- Datasets: ISIC 2018 provides dermoscopic training and in-domain evaluation data, PH2 provides close-OOD dermoscopy, and BUSI and CBIS-DDSM provide stronger modality shifts.The datasets vary in acquisition source, distribution, imaging modality, and target structure.
- Representation analysis: CKA is computed across the image encoder, prompt encoder, and mask decoder against zero-shot MedSAM representations.Higher CKA values indicate stronger preservation of the original MedSAM representation.
4 Experiments and results
Across prompt-perturbation experiments, full fine-tuning and encoder-only LoRA were the most robust overall, while far-OOD performance often declined after adaptation. CKA linked far-OOD performance more closely to decoder-side representation preservation than to encoder similarity.
- At higher evaluation jitter, zero-shot outperformed all adapted models on BUSI and CBIS-DDSM, while full fine-tuning and encoder-only LoRA stayed close to zero-shot far-OOD performance.
- Standard LoRA had the largest ID and close-OOD gains but a far-OOD mean gain of -0.176, whereas encoder-only LoRA reached -0.043.
- Random 0–100-pixel training jitter generally produced the best adapted models, especially for full fine-tuning and encoder-only LoRA.
- Full fine-tuning achieved the best far-OOD mean gain (-0.018) and best overall mean gain (-0.001) across adaptation methods.
- Encoder-only LoRA was the strongest parameter-efficient alternative, outperforming standard LoRA and VPT under OOD shift.
- Far-OOD gains correlated strongly with decoder-side CKA, while encoder-layer CKA was weakly negative, indicating encoder similarity alone did not explain robustness.
- Pooled CKA observations were non-independent, so reported p-values are descriptive rather than strict hypothesis tests.
5 Conclusion
MedSAM adaptation was evaluated under joint domain shift and noisy bounding-box prompts. Full fine-tuning offered the strongest overall tradeoff, while encoder-only LoRA was the best parameter-efficient alternative, and variable 0–100 pixel jitter produced reliable adapted models.
- Prompt robustness depends strongly on both adaptation strategy and training jitter regime.
- Full fine-tuning provides the strongest overall tradeoff across adaptation methods.
- Encoder-only LoRA is the best parameter-efficient alternative and preserves far-OOD performance better than several competing methods.
- Variable 0–100 pixel jitter training produces the most reliable adapted models.
- Because training is limited to dermoscopy, far-OOD results reflect combined modality and target-structure shift and should primarily guide method rankings.
A How hard of a task does each evaluation jitter represent?
The same pixel-level evaluation jitter creates different effective prompt difficulty across datasets. Far-OOD datasets experience substantially larger relative ROI expansions, especially for small lesions in CBIS-DDSM.
- Fixed pixel jitter does not represent the same challenge across datasets.ISIC 2018 and PH2 show relatively moderate bounding-box area increases, whereas BUSI and CBIS-DDSM show much larger expansions.
- Jitter-aware evaluation is necessary for assessing robustness under noisy ROI prompts.
B Boundary robustness under evaluation jitter
Boundary error generally worsens as evaluation jitter increases, but degradation varies across adaptation methods and datasets. Far-OOD performance is especially vulnerable for standard LoRA and VPT.
- Adapted models achieve low median HD95 values on ISIC 2018 and PH2 with clean bounding-box prompts.On far-OOD datasets, method differences become larger.
- Encoder-only LoRA, decoder-only fine-tuning, and full fine-tuning show relatively low far-OOD boundary errors.
- Standard LoRA and VPT variants have wider HD95 interquartile ranges, particularly on BUSI and CBIS-DDSM.
- Boundary error generally increases as evaluation jitter expands the bounding box, with degradation rates varying by method and dataset.The largest errors occur for standard LoRA and VPT on far-OOD datasets.
C How does CKA similarity correspond to segmentation performance
CKA similarity is related to segmentation performance, but the relationship depends on the network region being compared. Decoder- and output-side preservation aligns more strongly with far-OOD robustness than encoder similarity alone.
- Higher decoder-side and output-side CKA similarity is associated with better far-OOD performance.The strongest correlations appear in the far-OOD regime across datasets and jitter levels.
- Encoder similarity alone does not track robustness as reliably as decoder and output representations.
- Figure 4 compares performance gain over zero-shot with overall CKA similarity separately for each adaptation method.Each point represents one domain and training-jitter configuration, aggregating ID, near-OOD, and far-OOD settings.
- Figure 5 repeats the CKA-performance comparison by regime across four datasets, three training jitters, and five evaluation jitters.
- Fine-tuning largely preserves encoder representations but causes sharper similarity drops in decoder and token components.VPT and decoder-only tuning show particularly large decoder-side changes, while LoRA variants and full fine-tuning better preserve decoder structure.
D Degradation of performance depending on bounding box jitter
Bounding-box jitter degrades Dice across all methods, but variable-jitter training and adaptation strategy substantially affect robustness. Full fine-tuning and encoder-only LoRA degrade most slowly, while heavy jitter causes severe far-OOD failures.
- All methods lose Dice as bounding-box jitter increases, but their degradation rates differ.
- Full fine-tuning and encoder-only LoRA have the shallowest degradation curves, especially after variable-jitter training.
- Standard LoRA and VPT decline more quickly under stronger bounding-box perturbations.
- Far-OOD performance is more sensitive to prompt noise than in-domain performance across datasets.
- Under heavy jitter, CBIS-DDSM Dice collapses below 0.2 for every method.
- Training with variable prompt noise improves robustness to realistic bounding-box perturbations.
E Segmentation performance breakdown across jitter levels
The reported jitter analyses organize segmentation performance and representational similarity across training and evaluation noise levels. Results are summarized by dataset, training setup, method, evaluation jitter, Dice, and CKA.
- Table 5 reports Dice performance and CKA at zero jitter, averaged evaluation jitters, and 200 px jitter.
- CKA columns identify the best and second-best adapted methods within each dataset and training-jitter setting.
- Table 6 reports Dice for models trained with 0 px, 20 px, or random 0–100 px jitter and evaluated from 0–200 px.
- Performance values are means over 3 seeds, with underlining and bold marking within-block and across-setting best values.
- The tables are indexed by dataset, training jitter, method, and evaluation jitter levels E0, E20, E50, E100, and E200.