Source-linked AI summary
A Few Cases Are All You Need: An Empirical Study of Annotation-Efficient LoRA Fine-Tuning of MedSAM3
Sachin Dudda Nagaraju, Bendik Skarre Abrahamsen, Ashkan Moradi, Mattijs Elschot
TL;DR
Expert annotations for medical segmentation are costly and scarce, while zero-shot medical foundation models remain unreliable for some structures. This study adapts MedSAM3 with LoRA using 1–10 cases across abdominal and cardiac tasks, finding that 10 cases can produce competitive segmentation with faster training.
Problem
The number of expert-annotated cases needed for clinically useful adaptation remains insufficiently studied despite costly annotations and unreliable zero-shot segmentation for difficult structures.
Method
The study systematically adapts MedSAM3 with LoRA across five abdominal organs, CT and MRI, and cardiac segmentation, using fixed protocols and 1, 2, 5, or 10 cases.
Results
With 10 annotated cases, the models are competitive with specialist systems using over 100× more data, generalize to cardiac segmentation, and train 2–3× faster than nnU-Net.
Takeaways & Limitations
As few as ten expert annotations can support clinically useful, parameter-efficient segmentation across abdominal and cardiac use cases within the evaluated scope.
Abstract
from arXiv · showhide
Medical image segmentation is essential for clinical workflows such as treatment planning and disease assessment. While specialist tools like TotalSegmentator and MRSegmentator achieve strong performance, they require large annotated datasets for training. Medical foundation models offer a promising alternative through large-scale pretraining that reduces the annotation burden for new tasks, but zero-shot performance remains limited. Parameter-efficient adaptation via Low-Rank Adaptation (LoRA) enables efficient specialization with few trainable parameters, but a key question remains: how many expert-annotated cases are needed to achieve clinically useful segmentation performance? We address this by adapting MedSAM3 with LoRA for five abdominal organs (liver, kidneys, spleen, gallbladder, and pancreas) in CT and MRI using only 1, 2, 5, and 10 annotated cases, evaluating on AMOS22 dataset. With just 10 cases, models achieve performance competitive with specialist systems trained on orders of magnitude more data. Notably, this includes reliable gallbladder segmentation (Dice 0.68 CT, 0.59 MRI) where existing tools fail almost completely (Dice 0.0004), while remaining within 5--10% of MRSegmentator for liver, kidneys, and spleen using over 100 times fewer annotations. Furthermore, external validation on the Whole Heart Segmentation dataset shows that the approach extends to cardiac segmentation, a use case beyond the scope of TotalSegmentator (MRI) and MRSegmentator, achieving competitive left ventricle (LV) performance with only 10 annotated cases. Training requires only3--5,hours per organ on a single GPU, approximately 2--3 times faster than nnU-Net. These findings suggest that ten annotated cases are sufficient for clinically useful segmentation, effectively reducing bottlenecks for both image annotation and training time.
1 Introduction
This study examines whether LoRA adaptation of MedSAM3 can reduce the expert-annotation burden for medical image segmentation. It evaluates annotation efficiency across abdominal organs and modalities, compares against established systems, and tests cross-center cardiac generalization.
- Motivation: Medical image segmentation supports treatment planning, surgical guidance, disease assessment, and longitudinal monitoring, but expert volumetric annotation remains expensive, time-consuming, and difficult to scale.These annotation demands limit model development across organs, modalities, and clinical sites.
- Motivation: Zero-shot medical foundation models remain unreliable for small or low-contrast structures, while full fine-tuning is computationally expensive and prone to overfitting with limited annotation.Specialist systems perform strongly but depend on large curated datasets, restricting deployment in low-data settings.
- Study design: The study adapts MedSAM3 with LoRA for five abdominal organs in CT and MRI using 1, 2, 5, and 10 annotated cases, evaluated on AMOS22.The comparisons include zero-shot foundation models, TotalSegmentator, MRSegmentator, and nnU-Net.
- Contributions: At 10 cases, the approach achieves competitive accuracy against fully supervised specialist systems with over 100× fewer annotations and evaluates cross-center cardiac generalization on WHS.The cardiac experiments assess generalisability beyond tasks covered by the supervised specialist systems.
2 Related Works
Related work spans strong but annotation-intensive specialist segmentation systems, medically pretrained foundation models, and few-shot parameter-efficient adaptation. However, prior studies use heterogeneous experimental settings, leaving unclear how performance changes with the number of fully annotated patient cases.
- Specialist segmentation systems: 1,204 CT scans trained TotalSegmentator to segment 104 anatomical structures, illustrating the scale of specialist-tool supervision.nnU-Net remains a widely used baseline because it automatically adapts preprocessing, architecture, and training to each dataset, but requires large expert-annotated datasets and task-specific training.
- Medical foundation models: More than 1.5 million image–mask pairs supported MedSAM’s large-scale medical pretraining, addressing variability in SAM’s zero-shot medical performance.Later methods including MA-SAM and Medical SAM Adapter incorporated parameter-efficient adaptation, as described in the supplied passage.
- Few-shot adaptation: A single labelled volume can suffice for lightweight SAM adaptation, while combining data synthesis with LoRA has helped low-data brain tumour and abdominal CT segmentation.These findings motivate parameter-efficient adaptation for few-shot medical segmentation.
- Open methodological gap: Prior few-shot studies used 2D slices, dataset fractions, or fixed support sets, so performance scaling with fully annotated patient cases remains unclear.Recent SAM3-based methods also mostly target specific organs, modalities, or larger training settings.
3 Methodology: Annotation-Efficiency Study Design
The study measures how MedSAM3 segmentation performance changes with few expert-annotated cases rather than introducing a new architecture. It evaluates LoRA-adapted models across abdominal CT/MRI tasks and extends the 10-case protocol to cardiac segmentation.
- Study design: MedSAM3 is adapted with LoRA while pretrained weights remain frozen and only low-rank parameters are updated.LoRA uses rank r ≪ min(m, n), enabling task-specific adaptation with few trainable parameters.
- Protocol selection: The development phase evaluates ranks r ∈ {4, 8, 16} at the 10-case budget using validation Dice monitored every five epochs.The validation set contains 30 CT and 30 MRI cases held out from TotalSegmentator and is not used for early stopping or learning-rate scheduling.
- Study design: Abdominal experiments train organ- and modality-specific binary models for five organs in CT and MRI using 1, 2, 5, or 10 annotated cases.Models are evaluated on the independent AMOS22 benchmark to assess generalization.
- External validation: Cardiac evaluation tests models trained on 10 annotated CT and MRI cases from selected centers against held-out centers for left- and right-ventricle segmentation.This experiment examines whether the annotation-efficiency trend extends beyond abdominal anatomy using the WHS dataset.
- Model adaptation: LoRA is applied across MedSAM3’s vision, text, geometry, DETR, and mask-decoder components, with the target organ name supplied as the text prompt.Training predicts a binary mask and uses combined Dice and focal loss.
- Protocol selection: The final protocol fixes LoRA rank r = 16 and 30 training epochs, uses the final checkpoint directly, and applies the same procedure across organs and modalities.No validation-based model selection is performed in Phase 2; Dice similarity coefficient is used for evaluation.
4 Experiments and Results
Experiments established a fixed LoRA protocol and evaluated annotation efficiency, specialist-tool comparisons, computational cost, and cross-center cardiac generalisation. MedSAM3+LoRA reached competitive segmentation performance with 10 annotated cases while using less data and training time than specialist approaches.
- Phase 1: Protocol selection: LoRA rank r=16 was selected for final evaluation, and all six kidney, liver, and spleen modality combinations plateaued by epoch 30 without observed overfitting.The fixed protocol used 30 epochs and did not require validation-set or per-organ epoch tuning.
- Phase 2: Annotation efficiency: MedSAM3+LoRA Dice improved consistently from 1 to 10 annotated cases, with the largest gains between 1 and 5 cases and diminishing returns thereafter.The comparison covered all five abdominal organs in CT and MRI, against nnU-Net budgets of 10, 50, 100, and full annotation sets.
- Phase 2: Annotation efficiency: At 10 cases, nnU-Net reached Dice 0.11 for MRI spleen and 0.39 for CT pancreas, while MedSAM3+LoRA remained substantially stronger at the same budget.This identifies 10 cases as a practically meaningful operating point for foundation-model adaptation.
- Phase 2: Specialist-tool comparison: With 10 cases, MedSAM3+LoRA achieved gallbladder Dice 0.684 on CT and 0.633 on MRI, whereas TotalSegmentator and MRSegmentator produced Dice ≤0.0004.The specialist tools used over 1200 annotated scans in total, compared with 10 cases per organ and modality for MedSAM3+LoRA.
- Phase 2: Computational efficiency: MedSAM3+LoRA converged in 30 epochs using approximately 3–5 GPU-hours per run, compared with nnU-Net’s 8.5–11 GPU-hours per organ.All experiments used a single NVIDIA A40 GPU with 48 GB VRAM.
- Phase 3: Cardiac generalisation: WHS evaluation tested same-modality 10-shot models across centers, using 10 training cases and 20 held-out cases for cross-center generalisation.CT used center A for training and center B for testing; MRI used center E for training and centers C and D for testing.
5 Conclusion
MedSAM3+LoRA achieves annotation-efficient medical image segmentation with 10 annotated cases, matching specialist tools on large organs and reliably segmenting the gallbladder. It also trains faster than nnU-Net while updating only 2.15% of parameters.
- 10 annotated cases match specialist tools trained on over 100× more data for large organs.The comparison is presented as evidence of annotation efficiency.
- Dice 0.68 CT and 0.59 MRI demonstrate reliable gallbladder segmentation, whereas evaluated specialist tools achieve Dice ≤0.0004.The gallbladder result is highlighted as a setting where specialist tools fail almost completely.
- 2–3× faster training than nnU-Net is achieved by updating just 2.15% of parameters.