Source-linked AI summary
IMVS: Interactive Medical Volume Segmentation with Test-Time Adaptation - A New Method for Annotating Radiology Datasets
Abhilaksh Singh Reen, Kushal Borkar, Ritvik Mahapatra
TL;DR
Slice-wise interactive methods ignore inter-slice continuity, while fixed volume propagators can drift under clinical distribution shift, leaving repeated correction effort. IMVS combines scribble-driven online adaptation with frozen mask propagation; across datasets, it matches strong baselines in quality while reducing interaction and annotation effort.
Problem
Slice-wise methods ignore inter-slice continuity, while fixed 3D and video propagators can drift under distribution shift, low contrast, and pathology and require re-prompting.
Source: §1 · p. 2Method
IMVS closes the loop by combining an online-adapted 2D SMA trained from scribbles, a frozen VMT that propagates corrected masks, and soft teacher-student alignment.
Source: §1 · p. 2Results
Across datasets, IMVS matches strong baselines in final accuracy and reaches target with fewer interactions; it is 14.4× faster than Manual With Copy.
Source: §4.3 · p. 7, §4.2 · p. 7Takeaways & Limitations
Corrections are reused across subsequent slices through adaptation and propagation, supporting an annotation-efficiency approach rather than a new segmentation primitive.
Source: §1 · p. 2, §3.4 · p. 5IMVS segments one target per pass; multi-label volumes require sequential single-label sessions, and gains are smallest on tortuous or discontinuous structures.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Annotating large radiology datasets is bottlenecked by the manual effort of delineating structures slice-by-slice in 3D volumes. Interactive methods reduce this effort but stay interaction-inefficient: slice-wise methods (including many foundation models) ignore inter-slice continuity, while 3D and video-based methods propagate a prompt with a \emph{fixed} propagator that never adapts to the target volume, so it drifts on low-contrast or pathological structures and must be re-prompted. We present IMVS, a human-in-the-loop annotation framework that composes three components into a closed loop rather than a new segmentation primitive: a lightweight 2D Slice Mask Adapter (SMA) fine-tuned online from user scribbles, a frozen Volume Mask Tracker (VMT) that propagates corrected masks across adjacent slices, and a soft teacher--student alignment that limits forgetting. The SMA is backbone-agnostic (UNet++, DeepLabV3, TransUNet). Across 8 public CT/MRI datasets, IMVS matches strong interactive baselines in quality while sharply cutting annotation effort: $14.4\times$ faster than a proficient copy-based manual workflow ($22.3\times$ over naive manual), $4.6\times$ over slice-wise and $1.9\times$ over 3D interactive methods. MedSAM2 and ScribblePrompt stay competitive or stronger on well-delineated organs; IMVS's advantage is largest on challenging targets and on interaction efficiency. Source code and Demo Video: https://github.com/AbhilakshSinghReen/imvs.
1 Introduction
Interactive volume methods either refine slices without using continuity or propagate prompts with fixed models that can drift. IMVS composes online adaptation and mask propagation into a closed loop that reuses corrections across slices.
1 Introduction
- IMVS couples mask propagation with online adaptation, so each correction updates a lightweight per-slice model for upcoming slices instead of being repeated across the volume.The paper frames this as a human-AI collaboration system for annotation efficiency, not a new segmentation primitive.Source: §1 · p. 2
- The closed loop pairs a lightweight, online-adapted 2D Slice Mask Adapter with a frozen Volume Mask Tracker and soft teacher-student alignment.Backpropagation is confined to the SMA, and the SMA is validated with UNet++, DeepLabV3, and TransUNet.Source: §1 · p. 2
- The empirical study covers 8 CT/MRI datasets and a 9-annotator user study to measure efficiency gains and characterize where IMVS helps.The supplied passage also reports that the SMA is backbone-agnostic across UNet++, DeepLabV3, and TransUNet.Source: §1 · p. 2
2 Related Work
Prior interactive segmentation uses boxes, clicks, or scribbles, with methods ranging from iterative refinement to medical foundation and volume-oriented models. IMVS is positioned as an interactive, weakly supervised adaptation-and-propagation approach that amortizes corrections across slices.
2 Related Work
- Interactive segmentation takes boxes [24], clicks, or scribbles; f-BRS and RiTM use iterative inference-time refinement, while iSegFormer applies a transformer backbone to medical images.Source: §2 · p. 2
- Foundation models such as SAM and MedSAM [14] generalize across targets but degrade under medical distribution shift and are typically applied per-slice; PRISM and nnInteractive [5] address volumes more directly.The cited taxonomy places IMVS among scribble-driven, iteratively refined methods that amortize interactions across slices rather than maximize single-mask accuracy.Source: §2 · pp. 2–3
- IMVS uses user scribbles as its dominant adaptation signal, with teacher and propagation terms stabilizing continual refinement and soft alignment guarding against forgetting.Its division of labour keeps the attention-based, memory-bank propagator frozen and recovers its errors through online SMA adaptation.Source: §2 · p. 3
3 Method
IMVS alternates scribble-driven adaptation of a lightweight 2D model with frozen mask propagation, reusing corrections across slices while preserving source-domain knowledge. Its loop combines user corrections with propagation and teacher signals for stabilization.
3 Method
- IMVS alternates interactive correction and automatic propagation, updating the student from scribbles so each correction guides segmentation of slices ahead.The annotator starts where the target first becomes clearly visible; propagation runs for up to K=17 slices before a correction is required.Source: §3 · pp. 3–4
3.1 Slice Mask Adapter (SMA)
- The SMA is an interchangeable 2D segmentation network with a frozen teacher and adaptive student; scribbles are rasterized into two guidance channels and fused with the image.UNet++, DeepLabV3+, and TransUNet are evaluated, with UNet++ adopted as default.Source: §3.1 · p. 4
3.2 Volume Mask Tracker (VMT)
- The frozen VMT propagates the accepted mask using recent feature-mask pairs, with attention over preceding slices and a six-slice memory buffer.Short-term attention covers the two preceding slices, while long-term attention supports appearance consistency and drift prevention.Source: §3.2 · p. 4
3.3 Online Adaptation Objective
- Online adaptation combines VMT and teacher consistency, cross-entropy against the user-corrected mask, and soft alignment of student and teacher batch-normalization parameters.The student updates only when total loss exceeds γ=0.05; the frozen propagator prevents pseudo-label drift from compounding, and the adapted student carries to subsequent volumes as a warm-start.Source: §3.3 · p. 5
3.4 Human-AI Interaction Workflow
- Each correction is incorporated by the SMA and reused through propagation, making user input guidance for future model behavior as well as local error correction.The workflow has the annotator intervene when propagated masks become unacceptable, rather than refine every slice.Source: §3.4 · pp. 5–6
3.5 Training and Implementation
- Training uses three consolidated datasets, while five other datasets are held out for zero-shot evaluation; training scribbles are simulated from ground truth.The VMT is pretrained on YouTube-VOS 2019 and fine-tuned on the consolidated set; the SMA uses ImageNet-initialized backbones.Source: §3.5 · p. 6
4 Experiments and Results
Across eight CT/MRI datasets, IMVS reduces the interactions needed to reach matched quality and cuts annotation time, while its advantage is smaller on discontinuous structures. Ablations show that online SMA adaptation is central to the efficiency gains, whereas adapting the VMT adds cost without improving accuracy.
4.1 User Study and Stopping Criterion
- Nine residents used a shared qualitative acceptance criterion, with aggregated masks setting stopping thresholds of DSC 0.885, NSD 0.894, and HD95 4 mm across eight CT/MRI benchmarks.Mean pairwise inter-annotator DSC was 0.89 ± 0.04.Source: §4 · p. 6, §4.1 · p. 7
4.2 Annotation Efficiency
- 14.4× faster than Manual With Copy and 22.3× faster than naive Manual, IMVS was fastest on 5 of 8 datasets but not uniformly best.It was most efficient by interaction count on 4 of 8 datasets and stayed within one interaction of MedSAM2 elsewhere.Source: §4.2 · p. 7
4.3 Segmentation Quality
- Final quality clustered across methods once enough interactions were spent, so IMVS reached the target sooner rather than surpassing the quality plateau.IMVS is reported to match strong baselines in accuracy, not to be best in raw quality on every dataset.Source: §4.3 · p. 7
- 7.1 interactions on average reached DSC ≥0.90, versus 10.1 for MedSAM2 and 13.1 for iSegFormer; IMVS needed the fewest on every dataset.Table 2 reports interactions per dataset at the matched-quality threshold.Source: §4.3 · p. 7, Table 2 · p. 8
- On tortuous hepatic vessels, VMT tracked only ∼4–6 slices versus up to 17 on well-defined organs; SMA adaptation typically recovered tracking with a single scribble.Every method degraded on the vessels.Source: §4.3 · p. 8
4.4 Ablation: Contribution of Each Component
- Online SMA adaptation cut interactions from 18 to 4 and raised DSC from 0.86 to 0.92, while adapting VMT added no accuracy and increased VRAM 6× and latency 14×.The method extended tracked span from 3 to 12 slices; UNet++ was the default backbone, memory-bank size 6 was optimal, and K=17 balanced correction frequency against propagation failures.Source: §4.4 · p. 8, Table 3 · p. 8
5 Scope and Limitations
IMVS handles one target per pass; multi-label volumes require sequential single-label sessions, and native multi-object propagation remains future work.
5 Scope and Limitations
- IMVS handles one target per pass; multi-label volumes require sequential single-label sessions, with native multi-object propagation deferred to future work.Efficiency also declines on tortuous or discontinuous structures, where more interactions are needed.Source: §5 · p. 8
6 Conclusion
IMVS combines online-adapted SMA, frozen VMT propagation, and soft teacher-student alignment in a closed annotation loop. Its reported benefit is annotation efficiency through interaction reuse and online adaptation.
6 Conclusion
- IMVS combines an online-adapted 2D SMA, frozen VMT propagator, and soft teacher-student alignment in a closed loop to improve annotation efficiency.The paper identifies topology-aware and multi-object propagation as future extensions.Source: §6 · p. 9