Source-linked AI summary
Anatomy-Aware Promptable Segmentation with Online Interactive Training for AUTOPET V
Pablo Lozano-Jimenez, Sergio Romero-Tapiador, Ruben Tolosana
TL;DR
Whole-body FDG/PSMA PET/CT lesion segmentation remains difficult because models face domain shifts, low-contrast lesions, and limited clinical alignment. The paper develops a two-stage nnU-Net-based system with shared-head organ supervision, online scribble refinement, and tracer-aware routing. Organ supervision provides the most stable backbone, interactive refinement improves segmentation over successive prompts, and PSMA-specific training gives the strongest tracer-wise performance.
Problem
PET/CT segmentation remains challenged by domain shifts, low-contrast lesions, and divergence from clinical perspectives, while manual assessment has inter-observer variability.
Method
The system incrementally combines nnU-Net pre-training, shared-head lesion–organ supervision, online scribble learning, and a classifier routing studies to combined or PSMA-specific models.
Results
Organ supervision gives the best trade-off between accuracy and robustness, interactive refinement steadily improves segmentation, and PSMA-specific training with tracer classification gives the best tracer-wise performance.
Takeaways & Limitations
A shared anatomy-aware backbone, online prompt refinement, and tracer-specific routing form the paper’s supported approach to robust multi-tracer PET/CT segmentation.
Abstract
from arXiv · showhide
We present an anatomy-aware, promptable model for whole-body lesion segmentation in FDG and PSMA PET/CT, developed for the AUTOPET V challenge. The proposed method is built as family of nnU-Net-based models and trained in two stages: i) a pre-training stage that produces a strong initial segmentation, and ii) an online interactive stage that learns to exploit scribble prompts, refining the prediction over successive interactions. Anatomical context is incorporated through organ supervision using a single shared head that predicts lesions and organs from the same features, which reduces false positives arising from physiological uptake. Also as the tracer (i.e., FDG/PSMA) is not provided at inference, we add a tracer classifier based on image processing and a random forest over coronal MIP features, routing each study to a combined FDG+PSMA model or to a PSMA-specific model. Across four-fold cross-validation the organ-supervised model achieves the best and most stable performance, the interactive stage improves the Dice score monotonically with each prompt, and PSMA-specific training yields the strongest tracer-wise results.
1 Introduction
AUTOPET V targets accurate whole-body lesion segmentation with fewer, clinically realistic interactions, addressing persistent challenges in manual and AI-assisted PET/CT analysis.
- Motivation: AI-assisted segmentation aims to reduce manual assessment problems including inter-observer variability and difficulty detecting complex patterns.Existing models still struggle with domain shifts, low-contrast lesions, and alignment with clinical perspectives.
- Challenge setting: The challenge uses a large heterogeneous dataset containing FDG and PSMA studies to test model generalizability.The dataset includes 1,014 FDG studies and 597 PSMA studies.
- Challenge setting: AUTOPET V introduces human-in-the-loop segmentation intended to improve lesion delineation while reducing interaction effort.The edition reduces allowed interactions from 10 to 5 and adds specialist-marked difficult lesions for clinical validation.
- Contribution: The proposed method builds on nnU-Net and combines PET/CT preprocessing with ideas from prior AUTOPET editions and related literature.The authors also make the method publicly available through GitHub.
2 Methods
The method incrementally develops nnU-Net-based PET/CT models with organ supervision, modality-aware processing, tracer routing, and online scribble-based refinement.
- Data and validation: The dataset combines FDG and PSMA PET/CT studies, and validation uses four class-stratified folds balanced by tracer and diagnostic category.FDG categories include negative, melanoma, lymphoma, and lung cancer; PSMA categories include negative and three scanners.
- Training strategy: Training proceeds in two phases: pre-training produces an initial segmentation, while interactive learning incrementally incorporates prompts and model errors.The interactive stage is fine-tuned from Phase-1 backbones rather than trained from scratch.
- Organ supervision: Version 3 uses one encoder and decoder with a shared lesion–organ output head, forcing lesion segmentation features to encode anatomical context.Organ labels cover nine structures obtained with TotalSegmentator.
- Modality-aware architecture: Version 4 separates PET and CT encoding and uses three decoders for organ segmentation, lesion detection, and fused lesion segmentation.The modality-specific encoders have independent weights, while the decoders perform task-specialized predictions.
- Tracer routing: Version 5 adds PSMA-only training and a tracer classifier because inference studies do not identify whether FDG or PSMA was used.The classifier uses image-processing features from coronal MIPs and a random forest.
- Tracer routing: Studies are routed to the PSMA-specific model only when classifier confidence exceeds 65%; otherwise, the combined model provides the fallback.This routing strategy is designed to make uncertain cases degrade gracefully.
- Interactive refinement: Phase-2 checkpoints are produced per tracer route by fine-tuning selected Phase-1 models to exploit scribble prompts from their initial predictions.The FDG route fine-tunes the best-performing combined-model fold for FDG, while routing determines the applicable tracer path.
3 Results
Across Phase 1, organ supervision provided the strongest and most stable backbone, while tracer-specific routing and PSMA training produced the best tracer-wise results. In Phase 2, interactive scribbles steadily improved segmentation, with tracer-dependent prompt widths.
- Phase-1 results: Organ supervision in Version 3 produced the largest single gain over Version 2 by incorporating anatomical context into lesion prediction.The shared features predict lesions and organs, helping distinguish physiological uptake from disease, especially in FDG studies.
- Phase-1 results: Version 3 was selected for Phase 2 because it achieved the best final metric and lower cross-fold variability than Version 4.Lower variance made the selected fold more representative for interactive fine-tuning.
- Phase-1 results: Version 5 yielded the best results overall, with PSMA-specific training outperforming Version 3 trained on the combined FDG+PSMA dataset for PSMA studies.The comparison supports routing PSMA cases to a dedicated PSMA model.
- Phase-2 results: Interactive scribbles improved segmentation monotonically for both tracers during Phase 2.Phase 2 fine-tuned the best Phase-1 fold for each tracer route because all five folds could not be trained interactively.
- Phase-2 results: The optimal scribble width differed by tracer, with wider guidance matching the larger and more diffuse FDG-avid lesions.Narrower guidance localizes corrections, whereas excessive width can extend beyond lesion boundaries and encourage over-segmentation.
4 Conclusion
The paper presents a two-stage anatomy-aware and promptable PET/CT segmentation system combining organ supervision, interactive scribble refinement, and tracer-specific routing. The reported design balances accuracy and robustness while improving tracer-wise performance.
- Conclusion: The system combines organ supervision, online interactive training with simulated scribbles, and tracer-specific routing through a lightweight classifier.These components form a two-stage approach for multi-tracer PET/CT lesion segmentation.
- Conclusion: Organ supervision through a shared head provides the best reported trade-off between accuracy and robustness and serves as the backbone.Interactive refinement improves segmentation steadily with minimal user effort.
- Conclusion: PSMA-specific training with a generalizable tracer classifier delivers the best tracer-wise performance while degrading gracefully on uncertain cases.