Source-linked AI summary
Domain Adaptive Relational Reasoning for 3D Multi-Organ Segmentation
Shuhao Fu, Yongyi Lu, Yan Wang, Yuyin Zhou, Wei Shen, Elliot Fishman, Alan Yuille
TL;DR
3D multi-organ segmentation can degrade when training and testing scans come from different scanners or protocols. DARR addresses this with super-resolution standardization, joint segmentation and jigsaw-based relational learning, and test-time puzzle adaptation, reporting a 29.60% average DSC improvement on target datasets without target-domain training data.
Problem
Domain shifts from different CT scanners or acquisition protocols can substantially degrade 3D multi-organ segmentation performance.
Method
DARR jointly trains segmentation with super-resolution and jigsaw-based spatial-relation learning, then performs puzzle training at test time.
Results
Over 29% improvement in mean DSC is reported across multiple target datasets, while the conclusion reports superiority on five public datasets.
Takeaways & Limitations
Capturing stable spatial relations among organs can bridge the domain gap between multi-site medical data without using target-domain data during training.
Abstract
from arXiv · showhide
In this paper, we present a novel unsupervised domain adaptation (UDA) method, named Domain Adaptive Relational Reasoning (DARR), to generalize 3D multi-organ segmentation models to medical data collected from different scanners and/or protocols (domains). Our method is inspired by the fact that the spatial relationship between internal structures in medical images is relatively fixed, e.g., a spleen is always located at the tail of a pancreas, which serves as a latent variable to transfer the knowledge shared across multiple domains. We formulate the spatial relationship by solving a jigsaw puzzle task, i.e., recovering a CT scan from its shuffled patches, and jointly train it with the organ segmentation task. To guarantee the transferability of the learned spatial relationship to multiple domains, we additionally introduce two schemes: 1) Employing a super-resolution network also jointly trained with the segmentation model to standardize medical images from different domain to a certain spatial resolution; 2) Adapting the spatial relationship for a test image by test-time jigsaw puzzle training. Experimental results show that our method improves the performance by 29.60% DSC on target datasets on average without using any data from the target domain during training.
1 Introduction
DARR targets performance degradation when 3D multi-organ segmentation models encounter scans from different scanners or protocols. It learns transferable organ spatial relations through jigsaw puzzles, standardizes resolution with super-resolution, and adapts relations at test time.
- Domain shift: 43.12% performance drop in DSC occurs when a V-Net trained in-house is directly tested on the MSD spleen dataset.The datasets differ in acquisition parameters, including pitch/table speeds and reconstruction settings.
- Motivation: Stable relative organ locations provide a transferable cue for unsupervised domain adaptation across CT datasets.DARR uses organ-location distributions and jigsaw recovery to learn spatial relations that are easier to transfer than appearance-dependent features.
- Approach: DARR jointly learns segmentation, spatial relations, and resolution standardization through a puzzle task and super-resolution network.The super-resolution component produces high-resolution outputs from low-resolution inputs while jointly training with segmentation and jigsaw objectives.
- Test-time adaptation: Test-time relative-position training adapts the model to unknown domains without retraining on the source domain.The method optimizes the jigsaw task on an individual test case to adapt the learned relational representation.
- Results: Over 29% improvement in mean DSC is reported when adapting to multiple target datasets.The reported benefits include a shared medical-image constraint, spatial-resolution mapping, and more robust low-level features.
2 Related Work
Related work spans domain-invariant feature learning, image adaptation, self-training, and self-supervised spatial tasks. DARR builds on jigsaw-based relational learning for 3D organ segmentation while addressing domain shift through its adaptation framework.
- Domain adaptation: Unsupervised domain adaptation commonly minimizes cross-domain differences at the feature or image level.Prior approaches include image-to-image translation methods based on modified CycleGAN frameworks.
- Self-training: Self-training methods use source-trained teachers and target pseudolabels, but domain shift can limit their effectiveness for adaptation.A cited semisupervised abdominal segmentation method trains a student on source data and target pseudolabels without delicately addressing domain shift.
- DARR: DARR combines a super-resolution network, V-Net segmentation, and a puzzle module for learning spatial relations among patches.The architecture overview identifies these three components as the framework’s core modules.
- Self-supervision: Jigsaw puzzles are among several self-supervised tasks used to learn patch relationships and other visual structure.Related tasks include patch relative positions, local context, color, and recognizing scans from the same patient.
3 Method
The method jointly trains super-resolution, segmentation, and patch-location reasoning, then uses the puzzle objective to adapt feature extraction during testing. Its design aligns image resolution and exploits stable organ spatial structure.
- Architecture: DARR contains a super-resolution network, a V-Net segmentation network, and a puzzle module for patch spatial relations.The super-resolution module upsamples low-resolution images before shared feature extraction and segmentation.
- Patch representation: The model partitions each image into W × H × L patches whose original coordinates define the location labels.Randomly permuted patches and correspondingly permuted labels form the puzzle-learning input.
- Training objectives: The super-resolution module minimizes mean square error between reconstructed and original patches.The reconstruction objective trains the output patch to match the original high-resolution patch.
- Training objectives: The segmentation loss uses ground-truth masks, while the puzzle loss classifies each patch’s correct location.Segmentation and super-resolution losses use labeled training data; the puzzle loss can also use testing data without manual labels.
- Test-time adaptation: At test time, minimizing the puzzle loss for a few iterations adapts the feature extractor to an unseen target image or domain.The adaptation relies on reasoning about organ spatial relations through the self-supervised task.
- Relational reasoning: Medical organs’ consistent locations and relative scales motivate solving relative positions from arbitrarily cropped 3D patches.The puzzle module flattens shared encoder features, concatenates them in permuted order, and predicts location probabilities with a softmax.
4 Experiments
Experiments evaluate DARR across five target datasets, compare it with established adaptation strategies, and isolate the contributions of its puzzle and super-resolution components. The results show strong domain-generalization performance and qualitative improvement in distinguishing spatially related organs.
- Experimental setup: DARR is trained on 90 high-resolution cases and adapted to five public target datasets covering multi-organ and single-organ segmentation.The Synapse evaluation includes eight abdominal organs, while the remaining datasets evaluate individual target organs.
- Comparisons: The study compares DARR against GAN-based, self-learning-based, and meta-learning-based domain adaptation methods, using Lower Bound and Upper Bound references.Upper Bound models are trained with target-domain ground-truth labels, whereas Lower Bound models are tested directly without DARR adaptation.
- Quantitative results: 29.60% average DSC improvement over Lower Bound results is reported, with DARR outperforming all compared methods by a large margin.The comparison is reported across the target datasets in Table 1.
- Quantitative results: DARR even exceeds Upper Bound results on the Synapse, MSD Liver, and MSD Spleen datasets without using target-domain data during training.The authors interpret this as evidence that learned organ spatial relations can bridge domain gaps across multi-site data.
- Comparisons: Self-learning lowers segmentation performance relative to its teacher, while MASF transfers mainly to large organs and performs poorly on smaller organs.These comparisons indicate that simple pseudolabeling and the evaluated meta-learning approach do not adequately address the multi-organ domain gap.
- Ablation study: The ablation compares VNET-Puzzle, VNET-SR, and full DARR, while qualitative results show DARR distinguishing left from right kidney after learning spatial relations.VNET-Puzzle adds patch-relation learning, VNET-SR adds super-resolution, and DARR combines both components.
5 Conclusions
The paper concludes that DARR addresses unsupervised domain adaptation for 3D multi-organ segmentation across scanners and protocols by learning relatively fixed spatial relations. It combines jigsaw-based relational reasoning with spatial-resolution standardization and test-time jigsaw training, and reports superiority across five public datasets.
- 5 Conclusions: DARR generalizes 3D multi-organ segmentation models to medical images acquired with different scanners or protocols.The method targets unsupervised domain adaptation across domains.
- 5 Conclusions: DARR formulates stable spatial relations among internal structures through jigsaw puzzles and adds spatial-resolution standardization plus test-time jigsaw training.These schemes are intended to preserve transferability of the learned relations across domains.
- 5 Conclusions: Experiments on five public datasets demonstrate the superiority of DARR.The conclusion summarizes the method's reported experimental outcome without specifying a separate metric here.