Source-linked AI summary
Unsupervised Adaptation of 3D CT Foundation Models for 3D CBCT Segmentation
Gauthier Miralles, Loic Le Folgoc, Vincent Jugnon, Pietro Gori
TL;DR
The paper addresses scarce CBCT annotations and severe CT–CBCT domain shift that limit cross-modality segmentation transfer. It introduces an architecture-agnostic UDA framework using redundancy-reducing feature alignment, and reports improved CBCT liver segmentation across interventional and radiation-therapy benchmarks without target annotations or inference-time optimization.
Problem
Scarce CBCT annotations and severe CT–CBCT domain shift limit the transfer of pretrained 3D CT foundation models for CBCT segmentation.
Method
The framework uses redundancy-reduction adversarial feature alignment and supports both CNN- and ViT-based foundation models without imposing network-architecture constraints.
Results
The method achieves the best results across interventional vascular and radiation-therapy CT–CBCT benchmarks without labeled target-domain data.
Takeaways & Limitations
Explicit feature-space adaptation is needed to bridge pretrained 3D CT models to CBCT, while the proposed framework remains lightweight and requires no inference-time optimization.
Takeaways & Limitations
The approach is demonstrated on liver segmentation, with broader multi-organ and interventional-imaging applications presented as potential extensions.
Abstract
from arXiv · showhide
Accurate 3D segmentation of cone-beam CT (CBCT) is critical for interventional and radiation therapy applications, yet it remains limited by two compounding challenges: the scarcity of annotated CBCT data and the large domain shift from diagnostic CT. Interventional CBCT exhibits fundamental modality differences from conventional CT, driven by acquisition and physics effects as well as contrast-specific vascular content, thereby limiting effective cross-modality model transfer. We propose a novel unsupervised domain adaptation (UDA) framework based on redundancy-reducing feature alignment, enabling 3D CBCT segmentation with no target-domain annotations or inference-time adaptation. Our framework is architecture-agnostic, seamlessly adapting both CNN-based and ViT-based foundation models. We evaluate our method on two challenging CT-CBCT liver segmentation benchmarks: one for interventional vascular procedures and one for radiation therapy, demonstrating that even large-scale pretrained segmentation networks require explicit feature-space bridging to generalize across acquisition modalities, and that our approach consistently outperforms existing pretrained foundation model and UDA strategies. To support reproducibility and benchmarking, we release the liver segmentations for a public CBCT dataset, along with the code, trained models, and weights.
1 Introduction
CBCT segmentation is hindered by scarce annotations and substantial CT–CBCT domain differences, which limit direct transfer of pretrained 3D CT models. The proposed framework addresses this setting through architecture-agnostic, redundancy-reduction feature alignment for CNN- and ViT-based models.
- Domain shift: CBCT differs from conventional CT through limited field of view, acquisition artifacts, physics-related degradations, and injected iodine contrast.These effects create high-intensity regions in interventional imaging.
- Domain shift: The CT–CBCT domain gap restricts transferability of pretrained 3D CT foundation models and can make full-field CT preprocessing unreliable on CBCT.Generative adaptation methods such as CycleGAN may also be unsuitable because they typically assume similar fields of view.
- Contribution: The work releases liver segmentations for a public CBCT dataset to facilitate future research and fair comparison.
- Proposed direction: The framework decomposes the network into a shared feature extractor, representation heads, and a task-specific prediction head for adaptation across CNN- and ViT-based backbones.The adversarial counterpart of the representation head is discarded at inference time.
2 Method
The method aligns source and target feature representations adversarially while preserving source-task supervision, using redundancy-reduction losses that are independent of the task head. Under a capacity assumption, the resulting optimization theoretically aligns the source and target marginal feature distributions.
- Design requirements and assumption: The method targets generic, architecture-flexible, lightweight adaptation without assuming a particular domain shift, while its theoretical guarantee relies on a sufficiently expressive adversary.Assumption 1 requires an adversary representation satisfying the stated feature constraints.
- Problem setup: The framework learns a shared labeling map from labeled source and unlabeled target samples by aligning their feature representations and supervising the source task.Source and target representations are driven toward the same marginal distribution, while source labels train the task-specific prediction head.
- Source domain label supervision: Source supervision trains the representation and prediction heads to solve the labeling task through a task-specific loss.The prediction head remains separate from the adaptation mechanism for the target task.
- Adversarial feature alignment: Adversarial alignment makes target adversary representations dissimilar from the representation head while making source representations similar.The strategy combines separation and alignment losses applied to source and target features.
- Redundancy-reduction based representation alignment: Redundancy-reduction losses impose correlation structure that aligns homologous feature dimensions and decorrelates different dimensions.The method forms a cross-correlation matrix from centered and normalized representations, with alignment and separation defined over its diagonal and off-diagonal entries.
- Task-agnostic alignment: Lalign and Lsep are task-agnostic, moving feature alignment from the prediction space to the representation space without requiring an adversary task-specific prediction head.This design avoids the task-specific adversary head used in MDD.
- Theoretical guarantee: Under Assumption 1, the adversary satisfying the feature constraints is a global minimizer, and optimal minimizers align source and target marginal feature distributions.The theorem states p(z) = q(z) at optimum; the proof uses the optimal adversary and first-order conditions under normalization.
3 Experiments and Results
The method was evaluated for 3D liver segmentation across radiation therapy and interventional CT–CBCT datasets, with comparisons spanning UDA methods, foundation models, and qualitative outputs. Results show robust adaptation across architectures and improved handling of CBCT intensity artifacts and field-of-view variation.
- Datasets: The evaluation used two CT–CBCT liver segmentation datasets: a public unpaired collection and a private clinical collection of interventional CBCT scans.The public dataset contains 130 CT and 39 CBCT volumes; the private collection contains 678 CT and 573 interventional CBCT volumes, with patient-level splits.
- Benchmarking: Table 1 compares UDA strategies and zero-shot foundation models using F1 on radiation therapy CBCT (DR) and interventional CBCT (DI).The table also reports the additional trainable parameters introduced by each UDA method.
- UDA comparisons: Feature-alignment strategies outperform self-training approaches overall, while DA-nnUNet consistently outperforms MDD-UNet on both datasets.Self-training is sensitive to poor initialization, which can cause error propagation during adaptation.
- UDA comparisons: The proposed method achieves the best results across all benchmarks while remaining lightweight and compatible with 3D foundation models.The hyperparameter study on DR identifies a broad near-optimal region, with the best run reaching F1 = 90.0%.
- Foundation models: Most foundation models fail to achieve satisfactory automatic zero-shot CBCT segmentation, whereas point prompts provide stronger initializations for SAM-Med3D and VISTA-3D.On DI, iodine-induced high intensities can degrade default preprocessing, while percentile normalization clips these intensity outliers.
- Qualitative results: After adaptation, the source-only model becomes more robust to high-intensity artifacts and field-of-view variations, producing more anatomically consistent liver segmentations.Figure 4 compares input, ground truth, source-only, zero-shot, and UDA-adapted predictions on DI and DR test sets.
4 Conclusion
The paper concludes that explicit redundancy-reducing feature alignment can bridge severe CT–CBCT shifts for 3D liver segmentation. The framework supports CNN and ViT foundation models without target annotations or inference-time optimization, although its demonstrated scope is liver segmentation.
- Conclusion: Explicit feature-space adaptation is required for pretrained 3D CT foundation models to transfer effectively to CBCT segmentation.The conclusion frames redundancy-reducing alignment as the mechanism for bridging modalities.
- Conclusion: The framework improves performance on both interventional vascular and radiation therapy CT–CBCT benchmarks.Its reported scope covers 3D liver segmentation under severe cross-modality domain shift.
- Conclusion: The method is architecture-agnostic, supports CNN- and ViT-based models, and requires neither target-domain annotations nor inference-time optimization.These properties are presented as supporting clinical practicality.
- Scope: The approach is demonstrated on liver segmentation, while the authors suggest potential extension to multi-organ tasks and other label-scarce interventional imaging applications.Suggested applications include artifact correction, denoising, and pose estimation.
- Conclusion: The work highlights explicit representation alignment as important for unlocking foundation models’ potential in CBCT.
A Reproducibility and Model Integration
The release provides paired CT–CBCT liver masks, organized preprocessing, common UDA comparison settings, model-integration details, and scripts for reproducible adaptation and evaluation.
- Public dataset release: The public release adds paired CT and CBCT liver masks for 39 Pancreatic-CT-CBCT-SEG cases in NIfTI format.The dataset includes images and masks with the naming conventions LiverCT_XXXX.nii.gz and LiverCBCT_XXXX.nii.gz.
- Preprocessing: All images and labels are resampled to isotropic 1.8 mm spacing, using continuous interpolation for images and nearest-neighbor interpolation for masks.Training and test partitions are defined at the patient level.
- Common comparison setup: UDA comparisons use a shared five-stage 3D U-Net with 64 base channels, two 3 × 3 × 3 convolutions per stage, skip connections, and standard augmentations.The common backbone isolates adaptation-objective effects from backbone-architecture differences.
- Model integration: VISTA-3D integration decomposes the network into feature extractor ψ, representation head f, and task head g, while f′ is used only during adaptation.After training, f′ is removed and inference follows the original feature and task paths.
- Reproducibility: The repository supplies configuration-driven adaptation and evaluation scripts, checkpoints, preprocessing, validation, scheduling, and metric-computation procedures.Source-only and adapted checkpoints are included for the reported experiments.