Source-linked AI summary
Multimodal pseudo-CT synthesis for PET attenuation correction using separate modality encoding and topogram conditioning
Rory Bell, Artemis Bouzaki, Jiaming Cao, Jasmine Morrison, Chelsea Sargeant
TL;DR
The paper addresses CT-free PET attenuation correction by generating pseudo-CTs from NAC-PET, MRI, and 2D topograms. It uses separate PET and MR encoders with multi-scale fusion and FiLM-based topogram conditioning, achieving promising validation performance for downstream PET attenuation correction.
Problem
CT-based PET attenuation correction introduces additional ionising radiation, motivating pseudo-CT generation from NAC-PET, MRI, and 2D topograms.
Method
A multimodal 3D patch-based U-Net uses separate PET and MR encoders, multi-scale feature fusion, and FiLM-based topogram conditioning at the bottleneck.
Results
The validation model achieved a CT mu-map MAE of 0.005454, whole-body PET SUV MAE of 0.0379, mean organ bias of 2.45%, and brain outlier score of 0.024.
Takeaways & Limitations
The multimodal architecture integrates complementary crossmodality information, and validation results demonstrate promising performance for downstream PET attenuation correction.
Abstract
from arXiv · showhide
We participated in the BIC-MAC Challenge with a multimodal 3D patch-based U-Net for pseudo-CT generation from NAC-PET, MRI, and 2D topograms. By using separate PET and MR encoders, multi-scale feature fusion, and FiLM-based topogram conditioning at the bottleneck, we obtain a model that integrates complementary cross-modal information while reducing reliance on precise voxel-wise correspondence between modalities. Our final submission can be found: https://github.com/rrr-uom-projects/BIC-MAC-MICCAI2026
1. Introduction
The BIC-MAC Challenge investigates CT-free PET attenuation correction by generating pseudo-CTs from NAC-PET, MRI, and 2D topograms. Because generated pseudo-CT quality and downstream PET correction both matter, the approach targets both outcomes.
- CT-based PET attenuation correction adds ionising radiation, motivating pseudo-CT generation without CT.
- The challenge uses NAC-PET, MRI, and 2D topograms to generate pseudo-CTs for PET reconstruction.
- The 75-subject training set supports evaluation of both CT quality and downstream PET correction.
2. Methods
The method uses separate volumetric PET and MR encoders, multi-scale fusion, and topogram-conditioned bottleneck modulation to combine complementary modalities without requiring precise voxel-wise alignment.
- 2.1 Multimodal architecture: Separate PET and MR 3D encoders learn modality-specific features before multi-scale fusion, avoiding early channel-wise fusion of imperfectly aligned inputs.
- 2.1 Multimodal architecture: At four encoder scales, concatenated PET-MR features are projected with 1×1×1 convolutions and reweighted by squeeze-and-excitation gates for U-Net skip connections.
- 2.1 Multimodal architecture: A pretrained ResNet-18 extracts a 256-dimensional topogram feature that conditions the shared PET-MR bottleneck through FiLM rather than spatial fusion.
- 2.1 Multimodal architecture: Zero-initialised FiLM parameters make topogram conditioning initially an identity mapping, allowing its influence to be learned progressively.
- 2.2 Implementation details: The model uses 64×64×64 positive-foreground patches, L1 loss within the prediction mask, and 71 training subjects with 4 internal validation subjects.
- 2.3 Augmentation and architectural ablations: Alternative augmentation, fusion, convolutional, registration, loss, and MRI-masking configurations did not improve validation CT or PET reconstruction metrics.
3. Results
On four internal validation subjects, the final submission produced pseudo-CTs and achieved reported CT and PET attenuation-correction metrics.
- 0.005454 CT mu-map MAE and 0.0379 whole-body PET SUV MAE were achieved on the validation set.
- 2.45% mean organ bias and 0.024 brain outlier score were reported for region-specific PET metrics.
- Figure 2 presents coronal pseudo-CT views generated for four internal validation subjects.
4. Conclusion
The proposed multimodal architecture combines modality-specific volumetric extraction with topogram-based conditioning for pseudo-CT generation. Validation results demonstrate promising downstream PET attenuation-correction performance.
- The architecture integrates modality-specific volumetric feature extraction with topogram-based conditioning to combine complementary crossmodality information.
- Validation results demonstrate promising performance for downstream PET attenuation correction.