Source-linked AI summary
Unified Multi-plane Autoregressive Diffusion for 3D Multi-contrast MRI Synthesis
Yejee Shin, Geonhui Son, Jinglu Wang, Minwoo Jung, Yan Lu, Dosik Hwang
TL;DR
Acquiring all MRI contrasts takes time and can be uncomfortable, and efficient 3D synthesis must preserve anatomical coherence across planes. MPAD combines isotropic 3D latents, 2D diffusion, and autoregressive synthesis across planes. Compared with 3D latent diffusion baselines, it reduces training and inference FLOPs while also reducing inference time and memory and improving reconstruction quality.
Problem
Acquiring all contrasts increases scan time and patient discomfort, while 3D latent diffusion still faces efficiency, inference-time, and multi-contrast fidelity constraints.
Source: §1 · pp. 2–3Method
MPAD uses isotropic 3D latents and 2D diffusion to reconstruct masked target slices, then generates slices autoregressively across planes using previously synthesized volume as a conditioning prior.
Source: §1 · p. 3Results
MPAD reduces training FLOPs by 7× and inference FLOPs by 3× versus 3D latent diffusion baselines, while reducing inference time and peak memory and improving reconstruction quality.
Source: §1 · p. 3Takeaways & Limitations
MPAD combines computational efficiency with global 3D anatomical fidelity, with refinement settings offering flexibility for different deployment constraints.
Source: §5 · pp. 14–15Sequential cross-plane refinement remains part of inference; fewer refinement iterations reduce inference time to approximately 1.8 s per volume but trade off against multi-plane consistency.
Source: §4.5 · p. 14
Abstract
from arXiv · showhide
Acquiring a complete set of magnetic resonance imaging (MRI) contrasts is time-intensive and uncomfortable for patients, despite the diagnostic value of multi-contrast imaging. This motivates synthesizing missing contrasts from those already acquired, which is an inherently 3D problem requiring anatomical coherence across axial, sagittal, and coronal planes. However, fully 3D generative models are often impracti- cal under computational resources that scale cubically with volume size. We propose a unified Multi-Plane Autoregressive Diffusion (MPAD), a latent diffusion framework that achieves full-volume 3D synthesis using efficient plane-wise 2D operations while preserving volumetric coherence. A 3D autoencoder first compresses MRI scans into an isotropic 3D la- tent representation. A 2D diffusion model is then trained to reconstruct masked latent slices of the target contrast, conditioned on both source- contrast slices and unmasked target-contrast slices. During inference, we introduce plane-wise autoregressive synthesis with inter-plane priors. Slices are generated autoregressively in random order within one plane orientation to maintain intra-plane continuity, then propagated as con- ditioning priors to orthogonal plane orientations to enforce inter-plane consistency. Compared to 3D latent diffusion baselines, MPAD reduces training and inference FLOPs by 7x and 3x, respectively, while also lowering inference time and peak memory consumption. Experiments on multiple datasets demonstrate that MPAD achieves superior perfor- mance, generating high-fidelity 3D volumes and supporting one-to-many translation within a single unified model.
1 Introduction
Acquiring every MRI contrast increases scan time and patient discomfort, while existing 2D and 3D synthesis approaches face coherence or computational challenges. MPAD uses latent 2D diffusion with multi-plane autoregressive inference to synthesize coherent 3D volumes efficiently.
1 Introduction
- Acquiring all contrasts increases scan time and patient discomfort, motivating synthesis of missing contrasts from those already acquired.The passage also notes burdens on scanner throughput and motion-artifact susceptibility during prolonged scans.Source: §1 · p. 2
- Earlier 2D methods lack volumetric coherence, while 3D generative models better capture anatomy but have computational and memory costs that grow cubically with volume size.Even latent 3D diffusion retains practical limits in efficiency, inference time, and fidelity across multiple contrasts.Source: §1 · pp. 2–3
- MPAD encodes volumes into isotropic latents and reconstructs masked target-contrast slices with 2D diffusion conditioned on source and unmasked target slices.This operates on 2D latent slices rather than full 3D volumes, avoiding cubic computational scaling.Source: §1 · p. 3
- Slices are generated autoregressively within each plane, then the synthesized volume conditions generation in orthogonal planes to promote inter-plane consistency.Random within-plane slice order is used to maintain intra-plane continuity.Source: §1 · p. 3
- 7× lower training FLOPs and 3× lower inference FLOPs than 3D latent diffusion baselines, with reduced inference time and peak memory and superior reconstruction quality.The reported reductions are accompanied by improved synthesis fidelity.Source: Figure 1 · p. 2, §1 · p. 3
2 Related Works
Prior 2D synthesis methods offer contrast translation but do not ensure coherence across a 3D volume, while volumetric approaches face substantial computational costs. MPAD coordinates slice-wise diffusion across orthogonal planes to retain efficiency while building global 3D coherence.
2 Related Works
- Slice-based 2D methods can achieve strong fidelity, but lack explicit mechanisms for volumetric coherence and may produce anatomical inconsistencies between adjacent slices.Full 3D synthesis improves spatial continuity but has prohibitive costs for long-range dependencies.Source: §2 · p. 4
- Full 3D and latent-diffusion models retain substantial computational requirements, while 2D/2.5D approaches struggle to enforce long-range dependencies affordably.The related work identifies a need for efficient global 3D coherence without full 3D generation.Source: §2 · p. 4
- MPAD factorizes a volume into 2D slice sequences and propagates context across orthogonal planes, retaining slice-wise diffusion efficiency while avoiding full 3D models’ memory footprint.Its training overview pairs a 3D autoencoder’s isotropic latents with a 2D diffusion model for target-contrast latent slices.Source: §2 · pp. 4–5, Figure 2 · p. 5
3 Method
MPAD combines an isotropic 3D latent representation with 2D diffusion over anatomical-plane slices. At inference, autoregressive generation within planes and priors across planes support volumetric coherence.
3.1 3D Latent Representation Learning
- A KL-regularized 3D autoencoder maps each MR volume to a latent tensor and reconstructs it, producing isotropic features that support slicing along any spatial plane.Its objective combines reconstruction loss, a KL penalty on the approximate posterior, and an adversarial loss.Source: §3.1 · pp. 5–6
3.2 2D Diffusion Training for Anatomical Planes
- The 2D diffusion model learns from randomly masked target-latent slices across randomly sampled axial, coronal, or sagittal orientations.Training conditions on source-contrast and unmasked target-contrast features, plus a target-contrast prompt; spatially adaptive normalization injects target-contrast information into intermediate features.Source: §3.2 · p. 6
3.3 Plane-wise Autoregressive Synthesis
- MPAD uses intra-plane autoregressive synthesis and inter-plane priors to generate slices across orthogonal orientations, combining four candidates by voxel-wise averaging.The aggregated latent is decoded into the target volume; combining orientations reduces plane-specific artifacts.Source: §3.3 · pp. 7–8
- Within each plane, slices are generated sequentially in random order; the first plane starts from noise at timestep T, while subsequent planes use completed volumes as priors from timestep τ < T.After all three planes are generated, a refinement pass on the first plane incorporates information from all planes; starting from τ reduces sampling steps while maintaining spatial consistency.Source: §3.3 · p. 8
4 Experiments and Analysis
Experiments on ADNI and IXI show that MPAD combines strong 3D synthesis quality with efficient 2D operations. Ablations identify gains from autoregressive and multi-plane conditioning, while the discussion notes that cross-plane refinement remains sequential.
4.1 Datasets
- Evaluation uses 737 ADNI volumes and 577 IXI volumes, with 100 subjects from each dataset reserved for evaluation and the remaining subjects used for training.ADNI includes T1w, T2w, and PDw scans from 1.5T GE scanners; IXI includes those contrasts from 1.5T and 3T Philips scanners across sites.Source: §4.1 · pp. 8–9
4.2 Implementation Details
- Training uses a 3D KL-regularized autoencoder followed by a 2D latent diffusion model, while inference generates across three orthogonal planes with 10 DDIM steps initially and 2 steps for subsequent planes.The autoencoder encodes volumes at 3 × 32 × 32 × 32 resolution; target latent slices are randomly masked at ratios from 0.7 to 1.0 during diffusion training.Source: §4.2 · p. 9
4.3 Comparison Results
- MPAD achieves the highest SSIM on five of six ADNI tasks, lowest NMSE, and over 2 dB better PSNR than the second-best method on PD→T1 and PD→T2; on IXI it ranks first across all metrics.Qualitative comparisons also report sharper boundaries and better tissue contrast across planes than baselines.Source: §4.3 · pp. 9–10, Table 1 · p. 10, Table 2 · p. 10
4.4 Ablation Results
- Autoregressive synthesis improves reconstruction over the non-AR baseline, and smaller slice groups yield steady gains; experiments use g=4 to balance accuracy and efficiency.Source-contrast conditioning supplies volumetric guidance, while previously generated target slices add slice-to-slice continuity and local anatomical consistency.Source: §4.4 · pp. 10–11
- Across both datasets, the 3D multi-modal conditioning encoder consistently outperforms its 2D counterpart, whose slices are processed independently; Gaussian-noise masking substantially outperforms learnable mask tokens.The 3D convolutions aggregate information across neighboring slices and depth.Source: §4.4 · p. 12
- Adding planes improves PSNR and SSIM, all three planes outperform fewer-plane combinations, and a final refinement pass adds a small consistent gain; plane ordering yields nearly identical metrics.The refinement pass lets the first plane use information from all completed views, while ordering robustness reflects conditioning on source information and previously synthesized volumes.Source: Table 5 · p. 13, §4.4 · pp. 13–14, Table 6 · p. 13
- Inter-plane priors consistently improve performance by giving later planes structural information from completed planes, supporting anatomical consistency across orientations.The priors also reduce denoising burden through a more informed initialization.Source: Table 4 · p. 12, §4.4 · p. 13
4.5 Discussion
- Cross-plane refinement remains sequential, and reducing refinement iterations brings inference time to approximately 1.8 s per volume with a trade-off against multi-plane consistency.The authors identify more efficient aggregation, parallel generation, distillation, and fewer-step sampling as future directions.Source: §4.5 · p. 14
5 Conclusion
MPAD performs full-volume 3D multi-contrast MRI synthesis using 2D diffusion operations and structured cross-plane priors. Its refinement depth trades inference time against multi-plane consistency, offering flexibility for different deployment constraints.
5 Conclusion
- MPAD combines plane-wise autoregressive generation, inter-plane priors, and multi-plane aggregation to enforce volumetric consistency without full 3D diffusion operations.A 3D autoencoder maps volumes to an isotropic latent representation that supports slicing along arbitrary plane orientations.Source: §5 · p. 14
- MPAD bridges computational efficiency and global 3D anatomical fidelity within a unified framework.The authors describe a controllable trade-off between inter-plane refinement and inference time, with distillation or parallelized multi-plane generation as future directions.Source: §5 · p. 15