Source-linked AI summary
ZODIAC: Zero-shot Octree-based Diffusion for Anatomical Completion
Miruna-Alexandra Gafencu, Vlad Bratulescu, Yordanka Velikova, Mohammad Farid Azampour, Nassir Navab
TL;DR
Recovering complete spine anatomy from partial, noisy ultrasound is an ill-posed problem, and synthetic corruption patterns may not match real occlusions. ZODIAC learns a diffusion prior over complete shapes in an adaptive octree and applies blended completion at inference, outperforming a supervised variant by 22% on HD95 in difficult volunteer cases.
Problem
Real ultrasound observations contain irregular missing structures and noise, while supervised methods rely on synthetically corrupted incomplete–complete pairs.
Method
ZODIAC learns an unconditional diffusion prior over complete anatomies in an adaptive octree and conditions generation on partial geometry through blended completion at inference.
Results
22% improvement in HD95 completion error is reported against the fully supervised TP-ODIAC variant on difficult volunteer cases.
Takeaways & Limitations
Zero-shot generative priors can provide a paired-supervision alternative for anatomical completion under unexpected missing regions.
Takeaways & Limitations
The partial observation is incorporated only at inference time, with no paired partial–complete training data required.
Abstract
from arXiv · showhide
Recovering the full 3D spine anatomy from intraoperative ultrasound is an ill-posed inverse problem, as the complete structure must be inferred from incomplete and noisy observations. Acoustic occlusions and limited field of view create large unobserved regions, while view-dependent artifacts lead to variability in expert annotations of the visible anatomy. Current supervised ultrasound shape completion methods rely on synthetically generated incomplete-complete paired data to learn conditional mappings under a predefined distribution of simulated occlusions. However, real intraoperative occlusions do not necessarily follow this distribution, which can limit generalization to patient data. As a result, accurate and robust completion from noisy partial observations remains an unsolved problem. We propose a zero-shot shape completion framework that reconstructs the entire lumbar spine from partial ultrasound observations without relying on simulated training data. To accommodate unseen and irregular patterns of missing structures, we introduce blended completion, a mechanism that integrates the learned anatomical prior with incoming partial geometry at inference time. The method learns a generative diffusion prior over full anatomical shapes represented in an adaptive octree structure, enabling efficient modeling of the complete spine in a single forward pass. Validation on phantom and volunteer data shows that decoupling completion from a predefined corruption distribution improves generalisation under real occlusions, outperforming a fully supervised variant by 22% on HD95 completion error. Code and data are available at https://github.com/miruna20/ZODIAC.
1 Introduction
Intraoperative ultrasound provides incomplete and uncertain views of spinal anatomy, making full 3D reconstruction ill-posed. ZODIAC addresses these challenges with a zero-shot octree-diffusion prior and inference-time blended completion.
- Ultrasound-guided spinal procedures require clinicians to mentally reconstruct 3D anatomy because bone shadows make the spine only partially visible.
- Shape completion must infer full anatomy from incomplete, noisy observations affected by occlusions, artifacts, and variable segmentation quality.
- Synthetic paired training data imposes predefined corruption patterns that may not represent irregular, patient-specific intraoperative occlusions and segmentation errors.
- Octrees concentrate computation and memory on occupied regions, supporting high-resolution volumetric learning more efficiently than dense grids.
- ZODIAC combines an adaptive octree representation, an unconditional generative prior, and blended completion to reconstruct unseen missing structures without paired incomplete–complete data.
2 Methodology
The method formulates completion as posterior inference from partial anatomy, learning a prior over complete shapes and conditioning generation on partial observations only at inference. It uses adaptive octrees within a two-stage diffusion pipeline.
- Shape completion approximates p(S | P), where P is a noisy partial observation and S is the complete anatomy.
- The zero-shot formulation learns p(S) from complete shapes without requiring paired partial–complete training data.
- Partial observations are incorporated only during inference rather than during prior training.
- Adaptive octrees concentrate resolution near anatomical surfaces while avoiding the memory cost of dense voxel grids.
- The training pipeline first learns an octree-to-latent representation, then trains a two-stage DDPM that generates complete shapes from noise.
Shape Representation.
ZODIAC represents each 3D shape with a geometric octree built by recursively subdividing space and retaining nodes containing surface points.
- Each shape is encoded as a geometric octree of depth D over the point cloud’s cubic bounding volume.
- Recursive subdivision retains only nodes containing surface points, producing a sparse representation of the anatomy.
- Leaf nodes span cubes whose side length decreases with node depth according to ℓ(Ω)/2^d.
Representation Learning.
Representation learning compresses geometric octrees into latent octrees and decodes them into continuous signed-distance fields for surface reconstruction.
- A VAE compresses the geometric octree O into a lower-dimensional latent octree Z.
- Each latent-octree leaf stores a C-dimensional feature vector encoding surface geometry.
- The decoder predicts binary split decisions and converts latent features into a continuous signed-distance field.
- The final surface is extracted from the decoded SDF using marching cubes.
Generative Prior.
The model learns a two-stage diffusion prior over complete shapes, generating coarse spine structure before refining high-resolution surface details.
- The two-stage DDPM learns the distribution of complete shapes and generates them coarse-to-fine.
- The LR diffusion stage denoises octree split states to capture global spine structure and inter-vertebral configuration.
- The LR representation recursively refines active octants from the base depth to capture coarse geometric structure.
- The HR graph UNet denoises per-node latent codes and recovers fine surface detail within the coarse envelope.
- The frozen VAE decoder converts the resulting latent representation into an SDF, which is converted to a surface using marching cubes.
2.3 Zero-shot Shape Completion
ZODIAC formulates completion as constrained generation from an unconditional shape prior, blending partial observations into every reverse diffusion step without paired incomplete–complete data.
- Completion samples from the learned prior while respecting the partial observation as a constrained generation problem.
- The zero-shot strategy operates directly on an unconditionally trained model and requires no paired partial–complete training data.
- At test time, the partial observation is blended into each reverse diffusion step to condition the generated shape.
Partial Observation Rasterization.
The partial point-cloud observation is converted into the model’s rasterized octree grid and paired with a one-sided occupancy mask.
- The partial point cloud is converted into an octree and rasterized into a base-depth split grid using the complete-shape procedure.
- The binary mask anchors positively observed occupied split entries while leaving unobserved entries free for prior-based completion.
- This one-sided masking treats absent ultrasound surfaces as unknown rather than confirmed empty space.
Blended Completion.
Blended completion injects a re-noised partial observation throughout low-resolution denoising, preserving observed structure while allowing the learned prior to generate missing anatomy before high-resolution refinement.
- The partial input is re-noised and blended into the LR diffusion stage at every denoising step.
- Re-noising keeps observed regions statistically consistent with the training distribution instead of injecting the clean partial grid directly.
- The completed low-resolution grid is passed to the HR diffusion stage, which denoises latent features before final SDF decoding.
- The update anchors observed entries to the re-noised partial grid and lets unobserved entries evolve under the learned prior.
- Blending is applied only at LR because that stage constrains global topology, while HR refinement handles fine details across observed and unobserved regions.
3 Experiments
Experiments evaluate ZODIAC on phantom and volunteer ultrasound data, using complete-shape training sets, standard shape metrics, and comparisons with supervised and zero-shot variants.
- Training data: Complete-shape training data combines 91 augmented VerSe20 lumbar meshes with 322 lumbar meshes from TotalSegmentator.Deformation augmentation accounts for different spine curvatures.
- Evaluation data: Evaluation uses two anthropomorphic phantoms and six volunteer ultrasound scans from three subjects in the Balgrist dataset.Each volunteer contributes one handheld and one robotic acquisition; scans are selected using clear anatomical anchors and identifiable vertebral levels.
- Metrics: Performance is measured with Chamfer Distance, F1-score at a 1% threshold, and 95% Hausdorff Distance, including vertebra-wise L1–L5 metrics.Predicted meshes are segmented into vertebral levels using proximity to ground-truth levels.
- Baselines: SITD independently completes each vertebra from partial ultrasound point clouds, with whole-spine metrics reported only when all five vertebrae are reconstructed.TP-ODIAC provides a supervised variant using paired partial-complete training data and serves as an expected upper bound.
- Ablation: ZODIAC-clean treats observed voxels as hard boundary conditions throughout denoising, restricting diffusion to unobserved regions.This ablation isolates the contribution of per-step blending and re-noising.
4 Results and Discussion
ZODIAC remains competitive with supervised baselines on clean data and improves robustness on volunteer and difficult acquisitions, while per-step blending is important for accuracy.
- Qualitative comparison: Volunteer acquisitions show anatomically and geometrically coherent ZODIAC meshes, whereas baselines can produce fragmented or collapsed surfaces.The qualitative difference is particularly evident on volunteer data.
- Comparison with TP-ODIAC: ZODIAC is within 3% of TP-ODIAC on whole-spine CD for volunteer data and matches it on the reported comparison metric.TP-ODIAC leads on clean, in-distribution phantom acquisitions.
- Difficult cases: 22%: on difficult volunteer cases, ZODIAC improves whole-spine HD95 over TP-ODIAC, while F1-score remains 11% lower.The difficult subset contains partially visible spines with one or more entirely absent vertebrae.
- Generalisation: ZODIAC generates anatomically coherent spines with expected landmarks when observations omit important structures such as lateral processes.The results suggest prior-driven completion degrades less than paired supervised models when test observations depart from training distributions.
- Ablation studies: ZODIAC-clean worsens all three metrics across settings, including whole-spine CD increases of 15–26% and HD95 increases of 13–16%.The largest difficult-subset losses are CD +26% and HD95 +16%, supporting per-step re-noising over clean-observation anchoring.
5 Conclusion
ZODIAC performs whole-spine completion from partial ultrasound without paired incomplete–complete training data. Its zero-shot prior-based formulation is designed for unexpected missing regions and shows strong volunteer-data results.
- Conclusion: ZODIAC combines a generative prior over complete anatomies with blended completion at test time and requires no paired incomplete–complete training data.The framework targets partial ultrasound observations with unexpected missing regions.
- Conclusion: ZODIAC matches the supervised variant under standard conditions and outperforms it by up to 22% in HD95 when observations contain unexpected missing regions.These evaluations are reported on volunteer data.