Source-linked AI summary
UltraDiff: Differentiable Ray Tracing in Ultrasound for Shape Optimization
Felix Duelmer, Magdalena Wysocki, Nassir Navab, Mohammad Farid Azampour
TL;DR
Differentiable rendering has largely focused on light transport, while ultrasound shape reconstruction relies on pre-segmented images. UltraDiff uses a differentiable, time-of-flight-gated path-tracing model for B-mode ultrasound; it reports improved synthetic reconstruction metrics and recovery of overall anatomy on a real phantom.
Problem
Differentiable rendering has mainly targeted light transport, while ultrasound reconstruction commonly fits surfaces to point clouds extracted from pre-segmented images.
Source: §1 · p. 2Method
UltraDiff models B-mode image formation as a time-of-flight-gated path integral and estimates the forward model and scene-parameter gradients using Monte Carlo sampling.
Source: §1 · p. 2, §2.1 · p. 2, §2.2 · p. 2Results
On synthetic data, UltraDiff reduces Chamfer distance by 3.8× and MAD by 6.8× versus the best baseline; on the real phantom, it recovers overall anatomy and principal curvature.
Source: §3.2 · pp. 3–4Takeaways & Limitations
UltraDiff brings differentiable path tracing to ultrasound and provides a foundation for richer acoustic interaction models and broader inverse problems.
Source: §4 · p. 4Synthetic targets are rendered with UltraDiff’s own forward model, which may bias comparisons in its favor; transfer to real data mitigates but does not eliminate this concern.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Physically-based differentiable rendering enables gradient-based optimization of scene parameters by matching rendered images to measurements, but has so far mainly focused on light transport. We extend this paradigm to medical ultrasound, where image formation resembles transient rendering: echoes are binned by time-of-flight rather than projected onto an image plane. We present UltraDiff, a modular framework for differentiable ultrasound ray tracing. UltraDiff formulates ultrasound image formation as a path-space integral, gated by travel time between the transducer and tissue interfaces, and derives a Monte Carlo estimator of both the forward model and its gradients with respect to scene parameters. We demonstrate this on an inverse geometry estimation: starting from a sphere, an SDF is optimized until simulated echoes match measured ones, recovering vertebral surfaces from simulated B-mode sweeps and from a real robotic acquisition of a spine phantom. Unlike state-of-the-art ultrasound shape reconstruction methods, which rely on pre-segmented images, our approach operates unsupervised on B-mode images through analysis-by-synthesis, while achieving competitive geometric accuracy. Implemented on top of Mitsuba 3, UltraDiff brings differentiable path tracing to a new sensing modality and provides a foundation for inverse problems in acoustic imaging.
1 Introduction
Differentiable rendering has mainly addressed light transport, while ultrasound reconstruction commonly fits surfaces to point clouds extracted from pre-segmented images. UltraDiff instead differentiates an ultrasound forward model so geometry can be optimized by matching simulated and measured B-mode images.
- Differentiable rendering has recovered geometry and materials from images through gradient descent, but most prior work targets light transport.Its path-space integrals, Monte Carlo estimation, and boundary-aware gradients are not specific to light.Source: §1 · p. 2
- Existing ultrasound reconstruction methods fit surfaces to point clouds extracted from pre-segmented images, tying accuracy to a separately trained segmentation model [Chen et al. 2024; Wu et al. 2025].Differentiable forward models offer an alternative by optimizing a scene representation to match measured image intensities.Source: §1 · p. 2
- UltraDiff formulates B-mode image formation as a time-of-flight-gated path integral and provides gradients with respect to scene parameters, including SDF geometry.The framework uses Monte Carlo path sampling and supports segmentation-free vertebra reconstruction from simulated and real robotic B-mode sweeps.Source: §1 · p. 2
2 Method
UltraDiff models time-resolved ultrasound pressure as a path integral and estimates it, along with scene-parameter gradients, using Monte Carlo sampling. Its implementation represents geometry as an SDF and optimizes it through a modular ray-tracing pipeline.
2.1 Time-of-Flight Path Integral
- B-mode pressure is modeled as a path-space integral over tissue-boundary interactions, with spatial throughput multiplied by a time-of-flight response that distributes echoes across time samples.The temporal response is Gaussian, with width set by transmitted pulse length.Source: §2.1 · p. 2
2.2 Differentiable Formulation
- Geometry gradients account for moving interaction vertices via the transport theorem; the estimator reuses forward samples and omits explicit visibility-discontinuity terms.The authors report that temporal-kernel and transducer-response smoothing of dominant discontinuities is sufficient for stable optimization.Source: §2.2 · p. 2
2.3 Shape Representation
- Geometry is represented by a voxel-grid signed distance function with cubic B-spline interpolation, and intersections are found by sphere tracing.The interpolation provides smooth optimization gradients.Source: §2.3 · p. 2
2.4 Implementation
- Reconstruction uses single-interaction paths, uniform per-element ray sampling reused across time bins, and Adam optimization of the SDF with an ℓ1 image loss.A coarse-to-fine schedule and Laplacian regularization complete the setup; two-bounce transport is demonstrated separately.Source: §2.4 · p. 3
3 Experiments and Results
UltraDiff’s analytic gradients closely track finite-difference references, and its path-space formulation also reproduces two-bounce ghost echoes. For shape reconstruction, it lowers synthetic geometric errors against the best baseline and transfers to a real spine phantom without changing the model or optimization settings.
3.1 Gradient Validation and Multi-Bounce Transport
- 0.97 and 0.95 cosine similarities measure agreement between analytic and finite-difference gradients for single- and two-bounce transport, respectively.The gradients concentrate in a signed band along the reflecting boundary; two-bounce paths produce a deeper ghost echo where a specular bounce reaches a second surface.Source: §3.1 · p. 3, Figure 2 · p. 3
3.2 Shape Reconstruction
- 3.8× lower Chamfer distance and 6.8× lower MAD than the best baseline on synthetic data; reconstructions closely match the ground-truth surfaces.Figure 3 compares ground truth, RoCoSDF, and UltraDiff, with color encoding per-point distance to ground truth in mm.Source: §3.2 · p. 3, Table 1 · p. 3, Figure 3 · p. 4
- Sub-millimeter MAD on the real phantom lies within the stated lumbar tolerances of up to 3.8 mm for image-guided pedicle screw placement.The same forward model and optimization settings transfer without modification; remaining deviations occur in regions of weak or ambiguous signal.Source: §3.2 · p. 4, Figure 4 · p. 4
4 Discussion and Conclusion
UltraDiff extends differentiable path tracing to ultrasound, enabling geometry optimization directly from B-mode images. The authors report competitive or better reconstruction accuracy than point-cloud methods, while identifying estimator and data limitations.
- UltraDiff’s modular path-space framework supports exchangeable transducer models, interaction terms, and priors, as well as multi-bounce transport.The authors report that it matches or exceeds point-cloud-based methods while operating on B-mode images without segmentation and transferring to real data without modification.Source: §4 · p. 4
- The gradient estimator omits explicit boundary terms from visibility discontinuities, and the smoothness prior can suppress structures seen in only a few views.The authors note that synthetic targets rendered by their own forward model may bias comparisons, although transfer to real data mitigates but does not eliminate this concern.Source: §4 · p. 4
- The framework is presented as a foundation for richer acoustic interaction models, including scattering and attenuation, and broader inverse problems such as transducer and trajectory optimization.Source: §4 · p. 4