Source-linked AI summary
Dirichlet Splatting: Differentiable Rendering for Wave-Based Inverse Problems
Xingyu Chen, Wuqiong Zhao, Xinyu Zhang, Tzu-Mao Li
TL;DR
Wave-based coherent imaging has a complex, oscillatory Dirichlet point response that Gaussian splats do not represent. Dirichlet Splatting pairs that response with surfel modeling and DSFW; on a five-reflector cold-start benchmark, the proposed pair succeeded on every trial. The method’s indirect-propagation model can recover ghosts at correct path lengths but incorrect physical locations.
Problem
Gaussian splats do not represent the complex, oscillatory coherent-sensing response, including sidelobes and phase that govern interference.
Source: §1 · p. 2Method
Dirichlet Splatting uses exact Dirichlet spectral footprints and surfel-aware coherent appearance, optimized with DSFW’s variable projection and residual dual certificates.
Source: §1 · p. 2Results
100.0% of five-reflector cold-start trials succeeded with the proposed pair, at a final complex spatial MSE of 5.32×10−9.
Source: §7.6 · p. 11Takeaways & Limitations
The paper presents Dirichlet Splatting as a step toward physically grounded differentiable rendering for radar, sonar, and other coherent sensing modalities.
Source: §9 · p. 12Indirect propagation is modeled as virtual surfels, so recovered support can contain ghosts at correct path lengths but incorrect physical locations.
Source: §8 · p. 12
Abstract
from arXiv · showhide
Wave-based coherent imaging, including terahertz tomography, synthetic-aperture acoustics, and millimeter-wave radar, forms images by Fourier-processing finite-length signals, with an exact point spread function that is not Gaussian but a Dirichlet kernel: complex-valued, oscillatory, and periodic. However, transplanting 3D Gaussian splatting to coherent sensing fails by construction; Gaussian splats discard the sidelobe energy (10-20% of the total) and the phase that governs coherent interference between reflectors. Our key idea is to replace the learned Gaussian footprint with the physically exact Dirichlet kernel of the finite-window DFT, modulated by a surfel that carries area, normal, and material, so that the rendering primitive matches the measurement physics instead of approximating it. We pair this primitive with a specialized solver, Dirichlet Sliding Frank-Wolfe (DSFW), that combines variable projection, residual dual certificates, and certificate-driven hard replacement of low-utility surfels, with periodic low-resolution coupled Levenberg-Marquardt correction, navigating the rugged loss landscape that breaks generic first-order optimizers. The Dirichlet kernel admits an O(1) closed-form evaluation, so the forward model matches FFT ground truth to machine precision while remaining differentiable end-to-end. On dense terahertz reconstruction, our method recovers reflector centers to 0.018 bin RMSE, 10-50x faster than waveform-level automatic differentiation, where Gaussian splats fail.
1 Introduction
Wave-based coherent sensors require differentiable rendering, but Gaussian primitives do not represent their complex, oscillatory point responses. Dirichlet Splatting uses the physical Dirichlet response and a surfel-aware model, with a specialized optimizer for its rugged loss landscape.
1 Introduction
- Gaussian primitives cannot represent the complex-valued, oscillatory point response produced by coherent wave measurements.The mismatch means sidelobes and phase that govern interference are lost when the response is approximated by a smooth real envelope.Source: §1 · pp. 1–2
- Dirichlet Splatting renders each primitive with the exact Dirichlet point-spread function and models it as a surfel with coherent appearance.The surfel-aware amplitude follows wave propagation physics, making the primitive a finite-area surface element rather than an isotropic point reflector.Source: §1 · p. 2
- DSFW addresses the rugged loss landscape with variable projection, residual dual certificates, hard replacement of low-utility surfels, and periodic coupled Levenberg–Marquardt correction.Generic first-order optimizers can encounter spurious gradient nulls in sidelobes and stall from cold starts.Source: §1 · p. 2
- The paper evaluates the method on real sub-THz tomography and includes surfel-modulation and solver ablations plus convergence analysis.The evaluation is described as part of the paper’s contributions.Source: §1 · p. 2
2 Related Work
Prior rendering approaches use Gaussian kernels for convenience or represent coherent effects without matching the finite-window DFT response. This work instead combines a physically exact Dirichlet primitive with sparse-recovery optimization inside differentiable splatting.
Differentiable rendering and splatting primitives
- The Gaussian kernel is convenient for optical PSFs but structurally mismatched to image formation by a finite-window DFT.Adding complex amplitude to a Gaussian does not give its smooth envelope the periodic sidelobe nulls or phase law of the DFT response.Source: §2 · p. 2
Wave-optical rendering and appearance models
- Wave-optical rendering models field propagation to an aperture, while this work models a surfel’s contribution after mixing, sampling, and Fourier transformation.The paper targets active coherent sensing with a known transmitted waveform and complex baseband returns.Source: §2 · p. 2
Differentiable coherent sensing
- Differentiating through the full waveform pipeline is physically exact but materializes the time-domain signal and FFT gradient at O(MN log MN) per step.Radar scene methods also either discard spectral information or approximate the analytic DFT response with a neural network.Source: §2 · pp. 2–3
Sparse recovery and variable projection
- The method combines sliding Frank–Wolfe, variable projection, and residual dual certificates within differentiable splatting, a combination the authors say had not previously been applied there.Off-the-grid sparse recovery supplies optimization components, while MUSIC and ESPRIT exploit signal-subspace structure but do not provide renderer gradients.Source: §2 · p. 3
3 Background and Motivation
A finite-window DFT maps coherent reflector returns to complex spectral measurements whose exact point response is a Dirichlet kernel. Gaussian splats omit that response’s sidelobe and phase structure.
3 Background and Motivation
- Each spectral bin records a complex amplitude, and nearby reflectors interfere through phase before intensity is observed.The sensor forms an L-axis spectral tensor by applying a finite-length DFT along each measurement axis.Source: §3 · p. 3
- A reflector offset by Δ from a spectral-bin center contributes the exact analytic Dirichlet kernel, whose sidelobes carry about 10% of spectral energy for a rectangular window.The kernel is complex-valued, oscillatory, and periodic; its sidelobe ratio and phase law are fixed by the DFT window length N.Source: §3 · p. 3
- A Gaussian splat has no zero crossings and cannot reproduce the periodic sidelobe nulls or phase law that govern coherent interference at sub-bin separation.The paper identifies the Dirichlet kernel as the finite-window DFT point-spread function and the Gaussian as, at best, a main-lobe approximation.Source: §3 · p. 3
- The figure traces a coherent array’s signal chain and shows each surfel becoming a Dirichlet PSF in the spectral tensor.Motion shifts the moving surfel off the f_D = 0 face.Source: Figure 2 · p. 3
4 Overview
Dirichlet Splatting reconstructs scenes from active coherent measurements by rendering oriented surfels directly in spectral space. Its renderer feeds DSFW, which uses variable projection and residual dual certificates.
4 Overview
- The renderer models oriented planar reflectors with complex reflectivity, bistatic foreshortening, and material-dependent Fresnel response, using the exact windowed-DFT PSF along each spectral axis.The surfels scatter the incident waveform once before it reaches the sensor.Source: §4 · p. 3
- The renderer’s projection, surfel-aware splats, and coherent compositing feed DSFW, an optimizer built on variable projection and residual dual certificates.Source: §4 · p. 3
5 Dirichlet Splatting
Dirichlet Splatting represents coherent scenes with planar surfels whose exact Dirichlet responses are projected onto measurement axes and summed as complex signals. Its appearance model encodes surfel geometry and wave interactions, while the Fourier-domain kernel supplies the point-spread response.
5.3 Spectral Rendering
- The renderer forms the spectral tensor by summing per-surfel splats, with each splat’s complex amplitude modulating its Dirichlet kernel product.The kernel separates the sensor model from surfel attributes.Source: §5 · pp. 3–4, §5.3.2 · p. 5
5 Dirichlet Splatting
- Each surfel contributes a complex amplitude modulated by a separable Dirichlet point-spread response, evaluated at its continuous Fourier center and measured on integer bins.The kernel is the exact response of a single point reflector for the per-axis FFT lengths.Source: §5 · pp. 3–4
5.1 Primitive Parameterization
- A surfel is a planar oriented disk parameterized by center, normal, area, complex reflectivity, velocity, and material impedance.The complex reflectivity carries magnitude and phase; impedance enters through the Fresnel factor.Source: §5.1 · p. 4
5.2.1 Complex Dirichlet kernel.
- The splat uses the complex Dirichlet kernel, the exact point-spread function of an N-point DFT, with an O(1) closed form per sample.A Taylor expansion handles removable singularities; tapered-window sensors use the corresponding window frequency response.Source: §5.2.1 · p. 4
5.2.2 Coherent appearance model.
- The surfel’s complex amplitude combines area, bistatic foreshortening, Fresnel reflectivity, and free-space spreading, with phase from round-trip propagation.For transverse-electric polarization, the Fresnel response varies with incidence angle and impedance; a perfect conductor is its |η| → 0 limit.Source: §5.2.2 · p. 4
5.3.1 Projection.
- Sensor-specific projection maps each surfel’s 3D position and velocity to Fourier-axis offsets, including range, Doppler, and azimuth in the three-axis example.The construction is exact under the stated stop-and-hop, constant-radial-velocity, negligible-range-walk, and Doppler-unambiguity regime.Source: §5.3.1 · p. 5
5.3.2 Coherent compositing.
- Coherent compositing adds complex phasors before the detector squares the result, modeling constructive and destructive interference without depth ordering or opacity.When inter-surfel phase is not modeled, the renderer can instead composite magnitude-squared splats via the Fejér kernel.Source: §5.3.2 · p. 5
6 Optimization
DSFW addresses the rugged loss landscape of Dirichlet splats by eliminating linear amplitudes, globally scoring candidate centers, and replacing low-utility surfels. It recovers ground-truth geometry from random initialization on every reported benchmark, while lacking a convergence guarantee.
6.1 Optimization Challenges
- Dirichlet sidelobes create gradient nulls and local minima, amplitude shrinkage trades positional accuracy for coefficient magnitude, and nearby splats have coupled center gradients.Plain AdamW from a cold start fails on the reported benchmarks; its diagonal preconditioning misses off-diagonal coupling.Source: §6.1 · p. 5
6.2 Variable Projection
- Variable projection solves the complex ridge-regression amplitudes in closed form for fixed centers, leaving a better-conditioned search over centers alone.Removing amplitudes from the search removes amplitude shrinkage as a degree of freedom.Source: §6.2 · p. 6, §6.1 · p. 6
6.3 Residual Dual Certificates
- The residual dual certificate scores candidate locations by how much a hypothetical splat would reduce the residual, providing a global signal that bypasses sidelobe minima.Its evaluation cost is dominated by candidate-voxel correlations and grows quickly with grid resolution.Source: §6.3 · p. 6
6.4 Dirichlet Sliding Frank–Wolfe
- DSFW keeps a fixed surfel budget, replacing the lowest-utility surfel with the certificate maximizer and periodically applying a coupled low-resolution correction.The budget reflects the resolvable-scatterer limit and avoids fitting sidelobes and noise, but classical SFW guarantees do not apply; utility accounts for amplitude and coherent-cancellation efficiency.Source: §6.4 · pp. 6–7
- DSFW recovers ground-truth geometry from random initialization on every benchmark in §7.3, but carries no convergence guarantee.With periodic Gauss–Newton correction and stabilizers, it achieves the lowest final loss among the evaluated optimizers.Source: §6.4 · pp. 6–7
7 Results and Application
Across synthetic and real measurements, Dirichlet Splatting is evaluated against splatting baselines and physics references, including tests of its forward model, solver, and reconstruction pipeline. Results span signal-space fitting, raw-signal imaging, 3D reconstruction, sub-THz imaging, and wavefront sensing.
7.1 Baselines
- Comparisons include 3DGS, coherent Gaussian methods GSRF and WRF-GS+, back-projection, and WaveformAD; experiments use shared scene parameters and controls unless noted.Experiments score against analytic FFT targets, raw range profiles, or measured captures depending on the task.Source: §7.1 · p. 7
7.2 Forward Model Validation
- Three Dirichlet primitives match the three-reflector finite-window response to −120 dB NMSE, while 96 Gaussian splats reach only −30.1 dB; joint position–velocity modeling also matches the FFT oracle to −120 dB complex NMSE.The Doppler test selects all three reflector velocities correctly; clamping velocity to zero degrades fit to −1.11 dB, and dynamic-scene reconstruction is outside scope.Source: §7.2 · pp. 7–8, Figure 4 · p. 7
7.3 Inverse Fitting in Signal Space
- Across the seven-scene signal-space benchmark, DSFW lowers mean complex MSE from 3.32×10−1 at initialization to 6.71×10−4, compared with 6.06×10−2 for Dirichlet-AdamW and 1.12×10−1 for SGD.DSFW recovers positions within 0.1 bin on 77.4% of cold-start trials, versus 0.07% for Dirichlet-AdamW.Source: Figure 5 · p. 8, §7.3 · p. 9, Figure 6 · p. 9
7.4 Inverse Imaging from Raw Signals
- Across seven dense targets, ours achieved the lowest mean support NMSE (0.136), versus 0.294 for GS+SFW, 0.387 back-projection, 0.389 3DGS, 0.391 WRF-GS+, and 0.406 image-domain GSRF.Raw-channel fitting yielded raw NMSE of 7.74×10−4 for ours versus 9.12×10−3 for GSRF.Source: §7.4 · p. 9, Figure 7 · p. 10
7.5 3D Reconstruction
- Three orthogonal views increase observed surface fraction from 0.35 to 0.596 and reach full-surface recall@2mm of 0.439 ± 0.011; on a matched three-view-cluster budget, median surface distance is 1.51 mm versus 1.92 mm for back-projection.Five clustered views reduce median surface distance to 1.49 ± 0.21 mm, while clustered views trade lower coverage for stronger recoverable scores.Source: §7.5 · p. 11, Figure 8 · p. 11
7.6 Ablations
- On the five-reflector cold-start ablation, Dirichlet plus SFW succeeds on 100.0% of trials, while replacing SFW with AdamW reduces success to 0.0% and raises MSE to 1.18×10−2.Replacing the Dirichlet kernel with a Gaussian while retaining SFW also drops success to 10.0%.Source: §7.6 · p. 11
7.7 Application: Sub-THz Imaging
- On real sub-THz letter scans, the Dirichlet reconstruction preserves three separated letters and measured finite-window sidelobes, whereas GS+SFW produces broad blobs that miss the sidelobe structure.Without pixel-accurate ground truth, the evaluation reports contrast, support compactness, and raw-profile fit residuals; the 3D-printed-object fit reaches 0.482 NMSE against stacked raw profiles.Source: §7.7 · pp. 11–12, Figure 9 · p. 12
7.8 Wavefront Sensing
- At one main-lobe FWHM separation, Dirichlet fitting reaches 0.021 waves residual RMS wavefront error versus 0.051 for a diffraction-matched Gaussian and 0.110 for back-projection; only Dirichlet and the oracle remain below λ/30 down to half an FWHM.At one FWHM, Gaussian and Dirichlet fits fail in 52% and 16% of trials, respectively.Source: §7.8 · p. 12
8 Limitations and Future Work
The method models indirect propagation as independent virtual surfels, but lacks a constraint tying them to primary geometry. Its forward model also assumes calibrated acquisition, and DSFW's utility ranking and stopping behavior can fail under calibration errors.
Indirect propagation
- Multipath and inter-object scattering can produce ghosts at correct path lengths but incorrect physical locations because virtual surfels are unconstrained by primary geometry.The optimizer lacks the method-of-images constraint that would tie those surfels to the primary geometry.Source: §8 · p. 12
Occlusion
- Opaque self-occlusion between surfels is not modeled, although the omission is benign for the reported experiments.The visibility term handles back-facing surfels; in partially transmissive modalities, occluded contributions cancel through coherent phase rather than a discrete visibility test.Source: §8 · p. 12
Hardware calibration
- The forward model assumes known aperture positions and a calibrated transceiver response; aperture-position errors create wavelength-scale phase errors.An uncalibrated antenna pattern creates per-aperture amplitude errors that the fit absorbs into α_i.Source: §8 · p. 12
DSFW failure modes
- Gross A_i mis-scaling can demote correctly placed surfels, while noise-floor miscalibration can cause noise fitting or premature stopping.An under-calibrated floor fits noise as splats; an over-calibrated floor stops short of the true sparsity.Source: §8 · p. 12
9 Conclusion
Dirichlet Splatting replaces the Gaussian footprint with an analytic Dirichlet point-spread function and models each primitive as a surfel with a wave-propagation-based complex amplitude. Dirichlet Sliding Frank–Wolfe addresses the rugged loss landscape induced by the oscillatory kernel.
Conclusion
- Dirichlet Splatting replaces the Gaussian footprint with an analytic Dirichlet point-spread function and represents each primitive as a surfel whose complex amplitude follows wave propagation.Dirichlet Sliding Frank–Wolfe addresses the rugged sidelobe landscape induced by the oscillatory kernel.Source: §9 · p. 12
A Mathematical Derivations
The appendix derives the finite-window DFT response and its derivatives, then connects separable kernels to FMCW measurements and surfel reflection. It also documents model assumptions, sensor parameters, renderer safeguards, and stabilizing corrections.
A.1 DFT response of a windowed exponential; A.2 Kernel derivative
- A windowed complex exponential has a Dirichlet DFT response whose magnitude-squared is proportional to the Fejér kernel.The bin-offset derivative includes cotangent terms whose factored form must be evaluated by a limit at integer offsets.Source: §A.1 · p. 14, §A.2 · p. 14
Appendix B.3 gives the limiting forms.
- Geometry derivatives combine amplitude changes with spectral-footprint motion, while range and azimuth coordinates do not depend on velocity.The renderer's bin offset is Δℓ,i = kℓ − μℓ,i.Source: Appendix B.3 · p. 14
A.3 Windowed kernels; A.4 Separable spectral response
- The finite-sum kernel supports sampled windows such as Hann, Hamming, and Kaiser, and separable signals yield a product of one-dimensional window spectra.For rectangular windows, each factor is a Dirichlet kernel; the renderer evaluates three one-dimensional kernels for a 3D response.Source: §A.3 · p. 14, §A.4 · p. 14
A.5 FMCW separable measurement model
- Under the FMCW model, range, Doppler, and array steering map to separable Fourier axes, each evaluated with a windowed DFT's closed-form Dirichlet kernel.The appendix describes the beat frequency, slow-time Doppler, and array steering phase underlying the mapping.Source: §A.5 · pp. 14–15
A.6 Validity of the separable model
- The separable FMCW model assumes stop-and-hop acquisition, constant radial velocity, and negligible range walk; acceleration requires a different Doppler response.Slow-time sampling also makes velocity ambiguous outside the unambiguous interval, without affecting forward-response separability.Source: §A.6 · p. 15
A.7 Fresnel response at a planar boundary
- The surfel Fresnel response follows boundary-field matching; narrow apertures leave reflectivity and impedance degenerate, so the experiments fix impedance to a material prior.The reported sub-THz experiments use metal targets, for which impedance is not estimated.Source: §A.7 · p. 16
B.1 Sensor parameters; B.2 Renderer truncation and noise floor
- Sensor constants come from device specifications, while radius-8 truncation captures over 99% of rectangular-window kernel energy for N in [64, 256].The streamed matrix-free accumulation reduces peak memory and per-step time by over an order of magnitude; the renderer also adds a learned or calibrated noise floor.Source: §B.1 · p. 16, §B.2 · p. 16
B.3 Training stabilizers and numerical safeguards
- DSFW uses periodic low-resolution Gauss–Newton correction and stabilizers including kernel continuation, pairwise repulsion, and a coarse-to-fine kernel-order schedule.A Taylor branch regularizes kernel derivatives near integer offsets, including nonzero multiples of N.Source: §B.3 · pp. 16–17
B.4 Hyperparameters and Reproducibility
The appendix specifies fixed experiment settings and candidate-grid construction, with seed and initial centers as the varying factors. Experiments ran on one GPU and averaged metrics over ten seeds.
B.4 Hyperparameters and Reproducibility
- The experiment settings in Table 4 are fixed across seeds; only the seed and initial centers vary.The table caption identifies the listed quantities as DSFW hyperparameters per experiment.Source: §B.4 · p. 17
- Experiments ran on one NVIDIA RTX A6000 with PyTorch 2.1 and CUDA 12.1, and metrics average 10 seeds.The seeds control uniform initial centers and tie-breaking among low-utility candidates.Source: §B.4 · p. 17
- Candidate grids span the sensor's range gate and azimuth field of view, and comparisons share a grid and splat budget.Multi-view and printed-object fits instead initialize centers from back-projection peaks.Source: §B.4 · p. 17, Table 5 · p. 17
C.1 Doppler validation
The Doppler extension is validated against FFT synthesis and a static control, then supplemented by cold-start and cross-scene convergence summaries.
C.1 Doppler validation
- Validation assumes constant radial velocity and no range walk; acceleration, tracking, occlusion changes, and dynamic topology are outside its scope.The protocol uses a three-reflector range–Doppler–azimuth cube with 96 range, 64 Doppler, and 8 azimuth samples.Source: §C.1 · p. 17
- The joint velocity model matches the FFT oracle to the numerical floor, while fixing Doppler center to zero worsens complex NMSE by more than two orders of magnitude.This comparison holds range and azimuth fixed.Source: §C.1 · pp. 17–18, Table 6 · p. 18
C.2 Cross-scene quantitative summaries
- Fig. 11 aggregates median complex spatial MSE trajectories, with interquartile shading, across the seven signal-space scenes.It complements the representative loss trace in the main paper.Source: Figure 11 · p. 18, §C.2 · p. 18
C.1 Doppler validation
- Across eight randomized scenes and four SNRs, mean velocity RMSE is 0.074 m/s, while the static control’s NMSE gap ranges from 11.97 dB to 8.13 dB at the endpoints.The experiment does not evaluate dynamic-scene reconstruction.Source: §C.1 · p. 18
C.2 Cross-scene quantitative summaries
- Fig. 12 summarizes two-dimensional cold-start success over initial-position offsets for five-reflector scenes, averaged across ten seeds.The caption reports a broad tested eight-bin attraction basin for the certificate update, with 77.4% aggregate success versus 0.07% for Dirichlet-AdamW.Source: Figure 12 · p. 18, §C.2 · p. 18
C.3 Per-component ablations
The ablation links reliable cold-start recovery to using the Dirichlet kernel with its tailored optimizer; runtime measurements characterize the cost and memory scaling.
C.3 Per-component ablations
- At MSE 1.18×10^-2, Dirichlet+AdamW trails Gaussian+AdamW at 8.54×10^-3; with the Dirichlet kernel fixed, DSFW yields a 106× MSE gap over AdamW.The passage attributes the rugged loss landscape to the Dirichlet kernel’s sub-bin phase and sidelobe structure, motivating co-design of kernel and solver.Source: §C.3 · p. 18, Table 11 · p. 18
C.4 Runtime and memory
- At 16,384 primitives on an RTX 5080, forward rendering costs 3.2 ms for Dirichlet versus 2.1 ms for Gaussian, and backward rendering costs 25.1 versus 23.2 ms.A DSFW step takes 50–180 ms versus 1–9 ms for AdamW, but DSFW reaches within 10–30 steps a loss AdamW does not reach within 2000 steps; mean reconstruction time for §7.4 is 16.4 s.Source: §C.4 · p. 18
- On 256^3 grids, memory is dominated by the complex output volume; primitive parameter buffers and temporary shader state account for the small increase with primitive count.Source: §C.4 · p. 18
- For the fused forward renderer, render time scales with primitive count, while peak allocation is dominated by output-grid size under the tested settings.The protocol varies primitive count on standard 2D and 3D signal grids.Source: §C.4 · p. 18, Figure 13 · p. 19, Table 7 · p. 19