Source-linked AI summary
Stochastic Liquid Deformation Fields: An SDE Generalisation of Closed-Form Continuous-Time Cells for Dynamic 3D Gaussian Splatting
Mingzhao Li, Arghya Pal
TL;DR
D-3DGS’s closed-form CfC field provides continuous-time deformation but omits the diffusion associated with its stochastic lineage. This paper restores diffusion with a training-only Gaussian perturbation of the time gate, finding parity with deterministic CfC on D-NeRF and no benefit from noise on NeRF-DS.
Problem
The deterministic CfC retains the drift of a noise-driven liquid system but drops diffusion, motivating a solver-free way to restore the stochastic term.
Method
The method adds a small Gaussian increment to each CfC cell’s time-gate pre-activation as a one-step SDE realization, with no solver or added parameters and exact CfC recovery at λ=0.
Results
On D-NeRF the stochastic field matches deterministic CfC and is ahead of the MLP, while on NeRF-DS deterministic CfC remains best and constant noise does not help.
Takeaways & Limitations
The CfC field has a clean SDE interpretation and a free noise knob, but standard interpolation benchmarks favor keeping that knob near zero.
Takeaways & Limitations
The clean, dense D-NeRF and NeRF-DS protocol under-tests corrupted or sparse supervision, where the proposed noise is intended to help.
Abstract
from arXiv · showhide
Deformable 3D Gaussian Splatting (D-3DGS) reconstructs dynamic scenes by deforming a canonical set of 3D Gaussians through a deformation field of frame time. Replacing its MLP with a stack of Closed-form Continuous-time (CfC) cells-a Liquid Neural Network that solves the Liquid Timeconstant ODE in closed form-gives the field continuous-time behaviour at feed-forward cost. That closed form, however, is only the deterministic limit of a noise-driven system, and drops the stochastic term usually credited for the robustness of liquid networks. We put it back: a small Gaussian perturbation is added to the time gate of every CfC cell, turning the deterministic field into a simple stochastic (SDE) one. The noise is used only during training, needs no solver, and reduces exactly to the CfC when switched off. On the synthetic D-NeRF scenes the stochastic field is on par with the deterministic CfC and beats the MLP baseline on most scenes; on the real-world NeRF-DS scenes the deterministic limit is already best and adding noise does not help. The study thus gives both a clean way to read the CfC field as an SDE and an honest account of when a plain noise term helps and when it does not.
I. INTRODUCTION
The paper replaces the temporally independent D-3DGS MLP with a solver-free CfC field, then restores a dropped stochastic term by perturbing each cell’s time gate. The resulting field is competitive on synthetic scenes, while deterministic CfC remains preferable on real-world scenes.
- D-3DGS uses a positional-encoded MLP to deform canonical Gaussians into dynamic scenes, but its sampled timesteps are architecturally uncoupled.
- CfC cells provide explicit continuous-time semantics through a learned sigmoidal time gate while retaining feed-forward inference without a numerical solver.
- The method adds a small Gaussian increment to each CfC cell’s time-gate pre-activation, restoring diffusion without changing the inference path.
- At λ=0, the stochastic field exactly reduces to the deterministic CfC, and training-only noise leaves inference unchanged.
- On D-NeRF, the liquid fields are competitive with or ahead of the D-3DGS MLP, while on NeRF-DS the deterministic limit remains best.
- The noise is harmless on synthetic data but does not help and slightly hurts on real-world data, where deterministic CfC is already difficult to improve.
II. RELATED WORK
Prior work spans dynamic Gaussian deformation, solver-based continuous-time models, stochastic neural dynamics, and liquid networks. This paper positions its approach as a solver-free stochastic extension of the CfC deformation field.
- Dynamic Gaussian methods include D-3DGS’s MLP deformation field and explicit-flow or dual-deformation alternatives such as Gaussian-Flow, Shape of Motion, and FLAG-4D.
- The paper identifies a gap: existing deformation methods do not treat Gaussian deformation as an explicit stochastic continuous-time function of t.
- Neural ODE and Neural SDE approaches model continuous-time dynamics with numerical integration, incurring solver cost during forward passes.
- Liquid Time-constant networks use explicit time constants and input-dependent forcing, while CfC analytically solves the associated ODE with a sigmoidal time gate.
A. Background: Liquid Deformation Field
The liquid deformation field represents canonical Gaussians and predicts time-dependent offsets using a stack of CfC cells. Its cell updates use a shared backbone, linear heads, and a sigmoidal gate, while the surrounding D-3DGS pipeline remains unchanged.
- Each canonical Gaussian contains position, rotation, scale, opacity, and color, and the deformation field predicts position, rotation, and scale offsets from encoded spatial coordinates and time.
- The field Fθ is instantiated as a stack of D CfC cells that map an input, hidden state, and elapsed-time signal to an updated state.
- A cell combines a shared backbone, four linear heads, and a sigmoidal time gate to compute its update.
- Equation (1) is the closed-form LTC solution; hidden state is reset per pass, threaded through the stack, and read out as deformation offsets.
- The canonical Gaussians, rasterizer, loss, density control, and optimizer schedule are inherited from D-3DGS, so the contribution is confined to the cell.
B. Stochastic (SDE) Time Gate
The method restores stochasticity to the CfC by perturbing its time-gate pre-activation with one Gaussian Euler–Maruyama increment. This first-order gate-only approximation preserves solver-free inference and exactly recovers the deterministic CfC when λ=0.
- Stochastic gate: The gate pre-activation s = Waz τ + Wbz is modeled as a one-step Itô process with Gaussian increment λε.The constant λ absorbs Δt in the Euler–Maruyama discretisation.
- Stochastic gate: The stochastic cell replaces the deterministic gate with σ(Waz τ + Wbz + λε).
- Properties: At λ=0, the increment vanishes and the stochastic field becomes identical to the closed-form CfC.
- Properties: Noise is sampled only during training, leaving deterministic feed-forward inference without a stochastic solver.
- Approximation scope: The formulation is a first-order, single-step approximation that perturbs the time response without modifying the candidate states.The authors do not claim a higher-order or hidden-state SDE.
C. Implementation
The implementation adds one training-time Gaussian perturbation to each cell’s time gate while keeping the deterministic liquid-field architecture otherwise unchanged. Noise is independently sampled per cell and forward pass, with no added parameters or test-time cost.
- Architecture: The model uses six CfC cells, hidden width 128, backbone depth 2, GELU activation, and time-readout widths of 32 or 64.The time-readout width is 32 for D-NeRF and 64 for NeRF-DS.
- Architecture: Only λ is newly introduced, while the backbone width matches the deterministic liquid field.
- Noise implementation: Noise is sampled independently per cell and forward pass and disabled at inference.Its training overhead is one Gaussian sample and one addition per cell.
IV. EXPERIMENTS
Experiments compare the stochastic field with retrained MLP and deterministic CfC baselines on synthetic D-NeRF and real-world NeRF-DS scenes using PSNR, SSIM, and LPIPS. Results are interpreted with a measured-run variability floor for PSNR differences.
- Datasets: The evaluation covers eight synthetic D-NeRF scenes and seven real-world NeRF-DS scenes with specular, dynamic objects.
- Protocol: The study reports PSNR, SSIM, and LPIPS against the original D-3DGS MLP and deterministic CfC baselines.Baselines use identical training protocols and seeds.
- Protocol: The proposed SDE-LNN uses calibrated noise level λ=0.05 unless otherwise stated.
- Statistical calibration: PSNR differences below approximately 0.1 dB are treated as ties because repeated Hell Warrior runs span 41.48–41.57 dB.The reported calibration standard deviation is approximately 0.05 dB.
B. D-NeRF: Eight Synthetic Scenes
On the eight synthetic D-NeRF scenes, the stochastic liquid field generally outperforms the matched MLP, with gains concentrated on high-frequency articulated motion, while retaining solver-free, size-matched inference.
- B. D-NeRF: Eight Synthetic Scenes: 38.28 vs. 38.19 dB mean PSNR: SDE-LNN exceeds the matched D-3DGS MLP baseline across the eight D-NeRF scenes.SDE-LNN attains the best PSNR on six of eight scenes, although the aggregate gap is within measured variance.
- B. D-NeRF: Eight Synthetic Scenes: Hook gains +1.24 dB over the MLP, with additional gains on Hell Warrior, Stand Up, and Jumping Jacks.The clearest improvements occur on scenes with high-frequency articulated motion.
- B. D-NeRF: Eight Synthetic Scenes: SDE-LNN and deterministic CfC are exactly size-matched for isolating noise effects, while the liquid field can be 39% smaller than the MLP on NeRF-DS.All compared fields use feed-forward inference without a numerical solver.
- B. D-NeRF: Eight Synthetic Scenes: The D-NeRF comparison uses PSNR, SSIM, and LPIPS at 800×800, with retrained MLP and SDE-LNN models under identical protocol and seeds.The table compares eight synthetic scenes and distinguishes best, second-best, and third-best cells.
E. Noise-level Study
The noise sweep finds no aggregate benefit from constant Gaussian noise: small noise is harmless on D-NeRF, whereas the deterministic CfC is best on NeRF-DS and positive noise slightly degrades it.
- E. Noise-level Study: No noise level produces an aggregate gain in the dataset-mean sweep.The study compares metrics as a function of λ.
- E. Noise-level Study: On D-NeRF, the mean remains unchanged up to λ=0.05, while larger λ gradually washes out real motion and lowers the mean.This identifies λ=0.05 as the largest tested synthetic-data noise level before quality begins to fall.
- E. Noise-level Study: On NeRF-DS, λ=0 gives the best PSNR and every positive λ is slightly worse.Here λ=0 is the deterministic CfC limit.
F. Qualitative Results
Qualitatively, the largest D-NeRF gains appear around fast-moving articulated limbs, where the stochastic liquid field preserves sharper silhouettes than the MLP.
- F. Qualitative Results: The stochastic liquid field keeps fast-moving articulated limb edges sharper and closer to the ground-truth silhouette than the MLP.The MLP shows smearing or faint trailing ghosts around the limb.
V. DISCUSSION
On dense interpolation benchmarks, deterministic CfC already performs well, so constant gate noise is neutral at best and can hurt real-world results. The SDE formulation remains useful as a solver-free, tunable framework, with adaptive stochasticity left for noisier supervision.
- Noise efficacy: Dense temporal supervision and interpolation let deterministic CfC fit well, leaving little error for constant noise to correct.On noisier real-world scenes, the noise can do more harm than good.
- Future direction: Adaptive stochasticity is proposed for jittery supervision, including fast D-NeRF motion and difficult NeRF-DS scenes.Suggested variants include annealing λ, input-dependent λ, or diffusion in the hidden state.
- Limitations: The clean, dense evaluation protocol under-tests corrupted or sparse supervision, such as jittered timestamps, pose noise, and few-view training.The study uses one constant, gate-level λ, and D-NeRF differences fall within reported run-to-run variance.
- Takeaway: The SDE view restores diffusion with deterministic inference, but on standard interpolation benchmarks its noise knob is best left near zero.Its supported value is a clean formulation and free tunability rather than an automatic quality gain.