PaperlayerView exact revision on arXiv

Source-linked AI summary

EvoGS: Modeling Deformation Evolution for Dynamic Gaussian Splatting

Wei Dong, Shahram Shirani, Jun Chen, Han Zhou

arXiv:2609.00994v1cs.CV

TL;DR

Existing dynamic 3DGS methods often estimate deformation independently at each timestamp, limiting temporal modeling for dynamic reconstruction. EvoGS maintains persistent deformation states, extrapolates and adaptively corrects them with MLP observations, and uses deformation-aware densification; experiments report improved quality across benchmarks, with limits under long occlusions and severe topology changes.

  • Problem

    Existing dynamic 3DGS methods typically estimate Gaussian deformations independently at each timestamp rather than modeling temporal context.

  • Method

    EvoGS models deformation as temporal evolution using persistent states, historical extrapolation, adaptive MLP-based correction, and deformation-aware densification.

  • Results

    EvoGS consistently surpasses baselines across Neu3D, NeRF-DS, and HyperNeRF scenes with higher PSNR/SSIM and lower LPIPS without increasing computational cost.

  • Takeaways & Limitations

    EvoGS improves dynamic novel view synthesis quality across multiple benchmarks.

  • Takeaways & Limitations

    EvoGS targets short-horizon updates with approximately uniform frame spacing and may be less reliable under very long occlusions or severe topology changes.

Abstract

from arXiv · show

Recent extensions of 3D Gaussian Splatting (3DGS) enable real-time novel view synthesis in dynamic scenes by learning time-conditioned Gaussian deformations. However, existing MLP-based methods typically estimate deformations independently at each timestamp, making them less robust to large or abrupt motions. To address this issue, we propose \textbf{EvoGS}, a 3DGS-based dynamic reconstruction framework that models Gaussian deformation as a temporal evolution process. EvoGS maintains persistent deformation states for each Gaussian, extrapolates future states from historical deformation states, and corrects the predictions with MLP-derived observations. The correction is adaptively weighted using a temporal residual memory and evolution statistics such as deformation velocity and trajectory deviation. To further improve reconstruction quality, EvoGS introduces deformation-aware densification. Clone and split operations are performed along corrected deformation directions, while an uncertainty-aware strategy suppresses densification for Gaussians with unstable deformation histories. Experiments show that EvoGS improves dynamic novel view synthesis quality and achieves competitive performance across benchmarks.

1. Introduction

Dynamic scene reconstruction needs temporally consistent, geometrically accurate, high-fidelity rendering, but frame-wise MLP deformation estimates can produce inconsistent trajectories and flickering. EvoGS models deformation as temporal evolution with persistent states, adaptive correction, and deformation-aware densification.

  • Dynamic 3D scene reconstruction targets temporal consistency, geometric accuracy, and high visual fidelity for scenes involving object motion.
  • NeRF-based methods capture complex motion but are too slow for real-time applications because of volumetric ray marching and implicit representations.
  • Existing dynamic 3DGS methods commonly estimate per-Gaussian deformation independently at each timestep, without modeling temporal context.
  • EvoGS maintains persistent corrected deformation states, extrapolates current states from history, and refines them with MLP-derived observations using evolution statistics.
  • Deformation-aware densification guides clone and split operations with corrected deformations and suppresses densification for Gaussians with unstable deformation histories.
  • EvoGS improves dynamic view synthesis quality across multiple benchmarks.

2. Related Works

Related work spans dynamic NeRF, dynamic Gaussian Splatting, and temporal state-space modeling. EvoGS distinguishes itself by applying persistent temporal evolution and adaptive correction to Gaussian deformation.

  • Dynamic NeRF: Dynamic NeRF methods represent temporal variation through time-conditioned inputs, latent embeddings, or explicit deformation fields mapped to canonical space.
  • Dynamic Gaussian Splatting: Dynamic Gaussian Splatting extends efficient, high-fidelity Gaussian rendering to time-varying scenes using deformation fields and spatial-temporal encoders.
  • Temporal State-Space Modeling: Temporal state-space models estimate latent states recursively from historical predictions and current observations, including Kalman-inspired adaptive correction.
  • Compared with baseline 4DGS, EvoGS preserves sharper details, more stable geometry, and temporally coherent appearances across varying timesteps.
  • Baseline 4DGS produces scattered, jittery, crossing, and outlier trajectories, whereas EvoGS yields compact, structure-aligned trajectories.
  • EvoGS models Gaussian deformation as persistent temporal evolution rather than independent per-timestamp regression, using historical trajectories, MLP observations, and evolution statistics.

3. Preliminary

3DGS represents scenes with anisotropic Gaussian primitives that are projected and alpha-composited for real-time differentiable rendering. EvoGS draws conceptually on recursive state prediction and correction while avoiding full covariance propagation.

  • Radiance Modeling via 3D Gaussian Splatting: 3DGS represents a scene as spatially continuous, anisotropic volumetric Gaussian primitives.
  • Radiance Modeling via 3D Gaussian Splatting: Each Gaussian is parameterized by center, scale, rotation, opacity, and spherical-harmonics color, with scale and rotation determining its covariance.
  • Radiance Modeling via 3D Gaussian Splatting: Projected Gaussian covariances form 2D ellipsoids that are rasterized and alpha-composited according to depth ordering.
  • Radiance Modeling via 3D Gaussian Splatting: This rendering pipeline avoids volumetric ray integration and enables real-time differentiable rendering.
  • Temporal State Prediction and Correction: Temporal state-space modeling provides a recursive prediction-correction framework for estimating latent states from historical predictions and current observations.
  • Temporal State Prediction and Correction: EvoGS uses this principle conceptually without implementing a classical Kalman filter or maintaining per-Gaussian covariance matrices because nonlinear deformation covariances are costly and difficult to define.

4. Method

EvoGS models dynamic Gaussian deformation as a persistent temporal evolution process, extrapolating states from history and correcting them with current MLP observations. It further uses evolution-aware correction and deformation-adaptive densification to stabilize reconstruction and align new Gaussians with motion.

  • Persistent deformation states: EvoGS stores corrected deformation states for each Gaussian in a persistent buffer and reuses them across subsequent timestamps.The buffer provides a local motion history for current-state prediction.
  • Historical state evolution: Second-order extrapolation propagates historical motion trends into a predicted deformation state before incorporating the current observation.The extrapolated prior is a lightweight short-horizon continuity mechanism rather than a strict constant-velocity assumption.
  • MLP-derived observation: The observation network Φ estimates current deformation using tri-plane spatial features, a B-spline temporal encoder, and a lightweight MLP.This design preserves local spatial structure and provides smooth temporal interpolation.
  • Evolution-aware correction: EvoGS predicts correction weights from temporal residual memory, deformation velocities, and angular deviation to balance historical priors against current observations.Residual memory records recent prediction uncertainty, while large displacements and directional changes indicate reduced predictive reliability.
  • Deformation-adaptive control: A deformation-adaptive densification module suppresses unstable Gaussians and aligns clone/split operations with corrected deformation directions.The uncertainty-aware trigger uses prediction-observation residuals and angular deviation alongside the standard gradient threshold.
  • Optimization: The prior regularizer discourages abrupt deviations from historical motion in visible regions while leaving the adaptive correction module to determine prior-versus-observation reliance.The regularization term optimizes the observation network Φ and uses stop-gradient operations.

5. Experiments

Experiments across Neu3D, NeRF-DS, and HyperNeRF show that EvoGS improves reconstruction quality, temporal consistency, and perceptual detail across multiple Gaussian backbones. Ablations support persistent state modeling, adaptive correction, and deformation-aware densification.

  • Quantitative Comparisons: EvoGS consistently surpasses baselines across Neu3D, NeRF-DS, and HyperNeRF, with higher PSNR/SSIM and lower LPIPS.
  • Quantitative Comparisons: EvoGS remains competitive with fully explicit approaches: EvoGS(Grid4D) reaches 32.44 dB PSNR and 0.944 SSIM on Neu3D, versus FreeTimeGS at 32.25 dB and 0.946 SSIM.
  • Visual Comparisons: EvoGS improves visual quality through fewer ghosting artifacts, sharper structural recovery, and more stable geometry across diverse real-world motions.
  • Temporal Consistency: EvoGS achieves more consistent geometry across time steps than 4DGS, whose independently predicted deformations produce serious structure distortions.
  • Ablation Study: Removing temporal evolution decreases PSNR and LPIPS, while adding tri-plane and B-spline encoders improves baseline performance by +0.3 PNSR.
  • Ablation Study: The learned adaptive correction reaches 25.04 dB PSNR / 0.868 SSIM, outperforming fixed blending at 24.64 dB / 0.863 and prediction-only at 23.52 dB / 0.842.

6. Conclusion and Limitation

EvoGS models Gaussian deformation as temporal evolution using persistent states, adaptive observation correction, and deformation-aware densification. It improves dynamic novel view synthesis across benchmarks, but its reliability is bounded by short-horizon updates and the underlying backbone.

  • Conclusion: EvoGS extrapolates current deformation from persistent corrected states and adaptively refines it with MLP observations, residual memory, and motion statistics.
  • Conclusion: Deformation-aware densification follows corrected deformation directions and suppresses Gaussians with uncertain motion histories.
  • Limitation: The design targets short-horizon updates with approximately uniform frame spacing; long occlusions can accumulate historical-state errors, and severe topology changes remain constrained by the backbone.
Loading 2609.00994v1…