Source-linked AI summary
LagrangeGS: Non-Conservative Lagrangian System on Dynamic 3D Gaussian Splatting
Shogo Sato, Takuhiro Kaneko, Shoichiro Takeda, Tomoyasu Shimada, Riku Inoue, Kazuhiko Murasaki, Ryuichi Tanida
TL;DR
Dynamic 3DGS physics-aware extensions do not enforce Lagrangian mechanics, leaving trajectories inconsistent, time reversal unavailable, and long-term geometry unstable. LagrangeGS addresses these issues with a non-conservative Lagrangian formulation, tractable decoupled dynamics, time-independent forces, and local rigid alignment. Experiments show improved long-term extrapolation together with consistent time reversal and counterfactual editing without retraining.
Problem
Physics-aware dynamic 3DGS extensions fit vector fields to visual deformations without satisfying Lagrangian mechanics, producing inconsistent trajectories, limited time reversibility, and geometric collapse during long-term extrapolation.
Method
LagrangeGS formulates dynamic 3DGS as a non-conservative Lagrangian system, using TD-LNN for decoupled tractable dynamics and time-independent forces, plus LRA for local rigidity.
Results
LagrangeGS improves long-term extrapolation and enables consistent time reversal and counterfactual physics-based editing without retraining on two dynamic scene benchmarks.
Takeaways & Limitations
The framework provides an explicit Lagrangian formulation for physically grounded dynamic-scene modeling and manipulation in 3DGS.
Takeaways & Limitations
TD-LNN requires distillation from a pretrained dynamic 3DGS model, and its decoupled particle dynamics may not capture complex interactions such as collisions or contact forces.
Abstract
from arXiv · showhide
Dynamic 3D Gaussian Splatting (3DGS) achieves photorealistic reconstruction of time-varying scenes, and recent physics-aware extensions improve extrapolation by explicitly predicting velocity fields. However, these extensions merely fit vector fields to visual deformations without satisfying Lagrangian mechanics, leading to three major issues: (i) physically inconsistent trajectories, (ii) lack of time-reversibility, and (iii) geometric collapse during long-term extrapolation. In this paper, we propose LagrangeGS, which formulates dynamic 3DGS as a non-conservative Lagrangian system. While this Lagrangian formulation fundamentally solves (i), a direct application of general LNNs to dynamic 3DGS requires a large velocity-Hessian inversion for millions of Gaussian particles. To overcome this computational bottleneck, we approximate the velocity-Hessian as an identity matrix, decoupling particle dynamics for computational tractability. For (ii), we restrict the non-conservative forces to be explicitly time independent, enabling consistent backward integration. Finally, to address (iii), we introduce local rigid alignment that regularizes particle trajectories. Extensive evaluations on dynamic scene benchmarks demonstrate that LagrangeGS enables stable long-term extrapolation, consistent time reversal, and counterfactual physics-based editing without retraining.
1 Introduction
LagrangeGS formulates dynamic 3DGS as a non-conservative Lagrangian system to address physically inconsistent trajectories, time irreversibility, and geometric collapse. Its TD-LNN and LRA designs enable stable extrapolation, time reversal, and counterfactual editing.
- Physics-aware dynamic 3DGS methods fit velocity or acceleration fields without satisfying Lagrangian mechanics, causing inconsistent trajectories, limited time reversibility, and long-term geometric collapse.
- LagrangeGS is evaluated on Dynamic Object and Dynamic Indoor Scene benchmarks across extrapolation, time reversal, and counterfactual physics-based editing.
- LagrangeGS formulates dynamic 3DGS as a non-conservative Lagrangian system, fundamentally resolving physically inconsistent trajectories and defining physics manipulations.
- TD-LNN approximates the velocity-Hessian as an identity matrix for tractability and uses time-independent forces for consistent time reversal.
- LRA binds particles into locally rigid structures to prevent geometric collapse during long-term extrapolation.
- The method enables stable long-term extrapolation, consistent time reversal, and counterfactual physics-based editing without retraining.
2 Related Work
Prior dynamic scene representations are primarily visual, while physics-aware approaches use vector fields or external simulation. LagrangeGS instead builds on LNNs while simplifying their otherwise impractical coupled dynamics.
- Dynamic NeRF- and 3DGS-based models represent time-varying scenes, but purely visual models lack physics priors and have limited extrapolation performance.
- Physics-aware dynamic 3DGS methods introduce priors through vector fields or external simulation, including predicted velocity fields.
- LNNs learn the Lagrangian directly in generalized coordinates, avoiding the canonical-coordinate requirement associated with HNNs.
- LagrangeGS approximates the velocity-Hessian as an identity matrix and restricts non-conservative forces to time-independent Q(q, ˙q) for exact backward integration.
3 Methods
LagrangeGS represents dynamic 3DGS with a non-conservative Lagrangian system, combining decoupled particle dynamics, time-independent forces, and local rigid alignment for tractable and stable simulation.
- Task formulation: The task models time-varying 3D scenes from synchronized camera frames and extends evaluation beyond interpolation to long-term extrapolation, time reversal, and counterfactual editing.The manipulations modify potential energy, initial velocity, and non-conservative forces.
- Overall architecture: LagrangeGS distills pre-trained Gaussian trajectories into a TD-LNN parameterized by potential energy, initial velocity, and non-conservative forces.Particles are partitioned into objects before trajectory distillation and photometric refinement.
- Time-independent Decoupled LNN: TD-LNN assigns unit mass and approximates the velocity-Hessian as an identity matrix, decoupling particle dynamics and avoiding full-system matrix inversion.The non-conservative force is restricted to Q(q, ˙q), with no explicit time dependence.
- Physics manipulations: Time-independent forces permit backward integration, while the learned potential and force provide a closed-form per-particle acceleration.The system can recover past states using a negative integration step.
- Local Rigid Alignment: LRA aligns TD-LNN candidate positions to locally rigid supports using a differentiable Kabsch fit, preventing geometric collapse while accommodating global deformations.Each support is defined from k-nearest neighbors in the initial state.
- Physics manipulations: At inference, LagrangeGS supports forward extrapolation, time reversal, and counterfactual editing by scaling initial velocity, potential gradient, and non-conservative force.These physics manipulations do not require retraining.
4 Experiments
LagrangeGS is evaluated on dynamic-scene benchmarks for extrapolation, time reversal, and counterfactual editing, using trajectories from pretrained DefGS and FreeGave models. It improves long-horizon stability and physical consistency, while photometric effects and modeling assumptions reveal important trade-offs and limitations.
- 4.1 Experimental setup: Experiments use Dynamic Object and Dynamic Indoor Scene benchmarks, with pretrained DefGS and FreeGave backbones and standard PSNR, SSIM, and LPIPS metrics.Evaluation covers short- and long-term extrapolation, time reversal, and counterfactual physics-based editing.
- 4.2 Visual analysis of trajectories and potentials: LagrangeGS produces physically consistent trajectories and recovers scene-specific potentials, including rotationally symmetric harmonic motion and a uniform gravitational field.FreeGave trajectories show erratic fluctuations, whereas LagrangeGS captures these potentials without structural priors.
- 4.3 Extrapolation: LagrangeGS preserves smooth motion and suppresses geometric drift, volume divergence, haze, and collapse during extended extrapolation for both DefGS and FreeGave backbones.On the Fan and Factory scenes, the baselines degrade after the training window while LagrangeGS maintains scene geometry across the extended horizon.
- 4.3 Extrapolation: LagrangeGS improves DefGS photometric metrics but moderately degrades FreeGave metrics, with FreeGave’s Dynamic Object gap remaining −3.2 dB for both RGB and silhouette PSNR.On Dynamic Indoor Scene, the gap narrows from −2.5 dB in RGB to −0.6 dB in silhouette PSNR, implicating appearance effects such as moving shadows.
- 4.3 Extrapolation: Under extreme truncation t < 0.3 on Dynamic Object, LagrangeGS outperforms FreeGave, showing resilience to temporal overfitting in data-sparse regimes.The performance gap narrows as the training window is shortened from the original horizon.
- 4.3 Extrapolation: LagrangeGS remains within 1.1× its initial foreground extent during rollout, whereas FreeGave expands up to 1.9×, confirming reduced geometric divergence.The foreground pixel-area extent is measured relative to its value at t = 0.
- 4.4 Time reversal: Backward integration inverts per-particle velocity directions and yields mean forward–backward cycle errors of 1.3% on Dynamic Object and 0.19% on Dynamic Indoor Scene.Recovered past states remain structurally and visually consistent with corresponding forward trajectories despite no exact time-reversal guarantee.
- 4.5 Counterfactual physics-based editing: Rescaling the learned potential field enables inference-time editing: decreasing β slows descent, increasing β accelerates it, and initial velocity or non-conservative force can also be varied.These counterfactual simulations require no retraining.
5 Conclusion
LagrangeGS integrates Lagrangian mechanics into dynamic 3DGS through TD-LNN and LRA, improving long-term extrapolation while enabling consistent time reversal and counterfactual editing.
- LagrangeGS integrates Lagrangian mechanics into dynamic 3DGS through two key components: TD-LNN and LRA.TD-LNN provides computationally tractable dynamics and consistent time reversal, while LRA binds decoupled particles into locally rigid structures.
- TD-LNN approximates the velocity-Hessian as an identity matrix and uses a time-independent system for computational tractability and consistent time reversal.
- LRA binds decoupled particles into locally rigid structures to prevent geometric collapse during extrapolation.
- Experiments on two dynamic scene benchmarks demonstrate improved long-term extrapolation and enable consistent time reversal and counterfactual physics-based editing.
- The framework establishes an explicit formulation of dynamic scenes as Lagrangian systems for physically grounded 3DGS modeling and manipulation.