Source-linked AI summary
Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints
Marc Finzi, Ke Alexander Wang, Andrew Gordon Wilson
TL;DR
Existing Hamiltonian and Lagrangian models struggle with sophisticated constrained systems because generalized coordinates make the learned functions difficult. The paper uses Cartesian states with constraints enforced explicitly by Lagrange multipliers, and reports two orders of magnitude better accuracy and data efficiency, especially on chaotic and extended-body systems.
Problem
Existing Hamiltonian and Lagrangian models still struggle with sophisticated constrained systems, motivating further abstraction to make learning easier.
Method
The method separates Cartesian state variables from explicitly enforced constraints using Lagrange multipliers, yielding CHNNs and CLNNs.
Results
Two orders of magnitude improvement in accuracy and sample efficiency is reported over state-of-the-art alternatives, especially on chaotic and 3D extended-body systems.
Takeaways & Limitations
Cartesian coordinates with explicit constraints improve trajectory-prediction accuracy and data efficiency for rigid-body systems whose state is fully observed in 3D space.
Takeaways & Limitations
The method requires known constraints and Cartesian coordinates are limited to systems in physical space.
Abstract
from arXiv · showhide
Reasoning about the physical world requires models that are endowed with the right inductive biases to learn the underlying dynamics. Recent works improve generalization for predicting trajectories by learning the Hamiltonian or Lagrangian of a system rather than the differential equations directly. While these methods encode the constraints of the systems using generalized coordinates, we show that embedding the system into Cartesian coordinates and enforcing the constraints explicitly with Lagrange multipliers dramatically simplifies the learning problem. We introduce a series of challenging chaotic and extended-body systems, including systems with N-pendulums, spring coupling, magnetic fields, rigid rotors, and gyroscopes, to push the limits of current approaches. Our experiments show that Cartesian coordinates with explicit constraints lead to a 100x improvement in accuracy and data efficiency.
1 Introduction
Hamiltonian and Lagrangian models learn abstract physical functions, but generalized-coordinate formulations still struggle with sophisticated constrained systems. The paper proposes Cartesian states with explicitly enforced constraints to simplify learning.
- Hamiltonians and Lagrangians provide more fundamental, compact descriptions from which physical-system differential equations can be derived.
- Existing Hamiltonian and Lagrangian models still struggle to learn sophisticated constrained systems.
- Generalized coordinates implicitly satisfy constraints, whereas Cartesian coordinates can describe the system with constraints enforced explicitly.
- The proposed formulation separates Cartesian state variables x from constraints Φ(x), enforced through Lagrange multipliers λ.
- 100 times less data is sufficient for accurate neural-network modeling when problems are embedded in Cartesian coordinates.
- The experiments use complex chaotic and 3D extended-body systems, where explicitly constrained CHNNs and CLNNs are 10 to 100 times more accurate than HNNs and DeLaNs.
2 Background on learning dynamical systems
Physical dynamics can be modeled directly as differential equations or indirectly through scalar Hamiltonian and Lagrangian functions. Learning the latter supports trajectory prediction while imposing physical structure.
- An ordinary differential equation specifies how a state z(t) changes through dynamics f(z,t), and neural networks can learn f from trajectory data.
- Hamiltonians and Lagrangians are scalar functions from which many physical systems’ differential equations can be derived.
- Hamiltonian states concatenate generalized coordinates q and momenta p, while Lagrangian systems use q and its time derivative.
- HNNs learn Hamiltonians and DeLaNs learn Lagrangians instead of directly learning dynamics.
- Learning H or L offers a simpler scalar target and incorporates physical structure into trajectory prediction.
3 Related work
Related work established structured neural models for physical dynamics and conservation, but this paper applies explicit constraints directly to Cartesian Hamiltonian and Lagrangian learning.
- Prior work introduced stabilization, symmetry-based conservation, dissipative extensions, and generalized Lagrangian models for physical-system learning.
- The paper is the first to learn Hamiltonians and Lagrangians with explicit constraints in Cartesian coordinates.
- Explicit constraints produce two orders of magnitude improvement in accuracy and sample efficiency over state-of-the-art alternatives, especially on chaotic and 3D extended-body systems.
4 Simplifying function approximation with a change of coordinates
Generalized coordinates simplify constraint satisfaction but make Hamiltonians and Lagrangians harder to learn. Cartesian coordinates yield simpler functional forms, while explicit constraints preserve the physical restrictions.
- Generalized coordinates automatically satisfy constraints but complicate the Hamiltonian and Lagrangian functions.
- For an N-pendulum, Cartesian coordinates represent each mass using position and momentum vectors x_i and p_i.
- In Cartesian coordinates, the Hamiltonian and Lagrangian use a constant diagonal mass matrix, whereas angular coordinates produce a nonlinear, time-varying mass matrix.
- The Cartesian formulation is linear in x and quadratic in p and ẋ, while the angular formulation is nonlinear in q with off-diagonal M(q) terms.
- Cartesian expressions are easier to learn because their functional form is simpler, including for three-dimensional systems.
5 Learning under explicit constraints
The method enforces Cartesian-coordinate constraints explicitly with Lagrange multipliers, yielding constrained Hamiltonian and Lagrangian dynamics that can be learned with neural networks.
- Explicit constraint formulation: Cartesian coordinates leave physical constraints unenforced, so the method adds Lagrange multipliers to enforce them explicitly.This separates the learned state representation from the constraint equations.
- Hamiltonian mechanics with explicit constraints: Hamiltonian constraints are formed from both position constraints Φ and their momentum-dependent time derivatives, collected as Ψ(z) ∈ R^2C.The augmented state includes time-dependent multipliers λ(t) for these constraints.
- Hamiltonian mechanics with explicit constraints: The constrained Hamiltonian dynamics project the original dynamics onto the constraint surface while preserving the Hamiltonian structure.The projection uses the Jacobian of the constraints and solves for the multiplier forces.
- Lagrangian mechanics with explicit constraints: The same explicit-constraint strategy yields constrained Lagrangian dynamics using holonomic constraints Φ(x) = 0, their Jacobian, and the mass matrix.The resulting formulation combines conservative and Coriolis-like forces.
- Learning: CHNNs and CLNNs parameterize the Hamiltonian or Lagrangian with neural networks and use the constrained dynamics equations for training and integration.The approach assumes known constraints and computable Jacobians, while exploiting constant mass matrices in Cartesian mechanical systems.
6 Embedding 3D motion in Cartesian coordinates
The paper represents extended rigid bodies using Cartesian points and explicit rigidity or joint constraints, avoiding non-Cartesian orientation coordinates while retaining learnable mass and energy functions.
- Extended bodies in Cartesian coordinates: Extended bodies are embedded in inertial Cartesian coordinates without Euler angles, quaternions, or axis angles.This provides a general representation for rigid-body dynamics in arbitrary dimensions.
- Rigidity Constraints: A rigid body is represented by its center of mass and d non-colinear body-fixed points, whose distance constraints determine its orientation and position.The rigidity conditions become explicit distance constraints between the points.
- Extended bodies in d-dimensions: The inertial coordinates satisfy x = Ry + x_cm, and the rotation matrix can be recovered from the center of mass and principal-axis points.Collecting these points into X gives R = XΔ.
- Joints between extended bodies: The framework encodes joints between extended bodies and simple links between point masses as explicit geometric constraints.For simple links, constraint enforcement needs connectivity structure but not the link length itself because the projection depends on constraint gradients.
- Summary: A system is specified by objects and constraint relationships, then assigned a mass matrix and neural-network potential to form H or L and integrate constrained dynamics.The resulting primitives include Obj0D, Obj1D, Obj2D, and Obj3D.
7 Experiments
Experiments evaluate constrained and baseline neural models on increasingly complex chaotic, coupled, magnetic, gyroscopic, and rigid-body systems. Across these settings, explicit-constraint models retain lower rollout error and require less data.
- Datasets and simulated systems: The benchmark suite includes N-pendulums, spring-coupled pendulums, magnetic pendulums, gyroscopes, and rigid rotors.The experiments extend prior simple pendulum settings to chaotic and 3D extended-body systems.
- Training details: Models are trained by rolling out differentiable ODE solutions over short trajectories and minimizing averaged L1 trajectory error.Training uses four timesteps per trajectory and minibatches of 200 after tuning all models.
- Evaluating performance: Evaluation measures bounded relative state error over horizons much longer than the training trajectories and compares CHNNs and CLNNs with Neural-ODEs, HNNs, and DeLaNs.The relative error is independent of data scale and approaches 1 when predictions become orthogonal to the ground truth or much larger.
- Performance on N-pendulum systems: As N increases in N-pendulum systems, all models degrade, but CHNNs and CLNNs consistently outperform competitors with an increasing relative-error gap.Training minibatches last 0.12s, while Figure 5 evaluates rollouts for 3s.
- Long-horizon predictions: In chaotic pendulums, accurate predictions extend beyond the 0.12-second training trajectory length, despite errors from small perturbations amplifying over time.The paper reports noticeable ground-truth divergence after 15s from a small initial perturbation while its models remain accurate over longer timespans.
- Performance on 3D systems: CHNNs and CLNNs outperform competing methods on 3D systems whether complexity arises from coordinate singularities or from spring coupling and magnetic fields.The comparison covers gyroscopes and rigid rotors as well as spring-coupled and magnetic pendulums.
8 Conclusion
The paper concludes that Cartesian coordinates with explicitly enforced constraints simplify Hamiltonian and Lagrangian learning, improving data efficiency and trajectory prediction accuracy by two orders of magnitude. The approach applies to fully observed rigid-body systems in 3D but not to some Hamiltonian systems outside physical space.
- Cartesian coordinates combined with explicit constraints improve data efficiency and trajectory prediction accuracy by two orders of magnitude.
- The approach can embed arbitrary extended-body systems into purely Cartesian coordinates.
- The method applies to rigid-body systems whose state is fully observed in 3D, including robotics settings.
- Cartesian coordinates cannot simplify learning for some Hamiltonian systems outside physical space, such as the Lotka-Volterra equations.
9 Broader Impacts
The paper relates accurate physical-system modeling to robotics, model-based reinforcement learning, and data-driven control. It argues that broadly applicable learned dynamics could reduce expert effort in designing safe and accurate controllers.
- Accurate physical-system models have applications in robotics, model-based reinforcement learning, and data-driven control systems.
- A model that learns the dynamics of arbitrary systems could reduce the expert time needed to design safe and accurate controllers in new environments.
Appendices
The appendices provide supplemental experiments, derivations, implementation details, benchmark descriptions, and analyses of generalized-coordinate complexity. They also report energy-conservation behavior and lower error variance for CHNNs and CLNNs.
- Appendix B reports energy conservation, linear-scale relative error, constraint drift, passively enforced constraints, and effective dimensionality.
- Appendix C derives constrained Hamiltonian and Lagrangian mechanics and the embedding of 3D motion in Cartesian coordinates.
- Appendix D documents implementation and training details for the proposed method and baselines, including constraint Jacobians.
- Appendix E describes the 3D benchmark systems and their Hamiltonians, while Appendix F derives their generalized-coordinate Hamiltonians.
- CHNNs and CLNNs outperform Neural ODEs, HNNs, and DeLaNs in true-energy conservation across the evaluated systems.On the 1-pendulum task, all models approximately conserve energy; baseline trajectories diverge more quickly on chaotic 3- and 5-pendulum systems.
- CHNNs and CLNNs have lower relative-error variance than the baseline methods on the pendulum systems.
B.2 Relative error in linear scale
The appendix examines linear-scale error, constraint removal and drift, effective dimensionality, constrained mechanics, and Cartesian formulations for extended bodies. These passages connect explicit constraints and Cartesian coordinates to model accuracy, complexity, and rigid-body representation.
- B.2 Relative error in linear scale: CHNNs and CLNNs have lower relative-error variance than other methods when pendulum errors are shown on a linear scale.
- B.3 Removed constraints and constraint violation: Removing explicit constraints from Cartesian CHNNs and CLNNs rapidly degrades performance, making it slightly worse than HNN after removing one constraint.The unconstrained Cartesian models can perform more than 100 times better than HNN, but this advantage disappears when constraints are removed.
- B.3 Removed constraints and constraint violation: Numerical integration introduces small constraint drifts even though the continuous-time constrained dynamics preserve constraints exactly.The violation decreases as the integrator relative tolerance becomes stricter and contributes only a small amount to rollout error.
- B.4 Effective Dimension: Effective dimensionality measures model complexity through the decay of the Hessian eigenspectrum, with smaller eigenvalues contributing less above a soft cutoff.
- B.4 Effective Dimension: CHNN eigenspectra decay more rapidly than HNN eigenspectra on harder N-Pendulum tasks, corresponding to lower effective dimensionality.The comparison uses normalized eigenvalues and averages a 95% confidence interval over 3 independently trained models.
- C.1 Constrained Hamiltonian and Lagrangian Mechanics: Constrained Hamiltonian mechanics augments the dynamics with constraints and Lagrange multipliers so the state evolution satisfies both equations of motion and constraint equations.
- C.1 Constrained Hamiltonian and Lagrangian Mechanics: The constrained Lagrangian formalism explicitly enforces holonomic constraints because they are not implicit in Cartesian coordinate choices.
- Extended-body systems: Cartesian coordinates give extended-body models a constant mass-distribution representation, including translational and rotational kinetic-energy components.Rigid bodies can be embedded in Cartesian space by selecting linearly independent points in the body.
D.1 Dataset generation
The paper benchmarks its approach on synthetic constrained physical systems whose data are generated from known Hamiltonians and numerical integrations. The benchmark spans linked pendulums, spring-coupled masses, magnetic pendulums, rigid bodies, gyroscopes, and systems whose Cartesian equations retain a simple form as complexity grows.
- Training datasets use 800 sampled initial conditions, with dynamics evaluated at 100 system-specific timesteps using an adaptive Runge-Kutta4(5) solver.
- Data-efficiency experiments generate 10,000 four-timestep trajectories and vary training-set size through nested subsets of those trajectories.
- The benchmark includes N linked pendulums with distance constraints, whose Hamiltonian contains kinetic-energy and gravity contributions.
- Additional systems test spring coupling, magnetic interactions, spinning tops, rigid bodies, and gyroscopes under Cartesian-coordinate Hamiltonians and explicit constraints.
- For N-pendulums, generalized-coordinate equations become very long, whereas Cartesian-coordinate equations keep the same functional form irrespective of N.