Source-linked AI summary
Lagrangian Neural Networks
Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, Shirley Ho
TL;DR
Neural networks struggle to learn physical symmetries and conservation laws. The paper introduces LNNs, which learn arbitrary Lagrangians without canonical coordinates; they conserve energy on a double pendulum, handle relativity without canonical coordinates, and model the 1D wave equation with a graph extension.
Problem
Neural networks struggle to learn physical symmetries and conservation laws, while Hamiltonian methods require canonical coordinates.
Method
LNNs parameterize arbitrary Lagrangians with neural networks and use them to compute dynamics without restricting the learned energy form.
Results
LNNs nearly conserve double-pendulum energy, with 0.4% mean energy discrepancy versus 8% for the baseline, and handle relativity without canonical coordinates where HNNs fail.
Takeaways & Limitations
LNNs provide energy-conserving models for systems with unknown canonical momenta and extend to graphs and continuous systems through a Lagrangian Graph Network.
Takeaways & Limitations
Hamiltonian methods require canonical coordinates, which may be unavailable or difficult to compute for some datasets.
Abstract
from arXiv · showhide
Accurate models of the world are built upon notions of its underlying symmetries. In physics, these symmetries correspond to conservation laws, such as for energy and momentum. Yet even though neural network models see increasing use in the physical sciences, they struggle to learn these symmetries. In this paper, we propose Lagrangian Neural Networks (LNNs), which can parameterize arbitrary Lagrangians using neural networks. In contrast to models that learn Hamiltonians, LNNs do not require canonical coordinates, and thus perform well in situations where canonical momenta are unknown or difficult to compute. Unlike previous approaches, our method does not restrict the functional form of learned energies and will produce energy-conserving models for a variety of tasks. We test our approach on a double pendulum and a relativistic particle, demonstrating energy conservation where a baseline approach incurs dissipation and modeling relativity without canonical coordinates where a Hamiltonian approach fails. Finally, we show how this model can be applied to graphs and continuous systems using a Lagrangian Graph Network, and demonstrate it on the 1D wave equation.
1 INTRODUCTION
Neural networks struggle to learn physical symmetries and conservation laws. LNNs address this by learning arbitrary Lagrangians without canonical coordinates or restricted kinetic-energy forms.
- Neural networks may fail to learn basic physical abstractions even after observing many examples.
- The central challenge is learning physical symmetries and conservation laws within neural-network models.
- Hamiltonian approaches require canonical coordinates satisfying strict Poisson-bracket relations.
- Lagrangians conserve total energy while permitting arbitrary coordinates, addressing datasets where canonical momenta are difficult to compute.
- LNNs learn unrestricted Lagrangians and target systems that Hamiltonian models or baselines handle poorly.
2 THEORY
Lagrangian mechanics assigns action values to paths and selects stationary-action trajectories. With a black-box Lagrangian, the Euler–Lagrange equations are numerically expanded to compute accelerations and integrate system dynamics.
- A system follows the path whose action is stationary, with the Lagrangian defined as kinetic energy minus potential energy.
- The Euler–Lagrange equation describes the physical path produced by the stationary-action requirement.
- For a parametric Lagrangian, the Euler–Lagrange equation is rewritten in vectorized form because analytical expansion is unavailable.
- Applying the chain rule introduces acceleration and velocity terms, after which a matrix inverse solves for acceleration.
- Given coordinates and velocities, the method computes accelerations from a black-box Lagrangian and integrates them to obtain system dynamics.
3 RELATED WORK
Prior work adds physics-inspired inductive biases or learns conserved quantities through neural Hamiltonians. DeLaN learns a restricted class of Lagrangians suited to rigid-body dynamics, whereas LNNs avoid that kinetic-energy restriction.
- Physics-informed neural networks include domain-specific models and general interaction networks that represent physical interactions through graph message passing.
- Hamiltonian Neural Networks learn invariant quantities by approximating a Hamiltonian with a neural network.
- LNNs instead learn Lagrangians so the learned kinetic energy is not restricted by the Hamiltonian formulation.
- DeLaN assumes kinetic energy is quadratic in velocity with a positive-definite, coordinate-dependent matrix, which works well for rigid-body dynamics.
- This DeLaN kinetic-energy assumption excludes systems such as charged particles in magnetic fields and fast-moving relativistic objects.
4 METHODS
Training LNNs requires differentiating twice through the learned Lagrangian and solving a Hessian-based system. The method therefore uses pseudoinverses and smooth activations, selecting softplus for the experiments.
- Computing LNN dynamics requires an inverse Hessian and backpropagation through the neural network.
- The matrix inverse scales as O(d3) with the number of coordinates d, and a pseudoinverse avoids potential singular matrices.
- Baseline, HNN, and LNN models use four-layer networks with 500 hidden units, a decaying learning rate starting at 10^-3, and batch size 32.
- Because LNNs use second-order derivatives, ReLU is unsuitable and the authors search over several activation functions.
- Softplus performed best on the double-pendulum search and was used for all experiments.
5 EXPERIMENTS
The experiments evaluate LNNs on double-pendulum, relativistic-particle, and wave-equation tasks, emphasizing energy conservation, coordinate flexibility, and graph-based modeling.
- Double pendulum: 8% versus 0.4% mean energy discrepancy for the baseline and LNN models, respectively, on the double pendulum.The comparison averages 40 random initial conditions over 100 time steps and measures discrepancy as a percentage of maximum potential energy.
- Double pendulum: The LNN conserves the double pendulum’s total energy significantly better than the baseline over long time periods.Both models perform similarly when modeling short-term dynamics.
- Relativistic particle: The HNN fails with non-canonical coordinates, succeeds with canonical coordinates, and the LNN learns accurate relativistic dynamics from non-canonical coordinates.The relativistic particle task uses a uniform potential and tests models under different coordinate choices.
- Wave equation with Lagrangian Graph Networks: The Lagrangian Graph Network models the wave equation by summing Lagrangian densities computed over adjacent gridpoint groups.For the 1D grid, the relevant groups use neighboring coordinates, and the density model is vectorized across the grid.
- Wave equation with Lagrangian Graph Networks: The Lagrangian Graph Network models the wave equation accurately and almost exactly conserves energy integrated across the material.The experiment uses periodic boundary conditions; the plotted waves make approximately three and a half passes across 100 grid points.
6 CONCLUSION
The paper introduces Lagrangian Neural Networks for learning arbitrary Lagrangians without canonical coordinates. Experiments demonstrate energy conservation, learning non-trivial canonical momenta, and a graph-based extension to the 1D wave equation.
- Lagrangian Neural Networks learn arbitrary Lagrangians without requiring canonical coordinates.
- The experiments demonstrate effective total-energy conservation on the double pendulum.
- The model learns non-trivial canonical momenta in a task where Hamiltonian learning struggles.
- A graph version of the model is demonstrated on the 1D wave equation.
A SOLVING EULER-LAGRANGE WITH JAX
The appendix presents a JAX implementation of the parametric Euler-Lagrange dynamics. The implementation differentiates the Lagrangian and uses its Hessian to compute accelerations, then supports gradient calculation for training.
- The implementation accepts a differentiable Lagrangian, coordinates, velocities, accelerations, and non-dynamical parameters.
- The JAX code uses the Lagrangian Hessian pseudoinverse together with coordinate and velocity derivatives to compute accelerations.
- Gradients of the training loss with respect to LNN parameters are obtained with jax.grad.
B EXAMPLE OF LAGRANGIAN FORWARD MODEL
The forward model derives physical dynamics from a learned Lagrangian by calculating the required derivatives. For a ball in gravity, these derivatives yield downward acceleration along q1 and constant velocity along q2.
- The example considers a ball falling under gravity g along the direction q1.
- The forward model obtains the dynamics by calculating the required derivatives of the Lagrangian.
- The resulting motion has downward acceleration along q1 and constant velocity along q2.
C INITIALIZATION
Classical initialization schemes were insufficient for the nonlinear Lagrangian objective, so the authors empirically optimized gradient behavior and fit an initialization formula across network architectures.
- Classical Kaiming and Xavier initialization schemes were insufficient for the nonlinear Lagrangian objective.These schemes target output-gradient mean zero and standard deviation one in regular neural networks.
- The authors optimized the KL-divergence between each parameter gradient and a univariate Gaussian across network depths and widths.They then fit an empirical formula to the optimization results.
- The empirical search used 2500 optimization steps per initialization setting across approximately 200 random hyperparameter configurations.The configurations varied hidden nodes from 50 to 300 and hidden layers from one to three.
- The model used 2 input coordinates and 2 input coordinate velocities sampled from univariate Gaussians.
- A 100-node, 4-layer model used weight-matrix shapes {(4, 100), (100, 100), (100, 100), (100, 1)} and initialization variances {0.22, 0.058, 0.116, 10}.Each matrix was sampled from a zero-mean Gaussian with its corresponding variance.