AI Poincaré: Machine Learning Conservation Laws from Trajectories

Ziming Liu; Max Tegmark

Source-linked AI summary

AI Poincaré: Machine Learning Conservation Laws from Trajectories

Ziming Liu, Max Tegmark

arXiv:2011.04698v2Nov 9, 2020cs.LGastro-ph.EPnlin.SIphysics.class-ph

TL;DR

The paper addresses the challenge of discovering conservation laws from trajectory data when the governing dynamics are unknown. It introduces AI Poincaré, which estimates trajectory-manifold dimensionality and uses it to identify conserved quantities. Across five Hamiltonian systems, the method recovers exact conservation laws and also reveals periodic orbits, phase transitions, and breakdown timescales for approximate laws.

Problem
Existing approaches do not provide fully automated, data-driven discovery of conservation laws from trajectories with unknown underlying dynamics.
Method
AI Poincaré samples trajectory manifolds with a neural walk-pull procedure and infers conservation laws from PCA explained-ratio changes across length scales.
Results
AI Poincaré agrees with ground-truth conservation-law counts for all five tested Hamiltonian systems and discovers periodic orbits, phase transitions, and approximate-law breakdown timescales.
Takeaways & Limitations
Trajectory geometry can support automated discovery of conserved quantities and related dynamical structure without domain knowledge or a physical model of trajectory generation.

Abstract

from arXiv · show

We present AI Poincaré, a machine learning algorithm for auto-discovering conserved quantities using trajectory data from unknown dynamical systems. We test it on five Hamiltonian systems, including the gravitational 3-body problem, and find that it discovers not only all exactly conserved quantities, but also periodic orbits, phase transitions and breakdown timescales for approximate conservation laws.

INTRODUCTION

The paper seeks to discover conservation laws directly from trajectory data without knowing the underlying dynamical equations. AI Poincaré uses machine learning to pursue this goal and tests it on five Hamiltonian systems.

AI Poincaré aims to auto-discover conservation laws from observed trajectories while treating the underlying dynamical equations as unknown.
The approach addresses a gap between data-driven discovery and prior methods that required hand-crafted features or embedded physical assumptions.
The method is motivated by viewing trajectory closures as manifolds whose dimensionality determines the number of conserved quantities.
The paper applies AI Poincaré to five Hamiltonian systems to test discovery of conserved quantities, periodic orbits, phase transitions, and breakdown timescales.

METHOD

AI Poincaré estimates the dimensionality of the manifold traced by a dynamical system and infers conserved quantities from the dimensionality reduction it reveals. Its walk-pull sampling and explained-ratio analysis operate across length scales to identify submanifold structure.

Conservation laws constrain the permissible state manifold, with n = N − dim(M) when each independent law removes one degree of freedom.
AI Poincaré estimates trajectory-manifold dimensionality and sets neff = N − ŝ as the effective number of discovered conservation laws.
The method treats manifold dimensionality as a length-scale-dependent renormalized quantity because finite samples prevent probing infinitesimal neighborhoods while large scales introduce curvature.
The algorithm prewhitens trajectory data, samples the manifold locally with Monte Carlo walk-pull steps, and estimates dimensionality using PCA explained ratios.
The pull step trains a neural network to map noisy samples back toward the manifold, with pulled-back points characterizing its local tangent space.
The explained ratio diagram varies PCA variance fractions with noise length scale L, revealing conservation-law submanifolds in an intermediate phase between two sampling regimes.

RESULTS

Across five Hamiltonian systems, AI Poincaré recovered conserved quantities and extended discovery to symbolic formulas, phase transitions, periodic orbits, and approximate-conservation breakdown timescales.

Basic results: AI Poincaré agreed with the ground-truth number of conserved quantities for all five systems and outperformed PCA, autoencoder, and fractal dimensionality methods.The discovery criterion used an explained ratio one order of magnitude below baseline, 0.1/N.
Symbolic formula discovery: AI Feynman produced symbolic formulas for conserved quantities in many cases after AI Poincaré identified their existence.The procedure fixed the conserved-value gauge by assigning values 1 and 2 on two chosen trajectories.
Phase transition discovery: For perturbed Kepler dynamics, AI Poincaré detected the transition associated with Runge-Lenz-vector breakdown after approximately ϵ^-1 orbits.The perturbation uses F ∝r^−(2+ϵ), with precession of approximately ϵ per orbit.
Phase transition discovery: For the double pendulum, the method found a periodic orbit with neff = 3 near θ0 ≈65° and a low-angle phase with separately conserved normal-mode energies.The low-angle regime has neff = 2, while the usual regular and chaotic phases both have neff = 1.
Phase transition discovery: For the magnetic mirror, AI Poincaré identified a periodic orbit near v0 ≈1.0 and low-energy adiabatic conservation of magnetic moment with neff = 2.The periodic orbit has neff = 3; the known regular and chaotic phases have neff = 1.
Approximate conservation: In the three-body system, tight-binary energy and angular momentum were initially approximately conserved, but tidal interactions eventually caused their breakdown.The approximate conservation increased neff by 2 before breaking down.

CONCLUSIONS

The paper presents AI Poincaré as a model-independent method for discovering conserved quantities from trajectory data. Across five Hamiltonian systems, it also found periodic orbits, phase transitions, and breakdown timescales for approximate conservation laws.

CONCLUSIONS: AI Poincaré discovers conserved quantities from trajectory data without requiring domain knowledge or a physical model of trajectory generation.The authors suggest applying it to raw experimental data and improving symbolic-formula discovery through learned geometric information.
CONCLUSIONS: Tests on five Hamiltonian systems showed discovery of conserved-quantity counts, periodic orbits, phase transitions, and approximate-conservation breakdown timescales.The conclusion identifies raw experimental data and improved symbolic formulas as future directions.

Appendix A: Preprocessing details

The preprocessing step whitens trajectory data and optionally removes dimensions associated with vanishing covariance eigenvalues before subsequent analysis.

Prewhitening applies an affine transformation that gives the trajectory samples zero mean and identity covariance.
Dimensions with vanishing covariance eigenvalues are removed because their eigenvectors define linear conserved quantities.
The preprocessing threshold uses ϵp = 0.001, apart from neural-network architecture and training settings.

1. Why prewhitening helps

Prewhitening separates noise and manifold-curvature scales, making the dimensionality signal easier to detect across the explained-ratio diagram’s phase transitions.

1. Why prewhitening helps: Prewhitening helps anisotropic manifolds by making their curvature scales more similar and widening the usable interval [La, Lb].
1. Why prewhitening helps: The first phase transition tracks the noise scale, while the second depends on the ellipse’s axis ratio and manifold curvature.
1. Why prewhitening helps: For the noisy ellipse, La ∼∆L, whereas increasing anisotropy makes the second transition less prominent and harder to detect.

2. Robustness to dimensionality reduction and noise

AI Poincaré remains effective without dimensionality reduction and tolerates moderate isotropic noise, while recovering conserved quantities across noise scales.

2. Robustness to dimensionality reduction and noise: Without removing linear conserved components, AI Poincaré discovers neff ≈6 quantities at L ∈[0.1, 0.2] and neff ≈4 at L ∼> 1.
2. Robustness to dimensionality reduction and noise: The robustness experiment modifies prewhitening by dividing each principal component by λ1/2_i +ϵn, with ϵn = 0.001.
2. Robustness to dimensionality reduction and noise: The four linear conserved quantities remain detectable at noise levels σ ∼< 0.01 because isotropic noise shifts all covariance eigenvalues equally.

Appendix B: Monte Carlo Module

The Monte Carlo module learns a global manifold representation from trajectory samples, then generates local walk/pull samples for PCA-based conserved-quantity estimates.

Appendix B: Monte Carlo Module: The pull network is trained on odd/even trajectory splits and learns the manifold globally from the full trajectory.
Appendix B: Monte Carlo Module: At inference, repeated walk/pull steps perturb points and project them back toward the manifold before PCA estimates local structure.
Appendix B: Monte Carlo Module: The method can average neff across starting points, while using one starting point by default saves computer time.
Appendix B: Monte Carlo Module: Testing 100 random starting points produced the correct conserved-quantity count 100 out of 100 times across all five physical systems.

Appendix C: Method comparisons

AI Poincaré outperforms the comparison methods for discovering conserved quantities by estimating manifold dimensionality locally across scales. PCA misses nonlinear structure, while auto-encoder and fractal estimates require manually chosen thresholds or scale ranges.

Method comparisons: AI Poincaré outperforms PCA because local dimensionality estimation can capture nonlinear conserved-quantity structure beyond globally linear constraints.PCA discovers only linear conserved quantities, whereas AI Poincaré applies PCA locally rather than globally.
Method comparisons: PCA discovers only the four linearly conserved quantities in the three-body problem, while the comparison spans five physics examples with differing ground-truth dimensionalities.The five-example benchmark includes ground-truth neff values of 1, 3, 1, 1, and 6, with 2 after linear dimensionality reduction for the three-body problem.
Method comparisons: The fractal method requires manually selecting an intermediate scale range where d log N/d log L is approximately constant.This tuning compensates for scale-dependent effective dimensionality caused by noise at small scales and manifold curvature at large scales.
Method comparisons: The auto-encoder recovers correct dimensionalities only after manually setting the accurate-reconstruction threshold to 10^-3, and its results depend strongly on that choice.The double-pendulum reconstruction-error curves lack sharp phase transitions, making the threshold unclear.
Method comparisons: AI Poincaré predictions for the Figure 2 results have small deviations from ground truth, so rounding to the nearest integer provides error correction.The stability test compares blue predictions with a dashed red ground-truth line.

Appendix D: Does AI Poincar´e overfit?

The appendix tests AI Poincaré for overfitting by training on odd-numbered trajectory points and evaluating on held-out even-numbered points. Across five physical systems and multiple length scales, testing loss is not systematically larger than training loss.

Appendix D: Does AI Poincaré overfit?: AI Poincaré's overfitting check uses odd-numbered trajectory points for training and even-numbered points for testing.Training and testing losses are tabulated at various length scales.
Appendix D: Does AI Poincaré overfit?: Testing loss is not systematically larger than training loss across all five physical systems and tested length scales, providing no evidence of overfitting.The evaluation holds out even-numbered trajectory points while training on odd-numbered points.

Appendix E: Does AI Feynman overfit?

The AI Feynman evaluation tests whether discovered formulas remain constant on an unseen trajectory and whether they match the ground-truth functional form. The reported comparison finds no systematic increase in test error over training error.

Appendix E: Does AI Feynman overfit?: Test error is not systematically larger than training error when the discovered formula is evaluated on a third trajectory unseen during training.AI Feynman is trained on two trajectories and evaluated using the discovered formula's mean and standard deviation along the third.
Appendix E: Does AI Feynman overfit?: A discovered formula counts as correct when it is mathematically identical to the ground truth up to 3% variations in fitted numerical coefficients.For the harmonic oscillator, affine transformations of the conserved energy are accepted through the functional form H′ = cx2 + dp2 + e with the coefficient-ratio condition.