Source-linked AI summary
Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems
Elizabeth Qian, Boris Kramer, Benjamin Peherstorfer, Karen Willcox
TL;DR
High-dimensional nonlinear dynamical systems need low-dimensional models, but traditional reduction can require intrusive solver access while generic machine learning ignores governing physics. Lift & Learn uses equation-informed lifting, projection, and operator inference to learn quadratic reduced models from original-model data. The resulting models preserve lifted physics, match data at least as well as an intrusive reduced model, and generalize beyond training conditions in the reported examples.
Problem
Low-dimensional models are needed for high-dimensional systems, while traditional reduction is intrusive and classical machine learning ignores problem physics.
Method
Lift & Learn identifies equation-informed lifting transformations, projects lifted original-model data onto a low-dimensional basis, and fits reduced quadratic operators.
Results
Lift & Learn models capture system physics at least as accurately as traditional intrusive reduced models and demonstrate generalizability on FitzHugh-Nagumo and compressible Euler examples.
Takeaways & Limitations
Preserving quadratic physics makes the learned models applicable beyond their training conditions and supports an upper bound on training-data residuals.
Takeaways & Limitations
The lifting transformation is generally non-unique, may be non-invertible, and each added auxiliary variable increases the learning problem’s dimension.
Abstract
from arXiv · showhide
We present Lift & Learn, a physics-informed method for learning low-dimensional models for large-scale dynamical systems. The method exploits knowledge of a system's governing equations to identify a coordinate transformation in which the system dynamics have quadratic structure. This transformation is called a lifting map because it often adds auxiliary variables to the system state. The lifting map is applied to data obtained by evaluating a model for the original nonlinear system. This lifted data is projected onto its leading principal components, and low-dimensional linear and quadratic matrix operators are fit to the lifted reduced data using a least-squares operator inference procedure. Analysis of our method shows that the Lift & Learn models are able to capture the system physics in the lifted coordinates at least as accurately as traditional intrusive model reduction approaches. This preservation of system physics makes the Lift & Learn models robust to changes in inputs. Numerical experiments on the FitzHugh-Nagumo neuron activation model and the compressible Euler equations demonstrate the generalizability of our model.
1. Introduction
Lift & Learn combines governing-equation knowledge with data-driven reduction by lifting nonlinear dynamics into coordinates with quadratic structure. It learns reduced quadratic operators non-intrusively while preserving system physics.
- Lift & Learn uses governing-equation knowledge to identify lifted coordinates in which low-dimensional models can be learned.
- Traditional projection methods require intrusive access to high-dimensional solver operators, whereas classical machine learning treats the solver as a physics-agnostic black box.
- Lifting transformations provide finite-dimensional coordinates with quadratic dynamics for a large class of nonlinear terms appearing in engineering PDEs.
- The method simulates original-model trajectories, lifts the data, projects it onto a low-dimensional basis, and fits reduced quadratic operators by operator inference.
- Its contributions include non-intrusive learning when lifted full models are unavailable and guarantees on the lifted model’s fit to data.
2. Projection-based model reduction
Projection-based reduction approximates high-dimensional dynamical systems in a data-derived low-dimensional subspace. POD-Galerkin preserves polynomial structure, but unavailable high-dimensional operators require alternative ways to obtain reduced operators.
- A semi-discrete nonlinear PDE becomes a system of n-dimensional ordinary differential equations after spatial discretization, and reduction seeks computational speed-ups.
- The POD basis consists of leading singular vectors of the snapshot matrix, and the reduced state approximates the full state in that basis.
- Galerkin projection defines the POD-Galerkin reduced model by projecting the full dynamics onto the POD subspace.
- When full operators have polynomial nonlinearities, the reduced model preserves that structure and can be evaluated using reduced matrix operators.
- When high-dimensional operators are unavailable, intrusive computation of reduced operators is impossible and alternative means are required.
3. Lift & Learn: Physics-informed learning for nonlinear PDEs
Lift & Learn identifies lifting transformations that expose quadratic structure in nonlinear PDEs, then learns efficiently evaluable reduced operators from lifted data generated by the original model.
- 3.1. Exposing structure via lifting transformations: Lift & Learn identifies a transformation that maps nonlinear PDE dynamics into a quadratic representation and applies it to data from the original nonlinear model.The lifting map may introduce auxiliary variables for non-quadratic terms.
- 3.1. Exposing structure via lifting transformations: A quadratic lifting requires a differentiable map with bounded derivative and lifted states governed componentwise by linear and quadratic functions.The lifted state is the higher-dimensional vector field satisfying the resulting lifted PDE.
- 3.1. Exposing structure via lifting transformations: The lifting map is generally non-unique, and each additional auxiliary variable increases the learning dimension; finding a minimal lifting remains open.The reverse lifting map is also non-unique and can act as a regularization choice when the lifting is not invertible.
- 3.2. Obtaining lifted reduced data: Lifted state snapshots are formed by simulating the original full model, applying the discrete lifting map node-wise, and projecting the transformed data onto leading POD modes.The POD basis is obtained from the SVD of the lifted data matrix.
- 3.2. Obtaining lifted reduced data: Reduced derivative data are obtained by reversing projected lifted states, evaluating the nonlinear full model, and applying the lifting Jacobian through the chain rule.This procedure supplies derivative data compatible with dynamics that are Markovian in the reduced state.
- 3.3. Operator inference: Linear and quadratic reduced operators are learned from reduced states and derivatives by solving independent least-squares problems through operator inference.The procedure minimizes snapshot-wise squared losses and reconstructs a symmetric quadratic operator after removing redundant Kronecker-product terms.
4. Finite difference analysis
The analysis considers noiseless, consistent finite-difference discretizations and establishes bounds for Lift & Learn’s quadratic reduced-model residual. These bounds relate the learned model’s mismatch to discretization and POD projection errors, while the method assumes only the original discretized nonlinear model is available and stable.
- Analysis setting: The analysis assumes data from a consistent finite-difference discretization of the original nonlinear PDE with no noise.The discretized nonlinear model is assumed available and stable; lifted discrete operators are required to exist for analysis but need not be available.
- Consistency: The lifted and original discrete dynamics are related through consistency of the discretizations and the lifting transformation.The analysis compares the continuous solution sampled at grid points, the semi-discrete state, and the corresponding lifted state.
- Model comparison: The non-intrusive Lift & Learn model can model the data at least as well as an intrusive reduced model while retaining a physics-respecting quadratic form.This structure permits an upper bound on the training-data residual.
- Proof strategy: The proof combines bounds for lifted dynamics, projected quadratic dynamics, and the inverse lifting map using triangle inequalities and Lipschitz properties.The resulting minimizer is bounded by an objective value constructed from the projected lifted dynamics.
- Residual bound: The residual measures mismatch between the postulated quadratic dynamics and the true dynamics, and Theorem 1 bounds it using full-model truncation and POD projection errors.The theorem expresses the bound through the singular-value tail ε2 = Σ_{i=r+1}^{ndw} σ_i^2 and nonnegative constants C0, C1, and C2.
5. Results
The experiments apply Lift & Learn to FitzHugh–Nagumo and Euler systems, showing that learned reduced models retain quadratic structure and achieve accuracy comparable to intrusive lifted POD models.
- FitzHugh–Nagumo: The FitzHugh–Nagumo cubic nonlinearity is transformed into quadratic form by adding lifted variables without approximation.The lifted system contains only quadratic nonlinear state dependencies.
- FitzHugh–Nagumo: The FitzHugh–Nagumo model is trained from 3600 snapshots generated by nine simulations over prescribed α and β parameter combinations.Each simulation uses 400 snapshots on a spatial grid with n = 512 nodes.
- FitzHugh–Nagumo: For r ≤10, Lift & Learn errors are nearly the same as lifted POD errors across training and both FitzHugh–Nagumo test sets.Same-regime test errors match training-scale errors, while new-regime errors increase similarly for both reduced models.
- FitzHugh–Nagumo: In the new FitzHugh–Nagumo parameter regime, accuracy is limited by the POD basis's ability to represent trajectories beyond the training regime.Lift & Learn recovers the intrusive lifted POD accuracy while avoiding the need for an available lifted full model.
- Euler equations: The Euler equations admit a quadratic specific-volume representation through a nonlinear state transformation without auxiliary variables.For constant γ, the transformation is invertible and ds = dw.
6. Conclusions
Lift & Learn uses lifting transformations to expose quadratic structure, then learns low-dimensional models from data that generalize beyond training conditions while preserving physics.
- The method uses lifting transformations to expose quadratic structure in nonlinear dynamical systems.
- Lift & Learn learns quadratic reduced models from data generated in the original nonlinear variables, without requiring a lifted full model.
- The physics-informed formulation respects lifted-variable physics, allowing the learned model residual to be upper bounded by full-model truncation and reduced-basis projection errors.
- Numerical experiments on two different test problems demonstrate the method's generalizability.