Source-linked AI summary
Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction
N. Benjamin Erichson, Michael Muehlebach, Michael W. Mahoney
TL;DR
Scientific neural models often omit physical knowledge, raising questions about their robustness and generalization. This paper incorporates Lyapunov stability into autoencoders for fluid-flow prediction and reports improved generalization, reduced prediction uncertainty, and lower hyperparameter sensitivity. The approach is evaluated on simulated and real-world flow and climate problems.
Problem
Neural-network models for scientific applications often ignore physical insights, motivating physics-informed priors that may improve generalization, parameter sensitivity, and robustness.
Method
The paper trains autoencoders with a Lyapunov stability-preserving prior for fluid-flow prediction and evaluates them on simulated and real-world problems.
Results
The stability-preserving models improve generalization performance and reduce prediction uncertainty compared with physics-agnostic models.
Takeaways & Limitations
Lyapunov stability provides a physically meaningful regularization mechanism that also reduces sensitivity to hyperparameters.
Takeaways & Limitations
The study assumes linear underlying dynamics, and prediction remains challenging for complex ocean dynamics with substantial error.
Abstract
from arXiv · showhide
In addition to providing high-profile successes in computer vision and natural language processing, neural networks also provide an emerging set of techniques for scientific problems. Such data-driven models, however, typically ignore physical insights from the scientific system under consideration. Among other things, a physics-informed model formulation should encode some degree of stability or robustness or well-conditioning (in that a small change of the input will not lead to drastic changes in the output), characteristic of the underlying scientific problem. We investigate whether it is possible to include physics-informed prior knowledge for improving the model quality (e.g., generalization performance, sensitivity to parameter tuning, or robustness in the presence of noisy data). To that extent, we focus on the stability of an equilibrium, one of the most basic properties a dynamic system can have, via the lens of Lyapunov analysis. For the prototypical problem of fluid flow prediction, we show that models preserving Lyapunov stability improve the generalization error and reduce the prediction uncertainty.
1 INTRODUCTION
The paper argues that physics-agnostic neural models can be brittle in scientific settings, motivating physics-informed learning for fluid-flow prediction. It introduces stability-preserving autoencoders using a Lyapunov-based prior to improve training and generalization.
- Physics-agnostic neural models can be brittle under noise, small training sets, and many hyperparameters.
- The paper designs stability-preserving models for fluid-flow prediction by learning an end-to-end mapping between input and target flow snapshots.
- The model uses an autoencoder with an additional component that learns the dynamics of the underlying physical process.
- A Lyapunov stability-promoting prior constrains empirical risk minimization as a physically meaningful regularization mechanism.
- The stability-promoting parameter controls a fitting-stability trade-off while supporting better training and generalization than physics-agnostic models.
2 PROBLEM SETUP
The paper models fluid-flow prediction as learning a low-dimensional dynamical system from high-dimensional, noisy observations. It adopts linear dynamics for tractability while recognizing that estimating the governing map from observations remains challenging.
- The system is modeled as a dynamical process in which the next state depends on the current state plus a small perturbation.
- Linear time-invariant dynamics are used as a tractable approximation to nonlinear systems.
- Estimating the linear map is challenging because data-driven learning accesses high-dimensional observations rather than the underlying state directly.
- Fluid-flow observations are modeled as measurements of a low-dimensional state, with perturbations representing modeling and discretization errors and measurement noise.
3 AUTOENCODER-TYPE MODELS FOR FLUID FLOW PREDICTION
The proposed autoencoder maps high-dimensional flow snapshots into a low-dimensional latent space, evolves them with a learned dynamics map, and decodes predictions back to observation space. Training combines prediction error with an inverse-consistency penalty so the latent dynamics remain isolated.
- The model learns a snapshot-to-snapshot map by composing encoder Ψ, latent dynamics Ω, and decoder Φ.
- The encoder compresses snapshots into a low-dimensional feature space while preserving coherent flow structure and suppressing fine-scale variance.
- The decoder maps time-evolved low-dimensional features back to the high-dimensional measurement space.
- Inference produces multistep predictions by repeatedly feeding each predicted snapshot back into the learned model.
- An additional penalty enforces approximate inverse consistency between encoder and decoder, with λ balancing the prediction and consistency objectives.
- The skip connection supports an identity-preserving encoder constraint so that only Ω captures the dynamics.
4 LYAPUNOV STABILITY AS A TOOL FOR PHYSICS-INFORMED LEARNING
The paper uses Lyapunov stability to define and enforce bounded, robust dynamics in physics-informed autoencoders. A stability-promoting prior based on a matrix P and its eigenvalues is incorporated into training.
- 4 LYAPUNOV STABILITY AS A TOOL FOR PHYSICS-INFORMED LEARNING: Stable approximations prevent small perturbations near the origin from producing unbounded predictions and act as a principled regularization mechanism.Numerical examples show improved generalization performance and reduced sensitivity to hyperparameters.
- 4.1 Lyapunov stability: Lyapunov stability requires trajectories initialized sufficiently close to an equilibrium to remain close for all future times.For asymptotic stability, those trajectories must also converge to the equilibrium.
- 4.1 Lyapunov stability: The second method of Lyapunov replaces full trajectory analysis with an energy-like function V that is positive away from equilibrium and nonincreasing along trajectories.This provides a practical stability-checking tool without computing every future trajectory.
- 4.1 Lyapunov stability: For linear dynamics, stability can be assessed through eigenvalues, while asymptotic stability requires all eigenvalues of A to have magnitude strictly less than one.Stable linear dynamics yield bounded trajectories and therefore bounded predictions when the encoder and decoder maps are bounded.
- 4.1 Lyapunov stability: The paper converts stability assessment into checking whether a symmetric matrix P satisfying A^⊤PA − P = −Q is positive definite.This formulation also simplifies gradient computation because P has real eigenvalues.
- 4.2 Physics-informed models based on Lyapunov’s method: The proposed autoencoder adds a prior that penalizes small negative eigenvalues of P, thereby promoting Lyapunov stability during training.The training objective includes this stability-promoting penalty, with γ set to 4 in the experiments; stability is preserved when κ is sufficiently large.
5 EXPERIMENTS AND DISCUSSION
Experiments compare physics-aware and physics-agnostic autoencoders on simulated cylinder flow and real-world Gulf of Mexico SST data. Across tuning configurations, stability-aware models improve generalization, reduce prediction uncertainty, and may face higher error on the more complex SST task.
- Experimental setup: The experiments evaluate stability-preserving and physics-agnostic autoencoders on periodic cylinder flow and six years of Gulf of Mexico SST data.The cylinder-flow dataset contains 250 snapshots, while the SST dataset contains 2,190 daily snapshots.
- Generalization results: Physics-aware models show improved generalization over a larger prediction range across varying learning-rate and weight-decay configurations.For cylinder flow, models are evaluated by future-field prediction and averaged over 30 initial conditions.
- Generalization results: The stability-promoting prior reduces prediction uncertainty for the cylinder-flow predictions.
- Stability interpretation: Adjusting weight decay can shrink all eigenvalues of Ω below magnitude one, but the physics-informed prior offers a more interpretable alternative.
- Limitations and scope: SST prediction error is substantially larger than in the earlier examples because seasonal ocean dynamics create a more challenging prediction problem.The authors report that deeper networks might improve performance, although an initial residual-block autoencoder did not outperform the reported results.
6 CONCLUSION
The paper concludes that Lyapunov-stability-preserving autoencoders provide a simple physics-informed enhancement for fluid-flow prediction. Across several tasks, the approach improves performance and remains practical despite requiring one additional tuning parameter.
- Conclusion: The proposed stability-enhancing prior produces a substantial performance boost across several fluid-flow prediction tasks.
- Conclusion: Physics-informed stability constraints improve generalization performance and reduce prediction uncertainty in the evaluated flow-prediction problems.
- Conclusion: The method requires an additional tuning parameter, but the authors report that this parameter is relatively robust to tune.
A NETWORK ARCHITECTURES
The experiments use similar shallow autoencoder architectures, with a wider design for the more complex SST dataset. The architecture tables specify the corresponding training batch sizes.
- Architecture choices: The autoencoders use similar architectures across experiments, with a wider first and last layer and a smaller dynamics layer for SST.The wider SST design is intended to provide more expressive power for the more complex problem.
- Training configuration: The cylinder-flow autoencoder is trained with a batch of 34 samples.
- Training configuration: The SST autoencoder is trained with a batch of 156 samples.
B VISUAL RESULTS
The visual results compare physics-aware predictions across fluid-flow and climate settings, showing accurate short-horizon flow forecasts and better climate-field approximation than the physics-agnostic model.
- Flow past cylinder: The physics-aware model accurately predicts future flow fields behind a cylinder from four initial conditions over a short horizon.The estimated snapshot closely matches the true target, while both models perform well in this simple case.
- Reconstructed dynamics: The physics-aware and physics-agnostic models learn different dynamics, reflected in their reconstructed eigenvectors.The physics-aware model’s eigenvectors have more structure than those of the comparison model.
- SST data: The physics-aware model provides better approximation quality for extrapolated climate temperature fields than the physics-agnostic model.The difference is especially distinct for the third example, with predictions evaluated five time steps into the future.