Source-linked AI summary
Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders
Romit Maulik, Bethany Lusch, Prasanna Balaprakash
TL;DR
The paper addresses the difficulty of reducing advection-dominated nonlinear PDEs with conventional POD-Galerkin methods. It combines convolutional autoencoders with LSTM latent-space evolution and parameter embedding, demonstrating stable reconstructions for advection-dominated physics and improved compression relative to POD-Galerkin formulations.
Problem
Reduced-order models struggle with nonlinear, multiscale PDE dynamics because POD-Galerkin truncation omits higher-basis nonlinear interactions, while high-fidelity simulations remain expensive.
Method
The framework separately trains a convolutional autoencoder for nonlinear compression and an LSTM for non-intrusive latent-space evolution, with control parameters concatenated into the latent representation.
Results
The CAE-LSTM framework produces stable reconstructions of high-dimensional advection-dominated physics in space and time and captures sharp-profile advection with only two degrees of freedom.
Takeaways & Limitations
Advection-dominated systems benefit from nonlinear spatial encoding and nonlinear temporal modeling through the CAE-LSTM combination.
Takeaways & Limitations
The POD-Galerkin formulation is computationally efficient, but its accuracy is limited by truncated higher-basis nonlinear interactions; the framework assumes the PDE can be solved on a discrete spacetime grid.
Abstract
from arXiv · showhide
A common strategy for the dimensionality reduction of nonlinear partial differential equations relies on the use of the proper orthogonal decomposition (POD) to identify a reduced subspace and the Galerkin projection for evolving dynamics in this reduced space. However, advection-dominated PDEs are represented poorly by this methodology since the process of truncation discards important interactions between higher-order modes during time evolution. In this study, we demonstrate that an encoding using convolutional autoencoders (CAEs) followed by a reduced-space time evolution by recurrent neural networks overcomes this limitation effectively. We demonstrate that a truncated system of only two latent-space dimensions can reproduce a sharp advecting shock profile for the viscous Burgers equation with very low viscosities, and a six-dimensional latent space can recreate the evolution of the inviscid shallow water equations. Additionally, the proposed framework is extended to a parametric reduced-order model by directly embedding parametric information into the latent space to detect trends in system evolution. Our results show that these advection-dominated systems are more amenable to low-dimensional encoding and time evolution by a CAE and recurrent neural network combination than the POD Galerkin technique.
1. Introduction
The paper develops a data-driven reduced-order modeling strategy for nonlinear, advection-dominated PDEs, combining nonlinear compression, recurrent latent-space evolution, and parameter embedding. It targets expensive simulations and limitations of conventional reduced bases while demonstrating the approach on Burgers and shallow-water equations.
- Motivation: High-fidelity nonlinear PDE simulations are computationally expensive, motivating reduced-order models for fast-query applications.The paper also identifies control, optimization, uncertainty quantification, and data assimilation as application areas.
- Motivation: Conventional ROMs can perform poorly for complex multiscale phenomena because reduced-basis truncation limits nonlinear dependence and higher-mode interactions.The paper specifically highlights missing higher-basis nonlinear interactions as an error source in POD-Galerkin evolution.
- Contributions: The proposed strategy uses convolutional autoencoders for nonlinear, highly compressed embeddings and LSTM networks to evolve states in latent space.The compression and temporal-evolution components are trained separately from snapshot data.
- Contributions: Parametric information is embedded directly into the latent space, enabling emulator generalization across control-parameter ranges such as viscosity.This design also permits independent training of the CAE and LSTM networks.
- Contributions: The framework is evaluated for non-intrusive modeling of advection-dominated viscous Burgers and inviscid shallow-water equations.These equations provide the paper’s demonstration settings for the proposed formulation.
2. Proper orthogonal decomposition
POD constructs a reduced basis from solution snapshots and projects the governing equations onto that basis for reduced-order evolution. Truncation improves efficiency but can omit higher-basis nonlinear interactions, limiting accuracy.
- Basis construction: POD identifies a reduced basis from snapshots to approximate the solution field in a lower-dimensional space.The basis is formed from snapshot data, with a practical reduced basis using the first Nr modes where Nr ≪ Ns.
- Reduced representation: The reduced basis coefficients capture the temporal effects of the system and reconstruct the solution through the POD approximation.
- Galerkin projection: Galerkin projection converts the coupled PDE system into reduced equations for the temporal coefficients, including a precomputed reduced-space Laplacian.The resulting formulation is called POD Galerkin projection (POD-GP).
- Reduced evolution: POD-GP solves Nr coupled ordinary differential equations with a fourth-order Runge–Kutta method, making forward solves efficient.
- Accuracy limitation: Truncation omits higher-basis nonlinear interactions, creating a significant source of forward-evolution error despite the reduced computational cost.The passage explicitly characterizes accuracy as limited even though reduced degrees of freedom make the solves efficient.
3. Deep neural networks
The proposed deep-learning strategy uses convolutional autoencoders for nonlinear compression and LSTM networks for equation-free latent-space time evolution. Separating embedding from forecasting provides flexible modeling of reduced dynamics.
- Autoencoders: Autoencoders learn lower-dimensional representations by encoding inputs into a latent vector and decoding that vector to reconstruct the original data.Training minimizes reconstruction error, and the representation is learned without supervision.
- Convolutional autoencoders: Convolutional autoencoders replace fully connected transformations with learned filters applied to local input patches.The paper describes one- and two-dimensional convolutions, including shifted patches and multi-channel inputs.
- LSTM evolution: LSTMs advance encoded states through time using gating mechanisms that control information flow and preserve information from past inputs and predictions.The internal state is updated using input and forget gates.
- Combined surrogate modeling: The framework separately trains a CAE for dimensionality reduction and an LSTM for latent-space temporal evolution.This separates snapshot embedding from data-driven time-series forecasting and avoids retraining the embedding when changing the state-evolution mechanism.
- Combined surrogate modeling: Compared with POD-GP, the formulation targets improved nonlinear compression and non-intrusive evolution of the reduced state.
4. Experiments
The experiments assess the framework on viscous Burgers and conservative inviscid shallow water equations. These problems test temporal shock advection and interpolation across initial-condition space, respectively.
- Representative problems: The viscous Burgers problem tests whether the framework accurately captures an advecting shock profile over time.
- Representative problems: The inviscid shallow water problem tests interpolation in initial-condition space using varying initial conditions.
4.1. Burgers
For viscous Burgers flow, a two-dimensional CAE latent space combined with LSTM time advancement reconstructs advecting shock profiles across Reynolds numbers, while POD-Galerkin struggles with convergence and instability.
- CAE-LSTM architecture: A one-dimensional convolutional autoencoder compresses the solution field to two latent degrees of freedom before recurrent time advancement.The CAE uses strided convolutional and pooling layers, while the LSTM advances latent representations autoregressively.
- CAE representations: Two latent variables consistently represent the Burgers dynamics, with latent-space magnitudes varying alongside the balance between advection and dissipation.Similar latent-space trends appear across the tested Reynolds numbers, suggesting a potentially universal compressed representation.
- POD-Galerkin comparison: POD-Galerkin converges slowly toward the shock profile and develops high-frequency instabilities as retained modes increase at higher Reynolds numbers.The instabilities arise from non-shock-capturing schemes and produce Gibbs oscillations near the advecting discontinuity.
- CAE-LSTM results: At Re = 4000, CAE-LSTM reconstructs sharp-profile advection using only two degrees of freedom, with LSTM outputs recursively fed back for subsequent predictions.The latent-space window is initialized with the first 10 true time steps, requiring an initial full-order simulation interval.
- Error comparison: Across Reynolds numbers, CAE-LSTM generally attains lower final-time L2 errors than POD-Galerkin at comparable two-dimensional compression.The authors report an order-of-magnitude lower error across different Re for the same two-degree-of-freedom latent space, while noting POD-Galerkin can match or outperform it at greater mode retentions and lower Re.
- Parametric extension: A parametric LSTM reproduces latent-space trends and full-order dynamics at viscosities absent from training, with averaged final-time reconstruction mean-squared error of 1.17e-4.The two-dimensional latent representation captures differences in shock sharpness across viscosities.
4.2. Shallow water equations
The shallow-water experiments use a two-dimensional CAE-LSTM to reconstruct and advance conserved-variable fields, including cases with unseen initial conditions and control parameters. At six latent dimensions, the model reproduces coherent spatial structures while reducing simulation cost relative to POD-GP.
- Parametric evolution: Unseen-parameter tests show that the CAE reconstructs shock profiles and the parametric LSTM reproduces latent-space dynamical trends.The tests use parameters excluded from training, including cases with different shock sharpness and a test parameter for both the CAE and parametric LSTM.
- CAE encoding: The two-dimensional CAE encodes the three conserved shallow-water variables as input and output channels, with data scaled to zero mean and unit variance.The decoder reconstructs fields from the latent space for a representative test simulation with an unseen initial condition.
- Parametric evolution: The CAE-LSTM conditions latent-space evolution on control parameters that determine the initial Gaussian pulse location.The parameterized LSTM is trained from CAE-compressed full-order data concatenated with parameter information.
- Computational cost: 1.746 seconds per simulation was required for CAE-LSTM latent-space forecasting, versus 24.67 seconds for POD-GP deployments using either 6 or 40 retained modes.Reconstructing a 100-snapshot simulation from latent space required 0.167 seconds; POD-GP’s main added cost was nonlinear-term reconstruction for flux calculations.
5. Discussion and Conclusions
The paper concludes that recurrent CAE models provide stable reduced-order representations for advective systems and can incorporate control parameters for parametric exploration. The approach achieves low computational cost, while unstructured-grid handling and LSTM initialization remain practical boundaries.
- Discussion and Conclusions: The recurrent CAE framework targets high-dimensional advective systems that exhibit slow convergence and instability under POD-Galerkin reduction.It combines nonlinear CAE embedding with equation-free LSTM dynamics to reconstruct physics in space and time.
- Discussion and Conclusions: Concatenating latent embeddings with control parameters enables interpolation between full-order sample points and supports rapid design-space exploration.Test datasets show recovery of physical trends for unseen control-parameter choices.
- Limitations and future work: The framework requires a burn-in window for the LSTM, with initial computation reported as less than 10% of full-order compute.The authors are exploring ghost points and data augmentation to reduce this initialization cost.
- Limitations and future work: Unstructured-grid information remains a challenge, particularly for problems with significant spatial anisotropy.The discussion identifies generalized moving least squares methods and point-cloud networks as promising directions.