Source-linked AI summary
POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition
Stefania Fresca, Andrea Manzoni
TL;DR
The paper addresses the high offline training cost of DL-ROMs, which remains dependent on the full-order dimension despite their efficient testing. It proposes POD-DL-ROM, combining prior randomized POD reduction with multi-fidelity pretraining, and evaluates it across several parametrized PDEs. The reported results show fast training and efficient approximations across scalar and vector, linear and nonlinear problems.
Problem
DL-ROMs avoid projection and linear-manifold limitations but their offline training can remain prohibitively costly as the full-order dimension Nh grows.
Method
POD-DL-ROM first reduces full-order snapshots with randomized POD, then uses multi-fidelity pretraining and DL-ROM learning on POD coordinates.
Results
Across linear and nonlinear scalar and vector parametrized PDEs, POD-DL-ROMs make ROM training extremely fast while providing efficient online approximations.
Takeaways & Limitations
POD-DL-ROM provides a turn-key snapshot-based strategy for approximating nonlinear time-dependent parametrized PDEs and can support more-than-real-time online simulation.
Takeaways & Limitations
DL-ROM offline training depends on Nh and may entail overwhelming training times and costs when Nh is moderately large.
Abstract
from arXiv · showhide
Deep learning-based reduced order models (DL-ROMs) have been recently proposed to overcome common limitations shared by conventional reduced order models (ROMs) - built, e.g., through proper orthogonal decomposition (POD) - when applied to nonlinear time-dependent parametrized partial differential equations (PDEs). These might be related to (i) the need to deal with projections onto high dimensional linear approximating trial manifolds, (ii) expensive hyper-reduction strategies, or (iii) the intrinsic difficulty to handle physical complexity with a linear superimposition of modes. All these aspects are avoided when employing DL-ROMs, which learn in a non-intrusive way both the nonlinear trial manifold and the reduced dynamics, by relying on deep (e.g., feedforward, convolutional, autoencoder) neural networks. Although extremely efficient at testing time, when evaluating the PDE solution for any new testing-parameter instance, DL-ROMs require an expensive training stage, because of the extremely large number of network parameters to be estimated. In this paper we propose a possible way to avoid an expensive training stage of DL-ROMs, by (i) performing a prior dimensionality reduction through POD, and (ii) relying on a multi-fidelity pretraining stage, where different physical models can be efficiently combined. The proposed POD-DL-ROM is tested on several (both scalar and vector, linear and nonlinear) time-dependent parametrized PDEs (such as, e.g., linear advection-diffusion-reaction, nonlinear diffusion-reaction, nonlinear elastodynamics, and Navier-Stokes equations) to show the generality of this approach and its remarkable computational savings.
1. Introduction
Conventional ROMs can struggle when linear trial spaces become large or poorly represent nonlinear solution manifolds. The paper proposes POD-DL-ROM, combining POD-based dimensionality reduction with multi-fidelity pretraining to accelerate DL-ROM training while retaining efficient testing.
- Full-order simulations are unaffordable for repeated or real-time approximation of parametrized PDEs.
- Conventional ROM efficiency breaks down when the linear trial subspace is too large or hyper-reduction of nonlinear terms becomes expensive.
- DL-ROMs learn both a low-dimensional nonlinear trial manifold and its reduced dynamics non-intrusively from full-order snapshots.
- DL-ROMs can outperform reduced-basis methods in accuracy and online efficiency on problems challenging for linear reduced spaces.
- POD-DL-ROM accelerates DL-ROM offline training through randomized POD dimensionality reduction and multi-fidelity pretraining.
- POD-DL-ROM combines deep learning's non-intrusive character with POD's simplicity and mathematical foundation.
- The resulting strategy reduces DL-ROM offline costs and improves online testing relative to POD-Galerkin ROMs, including cases where POD-Galerkin already performs adequately.
2. An overview of deep learning-based ROMs
The paper reviews projection-based ROMs and DL-ROMs for parametrized nonlinear PDEs, emphasizing their respective trial-manifold, dynamics, training, and testing trade-offs. POD-DL-ROM applies DL-ROM learning after randomized POD reduction to reduce dependence of training on the full-order dimension.
- A parametrized PDE is represented as a nonlinear dynamical system with solution snapshots collected across time and parameter values.
- The solution manifold can have intrinsic dimension at most nµ + 1, despite the full-order dimension Nh being much larger.
- POD-Galerkin ROMs approximate solutions in an n-dimensional linear subspace spanned by basis vectors and evolve reduced coordinates through a projected dynamical system.
- POD-Galerkin bottlenecks include subspaces much larger than the solution manifold's intrinsic dimension and expensive hyper-reduction for nonlinear operators.
- DL-ROMs replace projection-based trial manifolds and reduced dynamics with neural networks trained on full-order snapshots.
- Once trained, DL-ROMs can query any time and parameter instance without solving a dynamical system up to that time.
- DL-ROM training still depends on Nh and may incur overwhelming costs when the full-order dimension is moderately large.
3. A new deep learning-based reduced order model
POD-DL-ROM combines randomized POD data compression with DL-ROM learning to reduce training costs while retaining nonlinear-manifold modeling and efficient testing. Its architecture maps time and parameters to POD generalized coordinates, reconstructing the solution through the POD basis.
- POD-enhanced DL-ROMs: POD-DL-ROM performs two-step dimensionality reduction: randomized POD compresses FOM snapshots, then a DL-ROM maps (t, µ) to POD generalized coordinates.The POD stage serves data compression rather than defining the final reduced linear trial manifold.
- POD-enhanced DL-ROMs: The reduced linear dimension N can remain potentially large for data compression, while the nonlinear manifold dimension n is chosen much closer to the intrinsic dimension nµ + 1.This separates snapshot compression from the dimension of the learned nonlinear representation.
- POD-enhanced DL-ROMs: The network approximates POD intrinsic coordinates with a DFNN and models their nonlinear trial manifold using a convolutional autoencoder decoder.At testing time, the encoder can be discarded.
- POD-enhanced DL-ROMs: The reconstructed solution is recovered from the network output through the randomized POD basis matrix.The network output approximates the POD intrinsic coordinates computed from the FOM snapshots.
- Analysis: The framework admits a convergence proof when the POD-DL-ROM network is zero-extended and the input-output map is locally Lipschitz.The proof follows the approach presented in the cited prior work.
- Randomized POD: Randomized SVD constructs an approximate snapshot range, computes a smaller SVD, and recovers the rPOD basis matrix VN.The procedure uses a randomized range finder followed by SVD of a reduced matrix.
- Pretraining: Multi-fidelity pretraining initializes complex-task training from a model trained on a simpler task, and numerical results identify this combination with rPOD as key to reducing training time.The simpler task may differ in discretization, physical laws, parameter count, or parameter range.
- Vector problems: The approach extends DL-ROMs from scalar to vector problems and uses convolutional layers for high-dimensional spatial data.The vector extension treats the discretized components analogously to channels in RGB images.
4. Numerical results
The numerical assessment evaluates POD-DL-ROM across four parametrized PDEs, focusing on accuracy, training efficiency, testing efficiency, and pretraining. Results show accurate solution reconstruction, reduced computational costs, and sensitivity to training data and POD dimension.
- Experimental setup: The method is evaluated on four linear or nonlinear parametrized PDE problems, including advection-diffusion-reaction, cardiac electrophysiology, nonlinear elastodynamics, and incompressible Navier-Stokes.The experiments assess both training and testing computational times and the use of pretraining.
- Test 1: unsteady advection-diffusion-reaction equation: 32-fold speed-up is obtained with randomized SVD over exact SVD for N = 64 when constructing the POD space.The randomized SVD time increases with N, while exact SVD time is unaffected by N.
- Test 1: unsteady advection-diffusion-reaction equation: The error indicator decreases approximately as 1/Ntrain as the number of training-parameter instances increases.The reported trend uses Nepochs = 3723 for Ntrain = 500.
- Test 1: unsteady advection-diffusion-reaction equation: For N ≥ 64, the POD-DL-ROM error remains almost unchanged and nearly coincides with the error between the projected FOM solution and its approximation.At N = 16, the rPOD space is too small to accurately reconstruct the FOM solution.
- Test 2: coupled PDE-ODE Monodomain/Aliev-Panfilov system: Setting ωh = 0.5 halves the error compared with ωh = 1 in the reported optimization comparison.The weighting performs a second dimensionality reduction by matching the dimension of the problem's minimal components.
- Test 2: coupled PDE-ODE Monodomain/Aliev-Panfilov system: The POD-DL-ROM reduces training time by factors of 37.5 versus POD-Galerkin ROM and 5 versus DL-ROM, while remaining the most efficient ROM in both stages.Its testing time is also faster than that of DL-ROM because the neural network has fewer parameters.
- Test 3: nonlinear elastodynamics for hyperelastic compressible materials: The POD-DL-ROM captures elastodynamics displacement components accurately, with a maximum relative error of about 10^-3.The reported comparison uses n = 3 for the testing instance µtest = (2.88 Pa, 0.3987) at T = 15 s.
- Test 3: nonlinear elastodynamics for hyperelastic compressible materials: Pretraining strongly reduces total training and validation time, while testing 90 time instances takes 0.006 s for the reported elastodynamics case.The testing time is lower than the final time T = 22.5 s.
5. Conclusions
POD-DL-ROM combines randomized POD and multi-fidelity pretraining to accelerate DL-ROM training while retaining efficient online evaluation. Across several parametrized PDEs, it provides accurate, robust approximations and substantial computational savings.
- Contributions: POD-DL-ROM reduces FOM snapshots through randomized POD and initializes neural networks sequentially using lower-fidelity snapshots.The approach targets the DL-ROM training bottleneck associated with increasing FOM dimension.
- Evaluation: The method was assessed on linear and nonlinear, scalar and vector, time-dependent parametrized PDEs, including advection-diffusion-reaction, cardiac electrophysiology, elastodynamics, and fluid dynamics.The evaluation covers four distinct application classes.
- Results: POD-DL-ROMs matched the intrinsic dimension of the investigated problems while providing extremely efficient numerical approximations.The reported scope includes nonlinear time-dependent parametrized PDEs.
- Results: The technique enabled online simulations in more than real-time for physical phenomena occurring on a time scale of seconds.This conclusion concerns the online testing stage.