PaperlayerView exact revision on arXiv

Source-linked AI summary

A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs

Stefania Fresca, Luca Dede, Andrea Manzoni

arXiv:2001.04001v1math.NAcs.LG

TL;DR

Linear ROMs can require overly large spaces for parametrized time-dependent PDEs with propagating structures. The paper proposes DL-ROMs that learn nonlinear trial manifolds and reduced dynamics non-intrusively, and reports accurate low-dimensional approximations that outperform linear alternatives in the tested problems.

  • Problem

    Linear reduced basis ROMs can be inefficient for nonlinear time-dependent parametrized PDEs because global linear trial spaces may require dimensions much larger than the solution manifold’s intrinsic dimension.

  • Method

    DL-ROM learns both a nonlinear trial manifold and nonlinear reduced dynamics non-intrusively from FOM solutions using deep-learning models.

  • Results

    Across three test cases, DL-ROM provides sufficiently accurate solutions at dimension nµ + 1 and outperforms global and local POD-based linear ROMs.

  • Takeaways & Limitations

    DL-ROM can approximate parametrized PDE solution manifolds with very low-dimensional nonlinear representations where many POD modes would otherwise be needed.

  • Takeaways & Limitations

    The study focuses on numerical accuracy rather than computational efficiency, and comparison with linear ROMs is performed on one-dimensional test cases.

Abstract

from arXiv · show

Traditional reduced order modeling techniques such as the reduced basis (RB) method (relying, e.g., on proper orthogonal decomposition (POD)) suffer from severe limitations when dealing with nonlinear time-dependent parametrized PDEs, because of the fundamental assumption of linear superimposition of modes they are based on. For this reason, in the case of problems featuring coherent structures that propagate over time such as transport, wave, or convection-dominated phenomena, the RB method usually yields inefficient reduced order models (ROMs) if one aims at obtaining reduced order approximations sufficiently accurate compared to the high-fidelity, full order model (FOM) solution. To overcome these limitations, in this work, we propose a new nonlinear approach to set reduced order models by exploiting deep learning (DL) algorithms. In the resulting nonlinear ROM, which we refer to as DL-ROM, both the nonlinear trial manifold (corresponding to the set of basis functions in a linear ROM) as well as the nonlinear reduced dynamics (corresponding to the projection stage in a linear ROM) are learned in a non-intrusive way by relying on DL algorithms; the latter are trained on a set of FOM solutions obtained for different parameter values. In this paper, we show how to construct a DL-ROM for both linear and nonlinear time-dependent parametrized PDEs; moreover, we assess its accuracy on test cases featuring different parametrized PDE problems. Numerical results indicate that DL-ROMs whose dimension is equal to the intrinsic dimensionality of the PDE solutions manifold are able to approximate the solution of parametrized PDEs in situations where a huge number of POD modes would be necessary to achieve the same degree of accuracy.

1. Introduction

The paper motivates nonlinear, non-intrusive reduced order models for parametrized time-dependent PDEs because linear ROMs can be inefficient for propagating coherent structures. It proposes DL-ROMs that learn both the reduced trial manifold and reduced dynamics from FOM solutions.

  • Motivation: High-dimensional FOMs are computationally prohibitive for real-time and multi-query parametrized PDE scenarios.ROMs seek much lower-dimensional approximations that retain the physical features represented by the FOM.
  • Limitations of linear ROMs: Linear ROMs represent approximations through basis-function superpositions and obtain reduced dynamics by projection onto a linear test subspace.POD constructs such bases from snapshot matrices using singular value decomposition.
  • Limitations of linear ROMs: Propagating coherent structures make linear ROMs computationally inefficient when accurate approximations are required.The paper discusses transport, wave-type, and related phenomena as motivating cases.
  • Proposed approach: DL-ROMs learn a nonlinear trial manifold and nonlinear reduced dynamics non-intrusively using deep learning.The framework approximates the map from time and parameters to the high-dimensional PDE solution and uses FOM solutions for training.
  • Proposed approach: The approach combines a convolutional-autoencoder decoder for the trial manifold with neural networks for reduced dynamics, trained simultaneously through a weighted loss.This design targets parameters affecting both PDE operators and data while avoiding the projection stage and a more expensive LSTM.
  • Evaluation: The paper evaluates DL-ROM accuracy on three parametrized PDE test cases of increasing complexity.The section structure identifies numerical assessment in section 4 and conclusions in section 5.

2. From linear to nonlinear dimensionality reduction

The paper formulates parametrized PDE solutions as a low-dimensional solution manifold and contrasts linear projection-based ROMs with nonlinear dimensionality reduction. Its DL-ROM replaces the linear trial space and projected dynamics with learned nonlinear maps whose dimension can approach the manifold’s intrinsic dimension.

  • 2.1. Problem formulation: After spatial discretization, a parametrized time-dependent PDE becomes a nonlinear dynamical system with FOM state uh(t; µ) and parameter-dependent dynamics f.The parameter may encode physical or geometrical properties, initial or boundary conditions, or domain shape.
  • 2.1. Problem formulation: The solution set Sh over time and parameters forms a solution manifold in R^Nh with intrinsic dimension at most nµ + 1.Time contributes one coordinate alongside the nµ parameters, under the stated unique-solution assumption.
  • 2.2. Linear dimensionality reduction: Classical linear ROMs approximate uh using a global subspace spanned by n basis vectors, with generalized coordinates evolving through a projected reduced system.Galerkin and Petrov-Galerkin formulations impose orthogonality of the FOM residual to a low-dimensional test space.
  • 2.2. Linear dimensionality reduction: POD builds the linear trial space from FOM snapshots by applying an Xh-weighted singular value decomposition and selecting leading singular vectors.The resulting basis is orthonormal in the Xh inner product and provides the best snapshot reconstruction among n-dimensional subspaces.
  • 2.2. Linear dimensionality reduction: RB methods often require n ≫ nµ + 1 because a global linear trial manifold cannot efficiently represent hyperbolic or propagating solutions.Hyper-reduction may also be needed to assemble reduced operators without expensive Nh-dimensional arrays.
  • 2.3. Nonlinear dimensionality reduction: A nonlinear ROM replaces the linear reconstruction with a differentiable nonlinear map Ψh from n intrinsic coordinates to the Nh-dimensional state.The nonlinear trial manifold consists of Ψh(un(t; µ)) over time and parameter values.
  • 2.3. Nonlinear dimensionality reduction: The proposed nonlinear ROM seeks n close to the solution manifold’s intrinsic dimension and models reduced coordinates through a nonlinear map Φn(t, µ).The formulation requires n ≥ nµ + 1 and does not require affine parameter dependence.

3. A deep learning-based reduced order model (DL-ROM)

DL-ROM learns both a nonlinear reduced trial manifold and the reduced dynamics from FOM snapshots using deep neural networks. Training uses an encoder-augmented architecture, while testing evaluates only the decoder and dynamics networks for new time-parameter inputs.

  • DL-ROM construction: The approach is non-intrusive and data-driven, requiring the dataset, parameter values, and snapshot matrix rather than the FOM arrays.This construction relies on neural networks’ ability to approximate nonlinear maps and generalize to unseen data.
  • DL-ROM construction: DL-ROM uses two blocks to learn the reduced dynamics and the nonlinear reduced trial manifold, with intrinsic dimension n much smaller than the FOM dimension Nh.The method targets nonlinear ROMs whose dimension is nearly equal to the intrinsic dimensionality of the solution manifold.
  • Reduced dynamics learning: A deep feedforward neural network models the reduced dynamics as a function of time and parameters.The network defines the map from (t, µ) to intrinsic coordinates through repeated nonlinear transformations.
  • Reduced trial manifold learning: A convolutional-autoencoder decoder maps intrinsic coordinates into the high-dimensional ROM approximation and defines the nonlinear trial manifold.The decoder uses several layers, including convolutional layers, to produce an output in R^Nh.
  • Training and testing: Training optimizes network parameters using reconstruction and encoder-coordinate errors, with backpropagation and ADAM applied to FOM snapshots.The training architecture includes the encoder, whereas the testing architecture discards it and performs a forward pass.
  • Training and testing: Inputs and outputs are affinely normalized to [0, 1], using extrema computed on the training set for validation and testing data.FOM solutions are reshaped into matrices for the convolutional encoder and zero-padded when Nh is not square.

4. Numerical results

The paper evaluates DL-ROMs on Burgers, linear transport, and monodomain problems, emphasizing accuracy against POD-based linear ROMs. Across tests, low-dimensional DL-ROM manifolds often match accuracies requiring many POD basis functions, though excessive dimension can cause overfitting.

  • Test cases: The evaluation covers Burgers, linear transport, and nonlinear monodomain problems with wave or traveling-wave behavior.These tests increase in parametric and PDE complexity.
  • Test 1: Burgers Equation: At µtest = 976.32, DL-ROM halves the optimal-POD error indicator and avoids oscillations near large gradients.Its reconstruction mostly fits the FOM solution, including near its maximum.
  • Test 1: Burgers Equation: For Burgers’ equation, DL-ROM is more accurate than POD for n < 20, while both methods have almost the same accuracy at n = 20.The comparison uses the testing-set error indicator ϵrel.
  • Test 2: Linear Transport Equation: For another transport instance, n = 2 gives ϵrel = 8.74 · 10^-3, while a POD manifold with n = 50 still produces spurious oscillations.The optimal-POD reconstruction also cannot fit the discontinuity sharply.
  • Dimension and training effects: Increasing DL-ROM dimension improves accuracy only slightly in one test, while n = 40 causes overfitting and increases ϵrel in another.The latter occurs when model complexity exceeds the available training data.

5. Conclusions

The paper proposes DL-ROMs that learn both nonlinear trial manifolds and reduced dynamics non-intrusively, targeting parametrized PDEs with propagating coherent structures. Across three test cases, DL-ROMs achieve accurate low-dimensional approximations and outperform linear and local-POD ROMs, while computational speedup remains future work.

  • DL-ROMs learn both a low-dimensional nonlinear trial manifold and the reduced dynamics of parametrized nonlinear, time-dependent PDEs.The manifold uses a convolutional autoencoder decoder, while reduced dynamics use a feedforward neural network and the autoencoder encoder.
  • Three test cases show accurate DL-ROM approximations when the solution manifold has dimension nµ + 1.The tests cover problems with different levels of parametric dependence and PDE complexity.
  • At equal dimension, DL-ROM approximations are orders of magnitude more accurate than linear ROM approximations.Accuracy becomes comparable around O(10^2) POD basis functions, a dimension described as infeasible for more involved higher-dimensional parametrized problems.
  • Computational speedup against linear ROMs is deferred to future publications, especially for two- and three-dimensional problems.The current one-dimensional tests focus on numerical accuracy rather than computational efficiency.

Appendix A. Basic concepts of deep learning

Deep learning techniques are introduced as models that discover patterns and extract features from massive datasets to make predictions without hand-crafted features.

  • Deep learning models discover patterns and extract features from massive datasets to make predictions without hand-crafted features.The paper introduces this capability as background for the deep-learning models used by DL-ROM.

Appendix A.1. Deep feedforward neural network

A deep feedforward neural network maps inputs to outputs through compositions of nonlinear layers whose parameters are learned by minimizing a loss function. Training uses supervised input-output observations and gradient-based optimization with back-propagation, with stochastic gradient descent reducing computation for large datasets.

  • A DFNN models the relationship between input values and output values through a composition of simpler nonlinear functions or layers.The layered composition learns complex hierarchies of features.
  • Each DFNN layer applies a linear transformation followed by an activation function, parameterized by weights and biases.The full parameter vector collects the weights and biases across all layers.
  • Supervised DFNN learning finds parameters from input-output observations by minimizing a loss measuring mismatch between observed and approximated outputs.The loss is defined through per-example discrepancies over the training observations.
  • Gradient descent with back-propagation computes derivatives of the loss with respect to network parameters.Back-propagation supplies the derivatives needed for parameter updates.
  • Stochastic gradient descent reduces gradient-computation cost by approximating the full-dataset expectation with a small sample.This is useful when the training dataset is very large and a full gradient step is expensive.

Appendix A.2. Convolutional neural network

Convolutional neural networks process high-dimensional spatial data through local connectivity and shared filters, extracting spatial features efficiently. These properties provide memory and computational advantages over DFNNs for extremely high-dimensional data.

  • CNNs are suited to high-dimensional spatially distributed data because convolutional layers exploit spatial correlations among nearby input elements.Their local approach supports extraction of low-level features.
  • A convolutional layer uses convolutional kernels or filters to compute feature maps from its input.The layer contains a bank of filters, with each filter producing a feature map.
  • CNNs use shared parameters and local connectivity, making their outputs translation equivariant.Weights are shared across a feature map, while each neuron connects only to a local input region.
  • Shared parameters and local connectivity improve CNN memory and computational efficiency relative to DFNNs for extremely high-dimensional data.These properties make CNNs preferable to DFNNs in that setting.

Appendix A.3. Autoencoder neural network

An undercomplete autoencoder learns a low-dimensional code and reconstructs high-dimensional inputs through encoder and decoder functions. For large discretized inputs, convolutional autoencoders reduce parameters and computation through parameter sharing and local connectivity.

  • Autoencoders learn an approximation of the identity function by reconstructing the input at the output.This places autoencoder learning within unsupervised learning.
  • Undercomplete autoencoders constrain the hidden code dimension to be smaller than the input and output dimensions.
  • The autoencoder consists of encoder and decoder parts that transform the high-dimensional input into a code and reconstruct it as output.Figure A.24 depicts the autoencoder neural network.
  • The encoder maps a high-dimensional input xh ∈ R^Nh to a low-dimensional code x̃n ∈ R^n, with n ≪ Nh.The encoder is parameterized by weights and biases collected in θE.
  • The decoder maps the low-dimensional code x̃n back to an approximation x̃h ∈ R^Nh of the original input.Its weights and biases are collected in θD.
  • For large discretized inputs, convolutional autoencoders reduce parameters and computations through parameter sharing and local connectivity.These reductions apply to both forward and backward passes.
Loading 2001.04001v1…