PaperlayerView exact revision on arXiv

Source-linked AI summary

Operator inference for non-intrusive model reduction with quadratic manifolds

Rudy Geelen, Stephen Wright, Karen Willcox

arXiv:2205.02304v4math.NAcs.NA

TL;DR

Linear-subspace reduction can be limited for problems without a fast-decaying Kolmogorov n-width. This paper learns a quadratic manifold from data and combines it with operator inference; the resulting models are reported effective for problems with slow n-width decay.

  • Problem

    Linear subspace approximation is limited for problems that do not have a fast-decaying Kolmogorov n-width.

    Source: §1 · p. 1

  • Method

    The approach learns a data-driven quadratic mapping from projection residuals and combines the manifold with operator inference to infer reduced-model operators from simulation data.

    Source: §1 · p. 2

  • Results

    The combined quadratic-manifold and operator-inference approach yields effective reduced-order models for problems exhibiting slow decay in the Kolmogorov n-width.

    Source: §5 · p. 22

  • Takeaways & Limitations

    The quadratic component provides approximation directions orthogonal to the linear subspace without increasing the low-dimensional representation’s degrees of freedom.

    Source: §5 · p. 22

    As reduced dimension grows, full quadratic models are likely to cost more to solve than linear models achieving the same accuracy.

    Source: §3.1 · p. 11

Abstract

from arXiv · show

This paper proposes a novel approach for learning a data-driven quadratic manifold from high-dimensional data, then employing this quadratic manifold to derive efficient physics-based reduced-order models. The key ingredient of the approach is a polynomial mapping between high-dimensional states and a low-dimensional embedding. This mapping consists of two parts: a representation in a linear subspace (computed in this work using the proper orthogonal decomposition) and a quadratic component. The approach can be viewed as a form of data-driven closure modeling, since the quadratic component introduces directions into the approximation that lie in the orthogonal complement of the linear subspace, but without introducing any additional degrees of freedom to the low-dimensional representation. Combining the quadratic manifold approximation with the operator inference method for projection-based model reduction leads to a scalable non-intrusive approach for learning reduced-order models of dynamical systems. Applying the new approach to transport-dominated systems of partial differential equations illustrates the gains in efficiency that can be achieved over approximation in a linear subspace.

1 Introduction

Linear subspaces can be inadequate for problems without a fast-decaying Kolmogorov n-width [1]. The paper learns a quadratic manifold from data and combines it with operator inference to build non-intrusive reduced-order models for parametrized PDEs.

1 Introduction

  • Unlike the kernel-PCA approach in, this method explicitly derives a data-driven basis for representing the manifold’s quadratic components.The paper is also inspired by structural-dynamics quadratic manifolds built from vibration modes and modal derivatives.Source: §1 · pp. 1–2
  • Transport-dominated problems can resist linear-subspace approximation because their singular values decay slowly, while some existing basis transformations require problem-specific knowledge of advection.Nonlinear reduction approaches include adaptive methods, multiple local subspaces, and neural-network manifolds; scalability and interpretability questions persist for the latter.Source: §1 · p. 2
  • The framework infers a quadratic mapping by least squares from projection residuals, then combines the manifold with operator inference on time-domain simulation data.It uses physics-based data from off-the-shelf solvers without access to their code, alongside an explicit high-fidelity problem definition.Source: §1 · p. 2
  • The quadratic component acts as a data-driven closure for modal-truncation error, accounting for discarded modes without requiring additional phenomenological arguments.The closure term is computed from observational data in a physics-agnostic fashion.Source: §1 · p. 2

2 Data-driven learning of nonlinear manifolds for dimensionality reduction

The section builds nonlinear solution manifolds by fitting quadratic directions to errors left by a POD linear approximation. It also describes selecting a subset of quadratic terms and illustrates the resulting reconstructions on a 3D trajectory.

2.1 The linear dimensionality reduction framework

  • POD uses leading left singular vectors of reference-shifted snapshots to form a low-dimensional linear subspace, but acceptable reconstruction error may require a large basis.The POD basis minimizes least-squares snapshot reconstruction error; the residual is the part not captured by that subspace.Source: §2.1 · p. 3

2.2 Data-driven quadratic solution-manifolds

  • Equation (10) is mathematically identical to an independent concurrent formulation [45], while this paper differs in its analysis, optimization treatment, and reduced-model deployment.Source: §2.2 · p. 5
  • The quadratic mapping fits least-squares directions to the residual left by POD, with the quadratic basis constrained to the linear basis's orthogonal complement.The proposed approximate minimizer first computes the POD representation, then fits the quadratic component to the remaining error.Source: §2.2 · pp. 5–6

2.3 Data-driven learning of quasi-quadratic manifolds

  • Group-sparse regularization selects quadratic basis columns, yielding a quasi-quadratic map with q selected terms instead of r(r + 1)/2 possible terms.The reduced fitting problem can be decomposed into n independent least-squares problems, each involving q coefficients.Source: §2.3 · pp. 6–7

2.4 Constructing the quadratic manifold — An illustrative 3D trajectory

  • On the 3D trajectory, both nonlinear manifolds curve the subspace without adding reduced coordinates; the quasi-quadratic example uses one additional basis vector.The selected direction curves the manifold along a single axis, while the full quadratic formulation approximates the trajectory accurately.Source: §2.4 · pp. 8–9
  • The linear subspace gives the largest trajectory error, while the full quadratic manifold gives the lowest; selected terms can match the full quadratic result.The cross-term basis vector is zero, so retaining the first and last quadratic terms suffices in this example.Source: §2.4 · p. 9, Table 1 · p. 9

3 Nonlinear manifolds for non-intrusive model reduction

The section develops quadratic-manifold reduced models and a non-intrusive operator-inference procedure for learning them from simulation data. The quadratic state approximation adds directions without increasing the reduced-state dimension, while its computational cost and initial-condition fit depend on the formulation and setting.

3.1 Projection-based model reduction with nonlinear manifolds

  • The quadratic manifold adds directions to the state approximation while keeping the reduced model dimension at r.The additional quadratic terms enrich the state reconstruction without adding reduced coordinates.Source: §3.1 · p. 10
  • For a linear full-order system, the quadratic manifold yields a quadratic reduced-state model that implicitly represents unresolved dynamics.The quadratic term acts as a data-driven closure term introducing components from the orthogonal complement of the POD basis.Source: §3.1 · p. 10
  • The quasi-quadratic formulation uses fewer quadratic coefficients and can make the model more data-efficient while retaining reduced-state dimension r.It replaces the full quadratic operator with a lower-dimensional representation.Source: §3.1 · p. 11
  • Full quadratic models are likely cost-competitive only for small reduced dimensions; at larger dimensions their solve cost may exceed a linear model of equal accuracy.The full formulation has worst-case quadratic-term evaluation cost scaling as r_q^3; the comparison depends on the achieved compression.Source: §3.1 · p. 11
  • Exact initial-condition satisfaction requires choosing the reference state equal to the initial condition, which cannot generally hold for trajectories with multiple initial conditions.With multiple initial conditions, one must choose one as the reference state, so exact satisfaction is not guaranteed for all trajectories.Source: §3.1 · p. 11

3.2 Learning quadratic manifold reduced models with operator inference

  • Operator inference learns the quadratic reduced model from simulation data without requiring access to full-order operators or model code.The approach uses state snapshots and knowledge of the full-order model structure.Source: §3.2 · pp. 11–12
  • Operator inference fits reduced operators to projected snapshots, with regularization to inhibit overfitting and symmetry to avoid redundant quadratic coefficients.Time derivatives must be estimated, and the quadratic model has more unknown operator coefficients than the linear model.Source: §3.2 · p. 12
  • For a quadratic full-order model, the quadratic-manifold reduced model has quartic structure, whose O(r^4) operator coefficients may make inference underdetermined or ill-conditioned.This risk may remain even with column sub-selection.Source: §3.2 · p. 12

4 Demonstration of the approach

The demonstrations apply quadratic-manifold operator inference to transport-dominated advection and wave equations. The workflow builds POD and quadratic bases from snapshots, then infers reduced operators from projected data.

4 Demonstration of the approach

  • The workflow centers snapshots about a reference state, computes a POD basis from their SVD, represents states and time derivatives in POD coordinates, forms quadratic data, and obtains reduced operators through regularized least squares.The examples are the advection and wave equations.Source: §4 · pp. 12–13

4.1 Linear transport equation

  • At a fixed reduced dimension, the quadratic manifold captures more snapshot energy than the linear POD subspace.The advection example uses γ = 10^9 for the quadratic mapping; Figure 4 compares retained-energy spectra.Source: Figure 4 · p. 14, §4.1 · p. 17
  • For advection at r = 29, the quadratic model’s reconstructed solutions have less pronounced spurious oscillations, and its r = 29 trajectory is visually close to the exact solution.The models capture 80.9% of snapshot energy linearly and 98.1% quadratically; the space-time comparisons also include r = 15.Source: Figure 5 · p. 15, Figure 6 · p. 16, Figure 7 · p. 16
  • Reducing the number q of quadratic basis columns shrinks the quadratic operator and storage needs, but has little effect on simulation cost; small q yields only marginal gains over the linear model.At q = 351, some test instances were unstable and excluded from the reported error measure.Source: Figure 8 · p. 17, §4.1 · p. 18
  • A quadratic manifold yields more accurate advection predictions across reduced dimensions, with faster error decay; at r = 30, its accuracy roughly matches the linear model at r = 60.For r > 35, the quadratic error curve flattens because of conditioning in the operator-inference regression; quasi-quadratic errors lie between linear and fully quadratic results.Source: §4.1 · pp. 18–19, Figure 9 · p. 18

4.2 Two-dimensional wave equation

  • The quadratic wave model more faithfully reproduces wave amplitudes and suppresses nonphysical behavior; its absolute errors are also much smaller than the linear model’s.Both models use r = 40, but capture 60.1% (linear) and 98.1% (quadratic) of snapshot energy.Source: Figure 11 · p. 20, Figure 12 · p. 21, Figure 13 · p. 21

5 Conclusions and future work

The paper combines data-driven quadratic manifolds with operator inference to build non-intrusive reduced-order models, with quadratic terms adding approximation directions without increasing the reduced representation's degrees of freedom. It also introduces column selection to trade some approximation accuracy for lower inference complexity, while identifying computational challenges for nonlinear problems.

5 Conclusions and future work

  • Quadratic manifolds add directions orthogonal to the linear subspace without increasing the reduced representation's degrees of freedom, and operator inference yields non-intrusive reduced-order models from snapshots.For the linear PDE examples, the resulting reduced-order models have quadratic structure.Source: §5 · p. 22
  • Column selection reduces the quadratic basis size and inferred operator dimension, trading some model performance for lower inference complexity.The paper describes this quasi-quadratic representation as incurring an error penalty in the examples considered.Source: §5 · p. 22
  • Nonlinear extensions also face challenges in representing nonlinear terms and constructing a well-conditioned least-squares problem as reduced-space dimension grows.The paper identifies hyperreduction, polynomial lifting, explicit treatment of non-polynomial terms, and alternative regularization as directions for further work.Source: §5 · pp. 22–23
  • For nonlinear PDEs, a quadratic manifold can yield quartic reduced operators, making reduction of inferred operator coefficients essential.The proposed column selection identifies multi-way interactions between reduced state variables that can be neglected.Source: §5 · p. 23

A The SpaRSA algorithm

The section presents SpaRSA as a proximal-gradient algorithm for the sum-of-ℓ2 regularized problem. It also describes solving across regularization parameters and selecting a solution using an external criterion.

A The SpaRSA algorithm

  • SpaRSA solves the sum-of-ℓ2 regularized problem (20) using a proximal-gradient step with a positive step length.The step moves from the current estimate V to a new estimate V+ by solving a subproblem.Source: A The SpaRSA algorithm · pp. 23–24
  • The objective is separated by rows, and the resulting subproblem has a closed-form solution.The section introduces row-wise notation for the residual before stating the solution.Source: A The SpaRSA algorithm · p. 23
  • A decreasing sequence of regularization parameters can reuse each solution as the next starting point, with an external criterion selecting among solutions.One possible criterion is targeting a specified number q of nonzero columns in V.Source: A The SpaRSA algorithm · p. 23
Loading 2205.02304v4…