PaperlayerView exact revision on arXiv

Source-linked AI summary

Nonlinear model reduction for transport-dominated problems

Jan S. Hesthaven, Benjamin Peherstorfer, Benjamin Unger

arXiv:2602.01397v1math.NAcs.LGmath.OC

TL;DR

Transport-dominated and wave-like problems expose fundamental limitations of linear reduced-space approximations. This survey organizes nonlinear model reduction around nonlinear parametrizations, reduced dynamics, and online solvers, and classifies methods by their structural approach. It concludes that many methods can be viewed as generic nonlinear parametrizations combined with instantaneous residual minimization, while transformation-based and online adaptive methods expose additional structure.

  • Problem

    Transport-dominated problems with moving coherent structures can limit linear model reduction because their solution manifolds may have slowly decaying n-widths.

  • Method

    The survey organizes nonlinear model reduction around nonlinear parametrizations, reduced dynamics, and online solvers, and distinguishes transformation-based, online adaptive, and generic residual-minimization approaches.

  • Results

    Many nonlinear model reduction methods can be viewed as generic nonlinear parameterizations combined with instantaneous residual minimization, while transformation-based and online adaptive methods provide additional structural features.

  • Takeaways & Limitations

    Nonlinear model reduction provides a framework for addressing transport-dominated problems by allowing more expressive approximations than linear reduced spaces.

  • Takeaways & Limitations

    Transformation-based approaches may struggle to capture multiple simultaneously transported structures without extensions beyond a single global group action.

Abstract

from arXiv · show

This article surveys nonlinear model reduction methods that remain effective in regimes where linear reduced-space approximations are intrinsically inefficient, such as transport-dominated problems with wave-like phenomena and moving coherent structures, which are commonly associated with the Kolmogorov barrier. The article organizes nonlinear model reduction techniques around three key elements -- nonlinear parametrizations, reduced dynamics, and online solvers -- and categorizes existing approaches into transformation-based methods, online adaptive techniques, and formulations that combine generic nonlinear parametrizations with instantaneous residual minimization.

1. Introduction

Model reduction targets the high cost of repeated simulations in outer-loop applications, but linear reduced spaces can be fundamentally inadequate for transport-dominated and wave-like problems. This motivates nonlinear model reduction, which replaces linear approximation assumptions with nonlinear approximations and requires revised reduced-model formulations, analysis, and solvers.

  • Motivation: Repeated simulations across parameters, inputs, and initial conditions make outer-loop applications costly, motivating reduced models.Applications include design, control, optimization, uncertainty quantification, and inverse problems.
  • Scope: Projection-based model reduction represents reduced solutions with low-dimensional states evolving under reduced dynamics, using full-model information.The survey focuses on projection-based reduced models, including constructions that may not be classical vector-space projections for nonlinear parametrizations.
  • Linear model reduction: Linear model reduction uses one global low-dimensional reduced space to approximate the solution manifold with linear combinations of basis functions.The survey identifies balanced truncation, proper orthogonal decomposition, interpolation methods, and reduced basis methods as examples.
  • Linear model reduction: Transport-dominated problems with moving coherent structures can produce slowly decaying n-widths, fundamentally limiting linear model reduction.The limitation has also been observed empirically across advection-dominated and chemically reacting flows, wildfire dynamics, pattern formation, solar weather, detonation, plasma, and rotating-wave problems.
  • Nonlinear model reduction: Nonlinear model reduction permits reduced solutions to depend nonlinearly on their degrees of freedom, requiring revised formulations, analyses, and numerical solution methods.Standard Galerkin constructions and stability arguments generally cannot be transferred verbatim because a global linear projection and associated orthogonality principle may not exist.

2. Linear model reduction

Linear model reduction constructs a fixed low-dimensional subspace from full-model information and evolves reduced coordinates, but transport-dominated dynamics can make such spaces inefficient. The section reviews the offline and online workflow, lifting bottleneck, empirical interpolation, and numerical evidence for this limitation.

  • Reduced-space formulation: Linear model reduction approximates full-model solutions in a fixed subspace V of dimension n≪N across time and parameters.The reduced solution uses fixed basis functions while its coefficients vary with time and parameter.
  • Offline workflow: The offline phase collects full-model training data, trains a reduced space to approximate the solution manifold, and constructs reduced dynamics.The workflow separates information collection, reduced-space training, and reduced-model construction.
  • Online evaluation: Online evaluation can scale with the full dimension N because reduced states are lifted, the full right-hand side is evaluated, and the result is projected.Empirical interpolation can instead evaluate only n component functions when component evaluations are independent, circumventing this bottleneck.
  • Diffusion example: For the heat equation, the first n=20 singular vectors approximate the snapshot data up to double precision in relative Frobenius norm, with N=1024.The singular-value decay suggests that the diffusion snapshots are well represented by a low-dimensional space, while not guaranteeing full-solution accuracy beyond the snapshots.
  • Transport example: For the transport problem, n≥150 is needed for double-precision snapshot representation, because translated copies of a localized feature are inefficiently represented by a linear space.The solution manifold can be one-dimensional while remaining poorly approximated by a low-dimensional vector space.
  • Transport example: Linear model reduction can therefore require substantially larger reduced spaces for transport-dominated than diffusion-dominated problems at comparable accuracy.The numerical illustration measures efficiency by the reduced-space dimension required to reach a given accuracy.

3. Kolmogorov barrier

The Kolmogorov n-width measures the smallest worst-case error achievable by n-dimensional linear reduced spaces. For transport-dominated problems, wave-like and moving localized features can force slow decay, limiting linear model reduction.

  • The Kolmogorov n-width is the smallest worst-case approximation error achievable using an n-dimensional reduced space.Its decay rate indicates whether a low-dimensional linear space can meet a prescribed error tolerance.
  • Fast n-width decay makes linear model reduction effective, as in diffusion-dominated problems where decay can be exponential in n.Increasing the reduced-space dimension can then rapidly improve approximation accuracy.
  • 3.3. Slow n-width decay for linear advection problem: For the one-dimensional linear advection solution manifold, the Kolmogorov n-width is lower bounded by c/√n.This prevents any sequence of linear subspaces from achieving faster worst-case error decay for that example.
  • 3.3. Slow n-width decay for linear advection problem: The linear advection bound is substantially slower than the exponential decay available for the one-parameter diffusion-dominated problem.Smoother initial conditions can improve the n-width decay rate, and analogous results apply to wave equations.
  • Transport-dominated problems with wave-like phenomena and moving coherent features exhibit slowly decaying n-widths that fundamentally limit linear model reduction.The limitation arises when transported structures are difficult to approximate with a single global linear space.
  • 3.4. Kolmogorov n-width and linear control systems: For stable finite-dimensional linear control systems, Kolmogorov n-widths coincide exactly with Hankel singular values.This identity is computable, although constructing an explicit reduced model that realizes the approximation quality remains nontrivial.

4. The elements of nonlinear model reduction

Nonlinear model reduction replaces fixed linear reduced spaces with nonlinear parametrizations that can represent transport and other moving features more efficiently. Its effectiveness depends on coordinating the parametrization, reduced dynamics, and online numerical solvers.

  • Nonlinear parametrizations: Nonlinear parametrizations seek to overcome the Kolmogorov barrier by making reduced approximations nonlinear in their weight vectors.
  • Nonlinear parametrizations: Unlike linear parametrizations, nonlinear parametrizations induce trial sets or manifolds that need not have vector-space structure and may represent solutions with fewer weights.
  • Example: nonlinear parametrizations and linear advection: In the linear advection example, translation requires moving features across basis functions, potentially requiring prohibitively many basis functions for smooth motion.
  • Example: nonlinear parametrizations and linear advection: The translated step-function solution manifold is represented exactly by a nonlinear parametrization with only n=2 weights, whereas the linear-parametrization L2 error cannot decay faster than 1/√n.
  • Three elements: Nonlinear model reduction comprises three essential elements: nonlinear parametrizations, reduced dynamics for online weights, and numerical solvers for efficient online evaluation.
  • Three elements: A posteriori error estimation is mature for linear model reduction but remains in its early stages for nonlinear model reduction.

5. Transformation-based methods

Transformation-based methods seek coordinates or operators that make nonlinear solution features amenable to linear reduction. They separate effects such as transport, rotation, or deformation while retaining physically interpretable reduced models.

  • Transformation-based methods identify coordinates in which the solution manifold becomes suitable for linear reduction.
  • Transformations can act directly on full-model solutions or on reduced basis functions, producing distinct reduced models in general.
  • Combining linear reduction with transformations decomposes solutions into features whose evolution is described by application-specific transformations.
  • Symmetry-based methods separate dynamics associated with group actions from the remaining solution-field evolution, which is then linearly reduced.
  • Single global group actions struggle with multiple structures moving at different speeds, whereas shifted POD assigns separate shifts to low-rank components inferred from snapshots.
  • Transformation choices can align nonsmooth parameter-dependent features or advect basis functions, but general nonlinear ansätze may make online evaluation scale with full-model dimension.
  • For non-periodic boundaries or higher-dimensional wave propagation, designing suitable transformation operators becomes more difficult than using a simple periodic shift.

6. Online adaptive model reduction

Online adaptive model reduction adapts the reduced space during simulation because transport-dominated solutions may be locally low-dimensional in time even when one global reduced space is inefficient.

  • Online adaptive model reduction adapts the reduced space over time to exploit local-in-time low-dimensional structure in transport-dominated solutions.

6.1. Model reduction with online basis updates

Online basis-update methods evolve time-dependent reduced spaces so they can track moving structures and transport without requiring a high-dimensional online weight vector. This flexibility weakens offline–online separation and can introduce full-model-scale update costs.

  • Online adaptive methods represent solutions with basis functions that evolve over time, causing the reduced space itself to evolve.
  • Time-dependent bases can be generated through nonlinear parametrizations whose online weights evolve features and the basis during simulation.
  • Local-in-time approximation enables low-dimensional spaces to track moving coherent structures and transport without a high-dimensional online weight vector.
  • Online adaptation weakens offline–online decomposition because basis updates may cost proportional to the full-model dimension, although it can capture dynamics absent from training data.
  • This survey distinguishes adaptation over time from reduced-space construction across outer-loop iterations, which targets incomplete parameter-domain sampling instead.

6.2. Overview of online adaptive model reduction and time-dependent basis methods

Time-dependent basis methods include localized spaces, dynamic low-rank representations, transport-informed updates, and data-driven basis adaptation. They trade improved adaptability for challenges including conditioning, structure preservation, and update costs.

  • Localized methods select or weight multiple low-dimensional spaces online so each space represents a local regime rather than one globally accurate space.
  • Local regimes can be defined by parameter-domain regions or time windows, while dictionary methods select a few basis functions from a larger offline set.
  • Dynamic low-rank approximation evolves low-rank matrix factors by projecting full dynamics onto the tangent space of a fixed-rank matrix manifold.
  • Small singular values can poorly condition dynamic low-rank reduced dynamics, imposing step-size restrictions that motivate specialized integrators.
  • Structure-preserving extensions target Hamiltonian wave equations and conservation laws in kinetic problems, while related methods extend to tensors and stochastic PDEs.
  • The survey provides only a brief overview of dynamic low-rank methods and points to a dedicated survey for further detail.
  • Transport-informed approaches evolve bases using optimal transport, approximate Lax pairs, or characteristics; other methods update bases from new snapshots or residual information.
  • Online ADEIM uses sparse residual samples for low-rank basis corrections, while lookahead sampling anticipates upcoming dynamics and orthonormal updates preserve basis structure.

6.3. Adaptive discrete empirical interpolation method (ADEIM)

ADEIM adapts reduced bases and interpolation points online using low-rank updates informed by sparse residual data, while balancing approximation quality against online cost.

  • Basis updates: ADEIM uses low-rank basis updates, typically with rank r much smaller than the basis dimension n, often choosing r=1.Interpolation points are adapted alongside the basis.
  • Online adaptation: ADEIM adapts the basis and interpolation points over time, so the reduced representation evolves with the online state.The online weights can include the reduced state, low-rank basis-update variables, and interpolation-point-update variables.
  • Basis updates: The adapted basis is constructed to approximate vectors in a sliding window, using computable surrogates instead of unavailable full-model solutions.The window has length w, and the surrogates capture information without expensive full-model time stepping.
  • Update objective: The update objective minimizes residual-based approximation error in the sliding-window space, but solving its nonlinear optimization problem online can be expensive.Additional admissibility conditions for interpolation points are not included in the objective.
  • 46?
  • Online cost: At adaptation steps, evaluating the full-model right-hand side at all N components makes online costs at least linear in N; otherwise, sparse queries scale with n_s≪N.These costs are typically still cheaper than solving the corresponding full nonlinear system.

7. Instantaneous residual minimization methods

Generic nonlinear parametrizations let time- and parameter-dependent weights enter without a prescribed structural form, supporting residual-minimization methods beyond specialized transformation or adaptive-basis constructions.

  • Generic parametrizations: Generic parametrizations allow the weight vector to enter nonlinearly without restricting its structural form.The weights may depend on time and parameters, and may be combined with an offline weight vector.
  • Generic parametrizations: This generic formulation contrasts with transformation-based and online adaptive-basis methods, whose weight-vector structures are prescribed and algorithmically exploited.

7.1. Overview of this section

This section develops instantaneous residual minimization for generic nonlinear parametrizations, covering its numerical implementation, alternatives, and applications to trained representations.

  • Section scope: The Dirac–Frenkel variational principle is presented as a core building block for instantaneous residual minimization with generic nonlinear parametrizations.Neural Galerkin schemes are reviewed as methods built on this principle.
  • Numerical considerations: Practical implementations must approximate residual norms, discretize weight evolution in time, and address singular or poorly conditioned systems.The survey discusses collocation, time-stepping choices, and regularization strategies for these issues.
  • Alternative formulations: The section also relates Dirac–Frenkel methods to optimize-then-discretize and discretize-then-optimize schemes.
  • Applications: Applications include instantaneous residual minimization for snapshot-trained nonlinear parametrizations, including autoencoder-based and neural representations.

7.2. The Dirac–Frenkel variational principle

The Dirac–Frenkel principle evolves nonlinear-parametrization weights by making the residual orthogonal to the trial-manifold tangent space, equivalently minimizing the instantaneous residual norm.

  • Nonlinear trial manifolds: A nonlinear parametrization maps a weight vector to a trial manifold of representable functions and requires suitable regularity, including differentiability in the weights.The represented functions must also belong to the PDE solution space.
  • Residual formulation: The residual combines the parametrization’s weight derivative with the governing right-hand side evaluated at the represented solution.The chain rule makes the dependence on the weight derivative explicit.
  • Dirac–Frenkel principle: Dirac–Frenkel chooses the weight derivative so that the residual is orthogonal to all admissible tangent-space variations at the current time.The conditions act locally in time rather than globally across a time interval.
  • Limitations: The principle may fail to determine a unique weight derivative when the tangent-space dimension is smaller than the number of weights.The choice of inner product is also critical because L2 orthogonality may not control the error in the norm of interest.
  • Residual minimization: Minimizing the instantaneous residual norm yields conditions equivalent to the Dirac–Frenkel conditions when the trial set has a suitable manifold structure.
  • Relation to Galerkin projection: For linear parametrizations with orthonormal basis functions, Dirac–Frenkel reduces to the Galerkin reduced model and generalizes Galerkin orthogonality.The weight derivative is the coordinate vector of the projection of the right-hand side onto the tangent-space span.

7.3. Neural Galerkin schemes

Neural Galerkin schemes combine neural-network parametrizations with Dirac–Frenkel dynamics and instantaneous residual minimization. Their sampling points can remain static or adapt toward regions that dominate the residual, addressing the inefficiency of static sampling for moving local features.

  • Sampling strategies: Neural Galerkin schemes numerically evaluate the inner products underlying residual minimization using quadrature or spatial collocation.Static collocation points produce Monte Carlo estimators when sampled from a measure, while quadrature rules can assign nonuniform weights.
  • Neural Galerkin formulation: Neural Galerkin schemes evolve neural-network weights according to the Dirac–Frenkel variational principle.The weights depend on time and parameter, while the resulting dynamics enforce residual orthogonality through the neural parametrization.
  • Neural Galerkin formulation: The instantaneous residual-minimization formulation is a least-squares problem linear in the weight derivative despite the nonlinear solution parametrization.The weight derivative gives the best projection of the right-hand side onto the space spanned by the parametrization’s gradient components.
  • Sampling strategies: Adaptive sampling evolves collocation points with the weights and concentrates them in regions that dominate the residual, yielding more efficient residual-norm estimates than static sampling.The adaptation is described as being driven by a residual-informed potential.
  • Sampling strategies: Static collocation is costly for transport-dominated problems because moving sharp features require fine resolution across the whole spatial domain.Many static points can lie in near-zero regions for most time steps while still being needed to track a moving local feature.

7.4. Semi and fully discrete instantaneous residual minimization

Semi-discrete instantaneous residual minimization represents sampled nonlinear dynamics as a batch least-squares problem. Explicit time discretization preserves linear least-squares updates, whereas implicit discretization produces nonlinear optimization problems at each time step.

  • Semi-discrete formulation: The semi-discrete formulation uses batch gradients and batch right-hand-side values evaluated at the current sampling points.The batch gradient contains transposed parametrization gradients, and the batch right-hand side represents the sampled PDE right-hand side.
  • Semi-discrete formulation: The sampled residual-minimization problem is a Euclidean least-squares problem linear in the unknown update.Its matrix form represents the empirical objective up to a constant scaling factor.
  • Time discretization: Explicit Euler and other explicit Runge–Kutta discretizations lead to linear least-squares problems at every time step.Thus, explicit time integration requires solving only a linear least-squares problem at each step.
  • Time discretization: Implicit Euler makes the time-discrete problem nonlinear because the batch gradient and right-hand side are evaluated at the next iterate.Solving the resulting problem inherits typical difficulties of fitting a nonlinear parametrization.

7.5. Regularization and randomized time integration of Dirac–Frenkel dynamics

Dirac–Frenkel least-squares problems can be poorly conditioned or rank-deficient, motivating regularization and randomized updates. These remedies introduce trade-offs involving stability, bias, parameter redundancy, and the accuracy of the evolving trajectory.

  • Conditioning and collapse: Poorly conditioned or rank-deficient batch gradients can make Dirac–Frenkel updates nonunique and reduce the accuracy of the resulting evolution.Rank deficiency restricts residual orthogonality to a low-dimensional test space, leaving other residual components uncontrolled; this is called matrix singularity or tangent space collapse.
  • Regularization: Regularization can stabilize the sequence of least-squares problems, but bias may accumulate because each problem depends on previous time steps.Common regularizers can also preserve parameter redundancies and prevent recovery from degeneracy.
  • Regularization: Tikhonov regularization damps directions associated with small singular values, suppressing large and potentially unstable weight updates.Its strength α controls the smoothing, while truncated SVD discards small-singular-value directions instead of smoothing them.
  • Randomized time integration: Randomized right-sketching restricts weight updates to an s-dimensional random subspace, reducing the number of unknowns from n to s.Averaging ℓ independent random updates helps control variance, and the sketching dimension directly affects the conditioning of the sketched least-squares problem.

7.6. Instantaneous residual minimization with discretize-then-optimize (DtO) versus optimize-then-discretize (OtD) approaches

OtD derives continuous-time Dirac–Frenkel dynamics before time discretization, while DtO discretizes the PDE first and then minimizes a nonlinear residual at each time step. DtO can avoid OtD conditioning problems but generally has higher computational cost for explicit integration.

  • OtD and DtO: Optimize-then-discretize first derives continuous-time weight dynamics by enforcing residual orthogonality against the trial manifold’s tangent space.Time discretization is applied only after this evolution equation has been formulated.
  • OtD and DtO: Discretize-then-optimize first time-discretizes the PDE, then introduces the nonlinear parametrization and solves a residual-minimization problem for each resulting boundary value problem.This produces a sequence of spatial problems associated with the discrete time steps.
  • Trade-offs: DtO problems are typically non-convex at each time step, making them more expensive than OtD schemes with linear least-squares steps under explicit time integration.The comparison concerns the typical computational costs of the two formulations.
  • Trade-offs: DtO schemes can avoid ill-conditioning issues that typically arise in OtD schemes, including tangent space collapse or matrix singularity.The two formulations therefore exchange higher per-step optimization cost for avoidance of a characteristic OtD conditioning problem.
  • Linear analogue: For linear parametrizations, the DtO/OtD distinction corresponds respectively to least-squares Petrov–Galerkin and Galerkin formulations.This provides a linear-model-reduction analogue of the nonlinear formulation comparison.

7.7. Instantaneous residual minimization with trained nonlinear parametrizations for nonlinear model reduction

This section surveys trained nonlinear parametrizations, especially autoencoders, whose latent online states are evolved by instantaneous residual minimization. It also highlights the main computational challenge: although the reduced optimization variable is low-dimensional, the residual minimization can remain full-dimensional.

  • Trained nonlinear parametrizations: Trained nonlinear parametrizations use offline snapshot data to define online weights obtained by instantaneous residual minimization.The offline weights are fixed after training, while the online weights depend on time and parameters.
  • Autoencoders: Autoencoders encode full-model states into n-dimensional latent states and decode them into N-dimensional approximations, with n ≪ N.In model reduction, the encoder-decoder composition is trained to approximate the identity on solution-manifold data.
  • Training and structure: Autoencoder training commonly minimizes snapshot reconstruction error, with neural architectures and feature choices tailored to the application and online-cost requirements.Reported variants include deep convolutional networks, sparse output masks, fixed feature maps, and additional penalties for structure preservation or physics.
  • Online residual minimization: The decoder induces a vector-valued nonlinear parametrization, while its Jacobian enters the residual that determines the latent-state time derivative.The residual combines the decoder Jacobian, the latent derivative, and the full-model right-hand side evaluated at the decoded state.
  • Online costs: Online speedups are difficult because the residual is defined in R^N and the minimization is therefore carried out in the full-model state space, despite n ≪ N.Implicit time discretizations additionally require a nonlinear least-squares solve at each time step, while constraints can preserve mass, momentum, or energy.
  • Alternative architectures: Linear encoders can achieve reconstruction errors similar to generic nonlinear encoders while improving training robustness, and nonlinear decoders can yield faster error decay for transported features than the Kolmogorov n-width.These findings motivate combining simple encoders with more expressive nonlinear decoders.

8. Conclusions and outlook

The conclusions organize recent progress in transport-dominated model reduction around generic nonlinear parametrizations with instantaneous residual minimization, while distinguishing transformation-based and online adaptive methods. The outlook identifies underrepresented research directions and points to non-intrusive approaches that learn low-dimensional representations and reduced dynamics from data.

  • Conclusions: Nonlinear model reduction for transport-dominated problems has advanced substantially, with generic nonlinear parametrizations and instantaneous residual minimization providing a unifying perspective.The survey nevertheless keeps transformation-based and online adaptive methods distinct because their structural features may improve efficiency and robustness.
  • Conclusions: Transformation-based and online adaptive methods are distinguished because their additional structural features can be leveraged for improved efficiency and robustness.This classification complements the generic parametrization-and-residual-minimization viewpoint.
  • Outlook: Structure-preserving reduction, stochastic-system reduction, and control-oriented model reduction remain underrepresented research directions in the surveyed literature.The survey presents these as important directions that remain insufficiently covered despite the breadth of existing work.
  • Non-intrusive methods: Non-intrusive model reduction learns a low-dimensional representation and corresponding reduced dynamics primarily from snapshot data rather than directly deriving dynamics from governing equations.It is distinct from equation discovery, which uses data to infer governing physical laws.
  • Non-intrusive methods: Several non-intrusive approaches target transport-dominated regimes by learning nonlinear transformations that factor out transport before applying linear non-intrusive reduction.This line of work is described as analogous to transformation-based intrusive methods.
Loading 2602.01397v1…