PaperlayerView exact revision on arXiv

Source-linked AI summary

Inertial Manifold Neural Operator for Dissipative Time-Dependent Partial Differential Equations

Xiaoyang Xie, Clarence W. Rowley

arXiv:2608.23546v1math.NAcs.LGmath.DS

TL;DR

Long-horizon learning of dissipative PDE dynamics requires models that can represent low-dimensional long-time structure while retaining transient behavior. The paper introduces IMNO, which separates manifold dynamics from residual corrections, and reports stable, accurate performance across benchmarks, with limitations in strongly chaotic or structurally complex regimes.

  • Problem

    Long-horizon autoregressive PDE prediction can become unstable, while reduced-order models may neglect transient off-manifold behavior important for recovering full dynamics.

  • Method

    IMNO learns low-dimensional manifold dynamics together with a function-valued residual correction; its latent dynamics are modeled with a Neural ODE, and phase-dependent features support shift-equivariant residual updates.

  • Results

    Benchmark experiments show that IMNO often captures low-dimensional long-time dynamics while maintaining good training stability and accuracy, including on more complex systems where the reduction is difficult.

  • Takeaways & Limitations

    The framework provides a physically interpretable neural-operator representation that can preserve full PDE dynamics through explicit residual modeling while exploiting dissipative low-dimensional structure.

  • Takeaways & Limitations

    IMNO is less accurate for strongly chaotic dynamics, and its manifold component may not cleanly describe compressible Navier–Stokes dynamics, for which finite-dimensional inertial-manifold existence remains open.

Abstract

from arXiv · show

In this paper, we introduce the Inertial Manifold Neural Operator (IMNO) for solving dissipative time-dependent partial differential equations (PDEs). The long-time dynamics of such systems often exhibit an effective low-dimensional structure due to dissipation. Unlike standard neural operator architectures such as the Fourier Neural Operator (FNO), IMNO explicitly leverages the low-dimensional structure to achieve better physical interpretability, accuracy, and stability in long-horizon autoregressive training and prediction for nonlinear dissipative PDEs. For shift-equivariant PDEs, we further introduce a shift-equivariant variant (IMNO-SE) of the proposed neural operator, ensuring that a spatial shift in the input induces the same spatial shift in the output. This symmetry-preserving inductive bias substantially improves its performance in shift-equivariant PDEs. Extensive benchmark experiments are presented to evaluate IMNO's performance numerically.

1 Introduction

Dissipative PDEs often develop low-dimensional long-time dynamics, motivating reduced-order models and neural operators. IMNO combines these perspectives by learning manifold dynamics alongside transient corrections, with experiments showing stable long-horizon rollouts and effective low-dimensional extraction.

  • Dissipation can suppress transient degrees of freedom, yielding a simpler finite-dimensional description of long-time PDE dynamics.
  • Reduced-order modeling exploits low-dimensional structure to reduce simulation cost and improve understanding of dominant system mechanisms.
  • Inertial manifold theory explains how infinite-dimensional dissipative PDEs can have long-time behavior governed by a finite-dimensional invariant manifold attracting trajectories exponentially.
  • ROMs inevitably neglect transient behavior outside their low-dimensional manifold, which may matter for accurately recovering full dynamics.
  • IMNO decomposes solutions into dominant manifold dynamics and a function-valued residual correction, combining reduced-order interpretability with neural-operator expressiveness.The decomposition is described as making learning more structured and better conditioned for long-horizon autoregressive rollout.
  • Extensive experiments report stable long-horizon multistep rollout training, good accuracy, and effective extraction of low-dimensional long-time dynamics.

2 Background

The background defines neural operators as function-to-function solution-operator learners and motivates long-horizon autoregressive training for dissipative PDEs. It contrasts FNO’s Fourier-kernel construction with the need for architectures that remain stable over extended rollouts.

  • The problem setting considers nonlinear dissipative time-dependent PDEs whose solution semigroup defines evolution from an initial condition.
  • The learning objective is a one-step infinite-dimensional solution operator whose repeated application generates an autoregressive full-trajectory rollout.
  • Neural operators learn mappings between function spaces rather than fixed discretized vectors and should remain valid across spatial resolutions.
  • Kernel-based neural operators compose lifting, hidden integral-operator, and projection layers to map input functions to output functions.
  • FNO restricts kernels to translation-invariant, Fourier-parameterized convolutions, enabling global receptive fields and O(N log N) layer cost through FFTs.
  • Long-horizon autoregressive training can destabilize FNO-like models, while dissipative structure is important because repeated one-step errors may otherwise grow substantially.
  • The proposed approach assumes an invariant finite-dimensional inertial manifold that exponentially attracts trajectories and incorporates this structure for stable, accurate, interpretable learning.

2.4 Inertial Manifolds and Asymptotic Completeness

This section develops the mathematical setting for inertial manifolds, their finite-dimensional reduced dynamics, and asymptotic completeness. Spectral separation supports graph representations and exponential attraction, while flow-normal hyperbolicity provides orbitwise tracking and phase uniqueness.

  • 2.4.1 The Evolution Equation: The dissipative evolution framework uses a positive self-adjoint operator with compact inverse, whose high-frequency modes decay rapidly while nonlinearities couple modes.
  • 2.4.2 Inertial Manifold: Dissipative systems possess absorbing sets and, under compactness, global attractors, but an attractor alone need not provide a convenient low-dimensional dynamical description.
  • 2.4.2 Inertial Manifold: An inertial manifold is finite-dimensional, positively invariant, and exponentially attracting, allowing long-time dynamics to be represented by a closed finite-dimensional system.
  • 2.4.3 Spectral gap condition and existence of inertial manifolds: Existence results commonly rely on a spectral gap separating low and high modes, enabling fast-decaying components to be slaved to slow dynamics.
  • 2.4.4 Reduced dynamics: When represented as a graph over low modes, the inertial-manifold decomposition yields a closed reduced system whose solutions parameterize manifold trajectories.
  • 2.4.5 Asymptotic completeness: Asymptotic completeness upgrades setwise attraction to exponential tracking of every full-space trajectory by a trajectory on the manifold.
  • 2.4.5 Asymptotic completeness: Flow-normal hyperbolicity is a sufficient condition for asymptotic completeness and implies uniqueness of the associated asymptotic phase.

3 Inertial Manifold Neural Operators (IMNO)

IMNO decomposes PDE states into low-dimensional manifold dynamics and function-valued residual corrections, learning separate dynamics for each component. Its architecture uses resolution-independent encoder–decoder mappings, autonomous latent evolution, and latent-conditioned residual updates.

  • Architecture: IMNO decomposes each state into a low-dimensional manifold coordinate h(t) and a function-valued residual uR(t).The two components jointly represent the reduced dynamics and off-manifold correction needed to recover the full state.
  • Architecture: The learned manifold component represents asymptotic-phase dynamics, while residual components capture remaining degrees of freedom that decay rapidly.This decomposition follows the inertial-manifold and stable-leaf perspective.
  • Manifold representation: An encoder–decoder pair parameterizes the learned manifold using function-to-coordinate and coordinate-to-function mappings compatible with varying spatial resolutions.The encoder produces latent coordinates from input fields, and the decoder reconstructs manifold fields from those coordinates.
  • Dynamics: The latent coordinate evolves autonomously through a learned Neural ODE, while the residual dynamics depend on the current latent coordinate.IMNO uses forward Euler for the latent update and conditions residual-operator layers through latent-dependent affine corrections.
  • Residual dynamics: The residual operator is a conditioned neural operator that maps function-valued residual inputs to outputs while coupling off-manifold evolution to the finite-dimensional latent state.Its Fourier-based hidden layers retain the spectral-convolution structure of FNO while incorporating latent conditioning.
  • Approximation theory: Under finite-dimensional inertial-manifold assumptions, the paper states that IMNO can uniformly approximate the continuous one-step solution operator on compact subsets.The theorem assumes a finite-dimensional flow-normally hyperbolic inertial manifold and provides approximation for every ε > 0.

4 Shift-Equivariant Inertial Manifold Neural Operator

IMNO-SE separates spatial translation from intrinsic dynamics by working with a canonical representative and an explicit phase. Its construction is strictly shift-equivariant and reduces redundancy in the latent representation.

  • Shift symmetry: Translation-invariant PDEs have solution operators that commute with spatial shifts, so translated initial conditions produce translated solutions.Translation symmetry also creates group orbits of dynamically equivalent states in the original function space.
  • Motivation: Separating translation phase from intrinsic shape dynamics can produce more compact and physically meaningful reduced representations.The original IMNO does not automatically preserve this symmetry because its latent coordinate may entangle intrinsic shape with spatial translation.
  • Symmetry reduction: IMNO-SE fixes the phase of a reference Fourier mode to define a canonical representative, with the phase recording spatial translation and the representative encoding intrinsic shape.The reduced dynamics evolve in the quotient space H/G while the phase tracks motion along the symmetry direction.
  • IMNO-SE architecture: The shift-equivariant model parameterizes an inertial manifold in quotient space and extends the decomposition with latent, residual, and translation-phase variables.Phase alignment is applied before encoding, and the decoded manifold field is kept in the canonical frame.
  • Residual dynamics: The residual update depends on the current residual, latent coordinate, and translation phase, reflecting the coupled structure of off-manifold dynamics.The phase is incorporated through a spatial feature that transforms consistently under shifts rather than through an absolute coordinate embedding.
  • Equivariance: By construction, IMNO-SE maps any spatially shifted input to the correspondingly shifted output while reducing redundancy in the latent representation.Fourier-layer operations commute with translations, and the phase-aware embedding preserves this property.

5 Numerical Experiments

Experiments across dissipative PDEs show that IMNO and IMNO-SE generally provide accurate, stable rollouts while learning low-dimensional long-time dynamics. Performance is weaker in strongly chaotic regimes, and the manifold component may not independently describe highly coupled systems.

  • Burgers Equation: IMNO-SE achieves the best overall Burgers accuracy across short- and long-horizon rollouts, remaining stable where FNO does not and outperforming RNO with fewer parameters.The comparison uses matched Fourier modes and channel widths across models.
  • Nonlocal Burgers Equation: IMNO’s manifold component captures dominant long-time dynamics, while its residual correction preserves accurate full predictions through short-time transients.Latent trajectories also approach attractors consistent with the known inertial-manifold phase-space structure.
  • Kuramoto–Sivashinsky Equation: For nonchaotic KS dynamics at L = 4π, IMNO-SE accurately predicts both the full field and manifold component as solutions rapidly approach the inertial manifold.The learned latent trajectories approach two attractors with oscillatory transients, matching the two symmetry-related stable traveling waves.
  • Kuramoto–Sivashinsky Equation: When KS becomes chaotic at L = 8π, IMNO-SE is less accurate than FNO and RNO because reduced-manifold approximation errors can accumulate during long autoregressive rollouts.The paper identifies strongly chaotic dynamics as a limitation for IMNO, despite retained low-dimensional structure capture.
  • Navier–Stokes Equations: Increasing α improves the manifold prediction but degrades overall final-field accuracy, while IMNO-SE remains accurate in compressible Navier–Stokes even when the manifold component is not a clean standalone reduction.Strong coupling among velocity, density, pressure, and energy can prevent a simple reduced-order description, while the latent representation still provides useful global information for residual prediction.

6 Conclusion

The paper develops IMNO to combine inertial-manifold structure with neural operators for dissipative time-dependent PDEs. Benchmarks show effective long-time-dynamics capture, while highlighting challenges in chaotic regimes and manifold–residual training.

  • 6 Conclusion: IMNO learns low-dimensional long-time dynamics together with residual corrections, aiming for physically interpretable representations that preserve PDE structure.The framework is guided by inertial manifold theory, which decomposes solutions into a manifold component and a rapidly decaying residual.
  • 6 Conclusion: Benchmark results show good accuracy and training stability, including effective capture of low-dimensional long-time dynamics in many cases.For more complex systems, IMNO can still retain good neural-operator accuracy even when reduced dynamics are difficult to capture precisely.
  • 6 Conclusion: Increasing α often worsens predictive performance by making the residual more high-frequency and harder for the FNO-based component to learn.The authors identify joint manifold–residual training and loss design as directions for future work.
  • 6 Conclusion: Chaotic systems remain a limitation because latent prediction errors can accumulate during autoregressive rollout and distort both manifold and residual components.Periodic reprojection onto the learned manifold is proposed as a possible mitigation.
  • 6 Conclusion: Larger-scale simulations may benefit from rolling out mainly through manifold dynamics after the residual has sufficiently decayed.This could reduce the need to evaluate the full residual neural operator at every step.
  • 6 Conclusion: The work connects operator learning and reduced-order modeling, potentially clarifying their respective advantages and limitations.The authors present this connection as a direction for further scientific machine-learning research.

Data availability

The paper states that code and data for reproducing its results will be made available online.

  • Data availability: Code and data for reproducing the results will be available at the listed GitHub repository.The numerical experiments used NVIDIA A100 and L40 GPUs.

A. Proof of the Universal Approximation Theorem

Appendix A proves Theorem 1 through notation, preliminary properties, six lemmas, and an assembled final argument.

  • A. Proof of the Universal Approximation Theorem: Appendix A proves Theorem 1 by organizing the argument into notation, preliminary properties, six lemmas, and the final assembly.Sections A.1–A.3 correspond to these proof stages.

A.1 Basic settings and properties

The proof establishes the settings and continuity properties needed to construct an IMNO approximation using Fourier features, latent dynamics, and conditioned residual operators.

  • A.1 Basic settings and properties: Fourier projections onto finitely many modes converge uniformly on compact subsets, supporting finite-dimensional approximation of function-space inputs.The projections are uniformly bounded and converge to the identity as the truncation level grows.
  • A.1 Basic settings and properties: The proof assumes a continuous one-step solution operator on a Hilbert space and a finite-dimensional, flow-normally hyperbolic inertial manifold with a continuous phase map.These objects provide the theoretical setting for the approximation construction.
  • A.1 Basic settings and properties: The restricted one-step map on the inertial manifold is injective, enabling well-defined forward latent dynamics.Injectivity follows from the backward separation estimate.
  • A.1 Basic settings and properties: The construction defines exact one-step latent and residual maps before approximating them with neural components.The latent map is continuous and injective on the compact latent domain, while the residual map gives the exact next-step residual.
  • A.1 Basic settings and properties: The IMNO architecture comprises a Fourier-feature encoder, decoder, explicit latent update, and latent-conditioned residual neural operator.The residual operator uses pointwise maps, Fourier integral operators, and affine conditioning from the latent coordinate; the activation is chosen for MLP universal approximation.

A.2 Preliminary lemmas

The preliminary lemmas establish universal approximation results for the encoder, decoder, and conditioned residual operator used by IMNO. Their proofs reduce function-space approximation to finite-dimensional Fourier truncation and MLP approximation, while representing latent conditioning through constant input channels or additive terms.

  • Fourier neural operators provide the universal-approximation foundation for the theoretical construction.The preliminary material recalls the FNO universality theorem before proving approximation results for IMNO components.
  • The encoder can approximate any continuous map from a compact subset of H to the finite-dimensional latent space R^dh.Uniform Fourier truncation on compact sets reduces the infinite-dimensional problem to finite-dimensional MLP approximation.
  • The decoder can approximate any continuous map from a compact latent set into H using an MLP followed by band-limited Fourier synthesis.The MLP approximates retained Fourier coefficients, and inverse Fourier synthesis reconstructs the function-space output.
  • The conditioned residual operator is universal for continuous maps on compact subsets of H × R^dh.The proof augments the field with a spatially constant latent channel, applies FNO universality, and reproduces the result through additive latent conditioning.
  • A conditioned residual architecture can exactly reproduce the augmented-input FNO on the compact set used in the proof.Auxiliary channels retain local input features, while target channels reproduce the hidden features of the augmented FNO.

A.3 Main Proof

The main proof combines approximation results for the encoder, latent dynamics, decoder, and residual operator to establish one-step IMNO approximation on a compact input set. Error budgets and continuity control keep intermediate approximations within the domains required by subsequent components.

  • The proof combines separate approximation results for the encoder, latent dynamics, decoder, and residual operator.The construction allocates error budgets across these components before combining their estimates.
  • Continuity and compactness ensure that small latent-coordinate errors remain controlled after decoding and within the residual operator’s neighborhood.The proof explicitly uses a controlled neighborhood around the exact residual and latent pair.
  • The encoder and learned latent dynamics approximate the exact latent coordinate and its continuous one-step evolution on the compact domain.The approximate latent state remains in an enlarged compact latent set used for subsequent approximation.
  • The decoder approximates the next manifold component and yields a residual input close to the reference residual.The learned residual is formed by subtracting the decoded approximate manifold component from the current state.
  • The conditioned residual operator approximates the exact next-step residual when evaluated at the approximate residual and latent coordinate.The target residual is continuous on the compact domain, enabling application of the residual-operator universality result.
  • Combining manifold and residual estimates gives the one-step IMNO approximation bound, and taking the supremum over the compact input set proves the claim.The final estimate compares the IMNO prediction with the exact one-step solution using the triangle inequality.

B. Recurrent Neural Operator (RNO)

The Recurrent Neural Operator maintains a function-valued hidden state across time and updates it with gated recurrent dynamics implemented using neural operators. A pointwise projection maps the updated hidden state back to the solution space.

  • RNO carries a function-valued hidden state across time instead of predicting each next solution solely from the current field.This hidden state stores information across autoregressive time steps.
  • A pointwise lifting layer maps the solution field into a latent feature field before recurrent updating.The lifting layer is implemented by an MLP.
  • Gated recurrent dynamics update the hidden state using update and reset gates together with a candidate state.The update gate controls retention of the previous state, while the reset gate controls its contribution to the candidate.
  • The recurrent update incorporates nonlocal spatial interactions because its neural operators are implemented as FNO layers.The spectral convolution supplies the spatially nonlocal interaction mechanism.
  • The updated hidden state is projected back to the solution space to produce the next-time-step prediction.The projection layer Q is pointwise.
Loading 2608.23546v1…