PaperlayerView exact revision on arXiv

Source-linked AI summary

A physics-informed operator regression framework for extracting data-driven continuum models

Ravi G. Patel, Nathaniel A. Trask, Mitchell A. Wood, Eric C. Cyr

arXiv:2009.11992v1physics.comp-phcs.LGcs.NAmath.NAstat.ML

TL;DR

Learning unknown dynamics requires balancing model flexibility with physical structure, especially for complex, multiscale systems. This paper presents modal-space operator regression with neural networks and physical biases, and reports applications spanning diffusion and flows, including generalization to flow characteristics absent from training.

  • Problem

    For unknown dynamics, it remains an open question how to choose a model form that yields numerically well-posed extracted dynamics while balancing expressivity and physical structure.

    Source: §1 · p. 2

  • Method

    The framework composes a neural-network Fourier-symbol parameterization of differential operators with a neural-network point-wise nonlinear operator, allowing physical biases to be imposed.

    Source: §1 · p. 3, §7 · p. 31

  • Results

    The framework recovers heat and fractional heat equations from Brownian-motion and Levy-flight density measurements and is applied to flows that generalize to unseen particle densities, concentrations, and size ratios.

    Source: §1 · pp. 3–4, §7 · p. 31

  • Takeaways & Limitations

    The reported applications show that the framework covers local and nonlocal diffusion and single- and two-phase flow systems, including flow characteristics absent from training.

    Source: §1 · pp. 3–4

    The framework assumes field data lie in a Banach space on a compact domain and, for its Fourier-series reconstruction space, assumes arbitrarily smooth data.

    Source: §2 · p. 4, §2.3 · p. 7

Abstract

from arXiv · show

The application of deep learning toward discovery of data-driven models requires careful application of inductive biases to obtain a description of physics which is both accurate and robust. We present here a framework for discovering continuum models from high fidelity molecular simulation data. Our approach applies a neural network parameterization of governing physics in modal space, allowing a characterization of differential operators while providing structure which may be used to impose biases related to symmetry, isotropy, and conservation form. We demonstrate the effectiveness of our framework for a variety of physics, including local and nonlocal diffusion processes and single and multiphase flows. For the flow physics we demonstrate this approach leads to a learned operator that generalizes to system characteristics not included in the training sets, such as variable particle sizes, densities, and concentration.

1. Introduction

The paper addresses how to learn unknown dynamics while balancing physical structure with model expressivity. It introduces modal-space operator regression and tests it on synthetic PDEs and molecular-simulation problems, including flows that generalize to unseen system characteristics.

1. Introduction

  • Unknown dynamics pose a model-form challenge: black-box learning is expressive, while dictionary methods impose physical structure through predefined candidates.The paper positions its approach between these alternatives, aiming to impose physical biases without specifying a model-form dictionary in advance.Source: §1 · p. 2
  • The framework composes a neural-network parameterization of a differential operator in modal space with a point-wise nonlinear operator in physical space.Using a Fourier symbol reduces the modal operator to a scalar function of the wavevector and supports biases such as conservation form and isotropy.Source: §1 · pp. 2–3
  • The paper’s central design goal is to extract models that are accurate and stable for long-time integration by balancing inductive biases with data.The authors identify noisy, multiscale particle-simulation data and systems of PDEs as settings where physical biases and scalable architectures matter.Source: §1 · p. 3
  • The evaluation progresses from synthetic scalar PDEs to nonlocal diffusion and molecular-simulation flows, including single- and two-phase systems.The flow models generalize to particle densities, concentrations, and particle-size ratios not seen during training.Source: §1 · pp. 3–4

2. Abstract data-driven modeling framework

The framework learns an operator from field observations by fitting its action on sampled functions, then uses a modal reconstruction space and a neural parameterization to represent the dynamics. Its choices connect time stepping, boundary-conforming spaces, and physical structure to the regression problem.

2. Abstract data-driven modeling framework

  • The framework starts with time-series samples of fields in a function space and seeks dynamics whose predictions remain close to the observed fields.The stated setup assumes a Banach space on a compact domain and known boundary and initial conditions.Source: §2 · p. 4

2.1. Integral Evolution Operator

  • A finite-interval update operator maps the current state to its increment, with a uniform time mesh and one-point quadrature yielding an approximate step operator.The framework adopts explicit dynamics here, while the quadrature choice can correspond to explicit or implicit Euler updates.Source: §2.1 · pp. 4–5

2.2. Operator Regression

  • Operator regression fits a parameterized operator to its action on sampled input functions, using a norm to measure the mismatch.With a boundary-conforming reconstruction space, the formulation is equivalent to reduced-space PDE-constrained optimization and can use standard machine-learning optimizers.Source: §2.2 · pp. 5–6

2.3. Choice of reconstruction space

  • The reconstruction space must satisfy boundary conditions while supporting useful operator parameterizations; the paper uses Fourier series and assumes arbitrarily smooth data.Fourier symbols represent differential operators, and the Fourier transform provides a fast projection; sine and cosine series encode the stated boundary restrictions.Source: §2.3 · pp. 7–8

2.4. Choice of operator parameterization

  • The operator parameterization composes a modal-space network for differential operators with a physical-space network for point-wise nonlinearities.The mapping Π takes physical values to modal space, and the Fourier-symbol parameterization supports physical biases while replacing a high-dimensional mode-to-mode mapping with a one-dimensional function.Source: §2.4 · p. 8

3. Inductive bias

The paper builds physical inductive biases into the operator parameterization rather than relying on data augmentation alone. Its construction imposes translational invariance and can additionally encode reflective symmetry, isotropy, and conservation.

3. Inductive bias

  • The Fourier-based parameterization is translationally invariant because translation commutes with the learned operator.The point-wise nonlinear operator has no explicit spatial dependence, and the Fourier transform’s shift property supplies the invariance.Source: §3 · p. 9
  • The architecture can encode reflective symmetry through an odd point-wise nonlinearity and isotropy by making the Fourier symbol depend on wavevector magnitude.The paper also identifies conservation as an additional bias that can be imposed globally.Source: §3 · p. 10
  • Built-in biases restrict the model search to forms deemed feasible, avoiding the need to sample continuous transformations exhaustively through data augmentation.The paper motivates these biases with translational invariance and isotropy in mechanics and conservation of mass, momentum, or energy.Source: §3 · pp. 9–10

4. Validation for abstract operator regression problems

The validation shows that imposing isotropy improves the learned Laplacian operator’s agreement with the exact operator. The framework also accurately regresses the one-dimensional nonlinear Burgers spatial operator.

4. Validation for abstract operator regression problems

  • The validation fits known steady-state spatial operators on a periodic domain using sampled functions and their operator actions.The experiments include a nonlinear Burgers-motivated operator and the two-dimensional Laplacian; network and training hyperparameters are listed in Table 1.Source: §4 · pp. 10–11, Table 1 · p. 11
  • The learned Burgers operator is relatively accurate, with its action and modal and nonlinear functions agreeing closely with the exact operator.The figure reports relative error versus training-set size alongside comparisons of Im[g(κ)] and h(u).Source: §4 · p. 11, Figure 2 · p. 12
  • The isotropy-constrained regressed operator agrees more strongly with the Laplacian than the unrestricted fit.The figure compares operator applications, Fourier symbols, and relative error as training-set size varies.Source: §4 · pp. 11, 13, Figure 3 · p. 13

5. Diffusion Example: Extraction of anomalous diffusion from molecular data

The diffusion examples use operator regression to recover continuum descriptions of Brownian and Levy dynamics while testing how physical biases affect accuracy and generalization. The reported gains depend on the bias and data regime: some Brownian fits remain flawed, and noisy high-frequency data limit spectral accuracy.

5.1. Application of abstract theory

  • For periodic domains, Fourier-based regression trains continuum operators on coarse-grained particle data and validates them on molecular trajectories from unseen initial conditions.The abstract setup compares learned continuum evolution against a corresponding molecular trajectory.Source: §5.1 · p. 14

5.2.1. Generation of training data from 1D Brownian motion

  • One-dimensional Brownian training uses 100 trajectories of 51200 particles, binned into 256 cells after parameterized multi-von Mises initialization.Source: §5.2.1 · p. 15

5.2.2. Extracting a continuum model from Brownian Motion

  • Unrestricted fits match Brownian training evolution but fail square-wave validation; conservation stops density drift, yet both solutions remain flawed.The validation uses a square-wave initial condition evolved under the heat equation.Source: Figure 4 · p. 16, §5.2.2 · p. 17, Figure 5 · p. 18
  • Reflection lets the learned operator extrapolate to negative densities, where positive-only training data alone cannot identify h(u).With conservation and reflection biases, the learned h and g functions closely match the exact functions.Source: §5.2.2 · p. 17, Figure 6 · p. 18

5.2.4. Levy Flights

  • The learned fractional operator closely reproduces both Levy-flight training evolution and the square-wave fractional heat solution.The example uses a fractional diffusion exponent α/2 = 0.75.Source: §5.2.4 · pp. 19–20, Figure 7 · p. 20
  • Isotropy helps preserve symmetry and benefits long-wavelength modes, but all learned spectra deviate at short wavelengths as noise overtakes signal.The authors attribute high-frequency error to noise in the training data.Source: §5.2.4 · p. 20, Figure 8 · p. 21, Figure 9 · p. 21
  • With isotropy imposed, RMS error falls by multiple orders across β values; even β = 1 outperforms unrestricted fits, though added data brings no marked improvement.The comparison spans training-set sizes and degrees of anisotropy.Source: §5.2.4 · p. 22, Figure 10 · p. 22

6. Continuum mechanics: extraction of continuum mechanics models from molecular dynamics simulations

The section extracts continuum models from molecular-dynamics data to bridge micro- and macroscale descriptions, then evaluates their resolution sensitivity, field choices, and generalization. The learned models agree with held-out flow simulations through the simulated interval and remain stable at steady state beyond it.

6. Continuum mechanics: extraction of continuum mechanics models from molecular dynamics simulations

  • The study targets continuum models that bridge molecular and macroscopic scales whose required simulation ranges exceed even leadership computing capabilities.It evaluates pressure-driven planar Lennard-Jones and colloidal flows, aiming to generalize beyond training densities, concentrations, and particle sizes.Source: §6 · pp. 22–23

6.1. Application of abstract theory

  • The models use DCT/DST operator parameterizations for fields with Neumann or Dirichlet boundary conditions and include simulation parameters as inputs.The loss weights fast-timescale dynamics when β > 0, avoiding the loss of short-timescale behavior as longer-timescale dynamics are learned; density conservation is assumed and tested.Source: §6.1 · pp. 23–24

6.2. Generation of training data

  • LAMMPS simulations supply training data across LJ densities and colloidal concentrations and particle sizes; validation tests interpolate between training conditions and extrapolate to later times.Grid-bin averages are collected over particle simulations, with averaging used to reduce noise from fluctuations near bin boundaries.Source: §6.2 · pp. 24–25

6.3.1. Refinement study

  • At steady state, all three grid resolutions agree well with LAMMPS, but the N = 80 model performs worse, so subsequent studies use N = 20.The finer grid reveals additional flow features near the walls, which may make training harder.Source: Figure 11 · p. 26, §6.3.1 · p. 27

6.3.2. Model comparison

  • For LJ, two- and three-field models perform similarly and outperform the one-field model; for colloids, two and four fields perform similarly while six fields perform poorly.Accuracy improves with more training data before saturating; conservation shows no consistent LJ benefit but a small, more consistent colloidal benefit.Source: Figure 12 · p. 27, Figure 13 · p. 27, §6.3.2 · p. 28

6.3.3. Model generalization

  • Both learned flow models agree well with held-out LAMMPS simulations through the simulated interval and appear stable at steady state beyond it.The tests use LJ density ρ*=0.1 and colloidal parameters (c, d) = (0.15, 2.5), held out from training.Source: Figure 14 · p. 29, Figure 15 · p. 30, §6.3.3 · p. 31

7. Conclusion

The paper extends pseudo-spectral PDE discovery to systems of equations, multidimensional problems, and nontrivial boundaries, using neural networks to parameterize spatial operators with physical biases. It reports recovery of diffusion models and suitable continuum models from molecular simulations, with improved regression accuracy and extrapolation beyond training dynamics.

7. Conclusion

  • The framework parameterizes spatial operators as a Fourier multiplier composed with a nonlinear pointwise neural functional, allowing biases for conservation, translation invariance, isotropy, and reflective symmetry.The authors report recovering heat and fractional heat equations from Brownian-motion and Levy-flight density data, and continuum models from Lennard-Jones and colloidal flow simulations.Source: §7 · p. 31
  • The authors identify spectral-element discretizations for complex geometries and realistic engineering settings as future directions, including possible applications to turbulence, multiphase flows, and material mechanics.They also note that the framework may be generalized from molecular-dynamics data to nonsynthetic experimental data.Source: §7 · p. 31
Loading 2009.11992v1…