PaperlayerView exact revision on arXiv

Source-linked AI summary

Learned Coarse Models for Efficient Turbulence Simulation

Kimberly Stachenfeld, Drummond B. Fielding, Dmitrii Kochkov, Miles Cranmer, Tobias Pfaff, Jonathan Godwin, Can Cui, Shirley Ho, Peter Battaglia, Alvaro Sanchez-Gonzalez

arXiv:2112.15275v3physics.flu-dyncs.LGphysics.comp-ph

TL;DR

Classical turbulence solvers require high-resolution grids, while fully learned models can struggle with stability and generalization. This paper trains learned simulators on coarse grids and finds they outperform comparably coarse classical solvers across scientifically relevant measures, with training choices helping stability and generalization.

  • Problem

    Classical solvers often require high-resolution grids and substantial computational resources to capture turbulent dynamics accurately.

    Source: §1 · p. 1, §2 · p. 2

  • Method

    The paper trains fully learned simulators end-to-end to predict state differences from high-resolution-generated trajectories at low spatial and temporal resolutions.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    Learned simulators outperform comparably coarse solvers across scientifically relevant measures, particularly in preserving high-frequency information.

    Source: §6 · p. 9

  • Takeaways & Limitations

    Learned simulators may distill expensive high-resolution engineered simulators into low-resolution algorithms with improved performance-versus-efficiency tradeoffs.

    Source: §6 · p. 9

    Under more extreme generalization conditions, coarse Athena++ models outperform learned models.

    Source: §6 · p. 9

Abstract

from arXiv · show

Turbulence simulation with classical numerical solvers requires high-resolution grids to accurately resolve dynamics. Here we train learned simulators at low spatial and temporal resolutions to capture turbulent dynamics generated at high resolution. We show that our proposed model can simulate turbulent dynamics more accurately than classical numerical solvers at the comparably low resolutions across various scientifically relevant metrics. Our model is trained end-to-end from data and is capable of learning a range of challenging chaotic and turbulent dynamics at low resolution, including trajectories generated by the state-of-the-art Athena++ engine. We show that our simpler, general-purpose architecture outperforms various more specialized, turbulence-specific architectures from the learned turbulence simulation literature. In general, we see that learned simulators yield unstable trajectories; however, we show that tuning training noise and temporal downsampling solves this problem. We also find that while generalization beyond the training distribution is a challenge for learned models, training noise, added loss constraints, and dataset augmentation can help. Broadly, we conclude that our learned simulator outperforms traditional solvers run on coarser grids, and emphasize that simple design choices can offer stability and robust generalization.

1 INTRODUCTION

Classical turbulence solvers often need high-resolution grids and substantial computational resources, while fully learned simulators can struggle with stability and generalization. The paper presents a fully learned approach that performs well on coarse grids and studies choices that improve its robustness.

1 INTRODUCTION

  • Classical solvers can lose high-frequency information on coarse grids, while chaotic dynamics can amplify small changes in initial conditions.High-resolution grids preserve accuracy but require substantial computational resources.Source: §1 · p. 1
  • Fully learned simulators can work well near their training distribution but have limitations in stability and generalization.The paper evaluates how training choices affect generalization to initial conditions, rollout durations, and environment sizes outside training.Source: §1 · p. 2
  • Fully learned simulators predict turbulent dynamics on coarse grids more accurately than classical solvers across several metrics, particularly preserving high-frequency information.The paper also reports that Dil-ResNet outperforms more complicated, more heavily parameterized learned models across varied challenging domains.Source: §1 · p. 2
  • Tuning training noise and temporal downsampling can reliably produce stable learned-simulator rollouts.The paper also evaluates training choices for improving out-of-distribution generalization.Source: §1 · p. 2

2 RELATED WORK

Turbulence simulation matters across scientific and engineering applications, but traditional solvers can be expensive. Prior learned approaches range from solver-integrated models to end-to-end architectures, including models designed specifically for turbulence.

2 RELATED WORK

  • Turbulence metrics include velocity autocorrelation, variable histograms, cooling, and energy spectra, which reflect small- and large-scale flow dynamics and their interplay.Applications include forecasting, engineering design, and scientific questions.Source: §2 · p. 2
  • Traditional PDE-based turbulence solvers can be extremely expensive to run, motivating interest in more efficient simulation.Athena++ is described as a state-of-the-art solver that prioritizes preserving high-order statistics.Source: §2 · p. 2
  • Learned physical-dynamics models span graph and grid representations, fine-grained dynamics, simulation acceleration, and super-resolution.Prior examples include models of airfoil flow and transient low-Reynolds-number flows.Source: §2 · p. 2
  • Turbulence-specific learned approaches include solver components, PDE formulations, embedding spaces, and end-to-end architectures such as TF-Net and Fourier Neural Operators.The paper implements models from this literature as baselines.Source: §2 · pp. 2–3
  • Figure 1 compares predicted rollout frames with ground truth across four domains, with stated accuracy spanning 31 to 119 model steps in three examples.The caption also describes KS-1D as close to ground truth for its first 150 steps and plausible thereafter.Source: Figure 1 · p. 3

3 MODEL

The model learns grid-based state transitions end-to-end by predicting state differences. Its model set includes a general-purpose Dilated ResNet and specialized or constrained baselines, trained with a shared loss and optional noise.

3.1 LEARNED SIMULATION FRAMEWORK ON GRIDS

  • The learned simulator maps a grid state to its next state by adding a neural-network-predicted difference to the current state.Rollouts are generated by iteratively applying the model from the supplied initial state.Source: §3.1 · p. 3

3.2 MODELS

  • Dil-ResNet combines encode-process-decode with dilated CNN blocks and residual connections, using dilation for long-range communication while preserving local structure.The model is not specialized for turbulence; comparisons include U-Net, TF-Net, constrained variants, and Fourier Neural Operators.Source: §3.2 · p. 4

3.3 TRAINING

  • The models are trained to predict state differences using mean squared error, with normalized inputs and targets.Constraint-preservation losses are added to constrained models with relative weight 1.Source: §3.3 · p. 4
  • Optional Gaussian input noise is paired with a corresponding target adjustment to train models to correct small errors and improve rollout stability.The approach is described as helping prevent error accumulation.Source: §3.3 · p. 4

4 EXPERIMENTAL TURBULENT DOMAINS

The experiments cover four chaotic or turbulent PDE environments generated at high resolution, spanning one- to three-dimensional settings and varied flow conditions.

4 EXPERIMENTAL TURBULENT DOMAINS

  • The study uses four PDE environments with chaotic or turbulent dynamics, ranging from classic cases to compressible flows and non-uniform initial conditions.Training data is generated at high resolution.Source: §4 · p. 5
  • KS-1D is a one-dimensional chaotic, unstable PDE used to assess models on highly nonlinear dynamics.Its state is a scalar velocity at each grid point.Source: §4 · p. 5
  • INCOMP-2D models decaying incompressible flow in which small-scale eddies decay into large-scale structures through an inverse energy cascade.Its state contains two velocity components at each grid point.Source: §4 · p. 5
  • COMPDECAY-3D is transonic compressible decaying turbulence simulated with Athena++, with density, velocity, and pressure represented on a three-dimensional grid.The setup assumes an adiabatic equation of state with γ = 5/3.Source: §4 · p. 5
  • COOLMIXLAYER-3D models Kelvin–Helmholtz mixing with strong cooling, open boundaries, and non-uniform initial conditions.The environment is solved with Athena++ and uses the same state specification as COMPDECAY-3D.Source: §4 · p. 5

5 RESULTS

Across the tested turbulence settings, learned simulators are evaluated for performance, stability, efficiency, and generalization. Dil-ResNet often beats same-resolution Athena++, while training noise and temporal downsampling can improve rollout stability; generalization remains limited in some settings.

Spatial coarsening

  • Dil-ResNet outperforms same-resolution Athena++ across several metrics, including energy-field RMSE and higher-order velocity autocorrelations, while energy-spectrum error is lower than for both 32^3 and 64^3 Athena++.For feature and phase histograms and per-feature mean squared error, it beats 32^3 Athena++ but not 64^3 Athena++; different applications may favor different metrics.Source: §5 · p. 6, Figure 2 · p. 6

Stability and training noise

  • Adding Gaussian input noise σ = 0.01 worsens one-step predictions but produces more stable rollouts, particularly when models take many small steps.The proposed explanation is that broader training support helps map deviant inputs back toward the training distribution.Source: §5 · pp. 6–7

Temporal coarsening

  • Rollout error is minimized around ∆t = 0.032; larger steps make next-state prediction harder, while steps below 0.008 often destabilize rollouts through accumulated errors.Training noise can stabilize small-step rollouts to some extent.Source: §5 · p. 7, Figure 3 · p. 7

Comparison across learned models

  • Dilated ResNets consistently have the lowest error among the compared learned architectures, and training noise improves stability across models, although its effects vary by task.Noise harms performance in KS-1D, particularly benefits FNO rollouts in INCOMP-2D, and mainly stabilizes COMPDECAY-3D rollouts.Source: Figure 4 · p. 7, §5 · p. 8

Running time

  • 1000x: COMPDECAY-3D Dil-ResNet running on one GPU is faster than Athena++ on an 8-core CPU when downsampling time from ∆t = 0.0005 to 0.032 and space from 128^3 to 32^3.Source: §5 · p. 8

Constraint satisfaction and preserved quantities

  • Training noise helps preserve mean KS-1D velocity and keep INCOMP-2D divergence near zero, but does not prevent COMPDECAY-3D mass, momentum, and energy drift.The authors speculate these conserved quantities comprise only 5 of 163840 output variables in the 3D state.Source: §5 · p. 8

Generalization to longer rollouts

  • Dil-ResNet remains competitive with 32^3 Athena++ for COMPDECAY-3D turbulence decayed up to twice as long as its training trajectories.The state distribution changes over time as the flow decays into smaller structures.Source: §5 · p. 8

Generalization beyond the training distribution

  • Learned-model generalization is limited for more-compressive initial conditions and box sizes outside training; loss constraints and training on multiple sizes help, but far-out-of-range generalization remains unreliable.Across seeds, models trained on a single box size do not generalize outside the training range; training on multiple sizes improves performance but does not resolve generalization far beyond that range.Source: §5 · pp. 8–9, Figure 5 · p. 9

6 CONCLUSIONS

Learned simulators can outperform comparably coarse solvers on turbulent dynamics, including on high-frequency measures, while training choices improve extended-rollout stability. Generalization remains difficult in extreme conditions, though regularization and dataset augmentation help.

6 CONCLUSIONS

  • Dilated ResNet had lower log Energy Spectrum RMSE than both 32^3 and 64^3 Athena++, with lower RMSE persisting beyond the training duration.The results support learned simulation as a way to distill expensive high-resolution simulators into lower-resolution methods with improved performance-versus-efficiency tradeoffs.Source: §6 · p. 9
  • Training noise and temporal downsampling improved the stability and accuracy of extended rollouts in these experiments.Source: §6 · p. 9
  • Coarse Athena++ models outperform learned models under more extreme generalization conditions, but physically informed regularization improves generalization to different initial conditions on COMPDECAY-3D.Source: §6 · p. 9
  • For COOLMIXLAYER-3D, training on a larger range of box sizes improved generalization across an even larger range.Source: §6 · p. 9
  • The authors identify distilling expensive, high-resolution engineered simulators into low-resolution algorithms as a potential near-term role for learned simulators.They report applicability to diverse grid-based environments, including with generic CNN-based models.Source: §6 · p. 9

A APPENDIX

The appendix table presents hyperparameters for four basic models and clarifies how constrained variants and TF-Net’s spatial filter relate to those entries.

A APPENDIX

  • Table A.1 lists hyperparameters for four basic models.Source: Table A.1 · p. 14
  • Con-Dil-ResNet and Con-TF-Net use the architectural parameters of Dil-ResNet and TF-Net, respectively.Source: Table A.1 · p. 14
  • The constrained variants add a constraint weighted by the constraint weight.Source: Table A.1 · p. 14
  • TF-Net’s spatial-filter kernel is distinct from the kernel size in the first table row because it is used during decomposition.Source: Table A.1 · p. 14

B ADDITIONAL MODEL DETAILS

The appendix describes alternative learned architectures and constraint-based variants, alongside a compute limitation encountered for Fourier Neural Operators on 3D datasets.

B ADDITIONAL MODEL DETAILS

  • The U-Net baseline uses linear CNN encoding and decoding around a seven-layer U-Net whose downsampling and upsampling support long-range communication while preserving local structure.Source: B ADDITIONAL MODEL DETAILS · p. 14
  • TF-Net processes learned spatial, temporal, and difference-based components through separate U-Net encoders, then concatenates their layer outputs for a shared decoder.Source: B ADDITIONAL MODEL DETAILS · pp. 14–15
  • Con-TF-Net adapts its loss constraint to each dataset: zero mean for KS-1D, zero divergence for INCOMP-2D, and input-matched total energy for COMPDECAY-3D.Source: B ADDITIONAL MODEL DETAILS · p. 15
  • Con-Dil-ResNet adds the constraint terms to Dil-ResNet’s loss, isolating the loss-term effect from other TF-Net architectural features.Source: B ADDITIONAL MODEL DETAILS · p. 15
  • The Fourier Neural Operator applies learned matrix multiplications in the Fourier domain, with optional low-pass filtering by truncating modes.Source: B ADDITIONAL MODEL DETAILS · p. 15
  • The authors could not readily run the Fourier Neural Operator on 3D datasets because of compute and memory constraints.They speculate that Fourier-transform complexity in the backward pass contributes, possibly in a TensorFlow-specific way.Source: B ADDITIONAL MODEL DETAILS · p. 15

B.1 ADDITIONAL DATASET DETAILS

The appendix documents datasets spanning one-dimensional chaotic dynamics, two-dimensional incompressible decaying turbulence, and three-dimensional compressible decaying turbulence.

B.1.1 1D KURAMOTO-SIVASHINSKY (KS) EQUATION

  • KS-1D is a periodic one-dimensional PDE with unstable chaotic dynamics; trajectories use randomized initial-condition phases and a 75-unit warm-up.The cited work identifies the Kuramoto–Sivashinsky equation.Source: §B.1.1 · p. 15

B.1.2 2D INCOMPRESSIBLE DECAYING TURBULENCE

  • INCOMP-2D uses periodic-boundary incompressible Navier–Stokes DNS with random small-scale velocity fields and a 500-unit warm-up; warm-up transients are excluded from training and evaluation.The dataset table notes that its figures are dimensionless.Source: §B.1.2 · p. 15, Table B.1 · p. 16

B.1.3 3D COMPRESSIBLE DECAYING TURBULENCE

  • COMPDECAY-3D models periodic, transonic compressible turbulence with Athena++, using an adiabatic index of 5/3 and varied compressive-to-solenoidal initial components.Initial phases vary across trajectories, and simulations use a 0.05-unit warm-up.Source: §B.1.3 · pp. 16–17

B.1.4 3D COMPRESSIBLE RADIATIVE COOLING MIXING LAYER DYNAMICS

The section describes Athena++ simulations of a compressible radiative-cooling mixing layer and details the learned simulator’s CNN architecture and boundary handling.

B.1.4 3D COMPRESSIBLE RADIATIVE COOLING MIXING LAYER DYNAMICS

  • The simulations use a Kelvin–Helmholtz mixing-layer setup with opposing flows, periodic x/y boundaries, fixed z boundaries, and randomized boundary perturbations.They include a 2.32-unit warmup and were run with Athena++ (Stone et al., 2020).Source: §B.1.4 · p. 17
  • The mixing-layer dataset has fixed boundaries and less representative data because turbulent behavior is localized near the layer and differs across regions.Source: §B.1.4 · p. 17

B.2 ADDITIONAL MODEL DETAILS

  • The Dil-ResNet uses a CNN encoder and decoder around a dilated CNN processor with residual skip connections.Source: §B.2 · p. 17
  • Dilated filters expand the spatial field covered by a kernel without changing parameter count or resolution.For example, a kernel-3 filter with dilation 2 spans five pixels.Source: §B.2 · p. 17
  • Padding preserves periodic boundaries and fixes boundary values for fixed-boundary dimensions, while masking their loss and marking boundary states in the input.Source: §B.2 · p. 17

B.3 ADDITIONAL TRAINING DETAILS

Training uses noisy inputs and single-step supervised updates, with limited hyperparameter search and validation-based model selection.

Training noise

  • Gaussian noise is added to the input in some training runs, also slightly modifying the target difference and its normalization variance.Source: §B.3 · pp. 17–18

Loss

  • Models are trained on sampled input-output pairs one model step apart, without rollout during training.Source: §B.3 · p. 18

Optimization

  • Training uses Adam for up to 10M steps, with learning-rate decay from 1e −4 to 1e −7 over the first 6M steps; convergence usually occurs around 5M steps.Training took up to a week on an NVIDIA V100 GPU.Source: §B.3 · p. 18

Hyper-parameter optimization

  • Hyperparameter search was not exhaustive, and better test-trajectory performance may be possible with other settings.Noise scale was scanned per domain, while depth and latent size received informal tuning; research decisions used validation sets, and COOLMIXLAYER-3D used validation-based early stopping.Source: §B.3 · p. 18

B.4 ADDITIONAL RESULTS

The additional results assess runtime, temporal and spatial generalization, stability, and turbulence statistics. They show useful but imperfect generalization, with training noise and dataset augmentation helping in specific settings.

Additional results

  • Three instances per model were trained with three initialization seeds, and 3D turbulence performance was summarized quantitatively using the energy field.Figures and videos for 3D environments show a slice at y = 0, with x and z as the horizontal and vertical axes; spatial-resolution comparisons use a common 323 grid.Source: §B.4 · p. 18

Downsampling in time

  • Models trained and operate successfully across a wide range of time-step sizes in KS-1D, INCOMP-2D, and COMPDECAY-3D.The reported fair comparisons average performance only over time steps predicted by every model.Source: §B.4 · p. 18

Running time

  • ∼4s, ∼60s, and ∼1000s are Athena++ CPU runtimes for 1 simulation time unit at 323, 643, and 1283 resolutions, versus 1s for the learned model on an NVIDIA V100 GPU and 20-30s on an 8-core CPU.The learned model runs at reduced spatial and temporal resolution while preserving dynamics of the high-resolution 1283 Athena++ simulation.Source: §B.4 · p. 18

Cooling velocity

  • Augmenting COOLMIXLAYER-3D training data across box sizes improves cooling-velocity estimates at unseen sizes, but generalization outside the training domain remains imperfect and unreliable across seeds.A model trained only at box length 0.75 did not produce correct cooling velocities at other sizes; the authors speculate stronger inductive biases or dataset engineering may be needed.Source: §B.4 · p. 19

Other statistical metrics

  • The power spectral density measures how well models preserve information across spatial resolutions, with spectrum error computed over frequencies within a unit-radius sphere.The method averages Fourier magnitudes over frequency vectors of equal length to form a one-dimensional spectrum.Source: §B.4 · p. 19
  • The evaluation uses PDFs and joint PDFs alongside velocity autocorrelation and structure-function measures to assess statistical and spatial structure in turbulent flows.The paper identifies PDF shape and width as common metrics and presents autocorrelation errors over time.Source: §B.4 · pp. 19–20

KS-1D generalization

  • KS-1D learned simulations preserve qualitative turbulence features over rollouts to larger domains and longer trajectories, though they do not perfectly capture ground truth.Source: §B.4 · p. 20

Model comparison for generalization

  • Training noise keeps KS-1D net velocity and INCOMP-2D divergence errors bounded near zero, but does not fully prevent drift in COMPDECAY-3D conserved quantities.For longer rollouts, noise improves stability; adding a loss constraint improves generalization to more compressive initial conditions.Source: Figure B.5 · p. 24, Figure B.11 · p. 27

Related papers

Loading 2112.15275v3…