PaperlayerView exact revision on arXiv

Source-linked AI summary

Machine learning accelerated computational fluid dynamics

Dmitrii Kochkov, Jamie A. Smith, Ayya Alieva, Qing Wang, Michael P. Brenner, Stephan Hoyer

arXiv:2102.01010v1physics.flu-dyncs.LG

TL;DR

Large-scale fluid simulation is limited by the cost of resolving the smallest spatiotemporal features. This paper embeds deep-learning approximations inside standard CFD solvers, achieving comparable accuracy on much coarser grids with substantial speedups and generalization beyond training conditions.

  • Problem

    Resolving the smallest spatiotemporal scales makes accurate numerical simulation of important fluid problems computationally expensive.

  • Method

    The method learns local interpolation and solver components within standard numerical discretizations, using differentiable programming to train the solver end to end.

  • Results

    The method matches advanced solvers at 8–10× finer resolution while running 40–80× faster, across DNS and LES applications.

  • Takeaways & Limitations

    Embedding machine learning within physics-based numerical solvers expands the efficiency–accuracy trade-off while supporting generalization to different forcings and Reynolds numbers.

Abstract

from arXiv · show

Numerical simulation of fluids plays an essential role in modeling many physical phenomena, such as weather, climate, aerodynamics and plasma physics. Fluids are well described by the Navier-Stokes equations, but solving these equations at scale remains daunting, limited by the computational cost of resolving the smallest spatiotemporal features. This leads to unfavorable trade-offs between accuracy and tractability. Here we use end-to-end deep learning to improve approximations inside computational fluid dynamics for modeling two-dimensional turbulent flows. For both direct numerical simulation of turbulence and large eddy simulation, our results are as accurate as baseline solvers with 8-10x finer resolution in each spatial dimension, resulting in 40-80x fold computational speedups. Our method remains stable during long simulations, and generalizes to forcing functions and Reynolds numbers outside of the flows where it is trained, in contrast to black box machine learning approaches. Our approach exemplifies how scientific computing can leverage machine learning and hardware accelerators to improve simulations without sacrificing accuracy or generalization.

I. INTRODUCTION

The paper introduces machine-learned components inside standard CFD solvers to retain pointwise accuracy on much coarser grids. For two-dimensional turbulent flows, this approach improves computational efficiency while targeting generalization beyond training conditions.

  • I. INTRODUCTION: Accurate simulations require resolving the smallest spatiotemporal scales, making direct numerical simulation impractical for many important physical problems.Turbulent flows are especially demanding because their smallest eddies can be extremely small, forcing fine meshes for convergence.
  • I. INTRODUCTION: LES permits coarser grids but sacrifices accuracy and depends strongly on sub-grid-scale models, especially when rate-controlling scales remain unresolved.The passage identifies SGS modeling as a severe limitation of accurate large-scale simulation.
  • I. INTRODUCTION: The method replaces resolution-sensitive solver components with learned alternatives that interpolate local solution features on coarse meshes.It uses data-driven discretizations within a standard numerical solver rather than replacing the entire simulation with a black-box model.
  • I. INTRODUCTION: 10× coarser resolution in each dimension yields approximately 80-fold faster two-dimensional direct numerical simulation at comparable accuracy.The comparison is made against an advanced numerical method of similar accuracy.
  • I. INTRODUCTION: Unlike pure machine-learning replacements, the solver retains underlying numerical structure to improve stability and generalization beyond training data.Pure ML approaches often struggle with physical constraints and novel forcing terms, whereas the proposed approach works within a standard solver.

A. Navier-Stokes

The paper frames incompressible fluid simulation through the Navier–Stokes equations and accelerates a standard finite-volume solver by learning local approximations within its convection calculations.

  • A. Navier-Stokes: The Navier–Stokes equations describe incompressible fluids using velocity, forcing, pressure, convection, and diffusion terms.Pressure acts as a Lagrange multiplier enforcing incompressibility, while Reynolds number controls the convection–diffusion balance.
  • A. Navier-Stokes: Higher Reynolds numbers produce more convection-dominated and generally harder-to-model flows, with turbulence corresponding to Re ≫ 1.The Reynolds number characterizes the balance between convection and diffusion.
  • A. Navier-Stokes: DNS solves the Navier–Stokes equations directly, whereas LES solves a spatially filtered version with an additional sub-grid stress term.LES therefore requires a closure model for the unmodeled stress as a function of filtered velocity.
  • B. Learned solvers: The learned solver improves a standard finite-volume method on a coarse staggered mesh using differentiable programming and automatic differentiation.The implementation uses explicit treatment for convection, diffusion, and forcing, with implicit pressure treatment.
  • B. Learned solvers: A convolutional network generates local latent features that solver sub-components use to account for local solution structure while preserving physics-informed numerical biases.The approach includes learned interpolation and learned correction components focused on convection.

1. Learned interpolation (LI)

The method improves coarse-grid CFD by learning local interpolation or correction components within a standard numerical solver. Learned interpolation preserves polynomial accuracy and physical equation structure, while training is performed end to end against high-resolution trajectories.

  • Learned interpolation: Learned interpolation generates local, flow-dependent coefficients for convective fluxes instead of using fixed polynomial interpolation.The approach targets the face interpolation required by finite-volume discretization.
  • Learned interpolation: The interpolation preserves equation symmetries and scaling properties while retaining polynomial accuracy as mesh spacing vanishes.These constraints connect the learned method to the regime where traditional numerical methods are accurate.
  • Learned correction: Learned correction models add a neural residual to the coarse-grid solver and can account for both closure effects and discretization error.The residual is small over a single time step, making it suitable for temporally discretized correction.
  • Learned correction: Restricting learned correction models to classical closure forms, such as effective tensor viscosity, reduced performance and stability.
  • Training: Training minimizes cumulative point-wise velocity error by differentiating through the numerical solver against coarsened high-resolution trajectories.Including the solver in the loss makes training model consistent because the model receives its own outputs as inputs.

III. RESULTS

The learned solver maintains accurate turbulent-flow predictions on coarse grids and reproduces both pointwise vorticity behavior and spectral energy distributions. Evaluations include forced and decaying turbulence, with comparisons against high-resolution DNS and coarse-grid baselines.

  • Decaying turbulence: Decaying turbulence is evaluated through vorticity evolution, correlation with the reference solution, and energy spectra after the flow develops larger structures.The test starts from a random high-wavenumber initial condition and removes forcing during evolution.
  • Forced turbulence: ∼10× finer resolution is matched by learned interpolation: a 64 × 64 model reaches the pointwise accuracy of 2048 × 2048 DNS more closely than coarse DNS baselines.The comparison uses an initial condition excluded from training, and correlation loss after several Lyapunov times is expected for chaotic turbulence.
  • Forced turbulence: Learned interpolation accurately captures the energy distribution across the spectrum, whereas lower-resolution DNS loses high-frequency features.The spectrum comparison is performed after 10^4 time steps.

2. Computational efficiency

The learned solver converts coarse-grid accuracy into computational efficiency and generalizes across domain sizes, flow decay, and Reynolds numbers. Local operators support reuse beyond the training settings, while hardware acceleration improves throughput.

  • Computational efficiency: 12.5x higher floating-point throughput is achieved by the neural solver than by the baseline CFD solver on sufficiently large grids.Despite 150 times more arithmetic operations, the ML solver is only about 12 times slower at the same resolution.
  • Computational efficiency: 10× effective resolution gains across two spatial dimensions and time correspond to a substantial speedup on the TPU.The time-dimension gain follows from the Courant condition.
  • Generalization: Essentially identical performance is retained on larger domains with the same forcing because the learned operators depend only on local flow features.
  • Generalization: Decaying turbulence is handled by a model trained on forced Kolmogorov flow as the eddy scale and Reynolds number change over time.
  • Generalization: ∼7× finer resolution is matched at Re = 4000 after rescaling a model trained at Re = 1000 without retraining.The scaling halves the grid spacing to match the smaller eddies at the higher Reynolds number.

B. Comparison to other ML models

Learned interpolation performs best overall among the compared ML approaches, while learned correction remains competitive. Both methods show more consistent accuracy, stability, and generalization than black-box alternatives.

  • Learned interpolation performs best overall, with learned correction close behind across the considered configurations.Pure black-box methods occasionally outperform the others, but not consistently.
  • 8× versus 10× effective resolution for pointwise accuracy corresponds to roughly a factor-of-two runtime difference.
  • Learned interpolation and correction generalize consistently to other flows and show narrow performance spreads across random initializations.Their performance consistently dominates other models in the generalization tests.
  • 4× effective coarse-graining still produces a 5× computational speed-up.
  • Black-box ML methods are highly sensitive to random initialization and show less consistent statistical accuracy and stability.

IV. DISCUSSION

The method expands efficient CFD by combining learned discretizations with conventional numerical structure. It matches finer-resolution accuracy while reducing computational cost, with broader speedup prospects suggested for three-dimensional simulations.

  • 8–10× finer-resolution solver accuracy is achieved while computation runs 40–80× faster.The method learns local operators within traditional numerical discretizations.
  • Learned discretizations retain scaling and symmetry properties of the governing Navier–Stokes equations.These properties support generalization to different forcings and Reynolds numbers.
  • Figure 5 compares accuracy, stability, and generalization against baseline methods across trained and out-of-distribution flows.The tests include larger domains, decaying turbulence, and more turbulent flows.
  • 10× lower effective resolution is expected to yield 10^3–10^4 speed-ups for three-dimensional simulations.This is an expectation based on the stated scaling argument, not a demonstrated 3D result.
  • The approach expands the Pareto frontier by enabling faster expensive simulations or higher accuracy without additional costs.
  • Figure 6 evaluates LES through vorticity evolution, vorticity correlation, and a scaled energy spectrum.The spectrum is averaged after solutions have decorrelated from the reference solution.

Appendix A: Direct numerical simulation

The appendix describes a finite-volume CFD solver using a staggered-square mesh, explicit time integration, pressure projection, and explicit subgrid modeling for LES. Dataset generation and solver comparisons vary grid resolution while keeping corresponding simulation parameters.

  • The solver uses a staggered-square mesh in a finite-volume formulation, with real-space equations chosen for boundary-condition and geometry versatility.
  • Explicit Euler time integration advances finite-difference convection and diffusion operators, including a Van-Leer flux-limiter convection scheme.Diffusion uses a second-order central-difference Laplace approximation.
  • Pressure is handled by a projection method that solves a corresponding Poisson equation using fast diagonalization or FFT.The implementation choice depends on simulation-domain size and accelerator utilization.
  • LES baseline and ground-truth simulations explicitly include a Smagorinsky–Lilly subgrid-scale model with grid spacing ∆ and Cs = 0.2.
  • Dataset generation consists of burn-in, high-resolution simulation, and downsampling to a lower grid for training and evaluation.
  • Model comparisons retain corresponding simulation parameters while changing only the underlying grid resolution.Solver components are selected for efficiency at each resolution.

Appendix C: Learned interpolations

The learned interpolation constructs local interpolations with neural-network-generated coefficients rather than fixed polynomial coefficients. A constraint on the coefficient sum preserves first-order accuracy.

  • Interpolation first computes a local stencil for the target position and then forms a weighted sum of stencil values.
  • Neural-network outputs determine interpolation coefficients instead of using a fixed coefficient set.
  • The coefficients satisfy Σ_i a_i = 1, guaranteeing that the interpolation is at least first-order accurate.An affine transformation maps unconstrained network outputs into the valid coefficient space.

Appendix D: Neural network architectures

The paper uses fully convolutional neural-network components across pure ML, learned interpolation, and learned correction approaches, with architecture choices differing between physics-augmented and pure ML solvers. Additional error metrics and enlarged-domain tests support the reported accuracy conclusions.

  • Modeling approaches: Three modeling approaches—pure ML, learned interpolation, and learned correction—are built from fully convolutional neural-network components.The architectures are summarized in Fig. A1, alongside fixed rescaling and repeated components.
  • Physics-augmented solvers: Physics-augmented solvers use a basic ConvNet, with Nout = 120 for learned interpolation and Nout = 2 for learned correction.The 120 outputs represent 8 interpolations requiring 15 inputs each.
  • Architecture selection: Larger neural networks slightly improved accuracy but did not justify their increased computational cost.This finding guided the architecture choices for the physics-augmented models.
  • Pure ML solvers: Pure ML solvers use Encoder-Process-Decoder and ResNet models to predict physical-process acceleration before advancing the state with forcing.Both architectures use CNN blocks with skip-connections; they differ in hidden-state design.
  • Validation: Squared and absolute errors reached the same conclusions as correlation-based accuracy measurements for learned discretizations versus DNS at Reynolds number 1000.The comparison used Kolmogorov flow.
  • Validation: On 2× larger domains with fixed flow complexity, the models produced the same improvement as on smaller domains.The paper presents this as a generalization test enabled by model locality.

Appendix F: Details of overview figure

The overview comparisons combine extrapolated large-domain accuracy, retrained models at different coarse-graining resolutions, and metric-based comparisons between learned interpolation and DNS. Training details include Adam optimization and hyperparameter sweeps.

  • Overview figure: The Pareto frontier extrapolates model accuracy to an 8× larger domain, while correlation timing uses the 1× domain because the largest grid was computationally infeasible.Performance on the 1× domain matched the 2× domain for this purpose.
  • Overview figure: The 512×512 learned-interpolation model matches Fig. 2, while 256×256 and 1024×1024 models were retrained using the same training dataset.These models represent 2× coarser and 2× finer coarse-graining, respectively.
  • Training details: All models were trained with Adam using learning rate 10^-3, b1 = 0.9, and b2 = 0.99.The study also swept training-trajectory length, model capacity, and random initialization.
  • Accuracy metrics: Figure A2 compares learned discretizations with DNS using vorticity correlation, mean absolute error, and absolute kinetic-energy error.The comparison is for Kolmogorov flow at Reynolds number 1000.
Loading 2102.01010v1…