Source-linked AI summary
Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning
Yifei Guan, Ashesh Chattopadhyay, Adam Subel, Pedram Hassanzadeh
TL;DR
Data-driven SGS models can capture inter-scale transfers offline, but online instability and difficulty generalizing to higher-Re flows remain obstacles. This paper tests deep fully CNNs for 2D decaying turbulence and uses transfer learning to adapt them. The authors report accurate, stable LES-CNN results at higher Re, including up to 16× higher Re with a small amount of target-flow data.
Problem
Data-driven SGS models face unresolved challenges in online stability and generalization to higher-Re flows, despite the practical importance of extrapolation when DNS data are costly.
Source: §1 · p. 3Method
The paper uses a deep fully convolutional network for non-local SGS modeling and applies transfer learning by retraining part of the network with data from the new flow.
Source: §1 · pp. 3–4, §3.2 · pp. 8–9Results
Transfer learning enables accurate and stable LES-CNN solutions at higher Re, while models using 1% of target-flow training samples perform well at 8× or 16× higher Re.
Source: §5 · pp. 21–22Takeaways & Limitations
The results support transfer learning as a way to obtain accurate and stable LES-CNN solutions beyond the training flow, including at higher Re and grid resolution.
Source: §5 · pp. 21–22The methodology still needs study in more complex turbulent flows, including 3D, wall turbulence, and stratified flows.
Source: §5 · p. 22
Abstract
from arXiv · showhide
There is a growing interest in developing data-driven subgrid-scale (SGS) models for large-eddy simulation (LES) using machine learning (ML). In a priori (offline) tests, some recent studies have found ML-based data-driven SGS models that are trained on high-fidelity data (e.g., from direct numerical simulation, DNS) to outperform baseline physics-based models and accurately capture the inter-scale transfers, both forward (diffusion) and backscatter. While promising, instabilities in a posteriori (online) tests and inabilities to generalize to a different flow (e.g., with a higher Reynolds number, Re) remain as major obstacles in broadening the applications of such data-driven SGS models. For example, many of the same aforementioned studies have found instabilities that required often ad-hoc remedies to stabilize the LES at the expense of reducing accuracy. Here, using 2D decaying turbulence as the testbed, we show that deep fully convolutional neural networks (CNNs) can accurately predict the SGS forcing terms and the inter-scale transfers in a priori tests, and if trained with enough samples, lead to stable and accurate a posteriori LES-CNN. Further analysis attributes these instabilities to the disproportionally lower accuracy of the CNNs in capturing backscattering when the training set is small. We also show that transfer learning, which involves re-training the CNN with a small amount of data (e.g., 1%) from the new flow, enables accurate and stable a posteriori LES-CNN for flows with 16x higher Re (as well as higher grid resolution if needed). These results show the promise of CNNs with transfer learning to provide stable, accurate, and generalizable LES for practical use.
1 Introduction
This introduction frames stable, generalizable data-driven SGS modeling as a key challenge for LES. The paper uses a deep fully convolutional network for 2D decaying turbulence and evaluates offline accuracy, online performance, and transfer learning to higher Reynolds numbers and resolutions.
1 Introduction
- DNS is computationally prohibitive because it resolves all relevant scales, while LES reduces cost by resolving large structures and parameterizing subgrid effects.Source: §1 · p. 1
- SMAG provides numerically stable LES through diffusion but can miss local backscattering, whereas DSMAG can represent backscatter yet may cause numerical instability.Backscattering is transfer from subgrid scales to resolved scales; it is absent from purely diffusive SGS models.Source: §1 · p. 2
- Data-driven SGS models can perform well in a priori tests yet become unstable online, and proposed ad-hoc stabilization remedies can reduce their advantages.Reported remedies include removing backscattering, attenuating SGS feedback, or combining the data-driven model with eddy viscosity.Source: §1 · p. 3
- Generalization beyond the training flow matters because DNS data are costly, while robust and trustworthy LES requires models that work at higher Re.Prior studies found transfer learning enabled accurate generalization in simpler systems, but its effectiveness in more complex turbulent flows remained to be investigated.Source: §1 · p. 3
- A deep fully CNN is used for non-local SGS modeling in 2D-DHIT, with tests of offline forcing and transfer accuracy and online LES accuracy and stability.The authors also compare the offline and online tests with SMAG, DSMAG, and a local ANN-based SGS model.Source: §1 · p. 4
- Transfer learning is assessed for generalizing LES-CNN to flows with up to 16× higher Re, and an encoder-decoder architecture is used to generalize to higher grid resolutions.Source: §1 · p. 4
2 DNS and LES: Governing equations and numerical solvers
The section formulates 2D turbulence and its filtered LES equations, where unresolved effects enter through an SGS forcing term that must be estimated from resolved variables. It describes the DNS and lower-cost LES solvers and the DNS data-collection interval used for training.
2.1 Governing equations
- Filtering the vorticity–streamfunction equations introduces SGS forcing Π, which represents truncated small-scale effects and must be estimated from resolved (ω, ψ).The derivation assumes filtering commutes with spatial and temporal derivatives; the Gaussian filter is used in this work.Source: §2.1 · pp. 4–5
- Data-driven models learn the relationship between (ω, ψ) and Π without a prescribed functional form, unlike physics-based models that treat Π as purely diffusive.Source: §2.1 · p. 5
2.2 Numerical solvers
- DNS uses a Fourier pseudo-spectral solver on grids of 2048² for Re = 8000–64000 and 3072² for Re = 128000, with 15 independent runs per Reynolds number.The solver uses second-order Adams–Bashforth and Crank–Nicolson time integration; the implementation uses CUDA and double precision.Source: §2.2 · p. 5
- LES uses one-eighth the DNS resolution in each direction and a timestep 10 times larger, requiring 640 times fewer degrees of freedom while still needing an SGS model.The tested models are CNN, ANN, SMAG, and DSMAG.Source: §2.2 · p. 5
- Training data are collected from t = 50τ to t = 200τ, when the TKE spectra exhibit self-similarity following k−3 scaling.The figure illustrates this interval for Re = 32000.Source: Figure 1 · p. 6
3 Data-driven and physics-based SGS models for LES
The section constructs filtered DNS training data and describes data-driven CNN and ANN approaches alongside the SMAG and DSMAG physics-based SGS models. The CNN maps resolved-flow inputs to the SGS forcing term, while the physics-based models are diffusive.
3.1 Filtered DNS (FDNS) data
- DNS fields are Gaussian-filtered and then coarse-grained onto the LES grid; the filter size is set to 2ΔLES.The Gaussian filter is selected because it is compact in physical and spectral spaces and suits the solver and model representations.Source: §3.1 · p. 7
3.2 Fully convolutional neural network (CNN)
- The fully convolutional CNN learns a map from global, normalized ψ and ω inputs to Π, using same-size layers without pooling or upsampling.The architecture has 10 hidden layers; the authors identify the fully convolutional design as key to training an accurate SGS model.Source: §3.2 · pp. 8–9, Figure 3 · p. 9
- Training, validation, and testing data come from 15 independent DNS runs, split into 8, 2, and 5 runs, respectively.Snapshots are sampled every 10ΔDNS from the interval [50τ, 200τ].Source: §3.2 · p. 9
3.3 Multilayer perceptron artificial neural network (ANN)
- The ANN is a local, fully connected model with two 50-neuron hidden layers, trained on eight snapshots yielding 524288 samples.The authors report no substantial training-loss improvement from using 20 snapshots and did not conduct an exhaustive ANN–CNN comparison.Source: §3.3 · p. 10
3.4 Smagorinsky (SMAG) and dynamics Smagorinsky (DSMAG) SGS models
- SMAG uses a constant coefficient, while DSMAG dynamically estimates eddy viscosity; both models enforce nonnegative viscosity and are purely diffusive.DSMAG uses positive clipping to enforce νe ≥ 0.Source: §3.4 · pp. 10–11
4 Results
In 2D decaying turbulence, CNN-based SGS models accurately capture forcing and inter-scale transfers, but stable online LES requires sufficient training data. Transfer learning extends accurate LES to higher Reynolds numbers and grid resolutions.
4.1.1 Accuracy
- In a priori tests, CNN predicts SGS forcing and both forward and backward inter-scale transfers accurately, outperforming DSMAG and achieving higher accuracy than ANN.The reported CNN correlation coefficient exceeds 0.9.Source: §4.1.1 · pp. 11–12, Figure 4 · p. 12
4.1.2 Scaling of the CNN’s accuracy with size of the training set ntr
- Stable, accurate a posteriori LES-CNN requires ntr ≥ 30000; at smaller training sizes, backscatter accuracy lags forward-transfer accuracy, especially for small datasets.The apparent accuracy threshold is case-dependent, with no established a priori metric that guarantees stable online LES.Source: §4.1.2 · pp. 12–13
4.2 A posteriori analysis
- LES-CNN is stable and most accurate in short-term forecasts and long-term statistics, while backscatter removal makes it excessively diffusive.It best matches FDNS kinetic-energy spectra and vorticity-PDF tails; LES-ANN without post-processing blows up.Source: §4.2 · pp. 14, 15, 17
4.3 Transfer learning to higher Re
- At 4× and 8× higher Re, transfer-learned LES matches the accuracy of models trained on the target Re, while the unadapted Re = 8000 model deviates from FDNS.The transfer-learned model is retrained with ntr/100 target-flow samples.Source: §4.3 · pp. 18–19
4.4 Transfer learning to higher Re and higher LES numerical resolution
- 16× higher Re and 512 × 512 LES resolution are reached with transfer learning from 256 × 256 training data, using 1% as many samples from the target flow.The encoder-decoder architecture enables transfer between LES resolutions.Source: §4.4 · pp. 19–20
5 Summary and future directions
The CNN-based SGS model captures forward transfer and backscattering, and with sufficient training data yields stable, accurate LES; transfer learning extends its performance to higher-Re flows and resolutions. The conclusions are demonstrated in canonical 2D turbulence, with more complex flows left for future study.
5 Summary and future directions
- The study tests a non-local CNN-based SGS model on filtered DNS data in a priori and a posteriori analyses, including transfer learning from low to higher Re and resolution.The authors describe transfer learning as practically useful because it requires only a small amount of higher-Re data and re-training of a few layers; transfer between LES resolutions further broadens applicability.Source: §5 · pp. 20, 22
- At Re = 32000, the data-driven SGS model predicts unseen SGS forcing with c around 0.93 and captures both forward transfer and backscattering.The a priori model substantially outperforms DSMAG in correlation coefficient c.Source: §5 · pp. 20–21
- With 500 samples (1% of ntr) from the target flow, transfer-learned CNNs perform well at 8× or 16× higher Re; one example reaches Re = 128000 and NLES = 512.The example re-trains 2 of 10 convolution layers; the 500 samples come from Re = 128000.Source: §5 · pp. 21–22
- Increasing ntr from 10000 to 30000 and 50000 raises c from 0.90 to 0.92 and 0.93, respectively, and yields stable, accurate LES without post-processing or added eddy viscosity.At low ntr, backscattering accuracy is disproportionately lower than forward-transfer accuracy; the authors suggest this may contribute to instability, while cautioning that it may not explain all online instabilities.Source: §5 · p. 21
- With ntr = 50000 at Re = 32000, LES-CNN is stable and outperforms the tested alternatives on short-term forecasts, TKE spectra, and vorticity PDFs, including their tails.The authors attribute other models’ shortcomings primarily to excessive diffusivity and failure to capture backscattering.Source: §5 · p. 22
- The authors call for future study of CNN-based SGS models and transfer learning in more complex turbulent flows, including 3D, wall turbulence, and stratified turbulence.They also identify online training, accounting for non-Markovian effects, and interpreting accurate CNNs as directions for future work.Source: §5 · pp. 22–23
A Initial condition for DNS
The DNS initial conditions use randomly different vorticity fields with the same energy spectrum. The study focuses on the decaying phase after discarding an initial spin-up period.
A Initial condition for DNS
- The DNS initial conditions use randomly different vorticity fields while keeping the same energy spectrum.The paper follows previous studies in choosing these initial conditions.Source: A Initial condition for DNS · p. 23
- The prescribed initial spectrum peaks at 2kp, with kp = 10, and determines the magnitude of the Fourier coefficients of vorticity..Source: A Initial condition for DNS · p. 23
- Independent random phases η1 and η2 are drawn uniformly from [0, 2π] in the first quadrant, with specified sign changes in other quadrants.The initial vorticity field is applied at t = 0.Source: A Initial condition for DNS · p. 24
- The flow evolves from large-scale-dominated vorticity toward smaller-scale structures, then decays as small scales fade and coherent vortices merge and grow.The study focuses on the decaying phase from t = 50τ to 200τ and discards the first 50τ as spin-up.Source: A Initial condition for DNS · p. 24