PaperlayerView exact revision on arXiv

Source-linked AI summary

Ensemble Kalman method for learning turbulence models from indirect observation data

Xin-Lei Zhang, Heng Xiao, Xiaodong Luo, Guowei He

arXiv:2202.05122v4physics.flu-dyn

TL;DR

Data-driven turbulence models often require full-field Reynolds-stress data, while training through RANS can require adjoint solvers. This paper trains a neural-network turbulence model from indirect observations using an adaptive ensemble Kalman method, which learns a model that generalizes across periodic-hill flows with varying slopes.

  • Problem

    Data-driven turbulence models often require full-field Reynolds stresses, and training them through RANS typically requires adjoint solvers to obtain sensitivities.

    Source: §1 · p. 3

  • Method

    The authors train a neural-network nonlinear eddy-viscosity model from indirect observations using an iterative ensemble Kalman method with adaptive stepping.

    Source: §1 · p. 4, §2.2 · p. 8

  • Results

    The ensemble method learns the underlying closure from square-duct velocity data and generalizes from one periodic-hill flow to similar configurations with varying slopes.

    Source: §6 · p. 27

  • Takeaways & Limitations

    The non-intrusive ensemble framework can learn from different observation types without developing an adjoint solver for each objective function.

    Source: §1 · p. 4, §6 · p. 27

    Closely spaced training observations may impair performance because correlations in their observation errors are neglected.

    Source: §4.2 · p. 21

Abstract

from arXiv · show

In this work, we propose using an ensemble Kalman method to learn a nonlinear eddy viscosity model, represented as a tensor basis neural network, from velocity data. Data-driven turbulence models have emerged as a promising alternative to traditional models for providing closure mapping from the mean velocities to Reynolds stresses. Most data-driven models in this category need full-field Reynolds stress data for training, which not only places stringent demand on the data generation but also makes the trained model ill-conditioned and lacks robustness. This difficulty can be alleviated by incorporating the Reynolds-averaged Navier-Stokes (RANS) solver in the training process. However, this would necessitate developing adjoint solvers of the RANS model, which requires extra effort in code development and maintenance. Given this difficulty, we present an ensemble Kalman method with an adaptive step size to train a neural network-based turbulence model by using indirect observation data. To our knowledge, this is the first such attempt in turbulence modelling. The ensemble method is first verified on the flow in a square duct, where it correctly learns the underlying turbulence models from velocity data. Then, the generalizability of the learned model is evaluated on a family of separated flows over periodic hills. It is demonstrated that the turbulence model learned in one flow can predict flows in similar configurations with varying slopes.

1. Introduction

The introduction motivates more accurate, robust turbulence models while outlining the data and training challenges facing data-driven approaches. It presents ensemble-based learning from indirect observations as a non-intrusive, model-consistent alternative that avoids adjoint solvers.

1. Introduction

  • Traditional turbulence models trade predictive performance against practical usability, while more complex nonlinear models face robustness, cost, implementation, and generalization challenges.Industrial CFD is expected to continue relying on RANS solvers with turbulence models for decades, particularly in outer-loop applications.Source: §1 · pp. 1–2
  • Data-driven nonlinear eddy-viscosity models expand stress–strain mappings but remain local, often require full-field Reynolds stresses, and may generalize poorly from a priori training to RANS predictions.The cited discussion links this mismatch to the ill-conditioning of the RANS operator, which can amplify small Reynolds-stress errors in predicted velocities.Source: §1 · pp. 2–3
  • Model-consistent learning trains turbulence models against observations through the RANS solver, but neural-network training typically requires sensitivities and adjoint solvers.The adjoint-derived sensitivity is combined with the neural-network gradient by the chain rule.Source: §1 · p. 3
  • Earlier ensemble-based gradient approximation was less accurate than adjoint learning, while ensemble Kalman methods had not yet been applied to indirect-data turbulence-model learning.The introduction motivates ensemble Kalman training as using Jacobian and Hessian information rather than only explicit first-order gradient descent.Source: §1 · pp. 3–4
  • The proposed ensemble Kalman method with adaptive stepping outperformed the authors’ adjoint implementation in accuracy and robustness, including learning a generalizable periodic-hill model.The authors also report lower wall-time and implementation effort than the adjoint method.Source: §1 · p. 4
  • The framework uses measurable flow quantities instead of full-field Reynolds stresses, trains in the prediction environment, and avoids adjoints through non-intrusive implementation.It can use different objective quantities without additional code redevelopment.Source: §1 · p. 4

2. Reynolds stress representation and model-consistent training

The section embeds a tensor basis neural network in RANS and uses an adaptive-step ensemble Kalman method to train it from indirect observations. The method learns the tested model mappings and generalizes to periodic-hill flows with varying slopes.

2. Reynolds stress representation and model-consistent training

  • The workflow samples network weights, predicts Reynolds stresses, solves RANS for observable quantities, and updates weights by comparing predictions with observations.The observation operator can include post-processed quantities such as point velocities or drag.Source: §2 · p. 5

2.1. Embedded neural network for Reynolds stress representation

  • The tensor basis neural network represents Reynolds-stress anisotropy with scalar invariants and tensor bases, providing a more general algebraic model than linear eddy viscosity.The nonlinear model uses ten tensor bases formed from strain and rotation tensors; it requires no additional transport equation.Source: §2.1 · pp. 5–7

2.2. Ensemble-based model-consistent training

  • Adaptive stepping adjusts the observation-error covariance during iterative ensemble Kalman training, while the Kalman gain incorporates approximate Hessian and gradient information.The authors report that this combination accelerates convergence without substantial memory overhead.Source: §2.2 · p. 8

2.3. Comparison to other learning methods

  • The method is non-intrusive and derivative-free, and its ensemble of weights supports uncertainty quantification and training across flow configurations.The authors contrast its weight update, which includes approximate Hessian information, with the ensemble-gradient method.Source: §2.2 · pp. 8–9, §2.3 · p. 11
  • The proposed method learns the functional mappings in both cases and generalizes well to similar periodic-hill configurations with varying slopes.The comparison covers a square duct trained using Shih’s quadratic-model prediction and periodic hills trained using DNS results.Source: §2.3 · pp. 11–12, Table 2 · p. 12

3. Case setup

The study evaluates ensemble-based learning of neural-network nonlinear eddy-viscosity models from velocity data in a square duct and periodic-hill flows. The square-duct case uses a known model mapping as synthetic truth, while the periodic-hill case tests generalization across hill slopes.

3. Case setup

  • The cases assess learning against ground truth in the square duct and generalization in periodic hills, with comparisons to an adjoint-based method.The square-duct comparison targets training accuracy and efficiency.Source: §3 · p. 12, §3.1 · p. 13

3.1. Secondary flows in a square duct

  • The square-duct case uses Shih’s quadratic model as synthetic truth to test whether the method recovers the underlying mapping from scalar invariants to tensor-basis functions.That model can simulate secondary flows that a linear eddy-viscosity model cannot capture, and its explicit mapping provides a benchmark for function accuracy.Source: §3.1 · pp. 12–13
  • The square-duct setup uses two scalar-invariant inputs, four output g functions, 50 weight samples, and a two-hidden-layer network with five neurons per layer.The fully developed flow uses one streamwise cell, one-quarter of the domain by symmetry, and a 50 × 50 mesh.Source: §3.1 · p. 13

3.2. Separated flow over periodic hills

  • The periodic-hill model is trained at slope α = 1 and tested at α = 0.5, 0.8, 1.2, and 1.5, spanning different levels of flow separation.The separation extent decreases as α increases; the mesh has 149 streamwise by 99 normal cells after grid-independence tests.Source: §3.2 · p. 13
  • For periodic hills, the DNS data provide training observations, and the learned model is validated on similar geometries with varying slopes; the k–ε model is the baseline.The two-dimensional setup uses two scalar invariants and independent tensors; its input-feature magnitude ranges from 0 to about 100, motivating a 10-layer network with 10 neurons per layer and 50 weight samples.Source: §3.2 · pp. 13–14

4. Results

The ensemble method learns the duct closure from velocity data and improves periodic-hill velocity predictions, with transfer strongest to flows whose input features resemble the training case. Its performance is less consistent for Reynolds stresses and regions or cases poorly represented by the training data.

4.1. Flow in a square duct: learning underlying closure functions

  • In the duct, ensemble learning estimates velocity and Reynolds stresses similarly to adjoint learning, though adjoint velocity estimates are better; fewer observations increase errors, especially in velocity.The anti-diagonal-only observation case is reported in Appendix C.Source: §4.1 · p. 15, Figure 3 · p. 16, Figure 4 · p. 17
  • 0.47% velocity error and 5.8% Reynolds-stress error with the ensemble method, versus 0.1% and 4.5% with the adjoint method.Both methods achieve satisfactory velocity estimates and predict Reynolds stresses well; Table 4 compares estimation errors and time costs.Source: §4.1 · p. 15, Table 4 · p. 17
  • The duct model reconstructs scalar invariants and the θ-to-g mapping, but the mapping is less reliable at small θ1, where velocity is insensitive to g and cells are sparse.Only 30% of duct cells have θ1 below approximately 5.1.Source: §4.1 · p. 18, Figure 5 · p. 18, Figure 6 · p. 19

4.2. Flow over periodic hills: generalizability test to varying slopes

  • Training on four velocity profiles at x/H = 1, 3, 5, and 7 reconstructs the α = 1 hill flow and captures its separation bubble.The ensemble model also significantly improves bottom-wall friction estimates, with the reattachment point very close to the truth.Source: §4.2 · p. 19, Figure 7 · p. 20, Figure 8 · p. 21
  • Only 30% of periodic-hill cells have |θ1| > 5.0 and |θ2| > 3.8; training profiles away from separation leave large-invariant regions less improved.The authors suggest evenly spaced observations farther apart than one correlation length; for periodic hills, lc/H = 1.Source: §4.2 · pp. 20–21, Figure 9 · p. 22
  • For periodic hills, no ground-truth θ-to-g mapping is available; the ensemble mapping is nearly constant at about −0.098 for g(1) and 0.01 for g(2).The baseline mapping is g(1) = −0.09 and g(2) = 0.Source: §4.2 · p. 22, Figure 10 · p. 23
  • Both learned methods show substantial TKE discrepancies from DNS, and direct learning estimates Reynolds stresses in better agreement than ensemble learning.The authors note that velocity data cannot accurately reconstruct the divergence-free part of Reynolds stress, which does not affect velocity.Source: §4.2 · pp. 22–23, Figure 11 · p. 24
  • Ensemble-based learned models improve mean-flow velocity estimates over k–ε in all test cases; prediction error increases as extrapolation moves farther from training α = 1, with noticeable discrepancies at α = 1.5.For α = 1.2, the velocity profiles have a remarkable agreement with DNS data.Source: §4.2 · pp. 24–25

5. Discussion

The ensemble-based framework supports parallel training across flows and flexible learning from different observation data and loss functions. These properties make it applicable where adjoint solvers are difficult to develop.

5.1. Parallelization

  • Training cases from different configurations can run without communication, so wall time is not significantly increased when CPU cores match the number of configurations.The framework is also described as parallelizable for large data sets and able to handle random noise to avoid overfitting.Source: §5.1 · p. 25
  • For configurations with slopes α = 0.5, 0.8, 1.2, and 1.5, the tests compare velocity profiles and prediction errors across the field and recirculation region.The direct learned model is omitted at α = 0.5 because its solver diverges.Source: Figure 12 · p. 26

5.2. Flexibility in learning from different observation data

  • The derivative-free framework can use non-differentiable loss functions and disparate observations without requiring a separate adjoint solver for each objective.It approximates sensitivity from model inputs and outputs and can be applied to other physical systems where adjoint solvers are unavailable.Source: §5.2 · pp. 26–27

6. Conclusion

The paper proposes ensemble-based learning of nonlinear eddy viscosity models from indirect data, demonstrating learning from velocity observations and generalization across periodic-hill slopes. The method is non-intrusive and, in the tested steady flows and implementation, more efficient than the adjoint-based method.

6. Conclusion

  • The framework learns nonlinear eddy viscosity turbulence models from indirect data, without an adjoint solver, and addresses training–prediction inconsistency associated with direct data.The authors report improved generalizability and greater efficiency than the adjoint-based method for their implementation and steady-state test cases.Source: §6 · p. 27
  • The square-duct case demonstrates learning closure relationships from velocity observations, while periodic hills demonstrate generalization to similar configurations with varying slopes.The method runs in parallel and can learn from large sets of training flows simultaneously.Source: §6 · p. 27
  • The method’s scope is limited for large data sets such as unsteady three-dimensional flows by the computational cost of large matrix inversion.The framework also depends on observation-data position, uses a nonlinear eddy viscosity model under the weak equilibrium assumption, and requires further testing across flow classes.Source: §6 · p. 27

Appendix A. Practical implementation

The implementation samples neural-network weights, maps flow features through a tensor-basis model, and solves RANS to obtain predicted velocities. An iterative ensemble Kalman update adjusts the weights, with an inner loop adapting the update step length.

Appendix A. Practical implementation

  • The network is pre-trained to represent a linear eddy viscosity model before optimization, because random initialization may produce negative eddy viscosity and cause RANS divergence.The resulting weights initialize the optimization; subsequent ensemble members are sampled as i.i.d. Gaussian perturbations around them.Source: Appendix A · p. 28
  • Velocity and turbulence time-scale features produce scalar invariants and tensor bases; the invariants feed the network, whose coefficients combine with the bases to construct Reynolds stress.Training-case maximum and minimum invariant values scale test-case inputs, but global scaling can cluster features and cause convergence issues.Source: Appendix A · p. 28
  • The constructed Reynolds stress is propagated through the RANS equations to obtain velocity, while turbulence kinetic energy and dissipation rate come from their transport equations.Source: Appendix A · p. 28
  • The iterative ensemble Kalman method updates neural-network weights using a gain computed from the ensemble samples.The algorithm’s procedure is summarized given observation error, data, and sample variance.Source: Appendix A · pp. 28–29
  • An inner loop adjusts γ by inflating observation-error covariance, reducing the step parameter after misfit falls and increasing it otherwise, for up to five sub-iterations.Source: Appendix A · p. 29
  • The iteration stops when ensemble variance falls below observation error; otherwise, it continues until convergence or the iteration limit.Source: Appendix A · p. 29

Appendix B. Hessian matrix in the ensemble Kalman method

Appendix B relates the iterative ensemble Kalman weight update to the Kalman gain and explains how it implicitly incorporates approximations to the gradient and Hessian. For nonlinear observation operators, the ensemble provides a derivative-free approximation to the projected quantities.

Appendix B. Hessian matrix in the ensemble Kalman method

  • The update is connected to the common Kalman gain by expressing the model-error covariance and related quantities through a square-root matrix and its observation-space projection.The appendix establishes both forms of the gain using the cross-covariance between weights and predictions and the covariance projected into observation space.Source: Appendix B · pp. 29–30
  • For a nonlinear observation operator, the ensemble-estimated observation-space square root provides a derivative-free approximation to the projected square root matrix.The appendix identifies the gain as an ensemble-based derivative-free approximation to the inverse local Hessian times the negative local gradient.Source: Appendix B · p. 30
  • The ensemble Kalman weight update implicitly uses information from both approximated gradient and Hessian matrices.The appendix connects the update to the Kalman gain and to an ensemble-based approximation of the inverse local Hessian multiplied by the negative local gradient.Source: Appendix B · pp. 29–31

Appendix C. Sensitivity study of network architecture and observation data

The sensitivity study finds that training accuracy is not significantly affected by the tested network architectures in either the square-duct or periodic-hill cases. For the square duct, using velocity observations only along the anti-diagonal produces slightly inferior velocity and Reynolds-stress fields compared with full-field observations.

Square duct

  • Velocity and Reynolds-stress errors are similar across three architectures in the square-duct case, and the two-layer, five-neuron network predicts both fields well.The tested architectures are two hidden layers with 5 or 10 neurons per layer, and ten hidden layers with 10 neurons per layer; Table 7 summarizes the architecture and observation-data sensitivity.Source: Appendix C · p. 31
  • Anti-diagonal velocity observations yield slightly inferior velocity and Reynolds-stress fields than full-field observations in the square-duct case.The study selects the two-hidden-layer, five-neuron network for the square duct and the ten-hidden-layer, 10-neuron network for periodic hills.Source: Appendix C · p. 31

Periodic hill

  • Velocity and Reynolds-stress results do not vary significantly across the tested network architectures for the periodic-hill case.The study consequently uses the ten-hidden-layer, 10-neuron-per-layer baseline network for this case.Source: Appendix C · p. 31, Table 8 · p. 32

Appendix D. Implementation of direct learning method

Appendix D describes direct learning by fitting the tensor-basis neural network to DNS Reynolds-stress data, using input features from a baseline RANS prediction. It explains the regularization and optimization used to address ill-conditioning and fit the data.

Appendix D. Implementation of direct learning method

  • Direct learning uses the DNS deviatoric Reynolds-stress tensor as training data and scalar-invariant features from the baseline RANS prediction.The optimized model is a tensor-basis neural network.Source: Appendix D · p. 32
  • When tensor bases are small, coefficients can become extremely large and nonphysical, leading to negative eddy viscosity; regularization penalizes deviation from the zero prior.The regularization parameter λ is set to 10 to balance data fit against extreme coefficient values.Source: Appendix D · p. 32
  • The direct-learning least-squares objective is optimized using an autodifferentiated cost-function gradient and Adam updates.Source: Appendix D · p. 32
Loading 2202.05122v4…