Source-linked AI summary
Deep reinforcement learning for turbulent drag reduction in channel flows
L. Guastoni, J. Rabault, P. Schlatter, H. Azizpour, R. Vinuesa
TL;DR
Turbulent flow control is difficult because turbulence is nonlinear and multiscale, while accurate simulations are costly. The paper builds an RL environment around channel-flow DNS and compares DDPG with opposition control. In the minimal channel, DDPG achieves 43% drag reduction versus 26% for opposition control.
Problem
Turbulence is chaotic, multiscale, nonlinear, and expensive to simulate accurately, making effective control strategies difficult to find.
Source: §1 · p. 2Method
The authors couple channel-flow DNS to a configurable multi-agent RL environment in which agents use local flow observations to control wall-normal velocity.
Source: §1 · p. 2Results
43% drag reduction with DDPG versus 26% with opposition control in the minimal channel.
Source: §4.2 · p. 10Takeaways & Limitations
The environment supports customizable flow cases and control inputs for testing learned policies in fully turbulent simulations.
Source: §1 · p. 2, §5 · p. 12The study focuses on open-channel flow at uncontrolled Reτ = 180, and its policies do not re-laminarize the flow.
Source: §5 · p. 13
Abstract
from arXiv · showhide
We introduce a reinforcement learning (RL) environment to design and benchmark control strategies aimed at reducing drag in turbulent fluid flows enclosed in a channel. The environment provides a framework for computationally-efficient, parallelized, high-fidelity fluid simulations, ready to interface with established RL agent programming interfaces. This allows for both testing existing deep reinforcement learning (DRL) algorithms against a challenging task, and advancing our knowledge of a complex, turbulent physical system that has been a major topic of research for over two centuries, and remains, even today, the subject of many unanswered questions. The control is applied in the form of blowing and suction at the wall, while the observable state is configurable, allowing to choose different variables such as velocity and pressure, in different locations of the domain. Given the complex nonlinear nature of turbulent flows, the control strategies proposed so far in the literature are physically grounded, but too simple. DRL, by contrast, enables leveraging the high-dimensional data that can be sampled from flow simulations to design advanced control strategies. In an effort to establish a benchmark for testing data-driven control strategies, we compare opposition control, a state-of-the-art turbulence-control strategy from the literature, and a commonly-used DRL algorithm, deep deterministic policy gradient. Our results show that DRL leads to 43% and 46% drag reduction in a minimal and a larger channel (at a friction Reynolds number of 180), respectively, outperforming the classical opposition control by around 20 percentage points.
1 Introduction
Turbulence is important across natural and engineering settings, yet its chaotic, multiscale, nonlinear nature and costly simulation make effective flow control difficult. This work introduces a channel-flow RL environment based on direct numerical simulation to develop control policies without relying on modeled flow scales.
1 Introduction
- Turbulence is relevant to economic and environmental concerns, but its chaotic, multiscale, nonlinear dynamics and costly accurate simulations impede effective control.Depending on the goal, control may promote turbulence for mixing or suppress it to reduce drag.Source: §1 · p. 2
- The authors introduce a channel-flow RL environment using SIMSON direct numerical simulations, which resolve all time and length scales without approximation.The channel setup is more computationally efficient than geometries such as wings or turbine blades while capturing near-wall flow features.Source: §1 · p. 2
- Policies designed without exploiting limitations from modeled flow scales may underperform when applied to more realistic flows.The authors identify this as a reason to use fully resolved simulations when designing control strategies.Source: §1 · p. 3
2 Related work
Prior fluid-mechanics research applies deep learning to simulation, turbulence models, and reduced-order models, while DRL work spans flow modeling and active control. This paper extends active control to three-dimensional, wall-bounded turbulent channel flows, whose multiscale dynamics make them a challenging benchmark.
2.1 Deep learning in fluid mechanics
- Deep learning in fluid mechanics has been used to accelerate DNS, improve turbulence models, and construct reduced-order models; flow-field reconstruction and coarse-field super-resolution are other applications.Convolutional networks have been used for reconstruction and resolution enhancement, though GANs have been reported as more effective for the latter task.Source: §2.1 · p. 3
2.2 Reinforcement learning in fluid mechanics
- Fluid-mechanics DRL work includes turbulence modeling and active flow control, with applications ranging from agent movement in flows to control of flow dynamics.Other tasks include falling-fluid instabilities, convection, turbulent combustion, and engineering cases.Source: §2.2 · p. 4
- Wall-bounded channel flow extends DRL active control to a three-dimensional, multiscale setting with no obvious frequency or mechanism to target for drag reduction.Compared with earlier two-dimensional cylinder cases, it is a more nonlinear, chaotic, challenging benchmark and closer to realistic full-scale configurations.Source: §2.2 · p. 4
- The paper distinguishes its open-channel case from prior channel and Couette studies, including work that trained in a reduced-order model before applying control to the actual case.The authors note that benchmark cases can provide a starting point for the active-flow-control community.Source: §2.2 · pp. 4–5
2.3 Drag reduction in channel flows
- Traditional channel-flow control rescales wall-normal or spanwise velocity fluctuations sampled above the wall and applies them with opposite sign to suppress near-wall structures.Its drag reduction depends on the sensing-plane height; similar reductions have been reported for turbulent boundary layers.Source: §2.3 · p. 5
3 Methodology
The method couples SIMSON DNS of open-channel flow to a configurable, multi-agent RL environment. Agents use local observations and a shared wall-shear-stress reward to learn wall-normal actuation, while an MPI interface connects the Python learning process to the FORTRAN solver.
3.1 Simulation setup
- The simulations use an open channel with a no-slip lower wall and symmetry at the upper boundary, isolating near-wall dynamics from the opposite wall.Two domain sizes are used: a minimal channel that captures relevant near-wall statistics cheaply but permits only one low-speed streak, and a larger channel that captures interactions among several streaks.Source: §3.1 · p. 5
- Both domains use Reτ = 180 and SIMSON pseudo-spectral DNS; the minimal and larger domains have resolutions of 16 × 65 × 16 and 64 × 65 × 64, respectively.The solver uses Chebyshev modes wall-normal to the flow, with Crank–Nicholson integration for linear terms and Runge–Kutta for nonlinear terms.Source: §3.1 · p. 6
3.2 Reinforcement-learning environment and algorithm
- Cooperating agents share a translationally invariant local policy that sets wall-normal velocity at the lower wall, with actuation updated linearly between evaluations.Each agent observes sampled quantities in wall-parallel planes above its actuation; the sampled variables and heights are configurable.Source: §3.2 · pp. 6–7
- All agents receive a shared reward: wall-shear-stress reduction relative to uncontrolled flow, averaged across the wall and between actions.Reward averaging is intended to provide a more reliable estimate than an instantaneous value in mult frequency turbulence; the theoretical re-laminarization upper bound at this Reynolds number is 73.9%, with no guarantee it is attainable.Source: §3.2 · p. 7
3.3 Solver-DRL interface
- DDPG learns a deterministic policy from state-action-reward replay, with Gaussian exploration noise during training.The solver and Python learning process communicate through an MPI intercommunicator; STATE requests observations, CNTRL passes actions, and EVOLVE advances the simulation and supplies wall-shear stress for reward calculation.Source: §3.2 · p. 7, §3.3 · pp. 7–8
4 Experiments
The experiments validate the opposition-control implementation against prior results, then compare DRL with opposition control in minimal and larger channels. The minimal-channel policy also transfers to a larger domain without further tuning.
4.1 Validation and baseline
- The opposition-control implementation reproduced prior drag-reduction results, including 17.73% versus ≈14% and 25.67% versus ≈25%, supporting the simulation validation.The reported comparisons use different sensing planes and flow settings.Source: §4.1 · p. 9
4.2 Minimal channel learning and testing
- Minimal-channel learning used streamwise and wall-normal velocities at y+ = 15 for 400 episodes, and learning outcomes were sensitive to episode initialization.Each run used the same initial flow field across its episodes; some runs did not yield drag-reducing policies.Source: §4.2 · p. 9
- DRL altered minimal-channel velocity fluctuations more strongly than opposition control, reducing streamwise fluctuations and increasing wall-normal fluctuations while diminishing sweeps and ejections.Opposition control reduced fluctuation range but left the distribution shape and near-wall dominance of sweeps and ejections largely unchanged.Source: §4.2 · p. 10
- 43% versus 26% drag reduction: DDPG outperformed opposition control in the minimal channel after an initial drag increase.The testing was repeated across six initial conditions.Source: §4.2 · pp. 9–10, Figure 2 · p. 11
4.3 Larger channel testing
- 30% versus 20% drag reduction: the minimal-channel DDPG policy outperformed opposition control in the larger channel without further tuning.The advantage held across the tested initial conditions.Source: §4.3 · p. 11, Figure 3 · p. 12
- In the larger channel, DRL reduced sweeps and ejections but did not eliminate them as it did in the minimal channel.The paper suggests the policy may miss flow features, such as streak interactions, that are absent from the minimal-channel learning environment.Source: §4.3 · pp. 11–12
5 Conclusions
The study presents a configurable DNS-based reinforcement-learning environment and reports DDPG outperforming opposition control in two open-channel domains. The reported reductions exclude actuation energy costs and are limited to open-channel flow at Reτ = 180.
5 Conclusions
- 43% drag reduction in the minimal channel and 30% in the larger domain; improvements over opposition control are around 20 and 10 percentage points, respectively.The larger-domain reduction is smaller, but remains consistently better than the baseline.Source: §5 · p. 12
- The pseudo-spectral DNS environment supports customizable three-dimensional turbulent-flow cases and lets agents interact to maximize drag reduction.The number of input quantities and agents can be varied to adapt task difficulty.Source: §5 · p. 12
- Neither the learned policies nor the baseline accounts for actuation energy, so net energy savings are lower than the reported drag-reduction figures.The authors suggest incorporating actuation energy into the reward function in future work.Source: §5 · pp. 12–13
- The study covers open-channel flow at Reτ = 180; a policy effective at one Reynolds number may be less effective at higher Reynolds numbers.The authors note that different physical mechanisms can be responsible for drag at higher Reynolds numbers.Source: §5 · p. 13
Declarations
The declarations report the study's funding, conflict-of-interest statement, code and dataset availability, and author contributions.
Declarations
- The work reports support from SeRC, ERC grant 2021-CoG-101043998 DEEPCONTROL, and the Knut and Alice Wallenberg Foundation, with no conflicts of interest.The datasets and codes are included in the project repository.Source: Declarations · p. 13
- Luca Guastoni developed the DRL interface and performed learning runs and post-processing; Ricardo Vinuesa developed the project idea and co-wrote the paper.Jean Rabault, Philipp Schlatter, and Hossein Azizpour also provided input or feedback.Source: Declarations · pp. 13–14
Appendix A Effect of the episode length
Shorter training episodes did not show a significant drag-reduction disadvantage in the reported experiments, and may reduce learning time and computational cost.
Appendix A Effect of the episode length
- Shorter episodes did not provide significantly less drag reduction than the 3000-interaction episodes in the reported experiments.The shorter episodes tested used 1000 and 2000 interactions, compared with 3000 interactions (t+ ≈ 1500).Source: Appendix A · p. 14
- Learning appeared to improve faster at the beginning with longer episodes because they contained more interactions per episode.The initial episode length was chosen to include the transient (typically t+ ≈ 500) and a sufficiently long period afterward.Source: Appendix A · p. 14
- Shorter episodes may reduce total learning time and computational cost, which rise with agent count and resolution or Reynolds number, respectively.The passage presents this as a possibility rather than a measured cost reduction.Source: Appendix A · p. 14