Source-linked AI summary
Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning
Hongwei Tang, Jean Rabault, Alexander Kuhnle, Yan Wang, Tongguang Wang
TL;DR
The paper addresses the challenge of robust active flow control across varying Reynolds numbers. It uses PPO with an ANN to control four synthetic jets and smooth action changes, achieving drag reduction at trained Reynolds numbers and for previously unseen values between 60 and 400.
Problem
Robust and efficient AFC algorithms are needed for nonlinear, high-dimensional flows and imperfect or disturbed engineering environments.
Method
PPO with a two-layer fully connected ANN controls the mass flow rates of four synthetic jets, using linear interpolation to make actions vary smoothly.
Results
Drag reduction is approximately 5.7%, 21.6%, 32.7%, and 38.7% at Re = 100, 200, 300, and 400, respectively, with control also effective for unseen Re values between 60 and 400.
Takeaways & Limitations
The PPO-based ANN demonstrates robust and generalized active control across the tested Reynolds-number range and previously unseen intermediate conditions.
Abstract
from arXiv · showhide
This paper focuses on the active flow control of a computational fluid dynamics simulation over a range of Reynolds numbers using deep reinforcement learning (DRL). More precisely, the proximal policy optimization (PPO) method is used to control the mass flow rate of four synthetic jets symmetrically located on the upper and lower sides of a cylinder immersed in a two-dimensional flow domain. The learning environment supports four flow configurations with Reynolds numbers 100, 200, 300 and 400, respectively. A new smoothing interpolation function is proposed to help the PPO algorithm to learn to set continuous actions, which is of great importance to effectively suppress problematic jumps in lift and allow a better convergence for the training process. It is shown that the DRL controller is able to significantly reduce the lift and drag fluctuations and to actively reduce the drag by approximately 5.7%, 21.6%, 32.7%, and 38.7%, at $Re$=100, 200, 300, and 400 respectively. More importantly, it can also effectively reduce drag for any previously unseen value of the Reynolds number between 60 and 400. This highlights the generalization ability of deep neural networks and is an important milestone to active flow control.
I. INTRODUCTION
Active flow control offers industrial and environmental benefits, but robust strategies remain difficult for nonlinear, high-dimensional systems and uncertain actuator conditions. This work extends deep reinforcement learning for robust control across multiple flow conditions.
- AFC can reduce aerodynamic drag, manipulate bluff-body wakes, and optimize wind-turbine and gas-turbine performance.
- Designing efficient AFC strategies is difficult because Navier–Stokes dynamics are highly nonlinear and control spaces are high-dimensional.
- Engineering AFC must also adapt to environmental disturbances and actuator manufacturing or installation imperfections.
- Model-free, data-driven approaches are suitable for complex, high-dimensional, nonlinear systems.
- DRL discovers control solutions through trial-and-error even when no solution is known beforehand.
- The study extends prior work by using four symmetric synthetic jets and investigating robust DRL control across flow conditions.
A. Problem description
The simulation models viscous flow past a slightly off-center circular cylinder in a two-dimensional channel, with four synthetic jets providing active control around the cylinder.
- The benchmark domain contains a cylinder of diameter D in a 22D×4.1D two-dimensional channel.
- The cylinder is offset from the channel centerline by 0.05D, helping trigger vortex shedding above the critical Reynolds number.
- Four synthetic jets are symmetrically positioned on the cylinder’s upper and lower sides and controlled by ANN mass flow rates.
- The jets are located at 75°, 105°, 255° and 285°, with directions perpendicular to the cylinder wall.
- Synthetic-jet operation enforces zero total mass flow, while positive and negative injection velocities represent blowing and suction.
B. Numerical method
The numerical method solves nondimensional, incompressible two-dimensional Navier–Stokes flow in a cylinder benchmark domain with prescribed inflow, wall, outflow, and jet conditions. Mesh refinement and benchmark comparisons validate the simulation setup used for drag and lift analysis.
- Governing equations: The flow is modeled with time-dependent, incompressible Navier–Stokes equations and a continuity equation.
- Governing equations: The nondimensionalization uses D, U, ρ, and D/U, with Reynolds number defined as Re = UD/ν.
- Boundary conditions: The domain applies a specified inflow profile, no-slip conditions on walls and the cylinder, and a fully developed outflow condition.
- Boundary conditions: Four jet velocity profiles are prescribed radially, with mass flow rates Q_i, angular centers at 75°, 105°, 255° and 285°, and width ω = 10°.
- Discretization and solver: The IPCS finite-element method uses an unstructured mesh refined near the cylinder to resolve synthetic-jet effects.
- Validation: Mesh comparisons at Re = 100 agree with benchmark data, with discrepancies below 0.04% for listed quantities on the main mesh versus the fine mesh.
C. DRL control algorithm
The DRL controller learns closed-loop flow control from probe observations by selecting jet actions and optimizing a reward based on drag and lift. PPO training uses controlled action timing, smoothed mass-flow-rate changes, and bounded actuations to support physically stable learning.
- PPO couples a flow-simulation environment with an agent that observes 236 passive probes, selects jet actions, and receives reward feedback.The agent uses local flow quantities around the cylinder and wake as its state representation.
- The reward averages drag over T = 100δt and penalizes absolute average lift with β = 0.2.The lift penalty discourages a strategy involving persistent maximum-strength blowing in one direction.
- Actions are updated 200 times per episode and held constant for 100 numerical time steps between updates.The update interval balances responsiveness against allowing the flow enough time to exhibit a measurable effect.
- A smoothing interpolation makes instantaneous jet mass flow rates continuous at the numerical timestep, preventing non-physical jumps in pressure or velocity.The continuous control values support stable interaction between the PPO agent and the flow simulation.
- The normalized jet mass flow is constrained by |Q_i*| ≤ 0.05 to prevent non-physically large actuations and numerical divergence.Here Q_i is the mass flow rate of jet i and Q_ref is the reference flow rate intercepting the cylinder.
A. Active control for flow at higher Reynolds number
Higher Reynolds numbers make the cylinder wake more irregular, motivating tests of DRL control beyond the previously demonstrated Re = 100 case. The study trains controllers at Re = 200 and 400 and examines drag reduction against uncontrolled baselines.
- Reynolds number strongly influences flow complexity, wake irregularity, and the transition from laminar to turbulent cylinder flow.For this configuration, the wake becomes more irregular as Re increases.
- The mean drag coefficient contains steady symmetric and vortex-shedding contributions.The unsteady contribution is identified as the component that AFC can alter, providing an estimate of attainable drag reduction.
- The vortex-shedding contribution to drag increases with Reynolds number, motivating control tests at Re = 200 and Re = 400.These tests use trained ANNs and compare controlled drag coefficients with uncontrolled baseline flow.
- A controller trained across Reynolds numbers produces comparable average drag reduction while greatly suppressing drag oscillations relative to the Re = 400 comparison.The reported explanation includes four available jets instead of two and training across multiple flow conditions.
B. Effect of smoothing interpolation functions
The study compares interpolation schemes for converting discrete ANN actions into continuous control values. Linear interpolation between consecutive actions best suppresses lift oscillations while maintaining performance comparable to exponential decay.
- Direct interpolation between action updates generates continuous control values for the simulation time steps.The approach is simple to implement and supports policy training and action selection.
- The ANN action index j advances once per action-update interval, whereas the numerical-step index i advances every simulation time step.Action updates occur over a period T = 100δ.
- The exponential decay law reduces drag by approximately 8% at Re = 100 but leaves problematic jumps in lift.Its control values converge well, yet the controlled flow is not perfectly stable.
- Linear interpolation between two actions effectively eliminates lift-coefficient oscillations while matching the exponential law’s performance.The comparison identifies Eq. (20) as the best-performing smoothing law.
C. Training a model over a range of Reynolds numbers
A single PPO-trained ANN controls four synthetic jets across flow configurations at Re = 100, 200, 300, and 400. The global controller reduces drag and fluctuations, remains effective at unseen Reynolds numbers, and modifies the controlled wake dynamics.
- Training a model over a range of Reynolds numbers: A single ANN is trained to control flows at Re = 100, 200, 300, and 400 within one learning environment.The controller is designed to operate robustly across these distinct flow configurations.
- Training a model over a range of Reynolds numbers: 20.4% drag reduction is obtained at Re = 200, with large initial actuation followed by a pseudo-periodic low-actuation regime.The result comes from the PPO agent’s time-resolved control strategy.
- Training a model over a range of Reynolds numbers: 33.1% drag reduction is obtained at Re = 400, but actuation does not decrease substantially in the second phase because the higher-Re flow is inherently unstable.This behavior illustrates control of systems with pseudo-chaotic properties.
- Training a model over a range of Reynolds numbers: 5.7%, 21.6%, 32.7%, and 38.7% drag reductions are obtained at Re = 100, 200, 300, and 400, respectively.The global strategy reaches its typical drag-reduction level after a longer initial phase, with larger fluctuations especially at higher Re.
- Training a model over a range of Reynolds numbers: The global strategy greatly suppresses drag oscillations while achieving an average drag reduction close to the Re = 400 single-configuration strategy.The paper attributes this comparison to four-jet fine-grained control and greater variability during global training.
- Training a model over a range of Reynolds numbers: Active control increases the separated-wake area and expands vortex shedding, a flow morphology associated with reduced drag and lift fluctuations.FFT analysis also shows reduced fluctuation amplitudes and modified characteristic frequencies.
- Training a model over a range of Reynolds numbers: For Re ≥200, controlled drag is smaller than drag without vortex shedding, indicating proximity to the theoretical optimum.The unsteady contribution to drag becomes increasingly significant as Re increases.
- Training a model over a range of Reynolds numbers: The controller remains effective for previously unseen Reynolds numbers throughout 60-400 and beyond the training scope, including Re = 80.Only four reference Reynolds numbers were used during training, highlighting ANN generalization within the tested range.
IV. CONCLUSIONS
The study demonstrates robust, generalized active flow control across Reynolds numbers using PPO-trained neural networks, while identifying training variability and computational cost as remaining constraints.
- PPO-trained ANNs discover active control strategies across flows with multiple Reynolds numbers.
- Drag is also effectively reduced for previously unseen Reynolds numbers between 60 and 400.
- For Re ≥200, controlled drag is smaller than the symmetric-boundary-condition baseline, suggesting near-theoretical-optimum performance.
- Training randomness and exploration noise can produce slightly different control performance across runs, although qualitative strategies remain relatively similar.
- The selected problem is relatively simple and computational cost remains a challenge for broad DRL application in fluid mechanics.
- The methodology is anticipated to support substantially more complex problems, potentially alongside physical-invariance encoding or reduced-order hidden-feature identification.
Appendix B: Evaluation of momentum injected into the flow field using 4 jets
The appendix formulates how four synthetic jets inject horizontal momentum into the flow and evaluates its normalized, time-resolved behavior under active control.
- The appendix derives a mathematical formulation for momentum injected into the flow by four synthetic jets.
- The formulation evaluates each jet’s horizontal momentum injection rate and combines the four contributions into a total.
- Fig. 13 reports the time-resolved normalized momentum added by the four jets at Re=100, 200, 300, and 400.
- The injected horizontal momentum is normalized using the reference momentum intercepting the cylinder.
- Higher-Reynolds-number flows receive more injected momentum under the active flow control strategy.