PaperlayerView exact revision on arXiv

Source-linked AI summary

Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning

Hongwei Tang, Jean Rabault, Alexander Kuhnle, Yan Wang, Tongguang Wang

arXiv:2004.12417v1physics.flu-dyn

TL;DR

The paper addresses the challenge of robust active flow control across varying Reynolds numbers. It uses PPO with an ANN to control four synthetic jets and smooth action changes, achieving drag reduction at trained Reynolds numbers and for previously unseen values between 60 and 400.

Abstract

from arXiv · show

This paper focuses on the active flow control of a computational fluid dynamics simulation over a range of Reynolds numbers using deep reinforcement learning (DRL). More precisely, the proximal policy optimization (PPO) method is used to control the mass flow rate of four synthetic jets symmetrically located on the upper and lower sides of a cylinder immersed in a two-dimensional flow domain. The learning environment supports four flow configurations with Reynolds numbers 100, 200, 300 and 400, respectively. A new smoothing interpolation function is proposed to help the PPO algorithm to learn to set continuous actions, which is of great importance to effectively suppress problematic jumps in lift and allow a better convergence for the training process. It is shown that the DRL controller is able to significantly reduce the lift and drag fluctuations and to actively reduce the drag by approximately 5.7%, 21.6%, 32.7%, and 38.7%, at $Re$=100, 200, 300, and 400 respectively. More importantly, it can also effectively reduce drag for any previously unseen value of the Reynolds number between 60 and 400. This highlights the generalization ability of deep neural networks and is an important milestone to active flow control.

I. INTRODUCTION

Active flow control offers industrial and environmental benefits, but robust strategies remain difficult for nonlinear, high-dimensional systems and uncertain actuator conditions. This work extends deep reinforcement learning for robust control across multiple flow conditions.

  • AFC can reduce aerodynamic drag, manipulate bluff-body wakes, and optimize wind-turbine and gas-turbine performance.Source: §I · p. 1
  • Designing efficient AFC strategies is difficult because Navier–Stokes dynamics are highly nonlinear and control spaces are high-dimensional.Source: §I · p. 1
  • Engineering AFC must also adapt to environmental disturbances and actuator manufacturing or installation imperfections.Source: §I · p. 1
  • Model-free, data-driven approaches are suitable for complex, high-dimensional, nonlinear systems.Source: §I · p. 1
  • DRL discovers control solutions through trial-and-error even when no solution is known beforehand.Source: §I · p. 2
  • The study extends prior work by using four symmetric synthetic jets and investigating robust DRL control across flow conditions.Source: §I · p. 2

A. Problem description

The simulation models viscous flow past a slightly off-center circular cylinder in a two-dimensional channel, with four synthetic jets providing active control around the cylinder.

  • The benchmark domain contains a cylinder of diameter D in a 22D×4.1D two-dimensional channel.Source: A. Problem description · p. 2
  • The cylinder is offset from the channel centerline by 0.05D, helping trigger vortex shedding above the critical Reynolds number.Source: A. Problem description · p. 2
  • Four synthetic jets are symmetrically positioned on the cylinder’s upper and lower sides and controlled by ANN mass flow rates.Source: A. Problem description · p. 2
  • The jets are located at 75°, 105°, 255° and 285°, with directions perpendicular to the cylinder wall.Source: A. Problem description · p. 2
  • Synthetic-jet operation enforces zero total mass flow, while positive and negative injection velocities represent blowing and suction.Source: A. Problem description · p. 2

B. Numerical method

The numerical method solves nondimensional, incompressible two-dimensional Navier–Stokes flow in a cylinder benchmark domain with prescribed inflow, wall, outflow, and jet conditions. Mesh refinement and benchmark comparisons validate the simulation setup used for drag and lift analysis.

Governing equations

  • The flow is modeled with time-dependent, incompressible Navier–Stokes equations and a continuity equation.Source: B. Numerical method · p. 2
  • The nondimensionalization uses D, U, ρ, and D/U, with Reynolds number defined as Re = UD/ν.Source: B. Numerical method · p. 2

Boundary conditions

  • The domain applies a specified inflow profile, no-slip conditions on walls and the cylinder, and a fully developed outflow condition.Source: B. Numerical method · pp. 2–3
  • Four jet velocity profiles are prescribed radially, with mass flow rates Q_i, angular centers at 75°, 105°, 255° and 285°, and width ω = 10°.Source: B. Numerical method · p. 3

Discretization and solver

  • The IPCS finite-element method uses an unstructured mesh refined near the cylinder to resolve synthetic-jet effects.Source: B. Numerical method · p. 3

Validation

  • Mesh comparisons at Re = 100 agree with benchmark data, with discrepancies below 0.04% for listed quantities on the main mesh versus the fine mesh.Source: B. Numerical method · p. 4

C. DRL control algorithm

The DRL controller learns closed-loop flow control from probe observations by selecting jet actions and optimizing a reward based on drag and lift. PPO training uses controlled action timing, smoothed mass-flow-rate changes, and bounded actuations to support physically stable learning.

  • PPO couples a flow-simulation environment with an agent that observes 236 passive probes, selects jet actions, and receives reward feedback.The agent uses local flow quantities around the cylinder and wake as its state representation.Source: C. DRL control algorithm · p. 4, Figure 4 · p. 5
  • The reward averages drag over T = 100δt and penalizes absolute average lift with β = 0.2.The lift penalty discourages a strategy involving persistent maximum-strength blowing in one direction.Source: C. DRL control algorithm · p. 4
  • Actions are updated 200 times per episode and held constant for 100 numerical time steps between updates.The update interval balances responsiveness against allowing the flow enough time to exhibit a measurable effect.Source: C. DRL control algorithm · p. 6
  • A smoothing interpolation makes instantaneous jet mass flow rates continuous at the numerical timestep, preventing non-physical jumps in pressure or velocity.The continuous control values support stable interaction between the PPO agent and the flow simulation.Source: C. DRL control algorithm · p. 6
  • The normalized jet mass flow is constrained by |Q_i*| ≤ 0.05 to prevent non-physically large actuations and numerical divergence.Here Q_i is the mass flow rate of jet i and Q_ref is the reference flow rate intercepting the cylinder.Source: C. DRL control algorithm · p. 6

A. Active control for flow at higher Reynolds number

Higher Reynolds numbers make the cylinder wake more irregular, motivating tests of DRL control beyond the previously demonstrated Re = 100 case. The study trains controllers at Re = 200 and 400 and examines drag reduction against uncontrolled baselines.

  • Reynolds number strongly influences flow complexity, wake irregularity, and the transition from laminar to turbulent cylinder flow.For this configuration, the wake becomes more irregular as Re increases.Source: A. Active control for flow at higher… · p. 6
  • The mean drag coefficient contains steady symmetric and vortex-shedding contributions.The unsteady contribution is identified as the component that AFC can alter, providing an estimate of attainable drag reduction.Source: A. Active control for flow at higher… · pp. 6–7
  • The vortex-shedding contribution to drag increases with Reynolds number, motivating control tests at Re = 200 and Re = 400.These tests use trained ANNs and compare controlled drag coefficients with uncontrolled baseline flow.Source: A. Active control for flow at higher… · p. 7
  • A controller trained across Reynolds numbers produces comparable average drag reduction while greatly suppressing drag oscillations relative to the Re = 400 comparison.The reported explanation includes four available jets instead of two and training across multiple flow conditions.Source: A. Active control for flow at higher… · p. 7

B. Effect of smoothing interpolation functions

The study compares interpolation schemes for converting discrete ANN actions into continuous control values. Linear interpolation between consecutive actions best suppresses lift oscillations while maintaining performance comparable to exponential decay.

  • Direct interpolation between action updates generates continuous control values for the simulation time steps.The approach is simple to implement and supports policy training and action selection.Source: B. Effect of smoothing interpolation… · p. 7
  • The ANN action index j advances once per action-update interval, whereas the numerical-step index i advances every simulation time step.Action updates occur over a period T = 100δ.Source: B. Effect of smoothing interpolation… · p. 7
  • The exponential decay law reduces drag by approximately 8% at Re = 100 but leaves problematic jumps in lift.Its control values converge well, yet the controlled flow is not perfectly stable.Source: B. Effect of smoothing interpolation… · p. 7
  • Linear interpolation between two actions effectively eliminates lift-coefficient oscillations while matching the exponential law’s performance.The comparison identifies Eq. (20) as the best-performing smoothing law.Source: B. Effect of smoothing interpolation… · p. 7

C. Training a model over a range of Reynolds numbers

A single PPO-trained ANN controls four synthetic jets across flow configurations at Re = 100, 200, 300, and 400. The global controller reduces drag and fluctuations, remains effective at unseen Reynolds numbers, and modifies the controlled wake dynamics.

Training a model over a range of Reynolds numbers

  • A single ANN is trained to control flows at Re = 100, 200, 300, and 400 within one learning environment.The controller is designed to operate robustly across these distinct flow configurations.Source: C. Training a model over a range of… · pp. 7–8
  • 20.4% drag reduction is obtained at Re = 200, with large initial actuation followed by a pseudo-periodic low-actuation regime.The result comes from the PPO agent’s time-resolved control strategy.Source: Figure 6 · p. 8
  • 33.1% drag reduction is obtained at Re = 400, but actuation does not decrease substantially in the second phase because the higher-Re flow is inherently unstable.This behavior illustrates control of systems with pseudo-chaotic properties.Source: Figure 7 · p. 8
  • 5.7%, 21.6%, 32.7%, and 38.7% drag reductions are obtained at Re = 100, 200, 300, and 400, respectively.The global strategy reaches its typical drag-reduction level after a longer initial phase, with larger fluctuations especially at higher Re.Source: C. Training a model over a range of… · p. 9, Figure 9 · p. 10
  • The global strategy greatly suppresses drag oscillations while achieving an average drag reduction close to the Re = 400 single-configuration strategy.The paper attributes this comparison to four-jet fine-grained control and greater variability during global training.Source: C. Training a model over a range of… · p. 9
  • Active control increases the separated-wake area and expands vortex shedding, a flow morphology associated with reduced drag and lift fluctuations.FFT analysis also shows reduced fluctuation amplitudes and modified characteristic frequencies.Source: C. Training a model over a range of… · p. 9, Figure 10 · p. 10
  • For Re ≥200, controlled drag is smaller than drag without vortex shedding, indicating proximity to the theoretical optimum.The unsteady contribution to drag becomes increasingly significant as Re increases.Source: C. Training a model over a range of… · p. 10
  • The controller remains effective for previously unseen Reynolds numbers throughout 60-400 and beyond the training scope, including Re = 80.Only four reference Reynolds numbers were used during training, highlighting ANN generalization within the tested range.Source: C. Training a model over a range of… · pp. 10–11

IV. CONCLUSIONS

The study demonstrates robust, generalized active flow control across Reynolds numbers using PPO-trained neural networks, while identifying training variability and computational cost as remaining constraints.

  • PPO-trained ANNs discover active control strategies across flows with multiple Reynolds numbers.Source: §IV · pp. 11–12
  • Drag is also effectively reduced for previously unseen Reynolds numbers between 60 and 400.Source: §IV · p. 12
  • For Re ≥200, controlled drag is smaller than the symmetric-boundary-condition baseline, suggesting near-theoretical-optimum performance.Source: §IV · p. 12, Figure 12 · p. 12
  • Training randomness and exploration noise can produce slightly different control performance across runs, although qualitative strategies remain relatively similar.Source: §IV · p. 12
  • The selected problem is relatively simple and computational cost remains a challenge for broad DRL application in fluid mechanics.Source: §IV · p. 12
  • The methodology is anticipated to support substantially more complex problems, potentially alongside physical-invariance encoding or reduced-order hidden-feature identification.Source: §IV · p. 13

Appendix B: Evaluation of momentum injected into the flow field using 4 jets

The appendix formulates how four synthetic jets inject horizontal momentum into the flow and evaluates its normalized, time-resolved behavior under active control.

  • The appendix derives a mathematical formulation for momentum injected into the flow by four synthetic jets.Source: Appendix B · p. 13
  • The formulation evaluates each jet’s horizontal momentum injection rate and combines the four contributions into a total.Source: Appendix B · p. 13
  • Fig. 13 reports the time-resolved normalized momentum added by the four jets at Re=100, 200, 300, and 400.Source: Figure 13 · p. 13, Appendix B · p. 13
  • The injected horizontal momentum is normalized using the reference momentum intercepting the cylinder.Source: Appendix B · p. 13
  • Higher-Reynolds-number flows receive more injected momentum under the active flow control strategy.Source: Appendix B · p. 13

Related papers

Loading 2004.12417v1…