PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Neural Networks for Nonlinear Model Order Reduction of Unsteady Flows

Hamidreza Eivazi, Hadi Veisi, Mohammad Hossein Naderi, Vahid Esfahanian

arXiv:2007.00936v3physics.flu-dyncs.LGphysics.comp-ph

TL;DR

The paper addresses reduced-order modeling of nonlinear, high-dimensional unsteady flows with complex spatiotemporal behavior. It combines nonlinear autoencoder feature extraction with LSTM future-state prediction and compares the result with DMD- and POD-based models. Autoencoder-LSTM achieves at least R2 = 0.9405 on the reported test cases and outperforms DMD for multi-frequency and extreme-event flows.

  • Problem

    Nonlinear unsteady flows contain complex phenomena across spatial and temporal scales, while conventional ROM commonly relies on linear SVD-based compression.

  • Method

    A deep nonlinear autoencoder extracts low-dimensional flow features, and an LSTM uses feature sequences to predict future velocity fields; results are compared with DMD- and POD-based ROMs.

  • Results

    Autoencoder-LSTM achieves R2 of at least 0.9405 on the test cases and outperforms DMD for multi-frequency and extreme-event flows.

  • Takeaways & Limitations

    Deep neural networks provide a data-driven non-intrusive ROM framework for nonlinear unsteady-flow evolution prediction.

Abstract

from arXiv · show

Unsteady fluid systems are nonlinear high-dimensional dynamical systems that may exhibit multiple complex phenomena both in time and space. Reduced Order Modeling (ROM) of fluid flows has been an active research topic in the recent decade with the primary goal to decompose complex flows to a set of features most important for future state prediction and control, typically using a dimensionality reduction technique. In this work, a novel data-driven technique based on the power of deep neural networks for reduced order modeling of the unsteady fluid flows is introduced. An autoencoder network is used for nonlinear dimension reduction and feature extraction as an alternative for singular value decomposition (SVD). Then, the extracted features are used as an input for long short-term memory network (LSTM) to predict the velocity field at future time instances. The proposed autoencoder-LSTM method is compared with non-intrusive reduced order models based on dynamic mode decomposition (DMD) and proper orthogonal decomposition (POD). Moreover, an autoencoder-DMD algorithm is introduced for reduced order modeling, which uses the autoencoder network for dimensionality reduction rather than SVD rank truncation. Results show that the autoencoder-LSTM method is considerably capable of predicting fluid flow evolution, where higher values for coefficient of determination $R^{2}$ are obtained using autoencoder-LSTM compared to other models.

I. INTRODUCTION

Unsteady-flow ROM must capture nonlinear, multiscale dynamics while extracting features useful for future-state prediction. The paper proposes deep autoencoder-LSTM modeling and compares it with established non-intrusive ROM approaches.

  • Complex unsteady flows contain nonlinearities and phenomena across multiple spatial and temporal scales, complicating low-dimensional reduced-order modeling.
  • POD, DMD, and Koopman analysis are established ROM approaches for extracting flow features or representing dynamical evolution.
  • Most prior ROM studies rely on linear compression such as SVD, motivating nonlinear dimensionality reduction with autoencoders.
  • The proposed method uses a nonlinear deep autoencoder for feature extraction and an LSTM to predict future flow evolution from feature sequences.
  • The study evaluates nonlinear deep-learning ROM against linear and nonlinear non-intrusive frameworks on cylinder and oscillating-airfoil flows.

II. NUMERICAL ANALYSIS AND DATA SETS

The dataset comprises finite-volume simulations of cylinder and oscillating-airfoil flows spanning constant, varying, and oscillatory conditions. These cases provide periodic, variable-frequency, and moving-mesh test settings for ROM evaluation.

  • Two-dimensional Navier–Stokes equations are solved with a finite-volume method using second-order temporal and spatial discretization.
  • Cylinder test 1 uses Re = 3900 and a circular O-type computational mesh with 98,878 cells.
  • Cylinder test 2 decreases Reynolds number from 3355 to 676, producing weakened vortex shedding and changing frequency.
  • The second cylinder case contains 332 snapshots, with 166 used for training.
  • The oscillating-airfoil case models a sinusoidally pitching NACA0012 at Re = 1.35 × 10^5 and reduced frequency k = 0.1.

III. METHODOLOGY

The methodology compresses CFD snapshots into nonlinear latent features and uses recurrent dynamics to reconstruct future velocity fields. Autoencoder architecture and LSTM memory mechanisms support this reduced-order prediction pipeline.

  • A. Autoencoder Neural Network: CFD snapshots are flattened and passed through an autoencoder that maps each velocity state to a lower-dimensional latent representation.
  • C. Autoencoder-LSTM method for ROM: Sequences of extracted features are supplied to an LSTM to predict the velocity field at future time steps.
  • A. Autoencoder Neural Network: The autoencoder is trained for nonlinear dimensionality reduction and feature extraction, with decoding used to reconstruct the input flow field.
  • A. Autoencoder Neural Network: The encoder maps V(t) to latent features g(t), while the decoder reconstructs the velocity field from those features.
  • B. Long Short-Term Memory Network: LSTM gates regulate information entering, remaining in, and leaving the cell state, allowing information from earlier sequence steps to persist.

C. Autoencoder-LSTM method for ROM

The autoencoder-LSTM framework compresses flow snapshots into a lower-dimensional latent representation, then uses feature sequences to predict future velocity fields iteratively.

  • Autoencoder feature extraction: The autoencoder maps each flattened flow snapshot V(t) to lower-dimensional features g(t) and reconstructs the input data.The latent dimension satisfies n_g ≪ n_input.
  • LSTM prediction: The LSTM receives sequences [g(t_n), ..., g(t_n+p)] and predicts the velocity field at the next time step.The sequence contains p previous time units for next-step prediction.
  • Iterative forecasting: The ROM iteratively feeds newly predicted sequences back into the network to forecast unsteady flow evolution over multiple future steps.The procedure begins from an initial snapshot sequence and repeatedly predicts the next time step.
  • Training: The networks are trained using feedforward and backpropagation algorithms, with Adam used for the training runs.The implementation uses TensorFlow.
  • Evaluation: R2 and MSE are used to estimate prediction and reconstruction error, with MSE also serving as the training loss function.R2 is commonly between zero and one.

D. Hyperparameter Analysis

The hyperparameter analysis evaluates activation functions, network depth, and LSTM cell counts for reconstruction and prediction performance across the tested configurations.

  • Analysis setup: The analysis varies activation functions, autoencoder hidden layers, and LSTM cell counts using validation loss and test-set R2.The tested activation functions are tanh, ReLU, and softplus; LSTM cell counts are 10, 100, and 600.
  • Activation functions: ReLU performs best for autoencoder reconstruction, while tanh performs better for LSTM prediction.The comparison uses validation loss and test-set R2.
  • LSTM capacity: Increasing the number of LSTM cells leads to better predictions in the reported analysis.The analysis compares LSTM networks with different cell counts.
  • Autoencoder depth: An autoencoder with three hidden layers obtains the best autoencoder-LSTM prediction results among the tested depths.The comparison examines one, two, and three hidden layers.
  • Input and output dimensions: The cylinder test cases use 81401-node velocity snapshots, whereas the oscillating airfoil case uses 99473-node snapshots.The autoencoder input and output layers match the velocity-field dimension for each case.
  • Data normalization: Input data are normalized using the measured value, its time average, and its standard deviation to cover both negative and positive values.The normalized value is denoted V-hat.
  • LSTM input and output: The LSTM input is a three-dimensional matrix organized by latent feature size, sequence length p, and number of sequences.Its output layer has the full velocity-field dimension.

F. DMD with Autoencoder for Dimensionality Reduction

The paper replaces SVD-based dimensionality reduction in DMD with autoencoder latent features, then performs DMD in that reduced space and decodes predictions into velocity fields.

  • DMD background: DMD is a model-free reduced-order technique that decomposes fluid-flow measurements into dominant spatiotemporal coherent structures.Its standard formulation constructs snapshot and lagged matrices from measured system data.
  • Framework comparison: Figure 7 depicts autoencoder training, LSTM next-step prediction, and iterative autoencoder-LSTM forecasting from snapshot sequences.The figure shows feature extraction, reconstruction, and repeated future-flow prediction.
  • SVD-based DMD: Standard DMD avoids direct computation of the high-dimensional operator A by projecting it onto a low-rank subspace using SVD.The truncation rank determines the reduced subspace.
  • Modal prediction: DMD modes and eigenvalues are obtained through eigendecomposition of the reduced operator and are used to construct future-time approximations.The initial snapshot and DMD eigenvectors determine the initial modal amplitudes.
  • Autoencoder-DMD: The autoencoder-DMD approach uses latent vectors g as input snapshots instead of applying SVD rank truncation to the original velocity fields.The reduced matrices are G and G-prime, formed from consecutive latent features.
  • Prediction pipeline: DMD computes the mapping A = G-prime G-dagger in latent space, propagates the features forward, and passes predictions through the decoder to recover velocity fields.The method is illustrated as autoencoder-DMD in the paper.
  • Hyperparameter context: Figure 8 compares validation loss and R2 across activation functions, LSTM cell counts, autoencoder depths, and LSTM architectures.The panels include reconstruction and prediction results on test data.
  • Mode selection: The number of DMD modes is selected when the method reaches a specified accuracy threshold to avoid sub-optimal or erroneous modes.This procedure aims to retain efficiency with an appropriate parameter count.

A. Dimension Reduction with Autoencoder

The autoencoder provides nonlinear dimensionality reduction and extracts flow features through its bottleneck representation. Increasing bottleneck dimension improves reconstruction accuracy, while extracted modes differ from DMD modes because of nonlinear activation functions.

  • A. Dimension Reduction with Autoencoder: The autoencoder projects high-dimensional velocity fields into a low-dimensional bottleneck representation and reconstructs them at the output.The bottleneck dimension controls the compression level and reconstruction process.
  • A. Dimension Reduction with Autoencoder: Even with ng = 3, the velocity field is reconstructed with acceptable accuracy.
  • A. Dimension Reduction with Autoencoder: Larger bottleneck dimensions increase reconstruction accuracy and decrease approximation error.
  • A. Dimension Reduction with Autoencoder: The autoencoder extracts dominant flow features, producing modes that differ from DMD modes because nonlinear activation functions are used.The extracted modes may contain noise, so a two-dimensional Gaussian filter was applied for clearer representation.
  • A. Dimension Reduction with Autoencoder: The first three autoencoder features were visualized for cylinder test 1, cylinder test 2, and the pitching airfoil.

B. Comparison of Autoencoder-LSTM with DMD-based models

The autoencoder-LSTM framework is evaluated against DMD and autoencoder-DMD for reconstructing training data and predicting future velocity fields. It performs especially well for multi-frequency, non-periodic, and dynamic-stall flows, where linear DMD-based models are less accurate.

  • B. Comparison of Autoencoder-LSTM with DMD-based models: The comparison evaluates autoencoder-LSTM, DMD, and autoencoder-DMD on training-data reconstruction and testing-data prediction.All models use a latent-space dimension of 50, with ng = 50 for the autoencoder and r = 50 for DMD.
  • B. Comparison of Autoencoder-LSTM with DMD-based models: R2 reaches at least 0.9405 for autoencoder-LSTM across the test cases, while DMD is comparable only for cylinder test 1.Autoencoder-LSTM outperforms DMD for cylinder test 2 and the oscillating airfoil.
  • B. Comparison of Autoencoder-LSTM with DMD-based models: For cylinder test 2, autoencoder-LSTM obtains R2 = 0.9405 versus DMD’s R2 = 0.0826 on testing data.
  • B. Comparison of Autoencoder-LSTM with DMD-based models: Full-ranked DMD reaches R2 = 0.1776 and MSE = 1.81 × 10^-5, compared with autoencoder-LSTM’s R2 = 0.9405 and MSE = 1.06 × 10^-6 for cylinder test 2.
  • B. Comparison of Autoencoder-LSTM with DMD-based models: Autoencoder-LSTM accurately predicts dynamic-stall velocity variations and vortex shedding, whereas DMD and autoencoder-DMD fail to capture them perfectly.The same method is reported to perform well for both periodic and non-periodic flows investigated.

C. Comparison of Autoencoder-LSTM with POD-based models

Autoencoder-LSTM is compared with POD-LSTM, which predicts temporal amplitudes of POD modes. The autoencoder can compact more POD content into a smaller latent representation, while POD-LSTM becomes less accurate when predicting many higher-frequency modes for non-periodic flow.

  • C. Comparison of Autoencoder-LSTM with POD-based models: POD-LSTM uses POD modes as a basis and an LSTM to predict their temporal amplitudes, providing a non-intrusive alternative to POD-Galerkin modeling.POD-Galerkin systems can be inaccurate or unstable for transitional and turbulent flows and may struggle with highly nonlinear problems.
  • C. Comparison of Autoencoder-LSTM with POD-based models: With three latent modes, autoencoder-LSTM predictions contain more than three POD basis modes, extending through approximately mode 6.The autoencoder compacts a higher number of POD modes into three autoencoder modes while retaining comparable energy for higher-energy modes.
  • C. Comparison of Autoencoder-LSTM with POD-based models: For non-periodic cylinder test 2, POD-LSTM with 50 modes produces lower R2 than with rank truncation of 3.Higher-order POD modes are higher-frequency, making their temporal dynamics more challenging for the LSTM to predict.
  • C. Comparison of Autoencoder-LSTM with POD-based models: Autoencoder modes are correlated and unordered, so their dominant modes cannot be identified by energy ranking as in POD.The bottleneck size requires a priori selection and posterior validation, and an unsuitable choice requires retraining.

D. Prediction of Transient Dynamics

An LSTM network predicts the transient dynamics of a low-order cylinder-wake model at Re = 100 from a limited training segment, whereas DMD cannot capture the dynamics.

  • Test setup: The test case uses a two-dimensional viscous flow past a circular cylinder at Re = 100, represented by two energetic POD modes and a shift mode.The temporal dynamics are supplied by a mean-field model used for modal decomposition, reduced order modeling, and model identification.
  • Training setup: The model is integrated for 2000 time steps with ∆t = 0.1, using the first 340 steps for training and 60 previous steps for one-step prediction.The LSTM has one hidden layer containing 600 LSTM cells.
  • Comparison: DMD cannot predict the transient dynamics because it is essentially a linear model.
  • Results: Despite training on a small portion of the dynamics, the LSTM provides an excellent prediction of the nonlinear transient evolution.The prediction is evaluated over the whole reference-data time span.

E. Computational Cost

The autoencoder-LSTM requires offline neural-network training, while DMD is much faster and is reported as five orders of magnitude faster than CFD in the tested setting.

  • Cost metric: Computational costs are reported separately for offline training and online prediction as tROM/tCFD.The costs are averaged over three test cases.
  • Autoencoder-LSTM: The autoencoder-LSTM incurs its main computational cost during offline neural-network training.The proposed ROM includes both offline and online computations.
  • DMD comparison: DMD is much faster than autoencoder-LSTM because it is trained in one shot.
  • CFD comparison: DMD is five orders of magnitude faster than CFD in the reported RANS-based simulations.The paper notes that ROM costs would be much lower relative to CFD for DNS.

V. CONCLUSION

The paper introduces deep-neural-network ROMs that combine autoencoder-based nonlinear reduction with LSTM prediction, and reports strong performance across complex unsteady-flow cases.

  • V. CONCLUSION: The study investigates three cases: a cylinder at Re = 3900, a cylinder with Reynolds number decreasing from 3355 to 676, and an oscillating airfoil.
  • V. CONCLUSION: The proposed autoencoder-LSTM method uses an autoencoder to extract reconstructable low-dimensional flow features and an LSTM to estimate the next velocity field.
  • V. CONCLUSION: Performance is compared with DMD and POD-mode LSTM ROMs using coefficient of determination R2 and MSE.The study also assesses autoencoder dimensionality reduction in DMD instead of SVD rank truncation.
  • V. CONCLUSION: The results indicate excellent potential for deep neural networks in data-driven reduced order modeling of complex unsteady flows.
Loading 2007.00936v3…