Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems

Francisco J. Gonzalez; Maciej Balajewicz

Source-linked AI summary

Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems

Francisco J. Gonzalez, Maciej Balajewicz

arXiv:1808.01346v2Aug 3, 2018math.DScs.LGphysics.flu-dyn

TL;DR

The paper addresses computationally burdensome and intrusive nonlinear model reduction for high-dimensional dynamical systems. It proposes a convolutional autoencoder plus modified LSTM that learns and evolves low-dimensional manifold coordinates, and demonstrates performance across parameter-varying and long-term fluid prediction tasks.

Problem
High-dimensional projection-based model reduction can be intrusive, require special treatment of nonlinearities, and face stability limitations, while fully connected autoencoders do not scale well to DNS-sized inputs.
Method
A deep convolutional autoencoder learns nonlinear-manifold coordinates, a modified LSTM evolves them efficiently, and modular offline training jointly updates reconstruction and prediction objectives.
Results
Across three illustrative nonlinear model-reduction examples, the convolutional recurrent autoencoder outperforms traditional POD-Galerkin ROMs and produces promising results for parameter-varying and long-term-stability problems.
Takeaways & Limitations
The model provides a completely data-driven reduced-order approach for learning expressive low-dimensional representations and evolving them in time for fluid systems.
Takeaways & Limitations
The authors identify high-frequency decoding error as the main source of error and note that the method's full capabilities remain unknown.

Abstract

from arXiv · show

Model reduction of high-dimensional dynamical systems alleviates computational burdens faced in various tasks from design optimization to model predictive control. One popular model reduction approach is based on projecting the governing equations onto a subspace spanned by basis functions obtained from the compression of a dataset of solution snapshots. However, this method is intrusive since the projection requires access to the system operators. Further, some systems may require special treatment of nonlinearities to ensure computational efficiency or additional modeling to preserve stability. In this work we propose a deep learning-based strategy for nonlinear model reduction that is inspired by projection-based model reduction where the idea is to identify some optimal low-dimensional representation and evolve it in time. Our approach constructs a modular model consisting of a deep convolutional autoencoder and a modified LSTM network. The deep convolutional autoencoder returns a low-dimensional representation in terms of coordinates on some expressive nonlinear data-supporting manifold. The dynamics on this manifold are then modeled by the modified LSTM network in a computationally efficient manner. An offline unsupervised training strategy that exploits the model modularity is also developed. We demonstrate our model on three illustrative examples each highlighting the model's performance in prediction tasks for fluid systems with large parameter-variations and its stability in long-term prediction.

1 INTRODUCTION

The paper presents a non-intrusive, data-driven approach to nonlinear model reduction that learns a low-dimensional manifold and its dynamics from data. Convolutional encoding addresses scalability, while a modified LSTM evolves representations efficiently for parameter-varying and long-term prediction tasks.

High-dimensional simulations can involve millions or billions of degrees of freedom, motivating reduced-order models that retain large-scale behavior while lowering computational burden.
The proposed model uses a deep convolutional autoencoder to learn nonlinear-manifold coordinates and a modified LSTM to evolve those low-dimensional representations without costly full-state reconstruction at every step.
An end-to-end, two-step unsupervised training strategy exploits model modularity to learn representation and dynamics together from spatiotemporal data.
Projection-based reduction is intrusive, may require hyper-reduction for nonlinear systems, and can face stability challenges in high-Reynolds-number flows.
Fully connected autoencoders become computationally intractable at DNS-level sizes, motivating convolutional methods that exploit local, location-invariant correlations.

2 PROBLEM FORUMALTION

The problem formulation considers large nonlinear ODE systems obtained by spatially discretizing time-dependent PDEs, with states, time, and parameters defining the evolution. The goal is non-intrusive surrogate modeling from solution snapshots for computationally demanding applications, including single-trajectory and parameter-varying datasets.

The model targets a high-dimensional nonlinear ODE from semi-discretizing a time-dependent PDE, with x as the large spatial state and μ as a feasible parameter vector.
Parameters may encode initial or boundary conditions, material properties, or shape parameters in parameter-varying systems.
Large state dimension makes direct evaluation challenging in time-critical applications and multi-query settings requiring repeated solutions.
An offline-online strategy uses solution snapshots to construct a surrogate that approximates new solutions at a fraction of the original computational cost.
The formulation restricts the surrogate to non-intrusive, purely data-driven modeling that requires the snapshot dataset but no governing-equation information.
Datasets may consist of snapshots from one statistically stationary trajectory or multiple trajectories with varying parameters for transient, multi-query applications.

3 BACKGROUND

This section introduces deep learning, recurrent and convolutional architectures, and projection-based model reduction. It explains how POD provides low-dimensional linear coordinates while motivating nonlinear manifold representations.

3.1 Deep learning: Deep neural networks learn increasingly abstract representations through layered transformations of high-dimensional inputs.Training adjusts network parameters to reduce discrepancy between predicted and target outputs while seeking generalization beyond observed examples.
3.2 Recurrent neural networks: RNNs process sequences incrementally while retaining prior inputs in hidden states, but long sequences can cause vanishing or exploding gradients.LSTM units address this difficulty through additional gradient paths across multiple time steps.
3.3 Convolutional neural networks: CNNs exploit local connections and shared weights to detect location-invariant features in arrayed data.Stride reduces feature-map dimensions, whereas dilated filters enlarge the receptive field without losing resolution.
3.4 Projection-based model reduction: Projection-based model reduction approximates high-dimensional states in a low-dimensional affine subspace and determines coordinates by enforcing residual orthogonality.POD obtains this subspace from snapshot data using singular value decomposition.
3.5 Connection between autoencoders and POD: POD modes optimally represent data in the L2 sense, yielding low-dimensional linear coordinates for the input state.The modes are derived from the leading singular vectors of the snapshot matrix.
3.5 Connection between autoencoders and POD: POD is limited to an optimal linear manifold, whereas complex real-world data is often strongly nonlinear.This limitation motivates more accurate representations of the data-supporting manifold.

4 CONVOLUTIONAL RECURRENT AUTOENCODERS FOR MODEL REDUCTION

The approach combines nonlinear dimensionality reduction with learned feature dynamics: a convolutional autoencoder identifies a low-dimensional manifold, while an LSTM evolves its coordinates without requiring system operators. Its convolutional design targets high-dimensional data by exploiting local structure and shared filters, and joint unsupervised training couples representation learning with recurrent dynamics.

Model-reduction framework: The model follows a two-step reduction strategy: identify a low-dimensional manifold and efficiently evolve its intrinsic representation.The representation may be obtained with POD or an autoencoder, while the dynamics are modeled separately on the manifold.
Feature dynamics: An LSTM models the possibly nonlinear evolution of the reduced state vector, extending earlier approaches that used linear recurrent dynamics.The LSTM is used because reduced dynamics on the learned manifold need not be linear.
Convolutional autoencoder: A deep convolutional autoencoder learns an expressive low-dimensional representation from high-dimensional system states.The architecture applies convolutional processing before fully connected encoding, then reconstructs through a corresponding decoder.
Convolutional autoencoder: Convolutional layers exploit local correlations and shared filters, reducing trainable parameters relative to fully connected processing.The architecture uses a four-layer convolutional encoder and a four-layer transpose-convolutional decoder around fully connected layers.
Convolutional autoencoder: The convolutional autoencoder contains at most 330k parameters for Nh = 64, whereas a comparable 12-layer fully connected autoencoder requires over 22M parameters.This comparison is reported for the described architecture and representation size.
Training strategy: The method trains the convolutional autoencoder and recurrent model jointly through an unsupervised strategy designed to limit overfitting.The training approach exploits the modularity of the convolutional recurrent autoencoder.

5 NUMERICAL EXPERIMENTS

Three numerical examples assess nonlinear autoencoder-based model reduction for viscous Burgers flow, parameter-varying periodic vortices, and lid-driven cavity flow. The models improve reconstruction quality and maintain stable long-term predictions relative to POD-based approaches, while convolutional models address larger spatial systems.

5.1 Viscous Burgers equation: The Burgers example compares a 20-dimensional nonlinear autoencoder representation with an equivalently sized POD representation, each evolved by a single-layer modified LSTM.The autoencoder uses two encoder and two decoder layers with an intermediate layer of size 512; both models are trained for 100,000 iterations.
5.1 Viscous Burgers equation: The nonlinear recurrent autoencoder avoids the spurious oscillations seen in the L2-optimal POD reconstruction and provides an effective nonlinear reduced-order model.The POD-LSTM accurately evolves the optimal POD representation non-intrusively, but the autoencoder-LSTM better represents the nonlinear state reconstruction.
5.2 Parameter-varying flow in a periodic box: The periodic-box experiment varies Gaussian-vortex locations and signs across two-vortex and three-vortex initial-condition families at Re = 5 × 10^3.Vortex centers are randomly sampled within a square subdomain; the two-vortex case uses opposite signs, while the three-vortex case uses one positive and two negative vortices.
5.2 Parameter-varying flow in a periodic box: Across 512 new prediction runs, errors in the periodic-box models did not grow significantly over the training sequence, and predictions lacked commonplace POD spurious oscillations.The models also predict evolving vortex locations for the two-vortex case.
5.3 Lid-driven cavity flow: For the lid-driven cavity, convolutional recurrent autoencoder predictions nearly capture the exact solution after long-term prediction, whereas POD-Galerkin reconstructions show spurious oscillations.The comparison includes latent dimensions N_h = 8, 16, and 64; the N_h = 8 POD-Galerkin prediction diverges after t = 60 because of instabilities.
5.3 Lid-driven cavity flow: The convolutional model retains a high-frequency reconstruction error attributed possibly to strided transpose convolutions in the decoder, despite stable recurrent predictions.A final undilated convolutional layer is suggested as a possible filter for this noise.

6 CONCLUSIONS

The convolutional recurrent autoencoder combines nonlinear spatial compression with recurrent dynamics modeling to provide a data-driven reduced-order model. Across three examples, it addresses expressive representation, parameter variation, prediction stability, and non-intrusive modeling, while remaining limited by unresolved decoder and interpretability challenges.

Architecture: The convolutional autoencoder maps high-dimensional inputs to an expressive low-dimensional nonlinear manifold, while the modified LSTM models dynamics on that manifold.The architecture uses convolutional encoding for spatial data and avoids reconstructing the full state at every evolution step.
Training: The training strategy jointly weights autoencoder reconstruction error and modified LSTM prediction error during parameter updates.Each forward pass first creates temporary low-dimensional targets with the autoencoder, then advances the recurrent model.
Demonstrations: Three nonlinear model-reduction examples test expressive power, parameter-varying systems with large initial-condition changes, and long-term prediction of chaotic incompressible flows.The examples collectively assess representation quality, location-invariant feature learning, parameter variation, and stability.
Conclusions: The model is reported to outperform traditional POD-Galerkin ROMs in prediction quality, parameter variations, and stability while remaining data-driven and non-intrusive.The conclusion also identifies location-invariant feature learning as an additional advantage.
Future work: The main reported error is high-frequency and appears during decoding, motivating improved filtering strategies for the convolutional transpose decoder.The authors also identify multi-scale dynamic modeling and reconciliation with physical intuition as future challenges.