PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-level Convolutional Autoencoder Networks for Parametric Prediction of Spatio-temporal Dynamics

Jiayang Xu, Karthik Duraisamy

arXiv:1912.11114v2physics.comp-phcs.LGphysics.flu-dyn

TL;DR

Expensive high-fidelity simulation motivates a data-driven reduced-order model for complex spatio-temporal dynamics. The paper combines spatial and temporal convolutional autoencoders with an MLP to predict unseen parameters and future states, achieving accurate predictions and substantial computational reductions in the reported tests. The authors conclude that the approach is promising when adequate data are available but should not replace traditional model-reduction methods universally.

  • Problem

    High-fidelity PDE solutions are prohibitively expensive for repeated design, optimization, and uncertainty-quantification queries, motivating cheaper reduced-order and surrogate models.

  • Method

    A CAE encodes spatial snapshots, temporal convolutional networks process latent sequences, and an MLP maps spatio-temporal latent variables to unseen global parameters.

  • Results

    The framework accurately predicts discontinuities, waves, transients, and coherent-structure dynamics, with reported computational reductions exceeding three orders of magnitude and 600× in separate tests.

  • Takeaways & Limitations

    The results support effective data-driven predictive models when adequate data and careful training are available, including cases involving changed flow topology.

  • Takeaways & Limitations

    The authors do not recommend replacing traditional model-reduction methods or using CAEs instead of POD for all problem classes.

Abstract

from arXiv · show

A data-driven framework is proposed towards the end of predictive modeling of complex spatio-temporal dynamics, leveraging nested non-linear manifolds. Three levels of neural networks are used, with the goal of predicting the future state of a system of interest in a parametric setting. A convolutional autoencoder is used as the top level to encode the high dimensional input data along spatial dimensions into a sequence of latent variables. A temporal convolutional autoencoder (TCAE) serves as the second level, which further encodes the output sequence from the first level along the temporal dimension, and outputs a set of latent variables that encapsulate the spatio-temporal evolution of the dynamics. The use of dilated temporal convolutions grows the receptive field exponentially with network depth, allowing for efficient processing of long temporal sequences typical of scientific computations. A fully-connected network is used as the third level to learn the mapping between these latent variables and the global parameters from training data, and predict them for new parameters. For future state predictions, the second level uses a temporal convolutional network to predict subsequent steps of the output sequence from the top level. Latent variables at the bottom-most level are decoded to obtain the dynamics in physical space at new global parameters and/or at a future time. Predictive capabilities are evaluated on a range of problems involving discontinuities, wave propagation, strong transients, and coherent structures. The sensitivity of the results to different modeling choices is assessed. The results suggest that given adequate data and careful training, effective data-driven predictive models can be constructed. Perspectives are provided on the present approach and its place in the landscape of model reduction.

1. Introduction

The paper motivates data-driven reduced-order modeling for expensive spatio-temporal simulations and proposes a multi-level neural-network framework for non-intrusive prediction. It combines nonlinear compression, convolution, and regression to address limitations of linear trial spaces and governing-equation-based approaches.

  • Motivation: High-fidelity PDE solutions remain prohibitively expensive for repeated design, optimization, and uncertainty-quantification queries.Reduced-order and surrogate models therefore extract low-dimensional information offline for cheaper online prediction.
  • Limitations of existing methods: Linear trial spaces can become ineffective for advection-dominated and many multiscale problems, despite adaptive bases and closure modeling.Projection-based ROMs commonly construct these spaces using truncated proper orthogonal decomposition applied to solution snapshots.
  • Nonlinear reduction: Nonlinear trial manifolds, including autoencoder-based compression, have been pursued to overcome limitations of linear bases.Related applications to fluid dynamics have reported improved accuracy compared with linear bases.
  • Model-reduction approaches: Intrusive ROMs project governing equations onto reduced manifolds, whereas non-intrusive methods learn surrogate models directly from full-order solutions.Intrusive approaches may require additional approximations for computational efficiency, while non-intrusive methods use interpolation or regression.
  • Proposed approach: The proposed framework uses spatial and temporal convolutional neural networks, autoencoders, and multilayer perceptrons for parametric spatio-temporal prediction.Autoencoders learn nonlinear latent representations, while localized convolutions process coherent dynamic features.

2. Goals and Framework Overview

The framework targets prediction at unseen parameters, future times, or both by processing spatial snapshots and temporal latent sequences through nested convolutional networks. A CAE encodes spatial fields, temporal networks process the resulting sequence, and decoded outputs produce predicted physical states.

  • Problem setting: The framework assumes uniformly sampled spatial-field snapshots on a fixed Cartesian grid, with multiple variables depending on global parameters.A sequence Q(µ) contains the snapshots over nt time steps.
  • Prediction tasks: It addresses new-parameter prediction, future-state prediction, and combined prediction beyond the training time period.Future-state prediction proceeds one step at a time and can generate multiple subsequent steps iteratively.
  • Framework architecture: A convolutional autoencoder encodes each spatial snapshot into a latent sequence, while temporal convolutional networks process that sequence.The CAE is shared across the tasks, with TCN architectures used at the second level.
  • New-parameter prediction: For new parameters, a temporal convolutional autoencoder compresses the latent sequence into variables representing spatio-temporal evolution, and an MLP maps those variables to the new parameter.The predicted latent representation is subsequently decoded to recover the dynamics.
  • Future-state prediction: For future states, a TCN iteratively predicts subsequent spatially encoded latent steps, which are decoded into future physical states.The same top-level spatial encoder supports this temporal prediction pathway.

3. Network Details

The framework combines dense, convolutional, dilated-convolutional, and autoencoder components to compress and predict spatio-temporal data. Its three network levels use spatial encoding, temporal compression or prediction, and regression across parameters.

  • The framework introduces dense, convolutional, and dilated convolutional layers as its basic network components.
  • A dense layer applies a linear transformation followed by an element-wise activation, while fully connected stacks form multilayer perceptrons for nonlinear regression.
  • Dilated convolutions expand the receptive field exponentially with network depth, enabling efficient processing of long sequences; strided variants compress sequence dimensions.
  • The autoencoder bottleneck reduces dimensionality into latent variables that a decoder reconstructs, while the TCN predictor outputs the next latent variable from an input look-back sequence.
  • The CAE encodes and decodes individual spatial frames, using convolution-pooling blocks in the encoder and transposed convolutions in the decoder.
  • For new-parameter prediction, the TCAE compresses each spatial-latent sequence into one latent set, and an MLP regresses that representation to unseen parameters.

4. Numerical tests

Numerical tests evaluate the framework on advection, discontinuous compressible flow, transient cylinder flow, and transient ship airwake, assessing prediction accuracy, training-data sensitivity, and computational cost. The reported results show low errors in several settings, while future prediction accuracy depends on sufficient training snapshots.

  • Test cases: The tests cover wave advection, discontinuous compressible flow, transient cylinder flow, and transient ship airwake, including challenging propagation and transient dynamics.The study also compares modeling choices and reports computational timing for reduced-order prediction.
  • Linear advection: The advection model compresses 128 spatial degrees of freedom to 32 latent degrees of freedom using a three-layer CAE.The latent representation transforms the half cosine wave into a triangular wave with two linear halves.
  • Linear advection: CAE reconstruction preserves the advection-wave representation with the same training and testing error, indicating position-independent reconstruction precision for the translated shape.This result is attributed to conservation of the wave shape during translation.
  • Linear advection: 0.04% RAE is obtained for the predicted final advection state, with the error dominated by CAE reconstruction rather than TCN prediction.The TCN error is reported as negligible compared with the CAE error.
  • Training sensitivity: Future-state prediction accuracy improves rapidly with more training snapshots and begins to saturate after nt = 30; with nt = 20, the TCN is inaccurate.The observation is reported for 20 iterative future-prediction steps and illustrates the importance of sufficient training data.
  • Transient-flow results: The framework reports below-2% RAE for predicted velocity components in one transient-flow test and below-0.9% for final predicted velocity components in the ship-airwake test.The ship-airwake predictions remained accurate despite significant variations in unsteady patterns between testing and training conditions.
  • Computational cost: For 400 frames of the 3D unsteady flow field, prediction takes about 6.6 s versus 4011.5 s for the reference CFD simulation, yielding > 600× lower computational cost.The timing comparison uses the framework and a commercial parallel solver on different CPU configurations.

5. Perspectives

The framework offers a non-intrusive alternative to projection-based model reduction, with rapid development and execution but greater data demands and scope limitations. The authors present it as complementary to, rather than a replacement for, traditional methods.

  • Non-intrusive modeling simplifies development and makes predictions orders of magnitude cheaper than standard projection-based models.Training took a few GPU hours, while predictions took a few seconds.
  • Intrusive methods may require less data and offer more opportunities to enforce physical constraints.
  • The authors identify comparisons with intrusive methods as requiring disciplined evaluation across benchmark problems.They note that guarantees on consistency and optimality make intrusive techniques natural candidates for comparison.
  • Neural-network approaches generally require more data than linear and more structured compression techniques.The paper notes that expressivity and generalization must be balanced against data-collection cost.
  • The convolution operations are defined on Cartesian grids, while graph convolutions are proposed for generalized unstructured meshes.

6. Summary

The paper proposes a multi-level neural framework for parametric and future-state prediction of spatio-temporal dynamics. Tests show accurate predictions across discontinuities, waves, and changing flow topologies, while the authors retain a complementary view of traditional reduction methods.

  • The framework uses a convolutional autoencoder for spatial encoding and temporal convolutional networks for sequence processing and future-state prediction.Decoded bottom-level outputs provide high-dimensional predictions for desired conditions.
  • Tests demonstrate accurate prediction of discontinuities and waves beyond the training period in one-dimensional compressible flow.
  • Evaluations on transient two- and three-dimensional flows demonstrate parametric and future-state prediction despite significant changes in flow topology.
  • The convolutional autoencoder performs significantly better than POD for latent-variable extraction, especially during future-state prediction.
  • The authors do not recommend replacing traditional model reduction methods or POD for all problem classes.They consider the approach most compelling when adequate data is available.

Appendix A. Replacement of CAE with POD

The appendix replaces the top-level convolutional autoencoder with POD to create a direct comparison between nonlinear and linear latent spaces. POD compresses flattened, scaled snapshots using a truncated singular-vector basis and reconstructs the state from the compressed variables.

  • The comparison replaces the top-level CAE with POD and applies the same standard-deviation scaling to flattened coupled variables.Each flattened frame forms a column of the snapshot matrix S.
  • POD constructs its basis from the first nk left-singular vectors of the snapshot matrix S.The basis V has dimensions R^n×n_k.
  • The appendix refers to compression and reconstruction as encoding and decoding, respectively.

Appendix A.1. Discontinuous compressible flow

The discontinuous-flow appendix shows that POD can reconstruct the limited training data exactly yet fail when predicted waves move beyond the region represented in training snapshots.

  • The shock-tube training set contains 35 time steps, limiting the effective POD modes to nk = 35.Using all available modes yields exact reconstruction of the training data.
  • POD prediction fails when waves propagate outside the region containing training features.The appendix illustrates the failure at t = 0.25 s relative to training data ending at t = 0.1 s.

Appendix A.2. Transient flow over a cylinder

The CAE-based framework outperforms the POD-based alternative for transient cylinder flow because POD produces substantially larger spatial reconstruction errors. These spatial errors limit the accuracy of the final decoded prediction.

  • With nk = ns = 60, POD produces significantly larger spatial reconstruction errors than CAE in both training and testing.The comparison replaces CAE with POD while keeping the neural-network settings identical.
  • Temporal processing with TCAE or TCN is largely independent of the spatial encoder and slightly benefits from smoother encoded CAE variables.The first latent dimension is smoother than the physical point-monitor signal and the CAE latent variable.
  • POD-based decoding largely fails to represent the cylinder-flow dynamics, making POD performance the main limitation on final prediction accuracy.The reported relative error is noticeably larger than for the CAE-based result.

Appendix A.3. Transient ship airwake

For transient ship airwake, POD-based spatial compression increases reconstruction error and produces visibly poorer predictions than CAE during the strong initial transition. The remainder of the framework follows the POD-reconstructed variables reliably.

  • The comparison uses equal encoded dimensions, nk = ns = 20, with the same second- and third-level network architectures.RAE values for the different stages are summarized in Table A.14.
  • POD reconstruction deviates strongly from the truth during the first 100 frames, when the side-slip transition strongly affects the deck airwake.The discrepancy is also visible in the flow contours, where POD appears blurred and shifted while CAE is almost visually identical to truth.
  • The POD-based final prediction follows the POD-reconstructed variables tightly, indicating reliable performance of the rest of the framework.

Appendix B.1. Transient flow over a cylinder

The appendix examines latent-dimension sensitivity for the cylinder-flow CAE and TCAE. Reconstruction errors decrease rapidly at small latent dimensions before flattening or saturating.

  • CAE training and testing errors drop sharply for ns ≤30, then flatten, with decay saturating for ns ≥60.
  • TCAE testing error drops rapidly for nl ≤80, while its training-error decay is almost negligible.
  • Rapid decay in leading modes allows the spatio-temporal sequence to be represented with substantially fewer latent variables.

Appendix B.2. Transient ship airwake

For transient ship airwake, increasing latent dimensions is less effective because training and testing data differ substantially across both large- and small-scale physics. Even so, the model achieves low testing reconstruction errors.

  • CAE testing error saturates at a few modes because changing inflow side-slip angles limits similarity between training and testing data.Adding extensive latent dimensions therefore does not efficiently reduce testing error.
  • Testing RAE remains below 0.5% for CAE and 5% for TCAE despite the slow error decay.The reported values support efficient model reduction for the airwake case.

Appendix C. Training convergence history

Appendix C reports convergence histories for different framework levels across discontinuous compressible flow, transient cylinder flow, and ship airwake cases. Testing behavior varies with how much the dynamics differ across parameters or time periods, while all reported errors remain below 1e−2.

  • Sample convergence histories are reported for the CAE, TCN, TCAE, and MLP levels across the appendix test cases.The plotted loss values are evaluated on scaled network inputs and therefore should not be compared directly with the Sec. 4 MSE.
  • The training–testing loss gap increases as the dynamics depart more strongly across parameters or time periods.
  • In discontinuous compressible flow, testing loss closely follows training loss because the local dynamics are similar.
  • In the other cases, testing loss saturates earlier, especially for the ship-airwake TCAE because its dynamic patterns differ substantially.
  • All tested errors remain below 1e−2, while training loss achieves more than 4 orders of convergence in most cases.
Loading 1912.11114v2…