Source-linked AI summary
LaSDI: Parametric Latent Space Dynamics Identification
William Fries, Xiaolong He, Youngsoo Choi
TL;DR
High-fidelity forward simulations can make decision-making applications impractical when many runs are needed. LaSDI learns dynamics in compressed latent spaces and combines local models across parameter space. Its conclusions report that compression and local-versus-global identification perform differently depending on the problem.
Problem
High-fidelity forward simulations are computationally expensive for applications requiring many runs, including inverse problems, design optimization, and uncertainty quantification.
Source: §1 · p. 1Method
LaSDI compresses state fields and identifies latent dynamics using local models with trust regions that can cover parameter space.
Source: §1 · pp. 2, 4Results
Local versus global dynamics identification depends on the problem, while nonlinear compression outperforms linear compression for slowly decaying Kolmogorov n-width and linear compression outperforms nonlinear compression for fast decay.
Source: §5 · p. 29Takeaways & Limitations
The reported comparisons support choosing compression and dynamics-identification strategies case by case, balancing accuracy, computational cost, and speed-up.
Source: §5 · p. 29The study considers parameters affecting only initial conditions and uniformly distributed training parameters, whose number grows exponentially with parameter-space dimension.
Source: §5 · p. 29
Abstract
from arXiv · showhide
Enabling fast and accurate physical simulations with data has become an important area of computational physics to aid in inverse problems, design-optimization, uncertainty quantification, and other various decision-making applications. This paper presents a data-driven framework for parametric latent space dynamics identification procedure that enables fast and accurate simulations. The parametric model is achieved by building a set of local latent space model and designing an interaction among them. An individual local latent space dynamics model achieves accurate solution in a trust region. By letting the set of trust region to cover the whole parameter space, our model shows an increase in accuracy with an increase in training data. We introduce two different types of interaction mechanisms, i.e., point-wise and region-based approach. Both linear and nonlinear data compression techniques are used. We illustrate the framework of Latent Space Dynamics Identification (LaSDI) enable a fast and accurate solution process on various partial differential equations, i.e., Burgers' equations, radial advection problem, and nonlinear heat conduction problem, achieving $O(100)$x speed-up and $O(1)\%$ relative error with respect to the corresponding full order models.
1 Introduction
High-fidelity physical simulations can make decision-making applications impractical when many forward runs are needed. LaSDI learns dynamics in a compressed latent space and uses local models to build a parametric simulation framework.
1 Introduction
- High-fidelity forward simulations can make inverse problems, design optimization, optimal control, uncertainty quantification, and parameter studies intractable when many runs are required.Simulation information can be difficult or risky to obtain through experiments or in-situ measurements.Source: §1 · p. 1
- Reduced-space dynamics motivate identification: the radial-advection example compares a 9,216-degree-of-freedom full model with five POD or three auto-encoder latent variables.The paper describes the reduced dynamics as much simpler than the full-order dynamics.Source: §1 · p. 2, Figure 1 · p. 3
- LaSDI builds a parametric model from local latent-space ODEs, each tailored to a parameter-space trust region.The paper states that covering the parameter space with these trust regions provides a certain accuracy level throughout while enabling faster calculations than the high-fidelity model.Source: §1 · p. 4
- LaSDI generalizes DMD and OpInf [71] [72] [74] by allowing nonlinear compression and general nonlinear terms in its candidate library.DMD and OpInf use linear SVD compression and polynomial candidate terms, respectively.Source: §1 · pp. 4, 6
- The framework introduces a local latent-space learning algorithm for robust parametric prediction and global, local, and interpolated dynamics-identification algorithms.It also uses both linear and nonlinear compression techniques.Source: §1 · p. 6
- The paper reports good accuracy and speed-up across numerical experiments and compares global, local, and interpolated identification algorithms across four experiments.The experiments are described in Section 4.Source: §1 · p. 6
2 Dynamical system of equations
The paper formulates a parameterized dynamical system as a time-dependent ODE whose solution is a state trajectory. It assumes the parameter affects only the initial condition and notes that large systems or complicated domains can make forward solves impractically slow.
2 Dynamical system of equations
- The system is a time-dependent ODE for state u(t; µ), with initial state u0(µ) and velocity f.The paper treats it as a semi-discretized PDE and assumes µ affects only the initial condition.Source: §2 · p. 6
- The state trajectory is defined over time and a parameter domain D, with spatial discretization possible by finite difference, finite element, or finite volume methods.The spatial domain is denoted Ω.Source: §2 · p. 6
- Large state dimensions or geometrically complicated domains can make a forward simulation impractically slow, sometimes taking a day or a week.The paper presents LaSDI as a way to accelerate these computationally expensive simulations.Source: §2 · p. 6
3 LaSDI
LaSDI is organized as a four-step framework: data generation, compression, dynamics identification, and prediction.
3 LaSDI
- LaSDI consists of data generation, compression, identification, and prediction, which the paper describes in the following sections.Source: §3 · p. 7
3.1 Data generation
Data generation samples the parameter domain and solves the dynamical system to collect full-order simulation snapshots.
3.1 Data generation
- LaSDI samples training parameters µk from D and solves the full-order system to generate time-step snapshots for each parameter.The snapshots for each parameter are arranged into a matrix, then concatenated side by side into the overall snapshot matrix.Source: §3.1 · p. 7
3.2 Compression
LaSDI compresses simulation snapshots into latent coordinates, choosing linear or nonlinear compression according to the solution space’s Kolmogorov n-width decay. POD and auto-encoders provide the two compression routes, with the resulting latent trajectories used to identify dynamics.
3.2 Compression
- The Kolmogorov n-width guides compression choice: fast decay supports accurate linear compression, while slow decay makes nonlinear compression necessary.Singular-value decay indicates the n-width decay rate.Source: §3.2 · p. 7
3.2.1 Proper orthogonal decomposition: LaSDI-LS
- POD uses the method of snapshots to build a basis minimizing projection error, then maps snapshots to reduced coordinates that form latent trajectories for dynamics identification.The basis is formed from leading left singular vectors of the snapshot matrix.Source: §3.2.1 · pp. 7–8
- Singular values decay slowly for 2D Burgers and radial advection, favoring nonlinear compression, but quickly for 1D Burgers and heat conduction, where linear compression suffices.Source: Figure 3 · p. 8
3.2.2 Auto-encoder: LaSDI-NM
- LaSDI-NM uses an auto-encoder as nonlinear compression; its latent dimension is tuned to preserve reconstruction accuracy, since increasing dimension does not always improve error.The paper uses a masked-shallow encoder-decoder architecture.Source: §3.2.2 · pp. 8–9
3.3 Dynamics Identification
LaSDI identifies latent dynamics through regression and offers global, local region-based, and point-wise interpolated models. Locality, coefficient interpolation, and a staged fitting procedure provide alternatives for modeling parameter-dependent dynamics.
3.3 Dynamics Identification
- Dynamics identification fits a function to latent trajectories and their finite-difference time derivatives; the objective is accurate integrated dynamics, not sparse coefficients.The function is represented using a library that can include constant, polynomial, trigonometric, and exponential terms.Source: §3.3 · pp. 9–10
3.3.1 Region-based global DI
- Global DI fits one model using all training snapshots and covers the parameter space, which may suit cases where latent dynamics change little with parameters.Source: §3.3.1 · p. 10
3.3.2 Region-based local DI
- Local DI fits dynamics from nDI neighboring training parameters; distance-based regions assign query parameters, and global DI is the special case nDI = nµ.Other distance functions can be used, and nDI can be tuned.Source: §3.3.2 · p. 11
3.3.3 Point-wise interpolated DI
- Point-wise DI interpolates each dynamics coefficient for a query parameter, using element-wise radial basis functions or linear bi-variate splines.The RBF weights are obtained by least-squares regression; bilinear splines apply on rectangular regions when parameters lie on a uniform grid.Source: §3.3.3 · pp. 11–12
3.3.4 Notes on DIs
- For difficult fits, the procedure starts with a first-order polynomial and visual trajectory check, then can rescale data and enrich the function library if needed.Possible additions include higher-order or mixed polynomials, trigonometric terms, and exponentials.Source: §3.3.4 · p. 12
- Regression or coefficient interpolation yields an ODE whose right-hand side is determined by the selected function library and identified coefficients.Source: §3.3.4 · p. 13
3.4 Prediction
For a new parameter, LaSDI selects an appropriate dynamics model, initializes and solves it in latent space, then reconstructs the full state. The paper also separates auto-encoder training from dynamics identification.
3.4 Prediction
- For a query parameter, the model is selected by region membership when using local DI; interpolated DI may be more practical with non-uniform training points.Source: §3.4 · p. 13
3.4.1 Latent space dynamics solve and reconstruction
- LaSDI compresses the query’s full-order initial condition, integrates the latent trajectory with Dormand–Prince, and reconstructs full-state trajectories using the linear basis or auto-encoder decoder.Local error is controlled with a fourth-order method and the step uses a fifth-order formulation.Source: §3.4.1 · p. 13
- Auto-encoder training and dynamics identification can be separated in this framework, although simultaneous training is also possible.Source: §3.4.1 · p. 13
4 Numerical Results
Across four PDE test problems, LaSDI’s accuracy varies with compression choice, latent-space dynamics, training coverage, and parameter-space localization. Results show that more training data can improve accuracy, while the relative performance of linear and nonlinear compression depends on the problem.
4 Numerical Results
- The experiments use FOM-generated data for four PDE problems and evaluate accuracy over training and testing parameter sets.The reported problems are 1D and 2D Burgers, nonlinear heat conduction, and radial advection; accuracy is measured by maximum relative error at each testing parameter point.Source: §4 · pp. 13–14, Table 1 · p. 14, Table 2 · p. 15
4.1 1D Burgers
- 0.991% maximum relative error for LaSDI-NM with 25 training points, down from 9.215% with four; LaSDI-LS showed no error reduction from nine to 25 points.For nine training points, degree-1 systems tended to have larger error ranges than degree-0 systems, although the best range came from region-based LaSDI-LS with nDI = 3. Interpolated LaSDIs tended to have larger ranges than region-based models, while degree-0 systems achieved higher speed-up. Figure 10 compares the models with 25 training points.Source: §4.1 · pp. 17, 19, Figure 10 · p. 20
4.2 2D Burgers
- With 25 training points, region-based local dynamics identification outperformed global identification for LaSDI-NM on 2D Burgers, indicating strong parameter-space localization.LaSDI-NM achieved around 800 speed-up using a cubic system with cross-terms; higher-degree dynamics can be numerically unstable, while lower degree can increase speed-up.Source: §4.2 · pp. 19, 23, Figure 13 · p. 22
4.3 Time-Dependent Heat Conduction
- 88% maximum error for LaSDI-NM versus 3.16% for LaSDI-LS across the heat-conduction parameter space; simpler dynamics also achieved over 100x speed-up.LaSDI-LS errors decrease with latent-space dimension, and the authors attribute the trend to both improved compression and good dynamics-identification models. For w = 1.0 and a = 2.0, LaSDI-LS also had much smaller relative error than LaSDI-NM.Source: §4.3 · pp. 23–24, Figure 16 · p. 25
4.4 Radial Advection
- About 3% maximum relative error was achieved by the best local radial-advection case; the worst case was about 20% for LaSDI-NM with a larger parameter space and fewer training points.For global LaSDI-NM, the worst error was 40.46% on the larger parameter space and 17.18% on the smaller one. Local LaSDI was most accurate in each case, and enough training points supported good local-model performance across parameter-space sizes.Source: §4.4 · pp. 24, 26
5 Conclusion
LaSDI combines local latent-space dynamics models with global regression techniques, using linear or nonlinear compression to build parametric simulation models. Its performance depends on the case: local versus global dynamics identification and linear versus nonlinear compression have different advantages and costs.
5 Conclusion
- LaSDI builds parametric models by combining local latent-space dynamics identification with global regression and either linear or nonlinear compression.The framework introduces region-based and point-wise interpolated local dynamics models; POD and auto-encoders compress simulation data to sizes manageable for regression.Source: §5 · p. 26
- LaSDI has been applied to 1D and 2D Burgers equations, heat conduction, and radial advection problems.For radial advection, the reported error ranges include high-error cases associated with numerically unstable dynamics identified by DI; Figure 19 shows best-case errors across T.Source: Figure 18 · p. 27, Figure 19 · p. 28, §5 · p. 29
- Local versus global dynamics identification is case-dependent: local models outperform global models in several settings, while global models or larger local training sets can outperform smaller local sets in another.Local models outperform global LaSDI-NM for 1D Burgers with nine training points, 2D Burgers with 25, and all radial-advection cases; for 1D Burgers with 25 points, global or larger local models outperform smaller local models.Source: §5 · p. 29
- Auto-encoders outperform POD for problems with slowly decaying Kolmogorov width, whereas POD outperforms auto-encoders for problems with fast-decaying width.Nonlinear compression can reduce projection error with small latent dimensions for slowly decaying width, but costs more computationally; the authors recommend linear compression when singular values decay quickly.Source: §5 · p. 29
- The study uses predefined uniformly distributed training parameters and parameterizes effects on initial conditions; higher-dimensional parameter spaces and parameters that do not alter initial conditions remain unsupported settings.The authors identify adaptive or sparse sampling and parameterization beyond initial conditions as future directions.Source: §5 · p. 29
- The authors identify climate science, manufacturing, and fusion energy as potential future application areas for LaSDI.They describe this potential in terms of fast and accurate data-driven simulation capability.Source: §5 · p. 29