Source-linked AI summary
Accelerated surrogate dynamics for dynamical, stochastic system evolution
Marco Jochum, Ioannis Kouroudis, Gohar Ali Siddiqui, Taher Amine Hamzaoui, Manuel Gößwein, Alessio Gagliardi
TL;DR
High computational costs and high-dimensional inputs complicate simulation surrogates, while sequence models have difficulty covering both local and long-range dynamics. The paper combines a VAE with a Temporal Fusion Transformer to encode, propagate, and decode system states. Across diverse simulations, it reports excellent results and inference multiple orders of magnitude faster.
Problem
Physical simulations can be computationally costly, and sequence models face high-dimensional inputs and difficulty capturing both short- and long-range dynamics.
Source: §1 · p. 1, §3.1 · p. 10Method
A VAE encodes configurations into a lower-dimensional latent space, which a Temporal Fusion Transformer propagates before the decoder reconstructs configurations.
Source: §1.3 · p. 4, §3.2 · p. 10Results
Across diverse simulation cases, the pipeline reports excellent results, with inference multiple orders of magnitude faster.
Source: §1 · p. 2, §4 · p. 14Takeaways & Limitations
The framework is presented as a versatile surrogate for complex and stochastic dynamics that can expedite phenomena investigation.
Source: §1 · p. 2, §5 · p. 15
Abstract
from arXiv · showhide
Dynamic simulations are an entrenched way of gaining insight into the evolution of system dynamics. Their computational cost however is often prohibitively high, especially in cases of stochastic frameworks. Machine learning algorithms are especially suited as simulation surrogates. Nevertheless, they face some very distinct limitations. Firstly, the sheer dimensionality of these systems, however, precludes the use of traditional time series models who struggle with high dimensional feature spaces. Additionally, traditional time series focus exclusively on either long or short range effects, causing local or global drift given enough time. In this paper, we propose a framework that addresses those limitations. Our framework combines a Variational Autoencoder, with a convolutional or graph basis that reduces the dimensionality of the system. This latent vector is propagated in time using a Temporal Fusion Transformer model, which includes both long range and short range effect encoding, as well as static covariate support. We test our framework on three distinct cases, to prove its robustness and in all three we have achieved practically identical to the simulation results at a fraction of the time. Further, our framework is flexible enough to be adapted to any new system and provides an inbuilt uncertainty quantification for targeted experiment design.
1 Introduction
Physical simulations can be computationally costly, while existing sequence models struggle with high-dimensional systems that require both short- and long-range dynamics. The paper proposes a VAE–Temporal Fusion Transformer pipeline to reduce dimensionality, propagate latent states, and reconstruct configurations.
1 Introduction
- The pipeline is presented as versatile across diverse dynamical cases, with excellent results and the potential to expedite phenomena investigation.The introduction describes applications to stochastic and deterministic systems represented as space configurations and molecular graphs.Source: §1 · pp. 1–2
1.1 Methods
- The surrogate autoregressively maps past field configurations and simulation parameters to future configurations, using an encoder, multi-horizon propagator, and decoder.The propagator predicts multiple future states in one call.Source: §1.1 · pp. 2–3, Figure 1 · p. 3
1.2 Encoder E and Decoder D
- The VAE reduces configuration dimensionality and embeds physically similar configurations nearby; its separate training optimizes reconstruction while regularizing the latent space toward a standard normal.The reconstruction and regularization loss contributions are weighted equally.Source: §1.2 · pp. 3–4, Figure 2 · p. 4
1.3 Temporal Fusion Transformer
- The Temporal Fusion Transformer propagates latent states while combining short- and long-range dependencies, static and dynamic covariates, and variable-importance weighting.Its architecture includes LSTM processing for short-range dependencies and interpretable attention over temporal patterns.Source: §1.3 · pp. 4–5
2 Stochastic Methods
The section applies the surrogate to spin exchange and kinetic Monte Carlo (kMC), including solid-state-electrolyte boundary-layer formation. It reports close energy agreement, predicted kMC uncertainty, and inference substantially faster than simulation.
2.1 Metropolis Monte Carlo Model for ion separation
- The spin-exchange model represents a two-dimensional lattice of up-or-down spins and uses energy differences to guide spin exchanges toward lower magnetic energy.The coupling constant J determines whether neighboring spins favor parallel or antiparallel alignment.Source: §2.1 · pp. 6–7
- For spin exchange, the reconstructed energy evolution has less than 5% error against ground truth, with ResNet-style residual prediction used for image reconstruction.The model also appends predicted energy and spin ratio to the latent space and includes their errors in the loss.Source: §2.1 · p. 7
2.2 Kinetic Monte Carlo for Solid State Electrolytes
- For solid-state-electrolyte kMC, the VAE encodes full configurations, TFT predicts latent trajectories using simulation parameters as static covariates, and the decoder reconstructs configurations.The modeled system is a simplified three-dimensional lattice gas of mobile Li+ ions; its grid does not represent the actual electrolyte morphology.Source: §2.2 · p. 8, §2.3 · p. 8
2.3 Kinetic Monte Carlo
- Figure 5 presents predicted mean concentration fields across three parameter configurations and reports that the parameters affect both final profiles and preceding dynamics.The fields sum concentration along Z and average across simulation runs at timesteps 10, 20, 50, 100, 200, and 300.Source: §2.3 · p. 8, Figure 5 · p. 9
- The model predicts standard-deviation fields for the kMC simulation, representing its epistemic uncertainty across the same parameter configurations as Figure 5.Source: §2.3 · p. 9, Figure 6 · p. 9
- Training takes minutes and inference seconds, while simulation spans days; the framework’s inference is many orders of magnitude more efficient.Source: §2.3 · pp. 9–10
3 Non Stochastic Methods
The alanine-dipeptide surrogate reproduces key conformational and transition statistics using a graph-based latent representation and temporal forecasting. Its setup combines molecular graph encoding, coarse-grained training data, and latent-space prediction.
3.1 Dynamics of Molecular Systems
- The alanine-dipeptide model targets a conformational space spanned by dihedral angles ϕ and ψ, using a coarse-grained molecular dynamics surrogate.The system is a coarse-grained model of alanine dipeptide; hydrogens are removed from the depicted configurations.Source: §3.1 · p. 10, Figure 7 · p. 10
3.2 Latent dynamics
- A bond-graph encoder represents bonds as vertices and angles or torsions as edges, producing a latent embedding from each molecular frame.The decoder combines graph-based embeddings with the latent vector to predict time-dependent bond lengths, angles, and dihedrals; the architecture can train on arbitrary graph sizes and multiple molecules.Source: §3.2 · pp. 10–12
3.3 Dataset
- The dataset comes from 100 ns of alanine-dipeptide molecular dynamics in water, sampled every 100 fs and coarse-grained by averaging atom positions over windows.Multiple long sequences are sampled from random trajectory starting points to improve robustness to initial-point variation.Source: §3.3 · p. 12
3.4.1 Training
- The encoder-decoder is trained separately, then frozen while the temporal fusion transformer is trained on latent trajectories with latent-space and topology losses.Latent predictions use Gaussian regression; topology predictions use mean-squared error, with different weights for bonds, angles, and torsions.Source: §3.4.1 · p. 12
3.4.2 Prediction
- At prediction time, the encoder represents the starting trajectory segment and the temporal fusion transformer forecasts future latent states for decoding.The prediction procedure samples latent points using a variance derived from the MD trajectory, with an ICA-based scaling adjustment.Source: §3.4.2 · pp. 12–13
3.4.3 Results
- The surrogate’s free-energy surface agrees well with simulation, including the depth of the rate minimum at high ϕ values, and its transition probabilities closely match simulation.The free-energy surface is compared over dihedrals ϕ and ψ; transition probabilities are compared between the simulation and surrogate.Source: §3.4.3 · p. 13, Figure 8 · p. 13, Figure 9 · p. 14
4 Efficiency gain and Result physicality
The framework’s efficiency advantage is its much faster inference relative to simulation. For the first two simulations, the surrogate is also expected to reach an unchanged equilibrium state.
- Inference is multiple orders of magnitude faster than simulation, while KMC simulations can take days to weeks to converge.The training time is a fraction of the simulation time, and deployment requires even less compute; Figure 10 compares simulation, training, and inference time.Source: §4 · p. 14, Figure 10 · p. 14
- For the first two simulations, the surrogate is expected to reach the same unchanged equilibrium state, preserving stability despite possible accuracy loss.The paper presents steady-state convergence as important to physical correctness.Source: §4 · p. 14
5 Outlook
The paper concludes that its framework performs well across diverse simulation methodologies and increases computational efficiency by at least three orders of magnitude, depending on the case. It is presented as a way to accelerate dynamics investigation while preserving intrinsic physical uncertainty.
- Across multiple and diverse simulation methodologies, the framework performs well and yields at least three orders of magnitude higher computational efficiency, depending on the case.The conclusion also identifies acceleration of dynamics investigation and preservation of intrinsic physical uncertainty as potential advantages.Source: §5 · p. 15