PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning PDE solution operators with variable initial conditions via Latent Dynamics Networks

Stefano Maria Pizzamiglio, Stefano Pagani, Francesco Regazzoni

arXiv:2610.08475v1cs.LGcs.NAmath.NA

TL;DR

LDNet’s fixed initial-state assumption limits its use for PDEs evolving from varying starting states. This paper formulates latent-state initialization as encoder-free adaptation and compares auto-decoding with meta-learning; meta-learning accelerates inference and is associated with smoother, better-conditioned adaptation and physically meaningful latent organization.

  • Problem

    The original LDNet assumes a fixed initial condition, limiting its applicability when PDE systems evolve from varying initial states.

    Source: §4 · p. 27

  • Method

    The paper formulates initial latent-state inference from early observations as encoder-free adaptation and compares auto-decoding with meta-learning.

    Source: §1 · pp. 2–3, §2.4 · p. 4

  • Results

    Meta-learning accelerates latent-state inference by more than one order of magnitude and is associated with smoother, better-conditioned adaptation and latent representations reflecting physical-space topology.

    Source: §1 · p. 3, §4 · p. 28

  • Takeaways & Limitations

    The encoder-free IC-LDNet preserves resolution independence and supports subsampled spatial training with high-fidelity field recovery at inference across advection-diffusion-reaction, fluid dynamics, and nonlinear solid mechanics.

    Source: §4 · p. 28

    Meta-learning’s faster adaptation and latent-space topology come at the expense of robustness and ease of training, with sensitivity to hyperparameter choice.

    Source: §3.1.3 · p. 14

Abstract

from arXiv · show

In many-query scenarios, data-driven surrogate models provide an efficient alternative to high-fidelity solvers for simulating physical systems governed by Partial Differential Equations (PDEs). In this context, the Latent Dynamics Network (LDNet) has recently demonstrated remarkable performance in predicting the response of spatio-temporal systems, combining Neural Ordinary Differential Equations with nonlinear dimensionality reduction. However, the original formulation assumes a fixed initial condition, limiting its applicability to many real-world applications where a system evolves from varying starting states. In this work, we overcome this limitation while keeping the end-to-end training procedure of the original LDNet and its encoder-free nature, which preserves its intrinsic independence from spatial resolution and grid topology. We infer the initial latent state directly from a small set of early-time observations, treating latent-state initialization as an adaptation problem, and investigate two strategies: an auto-decoding formulation and a meta-learning approach in which the initial latent state acts as a task-specific context variable. We demonstrate the accuracy of the proposed methods across diverse physical phenomena, spanning advection-diffusion, fluid dynamics, and solid mechanics. Meta-learning markedly accelerates latent-state inference and induces smoother, better-conditioned optimization landscapes, and spontaneously organizes the latent space into a structured representation that reflects physically meaningful features of the underlying dynamics. The coordinate-based decoder enables training from spatially subsampled data while recovering high-resolution solution fields at inference. The resulting approach provides an efficient and resolution-independent surrogate modeling framework for many-query simulations of time-dependent PDEs with varying initial conditions.

1 Introduction

High-fidelity numerical solvers can be costly in many-query settings, motivating data-driven surrogates for PDE solution operators. This work extends LDNet beyond fixed initial conditions with encoder-free adaptation strategies, and reports faster latent-state inference and physically meaningful latent organization with meta-learning.

1 Introduction

  • High-fidelity methods can be computationally expensive in many-query settings or for real-time predictions, while data-driven models approximate numerical solvers as efficient surrogates.Source: §1 · p. 1
  • Neural fields represent continuous fields from coordinates to values, making them candidates for discretization-invariant operator learning.Source: §1 · p. 2
  • LDNet learns time-dependent PDE solution operators by evolving a compact latent state and decoding it to physical space, but its original form assumes a fixed initial condition.The paper positions its contribution as extending that framework to unseen initial states while retaining end-to-end training.Source: §1 · p. 2
  • The work formalizes initialization from variable initial conditions as encoder-free adaptation, preserving independence from spatial resolution and grid topology.Source: §1 · p. 2
  • Meta-learning accelerates latent-state inference by more than one order of magnitude and yields better-conditioned adaptation loss and physically meaningful latent organization.The comparison covers four benchmarks spanning advection-diffusion-reaction, fluid dynamics, and nonlinear solid mechanics.Source: §1 · p. 3

2 Methods

The methods extend LDNet to varying initial conditions while retaining its encoder-free framework. The problem is posed as learning a causal mapping from an input function to a spatiotemporal output field.

2 Methods

  • IC-LDNet addresses varying initial conditions through latent-state initialization strategies while retaining the original LDNet framework.The methods section reviews LDNet, introduces strategies for informing the model about the initial state, and presents latent-space regularization strategies.Source: §2 · p. 3

2.1 Problem

  • The problem is to learn a well-posed, causal mapping from an input function over time to an output field over space and time.Source: §2.1 · p. 3

2.2 LDNet: architecture

LDNet couples a latent dynamics network with a decoder: it evolves a latent state over time and decodes it into output values at spatial coordinates. The latent dynamics are integrated with a forward Euler scheme.

2.2 LDNet: architecture

  • The LDNet architecture couples a dynamics network that evolves a latent state with a decoder that maps it to output values across space and time.The continuous dynamics use a Neural ODE.Source: §2.2 · p. 3
  • The latent state is represented by s(t), and this work integrates the Neural ODE in time using a forward Euler scheme.Source: §2.2 · p. 3

2.3 LDNet: training

LDNet is trained with supervised input-output examples and a loss over observed times and spatial points. Its networks are trained end to end, and inference decodes the evolved latent state at requested space-time coordinates.

2.3 LDNet: training

  • LDNet is trained in a supervised manner on input-function and output-field pairs, with time and spatial evaluation points that need not follow a fixed grid or mesh.Spatial points may vary across samples and even across time points.Source: §2.3 · p. 3
  • The loss combines an error measure with regularization terms for the dynamics and decoder weights.The weights are optimized together.Source: §2.3 · p. 3
  • Both networks are trained simultaneously end to end; at inference, the latent state is evolved from s(0) = 0 and decoded to produce outputs at any space-time point.Source: §2.3 · p. 4

2.4 Initial-Condition Latent Dynamics Network

IC-LDNet adapts the initial latent state to variable initial conditions using early observations while retaining an encoder-free LDNet framework. The section presents auto-decoding and meta-learning strategies, then discusses regularization for training stability and latent-space uniqueness.

2.4 Initial-Condition Latent Dynamics Network

  • IC-LDNet embeds initial observations into the latent initial state, replacing standard LDNet’s fixed initialization.The observation mapping Ψ can use a snapshot at t = 0 or a short sequence of early-time observations.Source: §2.4 · p. 4

2.4.1 IC-LDNet with encoder

  • A conventional encoder can map initial conditions to latent states, but its data-format dependence and added cost led the authors to omit this variant.Standard encoders may compromise discretization invariance; the extra network also increases parameter, training, and inference costs.Source: §2.4.1 · p. 4

2.4.2 IC-LDNet with auto-decoder

  • AD-IC-LDNet assigns each training initial condition a trainable latent code and fits a new code from early observations for each unseen sample.The fitted code is then evolved through the Neural ODE and decoded to predict the trajectory.Source: §2.4.2 · pp. 4–5

2.4.3 IC-LDNet with meta-learning

  • Meta-IC-LDNet treats each initial latent state as task-specific context and learns shared dynamics and decoder parameters through an inner adaptation loop and an outer trajectory loss.At inference, the shared meta-parameters are fixed while context parameters are adapted using a short initial time interval.Source: §2.4.3 · pp. 5–6

2.4.4 Regularization strategies for training stability and latent-space uniqueness

  • Training instability and non-unique latent representations motivate curriculum learning and latent-state penalties, with the choice depending on the test case.Curriculum training extends the prediction horizon progressively but requires careful hyperparameter choices and longer training; trajectory-matching penalties add inner-loop optimization cost.Source: §2.4.4 · pp. 6–7

3 Results

The results section evaluates the proposed methods on four benchmark problems and compares their test-set accuracy. It also describes the tuning protocol and reports that the final two test cases use Meta-IC-LDNet alone.

3 Results

  • Four benchmark test cases are used to assess predictive accuracy and generalization, with the final two evaluated using Meta-IC-LDNet alone.The methods are compared using per-component test-set range-NRMSE; the supplied table caption gives the metric and scope but no values.Source: §3 · p. 7, Table 1 · p. 7
  • Unless otherwise specified, training uses a single NVIDIA A100 GPU with 64 GB VRAM on CINECA’s LEONARDO HPC cluster.Training budgets and inference times are summarized in Table 2.Source: §3 · p. 7
  • Hyperparameters are tuned for validation-set generalization, seed robustness is checked during tuning, and the test set is reserved for final inference.Final configurations are reported in Appendix A.Source: §3 · p. 7

3.1 Test case 1: 1D advection-diffusion-reaction

The ADR benchmark tests initial-condition inference on sine-wave states with varying amplitude and phase, comparing auto-decoding and meta-learning. Meta-learning improves average accuracy and error consistency, while its latent organization and adaptation landscape support faster inference.

3.1 Test case 1: 1D advection-diffusion-reaction

  • The ADR benchmark uses 1D sine-wave initial conditions with randomly sampled amplitude and phase, and evaluates 100 test samples from a 1,000-sample dataset.Solutions are evaluated at 101 spatial points and 100 time steps; the dataset is split into 800 training, 100 validation, and 100 test samples.Source: §3.1 · p. 8

3.1.1 AD-IC-LDNet on the ADR test case

  • 3.5 × 10−3 average test-set NRMSE is reported for AD-IC-LDNet, despite variation from 1.6 × 10−3 on the best sample to 3.5 × 10−2 on the worst.The test-set predictions follow optimization of each unseen sample’s initial latent state.Source: §3.1.1 · pp. 9–10, Figure 1 · p. 9
  • AD-IC-LDNet’s amplitude and phase vary along nearly perpendicular latent directions, but its phase encoding is nonperiodic and redundant.Diffusion and reaction influence movement along the amplitude direction, while advection influences drift along the phase direction.Source: Figure 2 · p. 9, §3.1.1 · p. 10, Figure 3 · p. 10

3.1.2 Meta-IC-LDNet on the ADR test case

  • 2.9 × 10−3 average NRMSE and a more evenly distributed test-set error are reported for Meta-IC-LDNet, whose worst prediction is more accurate than AD-IC-LDNet’s.Figure 4 shows the best and worst Meta-IC-LDNet predictions; the comparison with AD-IC-LDNet is reported in the text.Source: §3.1.2 · pp. 10–11, Figure 4 · p. 11
  • Meta-IC-LDNet organizes phase periodically in latent space, and its advection-dominated latent trajectory remains within the training region during a T = 50 rollout.The test-set codes are colored by initial-condition amplitude and phase; the longer rollout is superimposed on those codes.Source: §3.1.2 · p. 11, Figure 5 · p. 12
  • Meta-IC-LDNet’s latent initial velocity nearly correlates with PDE coefficients: diffusion and reaction affect amplitude motion, while advection affects phase motion.The radial and angular velocities are plotted against the corresponding coefficient combinations with linear fits.Source: §3.1.2 · p. 11, Figure 6 · p. 12

3.1.3 AD-IC-LDNet versus Meta-IC-LDNet: initial-code fitting on the ADR test case

  • Meta-learning yields a more regular, better-conditioned initial-code loss landscape than AD-IC-LDNet, enabling rapid gradient-based adaptation in three steps rather than 104 Adam iterations.AD-IC-LDNet’s landscape has multiple local minima; meta-learning’s circular level curves indicate similar maximum and minimum Hessian eigenvalues near the minimum.Source: §3.1.3 · pp. 11–12, Figure 7 · p. 13
  • Meta-IC-LDNet’s final smooth loss landscape follows a sudden Hessian-condition improvement between 17,000 and 18,000 epochs, but the method is sensitive to hyperparameter choice.The authors report that fast adaptation and correct latent-space topology come at the expense of robustness and ease of training.Source: §3.1.3 · pp. 12, 14

3.2 Test case 2A: 2D flow around a static circular cylinder

The static-cylinder benchmark tests latent-state identification from snapshots of a periodic shedding flow. Meta-IC-LDNet, trained with curriculum learning, produces a closed, physically organized latent trajectory and more uniform final-time errors than the auto-decoding variant.

3.2 Test case 2A: 2D flow around a static circular cylinder

  • Because each training sequence spans a full shedding period, test initial conditions also occur as intermediate training states, so this case mainly tests identification from a single snapshot.The dataset contains 60 overlapping sequences of 61 time steps, with an 80%-10%-10% train/validation/test split.Source: §3.2 · p. 15

3.2.1 AD-IC-LDNet on the static-cylinder test case

  • AD-IC-LDNet’s average test-set NRMSE is 4.1×10−3 for pressure and 4.7×10−3 and 7.1 × 10−3 for the two velocity components, respectively.Errors are unevenly distributed, with more noticeable degradation in one selected sample.Source: §3.2.1 · p. 17
  • AD-IC-LDNet arranges initial codes along an open one-dimensional manifold, and predictions degrade outside the training region, especially for test sample 3.The code position corresponds to initial-condition time within the shedding period.Source: §3.2.1 · p. 18

3.2.2 Meta-IC-LDNet on the static-cylinder test case

  • Curriculum learning yields a closed latent manifold whose codes correspond to lift coefficient and its derivative, reflecting shedding angle and velocity.The outer loss expands from the first two time steps to the full 61-step sequence.Source: §3.2.2 · p. 18
  • Meta-IC-LDNet achieves average NRMSEs of around 2.6 × 10−3, 2.7 × 10−3, and 4.1 × 10−3 for pressure and velocity components, respectively.The model captures the periodic flow and produces stable predictions over long roll-outs.Source: §3.2.2 · p. 18

3.3 Test case 2B: 2D flow around a rotating circular cylinder

The rotating-cylinder benchmark combines randomly varying cylinder rotation with flow prediction from spatially subsampled data. Meta-IC-LDNet learns latent coordinates associated with shedding and rotation, predicts flow and forces, and supports a rapid optimal-control demonstration.

3.3 Test case 2B: 2D flow around a rotating circular cylinder

  • Training uses 1 250 independently sampled spatial points per snapshot, while test-time predictions use 40 000 shared points and curriculum learning gradually exposes the full simulations.The original region-of-interest grid contains 80 000 points.Source: §3.3 · p. 21
  • The four-dimensional latent space encodes high-frequency von Kármán shedding and the low-frequency Magnus effect, with components related to lift variations, rotation, and rotation rate.The rotation-related principal components show correlation when compared with a time lag maximizing Pearson correlation.Source: §3.3 · pp. 22–23
  • On the test set, average NRMSE is 7.5 × 10−3 for pressure, 1.2 × 10−2 for x velocity, and 1.3 × 10−2 for y velocity.The model reconstructs the flow with good accuracy despite larger errors than in the static-cylinder case.Source: §3.3 · p. 23
  • The original model’s control-volume method gives highly accurate drag, while surface integration underestimates drag; fine-tuning on a denser near-cylinder grid improves both force coefficients.Both force-computation methods provide similarly satisfactory lift predictions.Source: §3.3 · p. 23
  • The fine-tuned surrogate recovers an angular-velocity signal whose optimal-control lift coefficient follows the OpenFOAM target, solving the control problem in approximately 16 seconds.The recovered angular velocity follows the original signal’s trend, although the difference between the signals is not negligible.Source: §3.3 · p. 24

3.4 Test case 3: nonlinear solid dynamics of a 3D cantilever beam

The 3D cantilever-beam benchmark tests Meta-IC-LDNet on nonlinear solid dynamics under varying loads and material properties. Predictions capture typical beam responses and physically interpretable latent modes, while large-displacement outliers remain more difficult.

3.4 Test case 3: nonlinear solid dynamics of a 3D cantilever beam

  • The benchmark varies independent Gaussian-process load histories and shear modulus across 1 000 sequences drawn from 500 configurations.The input includes constant shear modulus and variable external-load intensities.Source: §3.4 · pp. 24–25
  • Training randomly subsamples 500 of the mesh’s 1 025 points, while inference uses the full point set; additional inner-loop optimizations regularize latent trajectories against optimized codes.Three optimizations are performed at equidistant times within each sequence.Source: §3.4 · pp. 25–26
  • Predictions capture the overall physical trends even in the worst test case, but accuracy degrades for large-displacement samples that are outliers in the data distribution.The best example has only a slight torsional discrepancy; the worst has noticeable error.Source: §3.4 · pp. 26–27
  • The beam is the most challenging benchmark, with average test-set NRMSEs of 4.8 × 10−2, 4.7 × 10−2, and 4.5 × 10−2 for x, y, and z displacement.The samples with larger errors have very large displacements and are outliers in the data distribution.Source: §3.4 · p. 27
  • Latent principal components correspond mainly to distinct beam responses, including stretching, twisting, bending, and flexural-torsional coupling.Each component is varied individually while the others remain at their means, and the resulting codes are decoded into beam deformations.Source: §3.4 · p. 27

4 Conclusions

The conclusions present IC-LDNet as an encoder-free adaptation of LDNet for varying initial conditions, with auto-decoding and meta-learning as alternative strategies. Meta-learning is associated with faster inference and physically structured latent representations, but hyperparameter sensitivity and limited test samples constrain evaluation.

4 Conclusions

  • IC-LDNet addresses LDNet’s fixed-initial-condition assumption through encoder-free latent-state adaptation, while retaining independence from spatial resolution and grid topology.The approach is demonstrated across advection-diffusion-reaction, fluid dynamics, and nonlinear solid mechanics.Source: §4 · pp. 27–28
  • Meta-learning is associated with faster latent-state inference, smoother and better-conditioned optimization, and latent topology reflecting the underlying physical space.The authors suggest that training representations for rapid state inference may bias them toward dynamically meaningful reduced coordinates.Source: §4 · p. 28
  • Meta-IC-LDNet can be hyperparameter-sensitive and may require case-specific regularization; accuracy degrades on outliers, and cylinder test sets contain only 6 and 7 samples.The small cylinder validation and test sets limit the statistical significance of the reported generalization metrics.Source: §4 · p. 28
  • The authors identify further optimization and control applications and explicit physical or topological latent-space constraints as future work.The proposed constraints would be investigated for their potential to facilitate training and enforce representation properties.Source: §4 · p. 28

A Architecture details and hyperparameters

The section reports tuned hyperparameter configurations for four test cases, with the listed configurations organized by case and model variant.

A Architecture details and hyperparameters

B Training and validation

The section presents training and validation curves for final hyperparameter configurations, along with initial-code optimization for AD-IC-LDNet test examples and a rotating-cylinder fine-tuning run.

B Training and validation

Related papers

Loading 2610.08475v1…