Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting

Yuan Yin; Vincent Le Guen; Jérémie Dona; Emmanuel de Bézenac; Ibrahim Ayed; Nicolas Thome; Patrick Gallinari

Source-linked AI summary

Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting

Yuan Yin, Vincent Le Guen, Jérémie Dona, Emmanuel de Bézenac, Ibrahim Ayed, Nicolas Thome, Patrick Gallinari

arXiv:2010.04456v6Oct 9, 2020stat.MLcs.AIcs.CVcs.LG

TL;DR

Forecasting complex dynamics is difficult when physical knowledge is incomplete: pure data-driven methods may be insufficient, while standard physical models can be oversimplified. APHYNITY augments incomplete physical dynamics with a data-driven component constrained to capture only what the physical model misses, and experiments across three system classes show accurate forecasting and physical-parameter identification.

Problem
Forecasting complex dynamical systems is challenging because physical models may be incomplete and pure machine-learning methods may not extrapolate properly to new conditions.
Method
APHYNITY decomposes dynamics into a physical component and a complementary data-driven augmentation, jointly learning both while enforcing a well-defined decomposition.
Results
Across reaction-diffusion, wave-equation, and pendulum systems, APHYNITY achieves forecasting and parameter-identification performance close to complete physical models.
Takeaways & Limitations
The framework can leverage approximate physical models while adapting to different levels of prior knowledge and supporting accurate forecasting and parameter identification.
Takeaways & Limitations
Theoretical decomposition properties in partially observed settings remain an important direction for future work.

Abstract

from arXiv · show

Forecasting complex dynamical phenomena in settings where only partial knowledge of their dynamics is available is a prevalent problem across various scientific fields. While purely data-driven approaches are arguably insufficient in this context, standard physical modeling based approaches tend to be over-simplistic, inducing non-negligible errors. In this work, we introduce the APHYNITY framework, a principled approach for augmenting incomplete physical dynamics described by differential equations with deep data-driven models. It consists in decomposing the dynamics into two components: a physical component accounting for the dynamics for which we have some prior knowledge, and a data-driven component accounting for errors of the physical model. The learning problem is carefully formulated such that the physical model explains as much of the data as possible, while the data-driven component only describes information that cannot be captured by the physical model, no more, no less. This not only provides the existence and uniqueness for this decomposition, but also ensures interpretability and benefits generalization. Experiments made on three important use cases, each representative of a different family of phenomena, i.e. reaction-diffusion equations, wave equations and the non-linear damped pendulum, show that APHYNITY can efficiently leverage approximate physical models to accurately forecast the evolution of the system and correctly identify relevant physical parameters. Code is available at https://github.com/yuan-yin/APHYNITY .

1. Introduction

Complex dynamical systems remain difficult to forecast because pure machine learning may fail to extrapolate, while physical models can be incomplete. APHYNITY addresses this gap by combining an incomplete physical model with a data-driven augmentation whose complexity adapts to the physical approximation.

Pure machine-learning methods often cannot properly extrapolate complex physical dynamics to new conditions.
Hybrid model-based and machine-learning approaches commonly assume that the physical model adequately describes observed dynamics, an assumption often violated in practice.
APHYNITY decomposes dynamics into physical and data-driven terms, with the latter modeling only information not captured by the physical model.
The framework provides existence and uniqueness guarantees under mild conditions, supporting interpretability and generalization.
APHYNITY uses trajectory-based training and adaptive optimization to learn both components end-to-end while adjusting neural-network complexity to physical-model accuracy.
Across reaction-diffusion, wave-equation, and pendulum systems, APHYNITY achieves forecasting and parameter-identification performance close to complete physical models, including partially observable extensions.

2. Related work

Prior hybrid approaches combine physical models and machine learning but often use sequential correction, application-specific fusion, or lack theoretical grounding. APHYNITY instead jointly augments prior dynamics while addressing cooperation and decomposition issues.

Traditional data-assimilation methods apply probabilistic correction after prediction steps, while similar residual procedures are used in robotics and optimal control.
Sequential prediction-and-correction procedures prevent cooperation between prediction and correction.
APHYNITY augments model-based prior knowledge with neurally parameterized dynamics while ensuring optimal cooperation between the prior model and augmentation.
Earlier gray-box methods predicted unknown physical-model parameters with neural networks but did not address proper model-based and machine-learning cooperation.
Other empirical fusion strategies lack theoretical grounding, while specialized partially observed methods target video prediction with dedicated recurrent architectures.
Existing approaches do not resolve decomposition uniqueness or proper cooperation for parameter identification, and vanilla cooperation performs worse than APHYNITY experimentally.

3. The APHYNITY Model

APHYNITY decomposes incomplete physical dynamics into a physical component and a data-driven augmentation, while constraining the augmentation to capture only what the physical model cannot. Its formulation provides existence and uniqueness guarantees and supports joint trajectory-based estimation with adaptive optimization.

APHYNITY writes the unknown dynamics as a physical term Fp plus a data-driven augmentation Fa, with both components estimated from observed trajectories.
The optimization resolves the general non-uniqueness of F = Fp + Fa by making Fa complementary to the physical model rather than allowing it to capture all dynamics.
A minimizing decomposition exists when the physical-model family Fp is proximinal, and it is unique when Fp is a Chebyshev set.
Under these guarantees, the physical component is the metric projection of the unknown dynamics onto Fp, supporting identifiable physical parameters and accurate forecasting.
APHYNITY estimates physical and neural parameters jointly from trajectories, using an integral formulation with differentiable ODE solvers instead of derivative supervision.
Adaptive optimization iteratively minimizes relaxed objectives and updates λ, producing stable and robust results while accommodating different approximation levels of the physical model.

4. Experimental validation

The experiments validate APHYNITY on three challenging dynamical families spanning high-dimensional PDEs and a one-dimensional pendulum. The evaluation uses simulated data with known parameters to assess both forecasting and parameter identification.

APHYNITY is evaluated on reaction-diffusion, wave-propagation, and damped-pendulum dynamics from distinct physical application domains.
The reaction-diffusion and wave experiments involve high-dimensional discretized PDE states, making them more difficult to learn than the one-dimensional pendulum.
The experimental section introduces the three equation families and specifies a common L2 setting for the admissible-state function space.
Experiments use simulated datasets with known model parameters, enabling evaluation of both trajectory prediction and physical-parameter identification.

Reaction-diffusion equations

The paper studies reaction-diffusion, damped-wave, and damped-pendulum equations as representative dynamical systems, with physical models that may omit selected terms or leave parameters unknown.

Reaction-diffusion equations: Its local reaction terms are Ru(u, v; k) = u − u^3 − k − v and Rv(u, v) = u − v, while a and b are diffusion coefficients.
Reaction-diffusion equations: The reaction-diffusion system is a 2D FitzHugh-Nagumo-type model with state X = (u, v) over a compact rectangular domain and periodic boundary conditions.
Reaction-diffusion equations: The physical-model families include a diffusion-only PDE with unknown a and b, and a full PDE with unknown a, b, and k.
Damped wave equations: The damped-wave system uses state X = (w, ∂w/∂t) on a compact spatial domain with homogeneous Neumann boundary conditions, and distinguishes global wave damping from pendulum damping.
Damped wave equations: Wave physical models either estimate propagation speed c without damping or estimate c and damping coefficient k in the damped equation.
Damped pendulum: The pendulum model uses angle θ and angular velocity dθ/dt as its state, with proper pulsation ω0 and damping coefficient α.

Damped pendulum

For the damped pendulum, APHYNITY augments incomplete physical models with learned dynamics, improving forecasting and physical-parameter identification while adapting augmentation strength to the available physical knowledge.

Forecasting accuracy: Incomplete pendulum models can perform worse than purely data-driven models because they omit important dynamics such as friction.The compared incomplete models are Param ODE (ω0) and Hamiltonian models.
Forecasting accuracy: APHYNITY greatly improves forecasting accuracy with incomplete physical models, often approaching the true ODE when the same numerical integration scheme is used.Across the reported problems, the wave-equation example reaches -5.92 versus -5.24 for the true ODE in log MSE.
Physical parameter estimation: APHYNITY reduces physical-parameter identification error to 4% for the pendulum, compared with more than 13% for incomplete models.The reported errors cover the reaction-diffusion, wave-equation, and pendulum settings respectively.
Ablation study: Ablations report consistent gains for APHYNITY over naive physical-plus-data-driven cooperation across the three use cases and physical models.The ablations also examine the trajectory-based training formulation against supervision using finite-difference derivative approximations.
Flexibility: The learned augmentation norm falls as the physical model becomes more complete, reaching 8.5 for APHYNITY versus 132 and 623 for incomplete pendulum models.This norm is presented as an indication of physical-model imperfection and supports adaptation across different levels of prior knowledge.

5. Conclusion

APHYNITY augments approximate physical models with deep data-driven networks and performs similarly to models with fully known dynamics. The framework is flexible across approximation levels and is positioned for future partially observable applications.

APHYNITY augments approximate physical models with deep data-driven networks, performing similarly to models whose underlying dynamics are entirely known.
Across three classes of physical systems, APHYNITY adapts to different approximation levels of prior physical knowledge.
The framework is proposed as applicable to partially observable settings such as video prediction.
Analyzing the theoretical decomposition properties in partially observed settings remains an important direction for future work.

Appendix B. Proof of Propositions 1 and 2

The appendix proves that APHYNITY’s decomposition problem corresponds to a metric projection onto the physical-model function space. Proximinality gives existence, while the stronger Chebyshev condition gives uniqueness.

The optimization problem is equivalent to a metric projection onto the physical-model function space.
The proof reconstructs the full functional from observed trajectories and characterizes valid decompositions as physical plus augmenting components equal to that functional.
If the physical-model space is proximinal, the optimization problem admits a minimizing pair.
If the physical-model space is Chebyshev, the minimizing pair is unique and the physical component is the metric projection.

Appendix C. Parameter estimation in incomplete physical models

The appendix explains why fitting incomplete physical models alone can bias parameter estimates and leave residual trajectory error. APHYNITY instead uses a trajectory-based decomposition whose error behavior is better suited to discretely sampled data.

When the true dynamics lie outside the physical-model space, fitting the incomplete model alone has strictly positive minimum loss.
Incomplete physical models may absorb unexplained phenomena into estimated parameters, producing physically biased values such as a period that accounts for damping.
In experiments, APHYNITY produces experimentally better physical-identification results than incomplete physical models alone.
For the damped-pendulum example, APHYNITY’s residual formulation estimates a = 2 and recovers the desired solution with the right period.
Trajectory-based optimization controls error over integrated trajectories, whereas derivative supervision relies on approximate finite differences.
Finite-difference supervision necessarily introduces discretization error, whose divergence from true trajectories can become explosive, especially with sparse sampling.

Appendix E.1. Reaction-diffusion equations

The appendices specify experimental setups for reaction-diffusion, wave, and damped-pendulum dynamics, including discretization, datasets, architectures, and ablation comparisons. The reported ablations show that APHYNITY’s decomposition improves forecasting and parameter identification over vanilla augmentation.

Reaction-diffusion: Reaction-diffusion states use a 32 × 32 grid with periodic boundaries and a discrete Laplace operator.The state is X = (u, v) on Ω = [−1, 1]^2.
Reaction-diffusion: Reaction-diffusion data contain 1920 sequences, with 1600 for training/validation and 320 for testing, predicting 25 future time steps.The true PDE uses fixed a = 1 × 10−3, b = 5 × 10−3, and k = 5 × 10−3.
Architectures: A 3-layer ConvNet provides the data-driven derivative operator, while physical parameters are learned as the trainable vector (a, b).The same ConvNet architecture is used for the data-driven Neural ODE baseline.
Ablation: APHYNITY’s vanilla-augmentation ablation outperforms the comparison scheme in forecasting and parameter identification.For the Hamiltonian pendulum case, log MSE is -0.35 versus -3.97; for reaction-diffusion Param PDE (a, b), parameter error is 8.4% versus 2.3.

Appendix F.2. Detailed ablation study

The detailed ablations test APHYNITY’s decomposition, adaptive optimization, and trajectory-based supervision. Across the reported examples, the principled scheme improves forecasting and parameter identification over vanilla, non-adaptive, and derivative-supervised alternatives.

Supervision: Trajectory-based learning outperforms derivative supervision, with wave-equation log MSE = -4.64 versus -1.16 for Param PDE (c).The derivative-supervised ablation replaces trajectory loss with finite-difference derivative supervision.
Optimization: The adaptive optimization improves reaction-diffusion forecasting, with log MSE = -4.55 versus -5.10 for Param PDE (a, b).The comparison is against non-adaptive optimization.
Varying parameters: With varying reaction-diffusion coefficients, APHYNITY improves prediction log MSE from -4.32 to -1.32 and reduces errors in a and b to 11.8% and 18.7%.The coefficients a and b vary by sequence while k remains fixed.
Varying parameters: For complete reaction-diffusion physics, APHYNITY reduces the reported b error from 3.10 to 1.23 and also lowers the errors for a and k.The reported a and k errors change from 1.55 and 0.59 to 1.29 and 0.39.

Appendix G.2. Additional results for the wave equation

Additional experiments test APHYNITY when wave speeds or pendulum parameters vary across sequences. The method remains effective for parameter identification, while purely data-driven and incomplete physical models struggle with the harder settings.

Varying wave speed: When wave speed c varies uniformly from 300 to 400, prediction worsens slightly but parameter estimation for c and k remains consistent.The experiment uses 200 training sequences and 50 test sequences, with k fixed at 50.
Varying wave speed: The purely data-driven Neural ODE fails as wave-speed variation increases the difficulty of the task.This comparison is reported alongside the varying-wave-speed results.
Varying pendulum parameters: For pendulum sequences with varying T0 and α, pure data-driven models fail to extrapolate the dynamics, while incomplete physical models are worse because they omit friction.The varying ranges are T0 ∈ [3, 10] and α ∈ [0, 0.5].
Varying pendulum parameters: APHYNITY significantly and consistently improves pendulum forecasting and parameter identification in the varying-parameter setting.Models use the first 20 steps to predict the next 20 and estimate sequence-specific parameters with an encoder.