PaperlayerView exact revision on arXiv

Source-linked AI summary

Identifiability Guarantees for Drivers and Dynamics of Delayed Physical Systems

Julien Boussard, Antoine Debouchage, Théo Saulus

arXiv:2609.37944v3stat.MLcs.LGmath.DS

TL;DR

Identifying the drivers and drift of delayed physical systems is challenging because existing approaches may lack identifiability guarantees or rely on assumptions physical systems violate. The paper uses L0 regularization to establish SDDE identifiability guarantees and reports benchmark performance in driver identification and long-term statistics.

  • Problem

    The paper addresses how to identify SDDE drivers and drift when causal-discovery methods may rely on strong constraints that physical systems violate.

    Source: §1 · p. 1

  • Method

    L0-NDDE applies L0 regularization to provide identifiability guarantees for lagged and instantaneous SDDE drivers and drift under permissive assumptions.

    Source: §1 · p. 2

  • Results

    L0-NDDE outperforms other data-driven methods on a structural-driver benchmark and captures long-term statistics on 40 simulated low-dimensional chaotic systems.

    Source: §1 · p. 2

  • Takeaways & Limitations

    On chaotic systems, long-run statistics provide a well-defined target even when individual trajectories cannot be forecast over long horizons; L0-NDDE captures these statistics on simulated systems.

    Source: §1 · p. 2, §3.4 · p. 5

    The method is evaluated mostly on SDE-driven systems because, to the authors' knowledge, no comprehensive benchmark exists for lagged-system modeling.

    Source: §6 · p. 9

Abstract

from arXiv · show

A wide range of methods have been proposed, including physics-informed neural networks, which are powerful but do not guarantee identifiability of the dynamics, symbolic regression, which requires a set of precomputed operations, and causal discovery, which is more principled but usually relies on strong assumptions that physical systems may violate. In this work, we develop a theory-grounded method and prove that under a set of permissive assumptions, the structural drivers and drift of stochastic delayed differential equations are identifiable. Our method outperforms others on a benchmark for driver identifiability, and on a second benchmark to evaluate physical consistency of the learned dynamics.

1 INTRODUCTION

Physical systems with delayed feedback require models that represent both current and past states, yet existing approaches face limits in identifying their dynamics. This work develops identifiability guarantees for SDDE drivers and drift and evaluates the method on driver-identification and long-term-statistics benchmarks.

1 INTRODUCTION

  • Delayed feedback makes past states relevant to physical-system evolution, and SDDEs represent delayed connections unlike Markovian SDEs.Examples include climate and ecological or neural population dynamics.Source: §1 · p. 1
  • Physics-informed neural networks and symbolic regression can improve generalization, but require a known parametric form or precomputed functions and are computationally expensive.The approaches are discussed alongside limits in learning long-term statistics with delayed neural ODEs.Source: §1 · p. 1
  • Causal discovery can infer nonlinear, lagged causal structures, but commonly relies on constraints such as acyclicity that physical systems may violate.Physical systems may also violate the assumption of no instantaneous connections.Source: §1 · pp. 1–2
  • L0-NDDE provides identifiability proofs for lagged and instantaneous drivers and drift of SDDEs, under permissive assumptions.The guarantees cover population-level and finite-sample settings.Source: §1 · p. 2
  • L0-NDDE outperforms other data-driven methods on a benchmark for identifying structural drivers and captures long-term statistics on 40 simulated low-dimensional chaotic systems.The systems are from the Dysts dataset.Source: §1 · p. 2

2 RELATED WORK

Time-series causal discovery includes neural, constraint-based, permutation-based, score-based, noise-based, and cross-mapping approaches. This work applies L0-induced sparsity to SDDE inputs and derives identifiability guarantees for instantaneous and lagged drivers.

2 RELATED WORK

  • Time-series causal discovery methods include nonlinear autoregressive links, conditional-independence tests, permutation searches, constrained score optimization, noise-independence approaches, and cross-mapping.Examples include NGC, CUTS+, TCDF, PCMCI+, F-PCMCI, GRaSP, DYNOTEARS, VARLiNGAM, RCD, and TSCI.Source: §2 · p. 2
  • Causal methods often discretize time and rely on strong graph assumptions, while Bellot et al. (2022) provide driver-identifiability results for SDEs and Wu et al. (2024) apply them to SDDEs without guarantees.Acyclicity is one example of an assumption that may conflict with continuous-time dynamics.Source: §2 · p. 2
  • The paper uses L0-induced sparsity on input features to identify instantaneous and lagged SDDE drivers, with theoretical guarantees under discrete, finite observations.L0-induced sparsity had also been used in causal representation learning and Neural ODE regularization.Source: §2 · p. 2

3 METHOD

The method models delayed dynamics with a sparse driver mask and drift, then develops conditions for their recovery from population risk and finite observations. It also establishes well-posedness and ergodicity, and describes a differentiable implementation.

3 METHOD

  • The method proves identifiability of the driver mask and drift from finite observations, with population-level results, geometric ergodicity, and a finite-sample theorem.The model represents the system as an SDDE with instantaneous and delayed inputs and a sparse binary driver mask.Source: §3 · p. 3

3.1 EXISTENCE AND UNIQUENESS OF A SOLUTION

  • Under Assumptions 1 and 2, the SDDE has a unique strong solution that does not explode in finite time.The assumptions include non-degenerate noise, finite energy, local Lipschitz drift, and a radial Khasminskii condition.Source: §3.1 · p. 3

3.2 IDENTIFIABILITY OF THE DYNAMICAL DRIVERS

  • Under Assumptions 1–5, the drift G and driver mask M are identifiable at the population level.The setup assumes a model class containing the true pair, population-risk minimization, faithfulness, and a learned mask no denser than the true mask.Source: §3.2 · p. 4

3.3 L0-PENALIZED IDENTIFIABILITY

  • With an L0-penalized objective, every minimizer recovers the true mask and drift when each true driver has measurable impact and the penalty meets the theorem’s conditions.This result does not require the hard sparsity constraint of Assumption 5.Source: §3.3 · p. 5

3.4 ERGODICITY OF THE TRUE PROCESS

  • Under Assumptions 1, 2, and 6, the segment process has a unique invariant measure and converges to it exponentially quickly.The delay-adapted dissipativity condition permits chaotic dynamics near the center while requiring the present restoring force to dominate destabilizing delayed effects.Source: §3.4 · p. 5

3.5 FINITE-SAMPLE IDENTIFIABILITY

  • When the empirical error is below the positive margin, every measurable empirical-objective minimizer exactly recovers the driver mask with probability at least 1 − δ.The drift is recovered up to an error bound; consistency jointly depends on observation duration, mesh size, and model-class complexity, not sample count alone.Source: §3.5 · p. 6

3.6 IMPLEMENTATION AND OPTIMIZATION

  • The implementation uses relaxed L0 penalization with sigmoid mask parameters and MLP drift functions, trained on one-step increments with an Euler integrator for regularly sampled data.Gradient standardization and a gradient penalty are also used; a restoring-force correction addresses the mismatch between bounded tanh MLPs and assumptions requiring an unbounded drift.Source: §3.6 · pp. 6–7

4 DATASETS AND EXPERIMENTS

The experiments assess driver identification on CausalDynamics and learned dynamics on Dysts, using separate datasets, training setups, and evaluation metrics.

4.1 CAUSALDYNAMICS DATASET

  • CausalDynamics spans Simple, Coupled, and Climate systems, covering noise, confounders, lagged interactions, and dimensions from 3 to 10.It contains more than 14,000 simulated systems, including two Climate datasets.Source: §4.1 · p. 7
  • Each CausalDynamics graph contributes 10 time series of 1,000 timesteps; methods are compared using AUROC, AUPRC, and SHD averaged by system class.Hyperparameters are tuned on a Lorenz84 graph that is excluded from results, and the same settings are used on other graphs.Source: §4.1 · p. 7

4.2 THE DYSTS BENCHMARK

  • Dysts evaluation uses 40 low-dimensional chaotic systems, each trained on one noisy 50-Lyapunov-timestep trajectory and tested on three 250-timestep trajectories.Test trajectories use different initial conditions; the systems are selected because their Jacobians are available for metric computation.Source: §4.2 · pp. 7–8
  • Dysts metrics assess short-term accuracy, attractor geometry and chaoticity, and long-term statistics, with median scores and average fractional ranks reported per method.The metrics are NRMSE and VLPT; FLEE and DKY; and LSD and Dstsp, respectively.Source: §4.2 · p. 8
  • For Dysts hyperparameter selection, five randomly chosen systems are used to maximize VLPT before evaluation on the remaining systems.Each method receives its own selected parameter set.Source: §4.2 · p. 8

5 RESULTS

L0-NDDE performs strongly on both driver-identification and dynamics benchmarks. Its CausalDynamics rankings and Dysts comparisons favor it over feature-selection competitors, while NODE also performs well on Dysts.

5.1 CAUSALDYNAMICS BENCHMARK

  • L0-NDDE has lower average rank on all three CausalDynamics metrics and generally higher AUROC/AUPRC than competing methods across Simple, Coupled, and Climate datasets.It consistently achieves low SHD, though its tuned hyperparameters produce overly dense graphs on Coupled datasets.Source: §5.1 · p. 8, Table 1 · p. 9

5.2 DYSTS BENCHMARK

  • On Dysts, L0-NDDE and NODE rank better than other methods on every metric except NRMSE.AGL and PathReg have extremely high median Dstsp, an artifact when the compared distributions are very different.Source: §5.2 · p. 8
  • DKY, Dstsp, and VLPT associate significantly with both SHD and training MSE, while L0-NDDE combines low SHD with low training error by masking irrelevant inputs.The reported associations do not establish causation.Source: §5.2 · p. 8
  • L0-NDDE ranks best in five Dysts metrics and consistently ranks ahead of other input-feature-selection methods, especially for DKY and Dstsp.Its difference from NODE is not significant; NODE also performs very well.Source: §5.2 · p. 8, Table 2 · p. 9

6 DISCUSSION

The discussion presents a theory-grounded framework as a foundation for robust lagged-system modeling, while identifying approximation, evaluation, deployment, and uncertainty limitations.

6 DISCUSSION

  • The method optimizes an approximate L0 penalty because the true L0 loss is nondifferentiable.The authors note that existing methods also rely on approximations.Source: §6 · p. 9
  • The two empirical benchmarks evaluate methods mostly on SDE-driven systems, and the authors report no comprehensive benchmark for lagged-system modeling.Source: §6 · p. 9
  • The method has not yet been applied to real physical systems, and its computational requirements scale quadratically with input dimension.High-dimensional application would require an identifiable dimensionality-reduction technique.Source: §6 · p. 9
  • The method learns probabilities for input-output mappings but a deterministic drift; a fully probabilistic extension could better represent uncertainty in long-term autoregressive rollouts.Source: §6 · p. 9

AI USE STATEMENT

The authors describe generative AI use in proof development, writing, implementation, and literature identification, with human review of the resulting work.

AI USE STATEMENT

  • Generative AI assisted with proof development, writing, method implementation, and literature identification, but not with synthetic data generation or research design.Source: AI USE STATEMENT · p. 10
  • The authors sketched proofs before asking an LLM to review them, then manually checked every suggestion before incorporating it.Source: AI USE STATEMENT · p. 10
  • The authors report verifying and testing experimental code and manually checking the identified literature and references.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The appendix outlines the proof sequence from well-posedness through population identifiability and probabilistic results to finite-sample recovery. It also records the notation and reproducibility details needed to follow those proofs and experiments.

REPRODUCIBILITY STATEMENT

  • Experiments use publicly available peer-reviewed benchmarks and CPU runs, with code, environment management, parameters, and metric computation documented.The implementation code is shared in Section 3.6; poetry manages the environment.Source: REPRODUCIBILITY STATEMENT · p. 10

A PROOFS

  • The proof sequence establishes well-posedness, population identifiability under exact-risk and penalized objectives, then probabilistic results leading to finite-sample recovery.The probabilistic steps include uniform moment bounds, ergodicity, and exponential mixing of the sampled lagged state.Source: A PROOFS · p. 17

A.1 NOTATION

  • The drift depends on the lagged state Z_t = (X_t, X_{t−τ1}, …, X_{t−τq}), with q = 0 denoting the no-delay case.The effective horizon is T° = T − τ_max.Source: §A.1 · p. 17

A.2 NON-EXPLOSION AND EXISTENCE OF A STRONG SOLUTION

  • Under Assumptions 1 and 2, the SDDE has a unique strong solution for every permitted initial condition, with no explosion on finite time intervals.Existence is extended by induction over intervals of length τ_min.Source: §A.2 · pp. 18, 20

A.3 POPULATION IDENTIFIABILITY

This section establishes population identifiability of the drift and driver mask, including for a penalized objective under a minimum-signal condition. It also derives moment bounds for the segment process across uniform-in-time, stationary, and finite-horizon settings under the stated assumptions.

A.3 POPULATION IDENTIFIABILITY

  • Under Assumptions 1–5, the drift G and mask M are identifiable everywhere: non-degenerate noise gives full support, while faithfulness and sparsity rule out dropped and spurious edges.Risk optimality identifies the drift on the occupation-measure support; full support extends equality globally.Source: §A.3 · pp. 20–22

A.4 IDENTIFIABILITY WITH THE PENALIZED OBJECTIVE

  • With a nonzero mask, measurable impact of every true driver, and the sufficient λ-window, every penalized-objective minimizer recovers M and G globally without Assumption 4 (faithfulness).The minimum-signal condition smin > 0 makes dropping a true input strictly costly.Source: §A.4 · pp. 22–23

A.5 UNIFORM MOMENT BOUNDS

  • Under Assumption 6, the radial and residual bounds combine into a global dissipativity estimate when the total delayed-feedback bound B is below β0.The lemma sets ba = βa + εa and B = Σa ba < β0.Source: §A.5 · pp. 23–24
  • For every integer p ≥ 1 with finite initial-segment 2p moment, the process has a one-point moment bound whose constants depend on the noise, delays, and assumption bounds.The proof uses Itô’s formula and Halanay comparison to handle delayed feedback.Source: §A.5 · pp. 24–25
  • The one-point bound is lifted to the segment norm, with stationary and finite-horizon consequences recorded for the same range of p.The segment bound is stated under the assumptions of Lemma 3.Source: §A.5 · pp. 25–26
  • If the segment process admits an invariant probability measure, that measure has finite Up moment without requiring a prior moment assumption.The argument applies separately for each p.Source: §A.5 · pp. 26–27
  • The segment process has a uniform-in-time moment bound, established without any growth assumption on G.Taking the supremum over t ≥ 0 yields the bound for M2p(ϕ).Source: §A.5 · p. 27
  • For each fixed horizon H, the finite-horizon moment bound is uniform over initial segments in any ball BR.Its constant depends on p, H, the delays, and the constants of Lemma 3.Source: §A.5 · pp. 26–27

A.6 ERGODICITY OF THE TRUE PROCESS

Under Assumptions 1, 2, and 6, the segment process has a unique invariant probability measure and converges geometrically. The proof uses transition overlap to obtain this ergodicity, which supports the paper’s finite-sample analysis.

A.6 ERGODICITY OF THE TRUE PROCESS

  • For histories in bounded sets, a control steers one process copy toward another; after h > τ_max their segments coincide, enabling transition overlap.Invertible noise and Girsanov’s theorem bound the change-of-measure cost.Source: §A.6 · p. 28
  • Under Assumptions 1, 2, and 6, the segment process has a unique invariant probability measure and geometric convergence in a bounded Wasserstein metric.The theorem also establishes polynomial moments under the invariant measure.Source: §A.6 · p. 29
  • Geometric ergodicity stabilizes identifiability constants as T grows, while finite-sample analysis instead assumes finite stationary drift energy.Total-variation convergence alone does not control squared errors for an unbounded drift.Source: §A.6 · p. 30

A.7 MIXING OF THE SAMPLED PROCESS

Theorem 3’s geometric ergodicity in total variation yields exponential β-mixing for the segment process, its lagged state, and deterministic subsamples. This supplies the mixing control needed for finite-sample analysis without assumptions beyond Theorem 3.

A.7 MIXING OF THE SAMPLED PROCESS

  • The mixing bound follows from Theorem 3’s total-variation geometric ergodicity; additive non-degenerate noise makes transition kernels overlap after h > τmax.No assumption beyond those of Theorem 3 is needed, and the non-delayed case q = 0 is included.Source: §A.7 · pp. 30–32
  • The finite-sample analysis needs the sampled lagged state to forget its past exponentially, and Wasserstein contraction alone would not suffice for this total-variation requirement.The proof bounds the segment process first, then passes to the lagged state and its deterministic subsamples.Source: §A.7 · pp. 30–31
  • Exponential β-mixing holds for the segment process and lagged state, with bound Cβe^−γt for every t > 0.The same rate applies to the lagged state restricted to times t ≥ τmax.Source: §A.7 · p. 31
  • Deterministic subsamples inherit exponential β-mixing without regular grid spacing; if every spacing is at least Δmin, the bound decays as Cβe^−γmΔmin.The result follows by passing from the segment process to the lagged state and deterministic subsamples by monotonicity.Source: §A.7 · p. 31

A.8 FINITE-SAMPLE IDENTIFIABILITY

For stationary finite-grid observations, this section develops finite-sample identifiability guarantees for learned delayed systems, including high-probability exact support recovery by an L0 estimator under a margin condition and a stationary total-variation error bound under stated assumptions. It also gives conditions for ergodicity and compares the L0 approach—which avoids an initial estimator but assumes an optimization oracle—with adaptive group lasso guarantees that likewise assume an oracle.

A.8 FINITE-SAMPLE IDENTIFIABILITY

  • Under the stated assumptions and margin condition, every measurable L0 minimizer recovers the exact driver mask with probability at least 1 − δ, with a bound on prediction error once the masks agree.The guarantee identifies the mask, not the drift exactly from finitely many observations.Source: §A.8 · pp. 33, 36
  • Consistency requires a joint condition on observation duration, mesh, and class complexity; large sample counts alone do not suffice, and the stated bound is qualitative rather than a sharp rate.The finite-sample statement assumes exact lag observations; interpolated lag values require additional error analysis.Source: §A.8 · pp. 32, 33, 36

A.9 DISSIPATIVITY AND ERGODICITY OF THE LEARNED SYSTEM

  • An unconstrained tanh MLP drift is globally bounded and non-explosive, but cannot meet the stated exterior dissipation condition, so Theorem 6 does not apply automatically.The passage notes that bounded drifts can still be ergodic, giving dX_t = −tanh(X_t) dt + σ dW_t as an example.Source: §A.9 · p. 37
  • When the learned drift has bounded error from a dissipative true drift under the theorem’s conditions, its SDDE is non-explosive and has a unique invariant law, polynomial stationary moments, and geometric total-variation convergence.The hypotheses are structural conditions and do not automatically follow from fitting an unconstrained neural network.Source: §A.9 · p. 37
  • Adding linear damping to a globally bounded neural residual guarantees non-explosion, a unique invariant law, polynomial stationary moments, and geometric total-variation convergence.Post-training exterior confinement can leave the fitted drift unchanged inside a chosen box, but generally changes the learned process’s invariant law.Source: §A.9 · p. 38
  • Under the stated conditions, the learned SDDE admits a stationary total-variation error bound relative to the true invariant segment law.The setup assumes a stationary true process and a learned non-explosive SDDE with the same lags and diffusion; the proof uses stopped change of drift, Girsanov’s theorem, and Pinsker’s inequality.Source: §A.9 · pp. 38–39

B COMPARISON WITH ADAPTIVE GROUP LASSO ASSUMPTIONS

  • Compared with Bellot et al. (2022), the L0 approach extends identifiability guarantees to SDDE-driven systems and needs no initial consistent estimator, with an explicit population-theorem λ window.Both analyses assume an optimization oracle; the paper reports that the relaxed L0 penalty performs better than the AGL-penalized objective on Markovian systems in the CausalDynamics and Dysts benchmarks.Source: B COMPARISON WITH ADAPTIVE GROUP LASSO… · p. 40

C.1 IMPLEMENTATION DETAILS

The implementation relaxes L0 penalization for score-based optimization and uses gradient regularization for the learned functions. The benchmark protocol defines metric computation, tunes only the sparsity penalty coefficient, and reports the final hyperparameter setup.

C.1 IMPLEMENTATION DETAILS

  • L0-NDDE uses hard-concrete samples of a sigmoid-parameterized mask, then thresholds mask probabilities at 0.5 for inference.Training uses β = 0.33, multiple mask samples per batch, and a 100-iteration warm-up before adding the penalty.Source: §C.1 · p. 40

D.1 METRIC COMPUTATION

  • SHD, AUROC, and AUPRC use the mask probabilities for L0-NDDE but feature weights for AGL and L1 methods.For SHD, weighted methods use a 10−8 threshold; changing it did not affect performance because weights were generally 0 or above 10−3.Source: §D.1 · pp. 40–41

D.2 HYPERPARAMETER SEARCH

  • The experiments keep default method parameters and tune only the sparsity penalty coefficient, selected by maximizing AUROC on Lorenz84.The search evaluates the listed penalty coefficients on 10 Lorenz84 time series.Source: §D.2 · p. 41

D.3 FINAL HYPERPARAMETERS

  • The final hyperparameter table compares L0-NDDE, C-NODE, AGL, and No grad. pen., with penalty coefficients as the only method-specific tuned values.No grad. pen. does not use a gradient penalty; the implementation uses a 2-layer, 8-unit-per-layer architecture and βtemp = 0.33 for L0 methods.Source: Table 3 · p. 41, §D.3 · p. 41

D.4 FULL CAUSALDYNAMICS RESULTS

The full CausalDynamics results favor L0-NDDE over competing methods most of the time, and the gradient-penalty ablation reports a slight advantage for including the penalty.

D.4 FULL CAUSALDYNAMICS RESULTS

  • L0-NDDE outperforms competing methods most of the time on the expanded CausalDynamics benchmark results.Table 4 adds methods to the Table 1 comparison.Source: Table 4 · p. 42

D.5 NO GRADIENT PENALTY

  • L0-NDDE is slightly stronger with gradient penalty, which helps avoid overfitting spurious correlations.The caption also reports slightly better CausalDynamics benchmark performance with the penalty.Source: Table 5 · p. 42

E.1 TRAIN AND TEST SYSTEMS

The evaluation uses 40 Dysts systems, with separate hyperparameter-search and test sets, and assesses prediction, attractor geometry, and long-term statistics. Results include associations between training error or driver recovery and several performance metrics, alongside example trajectory comparisons.

E.1 TRAIN AND TEST SYSTEMS

  • 40 Dysts systems are split into 5 hyperparameter-search systems and 35 test systems, with one training and three test trajectories per system.Training trajectories span 50 Lyapunov times and are constrained to 1,000–10,000 timesteps; test trajectories span 250 Lyapunov times.Source: §E.1 · p. 42

E.2 METRICS

  • Short-term prediction is assessed with VLPT and NRMSE, while FLEE and DKY error measure reconstruction of geometry and attractor properties.FLEE is the squared error in the first Lyapunov exponent; DKY error is the squared difference in Kaplan–Yorke dimension.Source: §E.2 · p. 43
  • Long-term statistics are assessed with LSD for spectral differences and Dstsp for divergence between learned and true invariant distributions.Dstsp estimates KL divergence from predicted and true trajectories.Source: §E.2 · pp. 43–44

E.3 RESULTS

  • Figure 2 compares predicted and true trajectories on 26 Dysts systems for L0-NDDE, NODE, and C-NODE.The caption identifies the methods from left to right and the predicted and true trajectory colors as red and black.Source: Figure 2 · p. 45
  • DKY and Dstsp increase, while VLPT decreases with SHD and training error; these associations are statistically significant by Kendall-τ tests.LSD is also significantly associated with training error.Source: Figure 3 · p. 46

Related papers

Loading 2609.37944v3…