PaperlayerView exact revision on arXiv

Source-linked AI summary

Data-Driven Learning of Unknown Nonlinear Differential Equations Using Functional Analysis

Seyyed Shaho Alaviani, Yongzhi Qu, Gregory W. Vogl

arXiv:2609.04329v1cs.LGcs.AIcs.SYeess.SY

TL;DR

Nonlinear ODE discovery seeks data-derived models of systems whose dynamics are challenging to characterize. The paper introduces FINS, an interpretable functional-analysis approach using an integral function-space cost and incremental learning. It reports recovery capabilities for polynomial vector fields from one trajectory, including simultaneous learning of dynamics and external inputs.

  • Problem

    Discovering nonlinear ODE models from data remains challenging because nonlinearity is characteristic of many scientific and engineering systems.

    Source: §1 · p. 2

  • Method

    FINS formulates vector-field learning using functional analysis and operator theory, an integral function-space cost, and incremental updates from training samples.

    Source: §1 · pp. 2–3

  • Results

    FINS can recover exact physics for finite-degree polynomial vector fields from one trajectory and can learn unknown dynamics alongside external inputs in forced, unforced, and time-varying systems.

    Source: §5 · p. 34

  • Takeaways & Limitations

    The reported capabilities offer an interpretable data-driven modeling route across the stated polynomial and forcing settings.

    Source: §5 · p. 34

    Addressing the method’s curse of dimensionality remains future work.

    Source: §5 · p. 34

Abstract

from arXiv · show

In this paper, the problem of data-driven discovery of nonlinear ordinary differential equations (ODEs) is recast, and a new interpretable machine learning (ML) method is proposed. The proposed method aims to learn the unknown vector field of nonlinear dynamics without prior knowledge of the system's physics from only one single state trajectory's data. The proposed method has two fundamental differences with existing methods: 1) the formulation presented in this method is derived based on Functional Analysis and Operator Theory, and 2) the cost function is constructed in the function space as a distance between two functions as an integral, instead of the discrete-sum of errors used in existing ML approaches. An incremental learning algorithm is proposed to learn the unknown vector field to handle new training samples in an online manner. The proposed method can discover the unknown vector field from both forced and unforced autonomous and non-autonomous (or time-varying) dynamical systems. The proposed method is able to simultaneously discover unknown external forces as a function of time and unknown underlying dynamics. Finally, numerical examples are given to demonstrate the advantages of the proposed method.

1 Introduction

The introduction frames nonlinear ODE discovery as important across scientific and engineering applications, then presents FINS, an interpretable method built around functional analysis and an integral function-space cost. It positions this approach among existing methods and notes its capabilities and measurement requirements.

1 Introduction

  • ODE discovery is motivated by nonlinear dynamics across scientific and engineering fields, where data-derived models can support control, estimation, performance analysis, and prediction.The introduction cites circuits, epidemiology, neuroscience, medicine, and biological systems as examples.Source: §1 · pp. 1–2
  • Interpretable machine learning addresses the opacity of black-box models, while incremental learning is presented as relevant to memory, privacy, and sustainability constraints.The paper describes incremental learning as updating, accumulating, acquiring, and using knowledge.Source: §1 · p. 2
  • FINS uses functional analysis and an integral function-space cost rather than discrete-sum errors, with incremental learning for new samples and single-trajectory learning for polynomial vector fields.The paper presents this as an interpretable formulation that does not require prior knowledge of the system physics.Source: §1 · pp. 2–3, Figure 1 · p. 3
  • The method targets forced and unforced, autonomous and time-varying systems, can learn external forces alongside dynamics, and requires full-state or full input-state measurements.The introduction also says learning can use zero or nonzero initial conditions.Source: §1 · p. 4

1.1 Related Works

  • Prior approaches include sparse regression, symbolic regression, neural models, Koopman methods, and neural operators, each with stated limitations such as candidate-function choice, numerical-derivative sensitivity, or data demands.The cited discussion also notes challenges in selecting Koopman embedding functions and applying many existing autonomous-system methods to time-varying systems.Source: §1.1 · pp. 4–7
  • Neural-network approaches can be opaque, data-intensive, and architecture- or initialization-dependent, while several interpretable neural alternatives discussed are limited to autonomous systems.The paper also reports memory inefficiency, long training times, and suboptimal results for NODEs.Source: §1.1 · pp. 6–7

2 Preliminaries

The preliminaries establish the paper’s notation and introduce the operator and polynomial concepts used in its formulation.

2 Preliminaries

  • The notation defines real and natural numbers, vector and matrix spaces, derivatives, gradients, transposes, and vector norms.The passages also define set inclusion, Cartesian products, approximation, and vector cardinality notation.Source: §2 · p. 7
  • A Hammerstein operator is defined using a known nonlinear mapping and kernel, with an integral form over time also given.The definition specifies the mapping as f and the kernel as K.Source: §2 · pp. 7–8
  • The preliminaries introduce orthogonal polynomials in L2 and give the first three Hermite polynomials as examples.The listed examples are H1(y) = 2y, H2(y) = 4y^2 − 2, and H3(y) = 8y^3 − 12y.Source: §2 · p. 8

3 Main Results

The section formulates ODE discovery from a single partially observed trajectory as a function-space inverse problem and presents FINS, an incremental polynomial approximation algorithm, alongside sparse and non-polynomial cases and computational limits. It also reports the algorithm’s practical efficiency, comparable to SINDy, and gives a special case with complexity O(nτ²).

3.1 Problem Formulation

  • The formulation casts learning the unknown vector field from a partially observed trajectory as a function-space least-squares inverse problem using a linear operator.It applies to autonomous or nonautonomous systems, and the formulation also permits the external force to be absent.Source: §3.1 · pp. 8–11

3.2 A Proposed Algorithm

  • FINS approximates the optimal solution with finite-degree Hermite polynomials and incrementally updates its learning matrices as samples arrive.It is an indirect method that does not require estimating time derivatives; accuracy depends on the integral approximation and sampling intervals, and finite polynomial degrees limit what it can represent. The approach also has a curse-of-dimensionality limitation, though it can exactly discover hidden inputs represented by finite-degree polynomials.Source: §3.2 · pp. 11–14

3.2.1 Sparsity-Promoting FINS (FINS s)

  • FINS s seeks sparse polynomial vector fields because the authors expect most physical systems to have only a few relevant terms.It uses l0-norm minimization, which is non-convex and NP-hard; the passage describes convex relaxation and iterative thresholding as approximate solution approaches.Source: §3.2.1 · pp. 14–15

3.2.2 Discussion on Non-Polynomial Vector Fields

  • The framework includes the Mathieu equation, but finite Hermite-polynomial approximations cannot fully discover non-polynomial vector fields.They can recover polynomial components when present if the selected Hermite degrees are at least as high as those components’ degrees.Source: §3.2.2 · p. 15

3.3 Computational Complexity of Algorithm 1

  • Algorithm 1’s time complexity increases linearly with the number of training samples, while its estimated coefficient count grows exponentially with state dimension.The authors identify high-dimensional systems as a challenge and describe distributed computation as a way to make the algorithm computationally tractable in that setting.Source: §3.3 · p. 16
  • When µ_i = 1 for all i, the stated complexity reduces to O(nτ^2), and the referenced algorithm has space complexity O(τ^2).Source: §3.3 · p. 16
  • Algorithm 1’s computational time is comparable to SINDy’s, while unlike SINDy it learns incrementally.Its computational time is evaluated with Duffing, Lorenz, and Lorenz 96 systems in Subsection 4.1.Source: §3.3 · p. 16

4 Demonstration on Numerical Examples

The numerical section tests Algorithm 1 on several systems, including Duffing and Van der Pol dynamics. It reports a Duffing prediction-error advantage for FINS without noise and greater robustness than sparse FINS and SINDy with noisy data.

4 Demonstration on Numerical Examples

Example 1: Discovering Duffing Equation

Example 2: Discovering Van der Pol Equation

Example 3: Discovering Duffing Equation with Known External Force

The examples test FINS on forced dynamics, including a case with a hidden external input, and examine how polynomial-order choice affects learning. For a non-polynomial input, FINS cannot recover the input exactly.

Example 3: Discovering Duffing Equation with Known External Force

Example 6: Discussion on Non-Polynomial Hidden External Input

  • For a non-polynomial hidden input, FINS cannot recover the input exactly; the sinusoidal-input test used k(30010)=11,609,944 samples.The authors used a large dataset to examine how FINS handles this case.Source: Example 6: Discussion on… · pp. 26–27

Example 7: Discussion on Non-Polynomial Vector Field

The examples show that FINS can recover represented polynomial components of a non-polynomial vector field, while its computational profile differs between full training and incremental updates. Comparisons with network-based methods report faster training for FINS and trajectory-length-dependent convergence for SymNN.

Example 7: Discussion on Non-Polynomial Vector Field

  • In the non-polynomial example, FINS s's learned ydot dynamics are highly dependent on the sparsity threshold, unlike its xdot dynamics across suitable thresholds.The reported learned ydot expression includes constant, time, state, and interaction terms.Source: Example 7: Discussion on… · pp. 27–28

4.2 Evaluation of Computational Time

  • The paper compares computational time in seconds over 25 independent runs, including evaluations by state dimension n, with N denoting the highest degrees of SINDy monomials and Hermite polynomials.For Table 16, the cardinality of Dt is fixed for training time; Figures 13 and 14 are bar plots of Tables 14 and 16.Source: Table 14 · p. 29, Table 16 · p. 30, Figure 13 · p. 31

Example 7: Discussion on Non-Polynomial Vector Field

  • For non-polynomial vector fields, FINS and FINS s recover polynomial components represented in the finite Hermite span but cannot recover the full field exactly.The reported xdot = y component is recovered when the selected Hermite degrees cover its polynomial degree.Source: Example 7: Discussion on… · pp. 29–30

4.2 Evaluation of Computational Time

  • With fixed training-set cardinality, SINDy is faster than the proposed method for computation, while FINS's update time per new datum is significantly low.The incremental update comparisons are not directly comparable with existing methods because their incremental learning capability had not been shown in the literature.Source: Table 14 · p. 29, §4.2 · pp. 30–31, Table 16 · p. 30

4.3 Evaluation and Comparison with the State-of-the-Art Network-Based Algorithms

  • On Takens–Bogdanov data, SymNN did not converge with a 1-second single trajectory while the proposed method converged; SymNN could converge with 4 seconds of data.The comparison used a single trajectory without noise.Source: §4.3 · p. 33
  • Network-based methods using gradient-descent optimization were at least 105 times slower than FINS in the reported training-time comparisons.The authors identify this efficiency difference as potentially relevant to real-time applications.Source: §4.3 · p. 34

5 Conclusions and Discussion

The conclusion presents FINS as an interpretable functional-analysis-based method for learning nonlinear ODEs from a single state trajectory, including forced and time-varying systems. It also identifies unresolved questions about accuracy, noise robustness, dimensionality, convergence, and basis choice.

5 Conclusions and Discussion

  • FINS can discover exact underlying physics for finite-degree polynomial systems from one state trajectory and can learn unknown dynamics and external inputs simultaneously.The conclusion also states that results apply to time-varying forced and/or unforced systems.Source: §5 · p. 34
  • The method uses a continuous-sum training cost, which the conclusion says allows incremental learning algorithms.Its formulation is based on Functional Analysis and Operator Theory.Source: §5 · p. 34
  • Open research issues include proving finite-time accuracy and noise robustness, addressing dimensionality, studying infinite-data convergence, and comparing other orthogonal bases.These are listed as future research subjects by the authors.Source: §5 · p. 34

Appendix A.

Appendix A completes a proof by establishing that the operator T(f) defined in equation (8) is linear.

Appendix A.

  • The proof requires showing that the operator T(f) defined in equation (8) is linear.Source: Appendix A · p. 35
  • The appendix states that the required implication establishes the operator's linearity.Source: Appendix A · p. 35
  • The text concludes that the proof of Lemma 1 is complete.Source: Appendix A · p. 35

Appendix B.

Appendix B uses Hermite polynomials as an orthogonal basis and finite-degree approximations, then invokes Hilbert-space strict convexity to establish an equivalence.

Appendix B.

  • Hermite polynomials form an orthogonal basis in the stated L2 space.Source: Appendix B · p. 35
  • The appendix represents functions as infinite Hermite-polynomial sums and approximates f with polynomials of finite degree.It also gives a compact form for the finite expansion.Source: Appendix B · pp. 35–36
  • Strict convexity of the stated Hilbert space is used to conclude that equations (35) and (14) are equivalent.Source: Appendix B · p. 36

Appendix C.

Appendix C completes the proof of Theorem 1 by reducing optimization problem (14) to (16). The proof uses the definitions of Qk, Pk, and Uk.

Appendix C.

  • The appendix defines equivalence as equality of the optimal solutions of the two optimization problems.Source: Appendix C · p. 36
  • The proof obtains results from (36) and (15), from (37), (38), and (15), and from (39); Qk and Pk are defined in (17) and (18), respectively.Source: Appendix C · p. 37
  • Solving (14) reduces to solving (16), completing the proof of Theorem 1.The reduction follows from equations (36)–(42).Source: Appendix C · p. 37

Appendix D.

Appendix D proves Corollary 1 by establishing convexity of the cost in (16) and applying first-order optimality to its unconstrained minimization.

Appendix D.

  • Because Qk is positive semidefinite, the cost function in (16) is convex.Source: Appendix D · p. 37
  • The unconstrained minimization of (16) yields an optimal solution through the first-order optimality condition.Source: Appendix D · p. 38
  • The resulting condition implies (21), using that Qk is block-diagonal and symmetric.Source: Appendix D · p. 38

Appendix E.

Appendix E proves Corollary 2 through substitutions and integral approximations, including calculations that use the known function w(t).

Appendix E.

  • The proof proceeds by substituting expressions from (44)–(50) and using relations involving K(z), defined in (15).Source: Appendix E · pp. 38–40
  • The appendix approximates K(tk) from dataset Dt, with K(t0) set to zero.Source: Appendix E · p. 38
  • Since w(t) is known, the integrals in (48) can be computed analytically or approximated using (23).Source: Appendix E · pp. 39, 41

Appendix F.

Appendix F identifies updates to Q̃k and P_lk in Algorithm 1.

Appendix F.

Loading 2609.04329v1…