PaperlayerView exact revision on arXiv

Source-linked AI summary

Discovering Governing Equations from Partial Measurements with Deep Delay Autoencoders

Joseph Bakarji, Kathleen Champion, J. Nathan Kutz, Steven L. Brunton

arXiv:2201.05136v1cs.LGcs.CEmath.DS

TL;DR

Partial measurements hide dynamically important states, and learning interpretable dynamics directly from delay coordinates remains challenging. The paper combines time-delay embedding, a deep autoencoder, and SINDy to learn transformed coordinates and sparse governing equations. It demonstrates this approach on canonical systems and chaotic-waterwheel video, while identifying optimizer overfitting and local minima as an unresolved limitation.

  • Problem

    When measurements omit dynamically important states, closed-form dynamics generally cannot be learned directly in the measured variables, and useful coordinates for modeling delay embeddings remain unknown.

    Source: §1 · p. 1, §1.1 · p. 4

  • Method

    The method delay-embeds partial measurements and jointly learns a coordinate transformation with sparse dynamics using a deep SINDy-autoencoder framework.

    Source: §1 · p. 3, §1.1 · p. 4

  • Results

    6 terms versus the original Lorenz system’s 7 were learned from its first coordinate, and a Lorenz analogue was learned from chaotic-waterwheel video.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The results show that sparse, interpretable nonlinear dynamics and associated coordinates can be learned from incomplete or partial measurements.

    Source: §5 · p. 16

    The optimizer can overfit or become trapped in local minima, producing large estimation errors; suitable loss-function regularization remains an open question.

    Source: §5 · p. 17

Abstract

from arXiv · show

A central challenge in data-driven model discovery is the presence of hidden, or latent, variables that are not directly measured but are dynamically important. Takens' theorem provides conditions for when it is possible to augment these partial measurements with time delayed information, resulting in an attractor that is diffeomorphic to that of the original full-state system. However, the coordinate transformation back to the original attractor is typically unknown, and learning the dynamics in the embedding space has remained an open challenge for decades. Here, we design a custom deep autoencoder network to learn a coordinate transformation from the delay embedded space into a new space where it is possible to represent the dynamics in a sparse, closed form. We demonstrate this approach on the Lorenz, R\"ossler, and Lotka-Volterra systems, learning dynamics from a single measurement variable. As a challenging example, we learn a Lorenz analogue from a single scalar variable extracted from a video of a chaotic waterwheel experiment. The resulting modeling framework combines deep learning to uncover effective coordinates and the sparse identification of nonlinear dynamics (SINDy) for interpretable modeling. Thus, we show that it is possible to simultaneously learn a closed-form model and the associated coordinate system for partially observed dynamics.

1 Introduction

Partial measurements can leave hidden states that prevent closed-form models in measured variables; time-delay embedding can recover an attractor, but the coordinate transformation needed to model its dynamics remains unknown. The work combines delay embedding, a deep autoencoder, and SINDy to learn sparse dynamics in transformed coordinates.

1 Introduction

  • Takens’ theorem [7] gives conditions under which delayed measurements yield an attractor diffeomorphic to the full-state attractor, but the transformation back to useful coordinates is unknown.Learning interpretable, generalizable dynamics in delay coordinates has remained an open challenge.Source: §1 · p. 1
  • 6 terms versus the original Lorenz system’s 7 were learned from its first coordinate, and a Lorenz analogue was learned from chaotic-waterwheel video.The Lorenz model used fewer terms than the original system.Source: §1 · p. 3

1.1 Mathematical problem definition

  • With partial measurements, hidden variables can make the measured-variable dynamics impossible to represent as a closed-form model, creating the closure problem.The challenge is that the measured dimension is smaller than the full-state dimension.Source: §1.1 · p. 4
  • The method delay-embeds scalar measurements, then jointly learns a coordinate map and interpretable dynamics using a modification of the SINDy-autoencoder framework.The delay vector contains successive measurements separated by a chosen time increment; Takens’ guarantee requires embedding dimension n > 2m.Source: §1.1 · p. 4

2 Background

The background introduces time-delay coordinates and SINDy, then describes how autoencoders can learn coordinates in which sparse dynamics are discoverable. The combined model uses reconstruction, derivative-consistency, and sparsity losses.

Delay embedding

  • Time-delay measurements can be assembled into a Hankel matrix; SVD yields eigen-time-delays, whose leading components can encode signal shape.Their rank may be selected by Pareto-front analysis.Source: §2 · p. 5

SINDy

  • SINDy identifies governing equations as sparse regressions over candidate functions, using an L1 penalty or sequential thresholding to select active terms.The sparse coefficients specify which library terms enter the dynamics.Source: §2 · p. 5

SINDy autoencoders

  • Autoencoders can transform input coordinates before equation discovery when the candidate function library is poorly informed or includes operators such as non-local closures.The paper extends this approach to discovering high-dimensional equations from low-dimensional measurements.Source: §2 · p. 5
  • The SINDy-autoencoder combines reconstruction with latent-derivative agreement, measurement-derivative consistency, and coefficient sparsity.The latent derivative is compared with the chain-rule derivative, while the added delayed-input problem requires further loss terms.Source: §2 · pp. 5–6

3 Delay SINDy Autoencoders

The delay SINDy autoencoder learns coordinates and sparse dynamics from time-delay embeddings by adding losses that exploit embedding properties. Its objective is a predictive model with a sparse analytical form, while the loss choices and weights remain non-unique.

3 Delay SINDy Autoencoders

  • Takens' theorem guarantees a diffeomorphic map between the delay-embedded and original attractors only under certain conditions, but does not specify how to find it.The design objective is to approximate this mapping into coordinates where sparse dynamics can be represented.Source: §3 · p. 7
  • The method extends SINDy autoencoders with three losses that exploit time-delay embedding properties to learn coordinates and dynamics from delayed measurements.The target is a predictive model with a sparse analytical form.Source: §3 · p. 7
  • When n ≫ 2m, projecting the Hankel embedding onto p dominant SVD modes preserves most signal variance and can filter noise while improving computational efficiency.This linear projection is an optional preprocessing step before the nonlinear autoencoder.Source: §3 · p. 7
  • Constraining the first latent component to equal the measured signal reduces component permutations from m! to (m − 1)! and ties latent dynamics to that measurement.The measured variable typically has physical meaning because it is experimentally observed.Source: §3 · p. 9
  • The SINDy consistency loss integrates latent dynamics at the input sampling rate and matches the trajectory's first component to the delay embedding.The integration is implemented as a recurrent layer using an RK4 scheme.Source: §3 · p. 9
  • Loss terms and their weights are not unique; known system properties such as momentum or energy conservation can provide additional guidance.The method constrains the hypothesis class to make the inverse problem computationally feasible and the solution predictive of measurements.Source: §3 · p. 9

4 Results

The experiments recover Lorenz-like sparse dynamics from delay-embedded measurements and test the approach across canonical systems and waterwheel video. Results also show that initialization, optimization choices, and measurement suitability affect discovery.

4 Results

  • The experiments progressively optimize the maps and dynamics with known state, known equations, and then jointly learned coefficients; expanding the hypothesis class makes optimization harder.The study applies the full algorithm to Lorenz, Lotka–Volterra, Rössler, and waterwheel dynamics, while noting that measurement choice and discovered coordinates need not be unique.Source: §4 · p. 10

4.1 The Lorenz System

  • With known full state or known dynamics, the first two subproblems are solved with high accuracy, though small SVD rank p = 10 produces large errors near Lorenz lobe switches.The figure caption reports a computational advantage for the SVD layer at the cost of lower prediction accuracy.Source: §4.1 · p. 11, Figure 6 · p. 11
  • Initializing from dominant Hankel modes yields solutions closer to optimal than random initialization, particularly by making the learned maps smoother.The authors describe this initialization as transfer learning for data-driven modeling.Source: §4.1 · p. 12, Figure 7 · p. 12
  • Large coefficient perturbations can still yield Lorenz-like two-lobe attractors, but random initialization converges sporadically; periodic SINDy optimization improves sparsity and efficiency.The reported periodic refits use intervals of approximately 20–40 epochs.Source: §4.1 · pp. 12–13
  • The discovered Lorenz system has a similar attractor but is sparser, with 6 terms rather than 7 and 4 nonlinear terms rather than 2.Its latent representation is nearly a scaling and translation of the original attractor.Source: §4.1 · p. 13, Figure 9 · p. 14

4.2 Other systems

  • Lotka–Volterra discovered models are structurally similar, while the less-sparse Rössler model accurately captures bursting in the third dimension.The Lotka–Volterra attractor comprises limit-cycle families that depend on parameters and initial conditions.Source: Figure 9 · p. 14, §4.2 · p. 15
  • From waterwheel video, the model reproduces the rough attractor shape and chaotic dynamics, requiring third-degree polynomial features.Spoke angles are processed into angular velocity as the input measurement.Source: §4.2 · p. 15, Figure 10 · p. 15

5 Discussion

The discussion presents delay-embedding, autoencoders, and SINDy as a framework for discovering interpretable models from partial measurements. It reports that Lorenz-like models can be sparser than the original equations, while noting unresolved recovery and optimization limitations and proposing future extensions.

5 Discussion

  • The discovered Lorenz-like models use six terms versus seven in the original Lorenz equations, but have four quadratic terms versus two.The added sparsity therefore comes with more quadratic terms.Source: §5 · p. 16
  • The framework combines time-delay embedding, deep-autoencoder coordinate transformations, and SINDy to discover sparse, interpretable nonlinear systems from partial measurements.The paper frames this as explicitly learning a map from delay coordinates to coordinates where the dynamics can be efficiently represented.Source: §5 · p. 16
  • Perturbing the original equation coefficients at initialization can leave the optimizer over-fitting or trapped in local minima, producing large estimation errors.The authors leave suitable loss-function regularization as an open question.Source: §5 · p. 17
  • It remains unknown whether random parameter initialization can recover the original equations exactly from one-dimensional measurements.The authors suggest problem-specific symmetries or an invertible network as possible constraints, and note potential improvements to the L1-based SINDy algorithm.Source: §5 · p. 17
  • The authors propose a general framework for multidimensional model discovery from low-dimensional data and identify convolutional autoencoders as a future direction for video.They call for further theoretical and numerical investigations of combining delay embedding, SINDy, and autoencoders.Source: §5 · p. 17

A Delay embedding hyperparameters

The section describes how delay and embedding dimensions are selected, and why coordinate learning is combined with SINDy. The chosen delay aims to unfold the attractor, while dominant Hankel modes guide latent dimension and reconstruction-only coordinates may not yield sparse models.

A Delay embedding hyperparameters

B Lorenz waterwheel model

The waterwheel example uses video-derived angular velocity as a measurement for discovering Lorenz-like dynamics. The optimized model recreates chaotic dynamics, while sparse, noisy, low-frame-rate, and short-duration data make equation identification challenging.

B Lorenz waterwheel model

  • A third-order polynomial dictionary with n = 32 and a four-layer encoder/decoder recreates the Lorenz system’s chaotic dynamics from the waterwheel data.The discovered attractor has the same two-lobe structure as the original Lorenz system.Source: B Lorenz waterwheel model · p. 19
  • Sparse data, line-detection measurement noise, 24-frames-per-second video, and limited duration make identifying the differential equation challenging.The learned coefficients are presented in Table 1; its caption identifies them as coefficients of the Lorenz waterwheel system.Source: B Lorenz waterwheel model · p. 19, Table 1 · p. 19
  • The authors expect that more data and improved video-to-angular-velocity mapping could yield a sparser, more generalizing analytical model.They propose exploring machine-learning methods such as CNNs to incorporate that mapping into the SINDy autoencoder.Source: B Lorenz waterwheel model · p. 19
Loading 2201.05136v1…