Source-linked AI summary
Weak SINDy: Galerkin-Based Data-Driven Model Selection
Daniel A. Messenger, David M. Bortz
TL;DR
The paper studies data-driven recovery of sparsely represented nonlinear dynamics from noisy measurements. It develops a weak, data-driven Galerkin formulation and reports effective machine-precision recovery in the noise-free setting, with more predictable noise–coefficient-error behavior and smaller linear systems.
Problem
The paper addresses solving for sparsely represented nonlinear dynamics when noise is small and dynamic scales are challenging.
Method
WSINDy uses a weak formulation that can be viewed as a data-driven Galerkin method for solving for the dynamics.
Results
Effective machine-precision recovery is reported in the noise-free setting, alongside more predictable noise–coefficient-error behavior and smaller linear systems.
Takeaways & Limitations
The weak formulation provides a basis for recovering dynamic coefficients while reducing derivative-approximation dependence and linear-system size.
Takeaways & Limitations
The authors identify sparsification robustness and extensions to spatiotemporal datasets as areas requiring further investigation.
Abstract
from arXiv · showhide
We present a weak formulation and discretization of the system discovery problem from noisy measurement data. This method of learning differential equations from data fits into a new class of algorithms that replace pointwise derivative approximations with linear transformations and a variance reduction technique. Our approach improves on the standard SINDy algorithm by orders of magnitude. We first show that in the noise-free regime, this so-called Weak SINDy (WSINDy) framework is capable of recovering the dynamic coefficients to very high accuracy, with the number of significant digits equal to the tolerance of the data simulation scheme. Next we show that the weak form naturally accounts for white noise by identifying the correct nonlinearities with coefficient error scaling favorably with the signal-to-noise ratio while significantly reducing the size of linear systems in the algorithm. In doing so, we combine the ease of implementation of the SINDy algorithm with the natural noise-reduction of integration to arrive at a more robust and user-friendly method of sparse recovery that correctly identifies systems in both small-noise and large-noise regimes.
1. Problem Statement
System discovery seeks sparse nonlinear dynamical equations from noisy measurements, balancing interpretability, practicality, and robustness. WSINDy replaces pointwise derivative approximations with a weak formulation while retaining sequential thresholding for sparse recovery.
- Problem formulation: The problem is to recover a sparse weight matrix for a nonlinear dynamical system from measurements sampled at discrete timepoints.The dynamics are represented using a known family of candidate functions.
- Challenges: Measurement noise and varying dynamic scales make accurate system identification difficult, particularly for standard SINDy.SINDy is successful when noise is small and dynamic scales do not vary across multiple orders of magnitude.
- Existing approach: SINDy offers interpretable differential equations and easy implementation through sequentially thresholded least squares, but its sparsification is not highly robust.The article uses sequential thresholding to demonstrate the proposed weak formulation and notes that more robust strategies remain possible.
- Contribution and scope: The article provides rigorous justification and a robust algorithm for using weak dynamics instead of local pointwise derivative approximations.Experiments are restricted to autonomous ordinary differential equations because they are more amenable to analysis; PDEs and non-autonomous systems are future directions.
2. Weak SINDy
Weak SINDy reformulates system identification through weak equations evaluated against smooth test functions. This avoids relying on pointwise derivative estimates while accommodating discrete, noisy observations.
- Weak formulation: Weak SINDy replaces local derivative estimation with a weak formulation obtained by testing the differential equation over time intervals.Smooth test functions, including compactly supported non-constant functions, produce the weak equations used for identification.
- Residual construction: The generalized residual substitutes a candidate function from the trial basis and measured data for the exact dynamics and trajectory.With the true weights and noise-free trajectory, the residual vanishes for every admissible test function.
- Data limitations: Discrete measurements require numerical approximation of the weak equations, while measurement noise introduces additional error in the recovered weights.The data are discrete, so the weak residual cannot be evaluated exactly from observations.
2.1. Method Overview.
The method is a data-driven Galerkin procedure that computes weak residuals using analytically specified test functions and data-evaluated trial functions. It combines quadrature, covariance-aware regression, adaptive test-function placement, and sparse recovery.
- Method overview: WSINDy treats data-evaluated candidate functions as a trial basis and uses an analytically known test-function basis in a data-driven Galerkin method.Inner products in the residual are computed from the data and test functions rather than pointwise derivatives.
- Noise-free recovery: In noise-free trajectories, trapezoidal discretization with smoothly decaying test functions can recover the true coefficients to the forward solver’s effective machine precision.The accuracy is ultimately limited by the quadrature scheme and forward ODE solver tolerance.
- Noise handling: Measurement noise biases the weights, motivating generalized least squares with an approximate covariance matrix computed from the test functions.The residual analysis provides the basis for the covariance-aware formulation.
- Adaptive test functions: Adaptive placement of test functions near steep gradients improves recovery, while noisy-data recovery has significant digits scaling optimally with signal-to-noise ratio.The adaptive basis is constructed self-consistently and does not rely on pointwise derivative approximation.
- Large-noise regime: Even with large noise, WSINDy identifies the correct nonlinearities and accurate weights using far fewer test functions than timepoints.The weak formulation offers K = M(M − 1)/2 possible residuals, but exhaustive use can become intractable.
2.2. Algorithm: Weak SINDy.
The WSINDy algorithm builds integration operators and weak-feature matrices, then solves a regularized generalized least-squares problem with sequential thresholding to enforce sparsity.
- Weighted regression: The algorithm solves generalized least squares using an approximate covariance matrix derived from the action of test functions on the data.Residual analysis motivates the weighting used in the regression problem.
- Sparsification and regularization: Sequential thresholding with parameter λ enforces sparse coefficients, while ℓ2 regularization with coefficient γ addresses rank-deficient problems.The Lotka–Volterra system is identified as an example requiring regularization.
- Parameter conditions: Recovery requires λ < min_{w⋆≠0}|w⋆|, although the method is not sensitive to λ at low noise.The study does not include methods for choosing optimal λ and γ and mostly sets γ = 0.
- Linear system construction: WSINDy constructs integration matrices and computes a Gram matrix and right-hand side from test functions and data-evaluated candidate functions.The Gram entries are weak inner products between test functions and candidate functions.
2.3. Residual Analysis.
The residual analysis decomposes errors from measurement noise, nonlinear library evaluation, and numerical integration, motivating covariance weighting and adaptive localization near steep gradients.
- Nonlinearities in Θ(y) make E[R(w⋆)]≠0, so recovered coefficients are inherently biased.
- Measurement noise enters through trial functions evaluated at y=x+ϵ and through the weak-form left-hand side.
- When ϵ=0, integration error is the remaining error source, and sufficiently accurate quadrature makes ŵ=w⋆ the minimizer.
- The approximate covariance Σ=V′(V′)^T models the leading noise structure under approximately independent Gaussian residual columns.
- Localizing test functions near large |ẋ| approximately cancels R2 and R3, because perturbed observations remain close to the true trajectory.
- Regions with small |ẋ| are less reliable for noisy-data recovery because perturbations are more likely to leave the relevant phase-space region.
2.4. Test Function Basis (φk)k∈[K].
The test-function basis uses compactly supported unimodal piecewise polynomials and trapezoidal quadrature, with uniform or adaptive placement controlled by localization and overlap parameters.
- Test Function Basis: The basis S contains non-negative, unimodal, compactly supported piecewise polynomials normalized so that ∥φ∥∞=1.
- Numerical Discretization: The trapezoidal rule discretizes the weak derivative relation to order p+1 under endpoint root conditions.
- Basis Placement: Uniform-grid test functions are placed across [0,T], while adaptive placement samples centers from a cumulative distribution whose density is proportional to total variation.
- Parameter Selection: Larger ρ and larger s lead to better recovery of w⋆ in the reported parameter study.
- Basis Placement: The adaptive construction selects centers near steep gradients and builds each function from its center, polynomial degree, and support determined using rwhm.
- Algorithm: The algorithm forms integration matrices, computes G=VΘ(y) and b=−V′y, then applies regularized generalized least squares with sequential thresholding.
3. Numerical Experiments
Numerical experiments evaluate WSINDy on canonical nonlinear systems under noise-free, low-noise, and large-noise conditions. The method recovers dynamics accurately across parameter ranges, while performance depends on noise level and test-function design.
- Noise-Free Data: WSINDy recovers the correct weight matrix to within the ODE solver tolerance of 10^-10 across a wide range of parameter values.This accuracy is observed in the zero-noise regime for each tested dynamical system.
- Noise-Free Data: Increasing polynomial degree p produces convergence to ODE-solver accuracy, while recovery remains possible with the minimal basis size K = J.The experiments also report recovery over parameter sets spanning several orders of magnitude.
- Large-Noise Regime: At σSNR = 0.1, Duffing and Van der Pol trajectories remain qualitatively correct despite divergence in equilibrium relaxation or oscillation timescale.The Duffing trajectory diverges slightly while relaxing to equilibrium, whereas Van der Pol recovers the correct limit cycle with a slightly shorter dominant timescale.
- Large-Noise Regime: At large noise, Lotka-Volterra remains nearly indistinguishable from the true trajectory, while Lorenz captures the strange attractor despite expected chaotic divergence.The Lorenz trajectory remains close only for an initial interval, but the attractor is still captured.
4. Concluding Remarks
WSINDy reformulates sparse differential-equation discovery in weak form, avoiding pointwise derivatives and supporting generalized least-squares estimation. The concluding results indicate favorable noise scaling, smaller linear systems, and extensions beyond ODEs, while leaving computational tradeoffs for fuller analysis.
- Method: WSINDy reformulates model selection as a sparse regression problem using integral terms and a weak formulation.The method generalizes integral-term sparse recovery schemes for dynamical systems.
- Method: WSINDy avoids evaluating pointwise derivatives, which cause significant accuracy problems in conventional SINDy.Integration is a linear operator that permits covariance estimation and weighted least squares.
- Results: WSINDy’s coefficient error appears more predictable with signal-to-noise ratio and scales continuously without noise filtering.The authors contrast this trend with what cannot be expected from pointwise derivative approximations.
- Results and limitations: Preliminary efforts suggest WSINDy requires fewer data points but more overall floating-point operations than conventional SINDy.A more complete analysis is required to understand the tradeoffs between the two methods.
- Future directions: WSINDy is presented as an ODE proof of concept with natural extensions to spatiotemporal, multiresolution, and multiple-timescale settings.These extensions build on weak and variational formulations developed in numerical and functional analysis.