Tensor-Train Weak SINDy: Identifying High-Dimensional Nonlinear Dynamics

Will Houser; Vanja Dukic; David M. Bortz

Source-linked AI summary

Tensor-Train Weak SINDy: Identifying High-Dimensional Nonlinear Dynamics

Will Houser, Vanja Dukic, David M. Bortz

arXiv:2609.09434v1Sep 8, 2026cs.LGcs.CEstat.ML

TL;DR

High-dimensional data-driven dynamics discovery can be expensive in computation and memory, while tensor formats address poor scaling with tensor order. TT-WSINDy combines MANDy and WSINDy in the tensor-train format, and the authors show polynomial time and memory complexity in ODE dimension rather than exponential complexity.

Problem
High-dimensional dynamical-system discovery remains expensive in computation and memory, while classical tensor formats can scale poorly with tensor order.
Method
TT-WSINDy recasts WSINDy in tensor-train format, combines MANDy techniques with MSTLS and TT regression, and performs sparsification in a low-rank latent space.
Results
TT-WSINDy achieves time and memory complexities polynomial in ODE dimension rather than exponential, while matching flat WSINDy coefficients with relative errors on the order of 10^-4.
Takeaways & Limitations
The tensor-train approach makes searching an exponentially growing candidate-function space tractable while retaining weak-form robustness to noise and enabling dense temporal datasets through low-rank feature construction.
Takeaways & Limitations
The low-rank feature construction has time and memory complexities dependent on the data’s correlation structure, while the original construction requires time snapshots no larger than the search-space scale.

Abstract

from arXiv · show

In recent years, weak-form methods have made significant advances in data-driven discovery of dynamical systems. However, in high-dimensional settings, current techniques can prove expensive in both computation and memory. In this work, we introduce TT-WSINDy, which combines techniques of the Multidimensional Approximation of Nonlinear Dynamics (MANDy) and Weak Sparse Identification of Nonlinear Dynamics (WSINDy) methods, implementing requisite computations in the tensor-train (TT) format. We demonstrate that this method is able to search an exponentially-growing space of candidate functions -- performing weak-form transformation, regression, and sparsification -- without suffering from the curse of dimensionality.

1. Introduction.

Data-driven dynamics methods face inefficiency, solver inaccuracy, derivative-estimation sensitivity, and high-dimensional tensor-scaling challenges. Weak-form methods address derivative estimation and noise, while tensor-train representations target dimensionality-related costs.

Output-error methods can be computationally inefficient, solver-sensitive, and dependent on hyperparameter tuning.
SINDy reformulates nonlinear model selection as a linear system but requires derivative data that can be sensitive to noise.
Weak-form methods avoid unknown derivative data through test-function integration and integration by parts, while smoothing raw data for more noise-robust regression.
Classical tensor formats can suffer from the curse of dimensionality, including exponential Tucker storage and NP-hard CP-rank computation.
Tensor decompositions motivate high-dimensional data-driven methods that avoid forming corresponding full tensors in memory.

2. Notation and preliminaries.

The preliminaries formulate noisy nonlinear dynamics, explain SINDy and WSINDy, and introduce tensor operations and the tensor-train format. TT representations support pseudoinverse computation through operations on low-rank cores rather than full tensors.

WSINDy: The learning problem assumes a first-order D-dimensional ODE with noisy measurements and dynamics linear in known basis-function parameters.
SINDy: SINDy constructs a feature matrix from candidate functions and seeks a sparse coefficient matrix using pointwise derivative data.
WSINDy: WSINDy replaces derivative estimation with a weak formulation based on compactly supported test functions, avoiding dependence on derivative data and mollifying measurement noise.
Tensor preliminaries: Tensor operations include outer products, which combine modes, and contractions, which collapse matching adjacent dimensions in a matrix-multiplication analogue.
Tensor-train format: The tensor-train format decomposes a tensor into linked order-3 cores with shared ranks and supports efficient singular-value and pseudoinverse computations.
Tensor-train format: TT pseudoinverse computation has time complexity O(NIR^3 + KR^2), where I is the maximum mode size and R the maximum TT rank.

3. TT-WSINDy.

TT-WSINDy recasts the exponentially large WSINDy candidate library in tensor-train form and combines TT regression with sparse thresholding. Its two-stage TT-MSTLS-plus-MSTLS procedure preserves the weak-form problem while reducing computational and memory demands.

Problem construction: TT-WSINDy recasts the flat WSINDy learning problem in a tensor format whose low-rank latent space encodes many candidate functions with lower time and memory cost.The construction follows MANDy and combines TT regression with modified sequential thresholding least squares.
Problem construction: Because the full library contains JD product functions, its matrix formulation becomes computationally infeasible as the state dimension D grows.The library is formed from products of one-dimensional basis functions.
Problem construction: The weak feature tensor is built from feature cores using discretized test-function cross-correlation and retains a tensor-train structure.The test function is folded into the final core, while data are cross-correlated with its derivative for the weak formulation.
Tensor-train regression: TT-MSTLS performs a threshold-parameter line search around an unsparsified TT pseudoinverse solution, balancing residual change against coefficient sparsity.Its loss function penalizes deviation from the initial solution and encourages a sparse coefficient tensor.
TT-WSINDy algorithm: TT-WSINDy applies TT-MSTLS followed by standard MSTLS, with TT-STLS first reducing the candidate search space in the TT-induced latent space.Omitting TT-MSTLS and using the full support reduces the method to WSINDy.
TT-WSINDy algorithm: TT-MSTLS yields substantial memory and time improvements, while the stated complexity analysis gives polynomial scaling in the problem dimensions.The section explicitly frames this as circumventing the curse of dimensionality.

4. TT-WSINDy agrees with WSINDy.

The paper proves that TT-WSINDy is mathematically equivalent to WSINDy after matricization and that its tensor construction is the weak form of the corresponding strong-form tensor problem. Thus, the tensor representation changes computational organization without changing the solved weak regression problem.

Weak-form correspondence: Weak-form matrix multiplication is represented by discrete cross-correlation with the discretized test function and its derivative.This establishes the bridge between matrix weak features and tensor weak features.
Tensor–matrix equivalence: The tensor contraction and matricization identities imply W × Θ(X) = WflatΘflat(X), linking tensor and matrix feature representations.The result follows from the compatibility between tensor contraction and the chosen unfoldings.
Tensor–matrix equivalence: A tensor coefficient solution corresponds to a matrix SINDy solution through matricization, so the strong-form tensor problem and matrix SINDy problem are equivalent.The paper states this equivalence for the strong-form relation between the dynamics and feature representation.
Optimization equivalence: Proposition 4.6 shows that matricizing a TT-WSINDy solution produces a solution to the equivalent WSINDy optimization problem.This is the section’s main optimization-equivalence result.
Weak-form correspondence: The construction is the weak form of the MANDy tensor formulation, obtained by multiplying the strong-form relation by the test-function matrix.The resulting expression connects the matrix weak form, tensor contraction, and weak feature tensor.
Implication: Because TT-WSINDy solves the same weak regression problem as WSINDy, its computational savings arise from representation rather than a different regression target.The next section evaluates the resulting computation and memory savings for high-dimensional ODE discovery.

5. TT-WSINDy outperforms WSINDy in many dimensions.

TT-WSINDy performs weak-form regression and sparsification in tensor-train format, avoiding explicit formation of the exponentially large feature tensor. Its polynomial complexity, low-rank construction, and support reduction make high-dimensional searches feasible under stated sparsity conditions.

TT-WSINDy construction: TT-WSINDy performs requisite operations entirely in TT format without forming the full O(J^DM) tensor.This includes regression, masking, and pseudoinverse computations in a low-rank latent representation.
TT-STLS sparsification: The TT representation is preserved under Hadamard-product masking, so every feature-tensor iterate remains a rank-(M, ..., M) tensor train.This allows each pseudoinverse to be computed using TT operations rather than a full matrix representation.
Complexity: TT-STLS has estimated time complexity O((JD)^2M^3) and memory complexity O(JDM^2), both polynomial in the weak feature tensor’s shape parameters.The iteration bound is loose; in practice, time complexity is O(JDM^3).
Trade-offs: Matrix STLS can be preferable for runtime when M is sufficiently large, with the stated crossover condition M ≳ JD.The comparison assumes a constant number of iterations for matrix STLS.
Low-rank feature tensor: Low-rank construction reduces storage and can improve scaling in M by exploiting lower true ranks and low-pass filtering before truncated SVD.For J = 3, D = 10, and M ≈ 6000, storage falls from approximately 1.0 · 10^9 to 7.6 · 10^8, and a further reduction reaches approximately 1.1 · 10^8 in the cited example.
Support reduction: TT-STLS can output a much smaller linear problem than its input when the minimal coarse support is substantially smaller than the initial full support.This support reduction relies on sparsity of the true terms in the total basis-function search space.

6. Examples and results.

Across four ODE systems, weak-form regression improves robustness to noise, while TT-WSINDy reduces the support handed to the flat solver and achieves substantial speedups as Lorenz 96 dimension grows.

6.1. Weak vs. Strong Form TT Regression.: The experiments cover FPUT, Lorenz 96, Kuramoto, and Chua’s circuit, spanning quasi-periodic, chaotic, coupled-oscillator, and nonlinear-cross-term dynamics.Data include one or multiple trajectories, with 40 realizations regressed at each noise level.
6.1. Weak vs. Strong Form TT Regression.: Weak-form regression provides increased coefficient accuracy at both low and high noise levels across the tested ODE systems.A noise level exists where strong-form estimates are comparable, but weak-form robustness extends across the [0, 1] noise-ratio continuum.
6.2. TT-WSINDy Walltime.: The low-rank feature tensor construction was used for large datasets, and regression for Chua’s circuit terminated in an average of 1.33 seconds.The low-rank construction is motivated by data-heavy settings with M ≫ JD.
6.2. TT-WSINDy Walltime.: For D ≤11, coarse screening identified the true coarse support before the flat solve recovered the true dynamics; at D = 12, about 2000 spurious terms remained after screening.Both TT-WSINDy and flat WSINDy identified the true support, with identical coefficients and relative coefficient errors on the order of 10^-4.
6.2. TT-WSINDy Walltime.: For D = 6, . . . , 12, TT-WSINDy outperformed flat WSINDy by a speedup factor of 2 to 7.At D = 5, the flat solver was faster because coarse TT-MSTLS screening of the size 2^5 search space did not provide computational benefits.

7. Conclusion.

The paper extends WSINDy with tensor-train techniques to reduce time and memory costs while retaining weak-form sparsification, and reports polynomial rather than exponential scaling in ODE dimension. Future work targets screening reliability, construction tradeoffs, and PDE extensions.

7. Conclusion.: TT-WSINDy combines MANDy and WSINDy techniques to achieve time and memory savings over a rich candidate-function search space.The method translates the strong-form problem into weak form and applies existing WSINDy sparsification techniques within the tensor-train format.
7. Conclusion.: The method has time and memory complexities polynomial in ODE dimension rather than exponential, subverting the curse of dimensionality.A low-rank weak feature tensor construction also makes the algorithm tractable for datasets dense in the temporal axis.
7. Conclusion.: Future work will study TT-MSTLS screening statistics to reduce false negatives and ensure the fine WSINDy pass retains the full library of true terms.This targets the reliability of the coarse-to-fine identification pipeline.
7. Conclusion.: The original feature construction requires the number of snapshots to be at most on the same order as the search-space size JD, whereas the low-rank construction depends on data correlation structure.The authors identify understanding this tradeoff as a route to greater robustness.
7. Conclusion.: The authors plan to extend TT-WSINDy to PDEs, where dimensionality arises from incoming-data scale as well as potentially large search spaces.The paper points to existing strong-form PDE libraries and an initial MANDy adaptation as related groundwork.

A.1. MANDy problem construction.

The MANDy construction represents the feature matrix in tensor-train form, including the identity core needed for equivalence to matrix SINDy, and computes coefficients using a TT pseudoinverse.

A.1. MANDy problem construction.: The strong feature tensor is constructed in tensor-train form from feature vectors and feature cores.This provides the tensor representation used by the MANDy formulation.
A.1. MANDy problem construction.: Including the M × M identity matrix as its own core is necessary for the MANDy construction to be equivalent to the matrix SINDy problem.The coefficient tensor is then obtained from the TT representation.
A.1. MANDy problem construction.: The coefficient tensor is computed using a TT pseudoinverse of the feature tensor.This replaces direct matrix pseudoinversion within the tensor-train construction.

A.2. TT Pseudoinverse algorithm.

The TT pseudoinverse algorithm orthonormalizes tensor cores, splits the tensor at a learning-specific index, and is dominated by singular value decompositions with complexity O(NIR^3 + KR^2).

A.2. TT Pseudoinverse algorithm.: The TT pseudoinverse procedure is based on an established algorithm and operates on an orthonormalized tensor train.Its final core must be right-orthonormal, while the other cores must be left-orthonormal.
A.2. TT Pseudoinverse algorithm.: The algorithm splits the tensor train at index N, a choice specific to this learning problem although other split indices are possible.The more general version permits any index in the stated range.
A.2. TT Pseudoinverse algorithm.: The algorithm’s computational effort is dominated by singular value decompositions of matricized tensor cores.These SVDs produce the left- and right-orthonormal counterparts required by the pseudoinverse procedure.

A.3. Function-major feature tensor.

The function-major feature tensor provides an alternate representation of the feature data, alongside the dimension-major form. Although the two forms produce differently sized flat problems, they have the same memory and pseudoinverse-computation costs.

The function-major feature tensor is constructed from feature vectors containing evaluations of each f_j on a given equation.
The function-major and dimension-major representations correspond to flat problems of sizes O(JDM) and O(DJM), respectively.
Both feature tensors require O(JDM^2) memory and O(JDM^3) time to compute the pseudoinverse.

Appendix B. Optimizations for TT-WSINDy.

TT-WSINDy reduces repeated work through cached pseudoinverse information and an optional one-pass sparsification mode. The one-pass approach worked for Lorenz 96, while a general handoff heuristic remains future work.

TT-WSINDy caches TT pseudoinverse solves because the pseudoinverse does not depend on the left-hand-side data.
For each support, the implementation stores only an O(JD) low-memory set of traces and reuses it across threshold values.
A boolean one-pass parameter performs one coarse TT-STLS pass for medium-size search spaces before handing the problem to matrix MSTLS.
For Lorenz 96, the one-pass approach was sufficient, but a general heuristic for choosing TT sparsification versus flat-solver handoff is left to future work.