Source-linked AI summary
Model selection for dynamical systems via sparse regression and information criteria
Niall M. Mangan, J. Nathan Kutz, Steven L. Brunton, Joshua L. Proctor
TL;DR
Standard information-criterion model selection cannot feasibly evaluate combinatorially large sets of dynamical-system candidates. The paper combines SINDy sparse regression with AIC or BIC to sub-select and rank candidates, and reports that the correct model is the only strongly supported candidate in its examples.
Problem
Standard AIC/BIC model selection is typically limited to a small predetermined candidate set because evaluating every model in a combinatorially large space is computationally intractable.
Method
The algorithm uses SINDy sparse regression to sub-select candidates near the Pareto frontier, then evaluates the remaining models with AIC or BIC.
Results
In the demonstrated examples, the correct dynamical system is the only candidate model with strong AIC support.
Takeaways & Limitations
Information criteria provide a principled ranking that resolves ambiguity after sparse regression has reduced the candidate model space.
Takeaways & Limitations
The method depends on time-series duration and sampling, and above noise level ϵ = 1 it cannot robustly select the correct model in the Lorenz example.
Abstract
from arXiv · showhide
We develop an algorithm for model selection which allows for the consideration of a combinatorially large number of candidate models governing a dynamical system. The innovation circumvents a disadvantage of standard model selection which typically limits the number candidate models considered due to the intractability of computing information criteria. Using a recently developed sparse identification of nonlinear dynamics algorithm, the sub-selection of candidate models near the Pareto frontier allows for a tractable computation of AIC (Akaike information criteria) or BIC (Bayes information criteria) scores for the remaining candidate models. The information criteria hierarchically ranks the most informative models, enabling the automatic and principled selection of the model with the strongest support in relation to the time series data. Specifically, we show that AIC scores place each candidate model in the {\em strong support}, {\em weak support} or {\em no support} category. The method correctly identifies several canonical dynamical systems, including an SEIR (susceptible-exposed-infectious-recovered) disease model and the Lorenz equations, giving the correct dynamical system as the only candidate model with strong support.
1. Introduction.
Standard model selection uses information criteria to choose among candidate models, but evaluating a combinatorially large dynamical-model space is computationally intractable. The proposed framework combines SINDy-based sparse sub-selection with information criteria to enable principled ranking from time-series data.
- 1. Introduction.: Information criteria select models by balancing goodness-of-fit against model complexity.AIC and BIC use likelihood-based scores with penalties related to the number of free parameters.
- 1. Introduction.: Standard model selection usually evaluates a small predetermined set of candidate models, often ten or fewer.This limits the use of AIC/BIC for discovering nonlinear dynamics from rich time-series data.
- 1. Introduction.: Evaluating every model in a combinatorially large candidate set is computationally intractable because each model must be simulated and scored.The desired setting has q ≫1 candidate dynamical models.
- 1. Introduction.: The framework first uses SINDy to sub-select models consistent with the data, then applies AIC or BIC to rank the remaining candidates.This integrates sparse regression for nonlinear systems identification with information-criteria model selection.
- 1. Introduction.: The method is demonstrated on canonical systems including the SEIR disease model and Lorenz equations, with the correct model receiving strong support.The authors also examine effects of noise, time-series length, and other regression factors.
2. Background.
The background frames model selection as a search for parsimonious dynamical laws in an enormous model space. It introduces AIC and SINDy, then combines sparse Pareto-frontier sub-selection with information criteria to resolve ambiguity in sparse model selection.
- 2. Background.: Polynomial models up to degree 4 with 5 state variables yield approximately 10^38 possible models to construct, fit, and compare.This scale makes exhaustive model selection computationally intractable.
- 2.1. Model selection via information criteria.: AIC compares candidate models using likelihood while penalizing the number of free parameters.With residual sum of squares, it can be expressed as AIC = m ln(RSS/m) + 2k.
- 2.2. SINDy and sparse model selection.: SINDy represents dynamics in a library of candidate functions and uses sparse regression to identify a small set of active terms.The regression seeks coefficients that fit derivative data while keeping the coefficient matrix sparse.
- 2.2. SINDy and sparse model selection.: SINDy alone may not automatically select one model when the Pareto frontier lacks a sharp elbow or contains clustered models.Choosing the sparsity-promoting parameter is therefore a central difficulty.
- 2.2. SINDy and sparse model selection.: The proposed algorithm uses SINDy to sub-select candidates near the Pareto frontier before evaluating them with information criteria.This makes ranking feasible within a combinatorially large model space and supports automatic identification of a single best-fit model.
3. Methods.
The method combines SINDy-based sparse model sub-selection with information criteria to rank and select dynamical-system models. It uses representative time-series errors and AIC scores to identify the best-supported candidate.
- 3. Methods.: SINDy generates sparse candidate models from a library of possible terms by searching over sparsification parameters.
- 3. Methods.: The algorithm simulates each SINDy-identified model, computes its AIC against the observed time series, and ranks models by score.
- 3. Methods.: Sub-selecting models near the Pareto frontier makes information-criterion evaluation tractable when the full candidate space is combinatorially large.
- 3. Methods.: Each time-series observation is assigned an average absolute error over its entire trajectory before errors are aggregated for AIC.
- 3. Methods.: AIC differences classify models as strongly supported for ∆j ≤2, weakly supported for 4 ≤∆j ≤7, or unsupported for ∆j ≥10.
- 3. Methods.: For the single-variable polynomial example, the lower AICc range contains only the correct model, while SINDy-selected models are highlighted among all possibilities.
4. Results: Model selection.
SINDy sub-selects models from combinatorially large libraries, after which relative AICc ranks support for candidate dynamical systems. Across polynomial, cubic, SEIR, and Lorenz examples, the correct models receive strong support under the reported conditions.
- 4.1. 1-D polynomial model.: 63 possible models were evaluated for a single-state polynomial system, whose correct 3-term model was the only model with strong support.All other models in the combinatorial space fell within the no-support range.
- 4.1. 1-D polynomial model.: SINDy recovered the correct 3-term model and meaningful 1-, 2-, 4-, 5-, and 6-term reductions matching combinatorial least-squares results.The 1- and 2-term models corresponded to setting smaller true coefficients to zero.
- 4.2. 2-D cubic system.: 268,435,455 potential models made exhaustive enumeration infeasible for the 2-state, fifth-order library, so SINDy generated a tractable sub-selection.The cubic-system evaluation used 100 randomly selected initial conditions for cross-validation.
- 4.2. 2-D cubic system.: The correct cubic model was strongly supported, while every other SINDy-selected model had no support under relative AICc.The input consisted of a single time series for each state variable.
- 4.3. 3-D Disease transmission model.: For the SEIR system, excluding redundant R allowed relative AICc to identify only the correct 6-term model as supported among 1023 candidates.SINDy used 100 cross-validation measurements after selecting models from the reduced library.
- 4.4. Lorenz model.: For Lorenz dynamics, the correct 7-term model had strong support, while another model with an additional small term had weak support.The reported explanation is that chaotic trajectories diverge after small coefficient variations; sensitivity analysis is therefore needed in practice.
- 4.4. Lorenz model.: Correct-model selection succeeds with low enough measurement noise and abundant cross-validation data, but higher noise admits incorrect models into supported ranges.For Lorenz data, noise levels 0.2 and 0.5 caused other models to enter weak and eventually strong support regimes.
5. Practical implementation: noise and number of measurements.
The method’s practical success depends on measurement noise, cross-validation data, and time-series duration and sampling. Increasing noise can admit incorrect models, while additional cross-validation measurements can improve score separation.
- Noise: Across random noise instances, the true model maintains the lowest score over 10 instances.Data sub-sampling is suggested for testing models containing noise-fitted terms.
- Noise: At ε = 1, the method cannot robustly select the correct model; at ε = 5, more incorrect models appear supported.At these noise levels, the true model may not be evaluated because SINDy fails to select it, and relative AICc alone can still assign a lowest score of zero.
- Cross-validation: With ε = 0.5 and Ncross = 2, SINDy selects the correct model, but an incorrect 1-term model receives a lower AICc score.With Ncross = 4, the correct model has the lowest score; increasing Ncross to 60, 80, and 100 raises the relative score of the incorrect 8-term model versus the true 7-term model.
- Time-series design: For chaotic systems such as Lorenz, method success also depends on time-series duration and sampling.In the stated experiment, only the true model was supported with ε = 0.1 over t = 0 to tend = 1.
- Implementation: The algorithm first sub-selects models using SINDy, then evaluates the remaining roughly ten candidates with AIC or BIC.This reduces information-criterion computation from a combinatorially large candidate set to a manageable collection.
6. Discussion and conclusions.
The framework combines sparse regression with information criteria to address complementary model-selection shortcomings. In the demonstrated examples, AIC strongly supports only the correct dynamical model when the data conditions are adequate.
- Discussion and conclusions: AIC ranks candidate models by score and classifies them as having strong, weak, or no support.With enough samples, sufficient signal-to-noise, and/or a sufficiently large candidate set, typically only one model is strongly supported.
- Discussion and conclusions: In the examples, the only strongly supported candidate model is the correct model.
- Discussion and conclusions: The framework addresses standard AIC/BIC’s difficulty evaluating combinatorially many candidates and SINDy’s lack of a principled correct-model selection method.Sparse regression enables large candidate consideration, while information criteria evaluate the sub-selected models.