PaperlayerView exact revision on arXiv

Source-linked AI summary

Inferring biological networks by sparse identification of nonlinear dynamics

Niall M. Mangan, Steven L. Brunton, Joshua L. Proctor, J. Nathan Kutz

arXiv:1605.08368v1math.DS

TL;DR

Biological network inference lacks scalable approaches when interactions and nonlinearities are unknown. This paper introduces implicit-SINDy, which uses sparse null-space identification to infer rational nonlinear dynamics, accurately reconstructing three canonical biological models.

  • Problem

    Unknown network interactions and nonlinearities make information-theoretic model selection computationally challenging because candidate interactions scale combinatorially.

  • Method

    Implicit-SINDy casts rational nonlinear dynamics as implicit equations and selects the sparsest vector in a mixed state-derivative library's null space.

  • Results

    Implicit-SINDy accurately reconstructs enzyme kinetics, bacterial regulation, and yeast metabolism, recovering coefficients within 2% of their original values.

  • Takeaways & Limitations

    The method provides a parsimonious alternative for selecting nonlinear biological network dynamics from large libraries of candidate functions.

  • Takeaways & Limitations

    Practical implementation remains challenged by noise sensitivity and the need to reduce the number of time-series measurements.

Abstract

from arXiv · show

Inferring the structure and dynamics of network models is critical to understanding the functionality and control of complex systems, such as metabolic and regulatory biological networks. The increasing quality and quantity of experimental data enable statistical approaches based on information theory for model selection and goodness-of-fit metrics. We propose an alternative method to infer networked nonlinear dynamical systems by using sparsity-promoting $\ell_1$ optimization to select a subset of nonlinear interactions representing dynamics on a fully connected network. Our method generalizes the sparse identification of nonlinear dynamics (SINDy) algorithm to dynamical systems with rational function nonlinearities, such as biological networks. We show that dynamical systems with rational nonlinearities may be cast in an implicit form, where the equations may be identified in the null-space of a library of mixed nonlinearities including the state and derivative terms; this approach applies more generally to implicit dynamical systems beyond those containing rational nonlinearities. This method, implicit-SINDy, succeeds in inferring three canonical biological models: Michaelis-Menten enzyme kinetics, the regulatory network for competence in bacteria, and the metabolic network for yeast glycolysis.

I. INTRODUCTION · A. Biological networks · B. Model selection and information theory

The paper motivates implicit-SINDy as a data-driven alternative for inferring biological network dynamics and connectivity when information-theoretic model selection becomes computationally intractable. It generalizes SINDy to rational nonlinearities using overcomplete libraries and sparse regression, and demonstrates the method on three biological models.

  • I. INTRODUCTION: Implicit-SINDy generalizes SINDy to an implicit formulation that permits rational-function dynamics, motivated by overcomplete libraries – and sparse regression,.The method is proposed to infer both the dynamics and connectivity of biological networks, with accuracy and robustness demonstrated on three representative models.
  • A. Biological networks: Regulatory and metabolic networks are critical for cellular function, and failures in their control circuits can contribute to cancer and other deadly diseases.These networks can often be modeled using mass-action kinetics, which produces a relatively constrained set of network motifs.
  • A. Biological networks: Modern high-quality experimental data create opportunities for rapid, robust, and accurate data-driven identification of regulatory and metabolic network models.Such identification could accelerate understanding and control of biological functions, including disease treatment and metabolic engineering.
  • B. Model selection and information theory: Traditional biological network models are developed by conjecturing relationships between variables and iteratively modifying them against experiments, a process that can take years to become quantitatively predictive.Information criteria instead assign numerical scores to candidate models, but they depend on practitioners specifying a reasonable candidate set.
  • B. Model selection and information theory: Information-theoretic model selection is difficult for nonlinear dynamical networks because unknown connections and functional forms create a combinatorially large model space.Candidate models may require simulation, comparison with data, and information-criterion scoring, making exhaustive evaluation computationally intractable.
  • B. Model selection and information theory: For polynomials up to degree 4 in five variables, the metabolic-network example has 126 monomials and approximately 10^38 possible polynomial structures.General numerator and denominator polynomials produce an even larger space of rational functions.
  • B. Model selection and information theory: Large libraries of candidate nonlinearities combined with sparsity-promoting regression select a sparse subset of functional interactions, as in SINDy.Libraries provide many possible functional forms, while LASSO or elastic-net regularization promotes sparse selection.

C. Contribution of this work · II. SPARSE REGRESSION FOR DYNAMICAL SYSTEMS · A. LASSO and sparse approximation

This section extends SINDy to rational nonlinearities motivated by biological network dynamics, and frames the approach through sparse regression, overcomplete libraries, and LASSO-based feature selection. The method is validated on three biological models and positioned for practical inference in real systems.

  • C. Contribution of this work: The work extends SINDy to rational-function nonlinearities because metabolic and regulatory network dynamics commonly contain rational terms.Constructing a sparse-regression library for generic rational functions is difficult because they are not simply sparse combinations of a few basis functions.
  • C. Contribution of this work: The updated algorithm is validated on simulated enzyme-kinetics, cell-differentiation regulation, and seven-node glycolysis models before discussing real biological applications.The practical discussion includes challenges such as those arising in applying implicit-SINDy to real biological systems.
  • II. SPARSE REGRESSION FOR DYNAMICAL SYSTEMS: The network-inference method combines sparse regression, overcomplete libraries, and dynamical-systems theory into a broadly applicable mathematical architecture.These components are presented as key innovations enabling the method.
  • A. LASSO and sparse approximation: LASSO performs feature selection by finding a sparse solution vector a from an over-determined library Θ, thereby defining a criterion for variable selection.The method is described as one of the earliest sparsity-promoting regression techniques.
  • A. LASSO and sparse approximation: Sparse approximation uses the ℓ1 norm to represent measurements as linear combinations of library atoms, alongside related methods such as elastic-net regularization.Other ℓ1-based sparsity innovations include those cited in,, and, while atoms may come from training data or known bases.

B. Sparse identification of nonlinear dynamics (SINDy)

SINDy discovers nonlinear dynamical systems by regressing measured derivatives onto a candidate function library while enforcing sparsity to select active interactions. Although polynomial and trigonometric libraries cover many systems, rational nonlinearities arising from time-scale-separated biological kinetics can make standard libraries computationally prohibitive.

  • B. Sparse identification of nonlinear dynamics (SINDy): SINDy extends automated system identification through sparse regression, improving numerical robustness in noisy overdetermined problems compared with earlier compressed-sensing approaches –.
  • B. Sparse identification of nonlinear dynamics (SINDy): SINDy assumes dynamical systems are sparse in a function space and uses ℓ1-regularized regression to identify active terms.This convex optimization selects a sparse model from a combinatorially large set of candidate nonlinearities.
  • B. Sparse identification of nonlinear dynamics (SINDy): The workflow collects state time series and derivatives, constructs polynomial and trigonometric candidate libraries, and regresses derivatives onto these nonlinearities.State variables represent measured biological components, while total-variation regularization provides a numerically robust derivative estimate from noisy data.
  • B. Sparse identification of nonlinear dynamics (SINDy): Polynomial libraries up to order n correspond to biological mass-action kinetics involving complexes from monomers through n-mers.The polynomial and trigonometric library is sufficient for a large class of dynamical systems.
  • B. Sparse identification of nonlinear dynamics (SINDy): Time-scale-separated mass-action kinetics can reduce to rational functions, for which a standard polynomial library becomes prohibitively large and motivates a tractable rational-function library.

III. INFERRING NONLINEAR DYNAMICAL SYSTEMS WITH RATIONAL FUNCTIONS

The method generalizes SINDy to rational nonlinearities by rewriting dynamics as implicit equations and finding the sparsest nonzero coefficient vector in the library’s null space. An augmented library combines numerator monomials with derivative-weighted denominator polynomials, while alternating directions methods identify the sparse model.

  • III. INFERRING NONLINEAR DYNAMICAL SYSTEMS WITH RATIONAL FUNCTIONS: Rational nonlinearities require modifying sparse regression because they are not generally sparse linear combinations of a small rational-function library.The procedure therefore seeks the sparsest implicit ordinary differential equation rather than applying the original SINDy formulation directly.
  • III. INFERRING NONLINEAR DYNAMICAL SYSTEMS WITH RATIONAL FUNCTIONS: Multiplying each rational equation by its denominator converts the dynamics into an implicit form involving numerator polynomials and derivative-weighted denominator polynomials.The resulting augmented library is built from state monomials and denominator-library columns multiplied element-wise by the relevant derivative.
  • III. INFERRING NONLINEAR DYNAMICAL SYSTEMS WITH RATIONAL FUNCTIONS: The augmented library is typically only twice the size of the original polynomial library when numerator and denominator polynomial degrees are matched.In this common case, ΘN(X) = ΘD(X).
  • III. INFERRING NONLINEAR DYNAMICAL SYSTEMS WITH RATIONAL FUNCTIONS: Because the zero vector trivially satisfies the implicit equation, the algorithm identifies the sparsest nonzero vector in the library’s null space using alternating directions methods.The inferred sparse coefficients correspond to the active nonlinear terms in each dynamical equation.

A. Algorithm for sparse selection of rational functions.

The algorithm identifies sparse rational-function dynamics by finding sparse vectors in the null space of a library built from state and derivative time series, then selecting a parsimonious model across sparsity thresholds.

  • Algorithm: The method builds a functional library from state-variable and derivative time-series data, computes its null-space basis, and searches for sparse linear combinations using an alternating-directions method with threshold λ.The threshold controls the enforced sparsity of the inferred coefficient vector ξ.
  • Model selection: Varying λ generates models with different accuracy and sparsity, after which the algorithm selects the most parsimonious model from their Pareto front.Increasing λ reduces the number of terms, and the chosen model is identified at the Pareto plot’s sharp drop-off.

B. General formulation for implicit ODEs

The formulation extends implicit-SINDy beyond rational nonlinearities to general implicit ordinary differential equations using libraries of mixed state and derivative nonlinearities. It also supports higher-order derivatives, broadening the systems that can be identified.

  • General formulation: The method identifies general implicit ordinary differential equations beyond systems with rational function nonlinearities.This generality enables identification of additional systems beyond the rational-function examples explored later.
  • General formulation: A library of nonlinear functions of states and derivatives represents mixed terms with different powers and combinations.The formulation permits nonlinear equations containing varied combinations of state and derivative terms.
  • General formulation: The resulting implicit dynamics can be encoded as a sparse vector in the null space of the library.
  • General formulation: Higher-order derivatives can be incorporated by expanding the feature library, including formulations for second-order implicit dynamical systems.

IV. RESULTS

The implicit-SINDy architecture is tested on canonical biological network models and shown to be fast, accurate, and robust for inferring three systems.

  • Implicit-SINDy is fast, accurate, and robust for inferring Michaelis-Menten enzyme kinetics, bacterial competence regulation, and yeast glycolysis.These canonical models support potential broader application of the method to biological networked dynamical systems.
  • The method is validated on canonical models of biological networked dynamical systems.Validation on these models allows for potential broader application.
  • The tested systems span enzyme kinetics, bacterial regulatory dynamics, and yeast metabolic dynamics.The examples comprise Michaelis-Menten enzyme kinetics, the regulatory network for competence in bacteria, and the metabolic network for yeast glycolysis.

A. Simple example: Michaelis-Menten kinetics

For Michaelis-Menten kinetics, implicit-SINDy recovers the correct rational functional form and accurately fits coefficients using time-series data from only two initial concentrations. The inferred model matches the original on test trajectories generated from new initial conditions.

  • A. Simple example: Michaelis-Menten kinetics: The Michaelis-Menten dynamics reduce to a single state-variable equation with a rational function after fast binding/unbinding or steady-state assumptions.The model describes substrate binding and unbinding followed by irreversible product formation.
  • A. Simple example: Michaelis-Menten kinetics: Using time-series data from only two initial concentrations, implicit-SINDy extracts the correct functional form and accurately fits its coefficients, matching the original model on new initial conditions.The new initial conditions were not used during training.
  • A. Simple example: Michaelis-Menten kinetics: The algorithm uses measurable concentration trajectories x(t), calculates derivative trajectories from them, and includes a flux source j_x plus Michaelis-Menten consumption.V_max is the maximum reaction rate, while K_m is the concentration at half-maximal reaction rate.
  • A. Simple example: Michaelis-Menten kinetics: Implicit-SINDy sparsely selects four active terms from a 10-column library of polynomial functions up to degree four, with the Pareto front sharply dropping in error near 0.01 to 10^-5.The sharp drop occurs at four terms, indicating the most parsimonious model’s λ.

B. Regulatory network: B. subtilis competence

The method is tested on a two-state B. subtilis competence regulatory model with rational, Hill-function nonlinearities arising from time-scale separation. It recovers the rational structure exactly and identifies coefficients within 2% of their true values.

  • B. Regulatory network: B. subtilis competence: The two-state reduction represents ComK positive autoregulation and ComS repression through Hill-function positive and negative feedback loops.The Hill-function powers indicate cooperative involvement of x1 proteins in regulatory complexes, and the combined feedback enables switching between competence and vegetative growth.
  • B. Regulatory network: B. subtilis competence: The two equations reduce a six-state regulatory dynamical system, with each rational function produced by steady-state or time-scale-separation assumptions.This reduction provides the mechanistic basis for the rational nonlinearities used in the test model.
  • B. Regulatory network: B. subtilis competence: The benchmark uses 40 time series and requires identifying dependence on two state variables despite a numerator polynomial reaching fifth degree.The library must therefore include polynomials up to degree six, although the highest power need not be known beforehand.
  • B. Regulatory network: B. subtilis competence: The inferred rational function is factored into the original form, with identified coefficients within 2% error of the true coefficients.The result is validated against test data generated from the original model and summarized in Table I.

C. Metabolic network: yeast glycolysis

For yeast glycolysis, implicit-SINDy is tested on a larger, oscillatory network containing rational and polynomial equations. Given sufficient data, it recovers the network structure and coefficients, while revealing equation-specific data and library requirements.

  • C. Metabolic network: yeast glycolysis: Inferring glycolysis networks could accelerate understanding of central metabolism and support metabolic disease intervention.Glycolysis breaks down glucose to extract energy as ATP and NADPH, and its network took over 100 years to uncover.
  • C. Metabolic network: yeast glycolysis: The analyzed yeast glycolysis network is larger and oscillatory, with three rational-function equations and four polynomial equations.It had previously been used as a model-inference test case.
  • C. Metabolic network: yeast glycolysis: 7 terms were selected from 3432 library functions for Eq. (21c), comprising 5 numerator terms and 2 denominator terms; Table II compares true and extracted coefficients.The selected terms and Pareto front are shown in Fig. 6.
  • C. Metabolic network: yeast glycolysis: Coefficient functional-dependence errors were very small, below 0.1%, although some parameters showed incorrect x6 dependence after polynomial factorization.Eq. (21g) required more data, enabling higher-precision coefficient extraction; starred values have x6-dependent errors.
  • C. Metabolic network: yeast glycolysis: Equations x3, x4, x5, and x7 required no degree-six polynomial library and were inferred more quickly, whereas Eq. (21g) needed over twice as many measurements.The comparison is against the other rational-function equations, Eqs. (21c) and (21b).

V. CONCLUSIONS

The paper develops implicit-SINDy, a computationally manageable sparse method for identifying rational nonlinear dynamics from large libraries. It also highlights efficient data-driven model selection, while leaving statistical connections and practical robustness challenges for future work.

  • V. CONCLUSIONS: Implicit-SINDy uses an alternating-directions method to select a parsimonious sparse vector in a library null space, enabling rational nonlinear dynamics for reasonably sized biological networks.The library can include nonlinear derivative terms while remaining computationally manageable.
  • V. CONCLUSIONS: The data-driven method selects connectivity and dynamics from information in the data alone, with no parameter tuning beyond a sparsity threshold determined by a Pareto front.It functions like an efficient unsupervised learning algorithm that sparsely selects dynamics from many candidate possibilities.
  • V. CONCLUSIONS: A rigorous statistical connection between implicit-SINDy selection and information criteria such as AIC and BIC remains an open theoretical question.The authors identify this connection as a future direction for relating information-theoretic metrics to sparse selection.
  • V. CONCLUSIONS: Practical implementation still faces challenges in improving robustness to noise and reducing computational demands.The supplied passage identifies these as two remaining practical challenges.
Loading 1605.08368v1…