PaperlayerView exact revision on arXiv

Source-linked AI summary

Behavioral Mapping of Hierarchical Dynamical Models Accelerates Bayesian Parameter Inference

Jingsong Bai, Ayush Pandey

arXiv:2610.30683v1stat.MEcs.SYeess.SYq-bio.QM

TL;DR

Repeated simulations make Bayesian inference for complex nonlinear dynamical models costly. This paper uses a higher-level model's posterior behavioral compatibility to learn a support region that filters detailed-model MCMC proposals without requiring parameter or state correspondence. After initial setup costs, the approach reduces MCMC time by 8.02% for the phenomenological model and 15.21% for the mechanistic model.

  • Problem

    Bayesian inference for complex nonlinear dynamical models is often limited by the cost of repeated model simulation.

    Source: §1 · p. 1

  • Method

    The framework transfers reduced-model posterior information through behavioral compatibility and a learned GP support level set to filter detailed-model MCMC proposals without parameter correspondence.

    Source: §1.2 · p. 2, §3 · p. 4, §5.2.4 · p. 13

  • Results

    After initial setup costs, MCMC time is reduced by 8.02% for the phenomenological model and 15.21% for the mechanistic model.

    Source: §1.2 · p. 2, §5.2.6 · p. 14

  • Takeaways & Limitations

    The derived conditions ensure the behavioral-support region retains detailed-model parameters that meet the prescribed data-fit tolerance when the reduced posterior assigns sufficient compatibility mass.

    Source: §1.2 · p. 2

    The hard-restricted posterior is intentionally restricted to the behavioral-support region rather than preserving the baseline posterior's support.

    Source: Thresholding the behavioral support… · p. 5

Abstract

from arXiv · show

Bayesian parameter inference for complex nonlinear dynamical models is often limited by the cost of repeated model simulation. We present a behavioral mapping framework that uses the inferred posterior of a higher-level model to accelerate inference of a lower-level model without requiring state or parameter correspondence between them. Building on approximate simulation relations for hierarchically related models, we define the notion of behavioral compatibility and quantify the behavioral support of detailed model parameters using the reduced model posterior. We learn a level set of behavioral support using a Gaussian process and use it to filter detailed model parameter proposals before expensive simulations during Markov Chain Monte Carlo (MCMC) inference. We derive conditions under which the behavioral support level set retains all detailed model parameters that fit the experimental data within a prescribed tolerance. We apply this method to a hierarchy of gene-expression models fitted to cell-free experimental data. After initial setup costs, our approach speeds up MCMC by 8.02% for the phenomenological model and 15.21% for the mechanistic model.

1 Introduction

Bayesian inference for nonlinear dynamical models is costly because it relies on repeated simulations, motivating ways to use reduced models to accelerate inference without requiring parameter correspondence. The paper introduces behavioral compatibility and support, derives retention conditions, and demonstrates MCMC speedups for gene-expression models.

1 Introduction

  • Repeated full simulations make Bayesian inference for nonlinear ODE models computationally expensive, limiting the size of models that can practically be fitted.The introduction motivates using system-behavior information from hierarchically related reduced models to accelerate full-model inference.Source: §1 · p. 1

1.1 Related Work

  • The framework transfers information from a higher-level model posterior to accelerate detailed-model inference without requiring state or parameter mapping.Prior approaches commonly relied on parameter correspondence or detailed-model simulations to learn cross-fidelity relations or local approximations.Source: §1.2 · p. 2, §1.1 · p. 2

1.2 Contributions

  • The paper constructs a behavioral-support level set and derives conditions under which it retains the detailed-model region that fits the data within a chosen tolerance.This retained region is used to support computationally efficient MCMC inference.Source: §1.2 · p. 2
  • 8.02% for the nine-state phenomenological model and 15.21% for the 38-state mechanistic model are the reported MCMC speedups after initial setup costs.The application also compares inferred parameter distributions and GFP fits with baseline inference.Source: §1.2 · p. 2

2 Preliminaries

The preliminaries establish a finite-horizon output-space setting for comparing detailed and reduced dynamical models, with no assumed parameter correspondence. They also define data-fit tolerances and the posterior and MCMC notation used in the inference framework.

2 Preliminaries

  • The hierarchy orders models by complexity, but assumes no relationship or ordering between their parameters; both levels describe common measured outputs over a finite horizon.The general setting uses parameterized models and a finite-horizon L2 output metric, while allowing other finite metrics on a common measured output space.Source: §2 · pp. 2–3
  • The specialized linear analysis assumes the reduced system is observable over the finite horizon, making its observability Gramian positive definite and invertible.This assumption is specific to the linear analysis.Source: §2 · p. 3
  • Approximate simulation relations require bounded observations, transition matching, and coverage of the detailed model's initial-condition set.For deterministic continuous-time flows, transition matching keeps a related initial-state pair related over the experiment horizon.Source: §2 · p. 3

2.1 Experimental data and desired fit tolerance

  • The desired fit tolerance η defines detailed-model parameters whose simulated outputs lie within η of the measured trajectory.The reduced model has an analogous tolerance-defined set; the Bayesian notation distinguishes prior, posterior, and posterior mass within a fit set.Source: §2.1 · p. 4, §2.2 · p. 4

2.2 Bayesian inference and MCMC notation

  • MCMC effective sample size is approximately the chain length divided by integrated autocorrelation time, and implementations use an ensemble of walkers and steps.The paper denotes the number of walkers by Nw and the number of steps by Ns.Source: §2.2 · p. 4

3 Problem Formulation

The problem formulation uses posterior behavioral compatibility to identify detailed-model parameters supported by a reduced model, without requiring parameter or state correspondence. A learned support boundary can restrict inference, but this restriction does not preserve the baseline posterior's full support.

3 Problem Formulation

  • The goal is computationally efficient inference of the detailed-model posterior using information from an inferred posterior of a lower-state, higher-level abstraction.The formulation does not order parameter dimensions and allows reduced models with lumped parameters and potentially more parameters.Source: §3 · p. 4
  • For a detailed parameter θ, behavioral compatibility is the set of reduced-model parameters whose outputs lie within tolerance ϵ of its output.The reduced posterior probability mass on that compatible set defines the behavioral support assigned to θ.Source: §3 · p. 4
  • A support level γ defines the detailed-parameter region whose behavioral support meets or exceeds that threshold.When support is continuous, the region's relative boundary lies within the level set bϵ(θ) = γ, which is learned to filter detailed-model proposals.Source: §3 · p. 5
  • The formulation proposes a data-dependent behavioral prior based on the learned support region.Thresholding support yields a hard-restricted posterior whose support is limited to Kϵ,γ rather than preserving baseline-posterior support.Source: §3 · p. 5, Thresholding the behavioral support… · p. 5

Thresholding the behavioral support instead gives the hard-restricted posterior

  • The hard-restricted posterior is proper when its stated normalization condition holds.The construction is nonnegative and has a finite positive normalizing constant under that condition.Source: Thresholding the behavioral support… · p. 5

4.1 Nonlinear models

Approximate simulation relations provide a way to bound differences between detailed and reduced model outputs over a finite horizon. The resulting compatibility tolerance supports behavioral support estimation, and a sufficient posterior-mass condition guarantees retention of detailed parameters that meet the desired data-fit tolerance.

4.1 Nonlinear models

  • The proposed tolerance choice uses approximate simulation relations to link instantaneous observation precision to the finite-horizon compatibility tolerance.The relation is specified with observation precision ϵ/T over the horizon [0, T].Source: §4.1 · p. 5
  • A behavioral compatibility tolerance ϵ bounds the directed Hausdorff distance between detailed and reduced behavior sets under the stated approximate simulation-relation conditions.The conditions include a related fixed reduced parameter for each detailed parameter, a compatible initial pair, and transition matching over the horizon.Source: §4.1 · p. 5
  • The support can be consistently estimated from samples of a reduced-model Markov chain with invariant distribution equal to the reduced posterior.The stated result assumes the chain satisfies the ergodic theorem for bounded measurable functions.Source: §4.1 · p. 6
  • Continuity of the detailed output map and zero reduced-posterior probability on the compatibility boundary imply a closed support region and a well-posed level-set learning problem.These conditions precede use of the learned set to accelerate MCMC.Source: §4.1 · p. 6
  • If γ ≤ m̂π(ϵ − η), then every detailed parameter whose output fits the data within η belongs to the behavioral-support region Kϵ,γ.The result requires ϵ > η and γ ∈ (0, 1); the reduced posterior must assign sufficient mass to trajectories within ϵ − η of the data.Source: §4.1 · p. 6
  • For each detailed parameter in the desired fit set, the triangle inequality bounds its output discrepancy from a compatible reduced trajectory by η + (ϵ − η) = ϵ.This bound is the step used to establish inclusion in the behavioral-support region.Source: §4.1 · p. 6

4.2 Linear models with parametric initial conditions

For linear systems whose parameters occur only in initial conditions, the optimal reduced initial condition is a finite-horizon least-squares projection. This yields closed-form compatibility bounds and behavioral support in the specialized linear case.

4.2 Linear models with parametric initial conditions

  • The optimal reduced initial condition is the unique finite-horizon least-squares projection of the detailed initial condition.The result applies when parameters appear only in the initial conditions.Source: §4.2 · p. 6
  • If the reduced parameter domain contains each optimal reduced initial condition, the directed Hausdorff distance is the supremum of the minimum squared output discrepancy over detailed initial conditions.For detailed initial conditions bounded by R in Euclidean norm, the distance is bounded by R times the square root of the largest eigenvalue of E_T.Source: §4.2 · p. 7
  • For a fixed detailed initial condition, compatibility holds exactly when ϵ^2 − θ^T E_T θ ≥ 0; the compatible reduced parameters form the domain intersected with an ellipsoid centered at the optimal reduced initial condition.The matrix E_T is positive semidefinite.Source: §4.2 · pp. 6–7

4.2.1 Behavioral support in the specialized linear case

  • In the specialized linear case, behavioral support is the reduced-posterior probability mass of the compatibility set, and its level set can be computed directly when ellipsoidal probabilities are analytic.The closed-form formulation is used to validate and visualize the GP level-set learning procedure.Source: §4.2.1 · p. 7

4.3 Computational Implementation

The implementation learns a behavioral-support level set with a Gaussian process and uses it to screen detailed-model MCMC proposals. Thresholds control the efficiency–exploration tradeoff, while the cost analysis identifies when setup and filtering costs are offset by avoided simulations.

4.3.1 Efficiency-exploration tradeoff

  • Increasing ϵ retains a larger detailed-parameter region, while increasing γ makes the retained set smaller, exposing an efficiency–exploration tradeoff.The thresholds can be tuned, subject to the retention conditions for the desired data-fit level.Source: §4.3.1 · p. 7

4.3.2 Learning the behavioral compatibility level set using GPs

  • A Gaussian process approximates the behavioral-support level set, and proposals with posterior membership probability at or below ζ are rejected before detailed-model simulation.The GP is trained on evaluated parameters and empirical validity values, then used as a behavioral prior during detailed-model MCMC.Source: §4.3.2 · p. 8, §4.3.1 · p. 8

4.3.3 Threshold selection

  • ϵ sets the maximum accepted output discrepancy, η the desired data-fit tolerance, γ the minimum reduced-posterior compatibility mass, and ζ the minimum GP membership probability for simulation.Increasing γ or ζ makes the behavioral restriction more selective.Source: §4.3.3 · p. 8

4.3.4 Computational efficiency analysis

  • The expected number of detailed simulations avoided during MCMC is N(1 − pK), and guided inference is favorable when their cost exceeds reduced-model inference, GP training, and filtering costs.Reduced-model inference and GP training are one-time setup costs; reported improvements compare measured guided and baseline MCMC elapsed times.Source: §4.3.4 · p. 9

5 Applications

The illustrative linear-system example compares baseline and GP-guided MCMC sampling against a learned behavioral compatibility region. It sets thresholds from the observation-noise setup and displays the resulting region and sampling behavior.

5.1 Illustrative linear system example

  • The baseline MCMC explores outside the learned GP acceptance region in the illustrative example.Figure 1 compares baseline samples with GP-guided samples; the compatibility ellipse is computed at the reduced-posterior mean α ≈ 1.627.Source: Figure 1 · p. 9, §5.1 · p. 10
  • The example uses a two-state detailed linear model and a one-state reduced model, with synthetic observations generated using Gaussian noise of standard deviation 0.02.The detailed ground truth is approximately (0.96, 0.64), and the reduced posterior and GP training use the described inference setup.Source: §5.1 · pp. 9–10
  • With γ = 0.80 and ζ = 0.95, the selected thresholds are η ≈ 0.0656, qγ ≈ 0.1747, and ϵ ≈ 0.2403, satisfying Theorem 2's stated condition.The empirical behavioral compatibility region is shown in Figure 1a.Source: §5.1 · p. 10

5.2 Inference of gene-expression parameters

The study applies a three-level gene-expression model hierarchy to cell-free experimental data, using a high-level model’s posterior to guide inference in more detailed models. GP-guided inference reduced MCMC runtime for both detailed models, with fit errors and posterior distributions also compared against baseline inference.

5.2.2 Hierarchical modeling

  • The hierarchy runs from a two-state mRNA/GFP model to a nine-state phenomenological model and a 38-state mechanistic model, with GPhp and GPhm mapping from the high-level model.AutoReduce derives the high-level dynamics; BioCRNpyler constructs the phenomenological and mechanistic models.Source: §5.2 · p. 10, §5.2.2 · pp. 10, 12

5.2.1 Experimental data

  • The data comprise three replicate deGFP time series at six initial DNA concentrations, retained from 0.25 to 3 hours to exclude lag and stationary expression.Fluorescence was converted to deGFP concentration using 5542.4 fluorescence units per µM.Source: §5.2.1 · p. 10

5.2.4 Behavioral mapping and GP-guided inference

  • At ζ = 0.5, balanced test errors were 8.22% for GP-MPM and 12.30% for uniform random sampling with GPhp, and 0.94% and 2.57%, respectively, with GPhm.These comparisons used 200 independent test points through 200 acquisitions.Source: Figure 3 · p. 12, §5.2.4 · p. 13

5.2.3 Baseline Bayesian inference

  • Baseline mean condition-wise MAP RMSEs were 0.255 µM for the high-level model, 0.272 µM for the phenomenological model, and 0.327 µM for the detailed model.Baseline inference used 256 walkers and 40,000 steps for the detailed models, and 64 walkers and 10,000 steps for the two-state model.Source: Figure 2 · p. 11, §5.2.3 · p. 12

5.2.4 Behavioral mapping and GP-guided inference

  • At ζ = 0.8, GPhp retained 47.73% of compatible test points and rejected 99.36% of incompatible points; GPhm retained 96.23% and rejected 100.00%, respectively.At the stricter filtering level, balanced errors were 26.46% for GPhp and 1.89% for GPhm.Source: Figure 3 · p. 12, §5.2.4 · p. 13

5.2.5 Inferred trajectories and parameter distributions

  • Guided mean condition-wise MAP RMSE was 0.361 µM versus 0.272 µM baseline for the phenomenological model, and 0.309 µM versus 0.327 µM for the mechanistic model.Figure 5 compares baseline and guided GFP trajectories at six DNA concentrations.Source: §5.2.5 · p. 14, Figure 5 · p. 14

5.2.6 Computational performance

  • 8.02% and 15.21%: GP-guided MCMC reduced runtime for the phenomenological and mechanistic models, respectively, after one-time setup.Reduced-model inference took 6.64 minutes and GP training took 4.46 minutes as one-time setup costs.Source: §5.2.6 · pp. 14–15, Table 1 · p. 15

6 Discussion

Behavioral filtering can focus inference on parameter regions that realize behavior of interest, particularly in non-identifiable models. This speeds exploration but trades off coverage of compatible parameters, with the user able to tune the balance through tolerances.

6 Discussion

  • In non-identifiable models, behavioral filtering focuses detailed inference on parameter combinations that realize the behavior of interest in a relevant region.Exploring distant parameter ranges can require many detailed simulations while producing similar measured trajectories.Source: §6 · pp. 15–16
  • GP-guided inference trades faster exploration for reduced exploration of the full range of compatible parameters.The designer should train the GP on a parameter region considered relevant because identified combinations may not be implementable.Source: §6 · pp. 15–16
  • Behavioral filtering narrows the mechanistic model marginals of ktx and ku,tl, while kb,tx and ku,tx remain broad and the ktl–kdil association persists.The baseline and guided chains both retain the strong positive association between ktl and kdil.Source: §6 · p. 16
  • Users can tune the exploration–behavioral-focus balance through the compatibility tolerance ϵ, required posterior compatibility mass γ, and GP-filter strictness ζ.The framework requires a common measured output across hierarchy models and does not require state or parameter correspondence; future work would extend mapping across successive levels.Source: §6 · p. 16

7 Conclusion

The paper presents behavioral mapping to accelerate Bayesian inference across hierarchical dynamical models without requiring parameter correspondence. Its gene-expression application reports faster MCMC for both the phenomenological and mechanistic models.

7 Conclusion

  • Behavioral guidance reduces MCMC time by 8.02% for the nine-state phenomenological model and 15.21% for the 38-state mechanistic model after initial setup costs.The application connects a two-state high-level model to the two lower-level models.Source: §7 · p. 16
  • The behavioral mapping framework transfers posterior information through behavioral compatibility and a learned Gaussian Process support level set, without requiring parameter correspondence.Source: §7 · p. 16
  • The framework formalizes behavioral compatibility across model hierarchies and is described as applicable to other systems.The paper compares inferred parameter distributions and GFP fits with baseline inference.Source: §7 · p. 16
Loading 2610.30683v1…