PaperlayerView exact revision on arXiv

Source-linked AI summary

Joining Forces of Bayesian and Frequentist Methodology: A Study for Inference in the Presence of Non-Identifiability

Andreas Raue, Clemens Kreutz, Fabian Joachim Theis, Jens Timmer

arXiv:1202.4605v1physics.data-an

TL;DR

Limited experimental data and complex models can leave posterior distributions insufficiently constrained for feasible MCMC sampling. The paper combines profile likelihood to detect and address non-identifiability with MCMC sampling. In the cell-biology comparison, the methods agree for the extended identifiable setup, while their results differ substantially for the initial setup.

  • Problem

    Limited data and nonlinear models with many unknown parameters can leave posterior distributions insufficiently constrained, making MCMC sampling infeasible.

    Source: §1 · p. 2

  • Method

    The paper uses profile likelihood to detect non-identifiability and guide experimental design, then applies MCMC when the posterior is sufficiently constrained.

    Source: §2 · pp. 5–6

  • Results

    For the extended identifiable setup, posterior profiling and MCMC agree; for the initial setup, their results differ substantially, and profiling better reflects the extended setup than initial-setup MCMC samples.

    Source: §3 · p. 10

  • Takeaways & Limitations

    The authors suggest successive use of profiling and MCMC to support realistic assessment of uncertainty in model predictions.

    Source: §1 · p. 2

    For the initial setup, an artificial uniform prior made the posterior proper but could bias MCMC results.

    Source: §2 · p. 7

Abstract

from arXiv · show

Increasingly complex applications involve large datasets in combination with non-linear and high dimensional mathematical models. In this context, statistical inference is a challenging issue that calls for pragmatic approaches that take advantage of both Bayesian and frequentist methods. The elegance of Bayesian methodology is founded in the propagation of information content provided by experimental data and prior assumptions to the posterior probability distribution of model predictions. However, for complex applications experimental data and prior assumptions potentially constrain the posterior probability distribution insufficiently. In these situations Bayesian Markov chain Monte Carlo sampling can be infeasible. From a frequentist point of view insufficient experimental data and prior assumptions can be interpreted as non-identifiability. The profile likelihood approach offers to detect and to resolve non-identifiability by experimental design iteratively. Therefore, it allows one to better constrain the posterior probability distribution until Markov chain Monte Carlo sampling can be used securely. Using an application from cell biology we compare both methods and show that a successive application of both methods facilitates a realistic assessment of uncertainty in model predictions.

1. Introduction

Complex models with limited experimental data can leave parameter uncertainty insufficiently constrained, making reliable inference difficult. The introduction presents profile likelihood as a way to diagnose and iteratively address non-identifiability before using Bayesian MCMC to assess uncertainty in predictions.

1. Introduction

  • Limited data and nonlinear, high-dimensional models can leave the posterior too diffuse for feasible MCMC sampling.The paper frames this as a barrier to propagating uncertainty reliably to model predictions.Source: §1 · p. 2
  • Profile likelihood detects structural and practical non-identifiability, and can guide iterative experimental design to resolve it.At each fixed parameter value, the other parameters are re-optimized; profile shapes distinguish flat structural non-identifiability, plateauing practical non-identifiability, and identifiability when they decay sufficiently quickly on both sides.Source: §1 · pp. 2–3
  • The authors report that MCMC and profile-likelihood results agree without non-identifiability but can differ substantially when it is present.They caution that MCMC results in the latter setting can be misleading and propose applying the methods successively.Source: §1 · p. 2
  • Model likelihood describes agreement between model output and experimental data, while its shape contains information about parameter-estimate uncertainty.The paper introduces the likelihood under normally distributed measurement noise.Source: §1 · p. 2
  • A profile-based confidence interval includes parameter values whose likelihood-ratio statistic falls below the threshold for confidence level α.The threshold is the α quantile of a χ2 distribution with the specified degrees of freedom.Source: §1 · p. 3
  • Bayesian inference combines the likelihood with a prior to form a posterior, which MCMC can sample without directly computing its normalizing constant.For complex models, non-identifiable parameters can make the posterior improper, preventing chain convergence; even a proper posterior may be impractically slow to sample when it is insufficiently constrained.Source: §1 · pp. 4–5

2. Results

In the cell-biology application, profile-guided experimental design resolved identifiability problems, after which profiling and MCMC results agreed across parameters. The resulting samples supported uncertainty assessment for predicted molecular dynamics that could not be measured directly.

2. Results

  • The application used a six-equation ODE model with ten unknown parameters, including one nuisance parameter, to describe Epo–EpoR molecular dynamics.Radioactively labeled Epo measurements supported fitting binding, internalisation, and recycling dynamics; the initial setup measured extracellular Epo and membrane-bound Epo.Source: §2 · pp. 6–7
  • The initial setup had four structurally non-identifiable, two practically non-identifiable, and three identifiable parameters.One example of each case was displayed in Figure 3A; the other profiles were in supplementary figure 1.Source: §2 · p. 7
  • With additional experimental data, all parameters became identifiable, the posterior became proper without artificial prior assumptions, and profiling and MCMC agreed, including for kt.The extended-setup kt samples clustered near the initial setup’s MAP point, suggesting the initial samples at log10(kt) > 0 were misleading.Source: Figure 3 · p. 8, §2 · p. 9
  • For initial-setup kt, the profile placed the MAP point near log10(kt) ≈ −1.8, while most MCMC samples proposed log10(kt) > 0.Profiling and sampling agreed for identifiable kde and structurally non-identifiable kon, but differed substantially for practically non-identifiable kt.Source: Figure 3 · p. 8, §2 · p. 8
  • Extended-setup MCMC samples propagated parameter uncertainty to predictions of three molecular components that could not be measured directly.The samples enabled consideration of the full high-dimensional posterior for inference.Source: Figure 4 · p. 9, §2 · p. 9

3. Summary

The paper combines frequentist profiling with Bayesian MCMC, using profiles to detect non-identifiability and guide experiments until sampling can be used reliably. Its cell-biology application finds that profiling and MCMC can disagree under non-identifiability, motivating successive use of both methods.

3. Summary

  • The joint approach uses posterior profiles to assess parameter identifiability and guide iterative experimental design, helping ensure a proper posterior for reliable MCMC and uncertainty propagation to predictions.The authors report accurate results with this approach in a cell-biology application.Source: §3 · p. 10
  • With non-identifiability, initial-setup profiles reflected the identifiable extended setup better than the initial-setup MCMC samples, which could be inaccurate despite a proper posterior.A uniform prior made the initial setup's posterior proper; agreement between profiling and marginalised MCMC samples was observed for the extended setup.Source: §3 · p. 10

Supplementary Notes and Figures for Joining Forces of Bayesian and Frequentist Methodology: A Study for Inference in the Presence of Non-Identifiability

The supplementary material identifies the paper's authors, their affiliations, and an arXiv posting dated 21 February 2012.

Supplementary Notes and Figures for Joining Forces of Bayesian and Frequentist Methodology

1. Model description and experimental setup.

The cell-biology application models Epo and its receptor with six concentration variables and ten unknown parameters, then compares an initial measurement setup with an extended setup designed to resolve non-identifiability. The reported MCMC results depend on both the setup and sampling algorithm.

1. Model description and experimental setup

  • The Epo/EpoR model describes six molecular-compound concentrations with an ODE system and ten unknown parameters, including one nuisance parameter.The model maps compound dynamics to experimentally accessible quantities.Source: §1 · p. 13
  • The model represents Epo binding to EpoR, receptor internalisation and recycling, and intracellular or extracellular degradation of the ligand-receptor complex.Its variables include extracellular Epo, membrane EpoR and occupied complex, internalized complex, and degraded Epo in two compartments.Source: §1 · pp. 13–14
  • Radioactively labeled Epo was measured over time in extracellular, membrane, and intracellular compartments; the model maps these quantities to y1, y2, and y3.The measurements were collected in triplicate and reported as counts per minute.Source: §1 · p. 14
  • The initial setup measurements of y1 and y2 resulted in non-identifiability; adding y3 measurements and direct measurements of Bmax, kD, and Epo0 yielded an identifiable extended setup.The additional measurements were selected based on profiling results to resolve non-identifiabilities.Source: §1 · p. 14
  • MMALA converged for both setups, whereas SIM remained correlated and had not converged for either the initial or extended setup.The passage reports the algorithm outcomes across both experimental setups.Source: §1 · p. 14

2. Implementation of the numerical methods.

The numerical workflow uses CVODES to solve the ODE system and LSQNONLIN to estimate MAP parameters, with sensitivity equations supporting methods that require first-order sensitivities. Sampling convergence is monitored through autocorrelation and thinning, and posterior samples are propagated to predictions of unobserved compounds.

2. Implementation of the numerical methods

  • CVODES solved the ODE system, while MATLAB's LSQNONLIN trust-region method produced MAP estimates using local gradient and curvature information.Posterior profiles were calculated with an algorithm cited as.Source: §2 · p. 14
  • Sensitivity equations were solved alongside the ODE because MMALA and LSQNONLIN rely on accurate first-order sensitivities, for which finite differences should not be used in ODE systems.The simplified MMALA implementation was chosen for computational efficiency.Source: §2 · pp. 14–15
  • The SIM algorithm remained impractical in both setups: thinning of 1/100 did not yield independent samples, while MMALA was more efficient and converged.Thinning of 1/100 for the initial setup and 1/10 for the extended setup was sufficient for MMALA to obtain 104 independent samples.Source: §2 · p. 15
  • Posterior parameter uncertainty was propagated to unobserved-compound trajectories by evaluating trajectories for sampled parameters and estimating density at each time point.The density estimate used MATLAB's KSDENSITY function.Source: §2 · p. 15
  • Profile calculations took approximately 5 minutes for the initial setup and 1 minute for the extended setup; MMALA sampling took approximately 18.5 hours and 20 minutes, respectively.Source: §2 · p. 15
Loading 1202.4605v1…