Source-linked AI summary
Flexible statistical inference for mechanistic models of neural dynamics
Jan-Matthis Lueckmann, Pedro J. Goncalves, Giacomo Bassetto, Kaan Öcal, Marcel Nonnenmacher, Jakob H. Macke
TL;DR
Mechanistic neural models are hard to connect quantitatively to data because many have intractable likelihoods and existing fitting methods lack full posterior inference. The paper introduces SNPE, which adaptively trains a Bayesian mixture-density network with simulator data while handling failed simulations, missing features, and raw time series. It estimates posterior distributions on synthetic and in-vitro single-neuron data, including multivariate Hodgkin–Huxley parameter posteriors that produce voltage traces similar to recordings.
Problem
Many neural-dynamics models have intractable or expensive likelihoods, while heuristic fitting methods do not provide full posterior distributions.
Method
SNPE uses adaptive simulation rounds to train a Bayesian mixture-density network for likelihood-free posterior estimation, with extensions for failed simulations, missing features, and learned time-series features.
Results
The approach estimates posterior distributions on synthetic data and recovers multivariate Hodgkin–Huxley parameter posteriors whose voltage traces are similar to in-vitro recordings.
Takeaways & Limitations
Likelihood-free density estimation provides a flexible route to Bayesian inference for complex single-neuron models without model-specific inference algorithms.
Takeaways & Limitations
The proposed posterior-divergence calibration kernel was suggested but not investigated.
Abstract
from arXiv · showhide
Mechanistic models of single-neuron dynamics have been extensively studied in computational neuroscience. However, identifying which models can quantitatively reproduce empirically measured data has been challenging. We propose to overcome this limitation by using likelihood-free inference approaches (also known as Approximate Bayesian Computation, ABC) to perform full Bayesian inference on single-neuron models. Our approach builds on recent advances in ABC by learning a neural network which maps features of the observed data to the posterior distribution over parameters. We learn a Bayesian mixture-density network approximating the posterior over multiple rounds of adaptively chosen simulations. Furthermore, we propose an efficient approach for handling missing features and parameter settings for which the simulator fails, as well as a strategy for automatically learning relevant features using recurrent neural networks. On synthetic data, our approach efficiently estimates posterior distributions and recovers ground-truth parameters. On in-vitro recordings of membrane voltages, we recover multivariate posteriors over biophysical parameters, which yield model-predicted voltage traces that accurately match empirical data. Our approach will enable neuroscientists to perform Bayesian inference on complex neuron models without having to design model-specific algorithms, closing the gap between mechanistic and statistical approaches to single-neuron modelling.
1 Introduction
Mechanistic neural models are difficult to fit statistically because their likelihoods are often unavailable or expensive, while existing heuristic methods do not provide full posterior uncertainty. The paper proposes likelihood-free Bayesian inference using adaptive neural posterior estimation for single-neuron models.
- Many neural-dynamics models are defined only through simulators with intractable or computationally expensive likelihoods.
- Heuristic feature-matching methods require difficult multi-feature trade-offs, scale poorly in high-dimensional parameter spaces, and do not estimate full posterior distributions.
- Approximate Bayesian Computation compares simulated and empirical summary features while providing a principled framework for full Bayesian inference.
- SNPE sequentially trains a mixture-density network across multiple rounds of adaptively chosen simulations.
- The approach addresses failed simulations and missing features, and can learn relevant features directly from time-series data with recurrent neural networks.
2 Methods
The methods train a posterior density estimator on simulator outputs using proposal-prior correction and adaptive rounds, while extending inference to unstable simulations, missing features, and raw time series. Bayesian treatment of network weights supports continual learning across rounds.
- Sequential Neural Posterior Estimation: SNPE trains qφ(θ|x) with proposal-prior importance weights and a calibration kernel so it approximates the posterior at observed data xo.
- Sequential Neural Posterior Estimation: Adaptive rounds initialise the proposal at the prior and use each round’s posterior as the next proposal, supporting multimodal and heavy-tailed distributions.
- Training the posterior model: Bayesian inference over MDN weights lets the posterior network reuse simulations from previous rounds without storing all old samples or retraining from scratch.
- Bad simulations and features: An uninvestigated extension would define calibration-kernel similarity using divergence between the posterior distributions assigned to simulated and observed data.
- Bad simulations and features: A classifier predicts parameter settings likely to produce failed simulations, allowing SNPE to reject them before expensive simulation and modify the effective prior.
- Bad simulations and features: Missing features are represented with binary indicators, while an RNN can learn informative summaries directly from recorded neural time series.
3 Results
SNPE accurately and robustly estimates posteriors across simple models, Hodgkin–Huxley simulations, and in-vitro recordings, while addressing unstable simulations and missing features.
- 3.1 Statistical inference on simple models: SNPE recovers ground-truth parameters in Gaussian mixtures, GLMs, and simulated Hodgkin–Huxley models.For Hodgkin–Huxley simulations, true parameters lie near the inferred posterior mode and in a region of high posterior probability.
- 3.1 Statistical inference on simple models: SNPE accurately and robustly estimates posterior distributions in simple models, including multimodal posteriors where CDE-LFI performs worse.On GLMs, SNPE produces posterior estimates similar to PG-MCMC.
- 3.2 Statistical inference on Hodgkin–Huxley neuron models: For mouse visual-cortex recordings, the posterior mode produces a voltage trace similar to the data, while posterior samples describe parameter regions preserving model output.The inferred posterior covers 12 Hodgkin–Huxley parameters for cell 464212183.
- 3.3 Dealing with bad simulations and features: A classifier learns parameter regions that produce diverging simulations, improving simulation efficiency and identifying where the simulator is well-defined.The approach is demonstrated on an autapse model whose stability boundary is known.
- 3.3 Dealing with bad simulations and features: An augmented mixture-density network imputes missing feature values, allowing simulations with undefined features to remain usable.Missing features arise frequently when parameter regimes produce non-spiking behavior, such as undefined spike latency.
- 3.4 Learning features with recurrent neural networks: Recurrent neural networks are proposed for automatically learning relevant features from neural-model data.The paper motivates this strategy because hand-designed features may be informative but the space of possible features is immense.
4 Discussion
The paper presents SNPE as a flexible likelihood-free Bayesian inference method for neural models, extending density-estimation ABC to unstable simulations, missing features, and automatically learned features. It argues that the approach supports posterior estimation under common constraints in neural models and can extend beyond single neurons.
- Learned features: Recurrent neural networks can automatically define features from complete voltage traces and input currents instead of relying on hand-engineered summaries.A many-to-one recurrent network maps T time steps to an N-dimensional feature vector.
- Scope: The approach is presented as applicable to single-neuron models and potentially to populations of neurons.The authors frame this as extending Bayesian inference to complex neural models without model-specific algorithms.
Algorithm 1: Training SNPE
Training SNPE proceeds through sequential rounds that sample parameters, simulate observations, and retrain a Bayesian mixture-density network. Continual learning is implemented through a KL-divergence term that carries information about earlier rounds into later training.
- Algorithm 1: Training SNPE: Each round samples parameters from the current proposal and generates simulated observations from the simulator.The proposal is initialized with the prior before the first round.
- Algorithm 1: Training SNPE: The Bayesian mixture-density network is retrained after each simulation round, with optional neural-network expansion.The network is initialized with K mixture components.
- Implementation: Importance weights are normalized per round, and optimization uses Adam with gradient norms capped at 0.1.The weight prior precision is fixed to λ = 0.01.
- Continual learning through the DKL-term: The KL-divergence term implements continual learning by treating the previous round’s posterior as the current round’s prior.This allows information from earlier simulations to influence later variational inference.
- Continual learning through the DKL-term: The KL penalty on changes in posterior means increases across rounds as the previous posterior precision increases.The penalty is a quadratic term weighted by the previous round’s posterior precision.
D.1 Mixture-models
The supplementary experiments apply SNPE to mixture models, a generalized linear model, and Hodgkin–Huxley neurons using progressively richer parameter and feature settings. These sections describe the model forms, inference configurations, and comparison procedures.
- D.1 Mixture-models: SNPE is evaluated on two distinct Gaussian mixture models, including a two-component mixture with a common mean.The first model uses α = 0.5, σ1 = 1, σ2 = 0.1, and a uniform prior on θ over (−10, 10).
- D.1 Mixture-models: For the first mixture model, SNPE and CDE-LFI are run for 6 rounds with 1000 samples per round.SNPE is initialized with two mixture components.
- D.2 Generalised linear model: The generalized linear model uses a 10-dimensional parameter vector and cross-correlations between input and response as summary statistics.The prior is Gaussian with covariance σ²(F^⊤F)^−1, and SNPE uses 5 rounds of 5000 simulations.
D.4 Inference in in-vitro recordings from Allen Cell-type database
SNPE is applied to in-vitro mouse visual-cortex recordings to infer a posterior over 12 Hodgkin–Huxley parameters using five rounds of simulations.
- SNPE infers a posterior over 12 parameters from in-vitro recordings in the Allen Cell Type Database.The application uses mouse visual-cortex recordings and the same prior as the simulated Hodgkin–Huxley data.
- The in-vitro application runs five rounds with 5000 Hodgkin–Huxley simulations per round.The posterior is fixed as a mixture of two Gaussians.
- Autapse: The autapse system has unstable dynamics when J > 1.This condition follows directly from the model’s governing equation.
- Autapse: The autapse inference targets connection strength J and time constant τ using the trace mean as the feature of interest.The prior over both parameters is uniform, and the true parameters are (0.75, 1).
E Bad features
Non-spiking Hodgkin–Huxley simulations produce undefined spike-dependent features, so SNPE imputes their values during training.
- Many prior-sampled Hodgkin–Huxley parameter sets produce non-spiking behaviour, leaving spike-dependent features such as first-spike latency undefined.The issue occurs because latency is defined only when spikes are present.
- SNPE imputes undefined feature values through an additional layer whose values are learned during mixture-density-network training.The imputation is optimized alongside the network parameters.
- In one posterior example, the imputed latency is close to the mean latency, but this agreement is not generally observed.The figure compares latency with firing rate for posterior samples.
F Comparison with Sequential Monte-Carlo ABC
In a 10-parameter GLM, SNPE recovers reference posterior statistics more effectively than SMC-ABC despite using far fewer simulations.
- SNPE consistently performs better than SMC-ABC on a 10-parameter GLM, even when SMC-ABC receives orders of magnitude more simulations.This comparison is reported as evidence for performance in a higher-dimensional inference problem.
- SNPE recovers PG-MCMC posterior means, variances, and covariances well after 25000 simulations.The figure reports these statistics as the reference quantities for comparison.
- SMC-ABC performs worse with over 4 × 10^6 simulations in the same comparison.The SMC-ABC setup uses 1000 particles and a decreasing tolerance sequence.
G Supplementary Figures
Supplementary figures visualize full posterior estimates for GLM and Hodgkin–Huxley data, alongside parameter and noise-feature values.
- G.1 GLM posterior: Figure G.1 compares SNPE and PG-MCMC two-dimensional GLM posterior marginals against ground-truth parameter values.White and yellow contours represent 68% and 95% of SNPE posterior mass.
- Parameter panels: Supplementary parameter panels include ln g_Na, ln V_T, ln noise, ln g_K, and ln k_bn2.
- G.2 Hodgkin–Huxley posterior: Figure G.2 presents the full posterior inferred for synthetic Hodgkin–Huxley data.
- G.3 Real-data posterior: Figure G.3 presents the full posterior inferred for real Hodgkin–Huxley data.