PaperlayerView exact revision on arXiv

Source-linked AI summary

A Tutorial on Bridge Sampling

Quentin F. Gronau, Alexandra Sarafoglou, Dora Matzke, Alexander Ly, Udo Boehm, Maarten Marsman, David S. Leslie, Jonathan J. Forster, Eric-Jan Wagenmakers, Helen Steingroever

arXiv:1703.05984v2stat.CO

TL;DR

The paper addresses the difficulty of approximating marginal likelihoods when they are not analytically tractable. It presents bridge sampling and applies it to the EV model, finding accurate estimates for individual and hierarchical models.

  • Problem

    Marginal likelihoods are important in Bayesian modeling but are often analytically intractable and require numerical approximation.

  • Method

    The tutorial introduces bridge sampling and related methods, then applies bridge sampling to the Expectancy Valence model.

  • Results

    Bridge sampling provides accurate marginal-likelihood estimates for individual and hierarchical models, with accuracy, reliability, practicality, and ease of implementation.

  • Takeaways & Limitations

    Bridge sampling offers a reliable, easy-to-implement approach for estimating model marginal likelihoods.

  • Takeaways & Limitations

    Bridge sampling should not be used blindly, and it is unnecessary when the marginal likelihood can be determined analytically.

Abstract

from arXiv · show

The marginal likelihood plays an important role in many areas of Bayesian statistics such as parameter estimation, model comparison, and model averaging. In most applications, however, the marginal likelihood is not analytically tractable and must be approximated using numerical methods. Here we provide a tutorial on bridge sampling (Bennett, 1976; Meng & Wong, 1996), a reliable and relatively straightforward sampling method that allows researchers to obtain the marginal likelihood for models of varying complexity. First, we introduce bridge sampling and three related sampling methods using the beta-binomial model as a running example. We then apply bridge sampling to estimate the marginal likelihood for the Expectancy Valence (EV) model---a popular model for reinforcement learning. Our results indicate that bridge sampling provides accurate estimates for both a single participant and a hierarchical version of the EV model. We conclude that bridge sampling is an attractive method for mathematical psychologists who typically aim to approximate the marginal likelihood for a limited set of possibly high-dimensional models.

3 Mathematical Science, University of Southampton, UK

The marginal likelihood is central to Bayesian parameter estimation, model comparison, and model averaging, but is analytically available only in restricted cases. This tutorial introduces bridge sampling and related estimators to approximate it for complex, potentially high-dimensional mathematical-psychology models.

  • The tutorial specifically aims to bring bridge sampling to mathematical psychologists working with possibly high-dimensional models.Its motivating applications include hierarchical modeling and other complex problems.
  • The marginal likelihood supports parameter estimation, model comparison, and Bayesian model averaging.It normalizes posterior distributions, contributes to posterior model probabilities and Bayes factors, and weights model-averaged inference.
  • Analytical marginal likelihood expressions are available only for restricted examples, creating a pressing problem for nonlinear and hierarchical models.Hierarchical models can quickly become high-dimensional; one example reaches 128 parameters.
  • The tutorial focuses on bridge sampling and three special cases: naive Monte Carlo, importance sampling, and generalized harmonic mean estimation.The estimators are introduced progressively through a beta-binomial model, from simpler methods to the general bridge-sampling case.
  • Bridge sampling is presented as accurate, efficient, and relatively straightforward to implement for complex problems.The tutorial aims to explain the estimator and facilitate application through a step-by-step implementation scheme.

Analytical derivation of the marginal likelihood.

The beta-binomial model provides an analytically tractable running example whose marginal likelihood is then approximated with increasingly sophisticated Monte Carlo estimators. The examples illustrate how sampling choices affect estimator accuracy and stability.

  • For k = 2 correct responses in n = 10 trials under a Beta(1, 1) prior, the exact marginal likelihood is p(k = 2 | n = 10) = 1/11 ≈0.0909.The beta-binomial model is used because its marginal likelihood and posterior can be derived analytically.
  • The posterior for θ is Beta(3, 9) after observing 2 correct responses out of 10 trials.With a uniform prior, the posterior follows Beta(k + 1, n − k + 1).
  • The naive Monte Carlo estimator averages likelihood values evaluated at samples drawn from the prior distribution.Its estimate is based on N prior samples, with larger N recommended in practice.
  • Importance sampling can become unstable when the posterior is peaked relative to the prior, because a few high-likelihood samples can dominate the estimate and increase variance.The method therefore depends on suitable overlap and tail behavior between prior, posterior, and importance distributions.
  • Importance sampling instead draws from an importance density that samples high-likelihood regions more frequently than low-likelihood regions.A suitable density should resemble the posterior, be easy to evaluate, cover its domain, and have fatter tails.
  • The importance density is illustrated as a beta mixture combining a beta density fitted to the posterior with a uniform density across θ.Increasing γ increases the uniform component, producing a less peaked distribution with thicker tails.

hood

The generalized harmonic mean estimator uses posterior samples and an importance density with thinner tails, but finding a suitable density can be difficult in high dimensions. Bridge sampling relaxes these requirements by combining samples from a proposal distribution and the posterior distribution.

  • The generalized harmonic mean estimator, also called reciprocal importance sampling, evaluates samples from the posterior distribution.
  • A suitable importance density should have thinner tails than the posterior, be easy to evaluate, share its domain, and closely resemble it.Thinner tails help avoid inflated ratios in the estimator's summation.
  • Running example: In the beta-binomial example, posterior samples are probit-transformed to the real line and used to fit a normal importance density.The transformation matches the posterior's domain to that of the normal density.
  • Running example: The illustrative importance density uses mean µ̂ = −0.793 and standard deviation σ̂ = 0.423/1.5 to ensure thinner tails.The division by 1.5 was chosen for illustrative purposes only.
  • Bridge sampling: Bridge sampling estimates the marginal likelihood using samples from both a proposal distribution and the posterior distribution.The proposal should resemble the posterior and have sufficient overlap with it.
  • Bridge sampling: Bridge sampling imposes less strict proposal-tail requirements than importance sampling and generalized harmonic mean estimation, making it more robust to tail behavior.Its ratios are bounded and therefore do not inflate in the same way.

Interim Summary

The tutorial presents bridge sampling as the most general of four marginal-likelihood estimators and focuses on its use with mathematical-psychology models. It specifically targets accurate estimation for hierarchical models.

  • The beta-binomial example introduces four marginal-likelihood estimators in increasing sophistication, culminating in bridge sampling.The first three estimators provide context for the bridge sampler.
  • The remainder of the tutorial focuses on demonstrating bridge sampling for popular models in mathematical psychology.
  • Bridge sampling may obtain accurate marginal-likelihood estimates for hierarchical models.

Assessing the Accuracy of the Bridge Sampling Estimate

The tutorial assesses bridge-sampling accuracy through variance measures and applications to the EV model using individual-level and hierarchical data. It compares bridge-sampling estimates with established alternatives.

  • Accuracy assessment: Bridge-sampling accuracy is quantified using the coefficient of variation, while accounting for independent proposal samples and autocorrelated posterior MCMC samples.The posterior contribution is adjusted using the normalized spectral density.
  • Individual-level implementation: Bridge sampling estimates the individual-level EV model separately for each participant and compares the results with importance-sampling estimates.
  • Data and model: The EV model represents performance through three parameters corresponding to distinct psychological processes, including attention to losses relative to rewards.Values w < .5 characterize reward-seeking and w > .5 characterize loss-averse decision makers.

Data

The paper applies bridge sampling to individual-level and hierarchical EV models using data from 30 participants. The estimates closely agree with importance sampling and Savage-Dickey comparisons, while coefficient-of-variation results indicate low variance.

  • Data: The data comprise 30 healthy participants, each contributing T = 100 Iowa Gambling Task card selections.Three participants were excluded because of incomplete choice data.
  • Individual-level model: For the individual-level EV model, parameters are estimated separately for each participant, producing one marginal likelihood per participant.
  • Individual-level model: The individual-level procedure uses a multivariate normal proposal distribution fitted to probit-transformed posterior samples.The proposal's mean vector and covariance matrix are obtained by the method of moments.
  • Individual-level model: After convergence, the procedure yields each participant's marginal likelihood and coefficient of variation.
  • Individual-level results: The largest coefficient of variation is 2.07%, suggesting low variance in the bridge sampler.The measure refers to marginal likelihoods, not log marginal likelihoods.
  • Individual-level results: Bridge-sampling and importance-sampling log marginal likelihoods for the 30 participants correspond almost perfectly.
  • Hierarchical model: The hierarchical EV model draws participant parameters from separate group-level distributions, incorporating both between-participant differences and similarities.
  • Hierarchical results: Hierarchical bridge-sampling Bayes factors closely approximate Savage-Dickey Bayes factors of 1, with closely matching log marginal likelihoods.

Discussion

The discussion presents bridge sampling as an accurate, reliable, generic, and accessible approach for estimating marginal likelihoods, while emphasizing that method choice depends on the model and research context.

  • Bridge sampling estimates marginal likelihoods for both individual-level and hierarchical implementations of the Expectancy Valence model.
  • The bridge sampling estimator is superior to naive Monte Carlo, importance sampling, and generalized harmonic mean estimators for several reasons.It minimizes mean-squared error through the optimal bridge function.
  • Bridge sampling is more robust to proposal-tail behavior than importance sampling, facilitating applications to higher-dimensional and complex models.
  • The method is relatively straightforward to implement, with the tutorial's procedure requiring only minor code changes for other models.The unnormalized posterior and potentially the proposal function must be adapted.
  • The authors implemented bridge sampling in the bridgesampling R package to maximize accessibility.
  • Method selection should balance accuracy and implementation effort, considering model complexity, the number of models, researcher expertise, and available time.Bridge sampling is unnecessary when the marginal likelihood is analytically available, and alternatives may be preferable in some settings.
  • Savage-Dickey, importance sampling, product-space, reversible-jump, Bayesian information criterion, and path sampling may be more appropriate under specific goals or constraints.The alternatives differ in efficiency, implementation demands, assumptions, and scope.
  • The tutorial concludes that bridge sampling is reliable, easy to implement, and applicable to parameter estimation, model comparison, and Bayesian model averaging.

Appendix B Bridge Sampling Implementation: Avoiding Numerical Issues

The implementation rewrites the bridge-sampling calculation on a logarithmic scale to avoid numerical problems while preserving the marginal-likelihood estimate.

  • l* is chosen so terms in the sums remain manageable during computation.
  • The iterative scheme is run with respect to r̂ for numerical convenience.
  • Multiplying by exp(l*) yields the marginal-likelihood estimate.
  • Alternatively, adding l* to the logarithm of the result yields an estimate of the log marginal likelihood.

Appendix C Correcting for the Probit Transformation

The appendix explains how bridge-sampling expressions change under a probit transformation, including the transformed importance density and posterior samples.

  • The probit transformation changes the expression for the importance density through a change-of-variable factor.
  • The running example uses a normal distribution on ξ as the importance density.
  • ξ is defined as the probit transform of θ, ξ = Φ^-1(θ).
  • The specific importance density is expressed in ξ, whereas the general density gIS is specified in θ.
  • The generalized harmonic estimate can use either posterior samples or probit-transformed posterior samples.

Appendix D Details on the Application of Bridge Sampling to the Individual-Level EV Model

The individual-level EV application computes the unnormalized posterior and bridge-sampling terms for one participant, using log calculations to avoid numerical issues.

  • The appendix obtains the unnormalized posterior for the individual-level EV model.
  • The application focuses on one participant and drops the participant subscript thereafter.
  • The iterative scheme is run with respect to r̂ to avoid numerical issues.
  • Proposal-distribution samples are represented as κ̃_i = (ω̃_i, α̃_i, γ̃_i).
  • The implementation computes the logarithm of the unnormalized posterior rather than the unnormalized posterior directly.
  • Independent priors on w, a, and c determine part of the posterior calculation, while p denotes a uniform prior on [0, 1].

Appendix E Details on the Application of Bridge Sampling to the Hierarchical EV Model

The hierarchical EV implementation obtains the logarithm of the unnormalized posterior using proposal samples for both subject-level and group-level parameters.

  • The appendix explains how the logarithm of the unnormalized posterior is obtained for the hierarchical EV model.
  • κ̃_s,i represents the ith proposal sample for the individual-level parameters of subject s.
  • ζ̃_i represents the ith proposal sample for all group-level parameters.
Loading 1703.05984v2…