Source-linked AI summary
Informed Bayesian T-Tests
Quentin F. Gronau, Alexander Ly, Eric-Jan Wagenmakers
TL;DR
Bayesian t-tests have been less prominent than frequentist t-tests, despite proposed objective and subjective alternatives and the importance of prior choice. The paper introduces a flexible t-prior for standardized effect size, computes the Bayes factor with a single numerical integral, and proposes measures for departures from predictive matching and information consistency. It illustrates informed priors using expert elicitation, with one analysis yielding a Bayes factor of 11.5 favoring the null over the one-sided alternative.
Problem
Bayesian t-tests require prior specifications, but existing objective Bayes factors are limited in incorporating expert knowledge and some earlier formulations have documented shortcomings.
Method
The paper uses a flexible t-prior for standardized effect size and evaluates the resulting Bayes factor through a single numerical integral.
Results
A one-sided informed-prior analysis produced BF0+(d; 0.350, 0.102, 3) = 11.5, indicating that the data were about twelve times more likely under the null than the one-sided alternative.
Takeaways & Limitations
The specification contains previous objective and subjective t-test Bayes factors as special cases and supports informed prior distributions based on expert elicitation.
Takeaways & Limitations
The specified prior's goal is constrained by an information-consistency departure of two samples.
Abstract
from arXiv · showhide
Across the empirical sciences, few statistical procedures rival the popularity of the frequentist t-test. In contrast, the Bayesian versions of the t-test have languished in obscurity. In recent years, however, the theoretical and practical advantages of the Bayesian t-test have become increasingly apparent and various Bayesian t-tests have been proposed, both objective ones (based on general desiderata) and subjective ones (based on expert knowledge). Here we propose a flexible t-prior for standardized effect size that allows computation of the Bayes factor by evaluating a single numerical integral. This specification contains previous objective and subjective t-test Bayes factors as special cases. Furthermore, we propose two measures for informed prior distributions that quantify the departure from the objective Bayes factor desiderata of predictive matching and information consistency. We illustrate the use of informed prior distributions based on an expert prior elicitation effort.
1 INTRODUCTION
The t-test is a central inferential tool for comparing two means, while Bayesian two-sample t-tests depend critically on how priors are specified. The paper presents a flexible informed-prior framework that generalizes earlier Bayesian t-tests and quantifies departures from key objective desiderata.
- 1 INTRODUCTION: The t-test assesses whether two means differ and remains an inferential workhorse across the empirical sciences.In eight major psychology journals, 26% of 258,105 reported p-values tested a t statistic.
- 1 INTRODUCTION: Bayes factors compare hypotheses through ratios of prior-weighted marginal likelihoods, making prior specification central to Bayesian hypothesis testing.For the two-sample t-test, the null has nuisance parameters (µ, σ), whereas the alternative additionally includes standardized effect size δ.
- 1 INTRODUCTION: Earlier two-sample Bayesian t-tests use a grand-mean/effect-size parameterization, a right Haar prior for nuisance parameters, and a test-relevant prior on effect size.The Gönen et al. formulation uses a normal g-prior on δ and is easily calculated, while later approaches use Cauchy or hyper-prior specifications.
- 1 INTRODUCTION: Earlier Bayes factors have documented limitations: some are information inconsistent, and objective priors centered at zero cannot incorporate available expert knowledge.Information inconsistency means the Bayes factor favoring the alternative does not diverge as the observed t-value increases indefinitely.
- 1 INTRODUCTION: The paper generalizes the Rouder et al. Bayes factor with a flexible t-prior for standardized effect size that accommodates substantive domain knowledge.It also proposes measures of departure from predictive matching and information consistency and illustrates informed-prior use through expert prior elicitation.
2 THEORY
The paper extends Bayesian t-tests with a flexible t-prior for standardized effect size, yielding a Bayes factor computable through one numerical integral. It also formalizes departures from predictive matching and information consistency while allowing expert knowledge in prior specification.
- The proposed Bayes factor evaluates the numerator by numerical integration while retaining the framework of G¨onen et al. (2005).
- The flexible t-prior for standardized effect size uses location µδ, scale γ, and degrees-of-freedom κ hyperparameters.
- Researchers can incorporate expert knowledge by shifting the prior away from zero, whereas predictive matching requires centering it at µδ = 0.
- The flexible prior contains the Cauchy prior of Rouder et al. (2009) as the special case κ = 1 and µδ = 0.
- The proposed departure measures quantify differences between informed and objective priors with respect to predictive matching and information consistency.
- For uninformative data, the Bayes factor approaches 1, with BF10(dν<min) ≈0.98 when ¯y1 < ¯y2 and ≈1.02 when ¯y1 > ¯y2.
f(dinfo,ν | δ)π(δ)dδ diverges, it suffices to take
Information consistency requires the alternative to receive unbounded support under overwhelmingly informative data. The analysis shows how the t-prior’s degrees of freedom determine its departure from this criterion and the observations needed to overcome the prior.
- Information consistency requires infinite support for the alternative when the data are overwhelmingly informative.
- For a t-prior with κ degrees of freedom, the departure from information consistency is κ −1 because the t-distribution has κ −1 moments.
- A t-prior with κ = 3 has two moments and misses information consistency by two samples.
- The departure measures support recommendations for selecting degrees of freedom while incorporating expert knowledge through informed priors.
- Researchers seeking to retain information consistency should choose κ ∈(0, 1], including a Cauchy prior when κ = 1.
- The practical example demonstrates prior elicitation based on expert knowledge.
3 PRACTICE
The facial feedback replication was reanalyzed with an expert-informed prior for the standardized effect size. The informed one-sided Bayes factor still favored the null, despite the prior encoding a positive expected effect.
- The facial feedback hypothesis proposes that facial expressions can influence affective responses even without corresponding emotional experiences.
- A preregistered 17-lab replication used an independently vetted protocol to reassess the original pen-holding finding.
- The classical random-effects meta-analysis estimated a smile–pout difference of 0.03 [95% CI: −0.11, 0.16].
- Bayes factors favored the null hypothesis in all 17 studies, exceeding 3 in 13 studies, and the authors judged the results inconsistent with the original result.
- 3.1 Prior elicitation: Expert elicitation produced a positive t prior for δ with location 0.350, scale 0.102, and 3 degrees of freedom, while departing from predictive matching by ±0.0198 and information consistency by two samples.
- 3.2 Reanalysis of the Oosterwijk replication study: The informed one-sided Bayes factor was BF0+(d; 0.350, 0.102, 3) = 11.5, indicating the data were about twelve times more likely under the null than under the one-sided alternative.
2. The one-sided default Bayes factor equals
The default one-sided analysis applies numerical integration to obtain the posterior mass needed for the Bayes factor. Its reported result is BF0+(d; 2, 1) = 8.7.
- BF0+(d; 2, 1) = 8.7 for the default one-sided analysis.
- Numerical integration of the marginal posterior distribution provides the posterior mass needed for the Bayes factor correction.
- The informed reanalysis displays the elicited prior and associated posterior distributions, including a 95% credible interval and posterior median.
2. The solid line corresponds to the associated posterior
The informed and default Bayes factors both supported the null hypothesis, but their unrestricted posterior distributions differed noticeably.
- Both the informed and default Bayes factors yielded evidence for the null hypothesis.
- The unrestricted posterior distributions differed noticeably between the informed and default analyses.
4 CONCLUDING COMMENTS
The article presents an informed two-sample Bayesian t-test using a flexible t-prior that incorporates expert knowledge while retaining objective and subjective approaches as special cases. It also proposes measures for assessing departures from predictive matching and information consistency, and illustrates the approach in practice.
- Contribution: The proposed informed Bayesian t-test generalizes Rouder et al.’s Bayes factor so practitioners can incorporate expert knowledge through a flexible t-prior.The framework extends earlier two-sample Bayesian t-tests and permits informed prior distributions for standardized effect size.
- Contribution: The flexible t-prior contains Rouder et al.’s objective default prior as a special case and can include a location parameter specified from expert knowledge.The formulation also accommodates any proper prior on standardized effect size through the presented Bayes factor expressions.
- Contribution: The same formula can compute both subjective and objective Bayesian t-tests, and the proposed specification includes the subjective prior of Gönen et al. as a limiting case.The authors implemented the Bayesian t-test setup in the open-source statistical program JASP.
- Theoretical analysis: The proposed departure measures assess information consistency and predictive matching for informed prior distributions.These measures quantify how an informed t-test departs from its objective counterpart.
- Extensions: The t-prior is not the only possible effect-size prior: Bayes factors can also be obtained for any scale-mixture of normals by integrating the corresponding expression over a prior on g.The article illustrates the informed Bayes factor with an example and notes that alternative prior choices are possible.
- Interpretation: An informed Bayes factor quantifies relative evidence for competing hypotheses, but a complete Bayesian analysis also requires prior plausibilities for those hypotheses.Once prior plausibilities are specified, the Bayes factor can be multiplied by prior odds to obtain posterior odds.