Source-linked AI summary
Parameter inference with estimated covariance matrices
Elena Sellentin, Alan F. Heavens
TL;DR
The paper addresses parameter inference when the covariance of Gaussian-distributed data is estimated rather than known, so covariance uncertainty must enter the likelihood. It marginalises over the true covariance conditioned on its estimate, obtaining a modified multivariate t-distribution that is computationally comparable to the Hartlap-scaled Gaussian and improves inference.
Problem
Estimating a covariance matrix from limited simulations makes the precision matrix noisy, while the resulting likelihood is not actually Gaussian.
Method
The method marginalises the true covariance over its inverse-Wishart distribution conditioned on the estimated covariance, using P(Xo|µ,S,N).
Results
The resulting likelihood is a modified multivariate t-distribution with the same computational cost as the Hartlap-scaled Gaussian.
Takeaways & Limitations
Using the correct sampling distribution accounts for covariance uncertainty in parameter inference instead of relying on a debiased inverse covariance within a Gaussian likelihood.
Takeaways & Limitations
Using a different prior changes the degrees of freedom, although increasing the number of simulations can lessen the prior's influence.
Abstract
from arXiv · showhide
When inferring parameters from a Gaussian-distributed data set by computing a likelihood, a covariance matrix is needed that describes the data errors and their correlations. If the covariance matrix is not known a priori, it may be estimated and thereby becomes a random object with some intrinsic uncertainty itself. We show how to infer parameters in the presence of such an estimated covariance matrix, by marginalising over the true covariance matrix, conditioned on its estimated value. This leads to a likelihood function that is no longer Gaussian, but rather an adapted version of a multivariate t-distribution, which has the same numerical complexity as the multivariate Gaussian. As expected, marginalisation over the true covariance matrix improves inference when compared with Hartlap et al.'s method, which uses an unbiased estimate of the inverse covariance matrix but still assumes that the likelihood is Gaussian.
1 INTRODUCTION
The paper addresses parameter inference when a Gaussian likelihood uses a covariance matrix estimated from limited simulations. It marginalises over covariance uncertainty, yielding a modified multivariate t-distribution rather than incorrectly retaining a Gaussian likelihood.
- Gaussian data likelihoods are characterized by a mean vector µ and covariance matrix Σ.
- Estimating covariance from limited simulations introduces statistical noise in the precision matrix that propagates into parameter errors.
- Treating the likelihood as Gaussian after replacing the true precision matrix is fundamentally incorrect because the resulting likelihood is not Gaussian.
- Marginalising over the unknown covariance conditioned on its simulated estimate produces a modified multivariate t-distribution for inference.
2 REPLACING A TRUE COVARIANCE MATRIX BY AN ESTIMATOR
When the true covariance matrix is unavailable, the paper models its estimator as random and marginalises over covariance uncertainty. This replaces the Gaussian likelihood with an adapted multivariate t-distribution conditioned on the estimated covariance.
- Covariance estimation: A single observed data vector is combined with simulated vectors to estimate the covariance matrix S when repeated observations are unavailable.The resulting target is P(Xo|µ, S, N), conditioned on the estimate and the number of simulations.
- Covariance estimation: The sample covariance from Gaussian simulations follows a Wishart distribution, enabling inference for the unknown true covariance conditioned on S.Bayesian inversion with an independence-Jeffreys prior yields an inverse Wishart distribution for Σ.
- Modified t-distribution: Increasing the number of covariance estimates makes the inverse Wishart distribution more sharply peaked, reflecting improved covariance estimation.The method assumes a Gaussian sampling distribution, with asymptotic applicability to non-Gaussian sampling through the central limit theorem.
- Marginal likelihood: Marginalising the Gaussian likelihood over the unknown covariance produces an analytical likelihood conditioned on the estimated covariance S.The derivation integrates over the inverse Wishart distribution associated with the conditional covariance uncertainty.
- Modified t-distribution: The resulting likelihood is a cosmologist’s multivariate t-distribution, distinct from the standard t-distribution because one observed vector determines the peak while N simulations estimate covariance.It requires N > p, and differences from a Gaussian become important when the simulation count is comparable to the data-vector dimension.
3 ATTEMPTING TO DEBIAS A GAUSSIAN LIKELIHOOD
The Hartlap approach rescales an estimated inverse covariance while retaining a Gaussian likelihood, but finite-simulation uncertainty makes this sampling model incorrect. The paper shows that this uncertainty produces random contour scatter and that the modified t-distribution better represents the resulting likelihood.
- Hartlap et al. retain a Gaussian likelihood while replacing the true inverse covariance with a scaled inverse sample covariance matrix.
- 68% (90%) of estimated covariances place the deduced 1σ contour within the dark-blue (dashed-blue) boundaries.
- Debiasing the inverse covariance does not correct posterior bias or errors when the likelihood’s sampling distribution remains wrong.
- The t-distribution has a sharper central region and broader extreme wings than Gaussian alternatives, allowing more scatter away from the peak.
- Scaling a single estimated inverse covariance widens the Gaussian likelihood in a random way, propagating contour scatter into parameter inference.
- The t-distribution follows the correct cumulative-probability diagonal, whereas the Hartlap-scaled Gaussian is too broad.
4 COMPARISON OF THE DISTRIBUTIONS
The true χ2, estimated-covariance T 2, and Hartlap-scaled H2 follow different distributions, so confidence assessment changes when covariance uncertainty is included. The T 2 distribution broadens relative to χ2, especially with fewer simulations, while H2 understates scatter and parameter confidence intervals require adjustment.
- Hartlap comparison: ⟨H2⟩=⟨χ2⟩, so Hartlap scaling debiases the expectation value but underestimates statistical scatter.The Hartlap-scaled distribution is more sharply peaked than χ2.
- Distribution definitions: The true χ2 assumes an exactly known covariance, whereas T 2 also includes the random estimated covariance following a Wishart distribution.T 2 approaches χ2 only as N becomes very large.
- Dependence on simulation count: As N approaches infinity, the T 2 distribution tends toward the χ2 p-distribution; as N approaches p, their difference increases.The cumulative T 2 distribution rises more slowly than the χ2 distribution.
- Confidence assessment: T 2 must exceed χ2 to achieve the same confidence that the mean lies within the confidence contours.This produces larger Bayesian confidence intervals in parameter space.
- MCMC reweighting: The modified t-distribution can correct existing Gaussian MCMC chains through reweighting, without requiring substantially greater numerical effort.If Hartlap scaling was applied, it must first be removed.
- Posterior shape: After marginalisation over parameters, the t-distribution and Gaussian-based posteriors need not peak at the same location.Their maxima coincide only in the full parameter space before marginalisation.
5 CONCLUSIONS
Finite-simulation covariance estimates make the covariance, and therefore the likelihood, uncertain rather than exactly Gaussian. The paper addresses this by marginalising over the true covariance, yielding a modified multivariate t-distribution that better accounts for inference uncertainty than Hartlap-scaled Gaussian likelihoods.
- Problem: A covariance estimated from N simulations is random, so treating the resulting likelihood as Gaussian does not fully account for covariance uncertainty.The inverse covariance estimator is not unbiased without correction, and the sampling distribution itself affects parameter posteriors.
- Existing approach: Hartlap scaling debiases the inverse-covariance expectation but leaves the Gaussian likelihood unable to capture the full statistical scatter.The resulting posteriors can be too broad near their centres yet insufficiently broad in the extremes.
- Method: The paper marginalises over the inverse-Wishart distribution of the true covariance conditioned on the estimated covariance S.This produces the modified multivariate t-distribution P(Xo|µ, S, N).
- Implication: For finite-simulation covariance matrices, parameter inference should evaluate the modified t-distribution rather than a Gaussian likelihood.Both likelihoods have constant numerical complexity because they evaluate (Xo −µ)T S−1(Xo −µ).