PaperlayerView exact revision on arXiv

Source-linked AI summary

Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology

Justin Alsing, Benjamin Wandelt, Stephen Feeney

arXiv:1801.01497v2astro-ph.CO

TL;DR

Cosmological inference often has forward-simulatable models but intractable likelihoods, while high-dimensional data make likelihood-free comparisons expensive. The paper combines massive asymptotically optimal compression with DELFI, recovering high-fidelity posteriors for full-scale analyses from ∼10^4 simulations. The approach also supports extracting Bayesian evidence from the learned joint density.

  • Problem

    Cosmological likelihood-free inference must compress high-dimensional data and reduce simulation demands because forward simulations are expensive and ABC acceptance rates become impractical for high-fidelity inference.

  • Method

    The paper compresses data to one score statistic per parameter and uses DELFI to learn the joint density of parameters and compressed statistics.

  • Results

    The combined method recovers high-fidelity posterior inferences for full-scale cosmological analyses from ∼10^4 forward simulations in a six-parameter task.

  • Takeaways & Limitations

    The framework makes likelihood-free inference feasible for large cosmological analyses while providing posterior densities and Bayesian evidence from the learned joint model.

  • Takeaways & Limitations

    Score compression can be lossy under an approximate likelihood, and Fisher information is not guaranteed to capture all information for highly non-Gaussian or multimodal likelihoods.

Abstract

from arXiv · show

Many statistical models in cosmology can be simulated forwards but have intractable likelihood functions. Likelihood-free inference methods allow us to perform Bayesian inference from these models using only forward simulations, free from any likelihood assumptions or approximations. Likelihood-free inference generically involves simulating mock data and comparing to the observed data; this comparison in data-space suffers from the curse of dimensionality and requires compression of the data to a small number of summary statistics to be tractable. In this paper we use massive asymptotically-optimal data compression to reduce the dimensionality of the data-space to just one number per parameter, providing a natural and optimal framework for summary statistic choice for likelihood-free inference. Secondly, we present the first cosmological application of Density Estimation Likelihood-Free Inference (\textsc{delfi}), which learns a parameterized model for joint distribution of data and parameters, yielding both the parameter posterior and the model evidence. This approach is conceptually simple, requires less tuning than traditional Approximate Bayesian Computation approaches to likelihood-free inference and can give high-fidelity posteriors from orders of magnitude fewer forward simulations. As an additional bonus, it enables parameter inference and Bayesian model comparison simultaneously. We demonstrate Density Estimation Likelihood-Free Inference with massive data compression on an analysis of the joint light-curve analysis supernova data, as a simple validation case study. We show that high-fidelity posterior inference is possible for full-scale cosmological data analyses with as few as $\sim 10^4$ simulations, with substantial scope for further improvement, demonstrating the scalability of likelihood-free inference to large and complex cosmological datasets.

1 INTRODUCTION

Cosmological models often support sophisticated forward simulations without tractable likelihoods, motivating likelihood-free inference. The paper addresses its scalability by combining massive data compression with DELFI, demonstrated on JLA supernova data.

  • Forward simulations can represent complex physical and measurement processes that are difficult to incorporate into tractable likelihood functions.
  • Approximate likelihoods may produce inaccurate posterior tails, affecting dataset-tension assessment, inference combination, and model comparison.
  • Likelihood-free inference faces two scalability hurdles: retaining parameter information while compressing data and obtaining high-fidelity posteriors with feasible simulation budgets.
  • The proposed two-step scheme first compresses full data into heuristic summaries, then reduces them to n numbers—one per parameter—while preserving Fisher information.
  • DELFI learns the joint density of parameters and compressed statistics, enabling posterior extraction and high-fidelity inference with orders of magnitude fewer simulations than PMC-ABC.

2 MASSIVE ASYMPTOTICALLY-OPTIMAL DATA COMPRESSION

The compression framework first forms informative cosmological summaries and then applies score-based compression to obtain one statistic per parameter. Its optimality and practical value depend on the quality of the approximate likelihood and initial summaries.

  • The first compression maps N data values to M carefully chosen summaries, such as power spectra, correlation functions, or supernova-derived quantities.
  • The second compression reduces M summaries to n numbers, where n equals the number of inferred parameters.
  • The score-function compression uses the gradient of an approximate log-likelihood at a fiducial parameter point and preserves Fisher information when the approximation is adequate.
  • The method can use forward simulations to estimate the mean, covariance, and parameter derivatives needed for Gaussian score compression.
  • Initial summary selection is generally lossy, while score compression is reliable when the fiducial point is near maximum likelihood and the approximate likelihood is suitable.
  • After compression, subsequent inference is genuinely likelihood-free, although the approximate likelihood used during compression affects how optimally information is retained.

Asymptotic optimality

Score compression is asymptotically optimal when it preserves Fisher information under a suitable likelihood approximation. Its guarantees are limited for non-Gaussian or multimodal likelihoods and by the quality of that approximation.

  • Score compression preserves Fisher information when the assumed likelihood approximates the true likelihood and the score is evaluated near maximum likelihood.
  • Using an approximate likelihood for compression can be lossy when the true likelihood is unknown or computationally intractable.
  • Even with the exact likelihood, Fisher-information preservation does not guarantee maximal information retention for highly non-Gaussian or multimodal parameter likelihoods.
  • In the asymptotic Gaussian limit, score compression exactly preserves expected parameter uncertainties, motivating the term asymptotically optimal.
  • Information-maximizing neural networks are an emerging alternative that learns compression schemes directly from forward simulations without specifying a likelihood.

3 LIKELIHOOD-FREE INFERENCE

Approximate Bayesian Computation compares simulated and observed data using a distance threshold, whereas DELFI learns their joint density to infer posteriors and evidence from all simulations.

  • Approximate Bayesian Computation (ABC): ABC draws parameters from the prior, simulates mock data, accepts samples within a distance threshold, and repeats until enough samples are obtained.
  • Approximate Bayesian Computation (ABC): As ϵ approaches zero, ABC approaches the true posterior, but nonzero thresholds produce broader approximate posteriors and vanishingly small acceptance rates.
  • Approximate Bayesian Computation (ABC): Sophisticated ABC methods still scale poorly with parameter count, often requiring impractically many forward simulations for high-fidelity cosmological inference.
  • Density Estimation Likelihood-Free Inference (DELFI): DELFI learns a parameterized joint density from prior draws and forward simulations, then extracts the posterior and Bayesian evidence from the fitted density.
  • Density Estimation Likelihood-Free Inference (DELFI): DELFI uses all available forward simulations rather than retaining only accepted samples, so it may require far fewer simulations for high-fidelity inference than ABC.

Gaussian mixture density estimation

The paper parameterizes the joint density with a Gaussian mixture model, whose components can represent flexible densities and provide analytically tractable evidence integrals.

  • The joint density is parameterized with a Gaussian mixture model consisting of component weights, means, and covariances.
  • Gaussian mixture models can represent probability densities arbitrarily accurately when sufficiently many components are used and can be fitted with expectation-maximization.
  • The evidence integrals are analytically tractable, making Bayesian evidence available directly from the fitted mixture model.

Data compression

The compression scheme reduces the density-estimation problem to parameters and compressed statistics, keeping its dimensionality fixed as the dataset grows.

  • The two-step scheme compresses data before DELFI, reducing the joint-density dimensionality from N + n to 2n.
  • Samples of parameters and compressed statistics are generated by drawing prior parameters, forward simulating mock data, and applying the subsequent compression step.
  • Evidence estimated from compressed statistics is not numerically equivalent to evidence for the original data, but supports model comparison when models use the same compression.
  • After compression is prescribed, inference remains a 2n-dimensional density-estimation task regardless of dataset size or the number of first-level summaries.

Implementation

The implementation fits Gaussian mixtures to simulated compressed samples while addressing prior boundaries and over-fitting; hard boundaries require a separate evidence estimate.

  • Forward simulations produce samples {θ, t}, which are fit with a Gaussian mixture model using expectation-maximization and hard-prior-boundary handling.
  • A minimum threshold on mixture-component covariance diagonals is used to mitigate over-fitting in large Gaussian mixtures.
  • With hard prior boundaries, the evidence integral is no longer analytically tractable, so evidence is estimated by fitting a mixture to compressed statistics alone.

Sophistications

The delfi implementation uses a flexible neural density model and variational inference to learn the joint distribution needed for likelihood-free posterior estimation.

  • Sophistications: delfi parameterizes the joint distribution with a mixture density network, representing a Gaussian mixture model through a neural network.The mixture density network is fit to simulated samples.
  • Sophistications: Stochastic variational inference is used to fit the mixture density network to the samples.
  • Sophistications: These extensions provide a sophisticated implementation of the vanilla delfi setup described earlier.

Scaling with number of parameters and dealing with nuisances

After compression, inference complexity depends on the number of parameters rather than the original dataset size. Nuisance parameters can be pre-marginalized through summaries designed to retain information about target parameters while remaining insensitive to nuisances, although general optimal compression with nuisances remains future work.

  • Scaling with number of parameters: The compressed inference task learns a 2n-dimensional density, irrespective of the original dataset size.Complexity increases with the number of parameters n.
  • Dealing with nuisances: Typical cosmological applications involve at most about 10 parameters of interest, but additional nuisance parameters may also require simultaneous inference and marginalization.
  • Dealing with nuisances: Nuisance-marginalized inference can retain a 2n-dimensional problem by choosing summaries that preserve information about target parameters while being insensitive to nuisances.The fitted density is P(θ, tθ), pre-marginalized over nuisance parameters.
  • Dealing with nuisances: A general treatment of optimal compression with nuisances and learning the nuisance-marginalized posterior density is left for future work.

4 VALIDATION CASE STUDY: JLA SUPERNOVA DATA ANALYSIS

The JLA validation analyzes 740 type Ia supernovae with a six-parameter wCDM light-curve model under Gaussian likelihood assumptions. DELFI accurately reconstructs the posterior and estimates the Bayesian evidence, while requiring far fewer simulations than PMC-ABC.

  • 4.1 JLA supernova data: The validation uses 740 type Ia supernovae from the Joint Lightcurve Analysis sample.The data combine multicolor lightcurves and spectroscopic or photometric observations to estimate apparent magnitudes, redshifts, color, and stretch.
  • 4.2 wCDM and lightcurve calibration model: The model assumes a flat wCDM universe with six parameters: Ωm, w0, α, β, MB, and δM.The model includes luminosity-distance dependence, stretch and color calibration, and host-galaxy-mass dependence of the absolute magnitude.
  • 4.3 Likelihood assumptions: The validation assumes Gaussian-distributed apparent magnitudes with a fixed covariance matrix incorporating redshift, color, and stretch uncertainties.The covariance construction follows the JLA analysis, with α and β fixed to maximum-likelihood values for this study.
  • 4.7 Results: Posterior means and standard deviations converge after approximately 8,000 simulations, with residual mean scatter below 0.05σ and standard-deviation differences of a few percent.The analysis uses 20,000 simulations for excellent convergence, while noting that adequate inference is possible with far fewer.
  • 4.7 Results: The 12-component Gaussian mixture model captures substantial non-Gaussianity, parameter degeneracies, and hard prior boundaries in the simulated joint density.The mixture likelihood converges as components increase, and covariance regularization protects against over-fitting.
  • 4.7 Results: DELFI using 20,000 simulations closely reproduces the exact posterior, whereas PMC-ABC requires approximately 10^6 simulations and yields a broader approximation.The PMC-ABC run required more than 2 × 10^6 forward simulations in total, while DELFI required approximately 10^4 for the comparison described.
  • 4.7 Results: DELFI estimates the evidence as ln P(to) = 7.38, consistent with the nested-sampling estimate ln P(to) = 7.4(1).The evidence is obtained by fitting a Gaussian mixture model to compressed-data samples and evaluating it at the observed data.

5 CONCLUSIONS

The paper combines massive optimal compression with DELFI to make likelihood-free inference scalable for large cosmological datasets. In a validation case, it recovers high-fidelity posterior inference using far fewer simulations than ABC methods.

  • Likelihood-free inference avoids explicit likelihood assumptions but requires compressing large datasets into tractable summary statistics.The paper motivates compression as necessary for scalable simulation-based inference.
  • The two-step compression scheme reduces summary statistics to n numbers—one per parameter—while preserving Fisher information under an approximate likelihood assumption.After compression, subsequent inference remains genuinely likelihood-free.
  • ABC methods can require orders of magnitude more forward simulations for approximate posterior inference because acceptance rates vanish as the tolerance approaches zero.The paper’s comparison motivates DELFI for computationally expensive cosmological simulations.
  • DELFI learns a parameterized joint data-parameter density from forward simulations, enabling posterior and Bayesian evidence extraction.The approach provides a model for simulation-based inference rather than relying on ABC acceptance thresholds.
  • ∼10^4 forward simulations yielded high-fidelity posterior inference for a full-scale six-parameter cosmological analysis.The result is reported with scope for further improvement.
  • Together, massive compression and DELFI provide a framework for scalable likelihood-free inference from large cosmological datasets, including when simulations are computationally expensive.The authors present this as a new paradigm for simulation-based Bayesian inference in cosmology.
Loading 1801.01497v2…