Source-linked AI summary
Incorporating Nuisance Parameters in Likelihoods for Multisource Spectra
J. S. Conway
TL;DR
The paper addresses how to incorporate systematic uncertainties into multisource spectral likelihoods without the computational expense of pseudoexperiments or marginalization. It develops profile-likelihood methods for multiplicative, morphing, and predicted-spectrum statistical uncertainties, and argues for their use in measurements, searches, and discoveries. In complex fits, profiling produced posterior densities nearly identical to marginalization while requiring orders of magnitude less computation.
Problem
Systematic uncertainties must be incorporated into multisource spectral likelihoods, but pseudoexperiments and Bayesian marginalization are computationally very expensive.
Method
The paper represents three types of systematic uncertainties with nuisance parameters in a profile likelihood, including multiplicative factors, morphing parameters, and predicted-spectrum statistical uncertainties.
Results
In complex spectrum fits, profile-based and marginalized posterior densities were nearly identical, while marginalization required orders of magnitude more compute time.
Takeaways & Limitations
The authors propose profile likelihood as a unified approach to exclusion bounds, discoveries, and measurements across particle-physics analyses.
Takeaways & Limitations
The morphing approximation may not reliably represent efficiency changes, and accurate behavior may require additional shifted spectra and higher-order interpolation.
Abstract
from arXiv · showhide
We describe here the general mathematical approach to constructing likelihoods for fitting observed spectra in one or more dimensions with multiple sources, including the effects of systematic uncertainties represented as nuisance parameters, when the likelihood is to be maximized with respect to these parameters. We consider three types of nuisance parameters: simple multiplicative factors, source spectra "morphing" parameters, and parameters representing statistical uncertainties in the predicted source spectra.
1 Overview
The paper develops profile-likelihood fits for multisource spectra that incorporate systematic uncertainties through nuisance parameters, offering a computationally cheaper alternative to repeated pseudoexperiments or Bayesian marginalization.
- Binned Poisson likelihoods are used for measurements, searches, discoveries, and exclusion bounds.
- Systematic uncertainties must be incorporated into the likelihood, but pseudoexperiments and Bayesian marginalization are computationally very expensive.
- Profile likelihoods maximize over nuisance parameters while interpreting intervals for parameters of interest.
- The paper defines three main nuisance-parameter types for systematic uncertainties on source distributions.
- Profile likelihoods are argued to apply to searches and discoveries through pseudo-Bayesian posterior or likelihood-ratio interpretations.
2 Core of the Poisson Likelihood
The core likelihood models event counts in arbitrarily shaped multidimensional bins as independent Poisson variables whose expected counts combine contributions from multiple sources.
- Observed events are partitioned into nbin bins of arbitrary shape in observable space.
- Each bin count ni is modeled as Poisson-distributed with expected count µi.
- Expected bin counts combine integrated luminosity, source cross sections, and source- and bin-specific efficiencies.
- The full-spectrum likelihood is the product of the Poisson probabilities across bins.
- Without systematic uncertainties, minimizing −ln L yields parameter estimates and confidence intervals from the standard error ellipsoid.
3 Multiplicative Uncertainties
Multiplicative systematic uncertainties are introduced as nuisance parameters constrained by auxiliary measurements or probability densities and profiled together with the physics parameters.
- A 2% integrated-luminosity uncertainty is represented by replacing the measured luminosity with a nuisance parameter.
- A Gaussian constraint G(L|L̃,σL) anchors the fitted luminosity L to its measured value while L determines the expected bin counts.
- The Gaussian constraint contributes a penalty to the negative log likelihood and can be replaced by another probability density.
- The same construction represents multiplicative uncertainties in cross sections, overall efficiencies, or selected sources.
- Multiplicative nuisance parameters must remain positive; broad Gaussian constraints may require truncation or a log-normal alternative.
4 Shape Uncertainties and Morphing
Shape uncertainties can distort both the spectrum and its normalization, so the paper models them by morphing source efficiencies with nuisance parameters. Its vertical morphing scheme uses quadratic interpolation within |f| < 1 and linear extrapolation beyond it, while noting accuracy and computational trade-offs.
- Energy-scale uncertainties can change both spectral shape and the overall efficiency normalization, especially when event-selection thresholds are present.
- Vertical morphing represents efficiency changes using nominal and shifted spectra controlled by a morphing parameter f with nominal value zero.The shifted spectra are obtained by recalculating efficiencies after varying a simulation parameter, such as the energy scale.
- The initial linear treatment approximates shifted-efficiency differences as a first-order Taylor expansion but may not reliably reproduce the true spectrum.At f = ±1, that expression does not yield the measured shifted efficiencies exactly.
- The proposed interpolation is quadratic for |f| < 1 and linear outside that range, matching the measured spectrum exactly at f = ±1.Values inside the sampled range are obtained by Lagrange interpolation.
- More shifted spectra and higher-order interpolation can improve accuracy, but they increase computation and bookkeeping; multidimensional horizontal morphing is not straightforward.The usefulness of additional sampling can be assessed from whether fitted morphing parameters move far beyond the sampled one-standard-deviation region.
- Multiple morphing parameters for different systematic effects can be incorporated by adding their deviations from nominal efficiency linearly.
5 Statistical Uncertainties in Efficiencies
The paper addresses statistical uncertainty in predicted source efficiencies by adding nuisance parameters constrained by assumed uncertainty distributions. Although the full Barlow–Beeston construction is tractable in principle, its implementation can create discontinuities that destabilize MINUIT, motivating combined uncertainty treatments.
- Statistical uncertainty in predicted event counts arises from finite Monte Carlo samples and from other data-driven estimation methods.
- The Barlow–Beeston method assigns a separate multiplicative nuisance parameter βji to each source and bin, nominally constrained around one.The constraint can use a Poisson distribution for Monte Carlo counts, while alternatives such as log normal avoid parameters approaching zero.
- Despite introducing many parameters, the full Barlow–Beeston profile likelihood is tractable because each bin’s βji values can be optimized independently of other bins.
- Directly solving βji inside MINUIT can cause discontinuous objective-function jumps, corrupt the finite-difference Hessian, and drive fitted parameters to wild values.The authors report finding no remedy short of rewriting MINUIT.
- The full Barlow–Beeston construction is not essential when only the total predicted-count uncertainty matters, because independent source uncertainties can be combined.Combination is particularly straightforward when the uncertainties are Gaussian.
- For a single bin, one statistical-uncertainty parameter can be solved exactly from a quadratic equation, with µ the expected count and σβ its relative uncertainty.Other constraint functions may instead produce transcendental equations.
6 Practical Considerations
The fitting procedure must handle pathological bins and keep the likelihood’s bin set fixed as nuisance parameters change. Practical safeguards include data-independent bin combination and tiny positive contributions.
- Bin populations: Sparse or highly uneven Monte Carlo populations can create zero-content bins, single-event bins affected by morphing, and data events without predicted rates.These pathologies are especially relevant in multi-bin and multidimensional spectra.
- Bin populations: Bins should be combined using an algorithm independent of the observed data distribution to ensure a minimum statistical threshold in every fitted bin.Generating sufficient Monte Carlo everywhere is the most straightforward option but may be impractical or impossible.
- Bins entering/leaving the likelihood: The likelihood’s included-bin set must be established a priori and remain fixed, including when source contributions vanish as parameters change.Simply excluding bins with no expected events during fitting is insufficient.
- Bins entering/leaving the likelihood: A tiny positive contribution can prevent bins from leaving the calculation; the implementation uses minimum values of 10^-10 pb for signal cross section and 10^-10 expected events per source.The authors report that final results do not depend on these minimum values in practice.
7 Pseudo-Bayesian Posterior Densities
The paper uses the profile likelihood to construct posterior densities for exclusion bounds, while contrasting this maximization with full Bayesian marginalization. In complex fits, both approaches produce nearly identical posterior densities, but marginalization is far more computationally expensive.
- Measurements: For physical-parameter measurements, profile likelihoods can use the usual ∆(ln L) method to derive confidence intervals in multidimensional parameter space.
- Exclusion bounds: For exclusion bounds on a parameter such as a new particle’s cross section σX, the profile likelihood Lmax is treated as a likelihood in Bayes’ Theorem.This yields a posterior density for the parameter of interest.
- Comparison with marginalization: The full Bayesian alternative marginalizes nuisance parameters by averaging over their prior-weighted values instead of maximizing over them.
- Comparison with marginalization: Nearly identical posterior densities were obtained from profiling and marginalization in complex spectrum fits, while marginalization required orders of magnitude more computation.The authors therefore employ profiling as a near-perfect practical representation of full Bayesian marginalization.
8 Conclusions
The paper presents a mathematical and numerical profile-likelihood approach for fitting multisource spectra with systematic uncertainties represented by nuisance parameters. It aims to unify exclusion bounds, discoveries, and measurements across particle-physics analyses.
- Conclusions: The method incorporates multiple types of systematic uncertainties into profile-likelihood fits of multisource spectra.The approach is presented for a wide range of particle-physics data analyses.