Source-linked AI summary
OmniFold: A Method to Simultaneously Unfold All Observables
Anders Andreassen, Patrick T. Komiske, Eric M. Metodiev, Benjamin Nachman, Jesse Thaler
TL;DR
OmniFold addresses unfolding by estimating a reweighted particle-level distribution from measured detector-level data. It is formulated as an unbinned, machine-learning-based maximum-likelihood procedure whose finite iteration count provides practical regularization.
Problem
Unfolding seeks a truth-level reweighting of a synthetic particle-level distribution that maximizes the likelihood of the observed measured data.
Method
OmniFold iteratively updates the truth-level reweighting using a generalized expectation-maximization procedure, with particle-level information as latent variables.
Results
OmniFold provides an unbinned, machine-learning-based estimate of the maximum-likelihood true particle-level distribution given observed detector-level data.
Takeaways & Limitations
Finite iteration counts regularize the unfolding procedure when statistical fluctuations make the likelihood maximum undesirable in practice.
Takeaways & Limitations
The formulation assumes that the detector response is accurately modeled in the synthetic dataset.
Abstract
from arXiv · showhide
Collider data must be corrected for detector effects ("unfolded") to be compared with many theoretical calculations and measurements from other experiments. Unfolding is traditionally done for individual, binned observables without including all information relevant for characterizing the detector response. We introduce OmniFold, an unfolding method that iteratively reweights a simulated dataset, using machine learning to capitalize on all available information. Our approach is unbinned, works for arbitrarily high-dimensional data, and naturally incorporates information from the full phase space. We illustrate this technique on a realistic jet substructure example from the Large Hadron Collider and compare it to standard binned unfolding methods. This new paradigm enables the simultaneous measurement of all observables, including those not yet invented at the time of the analysis.
Appendix: OmniFold as a Maximum Likelihood Estimate
The appendix formulates unfolding as maximum-likelihood estimation of a truth-level reweighting under an assumed detector response. OmniFold’s update follows from generalized expectation-maximization, increasing likelihood while finite iteration regularizes the estimator.
- Likelihood formulation: The likelihood seeks a truth-level reweighting of the synthetic distribution that maximizes the probability of observing measured detector-level data.The formulation is unbinned and treats particle-level information as the latent variable.
- Likelihood formulation: The detector response is assumed to be accurately modeled by the synthetic dataset, with normalization enforced through a Lagrange multiplier.The response p(m|t) is identified with the simulated conditional distribution pSim.|Gen.(m|t).
- Maximum-likelihood solution: The stationary condition is a maximum because the likelihood functional is concave, so its second variation is non-positive.The derivation obtains the normalization multiplier and then establishes the maximum property from concavity.
- OmniFold update: Replacing the optimal reweighting on the left with νn and the previous iterate on the right with νn−1 yields the OmniFold update rule, whose discrete form is IBU.The update is connected to the stationary condition for the maximum-likelihood solution.
- Convergence and regularization: The update increases expected complete-data likelihood through generalized expectation-maximization, and therefore increases the log likelihood at each step.Particle-level information serves as the unobserved latent variable in the generalized EM construction.
- Convergence and regularization: Finite iteration regularizes the unfolding by reducing estimator variance at the cost of increased prior-dependent bias; with an invertible response, the procedure converges to the true solution.If the response is not invertible, null directions can make the maximum non-unique, while concavity still ensures attainment of a global maximum.