Source-linked AI summary
The RooStats Project
Lorenzo Moneta, Kevin Belasco, Kyle Cranmer, Sven Kreiss, Alfio Lazzaro, Danilo Piparo, Gregory Schott, Wouter Verkerke, Matthias Wolf
TL;DR
LHC analyses require versatile statistical tools supporting different inference methods, nuisance parameters, and combinations. RooStats addresses this need with reusable statistical interfaces built on ROOT and RooFit for simple and complex models. The project is beginning use by ATLAS and CMS and is planned for future analysis combinations.
Problem
LHC data analysis needs software that supports multiple statistical methods, systematic uncertainties, combinations, and both simple and complex models.
Method
RooStats implements major statistical tools through common interfaces built on ROOT and RooFit, including frequentist, Bayesian, likelihood-based, and hybrid methods.
Results
RooStats is starting to be used by ATLAS and CMS for data-analysis results and is planned for future analysis combinations.
Takeaways & Limitations
The same input models can be reused across statistical methods and applied to analyses with multiple parameters of interest and nuisance parameters.
Abstract
from arXiv · showhide
RooStats is a project to create advanced statistical tools required for the analysis of LHC data, with emphasis on discoveries, confidence intervals, and combined measurements. The idea is to provide the major statistical techniques as a set of C++ classes with coherent interfaces, so that can be used on arbitrary model and datasets in a common way. The classes are built on top of the RooFit package, which provides functionality for easily creating probability models, for analysis combinations and for digital publications of the results. We will present in detail the design and the implementation of the different statistical methods of RooStats. We will describe the various classes for interval estimation and for hypothesis test depending on different statistical techniques such as those based on the likelihood function, or on frequentists or bayesian statistics. These methods can be applied in complex problems, including cases with multiple parameters of interest and various nuisance parameters.
A. Lazzaro †
The paper identifies the authors’ institutional affiliations and the conference context under which the work was presented.
- The listed affiliations include CERN, KIT, and NIKHEF.
- The work was presented at ACAT2010 in Jaipur, India, from February 22–27, 2010.
- The publication is released under a Creative Commons Attribution-NonCommercial-ShareAlike licence.
1. Introduction
RooStats addresses the LHC’s need for versatile statistical software supporting multiple inference approaches, analysis combinations, and models ranging from simple counts to complex distributions.
- Motivation: LHC data analysis requires multiple statistical approaches, systematic-uncertainty treatments, and combinations across search channels and experiments.
- Motivation: RooStats is designed to apply different statistical methods to the same input model across simple counting and complex distribution-based analyses.
- Statistical approaches: Frequentist, Bayesian, and likelihood-based methods differ in their probability interpretations, likelihood principles, coverage properties, and nuisance-parameter treatments.
- Project foundation: RooStats began in 2008 as an ATLAS–CMS collaboration built on ROOT and RooFit.
- Software roles: RooFit supports parameter estimation and model description, while RooStats provides confidence-interval and hypothesis-testing tools.
2. RooStats Interfaces
RooStats maps statistical questions to reusable C++ interfaces for confidence intervals and hypothesis tests, with result objects that expose standard inferential quantities.
- Design philosophy: RooStats maps fundamental statistical concepts and questions to corresponding C++ classes and interfaces.
- Confidence intervals: IntervalCalculator computes confidence intervals represented through the ConfInterval interface, including multidimensional or disconnected regions.
- Interface overview: Figure 1 diagrams the testing and interval interfaces together with the classes that return their results.
- Hypothesis testing: HypoTestCalculator accepts a model, dataset, and null and alternate parameter lists to perform hypothesis tests.
- Hypothesis testing: HypoTestResult exposes null and alternate p-values, confidence levels, and significance in one-sided Gaussian standard deviations.
3. RooStats Calculators
RooStats provides common interfaces and calculator classes for confidence intervals and hypothesis tests across likelihood-based, Bayesian, frequentist, and hybrid methods.
- RooStats implements common IntervalCalculator and HypoTestCalculator interfaces for interval estimation and hypothesis testing.
- ProfileLikelihood Calculator: ProfileLikelihoodCalculator uses likelihood ratios to estimate confidence intervals and test hypotheses for parameter values, including multidimensional contours.It implements both calculator interfaces and returns LikelihoodInterval or HypoTestResult objects.
- Bayesian Calculators: Bayesian calculators construct posterior distributions from models, data, and priors, then derive credible intervals by analytical, numerical, or MCMC integration.BayesianCalculator handles one parameter of interest, while MCMCCalculator uses Metropolis-Hastings chains and can represent posteriors with histograms or kernel estimators.
- Neyman Construction: NeymanConstruction builds frequentist intervals by repeated-experiment coverage, requiring an ordering rule and allowing arbitrary scanned interval shapes.It returns a PointSetInterval based on independently evaluated parameter-space points.
- Hybrid Calculator: The HybridCalculator combines frequentist toy Monte Carlo with Bayesian marginalization of nuisance parameters for hypothesis testing.It compares test-statistic distributions generated under background-only and signal-plus-background hypotheses with the observed value to compute CLb and CLsb.
4. Combination of Results
RooStats addresses the logistical challenge of combining heterogeneous experimental results through workspaces that preserve models and data in a unified, shareable format.
- Combining experiments can improve measurement sensitivity or hypothesis-test power, but requires heterogeneous low-level inputs to form one statistical test.
- A RooWorkspace stores data and arbitrarily complicated models in a ROOT file, preserving the ingredients needed for combinations and enabling digital publication.RooFit’s RooWorkspace also supports model creation through a string-based workspace factory.
5. RooStats Utilities
RooStats provides utilities for p-value and Z-value calculations, signal-background separation, and systematic corrections to probability models, alongside tools for building models from text files.
- RooStats utilities calculate p-values and Z-values for number-counting experiments.Z-values represent significance in one-sided Gaussian standard deviations.
- RooStats::SPlot disentangles signal from background using an extended maximum likelihood fit and produces control-variable distributions to assess fit quality and validity.It supports arbitrary RooFit models and complements ROOT's TSPlot3 class.
- BernsteinCorrection augments a nominal probability density function with a positive-definite polynomial correction term for systematic variations.The Bernstein basis is used because its polynomials are positive definite.
- HLFactory builds RooStats input models from text files through a user-friendly wrapper around the RooFit workspace class.The paper notes that additional high-level factory tools were planned for future versions.
6. Conclusions
RooStats supplies reusable statistical tools through common interfaces, supporting both simple and complex LHC analyses. Distributed with ROOT, it was beginning to be used by ATLAS and CMS for data-analysis results and planned combinations.
- RooStats provides major statistical tools through common interfaces that reuse the same input model across simple and complex analyses.Supported cases include number counting, multiple parameters of interest, and various nuisance parameters.
- RooStats is distributed with ROOT, with tutorials and calculator-specific macros that produce interval limits, significance values, and statistical plots.The examples include confidence-level contours and test-statistic distributions.
- RooStats was starting to be used by ATLAS and CMS for data-analysis results, with future use planned for analysis combinations.