PaperlayerView exact revision on arXiv

Source-linked AI summary

CheckMATE: Confronting your Favourite New Physics Model with LHC Data

Manuel Drees, Herbi Dreiner, Jong Soo Kim, Daniel Schmeier, Jamie Tattersall

arXiv:1312.2591v2hep-phhep-ex

TL;DR

Interpreting LHC searches for arbitrary new-physics models requires detailed detector and analysis reproduction. CheckMATE automates this process from simulated event files and cross sections, then evaluates selected analyses to report exclusions and confidence information. Its close cutflow matching makes analysis runtime negligible relative to event generation and detector simulation, while its current coverage is concentrated on ATLAS supersymmetry searches.

  • Problem

    Accurately reinterpreting LHC data for models not previously analysed requires simulated events to pass through detector effects and experimental analyses.

  • Method

    CheckMATE accepts simulated event files and cross sections, processes them through modified detector simulation and selected analyses, and statistically evaluates the resulting signal regions.

  • Results

    121 seconds for event generation, 11 seconds for detector simulation, and about 1 second for the ATLAS trilepton analysis show that analysis runtime is negligible by comparison.

  • Takeaways & Limitations

    CheckMATE provides model exclusion decisions, confidence limits, and signal-region information through a workflow designed for straightforward use and extension.

Abstract

from arXiv · show

In the first three years of running, the LHC has delivered a wealth of new data that is now being analysed. With over 20 fb$^{-1}$ of integrated luminosity, both ATLAS and CMS have performed many searches for new physics that theorists are eager to test their model against. However, tuning the detector simulations, understanding the particular analysis details and interpreting the results can be a tedious task. CheckMATE (Check Models At Terascale Energies) is a program package which accepts simulated event files in many formats for any model. The program then determines whether the model is excluded or not at 95% C.L. by comparing to many recent experimental analyses. Furthermore the program can calculate confidence limits and provide detailed information about signal regions of interest. It is simple to use and the program structure allows for easy extensions to upcoming LHC results in the future. CheckMATE can be found at: http://checkmate.hepforge.org

Important Note

CheckMATE addresses the difficulty of accurately testing new-physics models against LHC searches by processing simulated events through detector and analysis implementations. It compares the resulting signal predictions with experimental limits and reports exclusions, confidence levels, and signal-region information.

  • LHC data have increasingly constrained BSM model parameter spaces, motivating accurate reinterpretation of experimental searches.
  • Complete models are predictive but can become difficult to reinterpret after modest modifications, while simplified-model limits rely on restrictive assumptions.
  • Different production modes, branching ratios, and decay topologies can make simplified-model mappings unsafe and produce weaker limits than a full analysis.
  • CheckMATE lets users provide event files and cross sections, select analyses, and determine whether a model is excluded at 95% confidence using the CLS method.
  • The package processes events through modified detector simulation and analysis modules before statistically evaluating signal yields in experimental signal regions.

II. USING CHECKMATE

Using CheckMATE requires event files, cross sections, uncertainties, and selected analyses, after which the program produces signal-region results and an Allowed or Excluded decision. Its analyses follow experimental cutflows closely, while implementation details support configurable and extensible workflows.

  • Users must provide a run name, selected analyses, event files, process information, cross sections, and cross-section uncertainties.
  • CheckMATE accepts HepMC, HepEvt, and LHE3 event formats, supports weighted events, and can receive options through parameter files or command-line input.
  • Parton-level LHE files are strongly discouraged because missing showers or differing jet clustering can alter acceptances and limits.
  • II. USING CHECKMATE: The program reports signal-region quantities and summarizes the final test with an Allowed or Excluded result.

III. INTERNAL DETAILS

CheckMATE’s internal workflow processes simulated event files through detector simulation, analysis, and statistical evaluation, while organizing intermediate and final outputs in structured folders.

  • Example setup: The example run uses CMSSM event files from MadGraph for gluino-pair and gluino-squark production, tested against an ATLAS zero-lepton search.The event files contain 10,000 events per production process, with cross sections and theoretical uncertainties supplied for normalization.
  • Results organization: The results directory contains analysis, Delphes, evaluation, progress, and result files, with progress.txt linking input files to cross sections, processes, timestamps, and output prefixes.The progress file also preserves information needed when adding data to an existing output directory.
  • Detector simulation: Each input event file is processed by Delphes, which produces one ROOT file per input file and records detector-simulation output in a log.The detector card is generated at run start, while ROOT trees remain available for inspection or later analysis.
  • Storage constraint: ROOT trees can consume substantial disk space, although tempMode can delete them after analysis; at least one complete ROOT file must still be stored.The approximate size is below 20 Mb for 1000 events, but it depends on the signal, generator, and analysis.
  • Analysis outputs: The analysis stage converts Delphes ROOT files into cutflow and signal files containing event counts, acceptances, weights, and normalized yields.Cutflow files track events surviving successive cuts; signal files report events in signal regions.
  • Statistical evaluation: The evaluation stage combines results across input files and processes, then compares each signal region with experimental limits and records the most sensitive regions.Outputs include event-number, r-limit, confidence-limit, and best-signal-region files.

B. Delphes Tunings

CheckMATE extends Delphes with experiment-informed detector-response functions and configurable object-quality checks to better reproduce ATLAS analyses.

  • Motivation: Delphes provides parametrized detector resolutions and efficiencies, but its simplified standard functions can deviate significantly from experimental measurements.The discrepancy depends on the model kinematics and the analysis considered.
  • Lepton response: The detector treatment models lepton momentum smearing and reconstruction or identification failures through energy- and position-dependent effects and efficiencies.Truth leptons appear as reconstructed objects with probability ϵ_l < 1.
  • Detector tuning: CheckMATE replaces most Delphes detector functions using publicly available experimental data.The improvements include detailed electron and muon reconstruction and identification efficiencies.
  • Jet and tau tagging: CheckMATE includes pT-dependent b-tagging and mis-tagging probabilities together with pT-dependent tau-tagging efficiencies for signal and background.Tau tagging distinguishes one-track and three-track candidates and infers charge from their tracks.
  • Object conditions: Multiple isolation and efficiency requirements are represented independently through flags, allowing analyses to apply different object definitions and checks.The current implementation supports at most 32 independent conditions per flag.
  • Detector-card construction: Before each run, CheckMATE merges shared analysis requirements into a detector card with the smallest necessary set of modules.This reduces computing time and detector-card complexity.

IV. ANALYSES

CheckMATE analyses reproduce experimental event selections, record cutflows and signal-region yields, and evaluate model limits using experimental reference data.

  • Analysis definition: Each analysis is defined by its experiment, jet-clustering settings, tagging efficiencies, isolation criteria, and other analysis-specific requirements.Users can choose any combination of available analyses, and each analysis may contain several signal regions.
  • Analysis code: The Root-based analysis code reads modified-Delphes ROOT files and records events passing cuts alongside final yields in each signal region.Counters and relevant kinematic variables are defined during initialization.
  • Experimental effects: When trigger-efficiency data are unavailable, CheckMATE tunes triggers to supplied cutflows and parameter scans to match experimental results as closely as possible.Experimental cleaning losses are represented by a flat efficiency factor of at most a few percent.
  • Cutflow reproduction: CheckMATE follows published cutflow tables exactly when they are available, using the same analysis code for validation and limit setting.This prioritizes reproducibility and user-checkable cutflows over analysis runtime optimization.
  • Reference data: Reference data from the corresponding experiments provide the backgrounds, uncertainties, observed events, and limits needed for statistical evaluation.The required information varies across analyses, but a minimum set of inputs is necessary.
  • Limit setting: CheckMATE evaluates limits using either a conservative r comparison with experimental 95% limits or a CLs likelihood calculation.The r method uses expected limits to select a signal region, while CLs requires background and observed-event information.

C. Currently Validated Analyses

Checkmate includes a broad set of validated ATLAS and CMS searches targeting missing transverse momentum with varied jet, lepton, and b-tag signatures. The implemented analyses cover hadronic, leptonic, monojet, multijet, and razor-like final states motivated by diverse BSM scenarios.

  • The implemented analyses broadly cover final states with missing transverse momentum combined with jets, leptons, and b-tags.
  • The monojet search selects a single hard jet recoiling against missing energy while vetoing additional jets and reconstructed electrons, muons, or hard photons.Its signal regions include models with extra dimensions, gravitino production, dark matter, and compressed spectra.
  • The all-hadronic stop search requires at least 6 jets, including 2 b-jets, significant missing energy, and a lepton veto.It targets stop-pair production but also applies to models producing top-quark pairs with missing energy.
  • The trilepton search uses six signal regions classified by Z-boson enrichment and progressively tighter missing-transverse-momentum requirements.The regions require three charged leptons, including at least one same-flavour opposite-sign pair.
  • The two-lepton search contains five signal regions, including Z-veto regions and regions sensitive to on-shell W bosons.These regions target electroweak chargino, neutralino, and slepton production.
  • CMS detector modelling uses default Delphes settings and ATLAS b-tagging efficiencies rather than tunings to the latest CMS data.

V. PERFORMANCE STUDIES

The performance studies validate Checkmate’s detector simulation and analyses against ATLAS results, while benchmarking computation and documenting its current scope. The program reproduces published cutflows and model exclusion curves with stated accuracy, but its analysis coverage and detector tunings remain incomplete.

  • ATLAS detector settings were improved and validated by comparing reconstructed Standard Model processes with experimental data.
  • Most evaluated cutflows have acceptance within 10% of the published signal-region values.Analyses outside this accuracy were checked for bugs and documented discrepancies.
  • Checkmate generally reproduces published model exclusion curves within the model’s 1σ theoretical uncertainty.The CMSSM parameter scan is shown as an example.
  • Computing Performance: 121 seconds generate 1000 events, 11 seconds perform detector simulation, and 1 second runs all five trilepton signal regions.Analysis runtime is negligible compared with event generation and detector simulation.
  • Outlook: The current implementation concentrates on ATLAS supersymmetry searches, although these analyses can also probe some non-supersymmetric BSM scenarios with invisible particles.Additional ATLAS and CMS exotic searches are planned.
  • Outlook: Checkmate already provides many validated analyses, and its structure lets users implement additional analyses for inclusion in the official library.
  • Outlook: Planned extensions include integrated event generation, an SLHA reader for supersymmetry, and FeynRules and SARAH integration for other models.

Appendix A: Getting Started

Getting started requires installing compatible Python and ROOT dependencies, compiling Checkmate, and testing the resulting binary. The setup instructions cover ROOT installation options, configuration, compilation, and a basic validation run.

  • Checkmate runs on Linux and MacOS X and provides an installation guide covering prerequisites and updated versions.
  • The required Python version is Python 2.7.X with X ≥3, and Checkmate does not work with Python 3.
  • ROOT must provide Minuit2, RooFit, and Python support, verified with three root-config commands.Missing packages require recompiling ROOT or installing a suitable local version.
  • Users without ROOT should build it from source rather than using precompiled binaries because the authors encountered internal linking problems with the binary version.
  • Running ./configure followed by make creates the Checkmate binary in bin/.The configuration step checks the ROOT setup and Python interpreter.
  • The installation is tested with ./CheckMATE testparam.dat, after which the program should finish by stating that the input is allowed.

Appendix B: Adding Analyses

Checkmate makes adding analyses extensible through an interactive manager, human-readable settings, and a general C++ analysis framework. New analyses define detector setup, signal regions, event selections, and automatically saved outputs.

  • Extensibility: Checkmate is designed to be easy to extend with new analyses or user-requested analyses.The program emphasizes simplicity and transparency alongside extensibility.
  • Analysis Manager: The Analysis Manager can list, add or modify, and remove implemented analyses.Adding or modifying an analysis prompts for its name, luminosity, detector setup, signal regions, and experimental observations and backgrounds.
  • Analysis framework: The general C++ framework supplies particle containers and reusable methods for phase-space reduction and overlap removal.It is based on a globally defined base class and supports final-state objects such as leptons and jets.
  • Analysis framework: A minimal example illustrates an analysis with three signal regions, a two-step cutflow, and isolated-electron selections.The example requires electrons with pT > 20 GeV, |η| < 2.5, and medium identification.
  • Outputs and objects: Particle containers are pT-sorted, and final signal, control, and cutflow numbers are automatically written to output files.The framework also provides access to particle four-vectors through P4().
  • Detector efficiencies: Electron efficiencies combine reconstruction and identification parametrisations, with tight identification using pT dependence multiplied by renormalised pseudorapidity dependence.Checkmate uses functional behaviour rather than only the illustrative discretised efficiency map.

3. Muons

The muon treatment models reconstruction through combined and standalone tracks and a detector-component efficiency map. The b-tagger balances tagging efficiency against light-jet and c-jet rejection using parametrised ROC behaviour.

  • Muons: Muon reconstruction distinguishes Combined, Standalone, and Combined+Standalone quality criteria.Combined requires inner-detector and muon-chamber tracks, while Combined+Standalone uses a standalone track when no combined track is reconstructed.
  • Muons: Checkmate assigns muon reconstruction efficiencies through a detector-component map in the η–φ plane.Different detector components receive different efficiencies because of material and geometry differences.
  • B-tagging: B-tagging quality is described by signal efficiency and background efficiency, with rejection used as the inverse background efficiency.Charm-jet and light-jet rejection are treated separately because charm jets are harder to distinguish from b-jet signals.
  • B-tagging: Choosing a b-tagging working point requires balancing signal quantity against purity because rejection weakens as signal efficiency increases.Checkmate uses separate ROC curves for light-jet and c-jet rejection and parametrises them internally.
  • B-tagging: The inclusive c-jet efficiency is assumed to be 40 % of the D∗-meson efficiency.The parametrisation uses D∗-containing jets as a reference.

5. Tau–Tagger

The tau-tagger models one- and three-prong candidates with separate working points and background treatments. Its parametrisations cover QCD-jet discrimination and electron-jet rejection, with explicit scope limitations.

  • Tau-tagging setup: Tau-tagging efficiencies distinguish one-prong and three-prong candidates across loose, medium, and tight working points.The corresponding signal and light-parton background functions are given in Equations (C10) and (C11).
  • QCD-jet discrimination: The QCD-jet tagging efficiencies do not depend on η, with |η| < 2.5 implied by tracking-detector coverage.This condition applies to the final efficiency functions for one- and three-prong jets.
  • Electron rejection: Electron jets require a separate rejection algorithm because they can resemble one-prong tau decays.The electron-background efficiencies depend on both transverse momentum and pseudorapidity.
  • Scope limitation: Checkmate does not implement a specific electron mistagging efficiency; failed electron-identification objects are counted as jets and tagged with the background efficiency.This is a stated limitation of the Delphes implementation.
  • Combined efficiencies: The final one-prong signal efficiencies combine rejection contributions from QCD jets and electrons for each working point.These combined efficiencies are shown in Figures 18a to 18c and use the corresponding functions and parameters.

1. atlas conf 2012 104

The validation programme compares Checkmate with published cutflows, distributions, and parameter scans across several LHC analyses. In the CMSSM validation, Checkmate gives a slightly weaker exclusion because it uses only the signal region with the best expected sensitivity.

  • CMSSM validation: Validation against the published CMSSM parameter scan found a slightly weaker Checkmate exclusion than the ATLAS result.The difference is attributed to Checkmate’s limit-setting procedure.
  • CMSSM validation: Checkmate selects the signal region with the best expected sensitivity, whereas ATLAS used a combined likelihood of all signal and control regions.This explains the difference in the exclusion curve for atlas conf 2012 104.
  • Distribution validation: Standard Model W and Z samples were validated using leading-jet pT and missing-transverse-momentum distributions in signal regions.The ATLAS backgrounds were estimated from Monte Carlo samples normalised with control-region data.
  • Validation limitations: The distribution validation is limited to hardest-jet pT < 600 GeV because of finite Monte Carlo statistics.This is an explicit validation boundary.
  • Stop validation: Stop-search validation used published cutflows and a pure stop-production model with the decay t̃ → tχ̃0_1.The validation model had right-handed top polarization in 95% of decays.
  • Additional validations: Other analyses were validated against all published cutflows, while the 0-lepton analysis also included a CMSSM parameter-scan validation.These validations cover trilepton and multijet plus missing-transverse-momentum searches.

6. atlas conf 2013 049

The section documents CheckMATE validation against ATLAS analyses, including cutflow agreement and implementation adjustments. It also records limitations affecting specific simplified models and exclusion interpretations.

  • Validation: Validation was performed against all published cutflows for the considered ATLAS analyses.
  • Implementation: 25% tighter jet-veto conditions were used to match cutflows and account for pile-up effects unavailable in CheckMATE.
  • Limitations: CheckMATE systematically underestimates cutflows for simplified chargino production with weak-boson-mediated decays, conservatively avoiding spurious exclusions.
  • Limitations: No validation had yet been performed with parameter scans for the considered analyses.
  • Implementation: 3% lower b-tagging efficiency was used than the experimental nominal value to improve agreement with cutflow data.
  • Exclusion results: A difference in exclusion appears for low squark masses and a heavier LSP, attributed by the authors to harder initial-state radiation from Pythia 6 settings.

9. cms 1303 2985

This section presents CMS analyses and simplified-model exclusion studies implemented in CheckMATE. It describes validation inputs, analysis assumptions, signal-region procedures, and exclusion curves for several strong-production scenarios.

  • Analyses and models: The analyses include hadronic αT and b-jet multiplicity selections, as well as distributions in HT for squark, bottom-squark, stop, and gluino models.
  • Validation: CMS validation used signal-region distributions and parameter scans because no CMS cutflow was provided.
  • Implementation: The implementation used Standard Model background distributions from the CMS note and ATLAS b-tagging efficiencies.
  • Limit setting: Jumps in CheckMATE parameter-scan limits arise from the single-signal-region limit-setting procedure.
  • Exclusion results: Exclusion curves cover simplified models with first- and second-generation squarks, bottom squarks, and gluinos decaying to light quarks or top quarks.
  • Exclusion results: The first- and second-generation squark exclusion differs at smaller squark masses and heavier LSP masses, which the authors associate with harder Pythia 6 initial-state radiation.
Loading 1312.2591v2…