PaperlayerView exact revision on arXiv

Source-linked AI summary

Parton Shower Uncertainties in Jet Substructure Analyses with Deep Neural Networks

James Barnard, Edmund Noel Dawe, Matthew J. Dolan, Nina Rajcic

arXiv:1609.00607v2hep-phhep-ex

TL;DR

Training jet-image networks on simulated events raises the question of whether they learn generator-specific approximations rather than features that carry over to real data. The paper compares shower models and studies pT-dependent zooming, finding substantial variation in background rejection across showers and improved performance with zooming. It concludes that the features used by these networks need careful scrutiny.

  • Problem

    Because labelled network training requires simulated data, it is important to determine whether a DNN learns generator-dependent approximations that may not be present in real data.

    Source: §I · pp. 1–2, §IV · p. 4

  • Method

    The study compares DNN performance across event generators and their shower and hadronisation models, and tests pT-dependent image zooming for W-versus-QCD tagging.

    Source: §I · p. 2, §III · p. 3

  • Results

    Parton-shower variation changes background rejection by up to 50%, while zooming improves network performance by around 10–20% across a wide pT range.

    Source: §I · p. 2

  • Takeaways & Limitations

    The results underscore the need to understand which jet-image features DNN taggers exploit before interpreting their performance on data.

    Source: §I · p. 2, §V · p. 7

    The study does not quantify real-world QCD effects such as colour reconnection, which are poorly understood despite being modelled in parton showers.

    Source: §IV · p. 4

Abstract

from arXiv · show

Machine learning methods incorporating deep neural networks have been the subject of recent proposals for new hadronic resonance taggers. These methods require training on a dataset produced by an event generator where the true class labels are known. However, training a network on a specific event generator may bias the network towards learning features associated with the approximations to QCD used in that generator which are not present in real data. We therefore investigate the effects of variations in the modelling of the parton shower on the performance of deep neural network taggers using jet images from hadronic W-bosons at the LHC, including detector-related effects. By investigating network performance on samples from the Pythia, Herwig and Sherpa generators, we find differences of up to fifty percent in background rejection for fixed signal efficiency. We also introduce and study a method, which we dub zooming, for implementing scale-invariance in neural network-based taggers. We find that this leads to an improvement in performance across a wide range of jet transverse momenta. Our results emphasise the importance gaining a detailed understanding what aspects of jet physics these methods are exploiting.

I. INTRODUCTION

Deep neural networks can learn generator-dependent approximations when trained on simulated jet images, leaving their robustness to parton-shower and detector effects unresolved. This paper compares shower models and studies pT-dependent zooming, finding substantial shower-related variation and improved performance with zooming.

I. INTRODUCTION

  • Hadronic-resonance tagging matters for Standard Model analyses and searches for beyond-the-Standard-Model states at the LHC.Earlier work proposed neural networks for jet-image classification, including hadronic W identification, while exploring truth-level simulation and detector effects.Source: §I · p. 1
  • A key unresolved issue is whether a network learns signal–background physics or generator-specific approximations, including soft-spectrum features modeled phenomenologically rather than through perturbative QCD.The study also raises robustness questions about detector smearing and pile-up.Source: §I · pp. 1–2
  • The study compares neural-network behavior across event generators, and thus across their parton showers and hadronisation models.For simplicity, the paper refers to these collectively as the parton shower.Source: §I · p. 2
  • Parton-shower variations change background rejection by up to 50%, while scale variations are negligible and pile-up degrades performance without changing the conclusions.The authors call for caution in applying these methods to data and for understanding which jet-image features networks use.Source: §I · p. 2
  • The paper applies pT-dependent zooming to jet images as a way to implement scale-invariant tagging, improving network performance by around 10–20% across a wide pT range.The W-versus-QCD comparison serves as a standard-candle study, with the approach framed as applicable to wider tagging and substructure problems.Source: §I · p. 2

II. NETWORK ARCHITECTURE, TRAINING, AND PERFORMANCE EVALUATION

The study uses a dense MaxOut neural network trained on simulated W and QCD jet images, then evaluates its background rejection as a function of signal efficiency. Training uses a held-out loss sample and early stopping.

II. NETWORK ARCHITECTURE, TRAINING, AND PERFORMANCE EVALUATION

  • The network takes 625 pixel inputs and uses dense MaxOut layers of 256 and 128 units, followed by fully connected 64- and 25-unit ReLU layers and a two-node sigmoid output.The reported results use the MaxOut architecture, following the choice in Ref..Source: §II · p. 2
  • Training uses about 3M signal and 3M background images in 50 < m < 110 GeV and 200 < pT < 500 GeV, with flat-pT weighting and 10% held out for cross-entropy evaluation.Training stops after 100 epochs or 10 epochs without loss improvement; tested Bayesian optimisation did not significantly improve performance.Source: §II · p. 2
  • Performance is measured as inverse background efficiency versus signal efficiency using a binned likelihood ratio, which highlights rejection differences at low signal efficiency.The measure can also be constructed from arbitrary jet observables or combinations of observables.Source: §II · p. 2

III. CONSTRUCTING A JET IMAGE

The paper constructs jet images through clustering, trimming, spatial alignment, pixelisation, and normalisation, with an optional pT-dependent zooming step. Zooming improves rejection across the studied pT range and makes W-jet images more compact while leaving QCD subjets comparatively smeared.

III. CONSTRUCTING A JET IMAGE

  • Jets are reconstructed from calorimeter towers with anti-kt at R = 1.0, trimmed with kt subjets of size r = 0.3, and subjets below 5% of the original jet momentum are discarded.The leading jet is selected, and all observables are computed from the trimmed jet.Source: §III · p. 3
  • Image construction translates the leading subjet to the origin, pixelises transverse energy, then rotates and reflects the image to align subjets or the principal component.Cubic-spline interpolation handles transformed pixels that do not align with the original grid.Source: §III · p. 3
  • The image-standardisation goal is to improve DNN performance across jet pT, while scale-invariant searches can span boosted and resolved phase space.Such searches may apply across a broad range of masses and kinematics.Source: §III · pp. 3–4
  • The zooming scale uses s = 2mW/pT and enlarges images by max(R/s, 1), standardising signal-subjet separation while tending to smear the subleading subjet in background images.Using actual subjet separation would standardise both signal and background, enhancing signal-like features in background images.Source: §III · p. 3
  • Images are cropped to 25 × 25 pixels and normalised so the squared pixel intensities sum to one; this does not preserve jet mass, which zooming also destroys.The construction follows the procedure in Ref. with optional zooming to reduce pT dependence.Source: §III · pp. 2–3
  • For PYTHIA images, zooming makes average W-jet shapes more compact and the lower-pT subjet better defined, while QCD subjets remain comparatively smeared.Figure 1 compares standard-preprocessed images in the top panels with zoomed images in the bottom panels; its colors encode average normalised pixel intensity.Source: §III · p. 4, Figure 1 · p. 4
  • Zoomed networks outperform unzoomed ones, with background rejection rising around 20% particularly at low signal efficiency; gains are around 10% for less-boosted W decays and just over 20% for 300 < pT < 500 GeV.The networks were trained and tested on samples with 50 < m < 110 GeV and 200 < pT < 500 GeV; Figure 2 shows PYTHIA default-shower results.Source: §III · p. 4, Figure 2 · p. 4

IV. EVENT GENERATOR DEPENDENCY

The section tests how generator and shower modeling affect DNN jet-image taggers, with generator choice producing substantial background-rejection differences. It compares shower-dependent ROC curves and radiation patterns, while treating their envelope as an approximate uncertainty from training on simulation rather than real data.

IV. EVENT GENERATOR DEPENDENCY

  • The study does not quantify effects such as colour reconnection, which are modeled in parton showers but remain poorly understood.Source: §IV · p. 4
  • The study treats the envelope of ROC curves from different generators and showers as an approximate uncertainty band for training on simulated rather than real data.The focus is algorithmic uncertainty from different shower-algorithm choices.Source: §IV · p. 5
  • The comparison covers five shower models across PYTHIA, SHERPA, and HERWIG, with Delphes used to incorporate detector effects such as smearing.The models include default PYTHIA and VINCIA, default angular-ordered HERWIG and its dipole shower, and default SHERPA.Source: §IV · p. 5
  • Relative to default PYTHIA, SHERPA and HERWIG dipole show more intense radiation in resolved subjets and a deficit between them, while HERWIG angular shows the opposite pattern.VINCIA is described as roughly similar to default PYTHIA; QCD radiation exhibits similar features.Source: §IV · p. 5
  • At fixed low signal efficiency, HERWIG angular and dipole background rejection is half PYTHIA’s; at 50% signal efficiency, changing generators gives around 40% uncertainty.The comparison uses a DNN trained on default PYTHIA and evaluated on events from other generators and showers.Source: Figure 4 · p. 6, §IV · p. 6
  • Factor-two variations of the common renormalisation and factorisation scales produce negligible changes in the ROC curves.The default scale is set to μ = pT,W.Source: §IV · p. 7

V. CONCLUSIONS

A network trained on PYTHIA performed better on PYTHIA test events than on HERWIG and SHERPA, with parton-shower changes shifting background rejection by up to a factor of two. The authors urge caution and identify data-driven unsupervised learning as one possible route toward generator independence.

V. CONCLUSIONS

  • Up to a factor of two in background rejection separated parton-shower samples, and a PYTHIA-trained network performed better on PYTHIA test events than on HERWIG and SHERPA.The authors caution against over-interpreting small ROC-curve changes given these uncertainties.Source: §V · p. 7
  • It remains unclear whether performance differences arise from perturbative parton-shower features or softer physics such as hadronisation modelling.The authors state that further study is needed to determine what drives the network performance.Source: §V · p. 7
  • Data-driven unsupervised learning that avoids training data is proposed as one possible route to event-generator independence.The authors identify this as an opportunity for further work.Source: §V · p. 7

Related papers

Loading 1609.00607v2…