Pulling Out All the Tops with Computer Vision and Deep Learning

Sebastian Macaluso; David Shih

Source-linked AI summary

Pulling Out All the Tops with Computer Vision and Deep Learning

Sebastian Macaluso, David Shih

arXiv:1803.00107v1Feb 28, 2018hep-phhep-ex

TL;DR

Boosted top-jet tagging needs effective ways to distinguish top jets from QCD in LHC analyses. The paper improves a CNN-based DeepTop tagger across architecture, training, preprocessing, sample size, and image channels, and finds stronger background rejection than conventional taggers. The authors also identify limited pT coverage, omitted pileup, and incomplete architecture optimization as scope boundaries.

Problem
Boosted top jets are important LHC signatures, but tagging them requires distinguishing their fat-jet substructure from light-flavor QCD.
Method
The paper improves the DeepTop CNN through changes to architecture, training, image preprocessing, training-sample size, and color pixels.
Results
∼2–3 or more in background rejection: the final CNN outperforms conventional top taggers across a wide range of tagging efficiencies.
Takeaways & Limitations
The approach can be straightforwardly extended to other jet types and offers lessons for designing deep neural networks for LHC applications.
Takeaways & Limitations
The study ignores pileup, restricts top pT to 350–450 and 800–900 GeV, and does not perform a proper architecture and hyperparameter scan.

Abstract

from arXiv · show

We apply computer vision with deep learning -- in the form of a convolutional neural network (CNN) -- to build a highly effective boosted top tagger. Previous work (the "DeepTop" tagger of Kasieczka et al) has shown that a CNN-based top tagger can achieve comparable performance to state-of-the-art conventional top taggers based on high-level inputs. Here, we introduce a number of improvements to the DeepTop tagger, including architecture, training, image preprocessing, sample size and color pixels. Our final CNN top tagger outperforms BDTs based on high-level inputs by a factor of $\sim 2$--3 or more in background rejection, over a wide range of tagging efficiencies and fiducial jet selections. As reference points, we achieve a QCD background rejection factor of 500 (60) at 50\% top tagging efficiency for fully-merged (non-merged) top jets with $p_T$ in the 800--900 GeV (350--450 GeV) range. Our CNN can also be straightforwardly extended to the classification of other types of jets, and the lessons learned here may be useful to others designing their own deep NNs for LHC applications.

1 Introduction

Boosted-jet tagging is important for LHC measurements and searches, motivating CNNs that learn discriminative jet features from image-like inputs. This work improves DeepTop and reports stronger performance than conventional taggers across two jet samples.

1 Introduction: CNNs can treat jets as images and learn spatially local and higher-level features without hand-engineered inputs.Jet-image intensities can use per-pixel pT and additional quantities such as track multiplicity.
1 Introduction: The paper improves DeepTop through architecture, training, preprocessing, larger samples, and multiple color channels, with about 1M training jets saturating performance.The color channels include calorimeter pT, track pT, track multiplicity, and muon multiplicity.
1 Introduction: ∼3–10 in the CMS sample and ∼1.5–2.5 in the DeepTop sample: cumulative improvements increase DeepTop background rejection in both samples.Training produced the largest single improvement, followed by architecture and training-sample size.
1 Introduction: ∼2–3 or more in background rejection: the CNN outperforms conventional top taggers across a wide range of tagging efficiencies.The comparison targets conventional taggers representing state-of-the-art and best-in-use methods.
1 Introduction: The approach is presented as a broader boosted-object-tagging case study that could extend to other jet types and related LHC applications.The authors note that more systematic hyperparameter scans could yield further gains.

2 Methodology

The study compares improved CNN top tagging on two simulated jet-image samples that differ in boost, detector information, image construction, and merging. The methodology benchmarks the CNN against conventional high-level-input taggers while using DeepTop as the baseline.

2 Methodology: The two samples differ in boost, detector inputs, calorimeter resolution, color channels, and whether top daughters must be contained within the fat jet.The DeepTop sample is moderately boosted and non-merged, while the CMS sample is higher-pT and merged.
2 Methodology: The merge requirement makes top jets more top-like and increases their potential discrimination against QCD jets.The authors expect the CMS-sample ROC curves to look better than those for the DeepTop sample.
2 Methodology: The CNN is benchmarked against high-level-input BDTs, including MotherOfTaggers for DeepTop and HTTV2-plus-N-subjettiness inputs for the CMS sample.The DeepTop comparison uses the published MotherOfTaggers ROC curve.
2 Methodology: The study starts from DeepTop, whose CNN was previously shown to perform comparably to a conventional high-level-input BDT.This establishes the baseline for evaluating the proposed improvements.

3 Improvements to the neural network

The authors improve the DeepTop CNN through revised training choices and a larger architecture, while noting that the design was selected mainly by trial and error.

Loss function: Replacing DeepTop’s mean-squared-error loss with binary cross entropy was chosen for classification and helps avoid slowdown near predictions of zero or one.
Optimizer algorithm: Switching from vanilla SGD to AdaDelta, tuning the learning rate, and reducing minibatches from 1000 to 128 improved training outcomes.
Architecture: Increasing feature maps from 8 to 64–128 and dense-layer width from 64 to 256 improved performance considerably.
Architecture: The final CNN processes 37 × 37 images with up to four colors through convolutional, pooling, dense, and two-neuron output layers.
Limitations: The architecture was chosen mainly by trial and error because limited resources prevented a thorough scan, leaving further gains possible.

4 Image preprocessing

The paper replaces DeepTop’s detector-resolution preprocessing with preprocessing before pixelation, using jet geometry to expose discriminating substructure.

Motivation: DeepTop’s preprocessing had decreased tagger performance, contrary to the usual expectation that preprocessing improves classifiers.
Method: The new procedure preprocesses before pixelation and uses the pT-weighted centroid and principal axis, differing from DeepTop’s sequence of pixelation, alignment, flipping, and normalization.
Visualization: Figure 2 averages 100k CMS top and QCD images on 37 × 37 pixels spanning Δη = Δφ = 3.2, with grayscale intensity representing per-pixel total pT.
Effectiveness: After preprocessing, average top images visibly exhibit three-prong structure while QCD images remain more dipole-like.

5 Other improvements

The authors study training-set size and color channels as additional improvements, finding saturation near one million jets and overfitting from color at smaller sample sizes.

Sample size: Figure 3 compares test accuracy with training sample size for CMS and DeepTop jets, shifting the DeepTop curve upward by 0.075 and overlaying an inverse-power-law fit.
Sample size: Performance is essentially saturated for ≳1M training jets, motivating the final 1.2M+1.2M-jet training sample.
Sample size: The inverse-power-law fit is presented only as a visual guide because uncertainties in test accuracies are not precisely understood.
Color: Adding color uses neutral pT, track pT, track multiplicity, and muon multiplicity per pixel for the CMS sample.
Color: Color channels caused significant overfitting with smaller training sets because the network learned image noise, but this disappeared at 1.2M+1.2M examples.

6 Final comparison

The improved CNN substantially increases background rejection across both jet samples and outperforms conventional high-level-input taggers. Training changes provide the largest individual gain, while architecture and sample-size improvements add substantial benefits.

Cumulative improvements: The ROC figures show cumulative improvement sequences for the DeepTop and CMS samples, with the final DeepTop tagger highlighted in orange.Figure 6 displays ratios of the improved ROC curves to the minimal DeepTop baseline.
Cumulative improvements: Training improvements provide the single largest boost, with over half of that gain attributed solely to the smaller minibatch size.The improved methods also reduce convergence from O(10^3) to O(10) training epochs.
Cumulative improvements: A factor of ∼3–10 higher background rejection for the CMS sample and ∼1.5–2.5 for the DeepTop sample results from combining all improvements.The aggregate effects are shown through cumulative ROC curves and metrics for both samples.
Comparison with conventional taggers: A factor of ∼3 higher background rejection than the CMS-sample BDT is achieved by the final CNN, while the DeepTop-sample CNN exceeds its BDT by ∼2.At tagging efficiencies near 0.1, the BDT and deep-learning tagger perform very similarly.

7 Outlook

The paper outlines future applications and identifies simplifying assumptions and scope limitations requiring further study.

7 Outlook: Pileup was ignored, and tagger stability was tested only in two narrow top-pT ranges.The authors say realistic pileup conditions and broader pT coverage should be checked for experimental use.
7 Outlook: The study omitted b-tagging because publicly available impact-parameter and secondary-vertex information was inadequate for accurate recasting.The authors expect adding b-tagging could further improve performance, but its deep-learning benefit remains untested.
7 Outlook: The tagger’s agreement between training and test samples does not resolve whether Pythia with default settings represents actual data.The authors identify generator dependence as an unresolved overtraining concern.
7 Outlook: A GPU cluster would enable broader architecture and hyperparameter scans, including tests of established CNN architectures.The architecture used was inspired by DeepTop, while alternatives such as ResNet and VGG were not evaluated.
7 Outlook: The approach could be extended to W/Z bosons, Higgses, BSM particles, and partially merged or fully resolved tops.The paper suggests that distinguishing merged categories could improve background rejection at fixed efficiency for merged tops.
7 Outlook: The authors place these limitations and extensions within a broader outlook for machine learning applications in high-energy physics.They mention full-event classification, generative modeling, and unsupervised discovery as future directions.

A Validating our DeepTop implementation

The authors validate their DeepTop implementation using the original sample, preprocessing, architecture, and training procedure, finding excellent agreement with the published result.

A Validating our DeepTop implementation: The validation reproduces the DeepTop setup with 14 TeV collisions, 350–450 GeV jets, centered 40×40 images, and 600k top plus QCD images.The sample uses the stated matching and no-merge requirements, with a 25%/25%/50% data split.
A Validating our DeepTop implementation: The default DeepTop architecture and training methods produce excellent agreement with the original tagger’s ROC curve.Figure 9 compares the implementation against a digitized published “DeepTop minimal” curve.

B Validating our HEPTopTaggerV2 implementation

The authors validate their HTTV2 plus N-subjettiness implementation against CMS, reproducing its ROC behavior with the stated variables and simplified cuts.

B Validating our HEPTopTaggerV2 implementation: HTTV2 clusters jet constituents into b- and W-consistent subjets and outputs mjet, frec, and Ropt.These represent jet mass, W-likeness, and an optimal jet radius that can differ from the input radius.
B Validating our HEPTopTaggerV2 implementation: N-subjettiness observables measure how likely a jet is to contain a specified number of subjets.The validated CMS-like sample uses 800–1000 GeV jets and R = 0.8, with slight differences from the authors’ CMS sample.
B Validating our HEPTopTaggerV2 implementation: The implementation’s ROC curve agrees well with the CMS result for an HTTV2 plus N-subjettiness tagger.The comparison uses the authors’ implementation overlaid on the digitized CMS curve in Figure 10.
B Validating our HEPTopTaggerV2 implementation: CMS optimizes simple window cuts on mjet, frec, and τ32 = τ3/τ2 for a target tagging efficiency.The implementation omits the ΔRopt cut because CMS reports it has the least discriminating power.
B Validating our HEPTopTaggerV2 implementation: HTTV2 failures to find at least three subjets produce no outputs and must be included in the tagger efficiency.This is an explicit efficiency-accounting boundary for HTTV2-based taggers.

C Importance of the merge requirement

The merge requirement materially affects top-tagging performance and may explain discrepancies between the DeepTop and CMS comparisons.

C Importance of the merge requirement: Non-merged tops produce fat jets lacking the full top-decay energy, weakening their distinction from QCD.The authors connect the performance gain from merging to more accentuated differences between top jets and QCD jets.
C Importance of the merge requirement: The fully merged selection substantially improves the CMS-sample ROC performance relative to removing the merge requirement.Without merging, fat jets may omit part of the top decay energy; the comparison appears in Figure 11.
C Importance of the merge requirement: A merge-requirement difference may explain why DeepTop performs roughly threefold worse than CMS on similar low-pT samples.CMS imposes a ΔR(t,q) < 0.8 requirement for low-pT tops, whereas DeepTop does not.