QCD-Aware Recursive Neural Networks for Jet Physics

Gilles Louppe; Kyunghyun Cho; Cyril Becot; Kyle Cranmer

Source-linked AI summary

QCD-Aware Recursive Neural Networks for Jet Physics

Gilles Louppe, Kyunghyun Cho, Cyril Becot, Kyle Cranmer

arXiv:1702.00748v2Feb 2, 2017hep-phphysics.data-anstat.ML

TL;DR

The paper addresses information loss and resource demands in image-based jet learning by introducing QCD-informed recursive networks over particle four-momenta and clustering trees. These networks improve accuracy and data efficiency over image-based approaches, while extending naturally to event-level classification over all stable particles.

Problem
Image-based jet methods discretize variable-length four-momenta into pixels, potentially losing information and requiring many parameters and large datasets.
Method
The paper recursively embeds four-momenta using event-specific sequential-recombination trees, jointly trains task-specific jet embeddings, and composes jet embeddings recurrently for full events.
Results
The recursive networks achieve significant accuracy and data-efficiency gains over previous image-based networks, including ROC AUC improvement from 0.8321 to 0.8807 for kt tower inputs without images.
Takeaways & Limitations
The approach provides QCD-motivated fixed-length embeddings for task-specific jet prediction and an event-level classifier operating on all detectable event particles.
Takeaways & Limitations
General infrared and collinear safety is difficult to guarantee because nonlinear network operations act on the embedding.

Abstract

from arXiv · show

Recent progress in applying machine learning for jet physics has been built upon an analogy between calorimeters and images. In this work, we present a novel class of recursive neural networks built instead upon an analogy between QCD and natural languages. In the analogy, four-momenta are like words and the clustering history of sequential recombination jet algorithms is like the parsing of a sentence. Our approach works directly with the four-momenta of a variable-length set of particles, and the jet-based tree structure varies on an event-by-event basis. Our experiments highlight the flexibility of our method for building task-specific jet embeddings and show that recursive architectures are significantly more accurate and data efficient than previous image-based networks. We extend the analogy from individual jets (sentences) to full events (paragraphs), and show for the first time an event-level classifier operating on all the stable particles produced in an LHC event.

1 Introduction

The paper replaces image-based jet representations with QCD-informed recursive networks that preserve particle four-momenta and adapt their topology to each event. It extends this construction from individual jets to full events through a sequence of jet embeddings.

Boosted hadronic decays merge into fat jets whose substructure must be distinguished from ordinary QCD backgrounds.
Image-based methods project variable-length four-momenta onto fixed η−φ grids, introducing many parameters and substantial data requirements.
Recursive networks use sequential-recombination clustering histories as event-specific tree topologies, while directly retaining particle four-momenta and substructure.
The resulting fixed-length jet embedding can be jointly trained with a classifier or regressor for task-specific optimization.
Full events are represented by feeding jet embeddings into a recurrent network, analogous to composing paragraphs from sentences.

2 Problem statement

The paper formalizes collision events as variable-size sets of particle four-momenta and clusters them into binary jet trees using sequential-recombination algorithms. It then defines jet- and event-level supervised prediction objectives.

Each collision event is a variable-size set of particles, with every particle represented by a four-momentum vector vi ∈ R4.
Sequential recombination algorithms recursively combine particle pairs by adding their four-momenta to form jets.
The algorithms use hyperparameters R, pt,min, and α, with jets below pt,min discarded.
After clustering, each retained jet is represented by a binary tree whose leaves correspond to subsets of the event particles.
Jet- and event-level models minimize separate loss functions for classification or regression targets.

3 Recursive embedding

The architecture recursively embeds binary jet trees into fixed-size vectors and jointly trains those embeddings with downstream predictors. A second recurrent stage composes jet embeddings into an event representation.

Individual jets: Each binary-tree node represents either a particle leaf or a recombination, with the hardest child designated as the left child.
Individual jets: Leaf nodes use particle inputs, while internal nodes recursively combine child representations through shared neural parameters.
Individual jets: The recursive network maps binary trees of varying shape and size into fixed-size jet embeddings.
Individual jets: Jet embeddings and downstream classifiers or regressors are jointly optimized by backpropagation through structure.
Individual jets: Gating lets the network preferentially pass information from the left child, right child, or both.
Full events: An event-level recurrent network consumes jet four-momenta paired with their embeddings, and its output feeds a classifier.
Full events: The two recurrence levels provide a QCD-motivated embedding operating on all event particles at hadron level.
Full events: Simpler comparisons use only jet four-momenta or directly sequence all event constituent four-momenta without jet embeddings.

4 Data, Preprocessing and Experimental Setup

The experiments evaluate boosted W tagging with particle-level, detector-tower, and image inputs, using matched signal and background samples. Event-level studies use separately generated samples with two high-pT jets in both classes.

The jet-level task distinguishes hadronically decaying W bosons from QCD jets, with both classes restricted to 200 < pT < 500 GeV.
The shared benchmark includes particle-level records, detector towers, and preprocessed 25×25-pixel jet images.
Training and testing each use 100,000 signal and 100,000 background jets, with evaluation restricted to 250 < pT < 300 and 50 ≤ m ≤ 110.
Particle and tower inputs are compared with and without projection into images after identifying constituents of the highest-pT anti-kt jet.
The event-level study uses newly generated W′ signal and QCD background samples because the borrowed samples were trivially separable.
Both jet- and event-level embeddings use dimension q = 40 and Adam training with validation-based early stopping.

5 Experiments with Jet-Level Classification

The jet-level studies evaluate how input representation, momentum source, tree topology, and gating affect recursive-network classification. Direct four-momentum inputs, suitable topologies, and gating improve performance, while robustness to collinear modifications depends on topology and is not guaranteed by construction.

Input representations: Removing image preprocessing improves kt-topology performance from ROC AUC 0.8321 to 0.8807 and Rϵ=50% from 12.7 to 24.1.The improvement is reported for tower 4-momenta.
Input representations: Direct variable-length four-momentum inputs avoid the information loss associated with image projection and outperform classifiers based on jet-substructure variables.The paper attributes possible image-level losses to pixelisation, transformations, lower resolution, particle superposition, and aliasing.
Momentum measurements: Particle-level inputs provide more accurate momentum measurements than the DELPHES tower scenario, motivating their separate evaluation.The particle scenario is described as an idealized setting in which tracking measurements are combined with calorimetry through particle flow.
Tree topologies: kt and C/A topologies significantly outperform anti-kt on tower and particle inputs, while descending-pT ordering can slightly outperform kt and C/A.Random trees perform poorly; descending-pT performance is not consistently superior across particle and tower inputs.
Gating: Gating significantly improves results, especially Rϵ=50%, while preserving the topology trends and using about 20 times fewer parameters than the cited MaxOut architectures.The gated networks have 48,761 parameters.
Infrared and collinear safety: kt-based recursive networks show greater robustness to collinear splits than descending-pT networks, but general IRC safety is difficult to ensure because of network nonlinearities.For collinear10-max, Rϵ=50% decreases by 4% for kt versus 10% for pT; the study also notes that training used only nominal classification loss.

6 Experiments with event-level classification

Event-level experiments show that QCD-motivated nested recurrence performs best when combining jet embeddings with recurrence over jets, while additional jets provide little benefit beyond the two hardest.

Experimental scope: The experiments primarily demonstrate the relevance and scalability of the QCD-motivated event-level approach rather than the signal process’s physics reach.Results use perfectly measured stable hadrons from PYTHIA in the idealized particles scenario; omitted tower experiments showed similar qualitative conclusions.
Number of jets: Significant improvement occurs when increasing the GRU input from the hardest jet to the two hardest jets, with little or no gain from adding more jets.The two highest-pT jets are expected to contain most of the discriminating information.
Topology of the binary trees: Descending-pT binary-tree topologies achieve the best results, but their advantage over k_t topologies is marginal.The comparison concerns the inner jet-embedding architectures.
Other variants: Adding jet-level embeddings to the event GRU improves ROC AUC from 0.9606 to 0.9875 and Rϵ=80% from 21.1 to 174.5 for the two-hardest-jets case.The baseline uses only the four-momenta of the jets.
Other variants: Directly feeding constituent four-momenta to the GRU performs significantly worse, although the ROC AUC remains 0.8925 when using the 50 highest-pT four-momenta.Grouping constituents into jets and structuring each jet as a tree makes relevant information easier to identify and learn.
Overall event-level result: The study reports the first classifier operating at hadron level on all particles in an event, using nested recurrence over jets and their constituents.Table 3 summarizes event-classification performance for this QCD-motivated construction.

7 Related work

The related work situates the approach among earlier neural-network applications, recent jet-physics activity, and methods connecting jet clustering structures with machine learning.

Earlier particle-physics applications: Neural networks have long been applied in particle physics tasks including quark-gluon discrimination, particle identification, Higgs tagging, and track identification.Earlier systems were often shallow multilayer perceptrons with expert-designed input features.
Event-level studies: Figure 5 evaluates event-classification performance as the maximum number of jets supplied to the GRU varies.The comparison covers the hardest jet, the two hardest jets, and additional jets.
Subsequent related methods: Recent work applied this method to quark/gluon tagging, used a variant to reconstruct jet charge, and used jet-clustering trees to define a substructure representation.These developments appeared amid broader activity linking deep learning and jet physics.

8 Conclusions

The paper concludes that QCD-informed recursive networks provide a direct alternative to image-based jet models and extend hierarchically to event-level classification over all hadrons.

Jet-level conclusions: The recursive networks work directly with variable-length particle four-momenta, avoiding information loss from pixelization in jet-image preprocessing.The authors report significant gains in accuracy and data efficiency relative to previous image-based networks.
Event-level conclusions: The method provides the first hierarchical event-level classification model operating on all hadrons of an event.Its structure is motivated by QCD and combines recurrence over jets with recurrence over their constituents.
Role of domain knowledge: Incorporating jet-algorithm domain knowledge into network architecture improves classification performance.The conclusion identifies architectural encoding of physics knowledge as a central result.
Robustness and future directions: Jet-tree architectures appear more robust to infrared radiation and collinear splittings than simpler pT-ordered architectures, despite the latter’s sometimes better apparent performance.The authors note that this robustness may outweigh a small performance loss.
Robustness and future directions: The topology affects classifier performance, and learning the embedding topology is identified as a future challenge because topology changes can make the loss non-differentiable.Standard backpropagation cannot directly optimize such discontinuous topology changes.

A Gated recursive jet embedding

The gated recursive jet embedding addresses limitations of a basic recursive activation by using reset and update gates to selectively combine child and local information.

Motivation: The basic recursive activation has two issues: equal treatment of inputs and forced nonlinear transformations that prevent unchanged information flow.These inputs include left-child, right-child, and local node information.
Recursive construction: For leaf nodes, the recursive construction uses local inputs including uk and vi(k).The passages specify these leaf cases directly in the recursive definition.
Recursive construction: The embedding uses shared learned parameters, ReLU activations, and element-wise multiplication.The parameter dimensions and operation definitions are given alongside the recursive equations.
Gated activation: Reset gates control how left-child, right-child, and local-node embeddings are selected and merged.The gated architecture can preferentially pass information along either child or their combination.
Gated activation: Update gates select among the candidate activation, left-child embedding, right-child embedding, and local node information.The final embedding is controlled by zH, zL, zR, and zN.
Relation to the ungated model: The gated recursive embedding generalizes the ungated version through fixed gate settings that recover the earlier activation.The ungated case uses zH = 1, zL = 0, zR = 0, zN = 0 and rL = rR = rN = 1.

B Gated recurrent event embedding

The event embedding applies a GRU to a pT-ordered sequence of jet four-momenta and jet embeddings, producing a final event representation for downstream tasks.

Event sequence: The event embedding is a GRU operating on a pT-ordered sequence of pairs containing each jet’s four-momentum and embedding.The sequence uses (v(tj), hjet^1(tj)) for j = 1, ..., M.
Event sequence: Each jet four-momentum is represented by the unprocessed features (φ, η, pT, m).The jet embedding accompanies these kinematic inputs in the recurrent sequence.
Event representation: The GRU recursively produces a final event output hevent at j = M.This output is the endpoint of the recurrent event-embedding construction.
Event representation: The event-embedding function has learned parameters for input and hidden-state transformations, with r denoting the embedding size.The listed parameters include Whx, Whh, Wrx, Wrh, Wzx, Wzh, and corresponding biases.
Experimental setting: The experiments consider sequences containing only the 1, 2, or 5 hardest jets, ordered by ascending pT.The number of jets included varies across the event-level experiments.

C Implementation details

The implementation enables efficient batch computation for dynamic tree topologies by synchronizing recursive operations across tree levels and events, then differentiates the full model chain automatically.

Batch computation: Tree-structured networks are difficult to batch because each example can have a different topology.This technical incompatibility can prevent efficient training on large datasets in standard implementations.
Batch computation: Activations at the same recursive-tree level can be computed together after deeper-level computations are complete.The same synchronization principle extends across multiple trees.
Batch computation: Synchronized computation across multiple trees enables batching of jet embeddings across many events.The implementation first traverses the recursion tree to organize the required operations.
Training: Automatic differentiation computes gradients through the recursive jet embeddings, event recurrence, and classifier on batches of events.The implementation uses native Python and Autograd, with parameters learned through the full model chain.