PaperlayerView exact revision on arXiv

Source-linked AI summary

JUNIPR: a Framework for Unsupervised Machine Learning in Particle Physics

Anders Andreassen, Ilya Feige, Christopher Frye, Matthew D. Schwartz

arXiv:1804.09720v1hep-phstat.ML

TL;DR

Particle-physics machine learning often targets discrimination, while learning the underlying physics from unlabeled data remains challenging. Junipr models jet probabilities using a neural network scaffolded by clustering trees, and the paper reports proof-of-concept discrimination and observable prediction. The authors also demonstrate a reweighting use case, subject to practical constraints when the distributions differ substantially.

  • Problem

    Basic neural networks can be limited by the observables they are given, motivating methods that can learn useful features from raw data.

    Source: §1 · p. 3

  • Method

    Junipr uses unsupervised learning to model collider-data probabilities, scaffolding its network around leading-order physics and binary jet-clustering trees.

    Source: §1 · pp. 5–6

  • Results

    A simple, untuned implementation showed strong discrimination power and effective predictivity for a broad class of observables in a simulated e+e− proof of concept.

    Source: §1 · p. 7

  • Takeaways & Limitations

    Junipr can reweight simulated events toward another sample, with a proof of concept demonstrated between Pythia tunes.

    Source: §6 · p. 38

    Practical reweighting requires the two sample distributions not to differ too greatly, or a few very large weights can dominate finite-statistics results.

    Source: §4.3 · p. 30

Abstract

from arXiv · show

In applications of machine learning to particle physics, a persistent challenge is how to go beyond discrimination to learn about the underlying physics. To this end, a powerful tool would be a framework for unsupervised learning, where the machine learns the intricate high-dimensional contours of the data upon which it is trained, without reference to pre-established labels. In order to approach such a complex task, an unsupervised network must be structured intelligently, based on a qualitative understanding of the data. In this paper, we scaffold the neural network's architecture around a leading-order model of the physics underlying the data. In addition to making unsupervised learning tractable, this design actually alleviates existing tensions between performance and interpretability. We call the framework JUNIPR: "Jets from UNsupervised Interpretable PRobabilistic models". In this approach, the set of particle momenta composing a jet are clustered into a binary tree that the neural network examines sequentially. Training is unsupervised and unrestricted: the network could decide that the data bears little correspondence to the chosen tree structure. However, when there is a correspondence, the network's output along the tree has a direct physical interpretation. JUNIPR models can perform discrimination tasks, through the statistically optimal likelihood-ratio test, and they permit visualizations of discrimination power at each branching in a jet's tree. Additionally, JUNIPR models provide a probability distribution from which events can be drawn, providing a data-driven Monte Carlo generator. As a third application, JUNIPR models can reweight events from one (e.g. simulated) data set to agree with distributions from another (e.g. experimental) data set.

1 Introduction

Junipr learns a probability distribution over collider data without labels, structuring its network around jet-clustering trees to make learning more tractable and interpretable. The paper presents proof-of-concept applications while noting that stronger testing is needed before judging performance on pressing physics problems.

1 Introduction

  • Junipr learns the probability distribution underlying particle-collider data through unsupervised learning.The framework is named “Jets from UNsupervised Interpretable PRobabilistic models.”Source: §1 · p. 5
  • Clustering trees scaffold both input processing and output probabilities, reducing complexity and organizing predictions into interpretable probabilities at individual tree nodes.The architecture is built around a leading-order description of the physics underlying the data.Source: §1 · p. 5
  • The tree can be built by any deterministic algorithm encoding momenta as a binary tree; real data has no labeled splitting history, and no single clustering algorithm is correct.For practical use, a fixed algorithm avoids marginalizing over all possible clustering histories, an extremely onerous computation.Source: §1 · p. 6
  • Junipr’s probability distributions support theoretically optimal likelihood-ratio discrimination, event sampling, and reweighting events from an imperfect simulator to agree with data.Sampling is possible, although generative models are better suited to that application.Source: §1 · p. 7
  • A simple, untuned implementation showed strong discrimination power and effective predictivity for a broad class of observables, but state-of-the-art performance remains untested.The authors describe the study as a proof of concept using simulated e+e− data.Source: §1 · p. 7

2 Unsupervised Learning in Jet Physics

Junipr models a jet’s probability density by decomposing its momenta into steps along a fixed clustering tree, without assuming a particular process generated the jet. Its neural implementation uses recurrent state and dense networks to predict tree termination, the next branching mother, and daughter momenta.

2.1 General Probabilistic Model

  • Junipr computes the probability density of a jet’s final-state momenta conditioned on the training-data selection criteria, without modeling the quantum-mechanical process that generated them.The model’s probability distribution is normalized over the physical phase space.Source: §2.1 · p. 8
  • A fixed clustering algorithm turns the jet into intermediate states, allowing its final-state probability distribution to factor into conditional probabilities at successive tree steps without loss of generality.Fixing the algorithm avoids marginalizing over all possible clustering trees, which would be intractable.Source: §2.1 · pp. 8, 10, Figure 2 · p. 9
  • At each step, Junipr predicts whether the tree ends, which candidate mother branches, and the daughter-momentum branching pattern, using a hidden state to carry global tree information.The branching function receives the participating momenta directly, so the hidden state need only encode global information.Source: §2.1 · p. 10

2.2 Neural Network Implementation

  • The implementation uses a recurrent neural network to update its hidden state from recently branched momenta, then dense networks output the full distributions for tree ending, mother selection, and branching.The product of those three probabilities across steps gives the jet probability density.Source: §2.2 · pp. 10–11, Figure 3 · p. 11, Figure 4 · p. 12
  • Branchings are parameterized by x = (z, θ, φ, δ); the implementation mainly discretizes these variables into a grid and predicts cell probabilities with a softmax.The discrete approach is faster; the paper also trains a continuous energy model to demonstrate that continuous branching variables are possible.Source: §2.2 · pp. 12–13, Figure 6 · p. 14

3 Training and Validation

Junipr is trained on clustered Pythia jets and validated by comparing its probabilistic predictions with observed outcomes. The section also describes discrete and continuous approaches to increasing branching-function resolution.

3.1 Training Data

  • The implementation is trained on 500k of 600k simulated quark jets, with 10k reserved for testing and 100k for validation.The jets are reclustered with C/A using Esub = 1 GeV and Rsub = 0.1; the approach is formally independent of the reclustering algorithm.Source: §3.1 · pp. 14–15

3.2 Approach to Training

  • Jets are represented as trees whose successive columns differ by a single 1→2 branching; Junipr uses only that branching’s momenta to update h(t), and can consider jets of arbitrary length.Maximizing the log likelihood allows each step t to be learned in parallel, providing computational efficiency and stability.Source: §3.1 · p. 15, §3.2 · p. 16

3.3 Validation of Model Components

  • Junipr’s predictions fit validation outcomes well for tree termination, mother selection, and branching, with early soft wide-angle and later harder, more-collinear branchings.The branching trends depend strongly on the chosen clustering algorithm and have no precise connection to the physical processes generating the data.Source: §3.3 · pp. 17–18, Figure 7 · p. 17, Figure 8 · p. 18
  • The validation plots show highest-energy candidates are most likely to branch next, while early and later branchings exhibit distinct soft-wide-angle and hard-collinear patterns.The latter patterns depend on the clustering algorithm and do not precisely identify the underlying physical processes.Source: §3.3 · pp. 17–18, Figure 8 · p. 18, Figure 9 · p. 19

3.4 Increasing the Branching Function Resolution

  • A discrete branching function with 10 bins per each of four coordinates has 10^4 outputs, while increasing bins directly makes dense-network computation scale quartically.Linear interpolation between the discrete bins removes most discretization effects on sampled observables at no extra training cost.Source: §3.4 · pp. 18–20
  • A continuous energy-model branching function permits arbitrarily high-resolution binning, but importance-sampling bias requires monitoring gradient error and increasing sample size above a 2% threshold.This mitigation slows training; the authors expect the faster 10-bin-per-coordinate discrete model to suffice in most cases.Source: §3.4 · pp. 20–21

4 Applications and Results

JUNIPR applies learned jet probability distributions to likelihood-ratio discrimination, sampling, and event reweighting. The section demonstrates separation of boosted-Z and quark jets, approximate agreement for sampled observables, and improved agreement after reweighting, while noting practical limitations of sampling and reweighting.

4.1 Likelihood Ratio Discrimination

  • Likelihood-ratio discrimination uses all jet momenta and requires N probabilistic models for N jet types; training samples need not be pure.The authors state that this approach is theoretically optimal and describe mixed-process samples as feasible for discrimination.Source: §4.1 · p. 23
  • Junipr’s likelihood-ratio discrimination is clearly superior to a 2D discriminant using 2-subjettiness and constituent multiplicity in the tight-mass-window Z/quark study.The comparison uses jets with masses between 90.7 and 91.7 GeV.Source: §4.1 · p. 23, Figure 12 · p. 24
  • Branching-level likelihood-ratio visualizations show discrimination at most branchings; later splittings can provide power comparable to the first.For the quark-jet example, the first-branching ratio is 10^-3.7; the authors suggest several possible contributors to later-branching separation.Source: §4.1 · pp. 23–24, Figure 13 · p. 25

4.2 Generation from JUNIPR

  • Junipr-sampled jet-mass and constituent-multiplicity distributions match the held-out Pythia validation distributions, with reasonable agreement in their joint distribution.The results use 100k sampled jets and 100k validation jets.Source: §4.2 · p. 27, Figure 15 · p. 27, Figure 16 · p. 27
  • Direct sampling can amplify early-step errors, and sampled 2-subjettiness does not match held-out Pythia data perfectly.The authors attribute feedback sensitivity to later-step distributions depending on earlier draws; they state the mismatch is addressed by reweighting.Source: §4.2 · p. 28, Figure 17 · p. 28

4.3 Reweighting Monte Carlo Events

  • Reweighting αs = 0.11 jets by Pαs=0.1365/Pαs=0.11 brings 2- and 3-subjettiness ratios and jet shape into relatively good agreement with the αs = 0.1365 distribution.The unreweighted distributions differ clearly from the target distribution.Source: §4.3 · p. 30
  • Reweighting is practical only when source and target distributions overlap and weights stay near unity; large weights can cause choppy distributions and large statistical uncertainties.In the toy study, roughly 0.1% of jets with Pαs=0.1365/Pαs=0.11 > 100 were discarded.Source: §4.3 · pp. 29–30

5 Factorization and JUNIPR

JUNIPR’s global hidden state carries information needed to model jet evolution, while its formal independence from the clustering algorithm does not guarantee equal performance across implementations. Tests with printer and anti-kt trees show that the model can learn non-QCD-like structures, with observable agreement depending on the clustering choice.

5.1 The Encoding of Global Information

  • Removing h(t) from the branching function produces intermediate-angle distributions that disagree with Pythia, while the baseline global model agrees essentially perfectly.The disagreement would lead to an incorrect probability distribution over final-state momenta.Source: §5.1 · p. 32, Figure 19 · p. 33

5.2 Clustering Algorithm Independence

  • Although JUNIPR is formally clustering-algorithm independent, its implementation may depend weakly on the algorithm because the architecture learns QCD-like trees most readily.The printer test shows that it can learn a non-QCD-like tree, but with less accurate sampled observables than with C/A clustering.Source: §5.2 · pp. 32–34
  • The printer implementation learns the printer emission structure, but its 2-subjettiness ratio agreement is worse than the C/A model’s.Its sampled jet mass–multiplicity distribution matches Pythia well, while the 2-subjettiness ratio match is somewhat worse.Source: §5.2 · pp. 33–35, Figure 22 · p. 35

5.3 Anti-kt Shower Generator

  • For anti-kt jets, h(t) preserves correlated emission directions that disappear when the branching function lacks the global representation.The sampled mass–multiplicity distribution is consistent with Pythia; the 2-subjettiness distribution has mild disagreement, on par with the C/A model.Source: §5.3 · p. 36, Figure 23 · p. 36, Figure 24 · p. 37

6 Conclusions and Outlook

JUNIPR models jet probabilities with a tree-structured neural network, enabling interpretable likelihood-ratio discrimination and event reweighting. The paper presents these applications as proof-of-concept tests and points to broader event-level and heavy-ion applications as future directions.

6 Conclusions and Outlook

  • JUNIPR computes jet probability densities with a recurrent network that follows a binary clustering tree and decomposes the density into contributions from each branching.The chosen clustering algorithm is not essential in principle, but an algorithm corresponding to the physics can improve performance and interpretability.Source: §6 · p. 37
  • Separately trained JUNIPR models discriminate boosted Z bosons from quark jets in a tight mass window, with discrimination power visualizable across each jet’s clustering tree.The authors suggest such studies might inspire new calculable observables.Source: §6 · p. 38
  • JUNIPR reweights Monte Carlo events to improve agreement with another sample, demonstrated by reweighting jets from one Pythia tune to match another.The authors propose small, granular reweighting as a possible alternative to isolating and fixing faulty generator components.Source: §6 · p. 38
  • The paper applies JUNIPR to probabilities for jet momenta and proposes extending it to whole events, particle-identification tags, detector data, and heavy-ion collisions.For heavy-ion collisions, comparing data and simulation probabilities could inform simulation improvements or understanding of the underlying physics.Source: §6 · p. 38
Loading 1804.09720v1…