PaperlayerView exact revision on arXiv

Source-linked AI summary

Local Diagnostics of Learned Event Representations for Anomaly Detection at the HL-LHC

Donatella Genovese, Philip Ploner, Simone Scardapane, Stefano Giagu, Thea Klaeboe Aarrestad

arXiv:2610.13437v1cs.AIhep-exhep-phphysics.data-an

TL;DR

Anomaly detection in learned event representations offers a model-agnostic route to new-physics searches, but can lose the immediate physical interpretation of hand-crafted observables. This paper compares supervised and self-supervised embeddings and develops local diagnostics; supervised objectives generally perform better, and the diagnostics recover signal-defining physics while distinguishing local discriminants from score correlations.

  • Problem

    Learned low-dimensional representations reduce reliance on hand-crafted observables, but how to connect their latent geometry to physical observables for interpreting flagged events remains largely unexplored.

    Source: §1 · pp. 2–3

  • Method

    The study learns Transformer event embeddings with supervised or self-supervised objectives, scores latent-space reconstruction errors, and uses a GMM-based pipeline to localize flagged events and rank observables.

    Source: §1 · pp. 3–4

  • Results

    Across the configurations and benchmark signals considered, supervised objectives generally yield higher anomaly-detection performance than self-supervised counterparts, and their best configurations improve over a raw-feature autoencoder on all held-out processes.

    Source: §1 · p. 4, §5 · p. 17

  • Takeaways & Limitations

    The local rankings recover the defining physics of HH →4b and the Z′, while the Z′’s strongest local discriminant is not the observable most correlated with its anomaly score.

    Source: §1 · p. 4

    Supervised representations use simulation process labels unavailable event-by-event in collision data and may depend on accurate physics and detector modeling; the self-supervised objectives considered still need improvement.

    Source: §5 · p. 18

Abstract

from arXiv · show

Anomaly detection, the identification of collision events that deviate from Standard Model expectations, is one of the most promising model-agnostic strategies for new-physics searches at the Large Hadron Collider. Searching for rare events in learned low-dimensional representations removes the need to commit to hand-crafted observables a priori, but sacrifices the immediate physical interpretation those observables provide. For anomaly detection to enter a discovery pipeline, flagged events must therefore be diagnosed locally, both to distinguish potential new physics from mismodeling or detector effects and to identify observables for targeted follow-up searches. We evaluate supervised and self-supervised neural embedding strategies on COLLIDE-2V, a 750-million-event dataset simulated under realistic High-Luminosity LHC conditions, using the reconstruction error of an autoencoder trained in the learned latent space as the anomaly score. Across the configurations and benchmark signals considered, supervised objectives, which exploit simulation process labels during training, generally outperform their self-supervised counterparts and, in their best configurations, improve on an autoencoder trained directly on kinematic features, raising the signal efficiency at 10% background acceptance on all held-out processes. We further introduce a local diagnostic pipeline that fits a Gaussian Mixture Model to the SM latent space, profiles each component through high-level observables without process labels, localizes the flagged events, and ranks observables by their local discriminating power. The ranking recovers the defining kinematics of the held-out processes, while correlating the anomaly score with the same observables quantifies how closely the score tracks them, revealing cases where the most discriminating observable is not the one the score follows most closely.

1 Introduction

The paper studies anomaly detection in learned event representations, where replacing hand-crafted observables makes local physical interpretation a challenge. It compares representation-learning strategies and introduces diagnostics for interpreting flagged events.

1 Introduction

  • Learned low-dimensional representations can avoid reliance on hand-crafted observables, but connecting their geometry to physical observables remains largely unexplored.The paper frames local diagnosis as a way to identify nearby Standard Model populations, local discriminants, and observables associated with the anomaly score.Source: §1 · p. 3
  • The study uses COLLIDE-2V, a 750-million-event dataset designed for the HL-LHC with broad Standard Model process coverage and realistic detector conditions.The dataset supports representation learning at a scale relevant to future collider analyses.Source: §1 · p. 3
  • A Transformer encoder learns event representations with SimCLR or VICReg, using supervised or self-supervised objectives; a dominant-background autoencoder’s latent-space reconstruction error supplies the anomaly score.No signal is used during training, and evaluation processes enter only at inference.Source: §1 · p. 3
  • Supervised representations generally improve anomaly-detection performance over self-supervised counterparts and, in their best configurations, beat the raw-feature autoencoder on every held-out process.The comparison uses a shared Transformer backbone and signals unseen during training.Source: §1 · p. 4
  • The local diagnostic framework recovers the defining physics of HH →4b and Z′ →n(µµ), while showing that the Z′’s strongest local discriminant is not the observable most correlated with its anomaly score.It uses a Gaussian Mixture Model to localize flagged events and rank observables by local separation and score correlation.Source: §1 · p. 4

2 Related work

Prior work spans learned collider representations, anomaly-detection methods, and model interpretability. The paper situates its contribution in the relatively unexplored question of where flagged events lie in learned background spaces and how their scores relate to local kinematics.

2.1 Representation Learning in High-Energy Physics

  • HEP representation learning spans metric, masked, joint-predictive, contrastive, and foundation-model approaches; prior latent-space anomaly detection studies report sensitivity effects from objective and embedding dimensionality.Existing foundation-model work cited here focuses on jet-level classification or generation, while some anomaly-detection studies use learned embeddings at event or jet level.Source: §2.1 · pp. 4–5

2.2 Anomaly Detection for New Physics Searches

  • Collider anomaly detection includes overdensity methods and unsupervised outlier detection, with learned-latent approaches also explored using autoencoders and signal-sensitive embeddings.Prior comparisons report that embedding dimensionality and training objective affect downstream sensitivity.Source: §2.2 · pp. 5–6

2.3 Representation Analysis and Model Diagnostics in High-Energy Physics

  • Interpretability work includes post-hoc attribution and interpretable architectures, but prior latent-space studies rarely localize anomaly-flagged events within the learned background space or examine local score–kinematics correlations.Existing examples include Gaussian-mixture latent priors for jet-image anomaly detection and contrastive spaces relating latent distances to phenomenological similarities.Source: §2.3 · p. 6

3 Methods

The methods combine learned event embeddings, reconstruction-error anomaly scoring, and local latent-space diagnostics. The pipeline is evaluated on simulated HL-LHC events using held-out processes and multiple representation-learning configurations.

3 Methods

  • An SM-trained encoder maps events to a latent space; a dominant-background autoencoder scores anomalies, while a GMM characterizes the broader SM space for diagnostics.The GMM partitions the embeddings, profiles components without process labels, and supports localization and observable ranking.Source: §3 · p. 6, Figure 1 · p. 7, §3.3 · p. 11

3.1 Dataset

  • COLLIDE-2V contains 750 million simulated HL-LHC events under realistic detector conditions, with twelve SM training processes and two held-out evaluation processes.The Full-Reco view represents events with jets, electrons, muons, photons, and missing transverse energy.Source: §3.1 · pp. 6–7

3.2 Neural embeddings

  • Four SimCLR/VICReg configurations learn embeddings with supervised or self-supervised objectives, using physics-motivated or masking augmentations where applicable.Supervised SimCLR groups events sharing a process label as positives; supervised VICReg adds classification loss and drops its invariance term.Source: §3.2 · pp. 8–10

3.3 Anomaly Detection and Latent Space Diagnostics

  • A fully connected autoencoder trained on dominant-QCD embeddings uses per-event reconstruction MSE as its anomaly score, with the threshold calibrated for 10% QCD acceptance.A GMM fitted to SM training embeddings provides posterior responsibilities for assigning events to components.Source: §3.3 · pp. 10–11
  • Components are profiled without process labels, flagged events are localized by maximum posterior responsibility, and observables are ranked by local signal-background separation and score correlation.Known simulated signal identities validate the diagnostic steps retrospectively; application to collision data would compare flagged events with an appropriate local background reference.Source: §3.3 · p. 11

3.4 Experimental setup

  • A shared Transformer backbone is scanned over dmodel ∈ {32,64,128,256}, with five seeds and AD measured by AUROC and efficiency at 10% QCD acceptance.The GMM dimension, number of components, and covariance structure are selected as described in Section A.4.Source: §3.4 · pp. 11–12

4 Results

Across the evaluated configurations, supervised embeddings outperform self-supervised embeddings in linear-probe quality, and the best supervised configurations improve anomaly-detection efficiency over a raw-feature autoencoder on both held-out processes. Local diagnostics profile latent regions without process labels, locate flagged signals, and distinguish observables that separate signals from local backgrounds from those most associated with the anomaly score.

4.1 Comparison of embeddings

  • Among the evaluated objectives, each supervised model outperforms its self-supervised counterpart in linear-probe quality across the 12 SM processes.The comparison uses macro-averaged AUROC for the best configurations; VCReg, SupCon, and CE cluster above SimCLR and VICReg.Source: §4.1 · p. 12, Table 4 · p. 13

4.2 Anomaly detection performance

  • 97.2% signal efficiency versus 77.3% for Raw AE on HH →4b is reached by VCReg at dmodel = 256; on Z′ →n(µµ), all but VCReg at dmodel = 32 exceed the 96.8% baseline.The results are reported for the held-out processes; Table 5 covers AUROC and signal efficiency at 10% background acceptance across models and embedding dimensions.Source: Table 5 · p. 13, §4.2 · p. 13

4.3 Localizing and explaining flagged anomalies

  • Without process labels, the component profiles identify C4 as high-multiplicity and heavy-flavour, C2 as leptonic with hadronic activity and missing energy, and C3 as showing no marked shift.In C4, nb-tag shifts by +2.5 standard deviations, njets by +2.0, and HT by +1.4; in C2, nleptons shifts by +1.5, njets by +1.6, and MET by +1.1.Source: §4.3 · p. 14, Figure 2 · p. 14
  • AE-flagged HH →4b events primarily occupy C5 and C4, while 88% of flagged Z′ →n(µµ) events fall into C2.Fractions are normalized independently for each population, so the component occupancies indicate localization rather than signal purity.Source: §4.3 · pp. 14–15, Figure 3 · p. 15
  • Within the dominant components, local Wasserstein-1 rankings select nb-tag for HH →4b in C5 and nleptons for Z′ →n(µµ) in C2.In C5, the local SM background has a median of one b-jet, compared with two for the HH →4b signal.Source: §4.3 · p. 15, Table 6 · p. 16
  • For HH →4b in C5, the anomaly score correlates most strongly with nb-tag, but for Z′ →n(µµ) it correlates more with secondary kinematics such as HT and Mj j than with nleptons.The paper distinguishes the observables characterizing an anomalous population from those associated with variation in the anomaly-score response.Source: §4.3 · pp. 15–16

5 Conclusion

Across the configurations and benchmark signals considered, supervised objectives generally outperform self-supervised counterparts, while a GMM-based pipeline profiles local SM regions and anomaly-score associations. The diagnostics recover defining physics but show that the most locally discriminating observable need not be the one the score follows most closely.

5 Conclusion

  • Supervised objectives generally achieve higher linear-probe quality and anomaly-detection performance than their self-supervised counterparts across the configurations and benchmark signals considered.A GMM-based pipeline partitions the SM latent space and profiles regions through high-level observables without labels.Source: §5 · p. 17
  • The local diagnostics recover heavy-flavour content for HH →4b and lepton multiplicity for the Z′, but the Z′ score tracks secondary kinematics instead.The Z′ signal is routed to a well-defined leptonic region even though the score is driven by secondary kinematics rather than lepton multiplicity.Source: §5 · p. 17
  • Process labels are unavailable event by event in collision data, and supervised representations learned on simulation may depend on physics and detector modeling accuracy.Future work targets improved self-supervised representations, including JEPA or foundation-model-like approaches for unlabeled data.Source: §5 · pp. 17–18

A Experimental details

The appendix describes how reconstructed events are transformed into fixed-length inputs and augmented to train the representation models. Transformer inputs use object tokens and an event token, while the baseline uses the flat vector directly.

A.1 Preprocessing and event representation

  • Continuous features are transformed as appropriate, then robust-normalized using training-SM medians and interquartile ranges; object groups are zero-padded and retain their true multiplicities.Azimuth is encoded as (sinφ, cosφ), while φ, multiplicity scalars, and one-hot variables are not robust-normalized.Source: §A.1 · pp. 18–19
  • The Transformer projects each object slot into a group-specific token and adds an event token from multiplicity counts; baseline models use the flat vector directly.No positional embedding is used, preserving permutation invariance within each object group.Source: §A.1 · p. 19, Figure 6 · p. 20
  • The encoder trains on 1,000,000 events per SM class, with 100,000 per class reserved for validation and 100,000 for testing.Source: §A.1 · p. 19

A.2 Augmentation operators and strategies

  • Physics augmentations rotate all object azimuths by a shared angle and shift all real-object pseudorapidities by a shared displacement.The shared shifts preserve inter-object azimuthal and pseudorapidity differences.Source: §A.2 · pp. 19–20
  • Transverse-momentum smearing adds independent Gaussian noise to each real object's log(1+pT), then maps the result back to normalized feature space.MET uses the same prescription with its own interquartile range; the stated smearing scale is σpT = 0.1.Source: §A.2 · p. 20
  • Random masking can zero each object's features jointly or independently zero feature dimensions, with masking strategy and probability treated as hyperparameters.MET is treated as a single object under particle-level masking.Source: §A.2 · pp. 20–21

A.3 Ablation Study

The ablation study uses staged scans to select representation-model configurations, assessing quality with linear-probe AUROC on a computationally reduced dataset.

A.3 Ablation Study

  • Ablations use the reduced 12-class SM dataset for computational tractability and assess linear-probe AUROC averaged over five random seeds.Source: §A.3 · p. 21
  • The scan varies augmentation strategy, loss-function hyperparameters, and backbone architecture in three stages, fixing other parameters at defaults in each stage.Source: §A.3 · p. 21
  • Optuna tunes the remaining backbone and optimization hyperparameters for each model, with final configurations reported in Table 12.Source: §A.3 · p. 21

A.3.1 Augmentation Strategy

Augmentation strategy effects vary by objective: SupCon results are similar across strategies, SimCLR is sensitive to the masking choice, and VICReg performs lower but is most stable with Physics augmentation.

A.3.1 Augmentation Strategy

  • SupCon's three strategies span only ~0.004 AUROC; Physics augmentation gives the best mean, 0.8991±0.0006.Source: Table 7 · p. 21, §A.3.1 · p. 21
  • For SimCLR, Random particle masking yields 0.7712 ± 0.0003, while Random feature masking achieves the best mean, 0.8514 ± 0.0020.Source: Table 7 · p. 21, §A.3.1 · p. 21
  • VICReg scores approximately 0.72 across all three strategies; Physics augmentation is selected as most stable, at 0.7203 ± 0.0097.Source: Table 7 · p. 21, §A.3.1 · p. 21

A.3.2 Loss Function Hyperparameters

The ablations show distinct sensitivities across VCReg, VICReg, SupCon, and SimCLR: VCReg is robust once variance regularization is strong, while SimCLR benefits from a lower temperature.

A.3.2 Loss Function Hyperparameters

  • VCReg reaches 0.9030 ± 0.0007 mean AUROC at α = 25; β = 1 is optimal, and the total hyperparameter sensitivity is modest at ∼0.003.Increasing β consistently reduces performance, while α = 50 retains most of the gain but increases seed variance.Source: §A.3.2 · p. 22, Table 8 · p. 22
  • VICReg performance remains near ∼0.72 AUROC across invariance weights, with λ = 25 marginally best.The scan fixes µ = 25 and ν = 1 and varies λ from 0 to 50.Source: §A.3.2 · p. 22, Table 8 · p. 22
  • SupCon’s AUROC is nearly flat across temperatures, while increasing λcls monotonically improves it to 0.9019 ± 0.0001 at λcls = 0.50.The temperature scan spans 0.05–0.20; the classification-head-weight scan fixes τ = 0.10.Source: §A.3.2 · p. 22, Table 9 · p. 22, Table 10 · p. 23
  • SimCLR performs best at τ = 0.10, exceeding τ = 0.20 by 0.034 AUROC, consistent with harder negative mining being important for this objective.Source: §A.3.2 · p. 22, Table 9 · p. 22

A.3.3 Backbone Architecture

Embedding-width sensitivity differs by objective; the Gaussian-mixture diagnostic uses a PCA-reduced SM latent space and selects seven components by an occupancy criterion. Observable comparisons also have population and object-definition constraints.

A.3.3 Backbone Architecture

  • Larger dmodel benefits all four objectives, but self-supervised models improve and stabilize more as width grows, while supervised models vary only marginally.The results suggest label information makes representation quality less sensitive to embedding width than it is for purely self-supervised objectives.Source: §A.3.3 · p. 23

A.4 Gaussian Mixture Model

  • The GMM is fit to SM embeddings after PCA reduces dmodel = 256 to 64; K = 7 is selected because it is the finest partition satisfying the 1/(5K) occupancy floor.At K = 8, the least-populated component falls to 12.8% of its uniform share, below the floor; the occupancy transition is shown in Figure 7.Source: §A.4 · pp. 23–24, Figure 7 · p. 24

A.5 Observable definitions

  • Observable values are computed from raw vectorized fields before normalization, so they remain in physical units.Source: §A.5 · p. 24
  • Mjj and |Δηjj| comparisons use 81% of SM events and MT uses 28%; the six-bit BTag OR reaches a 0.24 mistag rate.The three observables are evaluated only where the required jets or lepton exist; the BTag bits are independent working-point draws, not nested cuts.Source: §A.5 · p. 25

B.1 Probe results

The per-process linear probes show that supervised models are close and generally lead, while performance varies substantially with the SM process.

B.1 Probe results

  • VCReg has the highest per-class AUROC in nine of twelve SM processes, with SupCon and CE leading in the other three; VICReg trails every class.SimCLR is near the raw-feature baseline, slightly above it for most classes but below it for Z →bb and Z →cc.Source: §B.1 · p. 26, Table 14 · p. 26
  • Supervised-model AUROC exceeds 0.97 for all three t t̄ channels but is lowest for W →q q̄, γ, QCD inclusive, and Z →q q̄.The passage characterizes the latter final states as the least distinctive in the set and reports heavy-flavour final states as easier than light-flavour ones at fixed production mode.Source: §B.1 · p. 26, Table 14 · p. 26

B.2 Sensitivity across held-out processes

Across the held-out processes, supervised objectives lead, and their advantage over the raw-feature autoencoder varies with signal characteristics and embedding width.

B.2 Sensitivity across held-out processes

  • SupCon and VCReg are the top two models on all three proxies; on two BSM signals both outperform Raw AE, with VCReg achieving a factor of 3.7 in signal efficiency on H →aa →4b.Source: §B.2 · p. 27
  • Both supervised models improve on proxies up to dmodel = 256, but BSM optima vary: VCReg peaks at 128, while SupCon peaks at 32 for H →aa →4τ and 256 for H →aa →4b.At dmodel = 32 on VBF H →b b̄, both fall below the Raw AE baseline.Source: §B.2 · p. 27
  • Self-supervised models remain below Raw AE on every proxy; on BSM signals, SimCLR only exceeds the baseline in AUROC for H →aa →4τ at dmodel = 64, while VICReg has the largest seed dispersions.VICReg falls below 0.5 AUROC on Z′ at dmodel = 128.Source: §B.2 · p. 27
  • At the 10% QCD-acceptance threshold, γ, Z →q q̄, and W →q q̄ pass at 10–13%, versus 71% for W →ℓν, 64% for Z →b b̄, and over 90% for each t t̄ channel.Source: §B.2 · p. 27

B.3 Additional interpretability results

The appendix expands the local diagnostics for AE-flagged HH →4b and Z′ →n(µµ) signals, showing observable distributions within the mixture components where each signal is localized. It also shows the full Standard Model observable shapes across the seven mixture components underlying the median-based summary.

B.3 Additional interpretability results

  • Table 16 reports AUROC and signal efficiency at 10% background acceptance, averaged over five seeds, for 20,000 events per process drawn from dedicated 50,000-event productions.The QCD reference matches that used for HH →4b, and the event counts match the proxy signals in Table 15.Source: Table 16 · p. 28
  • For both AE-flagged signals, the top-ranked observable visibly separates signal from local background, with a markedly wider gap for Z′ →n(µµ) than for the HH →4b proxy.Figures 8 and 9 show normalized distributions for components holding at least 5% of each flagged signal; the other observables overlap more closely.Source: §B.3 · p. 28, Figure 8 · p. 29, Figure 9 · p. 30
  • Figure 10 shows the full shapes of eight observables across the seven Standard Model mixture components, supplementing the single standardized median deviation reported for each region in Figure 2.The mixture is fitted on embeddings alone, without process labels, and regions are described by the physics of their assigned events.Source: §B.3 · p. 28, Figure 10 · p. 31
  • Table 17 reports VCReg with dmodel = 256 AUROC against QCD and each Standard Model process's acceptance at the threshold calibrated to accept 10% of QCD events.The QCD validation threshold is applied unchanged to 20,000 test events per process; results are mean ± std over five seeds.Source: Table 17 · p. 31

Related papers

Loading 2610.13437v1…