Learning to Trigger: Reinforcement Learning at the Large Hadron Collider

Motivation

• Online trigger menus are costly, hand-tuned, and vulnerable to pileup and background-composition shifts, motivating autonomous threshold adaptation under bandwidth and latency constraints [5].The paper targets the largely unexplored problem of continuously retuning a fixed threshold menu online while preserving rare-signal sensitivity.Source: §1 · p. 1

Problem formulation

• The framework formulates adaptive thresholding as sequential decision-making, using streaming rates and distributional summaries to optimize long-horizon physics utility under hard background-rate constraints [12] [14].This enables policies to model nonlinear and delayed effects and allocate rate budgets more effectively than manual retuning in non-stationary environments.Source: §1 · p. 2

Experimental scope

• The study evaluates two trigger classes trained on simulation and deployed on real CMS collision data, asking whether RL improves signal efficiency while maintaining comparable or better rate stability than control baselines.It also contrasts simulation-to-data deployment with test-time training.Source: §1 · p. 2

Evaluation framework

• The evaluation framework measures feasibility with InBand fraction and performance with signal efficiency, while showing that a simple DQN can adapt trigger thresholds in real time.InBand fraction is the proportion of events whose trigger rate remains within the target bandwidth tolerance.Source: §1 · p. 2

State representation

• A length-K event-sequence representation with physics-informed features preserves distributional context that scalar-error PID control [5] cannot, and a recurrent encoder compresses it into a fixed-size state.The representation distinguishes global luminosity shifts from local score-distribution changes near the threshold.Source: §1 · p. 2

Policy optimization

• The paper adapts GFPO [17], a GRPO variant [18], and introduces GFPO-F and GFPO-FR to filter rollouts by accepted background rate and signal efficiency.Both variants improve stability without degrading efficiency on simulated events and real CMS collision data, transfer with negligible performance loss, avoid deployment fine-tuning, and tolerate large anomaly-score-scale variations.Source: §1 · p. 2

Beyond the LHC

• Combining sequence-based neural states with GFPO-F and GFPO-FR improves online anomaly-detection benchmarks under streaming distribution shifts, extending the framework beyond the LHC.This suggests applicability beyond particle physics while retaining the same adaptive-control principles.Source: §1 · p. 2

Real-data contribution

• The work demonstrates the first end-to-end RL-trained policy using real CMS collision data, learning threshold updates from streaming experience rather than minimizing a global cost function [5].The demonstration uses CMS Runs 283408 and 283876 and builds on the self-driving trigger framework of Emami et al. [5].Source: §1 · p. 2

Reinforcement learning for scientific decision making

• LHC trigger control combines signal-efficiency maximization with strict background-rate constraints under non-stationary pileup and detector drift, unlike prior scientific RL applications.Related applications include self-driving labs, accelerator tuning, and tokamak regulation.Source: §2 · p. 2

Anomaly detection under drift

• The work adapts autoencoder threshold control to satisfy target rates under drift, contrasting with ADT’s binary thresholds and DSPOT’s rate-adaptation baseline.DSPOT’s false-positive level q maps to the target rate r∗, but it does not optimize signal efficiency.Source: §2 · p. 3

Selective updates

• The method adapts GFPO’s selective updates to streaming trigger control and introduces GFPO-F and GFPO-FR, which filter candidates by rate feasibility and signal utility.GFPO-F retains the top-K candidates with smallest rate deviation; GFPO-FR first selects candidates within an expanded tolerance band, then ranks them by signal utility.Source: §2 · p. 3

3 RL formulation in Adaptive Thresholding

• The controller updates one scalar threshold for either HT or AD to keep background acceptance near a target while maximizing held-out signal efficiency.HT uses total hadronic activity, whereas AD uses autoencoder reconstruction loss.Source: §3 · p. 3
• The MDP state summarizes recent events and controller context, actions are discrete threshold increments, and reward combines rate tracking, t̄t and h →4b efficiency [5], and move-stability penalties [28].The transition reflects drifting streaming data, so the agent does not assume access to environment dynamics.Source: §3 · p. 3

3.1 Warmup: DQN for single trigger

• Operational constraints motivate a safety shield that limits threshold changes to configurations respecting bandwidth, computing, and detector-safety requirements [33] [35] [36].The stated ATLAS Level-1 rate limit is approximately 100 kHz.Source: §3.1 · p. 4
• In single-trigger MC results, GFPO-FR reports 0.979 and GFPO-F reports 1.000, evaluated on the held-out remaining 20% after training on the first 80% of MC chunks.Source: Table 1 · p. 5
• DQN improves InBand over PID from 0.52 to 1.00 for HT and from 0.73 to 0.99 for AD on held-out MC, while frozen DQN-F degrades to 0.74 and 0.85.This isolates continued online adaptation as the source of DQN’s gain rather than a richer policy class.Source: §3.1 · p. 5

3.2 Critic Free Policy Optimization

• GRPO removes the learned critic and uses within-group reward normalization, comparing candidate threshold updates under the current context to reduce sensitivity to reward-scale and regime shifts.Each candidate is evaluated on a recent-event window without replay, inference-time generation, or an auxiliary verifier model.Source: §3.2 · pp. 6–7
• Setting γ = 0 is exact for GRPO because actions affect the next batch, uncontrolled conditions drift exogenously, and smoothness is enforced directly through the immediate move penalty.The paper reports that DQN at γ = 0 matches its γ = 0.95 counterpart.Source: §3.2 · p. 7
• L-GRPO collapses to AD InBand 0.541 on CMS data, while CPO’s recovery fallback activates on roughly half the AD chunks because its parameter-space guarantee does not ensure feasible proposed candidates.When all sampled candidates violate the rate budget, group normalization can reinforce infeasible actions; feasibility must therefore be enforced before advantage normalization.Source: §3.2 · pp. 7–8

4 Group-Filtered Policy Optimization

• GFPO adapts Group-Filtered Policy Optimization [17] to rate-feasible streaming trigger control, addressing groups with no feasible actions under distribution shift.The method chooses which candidates to keep and how to handle too few feasible candidates before advantage normalization.Source: §4 · p. 8
• When no feasible candidate exists, GFPO shields the update, while partial feasible sets are padded with the closest-to-feasible out-of-band candidates.GFPO performs GRPO-style normalization over a fixed-size kept set and restricts the policy update to that set.Source: §4 · pp. 8–9
• GFPO-F keeps candidates with the smallest absolute rate error, whereas GFPO-FR keeps feasible candidates first and then ranks them by signal efficiency.GFPO-F favors tighter rate control; GFPO-FR seeks higher signal efficiency within the safe region.Source: §4 · pp. 8–9

Diagnostics

• 30.8% of HT steps had zero feasible GRPO candidates, with an average feasible fraction of 0.58, causing vanilla GRPO to reinforce an out-of-band action.Figure 2 reports the failure on the latter 20% of Monte Carlo data with G = 16, averaged across 3 seeds.Source: Figure 2 · p. 9, §4 · p. 9
• GFPO-F achieved a 79.40% pure-feasible update rate, while GFPO-FR had 8.42% zero-feasibility groups versus 9.26% for GFPO-F and traded rate accuracy for signal efficiency.On HT, GFPO-F had MAE 0.004 versus 0.016 for GFPO-FR; on AD, in-band rates were 100% versus 97.9%.Source: §4 · pp. 9–10, Figure 3 · p. 9

5 Beyond HEP: Anomaly Detection Benchmarks

• The failure mode is domain-general when scalar-score thresholding, drifting backgrounds, and hard positive-rate budgets coexist, because policy drift can leave the feasible manifold.The LHC’s constrained bandwidth exemplifies this structure, but none of the three conditions is physics-specific.Source: §5 · p. 10

5.1 Experimental Setup

• The benchmarks use a shared streaming protocol with autoencoder scores, chunkwise threshold updates, a pre-registered false-positive target and tolerance, and domain-specific context variables.UNSW-NB15 preserves rate regulation with Period 1 training and frozen Period 2 deployment; NAB removes rate regulation and evaluates precision, recall, and F1 online.Source: §5.1 · pp. 10–11

5.2 Experimental Results

• UNSW-NB15 transfers both predictions: competing methods collapse to 0% in-band FAR, while GFPO-F and GFPO-FR reach nearly 4% and 4.9%.GFPO-F/FR achieve 1.5–2× the best classical baseline and dominate signal TPR on the hard Backdoors class.Source: §5.2 · p. 11, Figure 5 · p. 11
• NAB results show GFPO-F and GFPO-FR achieve the highest F1, 0.215 and 0.216, approximately 3% above oracle Constant-opt at 0.184.DQN, GRPO, and L-GRPO underperform Constant-opt despite optimizing the same reward, attributing gains to feasibility-filtered group optimization rather than RL alone.Source: §5.2 · p. 11, Figure 5 · p. 11

• Feasibility-first GFPO improves rate stability and signal efficiency across CMS trigger studies, transfers from simulation to CMS data, and shows the same failure mode and remedy on UNSW-NB15 and NAB.DQN improves over constant menus and PID, but non-stationary backgrounds and sharp local rate sensitivity undermine DQN and Lagrangian-GRPO.Source: §6 · p. 12
• A single scalar threshold couples trigger efficiencies: t̄t exceeds 95% in-band efficiency, whereas h →4b reaches only 29-36% under the same cut.The reward-mixing parameter α provides only partial control over this coupling.Source: §6 · p. 12
• The evaluation is limited to CMS Open Data, using Run 283408 for deployment and Run 283876 for autoencoder training, without testing generalization to ATLAS or LHCb [52].Source: §6 · p. 12
• Multi-trigger coordination under a shared rate budget is the natural extension, with feasibility-first filtering expected to become more important as the joint feasibility region tightens [53].Because PID treats each trigger as an independent scalar loop, the single-trigger results are described as a conservative estimate of the gap at trigger-menu scale.Source: §6 · p. 12

A Extended Related Work

• Unlike scientific RL targeting physical states, this work maintains background acceptance near a fixed target while maximizing signal efficiency under streaming, non-stationary dynamics and single-pass latency constraints.Search-heavy methods such as MCTS are therefore unsuitable for deployment, although logged or streaming experience can support off-policy training.Source: A Extended Related Work · p. 19
• Lagrangian-GRPO adapts dual variables but cannot resolve structural sampling failure under distribution shift, motivating feasibility filtering through GFPO-F and GFPO-FR.The comparison appears as a direct baseline in Section 3.2 and Appendix E.1.3.Source: A Extended Related Work · p. 19
• For anomaly detection, the method adapts an autoencoder threshold online, unlike active anomaly detection’s query policy and ADT’s assumptions of normalized scores with known global extrema.Background-only training, unbounded score scale, and non-stationary shifts make fixed thresholds and ADT’s original deployment assumptions unsuitable.Source: A Extended Related Work · p. 19
• Anomaly Transformer is excluded because independent LHC collisions lack the regular sampling and autocorrelation needed for its Association Discrepancy criterion, while the task regulates rates over windows.Threshold-based RL methods are therefore the relevant comparison class rather than point-wise anomaly classification methods.Source: A Extended Related Work · p. 19
• DSPOT provides a non-learning response to score drift by fitting generalized Pareto tails, but it tracks quantiles without closing the measured-rate loop or optimizing signal efficiency.Used as an adaptive trigger cut, DSPOT is competitive with PID in some regimes but falls short of RL controllers in rate stability and signal efficiency under pileup drift.Source: A Extended Related Work · p. 19
• GFPO-F filters candidate threshold updates by rate deviation, while GFPO-FR prioritizes feasible candidates and then ranks them by signal utility, making filtering a soft rate constraint.GFPO-F samples G candidates and retains the top-K by smallest |r −r∗|; GFPO-FR pads with closest-to-feasible candidates when necessary.Source: A Extended Related Work · p. 20
• The policy observes sliding windows of event summaries plus threshold-relative and control-context features, exposing score mass near the cut to anticipate pileup-driven shifts without a closed-form dynamics model.This contrasts with physics-informed forecasting approaches that embed mechanistic structure to predict system states under known or approximate dynamical laws.Source: A Extended Related Work · p. 20
• GRPO’s within-chunk group baseline suits drifting trigger distributions better than PPO’s stale critic, yielding bias O(G−1) versus PPO’s drift-dependent Θ(δk/((1 −γ)(1 −γλ))).GRPO recomputes its baseline from G on-policy rollouts at every update, aligning with the trigger loop’s multiple candidate threshold evaluations.Source: A Extended Related Work · p. 20

B Theoretical Justification

• 48% of the empirical drift evidence shows monotonic background-distribution shifts in both Monte Carlo and CMS Open Data, matching the affine shifts assumed by the theory.The reported HT and AD-score distributions exhibit monotonic location-and-scale changes across a run.Source: B Theoretical Justification · p. 21
• PPO’s critic mismatch is O(δk/(1 −γ)) under drift, and bootstrapping plus multi-epoch trajectory reuse compounds the resulting advantage bias across chunks.Under monotone drift, the critic mismatch has a matching lower bound proportional to δk/(1 −γ).Source: B Theoretical Justification · pp. 22, 24
• GRPO’s within-chunk group normalization is asymptotically conditionally unbiased, with empirical bias and conditional mean-squared error of order O(G^-1).The estimator uses only chunk-k information, avoiding dependence on earlier chunks.Source: B Theoretical Justification · pp. 22–23

C GFPO for Streaming Trigger Control: Pseudocode

• The algorithm trains the policy over Monte Carlo chunks and supports frozen deployment or one-update-per-chunk test-time training on CMS data.Threshold changes are clipped to a maximum magnitude during streaming deployment.Source: C GFPO for Streaming Trigger Control:… · p. 26
• GFPO-STEP samples grouped actions, computes rewards and background-rate errors, retains variant-specific rate-feasible actions, and updates the policy from the kept set.If no actions are retained, the threshold change is sampled from the policy; otherwise it is returned from the kept set.Source: C GFPO for Streaming Trigger Control:… · p. 26

D.1 Dataset Summary

• Background HT and AD scores drift downward over each run; CMS Run 283408 declines mildly with a heavy upper tail and remains globally higher-scoring than MC.This creates domain shift in addition to within-run drift.Source: §D.1 · p. 27, Figure 6 · p. 28, Figure 7 · p. 28
• The autoencoder uses latent dimension d = 2, trains on CMS Run 283876 for real data, and evaluates on CMS Run 283408 containing 1.99M background events.For MC, training uses one background sample and evaluation uses an independent background sample.Source: §D.1 · p. 27
• t¯t is well separated from MinBias background, whereas h →4b overlaps heavily and remains below ∼35% HT signal efficiency across methods, reflecting an intrinsic oracle-limited ceiling.Every HT method reaches > 97% on t¯t except Constant in Table 2, while h →4b tracks the oracle bounds.Source: §D.1 · p. 27, Table 4 · p. 28
• RL evaluation trains on the first 80% of MC chunks and reports the held-out final 20%, while oracle figures use the full sample without learning.The first 10 chunks are excluded, and reporting begins after 500K MC or 200K CMS events.Source: §D.1 · pp. 27–28
• GFPO-FR achieves the smallest oracle signal-efficiency gap on MC, while both GFPO variants exceed the CMS h →4b oracle in-band efficiency: 42.533 and 44.477 versus 41.028.The CMS excess occurs because in-band chunks concentrate in the run’s high-efficiency tail.Source: §D.1 · pp. 27, 29, Table 6 · p. 29

D.2 Training setup

• Training performs one chronological pass over 148 chunks, using 50,000-event chunks and 5,000-event micro-steps; policies use Adam and results average three random seeds.GFPO-F and GFPO-FR use G = 64 candidates per micro-step and K = 16 kept; GRPO and L-GRPO use G = 16.Source: §D.2 · p. 29

D.3 Models and Hyperparameters

• GFPO-F and GFPO-FR remain tightly clustered at high in-band rate and signal efficiency across 25 reward configurations, unlike baselines’ pronounced objective trade-off.The selected reward weights are λ1 = 0.25 and λ2 = 1.00, with α = 0.7.Source: §D.3 · p. 29, §D.2 · p. 29

D.4 Standard anomaly detection metrics for LHC

• GFPO-FR ranks top-two on 9/14 MC HT metrics and 7/14 MC AD metrics, while achieving the highest InBand rate among methods.The benchmark reports superiority or parity with all baselines, alongside the highest operational InBand rate.Source: §D.4 · p. 29, Table 7 · p. 34
• On CMS Run 283408, GFPO-FR ranks top-two on 7/14 HT metrics and 6/14 AD metrics, while GFPO-F and GFPO-FR maintain 95% and 98% InBand rates.These conventional metrics reflect signal efficiency without background-rate constraints, unlike the operational InBand comparison.Source: Table 8 · p. 35, §D.4 · p. 36
• For zero-shot CMS transfer, GFPO-FR and GFPO-F maintain 95% and 99% HT InBand rates versus 83.8% for PPO and 43.2% for DSPOT and PID.Baseline methods can lead on top-two conventional metrics, but those metrics do not impose background-rate constraints.Source: Table 9 · p. 36, §D.4 · p. 36

E.1.1 ADT

• ADT is adapted from fixed-window normal/abnormal classification to event-level regulation of the per-event accept rate around target r*_B within tolerance τ.The adaptation replaces label-dependent classification feedback with rate-regulation state, maps episodes to chunks, discretizes continuous trigger actions, and adds a safety shield.Source: §E.1.1 · p. 37
• The ADT adaptation adds a safety shield because excessive threshold updates can violate LHC rate budgets and cause DAQ deadtime.Vanilla ADT has no such shield; the implementation retains its l = 10 micro-step update cadence while other methods update every 5K-event window.Source: §E.1.1 · p. 37

E.1.2 DSPOT

• DSPOT fits a Generalized Pareto Distribution to excesses above an intermediate threshold and uses its target-risk quantile as the next-chunk trigger threshold, without rewards or policy updates.The reference window matches the chunk size, with q = r*_B = 0.25% and the intermediate threshold initialized at the 98th calibration percentile.Source: §E.1.2 · p. 37
• DSPOT reaches 45.2% AD and 39.8% HT InBand fractions in MC, with 93.7% and 98.1% InBand t¯t efficiencies, respectively.The InBand gap to PID reflects DSPOT’s 50-chunk GPD calibration burn-in while PID adapts from chunk one.Source: §E.1.2 · p. 38
• DSPOT lacks two-sided tolerance enforcement and signal-efficiency ranking, yielding 13.5% AD and 17.6% HT InBand in test-time training and 41.9% AD and 43.2% HT after sim-to-real transfer.Its tail estimator is less suited to HT’s full-distribution pileup shift than AD’s tail shift, and abrupt CMS excursions can exceed the MC-calibrated GPD support.Source: §E.1.2 · pp. 37–38

E.1.3 L-GRPO

• L-GRPO replaces fixed λ1 with a projected dual variable λt that increases during rate violations and decreases when the constraint is satisfied.The dual is updated once per chunk on the same timescale as threshold updates and is clipped to [0.05, 0.95].Source: §E.1.3 · pp. 38–39
• ≈61% of CMS AD steps remain zero-feasible for both GRPO and L-GRPO, while L-GRPO’s in-band rate oscillates around 54% without recovery.The dual changes reward weights but cannot alter the candidate proposal distribution; when no feasible candidate exists, it amplifies the least-infeasible update.Source: §E.1.3 · p. 40
• On CMS, L-GRPO’s rate trajectory stays systematically below the tolerance band, unlike GRPO and both GFPO variants, isolating reward formulation as the failure source.This behavior motivates GFPO’s feasibility-first candidate selection under distribution shift.Source: §E.1.3 · p. 40, Figure 15 · p. 41

E.1.4 CPO

• CPO moves rate tracking from the reward into a non-negative cost constraint, enforcing an expected per-step violation budget with d = 1.The cost is zero at the target, reaches 1 at the tolerance-band edge, and remains continuous beyond violations.Source: §E.1.4 · pp. 40, 42
• GFPO-F and GFPO-FR track the CMS AD background rate within the tolerance band for most of deployment, whereas GRPO and L-GRPO remain near or below its lower edge.The comparison uses MC-trained, frozen policies over a 73-chunk CMS deployment window.Source: Figure 15 · p. 41
• CPO samples G = 16 candidate cuts, uses group-relative cost advantages, and updates policy parameters through a linearized trust-region program.The update replaces separate value critics and uses conjugate-gradient Fisher solves, closed-form dual variables, and line search safeguards.Source: §E.1.4 · p. 42

E.2 GRPO Failure Mode

• 34–36% of AD training steps remain zero-feasible across G ∈ {4, 8, …, 256}, while HT stays above 24% even at G = 256.The persistent fractions indicate zero feasibility is a property of the policy distribution under the constraint, not merely sampling variance.Source: §E.2 · p. 42

E.3 Sequential Network Architectures

• The trigger-control problem is partially observed and non-stationary because recent event windows and threshold-to-rate mappings drift with pileup, detector conditions, and score distributions.This motivates encoding temporal context rather than relying on instantaneous rate error alone.Source: §E.3 · pp. 42, 44
• Group-relative methods can lack any feasible rollout on substantial training fractions, motivating feasibility-filtered updates rather than relying on larger group sizes alone.For AD with G = 16, 32.08% of steps have zero feasible candidates; even G = 256 leaves 24–34% zero-feasibility steps.Source: Figure 17 · p. 43, Figure 16 · p. 43
• Each state is a fixed-length K-event sequence with F features combining event physics, threshold geometry, pileup context, and chunk-level control variables.The implementation uses F = 22 features per timestep, including rate error, drift, last action, and feasibility.Source: §E.3 · p. 44
• All five RL agents share the same single-layer recurrent encoder and linear projection head, trained on MC and deployed on CMS Run 283408 for both triggers.The recurrent-cell ablation compares LSTM, GRU, vanilla tanh RNN, and vanilla ReLU RNN under fixed λ1 = 0.25 and λ2 = 1.00.Source: §E.3 · p. 44
• GRU achieves the highest HT InBand rate at 0.849 versus 0.789 for vanilla RNN, while signal efficiencies vary by under 0.1% across recurrent architectures.Across the five-agent averages, tt̄ overall spans 97.57–97.64% and h →4b overall spans 33.29–33.38%.Source: §E.3 · p. 44, Table 10 · p. 45
• Normalized thresholds, recency indices, pileup summaries, near-cut occupancy, EMA rate error, sensitivity probes, and look-ahead ratios provide anticipatory information about rate drift and threshold response.These features expose distribution shifts, accumulated deviations, local trigger-curve slope, and the shape of the cut neighborhood reachable in one or two actions.Source: §E.3 · pp. 45–46
• The look-ahead features evaluate rates at the next two action-grid points and use survival ratios to distinguish thin from fat tails near the operating threshold.They use ∆ = 1 GeV for HT and ∆ = 0.5 for AD and drift across the run.Source: §E.3 · p. 46

F.1 Additional DQN results

• DQN tracks the target-rate band more consistently than Constant and PID controllers for both HT and AD, while DQN-F further suppresses swings and recovers faster.Constant thresholds drift downward under pileup changes, whereas PID oscillates around the tolerance band; the same qualitative pattern appears for both observables.Source: §F.1 · p. 49, Figure 25 · p. 49

F.2 GRPO MC training

• GRPO, GFPO-F, and GFPO-FR concentrate near the 100 kHz target on MC, while Constant decays with pileup drift and PID frequently leaves the tolerance band.GFPO-F and GFPO-FR have the highest in-band rate among established baselines; GFPO-FR operates at 106.1 kHz compared with GFPO at 100.0 kHz.Source: §F.2 · p. 49, Figure 26 · p. 49, Figure 27 · p. 50
• RL-based agents achieve higher signal efficiency than PID and Constant for both t¯t and h →4b signals over the held-out MC run.The advantage is largest for the rarer h →4b signal, while t¯t efficiency on HT is already near saturation.Source: §F.2 · pp. 49–50, Figure 28 · p. 51
• GFPO-FR reaches roughly 48% HT and 40% AD h →4b efficiency in late MC chunks, versus roughly 24% for Constant, with GFPO-FR ≳ GFPO-F ≈ GRPO > PID > Constant throughout.Efficiency rises gradually as adaptive policies re-center thresholds to track the target background rate rather than applying a one-time correction.Source: §F.2 · p. 50

F.3 Sim-to-Real Transfer and Test Time Adaptation

• Test-time adaptation on CMS Run 283408 produces nearly identical results to test-time training, supporting MC training with deployment-time adaptation as a practical alternative to online weight updates.The adaptation study uses models trained on MC and deployed on real CMS collision data.Source: §F.3 · p. 50

F.4 Testing Time Training on Real CMS Data

• Real-data evaluation is necessary because detector streams can differ from simulation in feature distributions, background composition, detector response, and running conditions, especially for anomaly-detection triggers.The study therefore tests whether online threshold adaptation remains effective beyond controlled MC assumptions.Source: §F.4 · p. 50
• GFPO-F and GFPO-FR provide the best overall stability–efficiency tradeoff on real CMS data across both trigger settings.They are more consistently strong than PID, DQN, PPO, ADT, and GRPO across control and physics metrics, although PID remains competitive on isolated metrics.Source: §F.4 · p. 50
• Frozen MC-trained rollouts and test-time training on CMS differ by less than 2 percentage points in InBand rate and signal efficiency across all methods and triggers.This similarity shows that direct learning on real collision data offers little additional performance over the frozen transfer setup.Source: §F.4 · p. 50, Figure 29 · p. 52
• Online per-chunk threshold adaptation compensates for MC-to-CMS distribution shift without gradient-based fine-tuning at deployment.The policy generalizes from MC, while selecting ∆c at each micro-step keeps the background rate within the tolerance band.Source: §F.4 · pp. 50, 52

Degeneracy

• CPO entries agree to floating-point precision with zero per-seed standard deviation because the executed CMS trajectory is identical across three seeds despite differing policies.The executed action sequence is bit-identical across seeds, and the resulting metrics are therefore identical.Source: §F.5 · pp. 52–53

Selection mechanism

• At each micro-step, CPO samples 16 candidates from a five-action set, then executes the highest-reward feasible candidate or the minimum-cost candidate otherwise.For AD, actions are {-3, -1.5, 0, 1.5, 3}; for HT, they are {-2, -1, 0, 1, 2}, with feasibility requiring c_k ≤ 10^-9.Source: §F.5 · p. 52
• The policy only determines sampled candidates; because the operative optimizer effectively matches the full action set, the executed action depends only on data and the current cut.The omitted operative action has probability at most (4/5)^16 ≈ 0.03 under a uniform policy and much less after policy concentration.Source: §F.5 · p. 53

Test-time training

• Test-time training leaves the frozen rollout and all derived metrics unchanged because recovery and ordinary policy updates do not alter the candidate rewards or costs consulted by selection.The recovery branch activates on roughly half of CMS update chunks when the budget is violated, but its cost-gradient update remains screened from execution.Source: §F.5 · p. 53

G Ablation study on noisy anomaly scores

• Latent-space geometry and topology can materially affect anomaly separation, motivating explicit study of latent dimension before treating any threshold as reliable [80] [77] [79].Reconstruction-based anomaly detection is known to be sensitive to hyperparameters and score design [80] [77] [79].Source: G Ablation study on noisy anomaly scores · p. 54
• Changing the bottleneck from d = 2 to d ∈{4, 6, 8, 10, 12, 14, 16} tests whether representation capacity alters downstream threshold control.Autoencoders are trained on one MinBias sample and evaluated on a disjoint MinBias dataset to introduce deployment-like distribution shift.Source: G Ablation study on noisy anomaly scores · p. 54
• The ablation changes anomaly-score distributions while preserving qualitative controller behavior across latent dimensions.This supports evaluating threshold reliability when reconstruction-score variability changes with representation capacity.Source: G Ablation study on noisy anomaly scores · p. 54
• InBand ≈0.76–1.00 across latent dimensions for DQN, GRPO, GFPO-F, and GFPO-FR, with low MAE and P95|e|, unlike PID’s 0.30–0.46.PID is less reliable because its fixed coefficients depend on score calibration, whereas RL controllers generalize across latent dimensions.Source: G Ablation study on noisy anomaly scores · pp. 54–55
• The AD-trigger parameter in [5] is calibrated on a single MinBias sample at d = 2 and lacks generalizability to other latent dimensions.This calibration limitation helps explain PID’s sensitivity under latent-dimension changes.Source: G Ablation study on noisy anomaly scores · p. 55
• Near-perfect background constraint satisfaction from GFPO-F contrasts with GFPO-FR’s higher conditional signal efficiency and more frequent violations.The filtered objectives therefore expose the expected feasibility–efficiency trade-off among RL variants.Source: G Ablation study on noisy anomaly scores · p. 55
• RL controllers primarily use rate feedback over windows rather than absolute anomaly-score calibration, remaining robust to moderate changes in autoencoder capacity and score quality.This robustness is consistent with prior dynamic-thresholding observations.Source: G Ablation study on noisy anomaly scores · p. 55

H.1 UNSW-NB15

• 0.45%–0.55% is the tightest realizable UNSW-NB15 FAR band because chunk-level negatives quantize FAR in steps of ≈0.224%, making the LHC band unreachable.With FP = 2, the selected band contains one realizable count at FAR ≈0.45%; the LHC analogue avoids this quantization floor.Source: §H.1 · pp. 55–56
• 47.6% is the structural upper bound on UNSW-NB15 in-band rate, since 43 of 82 Period 2 chunks contain only attacks and necessarily have FAR = 0.GFPO-FR’s 6.1% in-band rate therefore corresponds to ≈12.8% of the achievable structural maximum.Source: §H.1 · p. 56
• GFPO avoids D-SPOT’s 0% in-band failure under ≈55% attack prevalence by using per-chunk FAR feedback and feasibility-filtered training instead of estimating a contaminated score tail.D-SPOT’s generalized Pareto fit is dominated by attack scores, producing a threshold that flags effectively all traffic.Source: §H.1 · p. 56
• ≈12% Backdoors TPR and ≈22% Exploits TPR let GFPO-F and GFPO-FR dominate learned baselines while satisfying the FAR budget, with test-time training widening the Backdoors gap.GRPO and L-GRPO collapse to near-zero TPR through zero-feasibility failure, whereas GFPO maintains Exploits performance without trading it off during adaptation.Source: Figure 31 · p. 55, §H.1 · p. 57

H.2 NAB

• NAB uses 24 real-world telemetry series with 405 chronological test chunks, robust causal z-score anomaly scores, and a reward of TPR − αFPR − β|δ|/∆.The agent observes eight-chunk histories and selects threshold deltas on a 21-bin grid, with α = 0.10, β = 0.005, and ∆ = 0.3.Source: §H.2 · pp. 58–59
• Classical NAB controllers achieve 11–20% precision with recall below 10%, while GRPO and L-GRPO collapse to approximately 20% precision and 3.1% recall.PPO and Constant-opt raise recall above 10% but reduce precision to 14%, exposing the tradeoff that GFPO improves on.Source: Figure 32 · p. 58, §H.2 · p. 59
• ≈32% precision and ≈19% TPR let GFPO-F and GFPO-FR surpass every NAB baseline on both axes and achieve the highest F1.This result isolates sequence-based state and adaptive thresholding because NAB removes the FAR constraint and uses a pure detection-quality reward.Source: §H.2 · p. 59, Table 14 · p. 63