PaperlayerView exact revision on arXiv

Source-linked AI summary

AI Agents Can Already Autonomously Perform Experimental High Energy Physics

Eric A. Moreno, Samuel Bright-Thonney, Andrzej Novak, Dolores Garcia, Philip Harris

arXiv:2603.20179v3hep-excs.AIcs.LG

TL;DR

The paper asks whether AI agents can autonomously perform realistic, end-to-end HEP analyses beyond narrowly scaffolded workflows. Using JFC, Claude Code and related agents plan, execute, review, and document analyses on open ALEPH, DELPHI, and CMS data, including a novel primary Lund jet-plane measurement.

  • Problem

    Existing HEP agentic systems are typically highly scaffolded, while realistic end-to-end benchmarks on experimental data remain lacking.

  • Method

    JFC orchestrates autonomous agents through sequential analysis and review phases, with literature-based knowledge retrieval and specialized multi-agent critics.

  • Results

    Claude Code produced complete analyses spanning event selection, background estimation, uncertainty evaluation, statistical inference, and publication-grade reporting, including a novel primary Lund jet-plane measurement in e+e− collisions.

  • Takeaways & Limitations

    AI agents can already execute substantial portions of standard HEP analysis pipelines and may reduce the technical implementation burden on physicists.

  • Takeaways & Limitations

    The demonstrated analyses are relatively standard, while more complex workflows with advanced multivariate methods, multiple control regions, or simultaneous many-channel fits remain to be demonstrated.

Abstract

from arXiv · show

Large language model-based AI agents are now able to autonomously execute substantial portions of a high energy physics (HEP) analysis pipeline with minimal expert-curated input. Given access to a HEP dataset, an execution framework, and a corpus of prior experimental literature, we find that Claude Code succeeds in automating all stages of a typical analysis: event selection, background estimation, uncertainty quantification, statistical inference, and paper drafting. We argue that the experimental HEP community is underestimating the current capabilities of these systems, and that most proposed agentic workflows are too narrowly scoped or scaffolded to specific analysis structures. We present a proof-of-concept framework, Just Furnish Context (JFC), that integrates autonomous analysis agents with literature-based knowledge retrieval and multi-agent review, and show that this is sufficient to plan, execute, and document a credible high energy physics analysis. We demonstrate this by conducting analyses on open data from ALEPH, DELPHI, and CMS to perform electroweak, QCD, and Higgs boson measurements. We present two of those results in a condensed short paper form -- a CMS Run1 Open Data $H\to τ^+τ^-$ to demonstrate performance on a well-established result, and the first Lund plane measurement on LEP data -- a genuinely novel result and, to our knowledge, the first produced autonomously by an AI agent. Rather than replacing physicists, these tools promise to offload the repetitive technical burden of analysis code development, freeing researchers to focus on physics insight, truly novel method development, and rigorous validation. Given these developments, we advocate for new strategies for how the community trains students, organizes analysis efforts, and allocates human expertise.

1 Introduction

The paper argues that routine HEP analysis work is technically burdensome yet largely delegable to autonomous AI agents. JFC combines high-level objectives, literature access, autonomous execution, and AI review while retaining human validation for scientific results.

  • Typical HEP analyses require extensive literature study, event-selection design, uncertainty quantification, unblinding, and statistical interpretation.
  • Repetitive analysis coding is slow and error-prone, consumes substantial researcher time, and often contributes little beyond an initial programming-learning phase.
  • JFC delegates event selection, background estimation, systematic uncertainties, statistical inference, and report writing to an autonomous Claude Code workflow.
  • The framework supplies methodology, domain conventions, and behavioral rules analogous to the guidance and standards provided to a graduate student.
  • Analyses were produced on archived ALEPH, DELPHI, and CMS open data, with optional literature retrieval or internet access.
  • The showcased analyses primarily demonstrate autonomous capability rather than constituting validated scientific results, while two results received closer correctness evaluation.
  • Human experts must validate AI-assisted analyses, and complex techniques, bespoke reconstruction, subtle systematic interactions, and final accountability remain human responsibilities.

2 Related Work

Prior HEP agentic systems generally automate constrained steps or specialized tasks, while the field lacks integrated autonomous end-to-end analysis over real experimental data. The paper positions JFC as combining autonomous planning, literature retrieval, and multi-agent review to address this gap.

  • Scientific AI systems span chemistry, materials science, mathematics, and HEP, but domain-specific data complexity and validation requirements shape their capabilities.
  • Existing HEP systems include workflow automation, anomaly detection, simulation pipelines, and phenomenology tools, but generally operate within predefined structures or specialized domains.
  • Badea et al. demonstrated substantive agent participation in LEP measurement through iterative physicist feedback, whereas JFC delegates review to specialized AI agents.
  • Other systems focus on deep-learning classification, assistant functionality, documentation, or controlled reproduction benchmarks rather than fully autonomous analysis.
  • Scientific literature retrieval remains difficult because relevant information spans multiple sections, figures, and papers.
  • The field lacks an integrated framework combining autonomous multi-step planning, literature knowledge retrieval, multi-agent review, and realistic end-to-end analysis of experimental data.

3 The JFC Framework

JFC orchestrates autonomous HEP analyses through sequential artifact-producing phases, literature retrieval, and specialized multi-agent review. It gives an agent broad strategic autonomy while inserting independent checks and a human gate before full unblinding.

  • JFC delegates execution and review across sequential phases, requiring each phase to produce a written artifact and pass independent review.
  • The agent receives a high-level physics objective and autonomously chooses event selection, background estimation, uncertainties, statistical methods, and result presentation.
  • The framework encodes analysis methodology, domain-specific conventions, and agent communication rules to provide operational context and reviewable behavior.
  • Task decomposition: The orchestrator uses strategy, exploration, processing, inference, unblinding, and documentation phases, each producing named analysis artifacts.
  • Unblinding: Full-data execution occurs only after human approval of the partially unblinded analysis and consistency checks.
  • Multi-agent review: Specialized reviewers detect analysis and documentation problems, while structured review records make human physics judgment more efficient.

4 Results

JFC-produced analyses reproduce established HEP analysis strategies and yield a CMS H→τ+τ− result consistent with CMS, alongside a candidate first e+e− Lund-plane measurement. The results demonstrate autonomous capability, but open-data constraints, incomplete iterative refinement, and the need for expert validation limit their status as scientific measurements.

  • Validation and limitations: The analyses should be interpreted as evidence of framework capability rather than independently validated measurements, especially because open-data releases omit some calibration and analysis inputs.The paper states that the two condensed analyses received closer examination, while the remaining results were not vetted to publication standards.
  • Quantitative assessment: Eight of nine scalar comparisons lie within |pull| < 2 of published references, while the ALEPH ΓZ result is −3.4σ low.The ΓZ discrepancy is traced to imperfect correction of energy-dependent selection efficiency at off-peak scan points.
  • CMS result: The CMS H→τ+τ− signal strength is 1.20 ± 1.13, within 0.16σ of the published 1.01 ± 0.41 value, but its uncertainty is nearly three times larger.The result is therefore presented as a correctness and reproducibility demonstration rather than a competitive measurement.
  • ALEPH result: The ALEPH primary Lund jet-plane density is a candidate first measurement of this observable in e+e− collisions, obtained from archived LEP1 data.The agent reports a mean primary-emission multiplicity of ⟨N⟩ = 4.751 ± 0.224 and stability under tested unfolding and correction variations.
  • Analysis quality: The framework produced analyses whose selections, uncertainty sources, and correction procedures were methodologically aligned with published HEP approaches.The agents used standard techniques including iterative Bayesian unfolding, HistFactory likelihoods, and Breit–Wigner lineshape fitting.
  • Review and refinement: Agents often diagnose problems correctly but defer fixing them, with iterative refinement identified as the main area for framework improvement.Review rounds caught mistakes, including an arbitrary background prior and an incorrectly implemented goodness-of-fit test.
  • Throughput: Each end-to-end analysis completed in roughly 10 hours of wall-clock time, typically passing review after one to two iterations.Figure 7 separates autonomous compute, rate-limit waits, and post-gate human review using per-analysis git logs.

5 Discussion

AI agents could automate substantial technical work in HEP analyses, shifting human effort toward analysis design, physical reasoning, validation, and review. The paper argues that responsible adoption requires benchmarks, review protocols, training, and institutional changes while preserving human accountability.

  • AI agents can shift physicists from implementation toward architecture, critique, physical reasoning, and judgment.The paper frames automation as reducing technical drudgery rather than replacing physicists.
  • Limitations and future work: Current limitations include model errors, visual-inspection weaknesses, dependency workarounds, prompt-context trade-offs, and an untested ceiling on complex analysis topologies.The paper specifically identifies multi-channel fits, transfer-factor backgrounds, custom reconstruction, and machine-learning components as extensions still requiring investigation.
  • Legacy data and reanalysis: Legacy open datasets could be systematically reanalyzed with modern software, enabling human physicists to evaluate and refine first-pass results.ALEPH and DELPHI analyses serve as a proof of concept for this use case.
  • Legacy data and reanalysis: AI agents could also re-execute published computational pipelines at scale to flag inconsistencies or confirm findings.The paper identifies automatic reproducibility as an application currently limited by verification resources and personnel.
  • Robust AI-assisted analysis: Responsible deployment requires end-to-end benchmarks and explicit review protocols, including independent reimplementation, baseline comparisons, and regression testing.The proposed multi-agent review complements rather than replaces human review.
  • Training and institutional adaptation: AI-assisted workflows require training in code evaluation, agent failure modes, precise specifications, and AI literacy alongside programming and physics.The paper also calls for updated authorship, accountability, disclosure, and documentation norms.
  • Rethinking graduate training: Graduate training could emphasize physical intuition, question formation, theoretical context, and critical evaluation while retaining code comprehension and validation skills.The motivation is that technical implementation currently occupies a substantial part of HEP graduate training.

6 Conclusion

The paper reports that structured prompts let LLM agents execute substantial HEP analysis pipelines, reproduce established measurements, and produce a novel Lund plane measurement. It presents these capabilities as support for responsible experimentation with new workflows, benchmarks, and review practices rather than physicist replacement.

  • LLM agents autonomously planned, executed, reviewed, and documented analyses using structured prompts and literature retrieval.The demonstrated pipeline covered archived ALEPH data and CMS Open Data.
  • The primary Lund jet plane density was reported as the first e+e− measurement and the first new HEP measurement produced autonomously by an AI agent, subject to verification.
  • The authors argue that agents can absorb much of the technically demanding implementation work while physicists retain responsibility for insight, creative questions, and judgment.
  • The community should develop workflows, benchmarks, and review practices to use these tools responsibly.

A Numerical results

The numerical-results appendix identifies the values plotted in the comparison figure and explains how agent measurements are compared with published references. It also defines the uncertainty and compatibility conventions used in the table.

  • Table A.1 lists agent-produced scalar measurements alongside the published reference value for each comparison.Each analysis group also links to a public code repository.
  • The table separates statistical and systematic uncertainties where the source provides that split, with σ_tot formed by quadrature or reported directly.Missing component splits are marked with em-dashes.
  • Compatibility is summarized with a pull comparing the agent and reference measurements using their combined uncertainties.The table flags bold pulls at |pull| ≥ 2.
  • The plotted numerical content is read directly from the measurements.yaml and references.yaml files that feed Figure 3.

B An Open and Reproducible Measurement of H →τ +τ −in the µτh Final State with CMS Open Data

This measurement reconstructs a categorized H →τ +τ − analysis in the CMS µτh final state using public 8 TeV Open Data and open-source tools. Its reproducible statistical implementation provides a transparent reference while remaining limited by the single final state and partial dataset.

  • The analysis measures Standard Model Higgs production in the µτh final state using 11.467 fb−1 of CMS Open Data collected at √s = 8 TeV.Events are divided into 0-jet, boosted, and VBF categories, with data-driven W+jets and QCD estimates and constrained t¯t backgrounds.
  • The measurement’s precision is limited by the single final state and partial 2012 dataset.It is presented as a transparent, dual-engine-validated reference implementation rather than a precision frontier result.
  • Introduction: H →τ +τ − directly probes the Higgs Yukawa coupling to leptons because tau leptons are the heaviest leptons and the decay branching fraction is roughly 6%.
  • Public collision data, published likelihoods, and mature open-source tools make a complete Higgs measurement reproducible outside the CMS collaboration.The setup also permits verification with an independent statistical engine.

B.2 Data and event selection

The CMS Open Data analysis selects H →τ +τ − events in the µτh final state, partitions them into three production-sensitive categories, and estimates key backgrounds using simulation and control regions. A DNN discriminant and simultaneous fit then extract the signal strength.

  • Data sample: The analysis uses CMS 2012 Run B and C TauPlusX data at √s = 8 TeV with 11.467 fb−1 and simulated signal and background samples.The dataset is triggered by an isolated muon and hadronic tau candidate matching the µτh topology.
  • Event selection: Events require an isolated tight muon, a medium-isolation hadronic tau candidate, opposite charge, and additional µτh selections, yielding 27,240 opposite-sign signal-region events.
  • Event categories: Selected events are divided into VBF, boosted, and 0-jet categories that isolate production features and order the sample by signal-to-background ratio.The VBF category requires mjj > 500 GeV and |∆ηjj| > 3.5; the boosted category requires pT > 100 GeV, while remaining events form the 0-jet category.
  • Background estimation: The dominant Z →τ +τ − background is simulated, while W+jets and QCD backgrounds are estimated with data-driven methods and fitted using control-region information.Their normalizations enter the fit as free parameters constrained by control-region statistics.
  • Background estimation: A b-tagged-jet inversion defines three t¯t-enriched control regions, whose simultaneous fit constrains one shared t¯t normalization across control and signal regions.The control-region t¯t purity ranges from 56% to 80%.
  • Signal extraction: Because neutrinos broaden the reconstructed di-tau mass, a single gradient-boosted DNN using fifteen kinematic and angular inputs separates signal from background across all categories.The signal strength is extracted from a simultaneous binned maximum-likelihood fit of the three DNN distributions and three t¯t control regions.

B.7 Results

The simultaneous fit finds a Standard-Model-compatible H →τ +τ − signal strength, with post-fit distributions describing the data across all three categories. The analysis also reports an observed upper limit consistent with its expected sensitivity.

  • Post-fit agreement: The post-fit model reproduces the data in all three categories without localized structure across large differences in event count and signal-to-background ratio.The W+jets, QCD, and Z →τ +τ − backgrounds are pulled within their pre-fit uncertainties.
  • Post-fit agreement: The S/(S+B)-weighted combination emphasizes the most signal-like bins, where the post-fit signal remains within the background uncertainty and the residual shows a mild positive excess.The fit is compatible with both the absence of signal and a Standard-Model-sized signal.
  • Analysis terminology: The DNN is labeled throughout the analysis, although it is a gradient-boosted decision-tree output rather than a neural network.

B.8 Independent validation

Independent statistical reconstruction validates the CMS measurement, while public analysis artifacts make the workflow reproducible. The accompanying Lund-plane result provides a separate open-data measurement with a quantified model offset and robustness tests.

  • CMS validation: The pyhf HistFactory workspace and an independently rebuilt Combine datacard reproduce the likelihood point-by-point across systematic variations and signal-strength scans.
  • CMS validation: Z = 1.152 is obtained by both statistical engines, which also agree on the in-situ t¯t normalization and confirm a normal fit with well-described data.
  • Reproducibility: The CMS workspace, code, intermediate artifacts, public data, and independent Combine validation are openly available for end-to-end reruns.
  • CMS validation: ˆµ = 1.20 ± 1.13 and µ < 3.72 summarize the CMS open-data measurement, which agrees with the Standard Model and the published CMS µτh result within 0.16 standard deviations.
  • Lund-plane result: The ALEPH Lund-plane measurement reports ⟨N⟩ = 4.751 ± 0.224 and places the density 9–18% above modern parton-shower plus hadronization predictions.The offset is predominantly an overall normalization removable by generator-specific rescaling kg = 1.10–1.17.
  • Lund-plane result: The Lund jet plane separates perturbative showering, running αs, and hadronization across a two-dimensional splitting phase space, while Z-pole data avoid underlying-event, MPI, and pileup effects.
  • Lund-plane result: This is the first published primary-LJP-density measurement in e+e− collisions, using the archived ALEPH dataset and an analytic NLL plus NLO benchmark.

C.2 Data and method

The ALEPH measurement uses 1994 Z-pole data, reconstructs primary emissions in the Lund plane, and corrects detector-level spectra with iterative unfolding. Systematic uncertainties are propagated through the correction chain and dominate the reported precision.

  • Data sample: The baseline measurement uses 1994 ALEPH peak data and detector-simulated PYTHIA 6.1 hadronic Z decays because only that run has full detector simulation.The peak window is |E − MZ| < 0.5 GeV with MZ = 91.188 GeV.
  • Observable definition: The primary Lund-plane density ρ is the average number of primary emissions per hemisphere per unit area in the (ln 1/∆θ, ln kt) plane.In the perturbative bulk it follows (2/π) CF αs(kt) and turns over near kt ∼ΛQCD.
  • Correction procedure: The detector-level spectrum is corrected with two-dimensional iterative Bayesian unfolding using a PYTHIA-derived response, efficiency, purity, and prior.A bin-by-bin split-sample correction provides an independent cross-check, and the global diagonal fraction is 0.944.
  • Systematic uncertainties: Ten systematic sources are propagated as bin-dependent shifts in the correction operator, with prior/model dependence, track weights, and binning non-closure among the largest contributions.Detector effects each remain below 2%, and underlying-event, MPI, and pileup systematics do not apply at the Z pole.
  • Measured distribution: The corrected density has the canonical triangular Lund-plane structure, with few-percent precision in the perturbative region and larger uncertainties near kinematic edges.
  • Systematic uncertainties: The total density-weighted uncertainty is 6.0%, while the integrated ⟨N⟩ uncertainty is 0.224 or 4.7%, making the measurement systematics-limited and prior-dominated.

C.4 Results

The corrected Lund-plane density is structurally consistent with expectations and robust to unfolding choices, with uncertainties dominated by systematics.

  • The density exhibits the canonical triangular Lund structure, follows the running of αs(kt) in the perturbative bulk, and turns over below kt ∼ΛQCD.
  • The corrected density is compatible with the correction Monte Carlo, with diagonal χ2/ndf = 0.33, worst per-bin pull 1.10σ, and no bins above 2σ.
  • Prior swaps change ⟨N⟩ by at most 0.02%, while rebuilding the correction operator changes it by at most 0.67%.
  • Independent SVD unfolding agrees with the baseline iterative result to 0.19% in ⟨N⟩, confirming stability under prior, operator, and algorithm choices.

C.5 Generator benchmark

Modern generator predictions lie below the measured Lund-plane density, with most of the discrepancy attributable to an overall normalization and the remaining shape covered by theory uncertainty.

  • The occupancy-weighted excesses are +12.4% for Monash, +9.0% for Vincia, +11.7% for default, and +17.7% for Sherpa.The comparison uses the same fiducial region for data and each generator.
  • Single best-fit rescalings kg = 1.134, 1.099, 1.128, and 1.173 remove the integrated excess for Monash, Vincia, default, and Sherpa respectively.The rescaling removes 44–60% of the diagonal χ2.
  • Before normalization, nominal diagonal χ2/ndf values are 9.9, 6.9, 8.2, and 14.7 for Monash, Vincia, default, and Sherpa.Including the LSS kt-dependent theory band reduces these to 0.9–1.5.
  • The covariance is dominated by a coherent normalization mode: its top eigenvector carries 66% of the variance and has |cos| = 0.93 with the density-normalization direction.
  • The robust comparison uses the diagonal inverse-variance metric because full-covariance generalized least squares produces biased rescalings in the presence of the coherent mode.
  • After normalization, the residual has a shallow U-shape, while residual-shape χ2/ndf = 0.4–0.6 with p ≃1 inside the theory band.The data resolve the differential structure, but it does not significantly disfavor the generators relative to theory.

C.7 Discussion

The Lund-plane measurement is presented as a novel, robust benchmark, while its generator interpretation remains bounded by comparison and modeling limitations.

  • The Sherpa AHADIC model is most discrepant, but the comparison varies generator, shower, and hadronization together, so it does not isolate a string-versus-cluster effect.The authors therefore do not claim string-versus-cluster discrimination.
  • The heavy-versus-light flavor cross-check finds b-enriched hemispheres 32.4% higher in reconstruction-level ⟨N⟩, while 1992–1995 corrected values agree within 0.11%.The tagged-sample comparison does not alter the baseline measurement.
  • Direct comparison with published pp Lund-plane densities is precluded by differing jet definitions, energies, and quark/gluon mixtures.The e+e− measurement instead provides a distinct quark-dominated reference environment.
  • The paper reports the first primary Lund jet-plane density measurement in e+e− collisions, obtaining ⟨N⟩= 4.751 ± 0.224 with 4.7% systematics-limited uncertainty.
  • The density lies 9–18% above modern predictions, but normalization removes the integrated excess and the residual U-shape remains within generator-theory uncertainty.

D Initial agent prompts

The initial prompts specify diverse HEP measurements across ALEPH, DELPHI, and CMS, while leaving the agent to construct substantial analysis details from concise natural-language goals.

  • The prompt set includes CMS H→τ+τ−, ALEPH Z lineshape and αs, Rb/Rc, EECs, and DELPHI neutrino-count, Lund-plane, EEC, and αs analyses.
  • The CMS prompt requests µτh observables, visible di-tau mass distributions, signal and Standard Model backgrounds, categorization, and simultaneous fitting.
  • The CMS instructions additionally request neural-network and missing-energy approaches, a tight anti-muon veto, data-driven W+jets normalization, and enlarged Z normalization uncertainty.
  • The Rb/Rc prompt specifies lifetime-based b- and c-tagging with a double-tag method to reduce dependence on MC tagging efficiency.
  • The EEC prompt defines EEC, E3C, and AEEC observables, detector correction, angular representations, and comparisons with modern generators.
  • The event-shape prompts request particle-level correction, modern unfolding, NNLO+resummation fits, simultaneous αs and power-correction extraction, and comparisons of perturbative prescriptions.
  • The DELPHI Lund-plane prompt specifies Cambridge/Aachen declustering along the harder branch and recording splitting variables in logarithmic coordinates.
  • The prompt frames the LEP Lund-plane measurement as novel because the observable had not previously been measured at LEP.
Loading 2603.20179v3…