Source-linked AI summary
Learning New Physics from a Machine
Raffaele Tito D'Agnolo, Andrea Wulzer
TL;DR
The paper addresses how to detect departures from a reference model without assuming which new-physics model causes them. It constructs a neural-network algorithm from likelihood-ratio testing and evaluates it in simple numerical experiments, finding good sensitivity across different hypothetical signals and little dependence on neural-network capacity or prior signal-region selection.
Problem
The central problem is discovering discrepancies between data and a reference model when the alternative distribution is unknown and conventional tests rely on specified new-physics models.
Method
The method trains a neural network to parameterize an alternative distribution and compares experimental data with a reference sample using a likelihood-ratio test statistic.
Results
The numerical experiments show good sensitivity to different hypothetical new-physics signals, little effect from mostly reference-compatible data, and limited dependence on neural-network capacity.
Takeaways & Limitations
The approach supports model-independent new-physics searches and can identify the most discrepant phase-space region for further investigation.
Takeaways & Limitations
The reported results come from a few simple one-dimensional examples, while Monte Carlo reference distributions also have systematic uncertainties that require treatment.
Abstract
from arXiv · showhide
We propose using neural networks to detect data departures from a given reference model, with no prior bias on the nature of the new physics responsible for the discrepancy. The virtues of neural networks as unbiased function approximants make them particularly suited for this task. An algorithm that implements this idea is constructed, as a straightforward application of the likelihood-ratio hypothesis test. The algorithm compares observations with an auxiliary set of reference-distributed events, possibly obtained with a Monte Carlo event generator. It returns a p-value, which measures the compatibility of the reference model with the data. It also identifies the most discrepant phase-space region of the data set, to be selected for further investigation. The most interesting potential applications are model-independent new physics searches, although our approach could also be used to compare the theoretical predictions of different Monte Carlo event generators, or for data validation algorithms. In this work we study the performance of our algorithm on a few simple examples. The results confirm the model-independence of the approach, namely that it displays good sensitivity to a variety of putative signals. Furthermore, we show that the reach does not depend much on whether a favorable signal region is selected based on prior expectations. We identify directions for improvement towards applications to real experimental data sets.
1 Introduction
The paper frames new-physics searches as tests for discrepancies between data and powerful reference models, while noting that conventional tests often depend on specifying candidate models in advance. It proposes neural networks as flexible alternative-distribution approximants for more model-independent searches.
- Motivation: Reference models such as the Standard Model and ΛCDM can describe existing and future data, but experiments may reveal previously unobserved phenomena or precise discrepancies.The paper identifies searching for discrepancies between data and reference-model predictions as a central priority.
- Problem: The general task is to test a predicted multidimensional distribution against repeated measurements, but many existing strategies are poorly suited to discovering unspecified discrepancies.The challenge arises because the relevant departure from the reference distribution is not known beforehand.
- Existing approach: Model-dependent searches use candidate new-physics models to identify favorable data subsets or correlated deviations before constructing standard hypothesis tests.This approach is physically informative even when the data remain compatible with the reference model.
- Model independence: In the physics sense, model-independent searches select alternative distributions without deriving them from a physical model, prioritizing flexibility so they can adapt to many underlying data distributions.This is intended to include scenarios not predicted by currently constructed models.
- Proposed approach: The paper parameterizes such flexible alternative distributions with artificial neural networks, motivated by their use as efficient and relatively unbiased function approximants.The paper presents the strategy as a likelihood-ratio hypothesis test converted into a neural-network training problem.
2 Conceptual Foundations
The paper formulates reference-model testing as a likelihood-ratio comparison against flexible alternative distributions, implemented with neural networks. This model-independent construction addresses limitations of fixed signal hypotheses and binned goodness-of-fit tests, especially in high-dimensional data.
- Likelihood-ratio testing: The reference-model test compares observed events with a composite alternative distribution n(x|w), selecting the best-fit parameters by maximum likelihood.The resulting likelihood-ratio statistic is evaluated against toy datasets generated under the reference hypothesis to obtain a p-value.
- Neural-Network Alternatives: The proposed method replaces piecewise-constant functions with fully connected neural networks whose weights and biases parameterize the alternative hypothesis.Neural networks are motivated as flexible distribution approximants that may require fewer events than histograms for multivariate distributions, although training data remain limited.
- Model-Dependent Tests: The Neyman–Pearson ideal statistic is optimal when the true new-physics distribution is known, but it cannot design an agnostic search without prior signal assumptions.An incorrectly chosen signal hypothesis can place weight only in the wrong phase-space region and lose sensitivity.
- Model-Independence: A model-independent search selects alternative distributions for technical flexibility rather than physical considerations, aiming to remain sensitive to many putative signals.Flexibility requires an approximating function family capable of mimicking a broad range of underlying data distributions.
- Binned Goodness-of-Fit Tests: Binned goodness-of-fit tests use piecewise-constant alternatives but suffer from arbitrary binning and dilution by bins that agree with the reference model.Removing non-discrepant bins improves sensitivity, but selecting those bins requires prior expectations about the signal.
3 The Algorithm
The algorithm trains a neural network to compare observed data with a reference sample, producing both a hypothesis-test statistic and a local estimate of distributional disagreement. Toy reference experiments calibrate the statistic into a p-value, while the learned log-ratio identifies discrepant phase-space regions; its application is limited by reference-model and sample-size assumptions.
- Interpretation: The learned log-ratio is used after a small p-value to identify the nature and location of discrepancies for further investigation and sanity checks.The algorithm’s fourth step applies f(x; bw) to learn the nature of tension with the reference hypothesis.
- Assumptions and limitations: Reference-sample fluctuations are assumed negligible for a sufficiently large sample, while imperfect Monte Carlo shape modeling remains a major limitation requiring nuisance-parameter treatment.The paper notes that fitting nuisance parameters into the network is left for future work.
- Training procedure: Monte Carlo integration and a binary target variable convert the likelihood-ratio objective into a single event-level loss suitable for neural-network training.Reference events receive y = 0 and data events y = 1; minimizing the loss is equivalent to standard supervised training.
- Hypothesis test: The test statistic is calibrated by retraining the same network on toy data generated under the reference model, yielding P(t|R) for p-value computation.The observed statistic tobs is compared with this reference distribution, with p obtained from its upper tail.
- Inputs and outputs: The algorithm compares a data sample with a reference-distributed sample and outputs a test statistic plus a learned log-ratio of their probability distributions.The log-ratio approximates log[n(x|T)/n(x|R)] and can localize disagreement in phase space.
- Simple case study: In a simple one-dimensional example, neural-network fits remain smooth, reproduce the well-sampled bulk accurately, and yield a test statistic with discriminating power between signal and background.Signal samples produce a tail excess near x = 0.8, while background departures occur mainly where few events contribute to t.
4 Numerical Experiments
Numerical experiments assess the neural-network test across multiple new-physics scenarios, cuts, dimensions, and network configurations. The method shows model-independent sensitivity and cut robustness, while two-dimensional applications and larger networks expose training-related limitations.
- Performance assessment: The ideal test uses the complete new-physics distribution to benchmark the neural-network test’s sensitivity.Reference-distributed toy samples estimate the ideal statistic distribution for comparison.
- Model-Independence: The neural-network test shows considerable discriminating power for the NP1 resonance scenario, although the ideal test is more sensitive.The NP1 test-statistic distribution shifts toward larger values, while the ideal benchmark has higher sensitivity.
- Model-Independence: The NP2 test achieves a median significance of 3.1 σ, comparable across the tested signal scenario, while NP3 reaches 2.6 σ against an ideal 4.1 σ.For NP2, the ideal median significance is 4.4 σ; for NP3, the neural-network result is reduced relative to the ideal benchmark.
- Model-Independence: The comparative study of three new physics models confirms the model-independent nature of the approach.The method is evaluated against an ideal test across distinct signal scenarios.
- (In-)Sensitivity to Cuts: A median significance of 3.1 σ is obtained for both x > 0.3 and x > 0.5, so selecting these favorable regions does not improve sensitivity.The result uses the same network architecture and training procedure as the no-cut case, with only the expected event count changed.
- Two Dimensions: In two dimensions, the significance rarely reaches 3 σ and has a median of 1.4 σ, indicating reduced sensitivity relative to the one-dimensional studies.The degradation is associated with smaller new-physics test statistics and larger reference-sample test statistics.
- Two Dimensions: The two-dimensional loss in sensitivity indicates that implementation improvements are needed before applying the method to multivariate data sets.The paper identifies training algorithms and network architectures as possible improvement directions, while retaining the method as an alternative to binned tests.
5 Alternative Loss Functions
The paper considers alternatives to maximum-likelihood training, showing that different loss functions can learn usable approximations of the data distribution and support the test statistic. Maximum likelihood is effective, but other losses remain relevant for future extensions and applications.
- The algorithm can be derived as a maximum-likelihood construction while allowing alternative loss functions through a different formulation.This alternative derivation offers more implementation freedom than the direct maximum-likelihood approach.
- The trained network must be used to evaluate the test statistic from its definition rather than directly from the final loss value.The required integral is evaluated separately with a Monte Carlo method using the reference sample employed for training.
- In the large-sample and large-network limits, several losses approach functionals whose minima recover the target distributional relation.The paper discusses square loss, maximum-likelihood loss, and cross-entropy as technically different choices with the same asymptotic target.
- Maximum likelihood was more effective in the authors’ study, although no rigorous optimality notion exists for composite alternative hypotheses.The authors therefore retain other loss functions as possibilities for future algorithmic developments and different applications.
6 Conclusions and Outlook
The paper concludes that neural networks can detect departures from a reference model with sensitivity to varied signals, while identifying computational and higher-dimensional limitations. It also points to Monte Carlo comparison and data validation as possible applications.
- The algorithm uses experimental data and a reference sample to search for departures from a reference model without specifying the responsible alternative model.The construction follows directly from the maximum-likelihood hypothesis test.
- The method showed good sensitivity to different hypothetical new physics signals and remained effective when most data agreed with the reference prediction.These experiments supported the claimed model-independent behavior.
- Sensitivity degrades when nondiscriminating input variables are added, but doubling the event count recovered sensitivity comparable to the one-dimensional case.The observed degradation was not considered to exhaust the method’s potential.
- Training time is the main computational limitation because the method uses a large reference sample, typically about 100 times the actual data.More efficient use of reference events, including weighted events, is suggested as a possible improvement.
- The approach could compare Monte Carlo generator predictions or validate raw data by comparing them with a reference sample.These applications may be easier than new-physics searches because sample sizes and reference data are more readily available.
A A Short Introduction to Neural Networks
This appendix introduces neural networks as layered function families whose parameters are fitted by minimizing a loss. It explains why small sigmoid networks can represent localized or piecewise-smooth features relevant to model-independent searches.
- A neural network is a family of functions built by composing linear and element-wise layers, with weights and biases as free parameters.The network input has dimension d, while the scalar output used here has dimension one.
- For a two-layer network with one-dimensional input, the architecture (1, Nneu, 1) contains Nneu hidden neurons and a scalar output.The number of free parameters depends on the chosen layer dimensions.
- Training fits the network weights by minimizing a loss function, typically using gradient-based updates when the nested loss is non-convex.No general algorithm guarantees a global minimum, so practical training seeks a sufficiently good local minimum.
- Neural networks can approximate any continuous function on a compact domain to arbitrary accuracy, motivating their use as flexible distribution parametrizations.The paper presents this approximation property as the reason networks can represent varied possible signals.
- A three-neuron sigmoid construction can produce a smooth peak, broad plateau, or rectangular function by adjusting weights and biases.Increasing the relevant weights sharpens transitions, while biases control the width of the non-zero region.
- Few-parameter networks can reproduce sharp features while remaining smooth elsewhere, reducing the look-elsewhere effect relative to more flexible alternatives.This property is presented as a reason neural networks are promising for searches without a known signal shape.