Source-linked AI summary
Transforming the Bootstrap: Using Transformers to Compute Scattering Amplitudes in Planar N = 4 Super Yang-Mills Theory
Tianji Cai, Garrett W. Merz, François Charton, Niklas Nolte, Matthias Wilhelm, Kyle Cranmer, Lance J. Dixon
TL;DR
The number of Feynman diagrams and integrals grows factorially with loop order, quickly making high-precision QCD amplitude calculations intractable. The paper trains Transformers to predict integer coefficients associated with tokenizable amplitude keys and studies same-loop, compressed-data, mixed-loop, and cross-loop tasks. Mixed-loop training yields faster convergence and higher accuracy on the higher-loop test set, while cross-loop models predict coefficients using a small subset of related lower-loop coefficients.
Problem
The number of Feynman diagrams and integrals grows factorially with loop order, quickly making high-precision QCD amplitude calculations intractable.
Source: §1 · p. 2Method
The paper trains Transformers to predict integer coefficients associated with tokenizable amplitude keys and studies same-loop, compressed-data, mixed-loop, and cross-loop tasks.
Source: §1 · p. 3Results
Mixed-loop training yields faster convergence and higher accuracy on the higher-loop test set, while cross-loop models predict coefficients using a small subset of related lower-loop coefficients.
Source: §9 · p. 24Takeaways & Limitations
Computing simplified scattering amplitudes to all loop orders would offer a new window into quantum field theory.
Source: §9 · p. 25Baseline models trained only at one loop achieve at most 3% accuracy when evaluated at unseen loop lengths, a length-generalization limitation for predicting higher-loop coefficients.
Source: §7 · p. 18
Abstract
from arXiv · showhide
We pursue the use of deep learning methods to improve state-of-the-art computations in theoretical high-energy physics. Planar N = 4 Super Yang-Mills theory is a close cousin to the theory that describes Higgs boson production at the Large Hadron Collider; its scattering amplitudes are large mathematical expressions containing integer coefficients. In this paper, we apply Transformers to predict these coefficients. The problem can be formulated in a language-like representation amenable to standard cross-entropy training objectives. We design two related experiments and show that the model achieves high accuracy (> 98%) on both tasks. Our work shows that Transformers can be applied successfully to problems in theoretical physics that require exact solutions.
1. Introduction
The introduction frames amplitude computation as increasingly difficult at higher loop orders and presents Transformers as a way to learn integer coefficients from tokenized amplitude terms. It outlines same-loop and cross-loop experiments and distinguishes this use of Transformers from prior work on symbolic mathematics.
1. Introduction
- At higher loop orders, bootstrap equations grow rapidly and become infeasible beyond about L = 9, motivating methods that can exploit patterns in the data.The bootstrap constrains candidate amplitudes with linear relations and physical-limit constraints; the number of relations and unknown coefficients grows by about a factor of 4 per loop.Source: §1 · pp. 2–3
- Amplitudes can be represented as tokenizable words called keys paired with integer coefficients, and proposed expressions are checked against function-space and physical-limit constraints.The physical-limit comparison uses form-factor operator product expansion predictions and is stringent enough to fix the answer uniquely.Source: §1 · p. 3
- Because keys and coefficients are token sequences, Transformers can be trained to predict coefficients, while the study focuses on learning amplitude properties rather than optimizing performance.Transformers use attention to learn complex nonlinear relationships between input features.Source: §1 · pp. 3–4
- Transformers accurately predict solution elements from other same-loop elements, work with compressed data, and show improved performance when lower-loop data augments training.The introduction also describes experiments probing learned linear relations and predicting across loop orders.Source: §1 · p. 3
- The cross-loop experiment tests whether related lower-loop coefficients can predict higher-loop terms, aiming to uncover relationships useful for the bootstrap program.The introduction notes that this relationship is suggested by the mathematical structure of the problem.Source: §1 · pp. 3–4
1.1. Related Work
- Earlier Transformer work addressed symbolic computation and related mathematical tasks, but had not applied Transformers to computations in the amplitude-bootstrap paradigm.Related approaches include symbolic amplitude computation, linear algebra, and tensor completion.Source: §1.1 · p. 4
2. Three-Gluon Form Factors in Planar N = 4 SYM Theory
The paper studies three-gluon form factors in planar N = 4 SYM, a theoretical model system for QCD, using a symbol representation that turns amplitudes into integer-valued coefficients indexed by letter sequences. It describes constraints among those coefficients and compact representations intended to manage the rapidly growing number of terms.
2. Three-Gluon Form Factors in Planar N = 4 SYM Theory
- Three-gluon form factors in planar N = 4 SYM are the focus; they involve three massless gluons and a massive color-singlet operator, and are the SYM analogue of gg → Hg.The known results through loop L = 7 are used for model training and evaluation.Source: §2 · p. 5
2.1. Symbols: A Simple Language for Amplitudes
- At loop L, the symbol represents an amplitude as a 2L-fold tensor of integer coefficients, with each 2L-letter sequence acting as a key.The symbol alphabet has six letters for this form factor; symbol terms encode information about derivatives and discontinuities, not individual Feynman diagrams.Source: §2.1 · pp. 5–6
- At L = 6 and higher, about half of the terms allowed by the adjacency and prefix/suffix rules still have zero coefficients, called nontrivial zeros.The table compares actual nonzero counts with naive counts and counts after excluding trivial zeros.Source: §2.1 · pp. 6–7, Table 1 · p. 7
2.2. Linear Relationships among Symbol Elements
- The symbol contains further linear correlations, including integrability, multiple-final-entry, and triple-adjacency relations.Functional integrability follows from commuting mixed partial derivatives and correlates coefficients associated with adjacent letter pairs.Source: §2.2 · pp. 7–8
- These linear relations can probe which symbol properties models learn at different training stages.The paper studies a subset in Section 6 and gives a fuller discussion in Appendix B.Source: §2.2 · p. 8
2.3. Compact Symbol Representations
- At eight loops, the symbol has around 1.7 billion terms, motivating compact representations based on the limited set of independent suffixes.The quad representation uses the first 2L − 4 letters plus one of eight suffix tokens.Source: §2.3 · p. 8
- At L = 6, the quad representation has 391,570 keys, compared with 5 million in the uncompressed format.It also removes dihedral symmetry and multiple-final-entry relations involving only the last four entries.Source: §2.3 · p. 9
- The study does not use the more compact octuple representation; it may be needed to push past L = 8, the highest loop order then known.The octuple format has 312,463 keys at L = 7 and 5.6 million at L = 8.Source: §2.3 · p. 9
3. Implementation Details
The experiments frame coefficient prediction as sequence-to-sequence translation, encoding keys and coefficients as token sequences and training with cross-entropy. The paper uses encoder-decoder Transformers and evaluates performance on randomly held-out elements, without fine-grained hyperparameter scans.
3. Implementation Details
- Keys and coefficients are token sequences, and models minimize cross-entropy on predicted coefficient sequences.Keys use 2L letter tokens; coefficients use signed base-1000 numerical tokens, and zero coefficients receive a plus sign token.Source: §3 · p. 9
- Most experiments use encoder-decoder Transformers with matching encoder and decoder layer counts, 8 or 16 attention heads, and dimensions of 256, 512, or 1024.The encoder and autoregressive decoder are linked by cross-attention.Source: §3 · p. 9
- Models have 4.5–245 million trainable parameters, while the best performance on many experiments comes from models below 35 million parameters.Accuracy is measured per element rather than per token.Source: §3 · pp. 9–10
- Each epoch is defined as a pass over 300,000 key-coefficient pairs, and metrics are reported on a randomly held-out test set.The test set is evaluated at each epoch; accuracy uncertainty includes both sampling error and model initialization.Source: §3 · p. 10
- The experiments omit fine-grained hyperparameter scans and typically do not use a separate validation set.Repeating one experiment with different train/test splits and the same hyperparameters produced qualitatively similar performance.Source: §3 · p. 10
4. Predicting Symbol Coefficients from Keys
Transformers are trained to predict coefficients from keys in loop-five and loop-six symbols, with zero detection separated from nonzero coefficient prediction. Nonzero prediction reaches high test accuracy, and the training dynamics show magnitudes being learned before signs.
Experiment setup
- The first experiment separates zero detection from predicting a nonzero coefficient from its key, because most coefficients are zero.The nonzero prediction models train on 62% of the L = 5 nonzero elements and 98% of the L = 6 nonzero elements.Source: §4 · pp. 10–11
Zero-versus-nonzero classification
- 99.96% at L = 5 after one epoch and 99.91% at L = 6 after one epoch on zero-versus-nonzero classification; L = 6 reaches 99.97% after two epochs.The L = 6 model had observed 3% of the symbol after one epoch and 6% after two.Source: §4 · p. 11
Nonzero coefficient prediction
- At both loop orders, models learn coefficient magnitudes before signs, with sign learning taking longer at L = 6.At L = 5, 97% of magnitudes are correct after two epochs while signs are near chance; at L = 6, sign learning occurs from epochs 40 to 70.Source: §4 · p. 11
- Before signs are learned, predicted sign balance can swing sharply between epochs, then approaches the near-even true mix around the midpoint of the sign-learning phase.The reported midpoint is epoch 7 for L = 5 and epoch 60 for L = 6; similar fluctuations occur across magnitude subsets.Source: §4 · p. 11
- 99.5% at L = 5 after 47 epochs and 99.3% at L = 6 after 199 epochs on nonzero coefficient prediction; Figure 3 plots accuracy across epochs.The best model among four initializations is reported for each loop order.Source: §4 · p. 11, Figure 3 · p. 12
- At L = 6, magnitude-only prediction reaches 97.7% accuracy after 50 epochs, while sign-only prediction remains at random-guessing performance after 100 epochs.These results suggest that learning magnitudes may be a prerequisite for learning signs.Source: §4 · pp. 11–12
5. Quad Representation of Symbols
Quad compression makes higher-loop symbol data more manageable, but coefficient prediction becomes harder as obvious symmetries and relations are removed. Transformers nevertheless achieve high test accuracy on the compressed L = 6 and L = 7 symbols.
Compression and trade-offs
- In the quad representation, coefficient learning is harder than in the full representation because it removes dihedral symmetry and relations involving up to four final entries, requiring larger models.Source: §5 · p. 13
L = 6 results
- 99% peak accuracy is reached on the quad-compressed L = 6 symbol after 192 epochs, while 95% takes 43 epochs.The full representation reaches 95% in 64 epochs, but in fewer passes through its larger training set; quad training requires more passes to converge.Source: §5 · p. 13
L = 7 results
- 99.1% test accuracy is achieved on the quad-compressed L = 7 symbol, with 98.5% reached after about 400 epochs.The larger training set partially accounts for the slower learning compared with L = 6.Source: §5 · p. 13
Model capacity and training set size
- With at least 3 million training examples, L = 7 test accuracy remains above 94% for all but the smallest model.The smallest model occasionally fails to emerge from the first plateau and stays below 50%.Source: §5 · p. 13, Table 2 · p. 14
Compression and motivation
- The quad representation retains predictive power with only about 41% of the symbol elements, while compression is motivated by the growth of higher-loop training costs.The L = 7 symbol has 93 million nonzero elements, and linear scaling from L = 6 would imply about 1,000 training hours.Source: §5 · pp. 12, 14
6. Model Characterization via Relationship Accuracy
The model learns different classes of linear relations at different stages, revealing how coefficient magnitudes and signs are acquired. Its learned embeddings also reflect dihedral symmetries at L = 5 and L = 6.
Coefficient prediction
- 98.47% of held-out L = 6 coefficients are correctly predicted after 200 epochs.The model is a 2-layer Transformer with d = 512 and 8 heads, trained on 9,732,932 elements.Source: §6 · p. 15
Relation learning by structure
- Group 1 equivalence relations are often satisfied before coefficient magnitudes are correct, because the model first predicts equal coefficients and later identifies their values.The model predicts matching signs for the pair even while each sign is individually correct only about 50% of the time.Source: §6 · p. 16
- Group 2 relations requiring opposite signs remain poorly satisfied until signs are learned, although their coefficient magnitudes are learned within a few epochs.Their accuracy then increases as the model learns which equal-magnitude coefficients should have different signs.Source: §6 · p. 17
- Group 3 mixed relations are satisfied for instances that decompose into pairs of equivalent terms when the model has learned Group 1 relations.About 40% of generated integ 0 instances can be expressed as pairs of identical coefficients.Source: §6 · p. 17
Learning dynamics
- The two accuracy plateaus correspond to first grouping equal-magnitude elements and then learning their magnitudes and signs, with sign fluctuations gradually shrinking.The fluctuations respect Group 1 equivalences while the model gradually learns Group 2 relations.Source: §6 · p. 17
Embedding geometry
- At L = 6, embedding-vector angles indicate cycle and flip symmetries more strongly than at L = 5, though three-component PCA distorts the geometry.At L = 5, the measured triangle angles are within 1.5°–3.7° of 60°; at L = 6, they are within 0.5°–1.0°.Source: §6 · pp. 17–18
7. Mixed-loop Training
Mixed-loop training combines the full L = 5 symbol with a subset of L = 6 data to examine whether lower-loop information helps prediction at higher loops. It does not improve L = 5 performance, but appears to improve L = 6 performance relative to training on a small L = 6 subset alone.
Generalization limitation
- Cross-length generalization is a major limitation: models trained exclusively at one loop achieve at most 3% accuracy on nonzero coefficients from another loop.The authors identify this failure as a barrier to predicting coefficients at unseen higher loops.Source: §7 · p. 18
Training setup
- The mixed training set combines the full nonzero L = 5 symbol with 263,880 nonzero L = 6 elements, or 5% of that symbol, and roughly equal amounts of zeros.Each loop-specific training set contains 517,760 examples, while a separate L = 6 training set matches the mixed set's 1,035,520-example size.Source: §7 · p. 19
Mixed-loop results
- The mixed-loop model reaches L = 5 performance benchmarks in almost twice as many epochs as the dedicated L = 5 model, consistent with the mixed set being twice as large.The mixed-loop model also exceeds about 90% overall accuracy, unlike the dedicated L = 6 model, but a same-sized L = 6 set produces highly similar results.Source: §7 · p. 20, Table 3 · p. 20
- Adding the full L = 5 symbol to a subset of L = 6 data appears to improve L = 6 prediction, though adding L = 6 data does not improve L = 5 tasks.The authors suggest that lower-loop features can enhance prediction from a handful of known higher-loop coefficients.Source: §7 · p. 21
8. Steps Toward Predicting the Next Loop
The cross-loop experiments test whether coefficients at loop L can be predicted from related coefficients at loop L −1. For L = 6, predictions remain accurate with fewer parent coefficients and under several changes to their values or ordering, though reducing both information sources together defeats learning.
Cross-loop prediction
- The setup targets coefficients at loop L using related lower-loop elements whose keys are formed by striking two letters from the target key.Each key has L(2L −1) strike-two parents, listed in the order of the letter pairs removed.Source: §8 · p. 21
- 98.1% of the 10,000 test examples were predicted correctly after 500 epochs using 66 L = 5 parent coefficients to predict L = 6 coefficients.The dataset was restricted to 767,500 unique examples to avoid train-test contamination; magnitudes and signs were learned simultaneously.Source: §8 · pp. 21–22
Reducing parent inputs
- 98.3%, 98.4%, and 98.1% accuracy were achieved with 55, 21, and 11 L = 5 parents, respectively, while 94.3% was achieved with 11 parents.The passage labels these results as k = 5, 3, 2, and 1, respectively; reducing to 21 parents (k = 2) had little impact relative to the full 66.Source: §8 · p. 22
Changing parent order
- 95.2% accuracy remained with randomly shuffled parent coefficients, while sorting by increasing value gave 93.9% for k = 5.The authors caution that ordering may implicitly encode key structure because some letter-adjacency conditions produce zero coefficients.Source: §8 · p. 22
Degrading parent values
- 98.4% magnitude accuracy was retained using parent magnitudes alone, but target-sign accuracy fell from 99.6% to 83.2%.With parent signs alone, overall accuracy was 93.3% and sign accuracy was 99%; additionally shuffling or sorting those inputs made learning fail.Source: §8 · pp. 22–23
Interpretation and scope
- The model can recover coefficients when parent information is scrambled or severely degraded, but not when both are done together.The experiments suggest redundant information among strike-two parents and the possible existence of a closed-form rule, which the authors have not yet recovered explicitly.Source: §8 · p. 23
9. Conclusions and Future Work
The paper reports that Transformers predict scattering-amplitude symbol coefficients and studies what they learn across representations, relations, and loop orders. The authors also identify cross-loop generalization and extensions beyond SYM as open challenges.
Learning dynamics
- Models learn coefficient magnitudes before signs; they cannot learn signs without magnitude information, though magnitudes can be learned without signs.The authors propose that relation learning progresses from grouping equal magnitudes to learning their values, then sign relationships and true signs.Source: §9 · p. 24
Compressed representation
- Transformers predict coefficients successfully from the compressed quad representation, where many trivial correlations between terms are removed.The authors say this bodes well for higher loops, where the space of possible coefficients and keys becomes large beyond L = 6.Source: §9 · p. 24
Mixed-loop training
- Adding a small percentage of higher-loop examples alongside substantial lower-loop data leads to faster convergence and higher accuracy on a higher-loop test set.The authors suggest relatively few coefficients may need to be supplied at unseen loops to predict the rest of a symbol.Source: §9 · p. 24
Cross-loop prediction
- In cross-loop strike-out experiments, models predict loop-L coefficients from a small subset of related coefficients at loop L −1.Lower-loop coefficient information can be scrambled or severely degraded, but not both at once, without hindering reconstruction.Source: §9 · p. 24
Alternative architectures
- LSTM and GRU models achieve performance comparable to Transformers on the L = 5 symbol.The authors note that other architectures may also be worth exploring.Source: §9 · p. 24
Future work and limitations
- The authors aim to train larger multitask models on information from multiple loops, but distributions of keys and coefficients change substantially between loop orders.They characterize generalizing across those changing distributions as a challenging domain-generalization task.Source: §9 · p. 25
- Applying the methods to QCD poses additional challenges: unlike SYM, QCD results generally span weights from 2L to zero and have complicated rational prefactors.The authors also note that physical-limit behavior is less understood in QCD than in SYM.Source: §9 · p. 25
Motivation
- The authors identify all-loop understanding of simplified scattering amplitudes as a potential new window into quantum field theory.They also cite potential spin-offs for collider physics.Source: §9 · p. 25
Appendix A. Mapping Generalized Polylogarithms to Symbols
This appendix describes how generalized polylogarithms are represented through symbols, built iteratively from derivatives and lower-weight functions. It also specifies the three-gluon form factor's alphabet and symmetry, and notes what the symbol leaves out.
Generalized polylogarithms
- The classical dilogarithm has weight two because it requires two non-trivial integrations over rational functions.At one loop, the dilogarithm is the only special function beyond the logarithm.Source: Appendix A · p. 29
- At L loops, the three-gluon form factor in SYM uses generalized polylogarithms of weight precisely 2L.More complex processes may require elliptic polylogarithms or functions beyond elliptic.Source: Appendix A · p. 29
Constructing symbols
- A weight-n polylogarithmic function is described through its total differential in terms of weight-(n −1) functions and logarithmic differentials of symbol-alphabet letters.Iterating this decomposition n times yields the symbol.Source: Appendix A · p. 29
Three-gluon form factor
- The symbol alphabet consists of six letters that are rational functions of the kinematic variables.The external momenta are the three gluons' four-momenta, and q^2 = s123 = s12 + s23 + s31.Source: Appendix A · p. 30
- The three-gluon form factor is invariant under the dihedral symmetry group D3, isomorphic to S3, acting on its symbol letters by cycle and flip transformations.The stated invariance follows from the arbitrary ordering of the external momenta.Source: Appendix A · p. 30
Scope of the symbol
- The symbol captures almost all critical information about a function but does not capture all of it, so boundary conditions are needed when integrating back.The symbol involves taking n derivatives.Source: Appendix A · p. 30
Appendix B. Model Evaluation on the Full List of Linear Relations
Appendix B evaluates additional linear relations during training, grouped as triple-adjacency, integrability, and multiple-final-entry relations. The relations are assessed by group, with Group 3 behavior varying by relation length.
Relation set and evaluation
- The appendix evaluates triple-adjacency, integrability, and multiple-final-entry relations, using the same architecture, dataset, and accuracy criteria as Section 6.The underlying physics for the listed relations is referenced to.Source: Appendix B · pp. 30, 32
Triple adjacency and integrability
- Triple-adjacency relations connect three terms, while integrability relations connect up to 14 terms; specified adjacent slots may occur anywhere in the key.The appendix lists one triple-adjacency relation and three integrability relations.Source: Appendix B · pp. 30–31
Multiple final entries
- There are 26 nontrivial multiple-final-entry relations involving up to the last four letters, with specified adjacent slots restricted to the key's end.When evaluated on the symbol, E is the same as F(L).Source: Appendix B · p. 31
Evaluation findings
- Figure B1 reports coefficient, magnitude, relation, and sign accuracy for Group 1 relations; Figures B2 and B3 give the corresponding metrics for Groups 2 and 3.The appendix notes that Group 3 behavior varies somewhat between short and long relations.Source: Figure B1 · p. 32, Appendix B · p. 32, Figure B2 · p. 33
- The first 16 listed final-entry relations are one-term restrictions on letters at the end of a key and are learned perfectly after a few epochs.The appendix omits detailed assessment of these relations; Group 3 results differ somewhat for short relations under four terms versus long relations over six terms.Source: Appendix B · pp. 31–32
Appendix C. Coefficient-from-Key: Model Ablations
The architecture ablations examine positional encodings, encoder-only and decoder-only models, and recurrent alternatives for coefficient-from-key prediction. Across architectures, the two-phase learning pattern persists, while relative-position models reach comparable final accuracy and recurrent-model performance depends on capacity.
Appendix C. Coefficient-from-Key: Model Ablations
- All tested architectures exhibit two-phase learning, suggesting this behavior reflects the problem rather than a particular architecture.Table C1 summarizes learning dynamics across architectures.Source: Appendix C · p. 34, Table C1 · p. 35
- Relative-position models slightly lag absolute-position models but reach comparable test accuracy after 200 epochs, with small differences between RoPE and xPos.Source: Appendix C · p. 34
- Across three seeds, reported threshold timings shift by up to 5–8 epochs, putting many architecture-performance differences within that uncertainty.Source: Appendix C · p. 34
- Decoder-only accuracy is scored on coefficient completions; encoder-only accuracy uses the coefficient tokens and the following end-of-sequence token.The encoder-only evaluation ignores output tokens after that end-of-sequence token.Source: Appendix C · p. 35
- At five loops, parameter-matched LSTM and GRU models perform slightly worse, while an h = 2048 LSTM performs at least as well as the Transformer and the GRU is not qualitatively worse.Table C2 reports learning dynamics for LSTM variants and the baseline Transformer.Source: Appendix C · p. 36, Table C2 · p. 36
Appendix C.2. Architecture Parameter Scan
The parameter scan varies Transformer depth and dimension for coefficient-from-key prediction at L = 6. Larger capacity compresses both phases of learning, while the number of attention heads appears less influential; results also vary somewhat across train/test splits.
Appendix C.2. Architecture Parameter Scan
- Doubling model dimension with other hyperparameters fixed appears to halve epochs to benchmarks, as increasing capacity compresses both learning phases.Table C3 reports the learning benchmarks and best overall accuracy across model sizes.Source: Appendix C.2 · p. 36, Table C3 · p. 37
- The number of attention heads appears to affect learning dynamics less than model dimension and depth.Source: Appendix C.2 · p. 37
- Accuracies across three alternative train/test splits are a little lower than the baseline, and the authors note that test-set choice may influence reported accuracy.Table C4 covers four random split choices for the five-loop task.Source: Appendix C.2 · p. 37, Table C4 · p. 37
Appendix D. Further Strike-out Experiments
Further strike-out experiments test what information in parent coefficients supports prediction. Removing multiplicities leaves substantial recoverable information, whereas retaining only zero/nonzero status sharply reduces accuracy.
Appendix D. Further Strike-out Experiments
- 83.4% overall accuracy after 350 epochs and 93.6% sign accuracy are achieved with sorted unique parents, suggesting useful coefficient information remains without multiplicities.Source: Appendix D · p. 38
- With zero/nonzero parent information alone, magnitude accuracy is 60.1% and sign accuracy is 61.7%, versus 93.5% and 99.0% when parent signs are clarified.Source: Appendix D · p. 38
- Table D1 compares overall, magnitude, and sign accuracy across strike-two variants and k-values, including the epoch of best overall accuracy and training-set size after duplicate removal.Source: Appendix D · p. 38, Table D1 · p. 39