PaperlayerView exact revision on arXiv

Source-linked AI summary

Normalizing flows for lattice gauge theory in arbitrary space-time dimension

Ryan Abbott, Michael S. Albergo, Aleksandar Botev, Denis Boyda, Kyle Cranmer, Daniel C. Hackett, Gurtej Kanwar, Alexander G. D. G. Matthews, Sébastien Racanière, Ali Razavi, Danilo J. Rezende, Fernando Romero-López, Phiala E. Shanahan, Julian M. Urban

arXiv:2305.02402v1hep-latcond-mat.stat-mechcs.LG

TL;DR

Flow-based lattice gauge theory sampling had been explored mostly in two dimensions, leaving higher-dimensional applications as a central gap. This paper develops gauge-equivariant spectral and residual flow architectures and demonstrates them for four-dimensional SU(3); its hybrid sampling results favor a prior closer to the target.

  • Problem

    Flow-based sampling for lattice gauge theory had been explored primarily in two space-time dimensions, while scalable models for four-dimensional SU(3) gauge theory remained a crucial step.

    Source: §I · p. 1

  • Method

    The paper develops gauge-equivariant spectral flows on untraced Wilson-loop eigenspectra and gradient-based residual flows that transform gauge links directly.

    Source: §I · p. 1, §VII · p. 13

  • Results

    In the hybrid sampling demonstration, performance improves with β = 0.5 rather than the Haar-measure prior at β = 0.

    Source: §VII · p. 13

  • Takeaways & Limitations

    The proof-of-principle application demonstrates these flow architectures for SU(3) gauge theory in four space-time dimensions.

    Source: §VII · p. 13

Abstract

from arXiv · show

Applications of normalizing flows to the sampling of field configurations in lattice gauge theory have so far been explored almost exclusively in two space-time dimensions. We report new algorithmic developments of gauge-equivariant flow architectures facilitating the generalization to higher-dimensional lattice geometries. Specifically, we discuss masked autoregressive transformations with tractable and unbiased Jacobian determinants, a key ingredient for scalable and asymptotically exact flow-based sampling algorithms. For concreteness, results from a proof-of-principle application to SU(3) lattice gauge theory in four space-time dimensions are reported.

I. INTRODUCTION

Flow-based sampling could help address costly ergodicity problems in lattice gauge theory, but prior proof-of-principle work was primarily two-dimensional. This paper develops flow architectures for higher-dimensional geometries and demonstrates them for four-dimensional SU(3) gauge theory.

I. INTRODUCTION

  • Prior proof-of-principle flow-based lattice gauge theory sampling was carried out primarily in two space-time dimensions, motivating scalable models for four-dimensional SU(3) theory.The work targets non-Abelian lattice gauge theories on in-principle arbitrary space-time geometries.Source: §I · p. 1
  • The paper develops spectral flows and gradient-based residual flows, demonstrating these architectures in a proof-of-principle SU(3) application in four space-time dimensions.The introduction also reports observations on training strategies and neural-network parameterizations.Source: §I · p. 1

A. Lattice gauge theory

  • Gauge fields are group-valued links on a discrete space-time lattice, and state-of-the-art configurations can involve O(10^11) floating-point values.The number of integration variables generally grows as O(L^d), with lattice extent L and dimension d.Source: A. Lattice gauge theory · p. 2
  • Small lattice spacings bring topological freezing and critical slowing-down, increasing autocorrelation times and the cost of obtaining statistically independent configurations.These problems limit the precision and reach of lattice studies as the continuum limit is approached.Source: A. Lattice gauge theory · p. 2

B. Normalizing flows for lattice field theory

Normalizing flows provide tractable-density generative models for lattice-field sampling, with exactness recovered through an acceptance step. The paper describes their training and evaluation, and discusses hybrid sampling with priors closer to the target.

B. Normalizing flows for lattice field theory

  • Flow models generate independent configurations with tractable density q(U), optimized to approximate target density p(U); an independence Metropolis step yields asymptotically exact sampling.Accepted configurations fully decorrelate the chain; rejections generate autocorrelations, and perfect models achieve perfect mixing.Source: B. Normalizing flows for lattice field… · p. 2
  • A flow maps prior samples through an invertible transformation, whose Jacobian and prior density determine the output density q.The model is composed of invertible layers and can be optimized to reduce the difference between q and the target p.Source: B. Normalizing flows for lattice field… · p. 2
  • Effective sample size per configuration estimates the effective fraction of independent samples from reweighting variance and can be computed using model samples alone.It is the inverse average squared reweighting factor; ESS = 1 when model and target densities are identical.Source: B. Normalizing flows for lattice field… · pp. 2–3
  • Consistent flow-model training and sampling require only a sampleable prior and computable unnormalized prior and target densities; priors closer to the target may simplify transformations and improve efficiency, while sacrificing some independent-sampling advantages.Source: B. Normalizing flows for lattice field… · p. 3

III. SCALABLE FLOWS FOR LGT

Scalable lattice-gauge flows must keep model evaluation and Jacobian computation tractable while respecting the fields’ group structure and gauge symmetry. The section presents masked coupling layers as a structured approach to these requirements.

A. Structure and requirements

  • Gauge-field links are group-valued and high-dimensional, so scalable flow design must accommodate compact-manifold transformations and tractable model evaluation.The cost of evaluating the Jacobian determinant can otherwise grow cubically with the number of variables.Source: §III · p. 3, A. Structure and requirements · pp. 3–4
  • Coupling layers transform masked active variables element-wise conditioned on frozen variables, producing a triangular Jacobian and tractable determinant.The figure depicts alternating masks combined sequentially to construct expressive transformations.Source: Figure 1 · p. 4
  • Gauge equivariance makes the flow commute with local gauge transformations; with a gauge-invariant prior, it implies a gauge-invariant model density.The paper reports that implementing equivariance was crucial for non-trivial sampling performance in previous studies [3] [5].Source: A. Structure and requirements · p. 4
  • Matching target symmetries can avoid modeling redundant information, while masking and other structural trade-offs may break discrete lattice symmetries.The paper also notes that equivariance can expose or avoid mode-collapse issues and ensures invariance of reweighted expectation values under target-preserving symmetries.Source: A. Structure and requirements · p. 4

B. Tractable Jacobians via masking

Masking makes conditional transformations computationally tractable by arranging the Jacobian as triangular. Alternating masks distribute updates across variables, with some symmetry and modeling trade-offs.

B. Tractable Jacobians via masking

  • Masked coupling layers update active variables element-wise conditioned on frozen variables, yielding a triangular Jacobian whose determinant costs only linearly with lattice volume.For the example map, the determinant is the product of the diagonal derivatives.Source: B. Tractable Jacobians via masking · p. 4
  • Full autoregression retains triangular structure but is generally expensive because of sequential computation and breaks translation equivariance unless restricted to local internal degrees of freedom.The work therefore uses masked autoregressive layers in coupling-layer form.Source: B. Tractable Jacobians via masking · p. 4
  • Auxiliary fields offer another route to tractable conditional Jacobians, but their added dimensionality generically reduces sampling performance and requires better models to compensate.This work achieves tractability through masking instead.Source: B. Tractable Jacobians via masking · p. 4
  • Masks control information flow between lattice regions; alternating them avoids blind spots so every field variable is transformed at least once.For nearest-neighbor interactions, a checkerboard pattern can gather conditioning information from immediate neighbors; masking can reduce discrete translation symmetry.Source: B. Tractable Jacobians via masking · pp. 4–5

C. Context awareness

Context-dependent transformations use frozen field information to model correlations while retaining tractable Jacobians. The paper develops scalable gauge-equivariant coupling flows, including loop-level transformations and direct-link residual flows.

C. Context awareness

  • Context functions use frozen nearby field values to parameterize active-variable transformations, balancing correlation awareness with tractable Jacobian computation.Fixed transformations are agnostic to correlations and are too restrictive for all but the simplest modeling tasks.Source: C. Context awareness · p. 5

IV. CONSTRUCTING GAUGE-EQUIVARIANT COUPLING FLOWS

  • The construction covers two families of gauge-equivariant transformations: flows on untraced Wilson loops and flows acting directly on gauge links.Spectral and gradient-based residual flows are concrete instances, and components can be combined in layer stacks.Source: §IV · p. 5

A. Loop-level flow

  • Loop-level coupling flows generalize earlier two-dimensional Wilson-loop transformations to arbitrary dimensions and transform eigenvalue spectra to obtain gauge equivariance.The paper considers spectral flows based on explicit diagonalization.Source: A. Loop-level flow · p. 5
  • A loop transformation induces a gauge-equivariant link map by pushing the difference between the original and transformed loop onto one constituent link.Gauge-invariant traced loops can provide context; passive loops complicate masks because changing an active link also changes loops containing it.Source: A. Loop-level flow · pp. 5–6

1. Spectral flow

The spectral-flow construction makes transformations of untraced Wilson loops permutation-equivariant by ordering eigenvalues in a canonical cell. Within that cell, coordinate choices and monotonic splines shape the transformation, with polar coordinates reducing modeling ambiguity.

1. Spectral flow

  • Eigenvalue spectra contain the gauge-invariant information of untraced Wilson loops, and conjugation-equivariant loop maps operate on that spectrum.The paper uses spectral flows based on explicit diagonalization.Source: §1 · p. 6
  • The flow diagonalizes the matrix, transforms its spectrum, then reapplies the eigenvectors to form the transformed matrix.The procedure addresses SU(3), whose eigenvalues are parameterized by two angles, while the method generalizes to SU(N).Source: §1 · p. 6
  • A canonical ordering selects one of six permutation-related cells, making the spectral transformation permutation-equivariant by construction.This avoids requiring context functions to disentangle eigenvalue permutations.Source: §1 · p. 6
  • Strictly monotonic rational quadratic splines transform coordinates on fixed-endpoint unit intervals, with parameters supplied by context functions.Monotonicity ensures invertibility.Source: §1 · p. 7
  • Polar coordinates reduce rescaling ambiguity relative to Cartesian coordinates; the sharpest cell corner is chosen as origin, and such ambiguity-minimizing coordinates typically yield higher-quality models.Figure 2 depicts the canonical cell and polar coordinates used for the transformation.Source: Figure 2 · p. 6, §1 · p. 7

2. Masking patterns

Masking patterns partition links and loops into active, frozen, and passive subsets, trading available context against layer count and evaluation cost. The paper describes mask designs and coupling geometries, with experiments favoring checkerboard-like masks and alternating direction and location couplings.

2. Masking patterns

  • Mask generation first partitions links into active and frozen sets, then classifies loops as frozen, active, or passive according to their active-link content.Dense masks can reduce the number of layers but remove context; sparse masks can expose more information while increasing evaluation cost.Source: §2 · p. 7
  • Masks with every active link surrounded by frozen links typically produce higher-quality models than stripe masks in the reported experiments.This pattern supplies nearby information without being excessively sparse.Source: §2 · p. 8
  • Alternating mask parameters can cover every link at least once, while additionally iterating active-loop directions can address loops that remain frozen or passive.The figure shows an example mask alternation and the remaining uncovered loop surfaces.Source: §2 · p. 8

3. Direction and location couplings

  • Direction couplings transform active links pointing one way; location couplings can transform links in all positive directions at a site using independent, non-overlapping active loops.The same mask-generation algorithm applies to location couplings with minor modifications.Source: §3 · p. 8, Figure 4 · p. 8
  • Alternating direction and location couplings improves model quality in experiments, particularly compared with location-only flows.The two coupling types differ in geometry and use of frozen information.Source: §3 · p. 8

4. Context information

Context functions can use additional frozen Wilson-loop shapes to recover information unavailable from frozen plaquettes alone; link-level residual flows impose their own gauge-equivariance constraint.

4. Context information

  • Plaquettes alone cannot include every frozen link, so 2 × 1 loops, chairs, and crowns provide a sufficient set for incorporating the available frozen-link information.More complicated loop shapes may also be computed explicitly, and nested gauge-equivariant transformations may be applied to frozen links before computing loops.Source: §4 · p. 8
  • Adding 2 × 1 loops, chairs, and crowns generally improves performance over using plaquettes alone.The experiments indicate that more complicated loop structures can improve the expressivity of the flow transformation.Source: §4 · p. 8

B. Link-level flow

  • Residual-block inversion costs more than forward evaluation, and uniqueness of the inverse is guaranteed when Lip(g) < 1.The inversion cost depends on how many fixed-point iterations are needed to reach the desired precision.Source: B. Link-level flow · p. 9
  • Residual flows transform gauge links directly, and each link's output must transform like an untraced loop containing that link as its first element.The function for a given link must transform with the corresponding local gauge transformation on its endpoints.Source: B. Link-level flow · p. 9

1. Gradient-based residual flow

Gradient-based residual flows transform gauge links using gradients of gauge-invariant potentials built from traced loops. Their potentials can incorporate frozen-link context, while invertibility requires suitable Lipschitz constraints.

1. Gradient-based residual flow

  • Residual flows can be defined by gradients of gauge-invariant potentials constructed from traced loops, analogous to a first-order finite-step approximation of Wilson flow.When the potential is solely the traced-plaquette Wilson action with constant global coefficients, the construction recovers standard stout smearing.Source: §1 · p. 10
  • The potential must be gauge-invariant, and invertibility requires a Lipschitz condition on its derivatives.For the plaquette-only potential, the cited condition is Lip(φ) < 8c, implying invertibility when c < 1/8.Source: §1 · p. 10
  • More general potentials can use real and imaginary parts of traced Wilson loops, including cross-terms and locally varying parameters conditioned on frozen-link gauge-invariant quantities.This provides a way to implement context awareness in the residual flow.Source: §1 · p. 10
  • Automatic differentiation can compute the Jacobian matrices and evaluate determinants exactly without volume-dependent internal dimensionality for these element-wise SU(N) maps.It also allows more complex potential parameterizations, including neural networks.Source: §1 · p. 10

2. Masking patterns and context information

The section describes masking and context-function design for the two flow families, then compares network architectures and training choices. In the experiments, convolution architectures generally train faster and perform better than axial attention, and Adam is effective for convergence speed and final model quality.

2. Masking patterns and context information

  • Residual-flow masking separates active and frozen links; active links must not occur in one another's transformation inputs, so potential terms must use non-overlapping loops.Frozen-link context coefficients can use traced loops of arbitrary shape that contain no active links.Source: §2 · p. 10

A. Neural networks

  • Translation-equivariant context networks can transfer between lattice geometries with negligible overhead; the architectures considered are standard convolutions, spatially separable convolutions, and axial attention.Spatially separable convolutions reduce parameters and arithmetic operations, while encoding exchange of lattice directions.Source: A. Neural networks · p. 11
  • Local multilayer perceptrons can increase context-function expressivity while preserving translation symmetry, and lower-precision network evaluation can reduce cost while remaining similarly expressive.The work uses gauge-invariant context inputs, allowing in-principle arbitrary activation functions.Source: A. Neural networks · p. 11
  • Convolution architectures generally train faster and result in higher performances than axial attention in the experiments.Source: A. Neural networks · p. 11

B. Optimization procedures

  • Adam proves effective in the experiments for both convergence speed and final model quality.Choosing when to stop training depends on the desired statistics and training dynamics.Source: B. Optimization procedures · p. 11

C. Improving stochastic gradients

The section introduces control variates to reduce stochastic-gradient variance and demonstrates their effect during spectral-flow training. Its numerical discussion also cautions that the reported ESS values are not fixed-compute benchmarks.

C. Improving stochastic gradients

  • Control variates target stochastic-gradient variance using a function with analytically known expectation and a free coefficient.For reverse-KLD loss gradients, the model-density partial derivative is used as a zero-expectation control variate.Source: C. Improving stochastic gradients · p. 12
  • As α approaches 1, the procedure recovers path gradients, and the authors generally observe convergence to this limit during long training.Each training step requires one additional reverse-direction flow evaluation, which can be costly when forward and reverse evaluation costs differ substantially.Source: C. Improving stochastic gradients · p. 12

VI. NUMERICAL DEMONSTRATION

  • The numerical demonstration reports basic experiments for SU(3) lattice gauge theory in four space-time dimensions, with a Wilson-action target at β = 1.The experiments compare priors at β = 0 and β = 0.5; an in-depth study of scaling with lattice volume and spacing is outside the stated goal.Source: §VI · p. 12

C. Improving stochastic gradients

  • The same-seed spectral-flow run with control variates shows improved model quality early in training and a pronounced second wind.The figure plots rolling averages with a 50-step window.Source: Figure 6 · p. 12

VI. NUMERICAL DEMONSTRATION

  • The authors do not attempt a fixed-compute performance comparison, so the reported ESS differences should not be treated as benchmarks.They attribute the differences primarily to unequal optimization and development stages and the lack of careful hyperparameter optimization.Source: §VI · p. 12

A. Spectral flow

The paper implements spectral and residual gauge-equivariant flows and demonstrates them in four-dimensional SU(3) lattice gauge theory. Its outlook emphasizes hybrid sampling and the scope of the proof-of-principle results.

A. Spectral flow

  • The spectral-flow implementation uses 288 alternating direction and location coupling layers, with rational quadratic splines to transform eigenvalue spectra.Its context functions use stacked 3D convolutions, and plaquettes are active loops while plaquettes, 2 × 1 loops, chairs, and crowns are frozen loops.Source: A. Spectral flow · pp. 12–13

VII. SUMMARY AND OUTLOOK

B. Residual flow

  • Residual flows use 128 layers and gauge-invariant potentials; reflection symmetry orthogonal to the active link yielded the best models among the tested symmetry choices.The authors report no observed invertibility violations, although they do not explicitly enforce the Lipschitz condition.Source: B. Residual flow · p. 13

VII. SUMMARY AND OUTLOOK

  • The work demonstrates gauge-equivariant flow architectures for non-Abelian lattice gauge theories in arbitrary space-time dimension in a proof-of-principle SU(3), four-dimensional setting.The target uses a lattice of size 84 and the standard Wilson action with β = 1; in-depth benchmarking is outside the paper's scope.Source: §VII · p. 13
  • A hybrid approach using conventional MCMC to sample a non-trivial prior showed better model performance when the prior was closer to the target, using β = 0.5 rather than the Haar-measure β = 0 prior.The authors note that this sacrifices some advantages of independence sampling, while solving the partition-function integral is not required for many ergodicity problems.Source: §VII · p. 13
  • At-scale LQCD applications remain prospective; the paper identifies incorporating dynamical fermion fields through pseudofermion-based flow algorithms as a necessary next step.The authors describe this as preceding further engineering and exploration of high-quality LQCD flow models.Source: §VII · p. 14

Appendix A: Masking pattern algorithm

The appendix describes how link masks are generated from lattice geometry and how those masks determine active and frozen loops. It also outlines parameter alternation for producing translated and rotated masks.

Appendix A: Masking pattern algorithm

  • Table I lists mask-generation parameter settings used for Figure 3 and the Section VI A results.The appendix also notes that a Python implementation and an interactive version of Figure 3 are available in the supplementary notebook.Source: Appendix A · p. 14, Table I · p. 15

1. Link masks

  • The link-mask algorithm combines a periodic site-phase mask with a direction mask to select active links on the lattice.Coordinates, selected shifts, and the width parameter determine site phases; orientations determine which link directions are enabled.Source: §1 · p. 14

2. Loop masks

  • The loop-mask algorithm splits active links individually, constructs masks for loops containing active links, and identifies frozen loops where those masks vanish.Splitting avoids cancellation between active links of opposite orientation; the algorithm also selects loops containing exactly one active link when computing orientation signs.Source: §2 · p. 15

3. Mask alternation

  • Mask alternation can generate unique translations by cycling active-phase values and rotations by cycling orientations and shift directions.The described alternative scheme sets orientations to the first entry of orients-to-shift in each iteration.Source: §3 · p. 15

Related papers

Loading 2305.02402v1…