PaperlayerView exact revision on arXiv

Source-linked AI summary

Neural Network Renormalization Group

Shuo-Hui Li, Lei Wang

arXiv:1802.02840v4cond-mat.stat-mechcs.LGstat.ML

TL;DR

Traditional Metropolis updates have diffusive behavior, motivating the use of Hybrid Monte Carlo, which the paper applies in latent space. Its framework uses invertible Real NVP transformations, and the appendix shows that symmetrizing the variational density lowers the loss and can improve variational free energy.

  • Problem

    Traditional Metropolis updates exhibit diffusive behavior, motivating sampling approaches that exploit Hamiltonian dynamics.

    Source: Appendix C · p. 8

  • Method

    The approach uses Real NVP bijectors for invertible transformations and applies Hybrid Monte Carlo using an effective energy function in latent space.

    Source: Appendix B · p. 8, Appendix C · p. 8

  • Results

    Symmetrization lowers the variational loss by a nonnegative KL term and can yield better variational free energy.

    Source: Appendix D · p. 9

  • Takeaways & Limitations

    Exploiting inversion symmetry provides a way to improve the variational free energy of the model.

    Source: Appendix D · p. 9

Abstract

from arXiv · show

We present a variational renormalization group (RG) approach using a deep generative model based on normalizing flows. The model performs hierarchical change-of-variables transformations from the physical space to a latent space with reduced mutual information. Conversely, the neural net directly maps independent Gaussian noises to physical configurations following the inverse RG flow. The model has an exact and tractable likelihood, which allows unbiased training and direct access to the renormalized energy function of the latent variables. To train the model, we employ probability density distillation for the bare energy function of the physical problem, in which the training loss provides a variational upper bound of the physical free energy. We demonstrate practical usage of the approach by identifying mutually independent collective variables of the Ising model and performing accelerated hybrid Monte Carlo sampling in the latent space. Lastly, we comment on the connection of the present approach to the wavelet formulation of RG and the modern pursuit of information preserving RG.

Appendix A: Training Algorithm

The training algorithm initializes a normalizing-flow model and iteratively samples latent variables, maps them to physical configurations, and optimizes a density-based loss. It stops when its criterion is met.

Appendix A: Training Algorithm

  • The procedure takes a normalized latent prior and an unnormalized target density as inputs and returns a normalizing flow with normalized density q(x).Source: Appendix A · p. 8
  • Training continues until the stop criterion is met, optimizing the mean of ln[q(x)]−ln[pi(x)] at each step.Source: Appendix A · p. 8
  • Each iteration samples latent variables from p(z), maps them to physical variables with x=g(z), and computes q(x).Source: Appendix A · p. 8

Appendix B: Details about the Real NVP Bijector

The bijector uses Real NVP, whose transformations split variables into groups and update one group using the other. The transformations are invertible and have tractable Jacobian determinants.

Appendix B: Details about the Real NVP Bijector

  • Real NVP provides a bijective neural network with a tractable Jacobian determinant and explicit likelihood, allowing efficient evaluation of q(x).Source: Appendix B · p. 8
  • Each transformation partitions its input into two groups and updates one group using information from the other.Source: Appendix B · p. 8
  • The update uses neural-network functions for scaling and translation, and the transformation is invertible with a triangular Jacobian whose determinant is efficiently computed.Source: Appendix B · p. 8
  • Randomly sampled partitions allow all variables to be updated; each block uses ten transformation steps and shares parameters across blocks on the same scale.Source: Appendix B · p. 8

Appendix C: Hybrid Monte Carlo in the latent space

The appendix applies Hybrid Monte Carlo in latent space, using the effective latent energy to integrate trajectories and accept or reject proposals. Accepted latent states are mapped back to physical configurations for estimating observables.

Appendix C: Hybrid Monte Carlo in the latent space

  • HMC integrates Hamiltonian equations using the energy gradient, which autodifferentiation can compute, and the method can operate in physical or latent variables.Source: Appendix C · p. 8
  • The latent-space energy U(z) is the negative log prior plus the target energy at g(z), adjusted by the model log density.Source: Appendix C · p. 9
  • The simulation samples velocities, performs leapfrog integration, and applies Metropolis acceptance using the change in total energy before mapping z to x for observable estimates.Source: Appendix C · p. 9

Appendix D: Symmetrized variational calculation

The appendix incorporates discrete inversion symmetry by symmetrizing the variational density over sign flips. This symmetrized model has a lower training loss and can achieve better variational free energy.

Appendix D: Symmetrized variational calculation

  • For a target invariant under x→−x, the symmetrized density combines q(x) and q(−x), making the resulting density inversion-symmetric.Source: Appendix D · p. 9
  • Symmetrization makes the variational loss lower by a nonnegative KL term and can therefore yield better variational free energy.Source: Appendix D · p. 9
  • The mixture can be generated by randomly flipping the normalizing-flow output’s sign, with the sign treated alongside the latent variable.Source: Appendix D · p. 9
  • In latent inference, the two sign choices are weighted by q(ηx), and the inferred latent vector is used to flip the physical variables accordingly.Source: Appendix D · p. 9

Appendix E: Additional results for a wider temperature range

Across the wider-temperature study, increasing network depth lowers variational free-energy error mainly at low temperatures, while samples from different depths are similar at high temperatures. The appendix attributes the low-temperature gap to shallow models’ difficulty capturing long-range correlations.

  • The wider-range analysis uses one network without Z2-symmetrized variational calculation and adds KL(q(x) ∥ q(−x)) regularization to discourage collapse to a single ferromagnetic solution.Source: Appendix E · p. 10
  • Deeper models consistently lower variational free-energy error, with their advantage concentrated at lower temperatures and near criticality; shallow models nearly match them at high temperatures.Source: Figure S1 · p. 10, Appendix E · p. 10
  • Shallow networks miss long-range correlations because independent Gaussian variables do not share a transformation block; deeper networks better capture low-temperature coherence, while all depths look similar at high temperatures.Source: Appendix E · p. 10, Figure S2 · p. 10
  • Figure S2 samples span network depths 2–5 by rows and temperatures T = 2.0–2.7 by columns.Source: Figure S2 · p. 10

Related papers

Loading 1802.02840v4…