Source-linked AI summary
Unsupervised learning of phase transitions: from principal component analysis to variational autoencoders
Sebastian Johann Wetzel
TL;DR
The paper asks whether unsupervised learning can identify phase transitions and order parameters without knowledge of the microscopic theory. It compares principal component analysis with variational autoencoders on Ising and XY model configurations, finding that latent parameters match known order parameters and reconstruction loss can identify transitions.
Problem
Order parameters may be difficult to define non-locally, motivating methods that identify phase transitions without prior microscopic or order-parameter information.
Method
The study applies unsupervised learning, progressing from principal component analysis to variational autoencoders, to Monte Carlo configurations of Ising and XY models.
Results
Latent parameters coincide with known order parameters, configurations form phase-related clusters, and reconstruction loss changes at phase transitions.
Takeaways & Limitations
Reconstruction loss is established as a universal indicator for phase transitions, while variational autoencoders handle nonlinear features in the data.
Abstract
from arXiv · showhide
We employ unsupervised machine learning techniques to learn latent parameters which best describe states of the two-dimensional Ising model and the three-dimensional XY model. These methods range from principal component analysis to artificial neural network based variational autoencoders. The states are sampled using a Monte-Carlo simulation above and below the critical temperature. We find that the predicted latent parameters correspond to the known order parameters. The latent representation of the states of the models in question are clustered, which makes it possible to identify phases without prior knowledge of their existence or the underlying Hamiltonian. Furthermore, we find that the reconstruction loss function can be used as a universal identifier for phase transitions.
I. INTRODUCTION
The paper motivates unsupervised methods for identifying phase-transition parameters when conventional order parameters are difficult to define. It progresses from principal component analysis to variational autoencoders, whose latent representations and reconstruction loss reveal phases and transitions.
- Complex systems can have order parameters defined only through complicated non-local constructions, motivating new phase-identification methods.
- Unsupervised learning finds structure and clusters in unlabeled data, extending phase-transition detection beyond labeled-training approaches.
- Autoencoders encode inputs into latent representations and decode those representations to reconstruct the original data.
- Variational autoencoders constrain latent-variable distributions, turning encoding and reconstruction into latent-variable modeling.
- The study progresses from principal component analysis to variational autoencoders to identify phase transitions without the microscopic theory or order parameter.
- The learned latent representation coincides with the correct order parameter, while reconstruction is more accurate in the ordered phase.
A. Ising Model in 2d
The two-dimensional Ising study uses Monte Carlo spin configurations on a square lattice to establish order-parameter definitions for ferromagnetic and antiferromagnetic phases.
- The two-dimensional square-lattice Ising model has a phase transition, unlike its one-dimensional counterpart.
- The model uses discrete spins s_i ∈ {+1, −1}, uniform interaction strength J, and nearest-neighbor interactions.
- The ferromagnetic critical temperature is Tc = 2.269, where the discrete Z2 symmetry is spontaneously broken below Tc.
- The simulations use a lattice with L = 28 and N = 784, sampling temperatures T ∈ [0,5] with 50000 ferromagnetic and 10000 antiferromagnetic configurations.
- The ferromagnetic order parameter is based on the expectation value of the absolute magnetization at fixed temperature.
- For the antiferromagnetic model, the order parameter is the expectation value of staggered magnetization computed using A_ij = (−1)^(i+j).
B. XY Model in 3d
The three-dimensional XY model tests whether variational autoencoders can identify phases when a continuous symmetry is involved. It uses Monte Carlo configurations and an order parameter based on the magnetization norm.
- The three-dimensional XY model is chosen because continuous symmetries require testing infinitely many realizations of the ordered phase.
- The model has unit-length spins in R^2, interaction strength J = 1, and critical temperature Tc = 2.2017.
- Monte Carlo simulations generate 10 000 independent configurations on a cubic lattice with L = 14 and N = 2744 over T ∈ [0,5].
- Its order parameter is defined analogously to Ising magnetization, using the L2-norm of a two-component magnetization.
III. METHODS
The methods compare linear and nonlinear unsupervised representations, using PCA, kernel PCA, autoencoders, and variational autoencoders to encode and reconstruct spin configurations.
- PCA reduces dimensionality through orthogonal linear transformations whose components are ordered by variance.
- Kernel PCA performs principal component analysis after projecting the data into a kernel space with a radial basis function nonlinearity.
- Autoencoders use an encoder to map inputs to latent variables and a decoder to reconstruct inputs while minimizing reconstruction loss.
- Variational autoencoders constrain encoded representations by learning a probability distribution over latent variables rather than an arbitrary encoding function.
- The phase-transition interpretation relies on common order producing a nonzero latent parameter that encodes correlations within ordered samples.
- For these models, the latent parameter corresponds to the broken-symmetry order parameter, with one-to-one correspondence sufficient to capture order-parameter derivative discontinuities.
A. Ising Model
For Ising models, unsupervised latent representations recover conventional order parameters, cluster phases, and reveal phase transitions through reconstruction behavior.
- Ferromagnetic Ising Model: Principal component methods and variational autoencoders correlate their leading components or latent parameters with Ising magnetization, while traditional autoencoders are less accurate.The principal component methods perform slightly better than the variational autoencoder, consistent with their use of fewer parameters.
- Ferromagnetic Ising Model: A one-dimensional variational autoencoder latent parameter forms three clusters corresponding to magnetizations near −1, 0, and +1.The outer clusters represent ordered configurations, while the central cluster represents the unordered phase.
- Ferromagnetic Ising Model: The latent parameter and reconstruction loss change sharply near Tc ≈ 2.269, allowing the phase transition to be inferred without prior knowledge of the order parameter.The reconstruction loss is described as a universal phase-transition indicator across physical models.
- Ferromagnetic Ising Model: For the Ising model, adding a second latent dimension changes the results only slightly because the second parameter remains close to zero.One latent parameter is sufficient to store most information in the latent representation.
- Antiferromagnetic Ising Model: In the antiferromagnetic model, the latent parameter strongly correlates with staggered magnetization, and latent clusters distinguish phases.The averaged magnetization, latent parameter, and reconstruction loss all serve as phase-transition indicators.
- Antiferromagnetic Ising Model: Reconstructions from the latent parameter reproduce ferromagnetic and antiferromagnetic spin order, including alternating microscopic structure.The antiferromagnetic reconstruction is also recovered after transforming the latent parameter by alternating spin signs.
B. XY Model
The variational autoencoder is tested on the three-dimensional XY model to assess whether it can represent phases associated with continuous symmetry breaking. Its two-dimensional latent representation is organized radially, and its norm tracks magnetization and phase-transition behavior.
- Continuous Symmetry: The XY-model variational autoencoder handles continuous-symmetry states and represents their different ordered-phase realizations in latent variables.The test addresses the infinitely many possible realizations of a broken continuous symmetry.
- Latent Representation: Two latent parameters model the XY phase transition best, with states distributed around a central cluster in the two-dimensional latent space.The radial symmetry motivates using the L2-norm of the latent vector as the relevant latent quantity.
- Phase-Transition Indicators: The L2-norm of the latent parameter vector correlates with the L2-norm of magnetization, while the latent norm and reconstruction loss indicate the phase transition.These relationships are obtained by averaging samples at the same temperature.
V. CONCLUSION
The study shows that unsupervised learning can observe phase transitions, with variational autoencoders extending the available methods for nonlinear data. Its most distinguished latent parameters coincide with known order parameters, while reconstruction loss serves as a universal phase-transition indicator.
- Unsupervised learning observes phase transitions across algorithms ranging from principal component analysis to variational autoencoders.
- The variational autoencoder extends the traditional autoencoder for handling nonlinear features and scaling to huge datasets.
- The most distinguished latent parameters coincide with the systems’ known order parameters.
- Reconstruction loss is established as a new universal indicator for phase transitions.
- The methods may help predict unseen phases or uncover unknown order parameters, including in quantum spin liquids.