PaperlayerView exact revision on arXiv

Source-linked AI summary

Discovering Phases, Phase Transitions and Crossovers through Unsupervised Machine Learning: A critical examination

Wenjian Hu, Rajiv R. P. Singh, Richard T. Scalettar

arXiv:1704.00080v2cond-mat.stat-mech

TL;DR

The paper asks how far unsupervised machine learning can decipher and distinguish phase behavior across classical spin models. Using PCA and autoencoders on Monte Carlo configurations and derived representations, it finds that these methods can identify ordering, transition types, critical behavior, and incipient orders, while raw representations can miss charge correlations.

  • Problem

    The paper examines whether unsupervised machine learning can distinguish different phases and transitions across classical spin models, including frustrated and highly degenerate systems.

  • Method

    The authors apply PCA and nonlinear autoencoders to Monte Carlo data from several Ising-family and XY models, using spin, plaquette, bond, and vorticity representations.

  • Results

    PCA recognizes order and symmetry breaking, distinguishes transition types and crossovers, locates critical behavior, and reveals incipient orders in the frustrated triangular antiferromagnet; the BSI model with J = 0 has no phase transition.

  • Takeaways & Limitations

    Unsupervised machine learning can be a useful tool for studying phase transitions, provided that the input representation matches the relevant physical correlations.

  • Takeaways & Limitations

    PCA fails to discriminate changes in opposite-sign charge configurations in the 2D XY model unless squared variables are supplied.

Abstract

from arXiv · show

We apply unsupervised machine learning techniques, mainly principal component analysis (PCA), to compare and contrast the phase behavior and phase transitions in several classical spin models - the square and triangular-lattice Ising models, the Blume-Capel model, a highly degenerate biquadratic-exchange spin-one Ising (BSI) model, and the 2D XY model, and examine critically what machine learning is teaching us. We find that quantified principal components from PCA not only allow exploration of different phases and symmetry-breaking, but can distinguish phase transition types and locate critical points. We show that the corresponding weight vectors have a clear physical interpretation, which is particularly interesting in the frustrated models such as the triangular antiferromagnet, where they can point to incipient orders. Unlike the other well-studied models, the properties of the BSI model are less well known. Using both PCA and conventional Monte Carlo analysis, we demonstrate that the BSI model shows an absence of phase transition and macroscopic ground-state degeneracy. The failure to capture the `charge' correlations (vorticity) in the BSI model (XY model) from raw spin configurations points to some of the limitations of PCA. Finally, we employ a nonlinear unsupervised machine learning procedure, the `antoencoder method', and demonstrate that it too can be trained to capture phase transitions and critical points.

I. INTRODUCTION

The paper examines whether unsupervised machine learning can distinguish phases, transitions, crossovers, and symmetry breaking across classical spin models, with special attention to frustrated and highly degenerate cases.

  • Scope: The study compares square- and triangular-lattice Ising models, the Blume-Capel model, the BSI model, and the 2D XY model.These systems span conventional ordering, frustration, vacancies, degeneracy, and power-law correlations.
  • Machine-learning goals: PCA can recognize order and symmetry breaking, locate the temperature region of sharpest change, and distinguish first- from second-order transitions and crossovers.Principal-component distributions and their temperature evolution provide the main signatures.
  • Limitations: The analysis depends on the representation supplied: bare spins, bond or plaquette variables, and local vorticity provide complementary inferences.Raw configurations fail to expose some charge correlations, including vorticity-related structure.
  • Frustrated systems: In the fully frustrated triangular antiferromagnet, PCA identifies incipient orders despite the absence of conventional long-range order.The model develops power-law correlations as temperature approaches zero, and PCA points toward nearly degenerate three-sublattice patterns.
  • BSI model: For the purely biquadratic BSI model with J = 0, conventional analysis indicates macroscopic ground-state entropy but no phase transition.The associated specific-heat and structure-factor peaks do not grow with system size.

III. METHODS OF MACHINE LEARNING

The paper distinguishes supervised and unsupervised learning by whether patterns are paired with desired answers, then uses unsupervised methods to analyze spin configurations without prior labels.

  • Learning paradigms: Supervised learning trains on patterns paired with desired answers, while unsupervised learning receives patterns without supplied answers.The paper uses unsupervised approaches for its spin-model analysis.

A. Principal Component Analysis

PCA centers Monte Carlo configuration data, extracts orthogonal directions of decreasing variance, and quantifies their projections to reveal physical patterns and temperature-dependent behavior.

  • Data preparation: The configuration matrix is centered by subtracting each column’s empirical mean before PCA extracts features.Rows represent Monte Carlo snapshots, with columns representing instantaneous degrees of freedom.
  • Sampling: For phase-transition studies, configurations are generated at evenly separated temperatures and assembled into a centered data matrix.The construction uses n uncorrelated configurations at each of t temperatures, giving M = nt rows.
  • Transformation: PCA transforms correlated variables into linearly uncorrelated principal components ordered by decreasing variance.Each successive component is constrained to be orthogonal to the preceding components.
  • Quantification: Principal-component values are inner products between configurations and weight vectors, while quantified components average those values over samples at the same temperature.This temperature-wise averaging connects PCA outputs to phase behavior.
  • Eigenproblem: The PCA weight vectors are eigenvectors of the symmetric matrix X^T X, and the corresponding eigenvalues represent variances along those vectors.Eigenvalues are conventionally sorted as λ1 ≥ λ2 ··· ≥ λN ≥ 0.

B. The Autoencoder Approach

The autoencoder provides a nonlinear alternative to PCA by compressing spin configurations through a small hidden layer and reconstructing them from that representation.

  • Autoencoder principle: An autoencoder learns compressed encodings by passing spin configurations through an intermediate layer with far fewer neurons than the original input.The reconstruction objective is to reproduce the input despite reduced information storage.
  • Architecture: Unlike PCA’s linear transformation, the autoencoder can learn nonlinear representations through encoder and decoder networks.The paper uses CNNs as both the encoder and decoder.
  • Architecture: The architecture illustrated in Fig. 2 uses two hidden neurons between the encoder and decoder.The hidden layer is the bottleneck through which the configuration is represented.

A. PCA results of the Ising model

PCA identifies ferromagnetic ordering in square-lattice Ising configurations and uses quantified components and finite-size scaling to estimate the critical temperature.

  • A single dominant principal component reflects the Ising model’s dominant ferromagnetic spin pattern.The first component’s weight vector is approximately uniform across lattice sites.
  • Below Tc, projections onto the two leading components bifurcate into two regions, whereas high-temperature configurations form one central blob.
  • The quantified first component follows the absolute magnetization, while the second component develops similarly to the susceptibility.
  • The second component’s weight vector resembles the lowest nonzero Fourier modes, linking subleading PCA structure to domain-wall arrangements.
  • Tc ∼ 2.278 ± 0.015 from finite-size scaling agrees reasonably with the exact Tc/J ≈ 2.269.

B. PCA results of the Blume-Capel model

PCA captures the Blume-Capel model’s transition behavior across parameter sweeps, locating a second-order critical point and distinguishing first-order transitions by their projection patterns.

  • PCA applied while sweeping ∆ reproduces the qualitative structure seen for the Ising model and provides transition-point estimates.
  • At fixed T = 0.4, the first-order transition occurs near ∆c ≈ 1.996.
  • First-order transitions produce projections concentrated along the second component with few intermediate points, unlike the more uniform spreading associated with second-order transitions.

C. Monte Carlo and PCA results of the BSI model

Monte Carlo analysis establishes that the BSI model lacks a phase transition at J = 0 but develops a sharp transition when J = 0.1. PCA distinguishes these regimes, identifies crossover behavior, and reveals how raw versus squared spins encode different aspects of the ordered states.

  • Monte Carlo analysis: At J = 0, the BSI specific-heat and structure-factor peaks do not grow with system size, indicating short-range order rather than a phase transition.With J = 0.1, the specific-heat peak near T ≈ 0.17 grows with system size, indicating a true transition.
  • Monte Carlo analysis: The J = 0 BSI model has ground-state entropy approximately 0.584, while adding J = 0.1 causes entropy to drop rapidly to zero below Tc.The J = 0 value exceeds ln 3/2 ≈ 0.5493, consistent with macroscopic ground-state degeneracy.
  • Monte Carlo analysis: For J = 0.1, the Binder-ratio crossing locates the critical point at Tc = 0.163, consistent with magnetization, susceptibility, and specific-heat signals.The transition signals have small finite-size effects and agree with the specific-heat peak.
  • PCA analysis: PCA on raw J = 0 configurations shows no dominant components and mixes temperatures because it averages the ±1 spins relative to zero.The relevant organization is between ±1 and 0 variables, not among the ±1 variables themselves.
  • PCA analysis: PCA on squared spins distinguishes low- and high-temperature regimes, with the first component measuring charge density and the second representing checkerboard order.Charge density changes from 2/3 at high temperature to below 0.5 at low temperature, while checkerboard order has smaller absolute variation.
  • PCA analysis: The absence of symmetry-breaking bifurcation in squared-spin projections identifies a gradual crossover and absence of a phase transition for J = 0.With J = 0.1, raw-spin PCA instead yields four low-temperature branches associated with four ground states, while squared-spin PCA reduces them to two charge-order states.

D. PCA results of the TLIM

PCA applied to the fully frustrated antiferromagnetic triangular-lattice Ising model reveals two equally weighted components associated with incipient ordering patterns despite no long-range order at finite temperature.

  • D. PCA results of the TLIM: Two equally weighted principal components emerge from raw-spin PCA for the fully frustrated triangular-lattice Ising model.The model develops power-law spin correlations as temperature decreases but has no long-range order down to T = 0.
  • D. PCA results of the TLIM: The leading components correspond approximately to three-sublattice patterns (m,0,-m) and (m,-m/2,-m/2).PCA indicates an emergent continuous XY-type symmetry between these patterns.
  • D. PCA results of the TLIM: These patterns are the triangular antiferromagnet’s two incipient orders, whose near degeneracy is lifted only by an irrelevant sixth-order anisotropy term.The result shows that PCA can identify incipient order directly from Monte Carlo data.

E. PCA results of the XY model

PCA recovers rotational structure from XY spin-angle configurations, while preprocessing is required to reveal vorticity-related behavior associated with the KT transition.

  • E. PCA results of the XY model: Angle-based PCA produces two equally weighted variances and concentric projection circles, recovering the XY model’s rotational symmetry.High-temperature samples cluster near the center, whereas low-temperature samples spread toward the circle’s periphery.
  • E. PCA results of the XY model: The quantified leading component changes abruptly around the known KT transition temperature Tc = 0.892.The temperature dependence is compared with a dashed line marking the true Tc.
  • E. PCA results of the XY model: Raw vorticity configurations produce no dominant principal component because equally weighted positive and negative vorticities cancel.This prevents PCA from capturing their temperature-dependent proliferation from the raw vorticity representation.
  • E. PCA results of the XY model: Absolute-vorticity preprocessing yields a clear dominant component, but its projections appear more like a crossover than a true phase transition.The representation marks vortex or antivortex presence without retaining their sign.
  • E. PCA results of the XY model: PCA primarily detects spatial patterns, symmetry, and symmetry breaking: discrete symmetry gives separated multi-fold patterns, while continuous rotational symmetry gives circular scatter.Distinguishing power-law correlations from long-range order still requires careful finite-size analysis of projected components.

F. Autoencoder results of the Ising model

An autoencoder trained on Ising configurations reconstructs essential domain structure and, even with one hidden neuron, identifies the transition region and distinct low-temperature branches.

  • F. Autoencoder results of the Ising model: With 200 hidden neurons, the autoencoder preserves essential input information, including domain area, domain position, and overall spin structure.Reconstructions lose detailed spin structures but retain these larger-scale features.
  • F. Autoencoder results of the Ising model: With two hidden neurons, the autoencoder reasonably separates the ferromagnetic phase from the high-temperature phase, similarly to PCA.The two-dimensional encoding is shown for raw Ising configurations.
  • F. Autoencoder results of the Ising model: With one hidden neuron, the transition temperature is located around 2.3: above it, mappings are nearly constant, while below it, two branches appear.The two low-temperature branches correspond to the Ising model’s two distinct ground states.
  • F. Autoencoder results of the Ising model: Even a single-hidden-neuron autoencoder can learn key phase-transition information and help locate the critical point.This extends the paper’s unsupervised-learning analysis beyond PCA.

V. CONCLUSIONS

PCA and autoencoders can identify phases, symmetry breaking, transition types, and critical behavior, but raw spin configurations can hide charge-like order and vortices.

  • Autoencoder methods can serve as useful unsupervised tools for studying phase transitions and distinguishing multiple physical scenarios.
  • PCA recognizes spatial order and symmetry breaking, with principal-component projections supporting estimates of transition temperatures and critical exponents.The dominant component can correspond to an obvious order parameter, while subleading components capture small-q behavior such as domain walls.
  • In the frustrated triangular-lattice Ising model, PCA reveals subtle incipient three-sublattice orders and their emergent XY symmetry despite the absence of an obvious order parameter.The two nearly degenerate ordering patterns are represented by principal components related by the emergent symmetry.
  • PCA distinguishes first- from second-order transitions and crossovers from phase transitions through the spread and degeneracy of low-variance projections.Comparable large variances and clustered scatter points also indicate the number of ordered phases and model symmetries.
  • PCA fails to detect BSI charge order and XY vortices from raw spins because these structures reside in squared or charge variables rather than the original spins.Applying PCA to squared local spin variables can reveal charge order or vortex proliferation, motivating analyses with different powers of the spin variables.
  • Real-space machine-learning analyses may connect to experiments in which different phases coexist across spatially varying samples.The paper gives confined cold atomic gases as an example where density and energy scales vary across the cloud.

Appendix: Autoencoder Architecture

The appendix details a convolutional autoencoder that processes raw 40 × 40 spin configurations, using convolution, pooling, and decoder upsampling layers. It also specifies alternative hidden-neuron paths and the training setup.

  • Encoder: The encoder uses Conv1 with 16 learnable 3 × 3 filters, stride 1, and zero padding of 1.The filters are spatially small and extend through the full depth of the input volume.
  • Encoder: Conv2 through Conv4 use 8 filters of size 3 × 3 with stride 1 and zero padding of 1.
  • Pooling and decoder: Max-pooling layers use 2 × 2 filters with stride 2, while the decoder replaces pooling with upsampling.The max-pooling operation returns the maximum value in each covered sub-region.
  • Decoder: Decoder layers Conv5 through Conv7 use 8 filters, followed by Conv8 with 16 filters and Conv9 with 1 filter.These decoder convolutions use size 3 × 3, stride 1, and zero padding of 1.
  • Latent representation: The architecture supports paths with 200, 2, or 1 hidden neurons, and uses raw spin configurations of fixed size N = 40 × 40 as input.The 200-neuron path is black-to-red-to-black; the 2- or 1-neuron path is black-to-blue-to-black.
  • Training: The network is trained for 100 epochs in each case using raw square-lattice Ising spin configurations.
Loading 1704.00080v2…