Source-linked AI summary
Kernel methods for interpretable machine learning of order parameters
Pedro Ponte, Roger G. Melko
TL;DR
In condensed matter, learning phase labels from large configuration datasets is useful, but interpreting learned features in terms of physical theory remains challenging. This paper tests SVMs on simulated two-dimensional spin models and finds that they learn interpretable order parameters and Hamiltonian constraints. The results also show that regularization affects whether the learned decision function is physically interpretable.
Problem
In condensed matter, relating learned phase-classification behavior to theoretical structure is challenging, particularly for more complicated models and non-linear or non-local order parameters.
Source: §I · p. 1Method
The paper applies SVMs with kernel choices to simulated configurations from three two-dimensional spin models to test whether they learn physical discriminators.
Source: §I · p. 1Results
SVMs learn physical discriminators across the studied spin models, including the Ising model's squared magnetization, for which the quadratic kernel reaches ≈97% mean test accuracy at L = 40.
Source: §I · pp. 1–2, A. 2d Ising Model · p. 3Takeaways & Limitations
SVM kernel choices can support interpretable classification by identifying physical discriminators, and the least complex near-optimal model is associated with the expected physical order parameter.
Source: §IV · p. 7For the Ising gauge theory, learning the physically relevant decision function can require prohibitively many samples; SVM training also becomes time-consuming above 10^5 samples.
Source: §IV · p. 7
Abstract
from arXiv · showhide
Machine learning is capable of discriminating phases of matter, and finding associated phase transitions, directly from large data sets of raw state configurations. In the context of condensed matter physics, most progress in the field of supervised learning has come from employing neural networks as classifiers. Although very powerful, such algorithms suffer from a lack of interpretability, which is usually desired in scientific applications in order to associate learned features with physical phenomena. In this paper, we explore support vector machines (SVMs) which are a class of supervised kernel methods that provide interpretable decision functions. We find that SVMs can learn the mathematical form of physical discriminators, such as order parameters and Hamiltonian constraints, for a set of two-dimensional spin models: the ferromagnetic Ising model, a conserved-order-parameter Ising model, and the Ising gauge theory. The ability of SVMs to provide interpretable classification highlights their potential for automating feature detection in both synthetic and experimental data sets for condensed matter and other many-body systems.
I. INTRODUCTION
Interpretable machine learning is important in physics because data patterns are meant to connect to theoretical postulates. This paper tests whether SVMs can learn physical discriminators from simulated spin configurations, including order parameters and local Hamiltonian constraints.
I. INTRODUCTION
- Interpreting machine-learning results matters in physics because connecting discovered patterns to theoretical postulates is part of the search for physical features.The introduction notes that interpretable results are desired when machine learning is intended to drive theoretical progress.Source: §I · p. 1
- For more complicated models, relating neural-network structure to non-linear or non-local order parameters is challenging, motivating evaluation of interpretable algorithms.The introduction notes prior success on the two-dimensional Ising magnetization, but says similar interpretations are generally challenging.Source: §I · p. 1
- SVMs are tested for learning the mathematical structure of physical order parameters while classifying simulated phase data from three two-dimensional spin models.The models are the ferromagnetic Ising model, the conserved-order-parameter Ising model, and the Ising gauge theory.Source: §I · p. 1
- The Ising gauge theory has a zero-temperature ground state defined by local plaquette constraints, and its discriminator encodes whether four-spin constraints are satisfied.The introduction says no polynomial of order less than four identifies the ground state.Source: §I · pp. 1–2
II. SUPPORT VECTOR MACHINES
SVMs classify samples by learning a separating hyperplane, with training determined by a margin-based optimization. Their dual form permits kernel choices that represent nonlinear features without explicitly constructing them.
II. SUPPORT VECTOR MACHINES
- An SVM predicts one of two classes by finding a hyperplane that separates the input samples.Here, inputs are spin configurations and labels identify the thermodynamic phase from which each configuration was sampled.Source: §II · p. 2
- The optimization uses slack variables to permit margin violations, with regularization balancing training errors against model complexity.A sample with slack greater than one is misclassified; slack-free training is the hard-margin case and requires linearly separable classes.Source: §II · p. 2
- The decision function assigns a new input to a class by the sign of its score, while the model does not provide probabilistic predictions.The learned parameters are used to make predictions on a test set.Source: §II · p. 2
- The dual formulation makes predictions using inner products between training and new samples, allowing those products to be replaced by a kernel.A degree-d polynomial kernel corresponds to monomials of the raw features up to order d, without explicitly generating those features.Source: §II · p. 2
III. RESULTS
The results section evaluates SVM phase classification on simulated finite-lattice spin configurations. It explores polynomial kernels and regularization, reporting test accuracy across training-set sizes with repeated train/test selections.
III. RESULTS
- The study applies SVMs to finite-lattice configurations sampled from classical Hamiltonians, using Ising spins as input features.The described training uses scikit-learn's Sequential Minimal Optimization algorithm, whose scaling is reported as O(N^2) or a smaller power for several kernels and data types.Source: §III · p. 2
- Polynomial-kernel behavior is explored, and grid search selects the regularization parameter C; training is generally slow on one core at 10^5 samples or more.Results are generally averaged across several train/test set choices for the same hyperparameters.Source: §III · p. 3
A. 2d Ising Model
For the 2d Ising model, the quadratic-kernel SVM classifies phases well and learns a decision function that tracks the squared magnetization. The linear kernel does not show the same accuracy improvement.
A. 2d Ising Model
- ≈97% mean test accuracy at L = 40 for the quadratic kernel, which reaches significant performance with only a few dozen training samples.The learned discriminator corresponds to the squared magnetization per spin, m2.Source: A. 2d Ising Model · p. 3, Figure 1 · p. 3
- The quadratic-kernel accuracy plateau increases with system size toward 100%, consistent with vanishing order-parameter fluctuations at the critical point in the thermodynamic limit.The passage states that perfect phase discrimination is then possible.Source: A. 2d Ising Model · p. 3
- The linear kernel’s accuracy is non-monotonic with training-set size and does not improve with increasing system size because a linear decision function cannot discriminate the phases.Source: A. 2d Ising Model · p. 3
- With C = 10^-6, the quadratic-kernel decision function agrees essentially perfectly with m2 after a linear rescaling; at C = 10^6, it instead sums local squared-magnetization contributions.The scale and offset are matched to ⟨m2⟩ at T = 1.6 and T = 2.9.Source: Figure 1 · p. 3, A. 2d Ising Model · p. 4
B. Conserved-order-parameter Ising model
For the conserved-order-parameter Ising model, quadratic SVMs learn a decision function that agrees with the analytical order parameter, including when trained with an assumed critical temperature different from the true one. The results also show that high classification accuracy alone does not ensure that the learned function matches the physical order parameter.
B. Conserved-order-parameter Ising model
- The model restricts the Ising configuration space to zero total magnetization and has a phase transition at the 2d Ising model’s critical temperature.It describes a half-filled lattice gas with nearest-neighbor attraction.Source: B. Conserved-order-parameter Ising model · p. 4
- SVMs are explored as a more systematic route to automated order-parameter detection than PCA-based feature-dimension reduction.PCA had provided the visualization used to determine the order parameter.Source: B. Conserved-order-parameter Ising model · p. 5
- The quadratic SVM decision function closely matches the analytical order parameter, while a linear kernel shows no clear accuracy improvement as training samples increase.For the quadratic kernel, test accuracy improves monotonically with sample count; the spatial dependence of the learned function at C = 0.0089 agrees well with the analytical form.Source: B. Conserved-order-parameter Ising model · p. 5
- Decision functions learned assuming T′c = 1.85 or 2.05 also agree well with the analytical order parameter, despite differing from the true critical temperature.The comparison uses T′c = 1.85, 2.05, and 2.269.Source: B. Conserved-order-parameter Ising model · p. 5
- At C ≳ 10^-3, test accuracy rises to about 90% and plateaus, but the learned decision function changes across the plateau.At C = 0.0089 the function agrees closely with the analytical order parameter, whereas at C = 1000 it does not.Source: B. Conserved-order-parameter Ising model · p. 5
C. 2d Ising Gauge theory
For the 2d Ising gauge theory, fourth-order polynomial SVMs classify ground states against infinite-temperature states, while lower-degree kernels fail. The learned decision function contains four-spin products whose coefficient pattern corresponds to the lattice’s plaquette structure.
C. 2d Ising Gauge theory
- The studied gauge theory has no conventional order parameter distinguishing ground states from finite-temperature states, and earlier feed-forward networks could not classify raw configurations.Convolutional networks succeeded by exploiting locality and translational invariance.Source: C. 2d Ising Gauge theory · p. 6
- For L = 5, only the fourth-order polynomial kernel discriminates ground states from infinite-temperature states above random guessing, and accuracy approaches 100% as training samples increase.The number of samples needed to learn the fourth-order discriminator increases with system size.Source: C. 2d Ising Gauge theory · p. 6, Figure 5 · p. 6
- The fourth-order decision function contains products of four spins, and 600 large-magnitude coefficients equal 25 plaquettes × 4! index permutations.This coefficient pattern provides evidence that the SVM learns the Hamiltonian’s locality from raw configurations.Source: C. 2d Ising Gauge theory · p. 6
IV. CONCLUSIONS
The paper finds that SVM kernels can provide interpretable physical discriminators for several condensed-matter models. The learned decision function depends strongly on regularization, while sample demands and training time constrain the method.
IV. CONCLUSIONS
- For the L = 5 Ising gauge example, 600 large fourth-order coefficients correspond to the 25 plaquettes and spin permutations.The coefficients are from classifying ground states against infinite-temperature states.Source: Figure 6 · p. 7
- SVMs can identify interpretable physical discriminators, including conventional order parameters, across the studied models.The authors contrast this kernel-based interpretability with neural networks.Source: §IV · pp. 6–7
- The expected physical order parameter appears in the least complex model that still achieves near-optimal test accuracy.The amount of regularization strongly affects the learned decision function.Source: §IV · p. 7
- Small regularization can yield equally or better-performing nonphysical decision functions that generalize to uncorrelated test samples.The authors argue this is not overfitting, while noting that such functions may reflect details of the training set.Source: §IV · p. 7
- Learning the physically relevant Ising gauge discriminator can require prohibitively many samples, and SVM training becomes time-consuming above 10^5 samples.The authors identify possible challenges near phase transitions and suggest exploring neural-network scalability for such cases.Source: §IV · p. 7
- The authors propose exploring SVMs on other classical models, with alternative kernels for continuous degrees of freedom, and on quantum phases using wavefunctions or density matrices.These are proposed directions, not results demonstrated in this study.Source: §IV · p. 7