PaperlayerView exact revision on arXiv

Source-linked AI summary

Neural Quantum States of frustrated magnets: generalization and sign structure

Tom Westerhout, Nikita Astrakhantsev, Konstantin S. Tikhonov, Mikhail Katsnelson, Andrey A. Bagrov

arXiv:1907.08186v3cond-mat.dis-nncond-mat.str-elquant-ph

TL;DR

Neural-network generalization from a small set of basis states is not assured even when a network can express the target state. The authors test ground-state learning in frustrated magnets and find sign generalization especially difficult near transitions and under strong frustration.

  • Problem

    Whether neural quantum states can generalize ground-state structure from limited basis samples remains an important question even when the ansatz can express the state accurately.

    Source: RESULTS · p. 3

  • Method

    The authors train neural networks on limited basis-state samples and assess generalization, including separate trials that encode signs or amplitudes in the network.

    Source: RESULTS · p. 5, DISCUSSION · p. 6, METHODS · p. 7

  • Results

    Across the studied models, sign-structure generalization falls near quantum phase transitions and remains low in frustrated regimes, while amplitudes generalize more easily.

    Source: RESULTS · p. 5, DISCUSSION · p. 6

  • Takeaways & Limitations

    For the studied models, generalization rather than expressibility is the main issue for NQS, with sign learning especially difficult under strong competing interactions.

    Source: DISCUSSION · p. 6

    The authors could not verify whether full-training expressibility extends to larger clusters because training on all basis vectors for 30 spins was too resource-demanding.

    Source: RESULTS · p. 5

Abstract

from arXiv · show

Neural quantum states (NQS) attract a lot of attention due to their potential to serve as a very expressive variational ansatz for quantum many-body systems. Here we study the main factors governing the applicability of NQS to frustrated magnets by training neural networks to approximate ground states of several moderately-sized Hamiltonians using the corresponding wavefunction structure on a small subset of the Hilbert space basis as training dataset. We notice that generalization quality, i.e. the ability to learn from a limited number of samples and correctly approximate the target state on the rest of the space, drops abruptly when frustration is increased. We also show that learning the sign structure is considerably more difficult than learning amplitudes. Finally, we conclude that the main issue to be addressed at this stage, in order to use the method of NQS for simulating realistic models, is that of generalization rather than expressibility.

RESULTS

Across the studied magnets, neural-network generalization worsens in frustrated regimes, especially for wavefunction signs; expressibility remains strong, but learning depends sharply on training-set size.

RESULTS

  • On the test dataset, generalization quality is poor in frustrated regions; square-lattice quality shows a large, smooth dip for 0.4 < J2/J1 < 0.6, while the triangular-lattice minimum occurs near J2/J1 ≈ 1.25.Generalization is easy in ordered phases and notoriously hard in disordered phases.Source: Figure 2 · p. 4, RESULTS · p. 4
  • Convolutional networks show smaller performance dips than dense networks on square and triangular lattices, likely because the implementation accounts for translational symmetry..Source: RESULTS · p. 4
  • For 24-spin clusters, Stochastic Reconfiguration and supervised learning follow similar patterns across J2/J1.The paper reports a correlation between generalization quality and SR accuracy.Source: RESULTS · pp. 4–5, Figure 2 · p. 4
  • Generalization emerges abruptly as the training set grows; frustrated Kagome cases require substantially more training data than the ordered phase.At J2/J1=0.6, reasonable generalization is possible, but the required training fraction is larger.Source: RESULTS · p. 5, Figure 3 · p. 6
  • For 24-spin clusters, full-ground-state training yielded overlaps above 0.94 across all J2/J1 values, so expressibility was not the limiting issue.This capacity does not ensure good generalization from a small subset.Source: RESULTS · p. 5
  • The authors could not verify whether the full-training expressibility result extends to larger clusters because training all basis vectors for 30 spins was too resource-demanding.The 30-spin Hilbert-space dimension is approximately 1.5·10^8.Source: RESULTS · p. 5
  • Under frustration, generalizing wavefunction signs becomes harder than generalizing amplitudes, although this difficulty is distinct from the quantum Monte Carlo sign problem.With exact signs supplied, amplitude generalization remains decent even deep in the frustrated regime.Source: RESULTS · p. 5
  • The main generalization patterns persist for 30-spin clusters and a 36-spin square lattice, where training-size dependence also shows a sharp transition.For the square lattice, increasing from 24 spins (K ≃2.7·10^6) to 36 spins (K ≃9·10^9) required only a 10-fold increase in the critical training set.Source: RESULTS · pp. 5–6, Figure 4 · p. 6

DISCUSSION

Across the studied models and architectures, sign-structure generalization falls near phase transitions and remains low in frustrated regimes, even when the networks can represent the states accurately. The discussion distinguishes this challenge from amplitude learning and considers implications for training and phase-transition detection.

DISCUSSION

  • Across all studied models, sign-structure generalization falls near phase transitions and remains low in frustrated regimes, potentially impairing NQS convergence.The authors report that this pattern holds qualitatively across the studied models and network architectures.Source: DISCUSSION · p. 6
  • In the Kagome 2-layer dense network, generalization overlap decreases for both signs and amplitudes in the frustrated region, but sign learning is harder.The comparison is shown for the Kagome lattice.Source: Figure 5 · p. 6
  • Even simple networks generalize amplitudes and can express both signs and amplitudes well, so expressibility alone does not ensure sign generalization.The authors recommend prioritizing architectures that generalize ground-state signs on moderately sized test systems.Source: DISCUSSION · p. 6
  • Generalization changes abruptly at a threshold training-set size, unlike the smooth behavior noted for some other learning models.The authors identify understanding sample requirements across system size, frustration, and training parameters as an open question.Source: DISCUSSION · p. 7
  • A dense-minima bias in the loss landscape may partially explain the abrupt generalization change, but the authors present this as a possible explanation.They relate this possibility to stochastic gradient descent's known bias toward wide minima.Source: DISCUSSION · p. 7
  • Although the generalization dip is undesirable for variational energy optimization, it could help identify phase transitions without supervision.The authors compare this potential use in spirit to approaches [55].Source: DISCUSSION · p. 7

METHODS

The methods train separate neural networks to encode wavefunction signs or amplitudes from exact-diagonalization data. They use probability-weighted, small training and validation subsets, assess trial-state overlap, and optimize task-specific losses.

METHODS

  • Separate networks encode log amplitudes as real outputs and signs as probabilities for a binary classifier, with sign inferred as positive at p ≥ 0.5.The paper contrasts this sign representation with the approach of.Source: METHODS · p. 7
  • Training configurations are sampled from the exact-diagonalization ground state with probability proportional to |ψ_i|^2, using small training and validation fractions to mimic incomplete Hilbert-space coverage.The authors note that practical NQS methods also rely on Monte Carlo samples covering only a tiny fraction of the basis.Source: METHODS · p. 7
  • Performance is evaluated by overlap with the exact eigenstate, using exact amplitudes for the sign-network trial state and exact signs for the amplitude-network trial state.This setup assesses the learned component while fixing the other component to its exact value.Source: METHODS · p. 7
  • The sign classifier is trained with binary crossentropy, while the amplitude network minimizes a separate loss on predicted and exact log amplitudes.The amplitude-loss equation is identified as the objective for amplitude-network training.Source: METHODS · p. 7
  • Training uses Adam with learning rates around 10^-4–10^-3, typically 64- or 128-sample batches, and early stopping as the main regularizer.The authors report that an unsuitable batch size can increase the critical training fraction by an order of magnitude.Source: METHODS · p. 7

Supplemental Material

The supplemental material accompanies the article and provides additional information about its numerical analysis.

Supplemental Material

I. NETWORK ARCHITECTURES

The paper compares dense networks with a periodic convolutional architecture intended to encode translational symmetry. The architectures use distinct output structures for sign and amplitude prediction.

I. NETWORK ARCHITECTURES

  • The one-layer dense network uses 64 hidden ReLU units and outputs either log amplitude or two sign-class probabilities.For 24 spins it has 1,728 parameters; for 30 spins, 2,112.Source: §I · p. 10
  • The two-layer dense network uses 64 ReLU units in each hidden layer and has 5,888 parameters for 24 spins or 6,272 for 30 spins.Source: §I · p. 10
  • The periodic CNN applies to the square-lattice J1–J2 model and is designed to preserve translational invariance through periodic convolutions and spatial averaging.The authors state that its prediction is invariant under translations of the input spin configuration.Source: §I · pp. 10–11
  • For the CNN sign output, weighted spatial averaging distinguishes even and odd y-momentum behavior before mapping the result to sign probabilities.The odd case flips the sign under a y-direction shift, while the even case leaves it unchanged.Source: §I · p. 10

A. 24 spins

For 24-spin clusters, sign-structure generalization changes sharply with training-set size, while its threshold varies by lattice and frustration. In frustrated regimes, signs are harder to generalize than amplitudes, whose generalization varies smoothly with training-set size.

  • Sign-structure generalization has a sharp training-size transition on square and triangular lattices, with the critical size fixed across frustration for square but increasing with frustration for triangular and Kagome lattices.Source: Figure S1 · p. 12, A. 24 spins · pp. 12–13
  • In the frustrated regime, sign structure is more difficult to generalize than amplitude structure on square and triangular lattices.Source: Figure S2 · p. 12, A. 24 spins · p. 13
  • Kagome amplitude generalization varies smoothly with training-set size, unlike the sharp transition reported for sign structure.Source: Figure S3 · p. 13, A. 24 spins · p. 13

B. 30 spins

For 30-spin clusters, the reported frustration-dependent generalization patterns resemble those for 24 spins. Sign generalization improves abruptly beyond a critical training-set size, and signs remain harder to learn than amplitudes in frustrated regimes.

  • With a two-layer dense network, sign structure is more difficult to generalize than amplitude structure in the frustrated regime.Source: B. 30 spins · p. 13
  • For 30-spin clusters, generalization quality versus J2/J1 follows patterns similar to those observed for 24 spins.Source: B. 30 spins · p. 13, Figure S4 · p. 14
  • Sign generalization improves abruptly at a critical training-set size, and larger systems require a smaller basis fraction to begin generalizing than 24-spin systems.Source: B. 30 spins · p. 13, Figure S5 · p. 14

C. 32 & 36 spins

The 32- and 36-spin square-lattice results examine how sign-structure generalization depends on training-set size to estimate critical training-set scaling. A separate comparison covers sign and amplitude generalization across three lattices.

  • A separate comparison presents sign- and amplitude-structure generalization for square, triangular, and Kagome lattices using a two-layer dense architecture.Source: Figure S6 · p. 15
  • For square-lattice clusters with 32 and 36 spins, sign-structure generalization is plotted against training-set size as part of estimating critical training-set scaling with system size.Source: C. 32 & 36 spins · p. 14, Figure S7 · p. 15
  • The 32- and 36-spin square-lattice plots vary training-set size to assess sign-structure generalization.Source: Figure S7 · p. 15
Loading 1907.08186v3…