PaperlayerView exact revision on arXiv

Source-linked AI summary

Apparent Compression, Real Stability: The Intrinsic Dimension of Learning a Quantum Wavefunction

Lu Wei, Yufeng Wang, Chenfeng Cao, Haibin Ling

arXiv:2609.33193v1cs.LGquant-ph

TL;DR

How many trainable directions VMC needs is difficult to measure because its samples come from the network and its gradients are noisy. The paper trains a latent vector through a frozen random map and finds that intrinsic dimension tracks a quantum phase transition, while small values can be misleading and subspace training was stable in the reported experiments.

  • Problem

    How many trainable directions VMC needs remains to be measured in a setting where samples come from the network itself and gradients are noisy.

    Source: §1 · p. 1

  • Method

    The authors train a latent vector that a frozen random map converts into network weights, applying stochastic reconfiguration without changing the training loop.

    Source: §1 · p. 2

  • Results

    The measured dimension rises about 4× across the Ising-chain phase transition at two system sizes, while a non-negative-amplitude network reaches its best energy in 8 random directions despite being unable to reach a lower energy.

    Source: §1 · p. 3, §6 · p. 10

  • Takeaways & Limitations

    The measured dimension tracks the phase transition but remains above a floor set by the random slice, even deep in the paramagnet where the ground state is nearly a product state.

    Source: §1 · p. 3, §6 · p. 10

    The stability comparison was observed for one optimizer configuration, and the authors do not claim that full-parameter NQS training is generally unstable.

    Source: §4.4 · p. 9

Abstract

from arXiv · show

How many directions in weight space does training need? The intrinsic dimension answers this with the smallest number of random directions in which training still reaches a target accuracy, and small values have motivated parameter-efficient methods such as LoRA. We measure it for variational Monte Carlo (VMC), which trains a neural network to represent the ground state of a quantum many-body system. VMC is a demanding test, because the network generates its own training samples and every gradient is noisy, and a revealing one, because the exact answer is known and every run can be scored. We train only a small latent vector that a frozen random map turns into the network's weights, with no change to the standard natural-gradient optimizer. We find that a small dimension can be misleading, while the stability it brings is real. On a magnet with a hard sign pattern, a network that cannot represent signs reaches its best energy in 8 of 28,642 directions, but only because no such network can go lower; once signs are learnable, neither the signs nor the magnitudes are cheap. The dimension rises across a quantum phase transition, so it tracks how difficult a state is at far less compute than fitting a scaling law, yet it never falls below a floor set by the random subspace itself, even where the ground state is nearly trivial. Training in the subspace, in contrast, never diverged in our experiments, whereas full-parameter training with the same settings did, and a control with matched solvers attributes the difference to the reduced dimension.

1 INTRODUCTION

This paper measures intrinsic dimension in VMC, where gradients are noisy but exact ground-state energies make every run scoreable. It finds that low dimension can reflect a model-class bound rather than a learned ground state, while subspace training remains stable and dimension tracks a phase transition above a random-slice floor.

1 INTRODUCTION

  • The intrinsic-dimension measurement extends to VMC, where the network samples its own training configurations and exact ground-state energies allow every run to be scored.The method restricts training to a random subspace while leaving stochastic reconfiguration unchanged.Source: §1 · pp. 1–2
  • The nonlinear Mapping Networks-inspired map trails the linear map on the frustrated magnet and is comparable on the Ising chain.The paper proposes no new training algorithm; it measures intrinsic dimension and reports findings from that measurement.Source: §1 · p. 2
  • Subspace training never diverged in 246 sign-structure runs, while matched full-parameter training diverged 41 times in 35 runs at the larger lattice.A control using the full-parameter solver attributes the stability difference to reduced dimension; seed dependence of final energy remains.Source: §1 · p. 2
  • Eight of 28,642 directions reach the non-negative model’s best energy, but that energy is already the lowest any non-negative wavefunction can attain.Once signs are learnable, neither sign nor amplitude is cheap; with every parameter trainable, a learned sign costs 9× in error against an exact sign at equal steps.Source: §1 · pp. 2–3
  • On the Ising chain, d* rises about 4× across the phase transition at two sizes and falls only to 64 deep in the paramagnet.The threshold is one for which d* does not change when the training budget is tripled; the remaining floor reflects the random slice despite a nearly product-state ground state.Source: §1 · p. 3

2 RELATED WORK

Prior intrinsic-dimension work spans supervised learning, control, and language-model fine-tuning, while related work studies weight generation, quantum-state optimization, sign structure, and phase transitions. This paper distinguishes training dimension from data or weight geometry and uses exact ground truth for controls.

2 RELATED WORK

  • Intrinsic dimension has been measured in supervised vision, control, Atari, and language-model fine-tuning, where low-dimensional fine-tuning motivates LoRA.For RoBERTa on MRPC, 200 projected parameters reach 90% of full fine-tuning performance; LoRA uses 10,000 times fewer trainable parameters than full GPT-3 175B fine-tuning.Source: §2 · p. 3
  • Mapping Networks and HyperRBMs generate weights through fixed maps, but neither measures the smallest trainable dimension.The paper’s nonlinear comparison is inspired by Mapping Networks, while the related constructions have additional architectural or training features.Source: §2 · p. 3
  • Prior NQS work develops optimization methods and parameter-efficient transfer, while compute scaling laws assign Hamiltonians exponents at a cost of about 105 A100 GPU-hours.The paper instead measures trainable dimension in VMC.Source: §2 · p. 3
  • Work on sign structure identifies learning or optimization challenges, while this study measures sign cost inside VMC training in both dimensions and optimization.Related work also reports that an exact-sign model at 6 × 6 did not reach a more accurate state.Source: §2 · p. 3
  • Data intrinsic dimension has a minimum at continuous transitions, whereas training intrinsic dimension rises across the transition and relaxes to a random-slice floor.The paper presents training-dimension measurements as complementary to analyses of trained NQS weights.Source: §2 · pp. 3–4

3 METHOD: SUBSPACE-RESTRICTED VMC

The method trains a latent vector through a frozen random map and evaluates the smallest dimension meeting an error threshold under a fixed run budget. It retains VMC’s stochastic-reconfiguration geometry and adds training-health monitoring for noisy objectives.

3 METHOD: SUBSPACE-RESTRICTED VMC

  • Subspace-restricted VMC provides an intrinsic-dimension measurement, a budget protocol for distinguishing underallocated blocks from hard-to-learn ones, and a health guard for latent training.The protocol is presented as reusable for objectives that can only be sampled.Source: §3 · p. 4

3.1 SETUP AND NOTATION

  • The autoregressive NQS assigns amplitudes to configurations, and training minimizes variational energy estimated by Monte Carlo sampling against an exact ground-state energy.The study uses a frustrated Heisenberg model on a 4 × 4 lattice and transverse-field Ising chains of lengths 16 and 24.Source: §3.1 · p. 4

3.2 FOUR TRAINING CONFIGURATIONS

  • TC1 trains all parameters; TC2 uses a random linear subspace, TC3 adds a tanh after the map, and TC4 trains only the last parameter group.The random maps are frozen; TC3 omits features and auxiliary losses in the published Mapping Networks construction, so it may understate that full construction.Source: §3.2 · pp. 4–5, Table 1 · p. 4

3.3 STOCHASTIC RECONFIGURATION IN THE LATENT SPACE

  • Stochastic reconfiguration operates in latent space through the pullback of the weight-space quantum geometric tensor, without changing the VMC training driver.The study uses dense SR for latent dimensions up to 4096 and minSR for full-parameter training; a matched-solver control at 6 × 6 also logged zero divergence events.Source: §3.3 · p. 5

3.4 HEALTH PROTOCOL FOR NOISY-OBJECTIVE TRAINING

  • The health protocol detects non-finite or upward-jumping estimates and estimates that fall below the exact-energy floor beyond sampling noise.The floor guard matters because a frozen map can amplify downward estimator corruption that checks for NaNs and upward jumps miss.Source: §3.4 · p. 5

3.5 THE OBSERVABLE

  • The observable d* is the smallest dimension whose median seed error meets threshold τ at coupling λ under a fixed run budget.The median matters because random-subspace orientation selects among distinct attractors; d* depends on budget unless the threshold is in a regime where it becomes budget-independent.Source: §3.5 · p. 5

4 EXPERIMENTS

Subspace training remains stable under noisy VMC gradients, but a low measured dimension can reflect a representational bound rather than a simple ground state. Sign-capable controls show that both amplitudes and signs can require many dimensions; across the Ising transition, the measured demand rises while remaining bounded below by the random slice.

4.1 SUBSPACE TRAINING SURVIVES GRADIENT NOISE

  • At d = 8 of 28,642 parameters, sign-free training reaches εrel ≈0.5271 only at the positive-cone bound, so this apparent 3,500× compression says nothing about ground-state amplitudes.The bound is E = −3.99975, within 0.006% of the −4 diagonal minimum; any amplitude function concentrated on the classical Néel and stripe configurations attains it.Source: §4.1 · p. 6

4.2 AN EXACT-SIGN CONTROL SEPARATES AMPLITUDE FROM SIGN

  • With the exact sign supplied, median εrel falls from 0.37 at d = 8 to 0.052 at d = 512 and 0.031 at d = 2048, yet no subspace reaches 10^-2.The full amplitude network reaches 2.6 × 10^-3, showing that amplitude training remains costly once the positive-cone bound is removed.Source: §4.2 · p. 7
  • With all 1,633 sign dimensions trainable, error falls from 0.42 at 2 amplitude dimensions to 0.053 at 415 and 0.039 at 1,024; once amplitude is resolved, the sign needs its whole block.At fixed 1,024 amplitude dimensions, 402 sign dimensions give 1.24× the complete-block error, while 128 and 32 give 2.4× and 7.3×.Source: §4.2 · p. 7
  • The sign-capable full network reaches median εrel = 0.044 in 600 steps, versus 4.9 × 10^-3 for the same amplitude network given the exact sign, a 9× error gap.Both use the same minSR solver and equal steps.Source: §4.2 · p. 7
  • At both Ising-chain sizes, d∗ rises about 4× across the transition, with the step edge inside the finite-size critical window.The edge lies between h/J = 1.0 and 1.05 for L = 16 and between 1.05 and 1.1 for L = 24.Source: §4.3 · p. 7, Figure 3 · p. 8
  • Deep in the paramagnet, d∗ relaxes only to 64 at h/J = 4 and 10, although at h/J = 10 a parameter-free product state is within εrel = 2.5 × 10^-3 of the ground-state energy.At h/J = 10, a d = 32 random slice stalls at median error 3.4 × 10^-2, 14 times worse than the product state; the plateau is converged.Source: §4.3 · p. 8, Figure 3 · p. 8
  • At τ = 10^-2, all 7 cells shared by the base and deep protocols were unchanged under a 3× larger step budget, with 0 shrinking; the control fails at tighter thresholds.For the L = 24 grid, the artifact hypothesis was selected; no coupling requires d = 256, so the step survives.Source: §4.3 · p. 8

4.4 FULL-PARAMETER TRAINING BREAKS AT THE LARGER LATTICE

  • At 6 × 6, all 70 subspace runs completed without divergence, while full-parameter training logged 41 divergence events in 35 runs and had catastrophic median-seed outcomes at 4 of 7 couplings.At equal steps, the d = 3072 subspace beat the full-parameter median at those four couplings but lost where the reference median avoided the bad basin.Source: §4.4 · p. 9

5 DISCUSSION

The discussion warns that proportional parameter allocation can underserve a small but demanding block. It also presents subspace training as a stabilizer and intrinsic dimension as a cheap probe whose value includes a random-slice floor.

5.1 PROPORTIONAL ALLOCATION AND SMALL PARAMETER BLOCKS

  • A shape-proportional 28-of-512-dimensional sign allocation leaves error 5.9× above an exact-sign control; even a quarter of the sign block leaves median error 1.24× higher once amplitude is resolved.The sign block holds 5.4% of parameters; the 5.9× comparison uses the same amplitude allocation.Source: §5.1 · p. 9

5.2 IMPLICATIONS FOR NQS PRACTICE

  • Subspace training is a stabilizer, while d∗ tracks coupling difficulty at roughly two orders less compute but includes random-slice cost; the variational-floor guard is recommended when E0 or a good lower bound is known.The guard costs nothing when E0 or a good lower bound is known.Source: §5.2 · p. 9

6 CONCLUSION

The paper brings intrinsic-dimension measurement to VMC by training a neural quantum state in a random weight subspace and scoring runs against exact energies. It reports that apparent compression can reflect a variational bound, while subspace training is robust and measured dimension tracks a phase transition above a random-slice floor.

6 CONCLUSION

  • Subspace training never diverged in the sign-structure study, whereas full-parameter training with the same optimizer settings diverged repeatedly at the larger lattice.Every run is scored against an exact energy.Source: §6 · p. 9
  • A non-negative-amplitude network reaches its best energy in 8 random directions, but cannot reach a lower energy; once signs are learnable, neither signs nor amplitudes are cheap.The conclusion identifies repeating the exact-sign control across other network families as a next step.Source: §6 · p. 10
  • Measured dimension rises across a phase transition but remains above a floor set by the random slice, even when the ground state is nearly trivial.The paper proposes comparing intrinsic dimension with compute scaling laws on a shared Hamiltonian.Source: §6 · p. 10

A BOTH GATES AND THE SCOPE OF THE 6 × 6 FAILURE

The appendix rejects median-referenced gates when the full-parameter reference has a bad-basin median, and reports a robust-gate failure at 6 × 6. That failure is a joint saturated-budget result, demonstrated under a specific optimizer configuration rather than a general claim about full-parameter VMC.

A BOTH GATES AND THE SCOPE OF THE 6 × 6 FAILURE

  • A median-reference gate inflates to 0.31–0.57 when the median seed is in the bad basin, so those d∗c values are discarded; the robust gate is 2× the best reference seed.Under the robust gate, d∗c at 6 × 6 is right-censored at every coupling.Source: A BOTH GATES AND THE SCOPE OF THE 6 ×… · p. 12
  • d∗c is right-censored above 3072 at every 6 × 6 coupling under the robust gate, but the saturated-budget failure cannot be attributed to the sign block alone.At d = 3072 the sign block is fully covered, while the amplitude block receives only 799 of 28,642 dimensions.Source: A BOTH GATES AND THE SCOPE OF THE 6 ×… · p. 12
  • Figure 4 compares per-seed relative error across J2/J1: reference seeds spread by more than an order of magnitude, while the two subspace seeds stay within 3.3× of each other.At four couplings the median reference seed is in the shaded bad basin.Source: Figure 4 · p. 13
  • The instability comparison is limited to plain SGD under SR or minSR; it does not establish that full-parameter NQS training is generally unstable.Better-tuned or adaptive full-parameter optimizers may close the gap.Source: A BOTH GATES AND THE SCOPE OF THE 6 ×… · p. 13

B THE VARIATIONAL-FLOOR INCIDENT AND AUDIT

The appendix specifies health checks for noisy energy estimates and audits a below-floor incident that ordinary NaN and upward-jump checks missed. The audit found no silent contamination among the base-protocol runs.

B THE VARIATIONAL-FLOOR INCIDENT AND AUDIT

  • A health event is triggered by a non-finite estimate, an upward jump, a variational-floor violation, or an extreme downward jump when E0 is unknown.The floor and downward-jump guards are necessary because noisy objectives can otherwise make such violations look like convergence.Source: B THE VARIATIONAL-FLOOR INCIDENT AND… · p. 13
  • One run recorded a “best” energy of −4.7 × 10^7 below the variational floor, triggering four rollbacks and an abort after NaN and upward-jump guards missed the downward corruption.A milder floor violation could have passed as convergence without warning.Source: B THE VARIATIONAL-FLOOR INCIDENT AND… · p. 13
  • Exactly one of 147 audited base-protocol runs was below the floor—the known incident—and that seed was rerun.The audit found no silent contamination.Source: B THE VARIATIONAL-FLOOR INCIDENT AND… · p. 13

C EXACT REFERENCE ENERGIES AT 6 × 6

The 6 × 6 reference energies are exact ground-state energies obtained by symmetry-reducing the Sz = 0 sector without using point-group symmetry. Checks against smaller-lattice diagonalizations, an independent implementation, and published values validate the calculation; DMRG references remain variational upper bounds.

Exact diagonalization

  • The reference calculation uses the Sz = 0 sector reduced by zero-momentum translations and spin inversion, retaining every rotation sector.The sector contains 126,056,625 states and the Hamiltonian has 9.3 × 10^9 nonzero entries.Source: C EXACT REFERENCE ENERGIES AT 6 × 6 · p. 13
  • Lanczos converges the lowest eigenvalue to 10−12 relative in 58–118 steps, taking about ten minutes per coupling on one CPU node.The basis and matrix structure are shared across couplings.Source: C EXACT REFERENCE ENERGIES AT 6 × 6 · p. 14

Validation checks

  • The exact 6 × 6 energies are checked against full-space 4 × 4 diagonalization, an independent 4 × 8 implementation, and published N = 36 values.Agreement is to 3 × 10−13, 3 × 10−14, and the reported energies at J2/J1 = 0.5 and 0.55 are −0.503810 and −0.495178 per site.Source: C EXACT REFERENCE ENERGIES AT 6 × 6 · p. 14

DMRG comparison

  • DMRG torus references lie 1.6 × 10−5 to 1.5 × 10−3 relative above the exact energies, with the largest gap in the frustrated window.The DMRG energies are variational upper bounds.Source: C EXACT REFERENCE ENERGIES AT 6 × 6 · p. 14

D MORE ON THE FOUR TRAINING CONFIGURATIONS

Reruns show that which attractor a seed reaches is unstable, while the tested nonlinear map compares differently across the two systems: it underperforms the linear subspace on the frustrated magnet but is roughly comparable on the TFIM.

Reruns

  • Reruns change which attractor a seed reaches: at equal steps, 2 of 3 d = 8 seeds but 1 of 3 d = 32 seeds reach the lower attractor, and none do in fresh 1000-step reruns.The rerun medians are 0.717 and 0.646, respectively.Source: D MORE ON THE FOUR TRAINING… · p. 14

Four training configurations

  • On the TFIM, TC3 is roughly comparable to TC2 and is better at d = 2, h/J = 0.6, with best ε values of 5.6 × 10−2 versus a TC2 median of 2.6 × 10−1.Together with the frustrated-magnet comparison, this supports that the tested mapping variant's weakness is specific to the frustrated system, not VMC noise as such.Source: D MORE ON THE FOUR TRAINING… · p. 14
  • At matched dimensions on the frustrated 4 × 4 magnet, TC3 lies above TC2 throughout and is slowest at d = 8, while TC2 reaches the shared sign-free bound at d = 8.TC3 has median ε = 1.46 at d = 8 within 880 steps.Source: Figure 5 · p. 14, D MORE ON THE FOUR TRAINING… · p. 14

E ADDITIONAL FIGURES AND TABLES

The supplementary results map errors and dimension thresholds across frustration, sign–amplitude allocations, step budgets, and system sizes. The staircase tracks a phase-transition window at the stated threshold, but its interior maximum is a finite-size and discrete-threshold artifact.

Dependence on frustration

  • At d = 512, median error is 0.083–0.103 for J2/J1 ≤0.55 and 0.051–0.075 for J2/J1 ≥0.60; the ranges overlap at d = 2048.At d = 128 the corresponding ranges are 0.256–0.278 and 0.203–0.225; the 6 × 6 error at d = 3072 peaks at J2/J1 = 0.55 and relaxes 3× into the striped side.Source: E ADDITIONAL FIGURES AND TABLES · p. 15, Table 4 · p. 15, Figure 6 · p. 15

Sign/amplitude dissection

  • From 415 amplitude dimensions onward, the learned sign matches the exact-sign control; with amplitude fixed at 1,024 dimensions, a quarter of the sign block leaves median error 1.24× higher.With 2–128 amplitude dimensions, a 402-dimensional sign block is sufficient; from 415 amplitude dimensions on, the sign needs its whole block.Source: Table 5 · p. 16, Figure 7 · p. 16

Budget control

  • At τ = 10−2, all 7 shared staircase cells remain unchanged under a 3× larger step budget, while 0 cells shrink; at tighter thresholds the control does not hold.At 3 × 10−3 the outcome is mixed, and at 10−3 the one comparable cell moves from 128 to 64, so dimension claims are restricted to τ = 10−2.Source: E ADDITIONAL FIGURES AND TABLES · p. 16, Figure 8 · p. 17

Pre-registered falsification

  • The L = 16 interior maximum d∗(1.2) = 256 is confirmed with 5 seeds but is flagged because its median is within +4% of threshold; at L = 24 no coupling requires d = 256.The corresponding L = 24 cell reads 8.0 × 10−3, identifying the interior maximum as a finite-size and discrete-threshold artifact while the step survives.Source: E ADDITIONAL FIGURES AND TABLES · p. 17

Threshold dependence

  • Across τ ∈ [5 × 10−3, 2 × 10−2], the step location drifts, but wherever a step exists its edge remains inside h/J ∈[0.9, 1.2] at both sizes.Below the window, h/J ≤0.8, d∗ remains in {32, 64}; at fixed τ the staircase is a level cut of the smooth error surface.Source: E ADDITIONAL FIGURES AND TABLES · p. 17

Two-size staircase

  • The deep-protocol staircase edge lies inside the critical window at both sizes, while the demand relaxes to d∗ = 64 deep in the paramagnet.Table 6 reports d∗(h) per cell at τ = 10−2; daggers mark values decided by a median within ±10% of threshold.Source: Table 6 · p. 17

Error surface

  • The d∗ staircase is a level cut of median error versus field and dimension, with larger fields beyond the transition requiring systematically larger d for the same accuracy.This pattern is reported across the scanned window h/J ≤2.Source: Figure 9 · p. 18

F FURTHER RELATED WORK

Related NQS work measures effective directions through converged quantum-geometric spectra or pruning, while sign-focused studies use explicit phase constructions or separate amplitude and phase networks. These approaches differ from the paper's dense random slice.

NQS complexity measures

Sign structure

  • Sign-focused studies include explicit phase ansätze that find low-energy states obeying the Marshall sign rule, separate amplitude and phase networks, and convolutional NQS benchmarks on the J1–J2 model.Szabó & Castelnovo (2020) suggest such Marshall-rule states likely keep VMC from reaching the ground state.Source: F FURTHER RELATED WORK · p. 18

Related papers

Loading 2609.33193v1…