Fixed-point neural samplers on discrete spaces

• Discrete sampling is computationally intractable in many practical settings, and enumerating the state space is generally infeasible even when individual configuration densities can be evaluated.MCMC avoids evaluating the normalizing constant but may mix prohibitively slowly across high-energy barriers or narrow pathways.Source: §1 · p. 1
• Continuous neural samplers have advanced through fixed-point training procedures, but earlier normalizing-flow approaches had scaling issues for practical systems.Later diffusion methods used target-density gradients and reuse of evaluated configurations to improve efficiency in materials systems.Source: §1 · p. 1
• Each DIG-IN iteration collects data, smooths importance weights using ESS control, and updates parameters with weighted cross-entropy.Source: Figure 1 · p. 2
• Gibbs labels provide a strong learning signal that helps avoid mode collapse, while tempered importance weighting balances variance and bias.The method also has convergence guarantees in certain settings and extends to transport between pairs of distributions.Source: §1 · p. 2
• On Ising and Potts models, DIG-IN is the only method reported to avoid ordered-phase mode collapse while giving accurate ln Z estimates.The reported experiments also cover amortized sampling on lattices with up to D = 2500 sites and phase diagrams for two alloy systems.Source: §1 · p. 2

• A continuous-time Markov chain evolves by waiting a random time and then jumping; its generator gives instantaneous transition rates, with rows summing to zero.Source: §2 · p. 2
• The reverse-time generator reweights forward jump rates by the ratio of marginal probabilities, called concrete scores.The passage contrasts generation, which can use one time direction, with path-space importance reweighting and free-energy estimation, which generally require both generators.Source: §2 · p. 2
• In single-site processes, the rates specify when a jump occurs, which site changes, and its replacement symbol.Source: §2 · pp. 2–3
• Masked diffusion adds a mask symbol and progressively masks each site according to a decreasing schedule, leaving masked sites masked thereafter.Source: §2 · p. 3
• A categorical denoiser predicts each masked site's symbol from revealed sites, enabling generation by revealing one site at a time in any order.With target samples, it can be trained by masking tokens and using the observed symbols as one-hot reconstruction labels.Source: §2 · p. 3

3.1 FROM ENERGY DIFFERENCES TO GIBBS SOFT LABELS

• Gibbs soft labels use single-site energy differences to provide target-relative probabilities for alternative symbols, rather than only the sampled token.For target-distributed samples, this objective has the same optimum as one-hot training with lower per-site variance.Source: §3.1 · pp. 3–4

3.2 FROM FIXED-POINT CONSISTENCY TO FIXED-POINT CONVERGENCE

• Self-generated samples can bias training contexts, so DIG-IN tempers importance weights by an ESS-based exponent to trade controlled bias for reduced variance.When sampler-target overlap is poor, the correction is softened so a few configurations do not dominate optimization.Source: §3.2 · pp. 4–5
• Under finite-state, positivity, exact-sampling, and exact-fitting assumptions, fixed-order training converges globally for fixed positive tempering exponents and for ESS-adaptive exponents bounded away from zero.At ν = 1, one population update recovers the target predictor from any positive initialization.Source: §3.2 · p. 5

3.3 BEYOND MASKED DIFFUSION

• For general discrete interpolants, DIG-IN constructs mutually time-reversed forward and backward dynamics and learns their quantities from endpoint energy functions.The constructed flux uniquely minimizes probability-weighted kinetic energy among those satisfying the Kolmogorov equation.Source: §3.3 · pp. 5–6
• For Potts 16 × 16, q = 3, the DIG-IN row reports ESS 9.44±1.34, |∆ln Z| .974±.007, and TVDmode .014±.011 in one evaluation condition.The table reports results across three inverse temperatures; its caption defines the metrics and reports mean ± standard deviation over three seeds.Source: Table 1 · p. 7
• In the interpolant training setup, intermediate states are sampled from endpoint pairs, whose energy-based labels can be reused across times and whose generated endpoints receive ESS-tempered importance weights.The passage describes this weighting because the target endpoint is generated by the current sampler rather than drawn from the target distribution.Source: §3.3 · p. 7

Neural samplers on discrete spaces

• DIG-IN combines Gibbs soft-label supervision with tempered importance weighting among discrete neural samplers, including autoregressive models, GFlowNets, and CTMC methods.The passage characterizes this combination as supporting stable, scalable training and mitigating mode collapse.Source: §4 · p. 7

Discrete interpolant and flow models

• Unlike data-trained approaches discussed here, DIG-IN learns local conditional distributions and temporal potentials from endpoint energies for general interpolants.The related-work passage notes that prior discrete flow and generator-matching methods typically do not learn mutually time-reversed dynamics.Source: §4 · p. 7
• Figure 2 reports amortized training across (β, h), with low TVDmode indicating mode coverage and accurate relative mode weights across conditions.Its panels show Potts samples at h = 0 across phases and average ESS, |∆ln Z|, and TVDmode over a 9 × 9 grid.Source: Figure 2 · p. 8

5 EXPERIMENTS

• The experiments cover Ising and Potts lattice spin models, larger lattices with amortized sampling over two thermodynamic parameters, and two alloy phase diagrams.The paper also points to further experiments, including a component ablation and a bridge between two Boltzmann distributions.Source: §5 · p. 8

5.1 ISING AND POTTS LATTICE SPIN MODELS

• DIG-IN avoids mode collapse in the ordered Ising and Potts phases while giving accurate ln Z estimates; LEAPS also avoids collapse, but its ln Z error is high.Away from the ordered phase, DIG-IN leads in |∆ln Z| except for critical Potts, where MDNS and PDNS are marginally better.Source: §5.1 · p. 8

5.2 AMORTIZED TRAINING

• Across thermodynamic conditions and a high-dimensional phase transition, DIG-IN achieves high ESS, low ln Z error, and low TVDmode, indicating mode coverage and correct relative mode weights.The sampler is conditioned on (β, h) for the Ising and Potts models.Source: §5.2 · pp. 8–9

5.3 ALLOY PHASE DIAGRAMS

• The alloy systems comprise Ag–Pd with one miscibility gap and Cu–Au with three ordered phases, evaluated over T ∈ [200, 900] K and ∆µ ∈ [−0.4, 0.4] eV.Each method trains for 60k steps per system on the same energy, grid, and conditioning distribution.Source: §5.3 · p. 9
• DIG-IN correctly recovers both Ag–Pd and Cu–Au phase boundaries without spurious boundaries, while FrOGS misses boundaries and adds spurious ones.The comparison uses one amortized sampler per alloy, conditioned on (T, ∆µ), with three seeds and 5,000 samples per grid point.Source: §5.3 · p. 9, Figure 3 · p. 9

6 CONCLUSION

• DIG-IN uses Gibbs labels from single-site energy differences as lower-variance supervision for a masked denoiser, then trains on self-generated configurations with ESS-tempered importance weights.The same Gibbs supervision extends to general discrete interpolants between two distributions.Source: §6 · p. 10
• Under exact sampling and exact population fitting, the iteration converges to the target for fixed reveal orders at any tempering exponent and for arbitrary reveal orders when the exponent is sufficiently close to one.These are guarantees for specified settings, not for every practical training setup.Source: §6 · p. 10
• DIG-IN avoids mode collapse in ordered Ising and Potts phases while giving accurate ln Z estimates, scales to amortized sampling on lattices up to D = 2500, and recovers Ag–Pd and Cu–Au phase diagrams.Source: §6 · p. 10
• Gibbs labels require single-site energy differences; for black-box energies, computing them requires D(N − 1) + 1 evaluations per configuration.For local energies such as lattice models and cluster expansions, the table costs about one energy evaluation.Source: §6 · p. 10
• The convergence guarantee does not cover all practical experimental settings, and the benefits of general interpolants and applications in biology and chemistry remain for future work.Source: §6 · p. 10

AI USE STATEMENT

• LLMs helped draft parts of the paper from the authors’ notes and polish grammar and consistency.Source: AI USE STATEMENT · p. 10
• LLMs helped develop preliminary proofs and implement parts of the experiment code and figure and table scripts.Source: AI USE STATEMENT · p. 10
• LLMs helped with literature search and reviewed the draft for errors; the authors state that the research question, method, and experimental design originated with them.Source: AI USE STATEMENT · p. 10

A.1 DAMPING THE FIXED-POINT ITERATION

• Damping mixes the Gibbs label with the current model prediction; with exact minimization, η = 1 recovers the undamped update, and damped and undamped iterations share fixed points.The practical EMA model is also used to collect samples.Source: §A.1 · p. 17

A.2 IMPORTANCE WEIGHTING AND FREE ENERGY ESTIMATION

• For any-order generation, importance weights follow the sampled reveal order, whose probability cancels when the order is sampled independently of token values; general forward-backward weights include transition-density ratios and holding steps.Source: §A.2 · p. 17
• For independent masked-diffusion or any-order samples, importance weights estimate free energy; the passage also identifies ∆F = −log(Z1/Z0) as an escorted Jarzynski and Crooks instance in discrete space.Source: §A.2 · p. 18

B.1 ITERATIVE PREDICTOR UPDATES AND CONVERGENCE STATEMENT

• The convergence analysis studies an idealized finite-state population iteration with a positive target and exact sampling and fitting; for arbitrary reveal orders, full weighting recovers the target heads, while a sufficient convergence threshold is ν > 1 − 1/(2D).The threshold is stated for a fixed ν under the analyzed assumptions.Source: B CONVERGENCE OF DIG-IN · p. 18, §B.1 · p. 19
• With a fixed reveal order, every fixed ν ∈ (0, 1) converges globally at rate O(kD−1(1 − ν)k); label-only fitting also converges globally from every positive initialization.Source: §B.1 · p. 19

B.2 GIBBS-LABEL FITTING DOES NOT AMPLIFY LOG-RATIO ERROR

• Gibbs-label fitting does not amplify the oscillation of the log-ratio to the optimal categorical predictor, making its associated operator non-expansive.Source: §B.2 · pp. 19–20

• For every tempering exponent, substituting the exact target predictor into the update makes it a fixed point.Its endpoint marginal is the target distribution.Source: §B.3 · p. 20
• At ν = 1, full importance weighting makes the next update recover the exact target predictor in one step.Source: §B.3 · p. 20
• When ν > 1 − 1/(2D), repeated contraction makes the predictor converge to the target at supported contexts and the marginal proposal converge to the target.Source: §B.3 · p. 21
• In the contraction regime, the fixed point is unique on all training-supported contexts.Source: §B.3 · p. 22
• The 2D factor is a sufficient worst-case bound that can be conservative in high dimensions, not a prescription for empirically optimal tempering.Source: §B.3 · p. 22

• For earlier sites, updated-head error is controlled by the discrepancy between the softened proposal’s and target’s suffix laws.Source: §B.4 · p. 22
• At the final site, the update sets the head exactly to the target conditional distribution.Source: §B.4 · p. 22
• The error bound vanishes at every site because its exponential factor dominates the polynomial factor.Source: §B.4 · p. 24
• Exact Gibbs-label fitting converges globally for every positive initialization under a fixed reveal order.Source: §B.4 · p. 24
• The argument also covers varying exponents bounded away from zero, with the corresponding error bound tending to zero.Source: §B.4 · p. 24

B.5 GLOBAL CONVERGENCE OF LABEL-ONLY FIXED-ORDER FITTING: PROOF OF THEOREM B.1(iv)

• With strict positivity on the finite product space, exact label-only fitting (ν = 0) converges globally to the target heads and joint proposal under a fixed reveal order.The proof uses uniform contraction of Gibbs transition kernels and backward induction through the reveal order.Source: §B.5 · pp. 24–25

B.6 ESS ADAPTATION PRESERVES FIXED-ORDER CONVERGENCE

• ESS-selected exponents stay uniformly positive, so exact population updates converge; after the proposal approaches the target, the rule selects ν = 1 and one update recovers the target.This result assumes the finite batch selects ν while the fitting update itself remains an exact population fit; using the batch for fitting is not covered.Source: §B.6 · pp. 25–26

B.7 TEMPERING SCALES THE LOCAL JACOBIAN

• At the target, tempering scales the population-update Jacobian by 1 −ν; (1 −ν)s0 < 1 ensures local asymptotic stability, while ν = 1 makes the Jacobian vanish.The local condition concerns small perturbations and can be weaker than the global sufficient condition.Source: §B.7 · pp. 27–28

B.8 ROBUSTNESS TO APPROXIMATE POPULATION FITTING

• Under the stated contraction regime, vanishing fitting errors preserve convergence, whereas persistent bounded errors guarantee convergence only to a controlled neighborhood.This conditional result does not establish that a particular neural architecture, replay buffer, or optimizer achieves the required error sequence; the argument also relies on strict positivity.Source: §B.8 · p. 28

C GENERAL DISCRETE INTERPOLANTS

• Section C recaps the general forward and backward CTMC construction, its minimum-kinetic-energy local flux, and the resulting training objective.It identifies the endpoint conditional used in the construction as minimizing kinetic energy.Source: C GENERAL DISCRETE INTERPOLANTS · p. 28, §C.1 · p. 29, §C.2 · p. 29

C.1 MINIMUM KINETIC ENERGY LOCAL FLUX

• The minimizing local flux is the product of the two endpoint conditional probabilities at the coordinate being updated.The construction imposes the Kolmogorov constraint while minimizing kinetic energy.Source: §C.1 · p. 29

C.2.1 LEARNING pd

• The interpolant-learning derivation uses Gibbs labels for supervision, applies cross-entropy fitting, and rearranges expectations to train without simulation or sampling from endpoint posteriors.The section also derives the objective's form after tempered importance reweighting.Source: §C.2 · p. 29, §C.2.1 · p. 30
• The p_d derivation follows the same three-step procedure, with its final step justified because the remaining term does not depend on newly added x_d.Source: §C.2.1 · pp. 31–32

C.2.3 FINAL OBJECTIVES

• When endpoint samples are unavailable from one distribution, the objective uses current-sampler samples for that endpoint with tempered importance weighting and ESS control.Only direct samples from one endpoint are assumed in this practical fixed-point iteration.Source: §C.2.3 · p. 33

D FURTHER RELATED WORK

• The method retains MDNS’s importance-weighted denoising cross-entropy but replaces one-hot targets with Gibbs soft labels, converting single-site energy differences into categorical supervision without differentiable energy extensions.The local energy information is used alongside tempered importance weighting.Source: D FURTHER RELATED WORK · p. 33
• For a sampled single-site reveal order, generation probabilities form a tractable product, enabling exact importance weights on an augmented path-and-order space, but not for the random-order marginal.The setting uses an unnormalized-energy target without target-distributed training samples.Source: D FURTHER RELATED WORK · p. 34
• The interpolant framework uses Gibbs labels for energy-based training between distributions, unlike prior data-trained flow or generator matching and endpoint-interpolant dynamics (Gat et al., 2024; Holderrieth et al., 2025b; He et al., 2026).Source: D FURTHER RELATED WORK · p. 34

E.1.1 TASKS

• The experiments use periodic two-dimensional lattices of q-state spins; the ordered q=3 Potts phase has three symmetry-related modes.The Ising model is treated as the q=2 Potts energy, and reference values come from a long Monte Carlo chain with cluster updates.Source: §E.1.1 · p. 34

E.1.2 EVALUATION METRICS

• Evaluation uses 8192 independent configurations per method, importance weights, and mode assignments; the reference mode distribution is uniform at zero field and estimated from samples when a field breaks symmetry.Ising modes are assigned by magnetization sign, while Potts modes are assigned by majority color.Source: §E.1.2 · pp. 34–35

E.1.3 HYPERPARAMETERS

• The DIG-IN, MDNS, and PDNS setups distinguish Gibbs versus endpoint-token labels, tempered versus full importance weights, and proximal damping; FrOGS runs use 60k steps rather than its published 200k configuration.DIG-IN uses an ESS target of 0.8; the alloy FrOGS configuration also differs from the published setup in other specified hyperparameters.Source: §E.1.3 · pp. 35–36, §E.3.4 · p. 39

E.1.4 COMPUTATIONAL COSTS

• For black-box energies, constructing the single-site difference table costs D(q − 1) + 1 energy evaluations per configuration, rather than one call for a local lattice energy.The reported black-box accounting uses 626 calls per Ising table and 513 per Potts table.Source: §E.1.4 · p. 36

E.2.1 SETUP

• For amortized Ising training, half the condition samples come from an adaptive acquisition map, which changes training-condition allocation but not labels, loss, or the fixed point.The acquisition scores grid cells using the trace of the target family’s Fisher information, estimated from model draws.Source: §E.2.1 · pp. 36–37

E.3.1 TASKS

• The alloy tasks use binary occupations on Ag–Pd and Cu–Au lattices, with DIG-IN and FrOGS trained over temperature and chemical-potential conditions and evaluated against exact reference values.The task description specifies DFT-fitted cluster expansions and an evaluation sweep with 5000 draws per state point.Source: §E.3.1 · pp. 37–38

E.3.2 CONSTRUCTING THE PHASE DIAGRAM

• DIG-IN’s draw-based free-energy curve reproduces the exact Cu–Au tie lines: 4 of 4 at 200 K and 2 of 2 at 700 K.The construction uses the lower convex hull of estimated free energy and criteria for accepting tie lines.Source: §E.3.2 · p. 39

F FURTHER EXPERIMENTS

• In the ordered Ising phase, only full DIG-IN is both mode-complete and accurate; removing damping or Gibbs labels leads to collapse or failure to order.Without importance weights, both modes survive but |∆ln Z| rises from 0.7±0.4 to 15.8±9.9 ×10−3 nats.Source: COMPONENT ABLATION · p. 42, Table 8 · p. 42

F.1 BRIDGING TWO BOLTZMANN DISTRIBUTIONS

• The two Ising transports recover the free-energy difference within 0.5±0.6 and 10.0±6.3 ×10−4 of its value, and reweighted draws split nearly evenly between modes.The ordered-to-ordered transport has lower ESS because paths pairing opposite modes lose their order midway.Source: §F.1 · p. 43

Amortized lattice models

• The lowest amortized-lattice ESS occurs at and just below the zero-field critical point: 0.20 for Ising and 0.56 for Potts.Figure 8 reports ESS as a fraction of 5000 draws per (β, h) cell, averaged over seeds.Source: §F.2 · p. 43, Figure 8 · p. 44

Alloys

• DIG-IN’s ESS stays high across the alloy sweeps and falls only at a small fraction of state points, mostly on the coldest isotherms; FrOGS’s is low almost everywhere.Figure 9 reports the fraction of 5000 draws per state point, averaged over three seeds, for Ag–Pd and Cu–Au.Source: §F.2 · p. 43, Figure 9 · p. 45