Source-linked AI summary
QuanVI: Score-based Variational Inference via Quantum Maximally Mixed States
Yuchen Cong, Zerui Tao, Chao Li, Zhe Sun, Qibin Zhao
TL;DR
EigenVI’s score-based VI formulation faces exponential scaling and unstable eigenvector selection in degenerate low-energy subspaces. QuanVI combines mixed-state density operators with QTN parameterization, and experiments report strong performance across synthetic and Bayesian posterior benchmarks while identifying local-structure and contraction-cost boundaries.
Problem
EigenVI’s eigenvalue formulation faces exponential growth with dimension and non-unique, perturbation-sensitive eigenvectors in nearly degenerate low-energy subspaces.
Source: §1 · p. 1, §3.1 · p. 3Method
QuanVI represents low-energy eigenspaces with mixed-state density operators and uses a compact QTN parameterization rather than an explicit global operator.
Source: §1 · p. 2Results
Takeaways & Limitations
The reported results and ablations identify expressiveness–cost trade-offs involving environment size, local window size, basis size, and numerical parameterization.
Source: §7 · p. 9QuanVI relies on local structure; larger local windows capture stronger dependencies but increase contraction cost.
Source: §7 · p. 9
Abstract
from arXiv · showhide
Score-based variational inference (VI) provides an alternative to Kullback--Leibler (KL)-based VI by minimizing the Fisher divergence between the variational distribution and the target. A prior score-VI approach formulates this optimization as an eigenvalue problem, with the variational distribution constructed from low-energy eigenstates. However, this eigenvalue-based formulation faces two high-dimensional obstacles: an intractably large parameter count due to exponential scaling and non-uniqueness of individual eigenvectors in degenerate or nearly degenerate low-energy subspaces. We propose QuanVI, a scalable quantum-inspired algorithm that combines a mixed-state density-operator formulation with a quantum tensor network (QTN) parameterization using the matrix product operator (MPO) structure. In degenerate low-energy subspaces, the density-operator formulation represents the subspace by its maximally mixed state rather than relying on a non-unique individual eigenvector, while the QTN parameterization compresses the density operator to avoid exponential parameter growth. Experiments and ablations show that QuanVI agrees with exact solutions in low dimensions and scales to high-dimensional synthetic and Bayesian posterior-approximation benchmarks, including challenging non-Gaussian targets.
1 Introduction
Score-based VI minimizes Fisher divergence, but EigenVI’s eigenvalue formulation faces exponential scaling and unstable eigenvector selection in degenerate low-energy subspaces. QuanVI combines mixed-state density operators with compact QTN parameterization and maintains strong approximation quality across high-dimensional synthetic and Bayesian benchmarks.
1 Introduction
- EigenVI turns Fisher-divergence minimization into an eigenvalue problem, but its coefficients grow exponentially with dimension and its selected eigenvector can be unstable in nearly degenerate low-energy subspaces.Score-based VI matches score functions rather than density values, making it attractive when normalized densities are difficult to compute.Source: §1 · p. 1
- QuanVI replaces individual eigenvectors with mixed-state density operators and uses a compact QTN parameterization to represent low-energy eigenspaces without storing the full KD × KD operator.The density operator can represent a degenerate subspace by its maximally mixed state; the MPO perspective motivates its compact representation under the local Markov structure introduced later.Source: §1 · p. 2
- Across synthetic targets and POSTERIORDB Bayesian benchmarks, QuanVI maintains strong approximation quality while avoiding EigenVI’s global eigendecomposition bottleneck.The evaluations cover high-dimensional scalability, structured non-Gaussian expressiveness, posterior approximation, and architectural and optimization ablations.Source: §1 · p. 2
2 Related work
QuanVI builds on score-based VI and tensor-network representations, differing from EigenVI’s pure-state formulation through its use of mixed-state density operators suited to degenerate eigenspaces.
2 Related work
- EigenVI represents a variational density by basis expansion and casts Fisher-divergence minimization as an eigendecomposition, while QuanVI extends this pure-state approach to mixed-state tensor-network representations.EigenVI’s scalability and stability challenges in high dimensions motivate QuanVI’s extension.Source: §2 · p. 2
- Unlike prior pure-state Born representations that square a tensorized amplitude, QuanVI uses a mixed-state density operator, which is better suited to degenerate eigenspaces in eigenvalue-based score-VI.Tensor networks, including tensor train formats and hierarchical Tucker, compactly represent high-dimensional arrays through local tensor contractions.Source: §2 · p. 2
3 Preliminaries
The preliminaries formulate score-based VI with orthogonal expansions as a minimum-eigenvalue problem and introduce tensor-network notation used to represent high-dimensional tensors.
3.1 Score-based VI with orthogonal function expansions
- EigenVI’s orthogonal-function variational family reduces Fisher-divergence minimization to a minimum-eigenvalue problem for a PSD matrix induced by the objective.The feature map uses orthonormal basis functions, and the coefficient tensor is normalized.Source: §3.1 · pp. 2–3
- The eigenvalue formulation has two high-dimensional limitations: a non-unique, perturbation-sensitive eigenvector in nearly degenerate eigenspaces and exponential growth of the coefficient tensor and score-based matrix.The coefficient tensor has KD entries, while the matrix has KD × KD entries.Source: §3.1 · p. 3
3.2 Tensor-network diagram notation
- Tensor-network diagrams represent tensors as nodes and modes as edges; shared-mode edges are contracted, while open edges remain free modes.The figure also gives a simple QTN example.Source: §3.2 · pp. 3–4, Figure 1 · p. 3
4 Method
QuanVI extends rank-one score-VI to mixed-state density operators and represents them with QTN parameterization. Its local-structure motivation and chain-like MPO design support evaluation and representation without constructing the full operator.
4 Method
- QuanVI extends the rank-one density-operator restriction to mixed states and represents the resulting operator with a QTN parameterization.The method is designed to mitigate instability from eigenspace degeneracy while keeping optimization tractable.Source: §4 · p. 4
4.1 From rank-one to mixed-state density operators
- The rank-r mixed-state formulation represents the subspace spanned by U’s columns uniformly, rather than selecting a single eigenvector.The formulation also allows complex-valued learnable tensors, which the text describes as natural for QTN representations and as providing additional expressive capacity.Source: §4.1 · p. 4
- The induced variational density is nonnegative and normalized when the density operator is PSD with trace one and the basis functions are orthonormal.Source: §4.1 · p. 4
4.2 From full operators to local tensorized representations
- QuanVI introduces local Markov dependence as a source of structure for representing the mixed-state operator without forming the full operator.The example assumes a path-graph Markov factorization with factors involving adjacent variables.Source: §4.2 · p. 4, §4.2.1 · p. 4
4.2.1 Locality from Markov dependency
- Under path-graph Markov dependence, each interior score depends only on neighboring variables, allowing the objective to use local operators rather than an EigenVI-style global operator.This locality aligns with the chain structure of an MPO.Source: §4.2.1 · pp. 4–5
4.2.2 QTN parameterization of density operators
- QuanVI uses a chain-like MPO-style QTN to parameterize a learnable low-dimensional representation of the density operator instead of explicitly forming the full operator.System wires correspond to data dimensions; ancilla wires add degrees of freedom for representing mixed states through purification.Source: §4.2.2 · p. 5
5 QuanVI Algorithm
QuanVI combines a QTN-parameterized density operator with local score-based training and local-contraction inference procedures. Its training parameter count and computational cost scale polynomially, rather than exponentially, with data dimension.
5 QuanVI Algorithm
- QuanVI represents variational densities with a QTN-parameterized density operator and provides separate training and inference procedures.Source: §5 · p. 5
5.1 Training
- Training contracts QTN representations of UΘ and U†Θ with local score-based loss operators.Source: §5.1 · p. 5, Figure 2 · p. 6
- Both the QTN parameter count and the cost of evaluating all D local training terms scale polynomially, rather than exponentially, with data dimension.Each local term is estimated using a mini-batch of B samples.Source: §5.1 · p. 5
5.2 Inference
- Density, marginal, and conditional queries use local contractions; marginalization replaces omitted-variable measurement matrices with identity, while conditionals are contraction ratios.Source: §5.2 · pp. 5–6
- QuanVI samples sequentially from one-dimensional conditional densities, using numerical inverse-CDF sampling at each coordinate.Previously sampled variables are fixed and future variables are replaced by identity operators when computing each conditional.Source: §5.2 · p. 6
6 Experiments
QuanVI is evaluated on synthetic targets, POSTERIORDB posterior tasks, and ablations. The experiments report comparisons with EigenVI and show how local modeling capacity and parameterization choices affect approximation quality.
6.1 Synthetic target distributions
- At D = 5, QuanVI is comparable to or better than EigenVI on most targets and performs strongly on Gaussian, X-shape, GMM3, and Ring, while default-window Funnel is less competitive.Increasing L substantially improves Funnel results, suggesting that this target requires stronger local modeling capacity.Source: §6.1 · p. 7
6.2 Bayesian posterior approximation
- Across three POSTERIORDB benchmarks, QuanVI performs comparably to EigenVI and substantially outperforms the remaining baselines.For hmm_bball_0, Figure 6 suggests QuanVI better captures posterior geometry than EigenVI, including distinctly non-Gaussian features.Source: §6.2 · p. 8, Table 2 · p. 8
6.3 Ablation study
- Beyond the default setting, larger local windows yield modest improvements on most tested settings but substantially benefit Funnel.The effective bond dimension scales with K^(L−1), so the exponential dependence is set by L rather than the full data dimension D.Source: §6.3 · pp. 8–9
- Adding environment wires generally improves over the pure-state baseline E = 0; E = 3 achieves the best mean in this sweep, while larger E can bring diminishing returns.The passage attributes those diminishing returns to increased model capacity and optimization difficulty.Source: §6.3 · p. 9
- Increasing basis size K generally improves approximation quality and reduces average KL, but increases runtime through larger local tensor contractions.The main experiments use a moderate K to balance expressiveness and efficiency.Source: §6.3 · p. 9
- On Funnel, the complex-valued dtype reduces mean KL from about 5.08 to about 3.58 compared with the real-valued parameterization.Its runtime overhead is moderate in this experiment, and it is the default when memory permits.Source: §6.3 · p. 9
7 Conclusion
QuanVI combines mixed-state density operators with MPO-motivated QTNs to represent low-energy eigenspaces without choosing a single eigenvector or explicitly constructing the global operator. Its conclusion reports accuracy and scaling across synthetic and Bayesian benchmarks, while the appendix details EigenVI’s high-dimensional limitations.
7 Conclusion
- QuanVI represents low-energy eigenspaces with mixed-state density operators and uses MPO-motivated QTNs instead of explicitly constructing the global operator.This replaces reliance on a single eigenvector and avoids explicit global-operator construction.Source: §7 · p. 9
- Experiments show that QuanVI retains low-dimensional accuracy and scales to high-dimensional structured non-Gaussian synthetic settings and Bayesian posterior benchmarks.Ablations identify expressiveness–cost trade-offs involving environment size, local window size, basis size, and numerical parameterization.Source: §7 · p. 9
- QuanVI relies on local structure; larger local windows capture stronger dependencies but increase contraction cost.The conclusion identifies richer dependency graphs and adaptive tensor-network architectures as future work.Source: §7 · p. 9
A.1 Details of the EigenVI formulation
- EigenVI’s Rayleigh-quotient solution uses an eigenvector of the smallest eigenvalue, but global-operator size grows exponentially and degenerate minimum eigenspaces make single-eigenvector selection unstable.These limitations motivate mixed-state and tensor-network alternatives.Source: §A.1 · p. 13
B.1.1 Detailed Complexity Analysis
QuanVI uses local tensor contractions to keep dependence on data dimension and environment size polynomial, with exponential dependence confined to the local window size. The appendix also describes the resources used and the path-structured synthetic targets used to evaluate high-dimensional behavior.
B.1.1 Detailed Complexity Analysis
- QuanVI’s dependence on data dimension D and environment size E is polynomial, while exponential dependence is confined to local window size L.L controls an expressiveness–efficiency trade-off: larger windows capture stronger local dependencies but increase memory and runtime.Source: §B.1.1 · p. 14
- With E = O(D), drawing N independent samples costs O(ND^2K^O(L)).Sequential sampling draws variables one at a time using the chain rule and local tensor contractions.Source: §B.1.1 · p. 14
C.1 Computational resources
- The experiments used CPU nodes, and no GPUs were used.Synthetic benchmarks ran on AMD EPYC 9654 nodes with 1.5 TiB memory; posterior experiments used Intel Xeon Silver 4316 nodes.Source: §C.1 · p. 14
C.2.1 Construction of High-dimensional Synthetic Targets
- Non-Gaussian synthetic targets extend two-dimensional base distributions with a nonlinear Markov chain, preserving their original geometry while adding local dependencies.Exact sampling, log-density, and score are available, and sampling and score evaluation scale linearly with D.Source: §C.2.1 · pp. 15–16
- The Gaussian target is constructed directly in D dimensions as a locally correlated distribution whose path-graph dependency structure matches the Markov assumption.Its precision matrix is tridiagonal, with diagonal entries 1 and neighboring off-diagonal entries 0.2.Source: §C.2.1 · p. 16
C.2.2 Hyperparameter settings for synthetic experiments
Synthetic experiments use five target families, dimensions from 5 to 100, and repeated forward-KL evaluations, with QuanVI using a fixed basis size and a sample budget proportional to dimension. The appendix reports instability in normalizing-flow results and describes posterior-coordinate standardization.
C.2.2 Hyperparameter settings for synthetic experiments
- Synthetic experiments cover xshape, ring, funnel, gmm3, and gaussian at D ∈ {5, 10, 20, 100}; each configuration uses five independent seeds, and all methods use forward KL with 10^4 Monte Carlo target samples.Source: §C.2.2 · p. 16
- EigenVI is skipped for D > 5 because its global basis grows exponentially, and those configurations are reported as infeasible.The study aims for comparable optimization budgets where applicable; mixture-of-Gaussians settings are target-specific on more difficult targets.Source: §C.2.2 · p. 17
- QuanVI uses K = 5 Hermite basis functions, 2000 optimization steps, and a training sample count proportional to dimension; its mixed-state variant uses E = 2.E = 0 is the pure-state variant.Source: §C.2.2 · p. 17
C.2.3 Result of NF
- Normalizing-flow results vary sharply across targets and dimensions, with extremely large KL divergence in several high-dimensional or strongly non-Gaussian settings, so they are omitted from the main comparison.The appendix includes the detailed results for completeness.Source: §C.2.3 · p. 18
C.3.1 Standardization
- Posterior parameters are mapped to unconstrained coordinates, then standardized with a full-covariance Gaussian preconditioner before variational training.The preconditioner is fitted before QuanVI training using GSM, or BaM for one target where GSM is numerically unstable; reference posterior draws are not used.Source: §C.3.1 · p. 18
C.3.2 Hyperparameter settings for posterior benchmark experiments
The local-window ablation finds substantially lower KL divergence with L = 3, especially at D = 5, but much higher parameter and runtime costs; L = 2 is used in the main experiments. Posterior benchmarks use shared settings where applicable across three targets.
C.3.2 Hyperparameter settings for posterior benchmark experiments
- Posterior experiments cover hmm, hmm_bball_0, and gpregr, repeat configurations with independent seeds, and report means and standard deviations.QuanVI and EigenVI share the standardization source for each target; QuanVI keeps fixed tensor settings without target-specific tuning.Source: §C.3.2 · pp. 18–19
- QuanVI uses E ∈ {0, 2}, K = 5, N_k = 2, B = 5000, and trains for 1000 steps in complex double precision.MoG uses 50 mixture components and 10,000 iterations; GSM uses 10,000 iterations, while BaM and ADVI each use 1000 iterations.Source: §C.3.2 · p. 19
C.4.1 Local window size ablation
- Increasing local window size improves expressiveness but rapidly increases parameter count; with E = 0, the stated count scales as (D − L + 1)K^2L.The ablation fixes E = 0 and K = 5 while comparing L = 2 and L = 3 on the Funnel target.Source: §C.4.1 · p. 20
- For D = 10, training with L = 3 takes more than 12 hours versus about 15 minutes with L = 2; L = 3 reduces KL substantially, especially for D = 5.The authors use L = 2 in the main experiments because parameter count and runtime grow quickly with larger windows.Source: §C.4.1 · p. 20
C.4.2 Environment size ablation
Moderate environment sizes can enrich QuanVI’s mixed-state representation, while very large sizes may make optimization harder. The supplied ablations also describe a basis-size trade-off and local-window runtime costs.
C.4.2 Environment size ablation
- The X-shape ablation uses D = 10, five seeds per E value, 500 iterations, and freshly resampled mini-batches of size 100.Each update uses a noisy Monte Carlo estimate of the score-based objective.Source: §C.4.2 · p. 20
- Table 6 reports training time across local window sizes L on Funnel and states that larger windows substantially increase computational cost.Source: Table 6 · p. 21
- Moderate environment sizes E can improve fit, whereas very large E may yield diminishing returns or harder optimization; larger E adds auxiliary degrees of freedom but increases tensorized optimization-problem size.Source: §C.4.2 · p. 21
- Increasing the Hermite basis size K generally improves approximation quality but increases runtime through larger local tensor contractions.The main experiments use a moderate K.Source: §C.4.2 · p. 21
C.4.3 Numerical precision ablation
On the challenging Funnel target, the complex-valued parameterization achieves lower mean KL than the real-valued one, with a moderate runtime increase. The comparison uses five seeds for each dtype.
C.4.3 Numerical precision ablation
- The comparison uses Funnel with D = 5, E = 0, batch size B = 5000, and 2000 optimization steps, averaging five random seeds per dtype.Source: §C.4.3 · p. 21
- Mean KL was about 3.58 for cfloat versus about 5.08 for float on the Funnel target.Both dtypes show noticeable variation across random seeds.Source: §C.4.3 · p. 21
- Training took about 1m30s with cfloat versus about 1m10s with float, a moderate runtime overhead in this experiment.The authors use complex dtype as QuanVI’s default when memory permits.Source: §C.4.3 · p. 21