PaperlayerView exact revision on arXiv

Source-linked AI summary

High-dimensional extreme eigenvalue problems: low-rank tensor parametrization and optimization

Pengfei Hao, Renfeng Peng, Yang Kuang, Bin Gao

arXiv:2609.30138v1math.NAcs.NAmath.OCphysics.comp-ph

TL;DR

High-dimensional extreme eigenvalue problems become difficult to solve directly as their ambient dimensions grow. This paper optimizes the Rayleigh–Ritz problem over bounded-rank TT tensors using a smooth manifold formulation and Riemannian methods. Experiments report accuracy comparable to full-space eigensolvers with less computation time on larger problems, while TT ranks decrease during iteration and overestimated ranks remain robust.

  • Problem

    Directly solving high-dimensional extreme eigenvalue problems suffers from the curse of dimensionality, while ambient-space eigensolvers have limited ability to exploit tensor structure.

    Source: §1 · pp. 1–2

  • Method

    The authors reformulate Rayleigh–Ritz optimization over bounded-rank TT tensors on a smooth parametrization manifold and develop LTT-RGD and LTT-RCG.

    Source: §1 · p. 3, §3.1 · p. 7

  • Results

    LTT-RCG achieves accuracy comparable to LOBPCG and PRIMME with less computation time on larger problems, while observed TT-rank reductions and accurate solutions with overestimated ranks demonstrate robustness.

    Source: §6 · p. 25

  • Takeaways & Limitations

    The TT representation permits computations using tensor cores when storing full vectors is impractical.

    Source: §6 · p. 25

    The approximation error from restricting the eigenvalue problem to bounded-rank tensors remains to be investigated, including how eigenvalue and eigenvector accuracy depends on rank.

    Source: §6 · p. 25

Abstract

from arXiv · show

High-dimensional extreme eigenvalue problems often arise from molecular vibrational models, electronic structure calculations, and quantum mechanics. Directly solving these problems suffers from the curse of dimensionality. Instead of tackling the problem in high-dimensional ambient space, we reformulate the problem through low-rank tensor formats, which can significantly reduce the computational cost and storage. Specifically, we consider the Rayleigh--Ritz problem on bounded-rank tensors in the tensor train format, which enables a more flexible choice of rank parameters. Moreover, we consider a smooth parametrization for bounded-rank tensors by introducing slack variables, leading to a smooth manifold structure. The original Rayleigh--Ritz problem is therefore transferred to an optimization problem on the manifold. We develop the Riemannian geometry and propose optimization methods for solving extreme eigenvalue problems. In practice, the Kronecker-product structure in Hamiltonian and PDE operators is employed to simplify the computation. Numerical experiments on the harmonic oscillator, the Laplace operator, the Schr\"odinger equation, the layered cluster problem, and the Bose--Einstein model demonstrate that the proposed method achieves accuracy comparable to full-space eigensolvers with reduced computation time and avoids storing full vectors. The results also show numerical rank reduction during iteration and robustness when the rank parameter is over-estimated. In addition, the fixed-point iterations illustrate that the proposed methods are able to serve as an inner eigensolver for solving nonlinear eigenvalue problems.

1. Introduction

The paper recasts high-dimensional extreme-eigenvalue computation as Rayleigh–Ritz optimization over bounded-rank TT tensors and develops a smooth manifold formulation for Riemannian methods. Experiments report accuracy comparable to full-space eigensolvers, with reduced computation time on larger problems and robustness to overestimated ranks.

1. Introduction

  • TT storage scales polynomially with dimension, mode size, and rank, offering a compact alternative when low-rank structure can represent the high-dimensional tensor.Directly solving the tensor-space problem still faces exponential growth in dimension.Source: §1 · p. 2
  • Bounded-rank TT tensors allow more flexible rank choices than fixed-rank tensors, whose set is not closed.This flexibility is useful when the target eigenvector’s rank is unknown.Source: §1 · p. 2
  • Ambient-space eigensolvers can become computationally intractable as operator and eigenvector sizes grow.The introduction notes existing methods are efficient for moderate-sized operators and eigenvectors.Source: §1 · p. 2
  • The bounded-rank tensor set is an algebraic variety with singular points, so standard Riemannian algorithms cannot be applied directly to it.The paper uses a smooth parametrization to formulate the problem on a manifold.Source: §1 · p. 3
  • The paper develops bounded-rank TT Rayleigh–Ritz optimization, its smooth parametrization manifold and geometry, and the LTT-RGD and LTT-RCG algorithms.Kronecker-product structure simplifies geometric operations, and the algorithms’ computational complexity is analyzed.Source: §1 · p. 3
  • Across five model classes, the proposed methods achieve accuracy comparable to LOBPCG and PRIMME, with polynomial scaling and robustness to overestimated rank parameters.LTT-RCG also solves the nonlinear Bose–Einstein eigenvalue problem using a low-rank TT representation.Source: §1 · p. 4

2. Preliminaries

The preliminaries establish the matrix and tensor notation used throughout and review the tensor train decomposition as a compact representation of multidimensional tensors.

2. Preliminaries

  • This section introduces the matrix and tensor notation used throughout the paper and reviews the tensor train decomposition, a format commonly called a matrix product state in computational physics.Source: §2 · p. 4, §2.2 · p. 5

2.1. Matrix and tensor notation

  • A tensor is a multidimensional array, while its unfoldings, vectorization, tensorization, and tensor products provide matrix-based ways to manipulate it.Tensor-network diagrams represent tensors as nodes and contracted modes as connected edges.Source: §2.1 · pp. 4–5

2.2. Tensor train decomposition

  • A tensor train represents each entry as a product of mode-indexed slices of successive cores, with boundary ranks r0 = rd = 1.The cores have sizes r_(k−1) × n_k × r_k; boundary cores reduce to matrices when singleton rank modes are omitted.Source: §2.2 · p. 5
  • The TT rank vector consists of the ranks of successive unfoldings, and the interface matrices collect core products on either side of a mode split.The admissible ranks satisfy r_k ≤ min{r_(k−1)n_k, n_(k+1)r_(k+1)}.Source: §2.2 · pp. 5–6

3. Geometry of low-rank tensor parametrization

This section develops the smooth-manifold formulation and Riemannian geometry of bounded-rank tensor-train tensors. It presents tangent-space projections and related operations, including gradients, vector transport, and a retraction that preserves right orthogonality.

3.1. Parametrization manifold

  • Slack-variable projectors lift bounded-rank TT tensors to a smooth parametrization manifold, with core enlargement ensuring coverage of the entire tensor space.The gauge-group quotient removes core nonuniqueness, and the resulting manifold projects to the bounded-rank tensor space.Source: §3.1 · pp. 7–8

3.2. Tangent space

  • Gauge conditions give each tangent vector a unique TT-core representation, while a backward-recursive Euclidean metric avoids forming full tensors or projector derivatives.The metric’s additional terms account for derivatives of the right-interface projectors.Source: §3.2 · pp. 8–10

3.3. Projection

  • The Riemannian gradient is computed by projecting the ambient gradient onto the tangent space through TT-core and interface recursions, without forming an explicit tangent-space basis.Positive-definite recursion matrices make the core updates well defined.Source: §3.3 · p. 10
  • The tangent vector constructed by the projection formula is the unique orthogonal projection onto the tangent space.For a smooth objective composed with the parametrization, the Riemannian gradient is obtained by projecting the ambient gradient (∇f(X), 0, …, 0).Source: §3.3 · p. 12
  • Projection-based vector transport maps tangent vectors between points, is linear in the input tangent vector, and acts as the identity when the destination is the source point.The method enables tangent vectors from different iterations to be combined.Source: §3.3 · pp. 12–13

3.4. Retraction

  • A retraction updates the right-orthogonal tensor-train cores using a tangent vector while preserving their right orthogonality.The construction satisfies the retraction conditions: it returns the base point at zero step and has the prescribed tangent as its differential there.Source: §3.4 · pp. 13–14

4. Optimization for the Rayleigh–Ritz problem

The Rayleigh–Ritz problem is optimized on the smooth parametrization manifold, with its Riemannian gradient computed by applying the operator and projecting onto the tangent space. Kronecker structure and TT-core operations keep the algorithms and their costs aligned with low-rank representations.

4. Optimization for the Rayleigh–Ritz problem

  • The Riemannian gradient is obtained by applying H to a TT tensor and projecting the result into the parametrization manifold’s tangent space.The objective depends on the parameter only through its tensor component, so the other ambient gradient components are zero.Source: §4 · p. 14

4.1. E

  • Kronecker terms are evaluated separately from their factors, avoiding formation of the full matrix H or the potentially higher-rank tensor H X.The projected terms are combined only after their tangent-core representations have been computed.Source: §4.1 · pp. 15–16

4.2. Riemannian algorithms

  • LTT-RGD uses the negative Riemannian gradient, while LTT-RCG combines it with a transported previous direction; both use Armijo backtracking and retraction.LTT-RGD’s gradient norm tends to zero under the stated regularity and backtracking assumptions; LTT-RCG accumulation points are stationary when its search directions satisfy the gradient-related condition.Source: §4.2 · p. 16

4.3. Computational complexity

  • O((s + 1)dmn^2r^3) is the per-iteration cost for both methods; for fixed m, n, r, and s, cost and representation storage grow linearly with d.The TT cores and Kronecker factors require O(dnr^2) and O(dmn^2) storage, respectively; the iteration avoids forming tensors of size n^d.Source: §4.3 · pp. 17–18

5. Numerical experiments

The experiments compare tensor-train methods with full-vector eigensolvers across five models, using common operators, reference eigenpairs, initial tensors, and residual tolerances for the linear comparisons. They assess accuracy, runtime, rank behavior, and use of LTT-RCG in a nonlinear fixed-point iteration.

5.1. Harmonic oscillator

  • For the separable harmonic-oscillator ground state, LTT-RCG reaches comparable final accuracy in substantially fewer iterations than LTT-RGD and reduces numerical TT ranks to (1, 1, 1, 1).The test uses rank parameter (1, 4, 4, 1), above the exact eigenvector rank.Source: Figure 6 · p. 19, §5.1 · p. 19
  • Both LTT methods scale polynomially with tensor order, mode size, and rank; LTT-RCG has slightly higher cost per iteration than LTT-RGD.Figure 7 varies one parameter at a time across tensor order, mode size, and rank.Source: Figure 7 · p. 19, §5.1 · p. 19

5.2. Laplace eigenvalue problem; 5.3. Schrödinger eigenvalue problem

  • All three solvers reach comparable final energy errors and residuals on the Laplace and fixed-size Schrödinger tests, while LTT-RCG requires less computation time.The Laplace test also reports agreement with the reference marginal density; the Schrödinger test compares Fourier-space density and central-slice errors.Source: Figure 8 · p. 20, §5.2 · p. 20, Figure 9 · p. 20

5.3. Schrödinger eigenvalue problem

  • As tensor order increases in the Schrödinger tests, LTT-RCG becomes faster than full-space solvers and remains applicable beyond their size limit, though its final residual rises at the largest orders.Table 2 reports mean times and final normalized residuals over three repetitions; its memory column is the size of one full vector, not total solver memory.Source: §5.3 · pp. 21–22, Table 2 · p. 22

5.4. Layered cluster eigenvalue problem

  • At r = 4, the layered-cluster eigenvector becomes representable: LTT-RCG errors fall below 5 × 10^-14, while r = 1, 2, 3 retain errors and residuals of at least 0.70.For r > 4, the smooth parametrization also handles tensors whose numerical rank is below the prescribed rank.Source: §5.4 · p. 23, Figure 12 · p. 23

5.5. Nonlinear eigenvalue problem

  • After four outer iterations, the Bose–Einstein density meets the stopping criterion, while chemical potential and energy have stabilized; the inner gradient decreases substantially in the first three subproblems.The final inner solve terminates after one iteration with zero relative density change.Source: §5.5 · p. 24, Figure 13 · p. 24

6. Conclusion

The conclusion presents LTT-RGD and LTT-RCG for extreme eigenvalue problems over bounded-rank tensor trains. Experiments report accuracy comparable to full-space eigensolvers with less computation time on larger problems, alongside robustness to rank overestimation.

6. Conclusion

  • LTT-RGD and LTT-RCG minimize the Rayleigh quotient over a low-rank tensor space using a smooth parametrization that includes tensors below the prescribed rank.The parametrization represents tensors throughout the bounded-rank space.Source: §6 · p. 24
  • Figure 14 shows frozen-potential fixed-point iteration for the three-dimensional Bose–Einstein problem with n = 50, ϵ = 1, and TT-rank parameter (1, 1, 1, 1).Source: Figure 14 · p. 25
  • For sums of Kronecker products, tangent-space operations optimize directly with TT cores without forming full tensors and have polynomial per-iteration cost.The operations include tangent-space representation, projection, vector transport, and retraction.Source: §6 · p. 25
  • LTT-RCG achieves accuracy comparable to LOBPCG and PRIMME with less computation time on larger problems, while tensor cores avoid storing impractically large full vectors.Numerical TT ranks also decrease during iteration, and accurate solutions are obtained with overestimated rank parameters.Source: §6 · p. 25
  • Future work includes preconditioners, adaptive rank selection, and analysis of how restricting to bounded-rank tensors affects eigenvalue and eigenvector accuracy.The proposed analysis would examine accuracy dependence on the rank parameter.Source: §6 · p. 25
Loading 2609.30138v1…