PaperlayerView exact revision on arXiv

Source-linked AI summary

Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives

A. Cichocki, A-H. Phan, Q. Zhao, N. Lee, I. V. Oseledets, M. Sugiyama, D. Mandic

arXiv:1708.09165v1math.NAcs.LG

TL;DR

The paper addresses the scalability limits of high-dimensional tensorized models and develops low-rank tensor-network representations and operations for data analysis and optimization. Across signal-separation applications, these representations support accurate estimation, including perfect reconstruction in tested QAM cases and slightly better damped-signal estimates than the KT method.

  • Problem

    High-dimensional Volterra-based models can require exponentially long eigenvectors, while raw tensor methods may be feasible only for small-memory, low-dimensional inputs.

  • Method

    The paper develops tensorization and low-rank tensor-network formats, especially TT and QTT, for compressed representations, contractions, signal separation, and multilinear regression models such as HOPLS.

  • Results

    The applications report good mixing-matrix estimation, zero bit error rates for tested 32-QAM and 64-QAM signals, and slightly better damped-signal estimates than the KT method.

  • Takeaways & Limitations

    Low-rank tensor representations can support signal separation from a single or a few mixtures and provide compressed models for broader tensorized data-analysis tasks.

  • Takeaways & Limitations

    Raw tensor-format methods require many data samples and are feasible only for systems with relatively small memory and low-dimensional inputs.

Abstract

from arXiv · show

Part 2 of this monograph builds on the introduction to tensor networks and their operations presented in Part 1. It focuses on tensor network models for super-compressed higher-order representation of data/parameters and related cost functions, while providing an outline of their applications in machine learning and data analytics. A particular emphasis is on the tensor train (TT) and Hierarchical Tucker (HT) decompositions, and their physically meaningful interpretations which reflect the scalability of the tensor network approach. Through a graphical approach, we also elucidate how, by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volumes of data/parameters, thereby alleviating or even eliminating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification (support tensor machines, canonical correlation analysis, higher order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and in the optimization of deep neural networks. Part 1 and Part 2 of this work can be used either as stand-alone separate texts, or indeed as a conjoint comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions.

Tensorization and Structured Tensors

Tensorization converts lower-order data or large parameter sets into structured higher-order tensors, often exposing low-rank tensor-network representations that support scalable analysis and computation. The section surveys reshaping, statistical, local-structure, and parameter tensorizations, with applications including source separation, harmonic retrieval, regression, classification, and signal reconstruction.

  • Tensorization methods: Tensorization generates higher-order structured tensors by reshaping, aligning, augmenting, transforming, or statistically expanding lower-order data.Examples include image and EEG tensors, Toeplitz and Hankel constructions, time-frequency transforms, cumulants, and derivatives of generalized characteristic functions.
  • Low-rank structure: Quantizing exponential, trigonometric, polynomial, and rational signals can produce exact or approximate low-rank tensor-train representations.A quantized exponential signal yields a rank-1 tensor, while broader function classes admit approximate low-rank TT formats.
  • Harmonic retrieval: Vandermonde rank cannot exceed the length of a Hankel tensor’s generating vector, and products such as t sin(ωt) yield bounded tensor ranks.The cited example gives ranks 3 and 2 for sin(ωt) and t, respectively, implying rank at most 6 for their product.
  • Applications: Low-rank Toeplitz and Hankel tensor networks support blind source separation and harmonic retrieval by representing structured signal tensors compactly.For mixtures, low-rank structure in hidden-source foldings implies a corresponding low-rank representation of the mixture tensor.
  • Blind identification: Derivative-tensor decompositions estimate mixing matrices effectively, with higher-order derivatives improving performance while accuracy deteriorates as the number of sources increases.Prior TT decomposition produced approximately 2 dB higher mean SAE than CP alone for derivative orders 7 and 8 with relatively many sources.
  • Constant-modulus separation: Constant-modulus tensor methods perfectly reconstructed the tested 32-QAM and 64-QAM signals with zero bit error rates.The 32-QAM case used five quantized tensors, while 64-QAM used nine tensors across increasing odd orders.
  • Regression and classification: Tensorization can turn nonlinear feature extraction into a linear mapping and reduce parameterization in multilinear regression models.It also supports constrained Tucker-based HOPLS, where independent and dependent tensors share latent vectors in the samples mode.
  • Signal reconstruction: For sinusoidal signals, folding exposes low-rank matrix or tensor structure, while doubling signal length improves SAE by 2 dB on average.Signals shorter than 576 samples showed about 12 dB worse estimation quality than signals of length 9 × 2^12 in the cited comparison.

Supervised Learning with Tensors

Tensor-based supervised learning extends regression and classification to structured multiway data while controlling model complexity through low-rank tensor representations. The section covers tensor regression, support tensor machines, HOPLS, probabilistic kernels, and higher-rank tensor classifiers.

  • Supervised learning models responses from paired inputs and outputs, supporting regression for continuous targets and classification for categorical targets.
  • Tensor representations preserve object structure and can reduce unknown parameters, addressing small-sample and high-dimensional learning settings.
  • Support tensor machines extend support vector regression to tensor inputs, learning a tensor weight model and bias from paired tensor samples.
  • Tensor regression represents the weight tensor with CP, Tucker, or related low-rank formats to obtain generalized multilinear regression models.
  • Higher-order partial least squares: HOPLS simultaneously performs constrained Tucker decompositions of independent and dependent tensors, while its loading dimensions trade off model fit and complexity.
  • Tensor kernels and classifiers: Probabilistic tensor kernels accommodate missing values and variable data lengths, while higher-rank tensor classifiers use mode-wise projections to improve discriminative ability.

Tensor Train Networks for Selected Huge-Scale Optimization Problems

Tensor-train networks make huge-scale optimization manageable by converting high-dimensional constrained problems into lower-scale subproblems over TT cores. Orthogonalized sub-trains, frame matrices, and interface matrices provide the linear structure needed for core-wise computations.

  • Low-rank TT networks approximate cost functions in huge-scale dimensionality-reduction and optimization problems, including eigenvalue, CCA, and HOPLS formulations.
  • These problems are reformulated as constrained optimizations and reduced to manageable lower-scale subproblems using tensor-network representations.
  • Orthogonalization: Orthogonalization reduces contraction complexity and improves algorithm robustness, while also making interface and frame matrices orthogonal under the stated conditions.
  • Frame equations: The frame equation expresses the vectorized tensor linearly in the selected core, with frame or interface matrices collecting the remaining cores.
  • TT core extraction: A TT network is split into left- and right-orthogonalized sub-trains so that a selected core can be extracted and optimized separately.
  • Two-core extraction: Two neighboring TT cores can be merged into a super-core and represented through a corresponding two-site frame equation for modified optimization schemes.

3.2 Alternating Least Squares (ALS) and Modified ALS (MALS)

ALS makes TT optimization feasible by updating one core at a time while fixing the others, with sweeps and orthogonalization used to solve successive local problems. MALS updates two neighboring cores and can adapt TT ranks, trading higher subproblem cost for potentially faster convergence.

  • Simultaneous optimization over all TT cores is complex and nonlinear, so the problem is replaced by sequential local optimizations.
  • ALS: ALS updates one core while keeping all others fixed, traverses the cores in forward and backward half-sweeps, and repeats until a stopping criterion is met.
  • ALS limitations: Standard ALS decreases the cost function but does not guarantee a global minimum and depends on an initial guess for the unknown TT rank.
  • MALS: MALS subproblems are more expensive than standard ALS, but convergence may increase considerably and TT ranks can be estimated adaptively.
  • ALS: Orthogonalization follows core updates and generally improves the conditioning of local quadratic optimization problems.
  • MALS: MALS merges two neighboring cores, optimizes their super-core, and splits it with truncated SVD or another low-rank factorization.

3.3 Tensor Completion for Large-Scale Structured Data

Tensor completion reconstructs high-dimensional structured arrays with missing or noisy entries by assuming a suitable low-rank TT approximation. ALS converts the reconstruction into smaller least-squares problems, while rank-adaptive procedures estimate unknown TT ranks during optimization.

  • Tensor completion seeks to reconstruct high-dimensional structured arrays when a large proportion of entries is missing or noisy.
  • The TT completion formulation minimizes reconstruction error on the sampling set while constraining the solution to a prescribed TT-rank manifold.
  • ALS completion: ALS converts large-scale TT reconstruction into smaller optimization problems over core slices, which can be expressed in scalar or vectorized form.
  • ALS completion: Convergence requires repeated sweeps over all cores, with orthogonalization after each sweep and sequential minimization of individual slices.
  • ALS completion: Each vectorized core-slice subproblem is a standard linear least-squares problem that can be solved efficiently for huge-scale datasets.
  • Rank adaptation: Because TT ranks are generally unknown, they can be reduced from Rmax or increased from Rmin until a residual tolerance or maximum rank is reached.

3.4 Computing a Few Extremal Eigenvalues and Eigenvectors

TT representations convert extreme eigenvalue problems for prohibitively large symmetric matrices into linked, much smaller eigenvalue subproblems. Tensor contractions, orthogonal cores, and block TT models support efficient computation of one or several extremal eigenpairs.

  • Traditional eigenvalue methods become prohibitive for symmetric matrices with dimensions such as I = 10^15 or higher.
  • Representing A and x in TT/MPO and TT/MPS formats converts the large-scale eigenvalue problem into smaller optimization problems through tensor contractions and frame equations.
  • For relatively small TT ranks, effective Hamiltonians A(n) are much smaller than A, reducing the original EVD to linked smaller EVD subproblems.
  • The local eigenproblem A(n)x(n) = λx(n) is solved iteratively using optimized contractions rather than explicitly forming A(n).Left and right contraction tensors are built through iterative sweeps that reuse information from neighboring steps.
  • Block-n TT jointly represents K mutually orthogonal eigenvectors, with one fourth-order core carrying the K-vector index.This uses nearly the same number of data samples as representing a single vector or matrix.
  • MALS dynamically controls TT ranks and approximation error by merging neighboring cores, optimizing them, and splitting them with truncated SVD.The truncated SVD estimates optimal ranks and quantifies the splitting error.

3.5 TT Networks for Tracking a Few Extreme Singular Values and Singular Vectors in SVD

TT networks formulate the computation of a few extreme singular values and singular vectors as constrained trace optimization problems. Core contractions reduce these huge-scale problems to SVDs of relatively small matrices.

  • The largest K singular values and corresponding singular vectors are computed by maximizing tr(U^TAV) subject to U^TU = I_K and V^TV = I_K.
  • The smallest singular values can be formulated with the Moore–Penrose pseudo-inverse and represented by an asymmetric tensor network after tensorization.
  • TT core contractions reduce the unfeasible optimization to local maximizations of tr((U(n))^T A(n) V(n)) with local orthogonality constraints.
  • The reduced problems require only the largest or smallest singular values of relatively small matrices A(n), allowing any efficient SVD algorithm to be applied.
  • For thousands or more singular values, block-n TT may become inefficient because many-loop tensor networks have computationally complex contractions.Vectorizing the orthogonal matrices can eliminate the loops and transform the network into standard TT formats.

U P RIˆK Ñ u P RIK, V P RJˆK Ñ u P RJK (3.46)

Tensor-network formulations extend generalized eigenvalue methods and canonical correlation analysis to huge-scale data and structured matrices. Recursive TT contractions produce smaller linked optimization problems while preserving orthogonality or covariance-related constraints.

  • Generalized eigenvalue decomposition: GEVD estimates extremal eigenvalues and eigenvectors of symmetric A and positive-definite B through trace optimization over an orthonormal matrix V.
  • Generalized eigenvalue decomposition: A change of variables W = B^1/2V reduces GEVD to a standard symmetric EVD with orthogonality constraint W^TW = I_K.
  • Trace-ratio optimization: Recursive contraction of TT cores reduces trace-ratio optimization to smaller trace-ratio problems involving contracted matrices A(n) and B(n).
  • Canonical correlation analysis: CCA seeks linear combinations of two centered datasets whose latent variables are maximally mutually correlated.
  • Canonical correlation analysis: Contractions of TT sub-networks transform the huge-scale orthogonal CCA problem into smaller optimization problems.
  • Canonical correlation analysis: Sparse CCA adds ℓ1 penalties to the projection matrices for computational tractability and physical interpretability of latent variables.

L P P RRn´1Jnˆ(Rn+Qn),

TT networks represent large-scale regularized regression and sparsity-constrained problems through contracted sub-networks optimized by ALS or MALS. Regularization and rank adaptation address computational complexity, but suitable TT ranks and network structures remain central challenges.

  • Regularized regression: The regularized regression objective combines data-fitting, cross-product, response, and γ-weighted smoothness terms.It is formulated for A, B, X, and L with dimensions matching the regression problem.
  • Regularized regression: Approximating A, B, and X in TT format converts the infeasible large-scale objective into sub-networks optimized by ALS or MALS.
  • Regularized regression: Regularization terms imposing smoothness or sparsity can reduce TT ranks and parameters while improving convergence and approximation quality.
  • MALS: MALS creates larger local optimization problems by merging neighboring cores before optimization and splitting them afterward.
  • Sparsity-constrained optimization: Non-smooth penalties such as ℓ1, ℓq, and Schatten norms remain insufficiently investigated experimentally in tensor-network settings.

3.9 Truncated Optimization Approach in TT Format

Truncated iteration methods combine low-rank tensor representations with iterative updates and rank truncation to solve huge optimization problems. In TT format, vector operations can be carried out through core tensors, although intermediate gradients and auxiliary vectors may require high ranks.

  • TT implementation: Large vectors and gradients can be represented as TT networks, whose addition is implemented through direct sums of corresponding core tensors.The resulting TT ranks generally increase and must be controlled by truncation or TT rounding.
  • Method: Truncated iteration methods combine a preconditioned gradient step with rank truncation under tolerance ϵ.The update vector may represent gradient descent, preconditioned gradient descent, or quasi-Newton steps.
  • Rank truncation: Truncation can use hard thresholding, which removes small singular values, or soft thresholding, which shrinks them toward zero.Soft thresholding corresponds to nuclear-norm minimization for matrices, a convex relaxation of rank.
  • Applications: Truncated iteration methods have been applied to linear systems, PDE discretization, extreme eigenvalue computation, and low-rank tensor completion.The cited applications use CP, Tucker, TT, and HT formats with iterative algorithms.
  • Advantages and limitations: A key advantage is that tensor networks need not be constructed for a specific cost function, while global convergence properties hold under restricted conditions.The approach is also suitable for large-scale compressed sensing through its analogy with low-rank truncation and sparse estimation.
  • Advantages and limitations: The approach requires auxiliary vectors and low-rank representations of intermediate quantities, which may instead have high TT ranks and require truncation at every iteration.These requirements can make intermediate iterations expensive even when the solution and right-hand side are well approximated.

3.10 Riemannian Optimization for Low-Rank Tensor Manifolds

Riemannian optimization treats fixed-rank tensor formats as smooth manifolds, enabling optimization through tangent-space directions and retractions while retaining low-parametric representations. The approach offers computational advantages, but its convergence theory and high-dimensional software support remain incomplete.

  • Manifold formulation: Riemannian optimization formulates tensor problems on smooth low-rank manifolds and uses their geometric structure to reduce solution costs.The cost function is defined on manifold points, and the framework includes many basic optimization problems.
  • Manifold formulation: The manifold formulation converts constrained optimization into unconstrained optimization performed on a constrained space.This is presented as a potential advantage of using low-rank tensor manifolds.
  • Tensor manifolds: Fixed-rank Tucker and TT tensor sets form smooth embedded submanifolds, supporting manifold-based optimization in these formats.For TT tensors, tangent-space points have ranks bounded by (2R1, …, 2RN−1).
  • Performance and applications: Riemannian algorithms can outperform standard gradient and conjugate-gradient methods, although their greater technical complexity limits widespread use.The paper reports successful applications to best Tucker approximation and tensor completion.
  • Optimization steps: Riemannian gradient descent consists of an ambient update, projection of the gradient onto the tangent space, and retraction back to the manifold.The ambient update is not computed explicitly in the illustrated iteration process.
  • Tensor manifolds: TT-rank approximation can be obtained by rounding or TT-SVD at O(NIR^3) cost under uniform mode-size and rank assumptions.The cited assumption is In = I and Rn = R.
  • Retractions: Projector-splitting retraction is a second-order retraction and can yield better convergence than a crude SVD-based approximation.The associated splitting order also admits an exactness result under stated manifold conditions.
  • Convergence: Convergence theory for non-convex manifolds remains incomplete, with unresolved issues involving curvature, singular points, spurious local minima, and rank adaptation.The paper also describes experimentally observed staircase convergence as ranks increase.

3.11 Software and Computer Simulation Experiments with TNs for Optimization Problems

Tensor-network optimization relies on specialized software libraries spanning TT, HT, Tucker, CP, and quantum-network computations. The surveyed experiments report significant performance advantages across diverse large-scale applications, while the field and its tooling remain developing.

  • Software ecosystem: Tensor decompositions and tensor-network algorithms require specialized software libraries that are developing rapidly.The ecosystem includes libraries for contractions, decompositions, optimization, eigenvalue problems, and scientific computing.
  • TT and HT tools: The TT Toolbox supports linear systems, the AMEn algorithm, symmetric eigenvalue decomposition, and inverse or pseudoinverse computations for huge matrices.It is available for MATLAB and Python and supports TT, MPS, MPO, and QTT networks.
  • TT and HT tools: TTeMPS provides MATLAB implementations for TT/MPS algorithms, including AMEn for multiple extremal eigenvalues and eigenvectors.Its object-oriented design accommodates algorithms in the TT/MPS format.
  • General tensor tools: Other toolboxes provide general-purpose CP and Tucker decomposition, constrained factorization, sparse or incomplete tensor handling, and user-friendly tensor-analysis interfaces.Examples include the Tensor Toolbox, N-Way Toolbox, Tensorlab, TDALAB, and TENSORBOX.
  • TT and HT tools: The Hierarchical Tucker toolbox focuses on HT networks and avoids explicit SVD computation when truncating tensors already in HT format.The toolbox is designed specifically for HT-type tensor networks.
  • Parallel and quantum-network tools: Uni10 targets complex tensor networks such as PEPS and MERA, while mptensor provides parallel tensor-network computation on supercomputers.These libraries extend tensor-network software toward large-scale and quantum-network workloads.
  • Experimental perspective: Comprehensive computer experiments across diverse applications have validated significant performance advantages, although tensor networks for huge-scale optimization remain in their infancy.The conclusion frames the technology as promising but still developing.
  • Applications: The surveyed tensor-network paradigms cover EVD, PCA, CCA, SVD, linear systems, tensor completion, dynamical problems, nonlinear functions, CP and Tucker decompositions, and classification.Applications span numerical methods, scientific computing, and data-analysis tasks.

3.12 Open Problems and Challenges in Applying TNs for Optimization and Related Problems

Applying tensor networks to optimization requires assumptions about low-rank approximability and acceptable approximation error, while several algorithmic and methodological challenges remain. Future work spans broader constrained problems, efficient contractions, suitable network structures, error and convergence control, interpretability, uniqueness, and visualization.

  • Tensor-network optimization assumes datasets admit sufficiently good low-rank approximations and that approximate solutions are acceptable.
  • Extending low-rank tensor networks to complex constrained optimization with multiple constraints, nonsmooth penalties, sparsity, or nonnegativity remains an open direction.
  • Parallel algorithms for broader optimization classes are needed because most algorithms described are sequential and parallelization is nontrivial.
  • Exact contraction of complex tensor networks can require exponentially increasing time with tensor order, motivating efficient and sufficiently accurate approximate contractions.
  • Open problems include selecting application-specific network structures, controlling approximation accuracy and TT-rank explosion, analyzing convergence, and establishing decomposition uniqueness and optimality.These goals are tied to achieving affordable storage and computational complexity and to developing faster or more reliable algorithms.
  • Further methodological work is needed to choose factor-matrix and core constraints that extract significant or desired latent variables with meaningful physical interpretations, alongside techniques for visualizing huge tensors.

Tensor Networks for Deep Learning

The chapter connects deep neural networks with tensor networks to address architectural limitations, parameter scalability, and unresolved questions about expressive power. It presents tensor-network representations and optimizations as a route to reduced-parameter models and explores architectures including HT, TT, and MERA.

  • Deep learning advantages: Deep neural networks provide stronger nonlinear and multilevel feature representations than shallow networks, supporting applications from vision and speech to structured-data analytics.The passages also describe interpretable high-level representations and hierarchical classification of large image collections.
  • Open challenges: DNN research still faces challenges involving generalization, fast learning, local minima, and rigorous bounds on architecture-specific expressive power.The chapter specifically targets the latter two challenges through links between DNNs and tensor networks.
  • Compressed neural models: Tensor-network reformulations preserve useful model properties while reducing complexity, including RBM sampling and interface behavior and the interaction structure of matrix-variate models.For the matrix-variate case, the number of free parameters is reduced from IJMK to IJ + KM + JM + IK.
  • Tensor-network optimization: Tensor networks represent high-order DNN parameters compactly, enabling reduced-rank optimization and mapping back to DNNs with considerably fewer parameters.The TN workflow constructs a corresponding network, applies reduced or canonical forms such as truncated SVD, and maps the optimized network back to a DNN.
  • TN architectures for DNNs: HT, TTNS, TT/MPS, TT/MPO, and TC architectures provide hierarchical or deep tensor-network analogues of DCNNs, with TT/TC supporting N hidden layers and standard HT using L = log2(N).The text identifies these networks as tools for indirectly simulating and optimizing DCNNs, while noting that looped architectures may offer additional performance and architectural possibilities.
  • Future perspectives: MERA and other looped or two-dimensional tensor networks are proposed as routes to more expressive and potentially improved DCNN architectures.The stated goals include modeling more complex long-term correlations and constructing DCNN families for classification, clustering, and prediction.

Discussion and Conclusions

The monograph reviews tensor decompositions and tensor-network approaches for large-scale machine learning and data analytics, emphasizing low-rank representations, graphical interpretation, and distributed optimization. It also discusses tensor-network designs for improved deep neural network architectures.

  • The work targets large-volume, multi-relational, and multi-modal datasets represented as tensors, covering supervised and unsupervised learning, tensor regression, and deep learning.
  • Graphical representations explain tensor algebra and connect flat linear algebra with multilinear algebra, focusing on Tucker, Tensor Train, and Hierarchical Tucker decompositions.
  • Low-rank tensor networks and contractions convert huge-scale optimization problems into smaller linked or distributed sub-problems with affordable complexity.
  • Tensor-network methods account for couplings among multiple variables and accommodate multimodal, incomplete, and noisy data.
  • The monograph discusses tensor networks as a basis for improved and optimized deep learning architectures, with potential applications in classification, clustering, and prediction.

1 Estimation of Derivatives of GCF

This section constructs higher-order derivative tensors of a generalized characteristic function and simplifies their estimation through tensor products, symmetrization, and partition-based formulas.

  • Nth-order partial derivatives of the estimated characteristic function are represented as Nth-order tensors with dimensions I × I × ⋯ × I.
  • Derivative tensors are formed from outer products of lower-order tensors indexed by partitions n1, n2, ..., nk whose orders sum to N.
  • The symmetrization operator S(A) produces a symmetric tensor by combining tensor permutations.
  • For derivative orders 5, 6, and 7, symmetrization can be applied once to the resulting tensor rather than separately to each component tensor.
  • The number of distinct partition arrangements is determined from the distinct partition values and their replication numbers.

2 Higher Order Cumulants

Centering mixtures at zero mean makes the first derivative vanish and simplifies higher-order cumulant computation by eliminating corresponding partitions.

  • Zero-mean centering makes the first derivative ψ(1) vanish.
  • The associated partition terms are zero for arbitrary partitions containing singleton parts.
  • Cumulant computation is simplified by ignoring those zero partition terms, with formulas provided for orders 2 through 7.

3 Elementary Core Tensor for the Convolution Tensor

This section defines an elementary core tensor from sparse binary Nth-order tensors and specifies its recursive structure. The resulting core tensors are orthogonal, with only a limited subset nonzero.

  • The elementary tensor S is assembled from core tensors S1 through S2N.
  • The construction starts from 2N sparse Nth-order tensors of size 2 × 2 × ⋯ × 2.
  • The sparse tensors contain ones only at locations satisfying a parity-style index condition involving complemented indices.
  • Its first sub-tensors select S2n−1 and S2n, while the remaining sub-tensors are defined recursively through index-shift relations.
  • The core tensors are pairwise orthogonal, and only N + 1 are nonzero; the remaining N − 1 tensors are zero.
Loading 1708.09165v1…