Infinite Tucker Decomposition: Nonparametric Bayesian Models for Multiway Data Analysis

• Classical multilinear models struggle with coupled nonlinear interactions, noisy observations, missing values, and binary or ordinal data.The passage notes that missing values must first be imputed for classical models and that treating non-real-valued data as continuous can degrade prediction.Source: §1 · p. 2
• InfTucker performs Tucker decomposition in an infinite feature space using latent Gaussian or t processes and probabilistic models for noise, missing data, and binary or continuous observations.The paper also introduces efficient variational inference to reduce computational cost.Source: §1 · p. 2
• On chemometrics and social-network datasets, InfTucker achieved significantly higher prediction accuracy than HOSVD, Weighted CP [1], and nonnegative tensor decomposition.Source: §1 · p. 2

• Tucker decomposition represents a K-mode tensor using a core tensor and one latent factor matrix per mode.The factor matrices can be grouped as a Tucker tensor with an identity core, yielding the compact form Y = W × U.Source: §2 · p. 3
• Vectorizing the Tucker representation expresses it through the Kronecker product of the mode-specific factor matrices and the vectorized core.Source: §2 · p. 3
• CP is a restricted form of Tucker decomposition, and alternating least squares has been used to solve both families.The paper’s notation uses vec(Y) to stack tensor entries into a vector; its matrix vec convention is row-wise.Source: §2 · pp. 2–3

3 Infinite Tucker decomposition

• InfTucker extends classical Tucker decomposition with flexible noise models, an infinite core for complex interactions, and a latent Gaussian-process or t-process prior.The latent tensor’s infinite Tucker decomposition is defined as a limit of finite decompositions using feature mappings of component-matrix rows.Source: §3 · p. 3

3.1 Tensor-variate Gaussian processes

• A tensor-variate Gaussian process is constructed from mode-specific covariance functions; the latent tensor has a tensor-variate normal distribution with covariance formed from their Kronecker product.The model uses a zero mean and a truncated core tensor with identity covariance factors.Source: §3.1 · p. 4

3.2 Tensor-variate t processes

• The finite t-process construction converges to a tensor-variate t process, realizing probabilistic infinite Tucker decomposition on location vectors induced by the component matrices.The formulation also provides a constructive definition for tensor-variate processes with general covariance functions.Source: §3.2 · p. 5

3.3 Noise models

• A probit likelihood models binary observations, a Gaussian likelihood models real-valued observations, and observed entries alone can contribute when values are missing.The paper focuses on probit and Gaussian models, while noting other likelihoods for ordinal regression, multiclass classification, and semi-supervised classification.Source: §3.3 · p. 5

4.1 Inference

• Probit inference uses data augmentation, while Gaussian-noise inference updates Y directly and applies variational EM only to observed entries.For the probit regression prediction integral, the method substitutes the mode of the approximate posterior for the η integral.Source: §4.1 · pp. 6–7, §4.2 · p. 8
• A variational EM algorithm approximates the intractable posterior with a factorized distribution over Z, M, and η, iteratively optimizing each factor to reduce KL divergence.The E-step updates the approximate posterior, followed by a gradient-based M-step.Source: §4.1 · p. 6, §4 · p. 6

4.2 Prediction

• Prediction derives distributions for missing entries under probit noise and regression under Gaussian noise.The probit predictive integral is approximated using the mode of the approximate posterior for η.Source: §4.2 · p. 8

4.3 Efficient Computation

• To avoid the prohibitive cost of a naïve implementation, the algorithm exploits covariance Kronecker structure and covariance-matrix SVDs to compute the needed operations.The exposition simplifies computation by assuming Eq [η] = 1, and states that the strategies adapt to Eq [η] ≠ 1.Source: §4.3 · pp. 8–9

5 Related Works

• InfTucker models joint interactions of two matrix modes, unlike PPCA and GPLVMs, which model one mode; related matrix-variate GPs used linear covariance functions.Those matrix-variate GP approaches addressed online recommendation and link prediction.Source: §5 · p. 9
• InfTuckergp and InfTuckertp achieved higher prediction accuracy than competing approaches; InfTuckertp improvements were significant (p < 0.05) except versus InfTuckergp on amino.Table 1 reports mean square errors with standard errors.Source: Table 1 · p. 10
• GP-based InfTucker reduces to probabilistic Tucker with a linear covariance function; its t-process version additionally marginalizes a covariance scaling hyperparameter.Unlike pTucker, InfTucker uses nonlinear covariance functions and handles non-Gaussian noise.Source: §5 · p. 10
• Unlike nonnegative and weighted tensor decomposition methods [1], InfTucker models nonlinear multiway interactions without explicitly solving core tensors.Source: §5 · p. 10
• The variational inference technique also reduces multi-task GP or t-process complexity from O(M^3N^3) time and O(M^2N^2) space to O(M^3 + N^3) time and O(N^2 + M^2) space.Source: §5 · p. 10

6 Experiments

• The experiments covered continuous and binary tensor data, comparing InfTucker with CP, Tucker decomposition, NCP, HOSVD, Weighted CP, and probabilistic Tucker decomposition.For larger and binary datasets, PTD used probit models and efficient computation techniques.Source: §6 · p. 11

6.1.1 Experimental setting

• The continuous experiments used amino, bread, and flow-injection tensors, with repeated 5-fold cross-validation for each dataset.The tensor dimensions were 5 × 51 × 201, 10 × 11 × 8, and 12 × 100 × 89, respectively.Source: §6.1.1 · p. 11

6.1.2 Results; 6.2.2 Results

• InfTuckergp outperformed prior approaches on all three continuous datasets, and InfTuckertp further outperformed InfTuckergp; both InfTucker models significantly outperformed other methods on binary networks.The binary comparison reports AUC across three networks and latent-factor dimensions r = 3, 5, 8, 10, 15, 20; the authors attribute InfTucker's advantages to its noise models and nonparametric Bayesian treatment.Source: §6.1.2 · p. 12, §6.2.2 · pp. 12–13, Figure 1 · p. 13

6.2.1 Experimental setting

• The binary experiments used Enron, Digg1, and Digg2 social-network tensors, which were very sparse.Their nonzero-element proportions were 0.01%, 0.024%, and 0.002%, respectively.Source: §6.2.1 · p. 12

• InfTucker models an observed tensor as a sample from a stochastic process on tensors, using tensor-variate Gaussian and t processes.Source: §7 · p. 14
• The framework models nonlinear interactions across tensor aspects, handles missing data and noise, and quantifies prediction confidence through predictive posterior distributions.Source: §7 · p. 14
• A variational method estimates InfTucker from data, and experiments on chemometrics and social-network datasets showed superior predictive performance over alternative tensor-decomposition approaches.Source: §7 · p. 14

• The proof assumes vec(W(r)) follows a zero-mean Gaussian distribution with identity covariance.Source: A Proof of Theorem 2 · p. 17
• Inverting equation (21) yields a relation between W(r) and φ(r)(U).Source: A Proof of Theorem 2 · p. 17
• An inner-product identity establishes covariance-function convergence, from which convergence in distribution follows.Source: A Proof of Theorem 2 · p. 17