Source-linked AI summary
High Dimensional Semiparametric Latent Graphical Model for Mixed Data
Jianqing Fan, Han Liu, Yang Ning, Hui Zou
TL;DR
High-dimensional graphical models and sparse PCA have been developed mainly for continuous or Gaussian data, leaving discrete and mixed data as a modeling challenge. This paper proposes a latent Gaussian copula model and rank-based methods for latent graph estimation and PCA, with convergence and recovery guarantees under stated conditions.
Problem
Graphical models and sparse PCA are principally developed for continuous or Gaussian data, while many applications contain discrete or mixed variables and high-dimensional sparse PCA for discrete data has received little attention.
Source: §1 · p. 2Method
The paper models binary and mixed observations through latent Gaussian-copula variables and uses rank-based approaches for latent graph estimation and principal component analysis.
Source: §1 · pp. 2–3Results
The proposed methods attain the same convergence rates for precision-matrix and eigenvector estimation as if latent variables were observed, with graph recovery and leading-eigenvector feature-selection consistency under similar conditions.
Source: §1 · p. 3Takeaways & Limitations
The methods provide a theoretically supported way to study dependence structure and variation patterns in unobserved latent variables.
Source: §8 · p. 23The paper focuses on binary data; although extension to discrete variables with more than two categories is described, its properties are left for future investigation.
Source: §8 · p. 23
Abstract
from arXiv · showhide
Graphical models are commonly used tools for modeling multivariate random variables. While there exist many convenient multivariate distributions such as Gaussian distribution for continuous data, mixed data with the presence of discrete variables or a combination of both continuous and discrete variables poses new challenges in statistical modeling. In this paper, we propose a semiparametric model named latent Gaussian copula model for binary and mixed data. The observed binary data are assumed to be obtained by dichotomizing a latent variable satisfying the Gaussian copula distribution or the nonparanormal distribution. The latent Gaussian model with the assumption that the latent variables are multivariate Gaussian is a special case of the proposed model. A novel rank-based approach is proposed for both latent graph estimation and latent principal component analysis. Theoretically, the proposed methods achieve the same rates of convergence for both precision matrix estimation and eigenvector estimation, as if the latent variables were observed. Under similar conditions, the consistency of graph structure recovery and feature selection for leading eigenvectors is established. The performance of the proposed methods is numerically assessed through simulation studies, and the usage of our methods is illustrated by a genetic dataset.
1 Introduction
The paper addresses the challenge of learning graphical structure and principal components for discrete or mixed data. It proposes a latent Gaussian copula model and rank-based methods for studying dependence and variation in unobserved variables.
1 Introduction
- Gaussian and Gaussian-copula graphical models target continuous data, while many applications contain discrete variables or mixtures of discrete and continuous variables.Existing work also leaves high-dimensional sparse PCA for discrete data relatively unexplored.Source: §1 · p. 2
- The proposed semiparametric latent Gaussian copula model treats observed discrete values as discretizations of latent continuous variables.It includes the latent Gaussian model when the latent variables are multivariate Gaussian.Source: §1 · p. 2
- The paper targets conditional independence and variation patterns among latent variables, complementing methods focused on relationships among observed variables.The authors argue that latent conditional-independence structure can provide deeper understanding of the unknown mechanism.Source: §1 · p. 3
- Rank-based methods estimate latent graphs and conduct latent PCA, with convergence rates matching those available if latent variables were observed.Under similar conditions, the paper establishes graph-structure recovery and leading-eigenvector feature-selection consistency.Source: §1 · p. 3
2 Background
The background establishes notation and reviews the Gaussian copula model. In that model, monotone marginal transformations yield latent Gaussian variables, enabling rank-based correlation estimation without estimating the transformations.
2.1 Notation
- The notation section defines vector and matrix norms, support, eigenvalue ordering, and matrix indexing used in the later development.It also defines inner products, diagonal matrices, and the identity matrix.Source: §2.1 · pp. 3–4
2.2 Gaussian Copula Model
- A Gaussian copula, or nonparanormal, model applies increasing marginal transformations so the transformed variables are jointly Gaussian with identity diagonal covariance.The inverse-correlation matrix encodes conditional independence in this model.Source: §2.2 · p. 4
- Rank-based correlations such as Spearman’s rho and Kendall’s tau estimate latent correlation without estimating the marginal transformations.The resulting estimate can be used in Gaussian graphical-model precision-matrix procedures.Source: §2.2 · p. 4
3 Latent Gaussian Copula Model for Binary Data
For binary data, the model represents each observed value as a thresholded latent variable following a Gaussian-copula distribution. The section estimates latent correlations from observed-data Kendall’s tau and establishes uniform error control under stated conditions.
3.1 The model
- Binary observations are indicators that latent nonparanormal variables exceed unknown cutoffs; multivariate Gaussian latent variables give the latent Gaussian special case.For binary outcomes, the paper states that the latent Gaussian copula and latent Gaussian model families are equivalent.Source: §3.1 · pp. 4–5
- The latent precision matrix’s zero pattern characterizes conditional independence among the latent variables, whose distributions may be skewed or multimodal.Only the transformed cutoffs are identifiable from binary data under the model.Source: §3.1 · p. 5
3.2 Rank Based Latent Correlation Matrix Estimator
- The estimator inverts the Kendall’s tau bridge function at estimated cutoffs to recover each latent correlation.The bridge is strictly increasing in latent correlation; the cutoffs are estimated from the observed binary-variable means.Source: §3.2 · p. 6
- Under conditions excluding perfectly collinear latent-variable pairs and controlling transformed-cutoff variation, the maximum estimation error sup_j,k |bR_jk − Σ_jk| is at most C log d/n with probability greater than 1 − d^−1.Source: §3.2 · p. 7
4 Latent Graph Structure Learning for Binary Data
The section adapts Gaussian graphical-model estimators to latent binary-data graphs by replacing the sample covariance with rank-based estimates. It establishes estimation and graph-recovery guarantees, including performance matching the case where latent variables are observed.
4.1 Graphical Lasso Estimator
- The rank-based correlation estimate replaces the unavailable latent sample covariance in graphical Lasso, and projection to the positive-semidefinite cone supports convergence of the coordinate-descent algorithm.The unprojected estimate may fail to be positive semidefinite, creating computational problems.Source: §4.1 · p. 8
4.2 CLIME Estimator
- CLIME is constructed by replacing its covariance input with the rank-based estimate and symmetrizing the resulting precision-matrix estimate.The paper uses the corresponding Gaussian-model convex-program procedure.Source: §4.2 · p. 8
4.3 Adaptive Graphical Lasso Estimator
- The adaptive graphical Lasso uses a nonconvex penalty to reduce ℓ1-penalty bias, with local linear approximation converting the optimization into weighted ℓ1 problems.Its adaptive weights use an initial graphical Lasso or CLIME estimate.Source: §4.3 · pp. 8–9
4.4 Theoretical Properties
- CLIME sign recovery holds with probability tending to one when every nonzero precision-matrix entry exceeds twice the threshold.The threshold is required to satisfy τ > 4Kλ.Source: §4.4 · p. 10
- The proposed CLIME estimator achieves the same rates for matrix estimation and graph recovery as if the latent variables were observed.The graph-recovery guarantee holds under the stated assumptions and sparsity-class condition.Source: §4.4 · p. 10
- Under the cited conditions, the adaptive graphical Lasso equals the estimator formed with the true support set with high probability.The paper calls this the strong oracle property.Source: §4.4 · p. 10
5 Latent Principal Component Analysis for Binary Data
The section estimates leading eigenvectors of the latent correlation matrix from a rank-based estimator, using sparsity-constrained PCA for binary data. It gives convergence and support-recovery guarantees under stated conditions.
5.1 Eigenvector Estimation via Two-stage Method
- The cardinality-constrained sparse PCA objective is NP-hard, motivating a convex relaxation that can be solved efficiently by ADMM.The leading eigenvector of the relaxed solution is used as the initial estimate.Source: §5.1 · p. 11
- The two-stage procedure initializes truncated power with the leading eigenvector from an ADMM-based estimator, then iteratively truncates and normalizes to obtain a k-sparse estimate.The resulting estimator is typically different from the global maximizer; convergence is established by Yuan and Zhang (2013).Source: §5.1 · pp. 11–12
5.2 Theoretical Properties
- For fixed k0, k, λ1, and λ2, the two-stage estimator’s rate agrees with the minimax optimal rate available if the latent variables were observed.When k0 increases with n, the stated rate is suboptimal; a refined truncated-power analysis under additional conditions achieves optimal scaling for k.Source: §5.2 · p. 12
- The section also establishes a sure-screening property for the supports of both the constrained and two-stage eigenvector estimators.The support sets are defined for the true eigenvector and both estimates.Source: §5.2 · p. 13
- Under similar minimum-signal assumptions to those used for Gaussian and Gaussian-copula data, both sparse eigenvector estimators achieve feature-selection consistency.The theorem’s proof uses the convergence rate and minimum-signal conditions; an improved rate can relax the latter condition.Source: §5.2 · p. 13
6 Latent Gaussian Copula Model for Mixed Data
The mixed-data model combines continuous observations with thresholded latent variables for binary observations. A rank-based estimator connects observed Kendall’s tau to latent correlations, after which the paper applies its graph-estimation and scale-invariant PCA procedures.
6.1 A Model for Mixed Data
- The model represents binary variables as thresholded latent continuous variables, with the combined continuous and latent vector following a Gaussian copula model.The latent Gaussian model is the special case where the latent vector is multivariate Gaussian; the copula family is strictly larger.Source: §6.1 · p. 13
6.2 Rank Based Latent Correlation Matrix Estimator
- For a binary-continuous pair, population Kendall’s tau is linked to the latent correlation by an invertible bridge function when the binary threshold parameter is fixed.The rank-based estimator uses the inverse bridge function, with the threshold parameter estimated from the binary sample proportion.Source: §6.2 · pp. 14–15
- The paper establishes error bounds for the rank-based latent correlation estimator and then reuses its precision-matrix and scale-invariant PCA procedures with the same theoretical guarantees.The mixed-data results are stated under assumptions (A1’) and (A2’).Source: §6.2 · p. 15
7 Numerical Results
Simulations compare latent-data methods with methods based on observed binary or mixed data, while the Arabidopsis analysis illustrates graph estimation and dimension reduction. Across these settings, the proposed latent methods perform favorably against AMLE and naive methods, with results varying by task and scenario.
7.1 Simulation Results for Graph Estimation
- The graph simulations use 100 replicates at n = 50, 100, and 200 across binary and mixed-data scenarios, including misclassification.The six methods include latent estimators, observed-data baselines, and benchmarks that assume the latent variables are observed.Source: §7.1 · p. 16
- L-GLASSO and L-GSCAD have higher graph-recovery TPR than AMLE at fixed FPR across four scenarios, and L-GSCAD improves on L-GLASSO.Without misclassification, L-GLASSO and the latent-data benchmark ZR-GLASSO have similar ROC curves; under misclassification, L-GLASSO and L-GSCAD are more robust than ZR-GLASSO and ZP-GLASSO.Source: §7.1 · pp. 16–17, Figure 1 · p. 17
- L-GLASSO has smaller precision-matrix estimation error than AMLE and the naive method across scenarios, while L-GSCAD further reduces error.Both latent estimators are robust to misclassification and the non-Gaussian continuous components; without misclassification, L-GLASSO has slightly larger error than ZR-GLASSO.Source: §7.1 · pp. 17–18, Table 1 · p. 18
7.2 Simulation Results for Principal Component Analysis
- L-PCA outperforms AMLE-PCA and Naive-PCA in feature selection across all four scenarios and is only slightly inferior to ZR-PCA.Figure 2 plots false-positive rate against true-positive rate for recovery of the sparse leading eigenvector.Source: Figure 2 · p. 19, §7.2 · p. 20
- L-PCA has smaller leading-eigenvector estimation error than AMLE-PCA and Naive-PCA across scenarios, with moderate information loss relative to ZR-PCA in three scenarios.In the misclassified binary-data scenario, L-PCA shows some advantage over ZR-PCA and ZP-PCA because it is less affected by extreme latent values.Source: Table 2 · p. 20, §7.2 · pp. 20–21
7.3 Analysis of Arabidopsis Data
- The Arabidopsis illustration analyzes 39 pathway-related genes measured across 118 microarrays and converts expression values to binary data to address batch effects.The paper notes that discretization entails information loss.Source: §7.3 · p. 21
- In Arabidopsis graph analysis, L-GLASSO yields 30%–60% different edges from AMLE and the naive method, depending on graph sparsity.At 10 total edges, only L-GLASSO recovers the CMK–MCT and CMK–MECPS dependencies, which the paper notes are known associations.Source: §7.3 · p. 21, Table 3 · p. 22
- For every considered support size, L-PCA's Arabidopsis match rate exceeds 80% and is significantly larger than AMLE-PCA's and Naive-PCA's.The selected genes tend to belong to one of the two pathway groups and are more likely to interact in the same biological pathway.Source: Table 4 · p. 22, §7.3 · p. 22
8 Discussion
The discussion presents a latent Gaussian copula model for binary and mixed data, using rank-based methods for latent graph estimation and PCA, with theoretical properties established. The PCA procedure focuses on the leading eigenvector, while extensions to sparse principal subspaces and discrete variables with more than two categories are described as possibilities.
8 Discussion
- Rank-based methods estimate latent graphs and conduct PCA for latent variables, with theoretical properties established for both procedures.Source: §8 · p. 23
- The PCA procedure focuses on the leading eigenvector; sparse principal subspaces could instead be estimated with a Fantope projection approach and refined by a truncated power method.Source: §8 · p. 23
- The methods can be extended to discrete variables with more than two categories, but properties of that procedure remain for future investigation.Source: §8 · p. 23
A.1 Proof of Lemma 3.1
The proof establishes that F(t; ∆j, ∆k) is strictly increasing in t by showing its derivative is positive across two cases. It justifies differentiating under the integral and analyzes the derivative's sign in each case.
A.1 Proof of Lemma 3.1
- The proof interchanges differentiation and integration using the dominated convergence theorem, relying on Φ(x) < 1.Source: §A.1 · p. 23
- It establishes derivative positivity separately for ∆j < t∆k and ∆j ≥ t∆k, using a nonnegative integrand in the first case and monotonicity in ∆j in the second.Source: §A.1 · p. 24
- F(t; ∆j, ∆k) is strictly increasing in t because its partial derivative with respect to t is positive.Source: §A.1 · pp. 23–24
A.2 Proof of Lemma 3.2
The proof establishes a uniform lower bound on the derivative of F, which yields a Lipschitz bound for its inverse. It treats two cases for the cutoff parameters and chooses a bound independent of them.
A.2 Proof of Lemma 3.2
- A uniform lower bound on ∂F(t; ∆j, ∆k)/∂t yields a Lipschitz bound for F^-1(τ; ∆j, ∆k).Source: §A.2 · p. 24
- For ∆j < t∆k, the proof uses a nonnegative integrand and bounds the derivative from below using η = min{−|t∆k| − 1, ∆j}.Source: §A.2 · p. 24
- For ∆j ≥ t∆k, a change of variable helps bound the derivative; taking L2 = max{L′, L′′} makes the bound independent of ∆j and ∆k.Source: §A.2 · pp. 24–25
A.3 Proof of Theorem 3.1
The appendix proves results for correlation estimation, graph estimation, eigenvector estimation, and mixed-data estimation using inverse-function bounds, concentration inequalities, and existing results. It also establishes monotonicity for the mixed-data bridge function and a support-containment probability bound.
A.3 Proof of Theorem 3.1
- A Lipschitz bound for Φ^-1 on a bounded interval supports the proof of Theorem 3.1, which combines cutoff and Kendall’s tau concentration bounds.Source: §A.3 · pp. 25–26
A.4 Proof of Theorem 4.1
- Theorem 4.1’s proof bounds the estimator’s maximum-entry error by twice the corresponding error for the rank-based correlation estimator, then applies Theorem 3.1.Source: §A.4 · p. 27
A.5 Proof of Theorem 4.2
- Theorem 4.2’s proof follows arguments from Theorems 1, 4, and 7 of Cai et al. (2011), with details omitted.Source: §A.5 · p. 27
A.6 Proof of Theorem 5.1
- Theorem 5.1’s proof relates eigenvector error to the squared sine of the angle between vectors and invokes Theorem 3.1.Source: §A.6 · pp. 27–28
A.7 Proof of Theorem 5.2
- By Theorem 5.1, the probability of the event that V1 is not contained in bV1(k) is less than 1/d.Source: §A.7 · p. 28
A.8 Proof of Lemma 6.3
- The proof of Lemma 6.3 shows H(t; ∆j) is strictly increasing in t and therefore invertible.Source: §A.8 · p. 29
A.9 Proof of Theorem 6.1
- Under Conditions (A1’) and (A2’), H−1(τ; ∆j) is Lipschitz in τ uniformly over |∆j| ≤ M and |τ| ≤ 1−δ, for a constant L3.Source: §A.9 · p. 29