PaperlayerView exact revision on arXiv

Source-linked AI summary

High Dimensional Semiparametric Gaussian Copula Graphical Models

Han Liu, Fang Han, Ming Yuan, John Lafferty, Larry Wasserman

arXiv:1202.2169v3stat.ML

TL;DR

High-dimensional graphical-model estimation must accommodate non-Gaussian and contaminated data. The paper proposes the nonparanormal skeptic, which uses rank-based correlations within Gaussian copula models and plugs them into sparse graph estimators. It achieves optimal parametric rates for precision-matrix estimation and graph recovery, while simulations show stronger robustness under contamination.

  • Problem

    Gaussian graphical models assume Gaussian data, although data may be non-Gaussian or contaminated by outliers.

  • Method

    The nonparanormal skeptic directly estimates correlations with Spearman’s rho and Kendall’s tau, then applies existing parametric graph-estimation procedures.

  • Results

    The nonparanormal skeptic achieves optimal parametric rates for precision-matrix estimation and graph recovery, with stronger performance than normal-score methods as contamination increases.

  • Takeaways & Limitations

    Rank-based nonparanormal estimators provide a balance of statistical efficiency and robustness and are invariant to marginal transformations.

  • Takeaways & Limitations

    The estimated correlation matrix may be non-positive-semidefinite, causing some graphical-lasso algorithms to fail unless projection or alternative algorithms are used.

Abstract

from arXiv · show

In this paper, we propose a semiparametric approach, named nonparanormal skeptic, for efficiently and robustly estimating high dimensional undirected graphical models. To achieve modeling flexibility, we consider Gaussian Copula graphical models (or the nonparanormal) as proposed by Liu et al. (2009). To achieve estimation robustness, we exploit nonparametric rank-based correlation coefficient estimators, including Spearman's rho and Kendall's tau. In high dimensional settings, we prove that the nonparanormal skeptic achieves the optimal parametric rate of convergence in both graph and parameter estimation. This celebrating result suggests that the Gaussian copula graphical models can be used as a safe replacement of the popular Gaussian graphical models, even when the data are truly Gaussian. Besides theoretical analysis, we also conduct thorough numerical simulations to compare different estimators for their graph recovery performance under both ideal and noisy settings. The proposed methods are then applied on a large-scale genomic dataset to illustrate their empirical usefulness. The R language software package huge implementing the proposed methods is available on the Comprehensive R Archive Network: http://cran. r-project.org/.

1. Introduction.

The paper addresses high-dimensional graphical-model estimation when Gaussian assumptions are unrealistic or data are contaminated. It proposes the nonparanormal skeptic, using rank-based correlations to combine modeling flexibility with robust, rate-optimal estimation.

  • Gaussian graphical models assume X is Gaussian, but real data may be non-Gaussian or contaminated by outliers.
  • The nonparanormal skeptic directly estimates the correlation matrix with Spearman’s rho or Kendall’s tau, avoiding explicit marginal-transformation estimation.The estimated correlation matrix is then used with graphical lasso, CLIME, or the graphical Dantzig selector.
  • The method achieves optimal parametric convergence rates for both precision-matrix estimation and graph recovery despite the larger nonparanormal model family.
  • Simulations show a robustness–efficiency tradeoff: normal-score estimators are slightly more efficient under low contamination, whereas the skeptic performs better under higher contamination.
  • The paper compares rank-based and normal-score estimators in simulations and applies the proposed methods to a large-scale genomic dataset.

2. Background.

The background defines nonparanormal graphical models, reviews normal-score estimation, and introduces rank-based correlation estimation for high-dimensional precision-matrix and graph recovery. The proposed skeptic plugs these correlation estimates into existing sparse-precision procedures.

  • Nonparanormal model: A nonparanormal distribution transforms each variable with a monotone univariate function so the transformed vector is multivariate Gaussian.
  • Graph representation: For nonparanormal data, zeros in the inverse correlation matrix encode conditional independences and therefore determine the undirected graph.
  • Normal-score estimator: The normal-score estimator transforms observations using empirical normal scores, then applies the graphical lasso to estimate the graph.
  • Normal-score estimator: The normal-score estimator’s high-dimensional rate includes an additional dependence on s, d, and logarithmic factors, and the paper states that these rates are not optimal.
  • Nonparanormal skeptic: The nonparanormal skeptic uses Spearman’s rho and Kendall’s tau, which are invariant under monotone transformations, to estimate the underlying correlation matrix directly.
  • Sparse estimation procedures: The estimated correlation matrix is plugged into graphical lasso, CLIME, graphical Dantzig, or Meinshausen–Bühlmann procedures for sparse precision-matrix and graph estimation.
  • Computational considerations: The estimated correlation matrix may be non-positive-semidefinite, which can prevent some graphical-lasso algorithms from converging; projection or alternative procedures can address this issue.
  • Theory: The paper’s main theoretical result shows that the Spearman- and Kendall-based estimators achieve optimal rates.

4. Theoretical Properties.

The nonparanormal skeptic estimates correlation structure with rank-based statistics and retains optimal high-dimensional rates when combined with parametric graphical procedures. Its guarantees cover concentration, precision-matrix estimation, and graph recovery under the nonparanormal model.

  • Concentration Properties: Spearman’s rho and Kendall’s tau estimators have fast exponential concentration to the underlying correlation matrix in the max norm.This concentration permits existing analyses of parametric graphical methods to be reused.
  • Main Theorem: Theorem 4.3 shows that plugging bSρ or bSτ into graphical lasso, graphical Dantzig, or CLIME achieves the same parametric convergence rates for precision estimation and graph recovery.The conditions are those that ensure consistency and graph recovery for the corresponding methods under Gaussian models.
  • Minimax Optimality: The corresponding nonparanormal skeptic estimators are rate optimal over parameter spaces where graphical lasso, graphical Dantzig, or CLIME are rate optimal under Gaussian models.This extends the rate-optimality guarantee to the larger nonparanormal model.
  • Implications: The nonparanormal skeptic can serve as a safe replacement for Gaussian graphical models because its added flexibility and robustness come at almost no cost in convergence rates.The paper frames this conclusion specifically in terms of rates of convergence.
  • Minimax Optimality: The inverse correlation matrix estimator achieves the parametric rate and is minimax rate optimal over M1(κ, τ, M) in ℓ1-risk.The minimax lower bound is established for inverse correlation matrix estimation over the same class.
  • Additional Result: Estimated marginal transformations converge uniformly over an expanding interval, supporting downstream nonparanormal applications such as discriminant analysis and principal component analysis.The paper notes that details of these downstream applications will appear in follow-up work.

5. Experimental Results.

Experiments compare normal-score, rank-based nonparanormal, and Gaussian graph estimators under Gaussian and non-Gaussian data with varying outlier contamination. Rank-based methods are especially advantageous as contamination increases, while performance reflects a robustness–efficiency tradeoff.

  • Summary of Results: For non-Gaussian data without outliers, npn-ns, npn, npn-spearman, and npn-tau perform similarly and all outperform normal.The reported comparison is npn-ns ≈ npn ≈ npn-spearman ≈ npn-tau ≫ normal.
  • Summary of Results: For non-Gaussian data with higher outlier contamination, npn-tau outperforms npn-spearman, which significantly outperforms npn and npn-ns; normal performs worst.The reported ordering is npn-tau > npn-spearman ≫ npn > npn-ns ≫ normal.
  • Summary of Results: For Gaussian data without outliers, normal, npn-ns, and npn perform similarly, while npn-spearman and npn-tau are slightly weaker.The reported comparison is normal ≈ npn-ns ≈ npn >slightly npn-spearman ≈ npn-tau.
  • Tradeoff: Across simulations, npn-tau and npn-spearman provide a good balance of statistical efficiency and robustness, especially when outliers contaminate the data.Both rank-based methods outperform normal-score-based alternatives under contamination, even at low levels.
  • Robustness: Rank-based estimators are invariant to marginal transformations, whereas Gaussian estimators lose performance when non-identity transformations are applied.The methods are also applied to a large-scale genomic dataset.

5.2. Numerical Simulations.

The simulations compare semiparametric and parametric graph estimators across transformations and contamination settings using glasso and Meinshausen–Bühlmann procedures. Rank-based nonparanormal skeptic estimators are generally robust, with a tradeoff between efficiency in clean data and robustness under contamination.

  • Simulation Design: The estimators use a two-step procedure: estimate the correlation matrix, then plug it into a parametric graph estimator.The parametric baselines are graphical lasso and Meinshausen–Bühlmann; CLIME and graphical Dantzig selector show similar behavior, respectively.
  • Non-Gaussian Data: With no contamination under non-Gaussian transformations, rank-based and normal-score nonparanormal estimators perform comparably and outperform the Gaussian estimator.This pattern is reported for the power and Gaussian CDF transformations.
  • Contaminated Data: At low outlier levels, npn-spearman and npn-tau significantly outperform npn and npn-ns, while all semiparametric methods outperform the corresponding parametric methods.The reported low levels are r = 0.01 for deterministic contamination and r = 0.1 for random contamination.
  • Contaminated Data: At higher contamination, npn-spearman and npn-tau outperform npn and npn-ns; npn-tau is strongest, suggesting greater robustness than Spearman’s rho.For deterministic contamination, oracle FPR and FNR do not show significant improvement over npn and npn-ns because the oracle score can coincide with ROC-curve intersections; under random contamination, the skeptic methods uniformly dominate them.
  • Gaussian Data: With truly Gaussian data and no outliers, normal, npn-ns, and npn are slightly better than the skeptic methods, indicating a negligible efficiency loss for the latter.When r > 0, parametric methods degrade immediately, whereas npn-tau is the most robust and Spearman’s method behaves similarly.

5.3. Gene Expression Data.

The study applies graphical-model estimators to large-scale gene-expression data, comparing methods through differential networks at gene and tissue levels. The results indicate substantial differences between normal, rank-based, and normal-score approaches, especially for gene networks.

  • Data analysis: The analysis estimates gene and tissue networks after preprocessing integrated genomic data to address batch effects and normalization.The gene analysis uses 2,000 highly variable genes; tissue analysis averages replicate arrays and retains 2,714 tissue types.
  • Network comparison: The estimated tissue graph is denser than the gene graph, so the study focuses on differential rather than whole-network visualization.Both graphs contain around 2,000 nodes and more than 1,500 edges.
  • Visualization: Differential-network figures encode edges unique to one method in red and edges unique to the other in black.The comparison panels include npn-sp versus normal and npn-sp versus npn-ns at gene and tissue levels.
  • Gene network results: At the gene level, npn-sp contains 1,235 edges absent from the normal graph, corresponding to approximately 80% of its 1,557 edges.This difference is interpreted as evidence that the gene-expression data are highly non-Gaussian.
  • Gene network results: At the gene level, 691 of 1,557 npn-sp edges, approximately 45%, are absent from the npn-ns graph, suggesting substantial outlier contamination.The paper relates this pattern to the dataset’s integration from many sources.
  • Tissue network results: Tissue-level differences are smaller: 22% of npn-sp edges are absent from the normal graph, while 12% of normal edges are absent from npn-sp.The paper suggests stronger tissue-level signal, while noting possible uninterpreted lab effects.

6. Conclusions and Discussion.

The supplied passage records acknowledgments to contributors who provided comments and suggestions.

  • The authors acknowledge Professor Peter Bühlmann, the Associate Editor, and three referees for helpful comments and suggestions.

A.1. Proof of Proposition 3.1.

The proof connects the rank-based correlation quantities to their population definitions, deriving the Spearman correlation representation through distributional identities.

  • The proof obtains the result for τ_jk directly from the definition of τ_jk.
  • For ρ_jk, the proof uses the fact that F_j(X_j) is Uniform[0, 1] to derive an expectation-based representation.
  • The derivation combines identities involving C and D, using C(j,1,2;k,1,3)=1−D(j,1,2;k,1,3).

A.2. Proof of Theorem 4.1.

The proof analyzes rank-based correlation estimators using U-statistic representations and concentration inequalities, while accounting for the finite-sample bias of Spearman’s rho.

  • The proof rewrites Spearman’s rho as a third-order U-statistic with a bounded but asymmetric kernel.
  • Hoeffding’s inequality for asymmetric kernels is applied after assuming n can be divided by 3.
  • Unlike the Kendall estimator, the Spearman estimator is biased, so the proof uses a separate finite-sample bias equation.
  • The bias term a_jk is bounded in magnitude by π/(n−2).

A.3. Proof of Theorem 4.2.

The proof establishes unbiasedness of the Kendall’s tau estimator and derives its concentration using its U-statistic representation and a bounded kernel.

  • The estimator bτ_jk is unbiased for τ_jk, since E[bτ_jk] = τ_jk.
  • The estimator bτ_jk can be represented as a U-statistic.
  • Its kernel is bounded between −1 and 1, enabling Hoeffding’s inequality for U-statistics.
  • Applying Hoeffding’s inequality yields the concentration result stated as equation (4.2).
  • The proof section also introduces Gaussian distribution notation and preliminary lemmas for later asymptotic analysis.

A.4. Proof of Theorem 4.6.

The proof of Theorem 4.6 combines Gaussian-maxima bounds, standardized empirical-process limits, and deterministic comparisons to control the transformed empirical distribution. It uses interval-specific arguments, truncation, and the mean value theorem to complete the result.

  • Pickands’ lemma supplies an almost-sure bound for the maximum of independent standard Gaussian variables.
  • Lemma A.2 bounds the empirical distribution function eF_j(t) away from 0 and 1 on the relevant intervals for sufficiently large n.
  • A standardized empirical-process limit controls deviations of the empirical distribution over sequences of sub-intervals.
  • Lemma A.4 compares the random sequence with a deterministic sequence, producing an almost-sure inequality involving generic constants and the Gaussian density.
  • The argument applies the mean value theorem and Φ^-1, using Fj(gj(t)) = Φ(t), then handles truncated values before concluding the proof.
  • The proof partitions the analysis into sub-intervals and exploits symmetry to reduce attention to one sub-interval.
Loading 1202.2169v3…