PaperlayerView exact revision on arXiv

Source-linked AI summary

A Tutorial on Regularized Partial Correlation Networks

Sacha Epskamp, Eiko I. Fried

arXiv:1607.01367v9stat.APstat.ME

TL;DR

Psychological network analysis needs an interpretable way to estimate direct associations among observed variables while limiting spurious relationships. This tutorial explains regularized partial correlation networks, their implementation in R, and handling of ordinal data, then demonstrates the approach with PTSD symptoms. The examples show that changing the EBIC hyperparameter changes network sparsity, while the method requires choices about caution versus discovery and may omit true edges.

  • Problem

    The tutorial addresses the need for clear guidance on estimating and interpreting regularized partial correlation networks and handling common analysis problems.

  • Method

    The paper explains partial correlation networks, uses EBIC graphical LASSO with polychoric correlations for ordinal data, and demonstrates estimation in R.

  • Results

    In the PTSD example, networks contained 105 edges with γ = 0, 95 with γ = 0.25, and 87 with γ = 0.5.

  • Takeaways & Limitations

    Regularization produces sparser, more interpretable network estimates by removing edges likely to be spurious, while researchers must choose the hyperparameter according to the relative importance of caution or discovery.

  • Takeaways & Limitations

    The hyperparameter choice is somewhat arbitrary, and a conservative value such as γ = 0.5 may omit true edges.

Abstract

from arXiv · show

Recent years have seen an emergence of network modeling applied to moods, attitudes, and problems in the realm of psychology. In this framework, psychological variables are understood to directly affect each other rather than being caused by an unobserved latent entity. In this tutorial, we introduce the reader to estimating the most popular network model for psychological data: the partial correlation network. We describe how regularization techniques can be used to efficiently estimate a parsimonious and interpretable network structure in psychological data. We show how to perform these analyses in R and demonstrate the method in an empirical example on post-traumatic stress disorder data. In addition, we discuss the effect of the hyperparameter that needs to be manually set by the researcher, how to handle non-normal data, how to determine the required sample size for a network analysis, and provide a checklist with potential solutions for problems that can arise when estimating regularized partial correlation networks.

Introduction

The paper introduces regularized partial correlation networks for psychological research, explaining their interpretation, estimation in R, and common analytic challenges. It also addresses ordinal data, sample size, and methodological choices affecting network estimation.

  • Motivation: Psychological networks represent variables such as symptoms, moods, or attitudes as nodes and estimated statistical relationships as edges.
  • Core model: Partial correlation networks are the most common model for psychological data assumed to be continuous and multivariate normal.
  • Regularization: Regularization penalizes model complexity, producing sparse networks by removing edges likely to be spurious.It jointly performs model selection and parameter estimation.
  • Tutorial scope: The tutorial explains partial correlations, regularized network estimation, R commands, and solutions to common analytic problems.
  • Non-normal data: For ordinal data, the tutorial recommends EBIC graphical LASSO with polychoric correlations as input.The authors emphasize that this methodology can handle ordinal variables common in psychological research.
  • Researcher guidance: The authors provide an FAQ covering power analysis and sample-size recommendations, alongside solutions to recurring research challenges.

Partial Correlation Networks

Partial correlation networks encode associations between variables after conditioning on all others and can be obtained from precision matrices or node-wise regressions. Their interpretation supports modeling unique associations, multicollinearity, predictive mediation, and possible causal pathways, while requiring caution about latent variables and common effects.

  • Definition: Partial correlation coefficients range from −1 to 1 and encode the remaining association between two variables after conditioning on all other information.
  • Visualization: In a weighted network, nodes represent variables and edge weights represent partial correlation coefficients; exactly zero coefficients produce no edge.
  • Estimation: A precision matrix can be inverted from the variance–covariance matrix and standardized to obtain partial correlation coefficients.The precision matrix is the inverse of the variance–covariance matrix.
  • Estimation: Node-wise multiple regressions provide an alternative route to partial correlations, with corresponding regression slopes proportional to the same coefficient.The equivalence applies when coefficients are appropriately standardized.
  • Interpretation: Partial correlation networks model unique interactions and map multicollinearity and predictive mediation across variables.Indirect connections such as A–B–C can indicate that A and C are connected through B without a direct edge.
  • Causal interpretation: Partial correlations may indicate potential causal pathways when all relevant variables are observed, but they should not be equated with a causal-model skeleton.Common effects and latent variables can induce edges that complicate causal interpretation.

LASSO regularization

LASSO regularization penalizes model complexity to remove likely spurious edges and produce interpretable partial correlation networks, while EBIC selects among networks with different sparsity levels. The manually chosen EBIC hyperparameter γ trades sensitivity for specificity, and simulations illustrate this trade-off.

  • Sampling variation produces nonzero partial correlations even between conditionally independent variables, motivating methods that limit spurious edges.
  • LASSO can set parameter estimates exactly to zero, yielding sparse networks by penalizing the sum of absolute parameter values.
  • The tuning parameter λ controls sparsity, so estimating networks across different λ values produces structures ranging from fully connected to fully unconnected.
  • EBIC's hyperparameter γ is typically set between 0 and 0.5; higher values prefer simpler models, whereas γ = 0 favors discovery and γ = 0.5 favors parsimony.
  • Graphical LASSO estimates partial correlation networks by inverting the sample variance–covariance matrix, and is implemented in R's glasso package.
  • In a simulation, γ = 0 selected a network with three weak spurious edges, whereas γ = 0.25 and γ = 0.5 selected a network matching the true structure.
  • The best γ depends on the usually unknown true network structure, and γ = 0.5 may omit true edges because it is fairly conservative.
  • A sparse or empty LASSO network does not establish that omitted edges are truly zero, because noisy data can produce false negatives.

Non-normal data

Partial correlation networks assume multivariate normality and linear relationships, creating challenges for ordinal, continuous non-normal, and binary data. Transformations or alternative network models can address some cases, but latent-normality assumptions remain a limitation.

  • Partial correlation network estimation assumes multivariate normal distributions, including normal marginal distributions and linear relationships between variables.
  • Continuous non-normal variables can be transformed toward marginal normality using a nonparanormal transformation based on cumulative distributions.
  • Ordinal variables can be modeled with threshold functions that map latent normally distributed scores into ordered observed categories.
  • Polychoric and polyserial correlations estimate relationships between latent variables and can be supplied to glasso with EBIC model selection.
  • These ordinal-data methods assume normally distributed latent variables, which may be implausible for symptoms with a real zero point.
  • For all-binary data, the Ising Model is recommended instead of partial correlation networks, while mixed categorical and continuous data can use the Mixed Graphical Model.

Example

The PTSD example estimates regularized partial correlation networks in R for 20 symptoms from 221 people, then examines network structure and centrality across hyperparameter settings.

  • The example analyzes 20 PTSD symptoms reported by 221 people with a sub-threshold PTSD diagnosis.
  • The R workflow uses EBICglasso with cor_auto, which automatically detects ordinal variables and estimates the required correlations.
  • 105 edges occur at γ = 0, 95 at γ = 0.25, and 87 at γ = 0.5, showing fewer edges at the largest hyperparameter.
  • Node strength sums absolute connected edge weights, closeness uses inverse total distance, and betweenness counts shortest paths through each node.
  • Figure 5 displays centrality indices for the three networks using standardized z-scores, with higher values indicating greater node importance.

Sample size selection and replicability

Network analyses can produce many statistics whose replicability and interpretability vary with sample size, network structure, and data characteristics. The tutorial therefore recommends assessing sample-size adequacy both before and after data collection.

  • High-dimensional network estimation produces multiple potentially non-replicating measures, including edge weights, network structures, and centrality indices.
  • Stability and accuracy depend on sample size, the true network structure, and other characteristics of the data.
  • A priori analyses assess whether the sample suits the expected network structure, while post-hoc stability analyses evaluate the stability of obtained results.

A priori sample size analysis

The tutorial frames network power analysis around expected weighted network structures and recommends simulations across plausible structures, methods, and sample sizes to evaluate estimation performance.

  • Network sample-size analysis requires a prior expectation of the weighted network, including its zero and non-zero edges and edge strengths.
  • Sensitivity is the true-positive rate, specificity is the true-negative rate, and correlations compare true with estimated edge weights or centrality indices.
  • LASSO estimation generally has high specificity, whereas sensitivity and correlation increase with sample size; exact values depend strongly on the expected network structure.
  • The netSimulator varies expected network structures, estimation methods, sample sizes, repetitions, and data-generation conditions to inform required sample size.
  • In the PTSD simulation, N = 250 produced correlations above 0.8 for edge weights and strength and above 0.7 for sensitivity.
  • Specificity was moderate in that simulation, possibly because 54% of edges were zero in the generating network.

Post-hoc stability analysis

Post-hoc bootstrapping evaluates the accuracy and stability of estimated network parameters and centrality measures, revealing substantial uncertainty in the PTSD example, especially for some centrality indices.

  • Nonparametric bootstrapping constructs confidence intervals for regularized edge weights and tests differences between edge weights or centrality indices.
  • Case-dropping bootstrapping assesses centrality stability by repeatedly estimating networks after subsampling without replacement.
  • The PTSD bootstrap showed sizable sampling variation around edge weights and poor stability for closeness and betweenness.
  • Strength was more stable, although few nodes differed significantly from one another in strength.
  • The CS-coefficient quantifies the proportion of data that can be dropped while retaining, with 95% certainty, a correlation of at least 0.7 with original centrality coefficients.

Common Problems and Questions

The tutorial identifies common estimation and interpretation problems in regularized partial correlation networks, including sparse or implausibly dense structures, unstable polychoric estimates, unexpected negative edges, and misleading group comparisons or latent-variable interpretations.

  • Sparse networks: Low sample sizes can produce networks with few or no retained edges because EBIC favors more parsimonious structures.In the example, reducing the sample from 221 to 50 participants produced a network devoid of edges.
  • Dense or unstable networks: Extremely dense networks or partial correlations near 1 and −1 may indicate an invalid or unstable correlation matrix rather than a meaningful structure.Pairwise polychoric estimation can yield non-positive-definite matrices and unstable corrections.
  • Dense or unstable networks: Low expected frequencies and skewed scores can bias polychoric correlations, producing extreme or unexpected partial correlations.The authors recommend comparing polychoric- and Spearman-based networks; substantial differences suggest polychoric-estimation problems.
  • Unexpected negative edges: Unexpected negative edges can arise from conditioning on a common effect, including when a subsample is selected using a function such as a symptom sum score.Such subsamples can create spurious negative edges and should be interpreted with care.
  • Comparing networks: Network connectivity should only be compared across groups estimated with roughly equal sample sizes, or assessed using a pooled-data permutation test.EBIC depends on sample size, and the NetworkComparisonTest package implements the permutation approach.
  • Network models versus latent variables: A sparse estimated network does not establish that omitted edges are absent, because LASSO may exclude edges when statistical power is insufficient.Network modeling also does not by itself establish that the data come from a causal system rather than a common-cause model.

Conclusion

This tutorial addresses the need for an accessible introduction to regularized partial correlation networks by explaining their estimation, implementation, and common problems. Its methods assume independent cases, so temporal dependence in time-series data requires other approaches such as graphical VAR.

  • Conclusion: The methodology is appropriate when cases can reasonably be assumed independent, as in the paper’s cross-sectional example.The authors note that cross-sectional analyses do not separate within- and between-person variation.
  • Conclusion: Graphical VAR provides an alternative for estimating partial correlation networks while accounting for temporal information.LASSO-based EBIC model selection for such models is implemented in graphicalVAR and sparseTSCGM.
  • Conclusion: The paper provides an accessible overview of LASSO regularization with EBIC model selection for estimating sparse partial correlation networks.It also explains partial correlation networks, R commands, power analysis, stability and accuracy analyses, and common estimation problems.
  • Conclusion: The tutorial fills a gap in accessible descriptions of the commonly used LASSO-and-EBIC procedure while acknowledging that psychological network analysis remains a developing field.Several related challenges and future directions remain outside the tutorial’s scope.
Loading 1607.01367v9…