PaperlayerView exact revision on arXiv

Source-linked AI summary

The Gaussian Graphical Model in Cross-sectional and Time-series Data

Sacha Epskamp, Lourens J. Waldorp, René Mõttus, Denny Borsboom

arXiv:1609.04156v6stat.MEstat.AP

TL;DR

Psychological network analyses need ways to distinguish relationships across independent, temporal, and multilevel data structures. This paper develops GGM-based estimation procedures for these settings, yielding temporal, contemporaneous, and between-subjects networks. Under stated assumptions, these networks characterize predictive relationships over time, within time, and on average.

  • Problem

    Cross-sectional analyses cannot disentangle short-term within-subjects relationships from between-subjects relationships, limiting interpretation of observed associations.

  • Method

    The paper extends GGMs to independent, temporally ordered, and multilevel data by proposing estimation procedures for temporal, contemporaneous, and between-subjects networks.

  • Results

    The paper separates variance into contemporaneous, temporal, and between-subjects network structures and showcases the methods in two empirical examples.

  • Takeaways & Limitations

    Under multivariate normality, stationarity, and Lag-1 factorization, the networks show how variables predict each other over time, within time, and on average.

  • Takeaways & Limitations

    Causal interpretations remain exploratory because the methods generate hypotheses but do not confirm causal relations.

Abstract

from arXiv · show

We discuss the Gaussian graphical model (GGM; an undirected network of partial correlation coefficients) and detail its utility as an exploratory data analysis tool. The GGM shows which variables predict one-another, allows for sparse modeling of covariance structures, and may highlight potential causal relationships between observed variables. We describe the utility in 3 kinds of psychological datasets: datasets in which consecutive cases are assumed independent (e.g., cross-sectional data), temporally ordered datasets (e.g., n = 1 time series), and a mixture of the 2 (e.g., n > 1 time series). In time-series analysis, the GGM can be used to model the residual structure of a vector-autoregression analysis (VAR), also termed graphical VAR. Two network models can then be obtained: a temporal network and a contemporaneous network. When analyzing data from multiple subjects, a GGM can also be formed on the covariance structure of stationary means---the between-subjects network. We discuss the interpretation of these models and propose estimation methods to obtain these networks, which we implement in the R packages graphicalVAR and mlVAR. The methods are showcased in two empirical examples, and simulation studies on these methods are included in the supplementary materials.

Introduction · The Gaussian Graphical Model

The paper presents the Gaussian graphical model as an exploratory network framework for cross-sectional and time-series psychological data, extending it to temporal, contemporaneous, and between-subjects structures. It defines GGM edges as conditional associations and discusses interpretations ranging from prediction to potential causal effects under strong assumptions.

  • Introduction: Psychological network models conceptualize observed processes as emergent behavior in which psychological, biological, and sociological components directly affect one another.
  • Introduction: A GGM estimates an undirected network of partial correlation coefficients for continuous, normally distributed data, conditioning each association on all other variables.
  • Introduction: GGMs can be estimated on repeated measures assumed temporally independent, while VAR models extend them by accounting for dependence between consecutive cases.
  • Introduction: The paper separates variance into contemporaneous, temporal, and between-subjects network structures and proposes estimation procedures implemented in mlVAR and graphicalVAR.
  • The Gaussian Graphical Model: The GGM represents variables as nodes and nonzero conditional associations as weighted edges derived from the standardized precision matrix.
  • The Gaussian Graphical Model: A zero partial correlation produces no edge, indicating conditional independence between two variables given the remaining variables.
  • The Gaussian Graphical Model: GGMs can always be interpreted as showing predictive effects, whereas causal interpretations require additional assumptions and may yield equivalent or over-saturated models.
  • The Gaussian Graphical Model: The paper focuses on causal-effect interpretations while also describing predictive use and relating GGMs to factor analysis and extensions of factor modeling.

The Gaussian Graphical Model and Multiple Regressions

Gaussian graphical models encode mutual predictability among variables through edges and regression coefficients, with these coefficients directly related to the inverse variance–covariance matrix. Zeros in that matrix indicate absent predictive relations, making mutual prediction the basis for interpreting network edges.

  • Predictive interpretation: A GGM edge indicates that one node predicts a connected node after controlling for all other network nodes.This relationship is represented by least-squares regression coefficients and the inverse variance–covariance matrix.
  • Predictive interpretation: The coefficient γij encodes how well variable j predicts variable i, with predictive effects that are naturally symmetric.If insomnia predicts fatigue, fatigue correspondingly predicts insomnia, so γij is proportional to γji.
  • Matrix interpretation: A zero in the inverse variance–covariance matrix indicates that one variable does not predict another.κij is proportional to both γij and γji, linking matrix zeros to absent predictive relations.
  • Applications: The network’s predictive quality is the cornerstone of applications that assess how variables predict each other.The passage gives recommender systems as an example, where product preferences inform recommendations.

The Gaussian Graphical Model and Structural Equation Modeling · Causal models.

The paper places GGMs and causal models within a broader SEM framework, while emphasizing that exploratory causal-model estimation is underidentified and plagued by equivalent specifications. Compared with exploratory DAG estimation, GGMs provide a unique, undirected representation, although conditioning on common effects can create spurious edges and limits causal interpretation.

  • The Gaussian Graphical Model and Structural Equation Modeling: The SEM framework represents observed variables, unobserved causes, causal effects, and factor loadings through jointly normal variables and structured matrices.The framework includes a diagonal Φ, a causal-effects matrix B, and a factor-loading matrix Λ; the covariance of ηC can be modeled in different ways.
  • The Gaussian Graphical Model and Structural Equation Modeling: SEM supports confirmatory testing of causal models, whereas exploratory estimation must proceed without assuming a fully specified causal structure.After conditioning on all causes, Φ indicates independent variables, while the SEM expression accommodates more complicated model setups through ηC covariance structures.
  • Causal models.: With no unobserved causes, B encodes the causal model among variables in yC, but exploratory estimation is underidentified because Σ has m(m + 1)/2 elements versus m parameters in Φ and m(m−1) in B.Without stringent restrictions, the number of model parameters exceeds the information available in Σ.
  • Causal models.: Different specifications of B can generate the same Σ, so exploratory causal-model search may yield equivalence classes of equally fitting DAGs with different causal interpretations.Cholesky and eigendecompositions are examples of alternative decompositions producing equivalently fitting B; the number of equivalent models increases with more variables.
  • Causal models.: The undirected GGM is saturated when all edges are present, has one unique inverse for Σ, avoids questionable causal directions and acyclicity assumptions, and is parameterized by partial correlations.Its concentration matrix K contains the same number of unique elements as Σ, unlike the exploratory causal formulation.
  • Causal models.: In the causal formulation, sparsity in B corresponds directly to sparsity in K, and κi j equals zero when variables lack a directed edge and common effect under the stated assumptions.The GGM therefore operates at a comparable structural level to causal modeling while representing conditional independence through concentration parameters.
  • Causal models.: Conditioning on a common effect can induce a spurious GGM edge, so an unexpected edge sign may indicate a collider rather than a direct causal relationship.Two positive causal effects on a third variable can produce a negative partial correlation; GGMs therefore do not necessarily estimate the causal skeleton.

Factor models. … Estimation.

Factor models and GGMs are closely connected because latent factors can produce clustered indicators and spurious GGM edges. GGMs can be estimated for independent-case data using likelihood-based or Bayesian methods, with regularization enabling sparse networks.

  • Factor models.: A factor in a factor model causes its indicators to cluster in the equivalent GGM, with all corresponding nodes connected by strong edges.The factor-model expression can be inverted to obtain the equivalent GGM.
  • Factor models.: GGM estimation and exploratory factor analysis are closely connected: clustering GGM nodes can help investigate the number of factors to extract.The paper proposes estimating and drawing a GGM as a natural first step in exploratory factor analysis.
  • Factor models.: GGM edges may be spurious because latent variables can induce associations that cannot be accounted for without explicitly modeling those latent variables.The same issue can arise when estimating exploratory directed acyclic graphs.
  • Data with Independent Cases: GGMs can be estimated when cases are independent, including cross-sectional data, aggregated subject data, and n = 1 time-series data with long measurement intervals.GGMs can also be estimated for shorter-interval time-series data, as discussed in the subsequent time-series framework.
  • Estimation.: Cross-sectional and n = 1 time-series analyses assume subjects or time points share a common distribution, because subject-specific means and networks cannot be estimated from one observation per subject.The corresponding random responses are defined for subjects or time points across all items.
  • Estimation.: The full likelihood is readily obtained for cross-sectional and n = 1 time-series data, and Σ can be estimated by maximum likelihood, least-squares, or Bayesian methods.These methods apply to the variance–covariance matrix Σ.
  • Estimation.: Sparse GGMs can be obtained through iterative edge addition and removal or regularization techniques such as penalized MLE, which jointly estimates model structure.The unregularized precision matrix K is obtained by standardizing the inverse sample variance–covariance matrix.
  • Estimation.: R packages support GGM estimation through covariance-matrix inversion, automated procedures, Bayesian structure estimation, and fitting estimated GGMs to data.Examples include corpcor, qgraph, BDgraph, ggm, and lvnet.

Temporally Ordered Data of a Single Subject

For single-subject time-series data, lag-1 vector autoregression separates directed temporal effects from contemporaneous within-time-point effects. Graphical VAR represents the latter as a Gaussian graphical model, generalizing GGM analysis to temporally ordered data.

  • Temporally Ordered Data of a Single Subject: Single-subject time series repeatedly measure one individual over time, often several times per day across a relatively short period.The paper frames these data as an increasingly popular basis for intraindividual and person-based research, including clinical applications.
  • Vector Auto-regression: Lag-1 VAR models current measurements as functions of the immediately preceding measurement occasion.The model includes variables at time t and time t −1, capturing effects between consecutive measurements.
  • Temporal and Contemporaneous Networks: The VAR coefficient matrix B encodes directed temporal predictive effects, while inverting residual covariance Θ yields a contemporaneous GGM after controlling for temporal effects.These structures form the temporal and contemporaneous networks, respectively.
  • Contemporaneous Networks: Graphical VAR models contemporaneous effects explicitly as a GGM alongside temporal effects in a VAR framework.This framework is also expressible through a conditional Gaussian distribution.
  • Temporal and Contemporaneous Information: When consecutive cases are independent and B = O, GVAR is equivalent to the independent-case GGM and therefore generalizes GGM analysis to temporally ordered data.Unlike regular VAR, GVAR models the contemporaneous structure as a GGM rather than leaving it saturated.

Temporally Ordered Data of Multiple Subjects

For time-series data from multiple subjects, GVAR modeling separates temporal, contemporaneous, and between-subjects networks while accommodating subject-specific and population-level structures. Several estimation strategies are available, but method choice depends on research goals and involves important complexity and assumptions.

  • Multilevel modeling: Individual networks encode subject-specific temporal and contemporaneous parameters, while multilevel models estimate expected population networks and random-effect deviations.Fixed effects describe average intraindividual effects, whereas random effects describe individual differences.
  • Network structures: GVAR models with multiple subjects separate variance into temporal, contemporaneous, and between-subjects networks.The between-subjects network represents relationships between subjects’ stationary means.
  • Multilevel estimation: Multilevel estimation reduces the need to estimate separate VAR models for every subject and provides shrinkage-based subject-specific estimates.Shrinkage pulls individual estimates together after estimating fixed effects and random-effect variance–covariance structures.
  • Two-step multilevel VAR: Two-step multilevel VAR estimates contemporaneous networks from residual regressions, extending an approach that otherwise does not estimate contemporaneous or between-subjects networks.The procedure sequentially models residuals within measurement occasions and symmetrizes the resulting coefficients.
  • Choosing the estimation method: Estimator choice is nontrivial because multilevel methods become complicated in high-dimensional settings and normally distributed parameters constrain how subjects may differ.The assumption allows differences in parameterization but not in network structure.

Cross-Sectional Data Analysis

Cross-sectional GGM analyses of one-time measurements cannot separate between-subjects relationships from short-term within-subjects relationships. They therefore identify a blended network unless one source of variation is assumed absent.

  • Within- and between-subjects variation: Cross-sectional data comprise multiple subjects measured only once, so the resulting analysis is commonly treated as a between-subjects analysis.The paper cautions that this label may be unwarranted because within-subject variation around individual averages cannot be distinguished.
  • Within- and between-subjects variation: In the model, the GGM from Ω represents a between-subjects network, whereas inverting and standardizing Σ_p yields a within-subjects network.These networks are defined from separate covariance structures for stationary means and within-subject variation.
  • The value of a cross-sectional analysis: With one response per subject, cross-sectional analysis cannot estimate subject-specific covariance matrices or disentangle between-subjects from short-term within-subjects relationships.The observed co-variation is an unidentified blend of these sources, preventing clear interpretation of relationships such as fatigue and concentration.
  • The value of a cross-sectional analysis: The cross-sectional GGM corresponds exactly to one component network only when short-term within-subject variation or between-subjects variation is assumed absent.Formally, this requires Σ∗ = O or Ω = O.

Within- and between-subjects effects

The section distinguishes within-subjects effects from between-subjects effects, modeling the latter in a separate GGM rather than interpreting cross-sectional networks as within-subject structures. It further explains how short-term interventions map to within-subject VAR effects, whereas sustained mean-level changes are represented by between-subjects networks.

  • Conceptual distinction: Between-subjects effects are conceptualized separately from within-subjects effects, and cross-sectional networks may reflect between-subjects structure when state-like variance does not dominate.This contrasts with interpreting cross-sectional networks as reflective of within-subject effects.
  • Simpson’s paradox: Within- and between-subjects networks can differ substantially, producing seemingly paradoxical relationships between personal deviations and subject means.Figure 3 contrasts within-subject predictions among personal deviations with between-subjects relations among different subjects’ means.
  • Interventionist accounts of causation: A short-term shock that sets X to x and evaluates Y shortly afterward corresponds to within-subjects VAR effects.This intervention represents an immediate manipulation at a particular time point.
  • Interventions at the mean level: A sustained manipulation that shifts a system’s expected level of X is not represented by stationary within-subjects VAR models but can appear in the between-subjects network.Such effects concern changes in intercepts and long-term convergence on X = x.

Empirical Examples · Reanalysis of Bringmann et al. (2013)

The empirical examples apply multilevel VAR and GGM analyses to intensive longitudinal datasets, showing interpretable temporal, contemporaneous, and between-subjects networks. A reanalysis of Bringmann et al. (2013) finds mostly aligned network methods, expected latent-variable patterns, and notable sign reversals.

  • Empirical Examples: The first example reanalyzed two independent ESM samples measuring personality domains and exercise with multilevel VAR methods.Sample 1 participants answered three times daily, and Sample 2 participants answered five times daily, with at least two hours between measurements.
  • Empirical Examples: A five-variable analysis focused on outgoing, energetic, adventurous, and happy feelings plus exercise, using the two-step multilevel VAR procedure.The model was estimated with the mlVAR package.
  • Empirical Examples: The estimated temporal, contemporaneous, and between-subjects networks retained only significant edges at α = 0.05.For contemporaneous and between-subjects networks, edges followed the “or” rule when either underlying regression was significant.
  • Empirical Examples: In the 17-item analysis, indicators of the three traits clustered across all three networks, while exercise remained connected to energetic and adventurous.Exercising was additionally connected to feeling angry in the between-subjects network.
  • Reanalysis of Bringmann et al. (2013): The Bringmann reanalysis used ESM data from 129 participants measured during baseline and posttreatment periods, with 60 measurements per period.The posttreatment period included mindfulness-treatment and control groups.
  • Reanalysis of Bringmann et al. (2013): The three resulting network methods were mostly aligned, and the temporal networks were very similar to those reported by Bringmann et al. (2013).The figure displays the resulting temporal, contemporaneous, and between-subjects network structures.
  • Reanalysis of Bringmann et al. (2013): The temporal and contemporaneous networks generally matched expectations from a unidimensional autocorrelated latent-variable model, except for a positive temporal fearful-to-pleasant edge.The two-step multilevel between-subjects network also contained a selective sad-to-relaxed edge based on β = 0.202, p = 0.046 and β = 0.151, p = 0.036.
  • Reanalysis of Bringmann et al. (2013): The sad-to-relaxed edge was notable because the estimated between-subjects correlation was strongly negative (−0.53), while the Bayesian network showed a similar positive cheerful-to-worry edge.These sign reversals were noteworthy under a unidimensional factor model.

Discussion · Limitations and Challenges · Conclusion

The paper presents GGMs as exploratory tools for modeling predictive relationships across cross-sectional and time-series psychological data, while emphasizing limitations in estimation, assumptions, and causal interpretation. It concludes that GGMs provide uniquely identified, well-parameterized hypothesis-generating models for psychological dynamics.

  • Discussion: GGMs represent partial correlations as undirected networks that can show which variables predict one another without requiring causal interpretation.
  • Discussion: GGMs can be estimated from repeated observations, while LASSO regularization performs especially well; graphical VAR extends them to temporally ordered data.
  • Limitations and Challenges: Multilevel VAR estimation is not feasible for larger datasets, and models with more than 20 variables run very slowly.
  • Limitations and Challenges: Pooling within-subject centered data and estimating fixed-effects models with LASSO can scale analysis and perform on par with multilevel estimation at higher sample sizes.
  • Limitations and Challenges: VAR analyses rely on assumptions that can be problematic, including conditional-distribution assumptions for time-series data and multivariate normality.
  • Limitations and Challenges: Non-normal data cannot be represented properly using only means and covariance matrices, so the GGM may fail to characterize the joint likelihood function.
  • Limitations and Challenges: Causal interpretations of edges remain exploratory and generate hypotheses rather than confirming causal relations, although GGMs can reveal predictive relationships without causal assumptions.
  • Conclusion: The GGM maps unique variance in cross-sectional, contemporaneous, and between-subjects time-series data, offering uniquely identified models that support research on psychological dynamics.

Appendix A. Glossary of terms … Multi-level modeling

The appendices define core network terminology and notation, then describe two-step multilevel VAR estimation for between-subject, temporal, and contemporaneous networks. Multilevel modeling pools subject-specific information but sequential univariate estimation omits some covariance relationships.

  • Appendix A. Glossary of terms: A Gaussian graphical model is an undirected network whose nodes are observed variables connected when they remain dependent after conditioning on all others.
  • Supplementary 1: Notation: Roman letters denote observed variables, Greek letters denote parameters or latent variables, and uppercase and lowercase nonboldface letters distinguish random variables from realizations.
  • Supplementary 1: Notation: The notation uses t and T for measurement occasions, p and P for subjects, bold lowercase letters for column vectors, and bold uppercase letters for matrices.
  • Supplementary 1: Notation: Vectors such as y[t,p] represent all item responses for subject p at time t, while omitted brackets indicate a single observation or subject.
  • Supplementary 2: Two-step multi-level VAR: Two-step multilevel VAR estimates the GVAR model by adding between-subject effects and estimating the contemporaneous network from residuals of the temporal model.
  • Supplementary 2: Two-step multi-level VAR: The method targets the between-subjects network K(Ω) = Ω−1 = Var(µP)−1 and distributions of temporal networks Bp and contemporaneous networks K(Θ).
  • Multi-level modeling: Multilevel modeling assigns distributions over subject-specific parameters, combining observations in a single model instead of estimating a separate VAR model for every subject.
  • Multi-level modeling: Sequential univariate estimation simplifies integration but fails to retain correlations between means, contemporaneous covariances, and correlations between temporal parameters.

Extending multi-level VAR: two-step multi-level VAR

The proposed two-step multilevel VAR extension estimates temporal and between-subjects networks first, then estimates contemporaneous networks from the first model’s residuals. Multilevel regressions also support fixed and random effects, symmetry adjustments, and optional edge thresholding for sparse networks.

  • Two-step estimation: The two-step method extends Bringmann et al.’s procedure to estimate temporal, between-subjects, and contemporaneous networks.Step 1 adds between-subject effects and yields temporal and between-subjects networks; Step 2 models residuals to obtain contemporaneous networks.
  • Step 1: Temporal and between-subjects networks: Including subject sample means as Level 2 predictors turns the models into regressions that estimate the between-subjects network.The dependent variable’s own mean is excluded, and the resulting regressions estimate a GGM between subject means.
  • Network construction: The resulting network matrices are symmetrized by averaging corresponding upper- and lower-triangular regression parameters.This produces each partial-correlation edge from standardized estimates in both directional regressions.
  • Step 2: Contemporaneous networks: Contemporaneous networks are estimated by regressing each variable’s residual on other variables’ residuals at the same time point.These residuals come from the multilevel model estimating temporal and between-subject effects.
  • Thresholding: Thresholding removes edges not significantly different from zero, producing sparser and more interpretable networks.For fixed effects, multilevel software provides standard errors and p-values for this procedure.

Supplementary 3: Simulation Studies

The simulations assessed mlVAR and graphicalVAR for estimating multilevel network structures in ESM data across varied network sizes, sample sizes, observation counts, and individual-network structures. Both methods generally performed well, although mlVAR was poorer when individual network structures were rewired.

  • Simulation design: Simulations varied node count, sample size, observations per person, fixed network structure, and whether individual networks were rewired.Nodes numbered 8 or 16; sample sizes and observations per person were each 50, 100, or 200.
  • Estimation and evaluation: The study compared mlVAR and graphicalVAR estimates with true fixed and subject-specific temporal, contemporaneous, and between-subjects networks.Performance was assessed using correlations between true and estimated networks, alongside sensitivity, specificity, bias, and mean squared error.
  • Results: With no edge rewiring, both methods generally showed good performance across network-recovery measures.Fixed temporal and contemporaneous effects had high correlations, sensitivity and specificity were high, and bias and mean squared error were generally low.
  • Results: When edges were rewired, both methods still showed good performance, but mlVAR estimation performed poorer than when structures were stable.The simulations used rewiring to create substantially different individual network structures and reduce information-sharing benefits across subjects.
  • Results: For stable structures, graphicalVAR performed identically, whereas mlVAR’s two-step procedure performed comparably or better for estimating between-subject network structures.The comparison reflects graphicalVAR’s subject-specific estimation without borrowing information across subjects, unlike mlVAR.

Supplementary 4: Stationary distribution · Supplementary Appendix A. Simulation results (no rewiring)

The stationary-distribution supplement derives the GVAR variance-covariance structure under stationarity and uncorrelated residuals, expressing Σ through B and Θ. The simulation appendix organizes no-rewiring evaluations by network size, estimator, and time point.

  • Supplementary 4: Stationary distribution: The GVAR model specifies an expression for the variance-covariance matrix of y_t.Matrix-indexing subscripts are omitted for notational clarity.
  • Supplementary 4: Stationary distribution: The derivation assumes stationarity and that residuals ε_T are uncorrelated with y_T−1.It then uses vectorization and the Kronecker product.
  • Supplementary 4: Stationary distribution: The resulting expression gives the elements of Σ in terms of B and Θ.
  • Supplementary Appendix A. Simulation results (no rewiring): The no-rewiring simulations include 8-node networks with identical network structures across subjects using two-step multi-level estimation.
  • Supplementary Appendix A. Simulation results (no rewiring): The appendix also evaluates 8-node no-rewiring networks with pooled and individual LASSO estimation.
  • Supplementary Appendix A. Simulation results (no rewiring): For 16-node no-rewiring networks with identical structures across subjects, the appendix reports two-step multi-level and pooled or individual LASSO estimation setups.

Supplementary Appendix B. Simulation results (50% rewiring

The simulation results are organized across three time settings: 50, 100, and 200.

  • The simulations report results at time 50.
  • The simulations also report results at time 100.
  • A third reported time setting is 200.
Loading 1609.04156v6…