Source-linked AI summary
How well do Network Models predict Observations? On the Importance of Predictability in Network Models
Jonas Haslbeck, Lourens J Waldorp
TL;DR
Network research has emphasized structure while giving limited attention to how well network nodes predict observations, despite predictability informing edge relevance and intervention prospects. The paper introduces nodewise predictability, supplies methods and reproducible software for cross-sectional and time-series networks, and illustrates its use in interpreting node-level prediction and interventions. It concludes that predictability complements network structure but depends on the network model being appropriate and, for cross-sectional data, does not establish causal direction.
Problem
Network models have received limited attention regarding how well nodes predict observations, although this matters for interpreting edge relevance and potential interventions.
Method
The paper introduces interpretable nodewise predictability measures and reproducible mgm and qgraph workflows for cross-sectional and time-series network models.
Results
Nodewise predictability quantifies how well a node is predicted by its neighbors, with example values of 0.55 for lonely and 0.13 for unfriendly.
Takeaways & Limitations
Predictability complements network structure by helping assess the practical relevance of edges, interpret intervention outcomes, and indicate how self-determined parts of a network are.
Takeaways & Limitations
Interpreting predictability as influence from neighbors requires an appropriate model, and cross-sectional networks generally leave edge direction unknown, making predictability an upper bound.
Abstract
from arXiv · showhide
Network models are an increasingly popular way to abstract complex psychological phenomena. While the study of the structure of network models has led to many important insights, little attention is paid to how well they predict observations. This is despite the fact that predictability is crucial for judging the practical relevance of edges: for instance in clinical practice, predictability of a symptom indicates whether a an intervention on that symptom through the symptom network is promising. We close this methodological gap by introducing nodewise predictability, which quantifies how well a given node can be predicted by all other nodes it is connected to in the network. In addition, we provide fully reproducible code examples of how to compute and visualize nodewise predictability both for cross-sectional and time-series data.
1 Introduction
Network models represent complex psychological phenomena through nodes and edges, but research has focused more on network structure than on how well nodes are predicted. This paper introduces nodewise predictability to quantify prediction from neighboring nodes and support interpretation of edges and interventions.
- 1 Introduction: Network models represent variables as nodes and their pairwise interactions as edges, making complex psychological phenomena easier to visualize and analyze.They support analyses of global structure through measures such as density and centrality.
- 1 Introduction: Psychological network research has increasingly examined clinical, health, personality, and experience-sampling data, with emphasis on global structure and node connectedness.Network models have also been used to offer alternative perspectives on psychological phenomena such as comorbidity.
- 1 Introduction: The paper adds nodewise predictability: how well a node can be predicted by all neighboring nodes in the network.Figure 1 illustrates prediction of node A from neighbors C and E.
- 1 Introduction: Edge interpretation should account for how much variance in a node is explained, because small edge weights can be practically meaningless in large samples.The paper contrasts explaining 0.5% versus 50% of a node’s variance.
- 1 Introduction: Nodewise predictability can estimate how much a node may be influenced through interventions on connected nodes.Across nodes, it also indicates whether a network is largely self-determined or influenced by factors outside the network.
- 1 Introduction: The paper addresses the lack of an easy-to-use tool by providing interpretable predictability measures and reproducible R-package workflows for cross-sectional and time-series networks.The code uses mgm and qgraph and reproduces figures from applications based on two published studies.
2 Methods
The paper estimates nodewise predictability by predicting each node from its network neighbors, then comparing predictions with observed values using distribution-appropriate measures. It applies this framework to cross-sectional and time-series networks and relates predictability to network parameters and centrality.
- Predictability workflow: Nodewise predictability is computed by estimating a network, predicting each node from its neighbors, and quantifying agreement between predictions and observations.Higher agreement indicates higher predictability.
- Network models: Cross-sectional data are modeled with pairwise Mixed Graphical Models estimated through ℓ1-regularized neighborhood regression.The regularization sets spurious edge parameters to zero, and its strength is selected by 10-fold cross-validation.
- Network models: Time-series data are modeled with mixed Vector Autoregressive models, where variables at earlier time points predict variables at the current time.The analyses use the mixed VAR implementation in the R-package mgm.
- Continuous variables: For continuous nodes, predictability is measured as the proportion of explained variance, with 0 indicating no prediction and 1 perfect prediction.Predictions are generated from the conditional mean based on neighboring nodes, after centering variables to remove intercept effects.
- Categorical variables: For categorical nodes, normalized accuracy removes the accuracy obtainable from the node’s marginal distribution alone.This prevents imbalanced categories from producing apparently high predictability through trivial majority-class predictions.
- Predictability and model parameters: Predictability is zero for disconnected nodes and is not adequately represented by centrality measures, particularly in Mixed Graphical Models.Centrality strongly correlates with predictability in Gaussian Graphical Models but much less in models containing categorical variables.
3 Predictability in Cross-Sectional Networks
Cross-sectional nodewise predictability is computed after fitting a mixed graphical model, then visualized alongside network edges using variance-explained and accuracy measures. The examples show that predictability adds information beyond edge structure and can be evaluated either within sample or out of sample.
- Reproducibility and evaluation: The example code is reproducible from downloaded preprocessed data and produces a network similar to the original, with ordinal variables treated as continuous rather than dichotomized.The analysis uses the same data for network estimation and predictability calculation, yielding within-sample prediction error.
- Estimation and prediction: The workflow fits a mixed graphical model, predicts each node from its connected neighbors, and quantifies closeness between predictions and observed values.For categorical variables, the procedure computes correct classification, normalized accuracy, and intercept-model accuracy.
- Visualization: The visualization uses qgraph with a weighted adjacency matrix, nodewise predictability measures, edge colors, and a force-directed layout.The layout improves readability by seeking similar edge lengths and few crossings, but node distance has no analytic relation to the network.
- Visualization: For continuous nodes, the blue pie segment represents the proportion of explained variance, while the binary node uses accuracy and normalized accuracy segments.For the binary variable, full-model accuracy combines intercept-model accuracy with the additional accuracy from the remaining variables.
- Interpretation: Nodes with more or stronger edges can be more predictable, but network structure alone does not reveal the absolute predictability level.In the example, lonely has predictability 0.55 and unfriendly has 0.13, despite their relative ordering being visually apparent.
4 Predictability in Temporal Networks
Temporal nodewise predictability is computed in a lagged VAR network and displayed as blue rings around mood-variable nodes. The example uses these patterns to identify mutually engaging mood groups and implications for possible interventions.
- Temporal prediction: In a VAR network, each node is predicted from all nodes at the previous time point, including the predicted node itself.The example estimates a lag-1 VAR model and computes predictability for each node.
- Visualization: The temporal visualization displays network edges, self-loops, and one blue predictability measure for each node.Predictability values are supplied as a vector to the qgraph pie argument.
- Network pattern: The mood network contains a positive self-engaging group of Cheerful, Enthusiastic, and Satisfied, and a negative self-engaging group of Irritated, Agitated, Restless, and Suspicious.Worrying is influenced by both groups, whereas Relaxed is relatively disconnected and has a weak negative influence on the negative group.
- Intervention implications: The observed network pattern supports judging intervention targets: changing one mood group is expected to affect Worrying, but intervening on Worrying is not expected to reverse those effects.Relaxed itself is not influenced by the modeled variables, so additional variables would be needed to target it.
5 Discussion
Nodewise predictability complements network structure by quantifying how strongly neighboring nodes determine a node and supporting judgments about edge relevance and intervention outcomes. Its interpretation depends on model appropriateness, causal direction, and the distinction between predicting from all neighbors versus from one specific node.
- Discussion: Nodewise predictability measures how well a node can be predicted by all its neighboring nodes, complementing network structure when judging edge relevance.For an appropriate network model, it provides an interpretable measure of influence on a node.
- Discussion: This measure can support predictions of intervention outcomes on an interpretable scale, helping researchers select treatments in clinical practice.The paper frames this as especially relevant when judging the practical significance of edges connected to a symptom or other node.
- Discussion: The analyses can extend to time-varying networks, enabling study of changes in edge parameters and nodewise predictability over time.A second model may incorporate variables inside and outside the changing network to investigate transitions between system states.
- Limitations: Predictability can be interpreted as influence from neighboring nodes only when the network model appropriately represents the phenomena under study.Unobserved common causes, logical dependencies, or multiple indicators of one construct can produce edges that do not represent mutual causal influence.
- Limitations: For cross-sectional models, predictability is an upper bound because edge direction is generally unknown and causal effects may point away from the target node or be bidirectional.This directional limitation does not apply to VAR models with lagged predictors, where time determines causal direction.
- Limitations: The paper does not address how well one node predicts another, a problem lacking straightforward solutions with correlated predictors.Suggested future directions include explained-variance decomposition and variable-importance methods extended to network models.