Moderated Network Models

1 Introduction

• Pairwise network models analyze multivariate dependencies, but their standard assumption excludes interactions that vary with other variables.Such moderation may be plausible in psychology, where omitted moderators can make an average model inappropriate, hide an effect, or help explain contradictory study findings.Source: §1 · pp. 1–2
• Split-sample approaches such as NCT and FGL approximate moderation with a step function and reduce sensitivity by estimating networks on half-sized datasets.The step-function approximation can be implausible when relationships change gradually with a moderator.Source: §1 · pp. 2–3
• The proposed Moderated Network Model extends pairwise models with specified moderators, fitting linear moderation effects while using the full dataset.The paper takes the first step by extending the GGM.Source: §1 · p. 3
• The paper proposes factor-graph visualization and evaluates MNM estimation against NCT and FGL for recovering moderation effects.The introduction identifies these as contributions and describes a simulation comparison, without reporting its outcome here.Source: §1 · p. 3
• The paper also provides a reproducible tutorial for fitting and visualizing MNMs with the R-package mgm and discusses model misspecification.The introduction places the tutorial and misspecification discussion in Section 4.Source: §1 · p. 3

2.1.1 Moderation and Interactions in Linear Regression

• In linear regression, moderation means the effect of predictor B on response A varies linearly with a third variable C, through the interaction parameter ω_BC.When β_B is zero but |ω_BC| is nonzero, a regression without the interaction term would miss the effect.Source: §2.1.1 · pp. 4–5

2.1.2 Regression vs. Network Semantics

• In the network perspective, pairwise interactions correspond to products of two variables in the joint distribution, while moderation effects correspond to three-way interactions.In the conditional regression perspective, the pairwise term is called a main effect.Source: §2.1.2 · p. 6, §2.3 · p. 9

2.2 Gaussian Distribution and Gaussian Graphical Model

• The Gaussian graphical model encodes conditional dependence through the inverse covariance matrix: nonzero entries indicate edges, and zero entries indicate conditional independence.The multivariate Gaussian provides the basis for the paper’s continuous-variable MNM construction.Source: §2.2 · p. 6

2.3 Construction of Moderated Network Model

• The paper constructs an MNM by adding three-way interactions to a multivariate Gaussian, allowing pairwise interactions to be linearly moderated by other variables.All pairwise interactions are included, but a nonzero three-way interaction does not require its corresponding pairwise interaction to be nonzero.Source: §2.3 · pp. 7, 9
• The MNM joint distribution factorizes into p conditional Gaussian distributions, so it can be estimated through p regressions with the appropriate moderation terms.Each conditional mean has the form of a moderated linear regression.Source: §2.3 · pp. 8–9
• The unconstrained nodewise algorithm does not guarantee a normalizable MNM joint distribution, preventing global goodness-of-fit analyses and model selection by that route.The authors note that out-of-sample prediction error can still support model selection.Source: §2.3 · pp. 9–10

2.4 Visualizing Moderated Network Models

• Factor graphs represent each pairwise or higher-order interaction as a factor node connected to its participating variables, including three-way moderation effects.The illustrated three-way interaction can be read as three moderation effects, depending on which pairwise interaction is being considered.Source: §2.4 · pp. 10–11, Figure 2 · p. 11

2.5.1 Estimate Nodewise Regressions

• MNM estimation uses ℓ1-regularized nodewise regressions, with tuning parameters selectable by cross-validation or an information criterion such as EBIC.The sparsity assumption is that most true parameters are zero; ℓ1 regularization shrinks estimates and can set small estimates to zero.Source: §2.5.1 · pp. 12–13

3.1 Data generation

• The simulation sampled 100 MNMs with 12 initial variables, six edges across three interaction types, and one additional moderator; each model had at most two edges per node.All nonzero pairwise and moderation parameters were set to 0.2, and 1,808 cases were sampled per model; performance was assessed across 12 increasing sample sizes.Source: §3.1 · pp. 14, 16

3.2.1 Moderated Network Models

• MNM estimation compared a known moderator, sequential searches over candidate moderators, and a model including all moderators at once, with tuning selected by EBIC.Sequential estimates were combined by taking parameters nonzero in any candidate model, a strategy expected to increase sensitivity while potentially reducing precision.Source: §3.2.1 · pp. 16–17

3.2.2 Network Comparison Test (NCT)

• NCT split observations at the moderator’s median and used permutation tests of edge differences; non-positive-definite covariance estimates in small samples were projected to the nearest positive-definite matrix.The procedure used 1,000 permutations and an α = 0.05 significance threshold.Source: §3.2.2 · p. 17

3.2.3 The Fused Group Lasso (FGL)

• FGL jointly estimated two GGMs on median-split data, penalizing both model parameters and differences between groups; its sequential version searched each variable as a moderator and combined detected effects.The sequential split-sample methods approximate a linear moderation effect with two constant functions on either side of the median.Source: §3.2.3 · pp. 17–18

3.3 Results

• Across MNM versions, sensitivity rose with sample size; sequential MNM had the highest sensitivity but did not reach precision 1, while versions with a known moderator or all moderators specified appeared to approach 1 on both metrics.The reported metrics are sensitivity and precision for pairwise and moderation effects, shown as sample size increases.Source: Figure 4 · p. 19, §3.3 · p. 19, Figure 5 · p. 20

3.4.1 Performance of Moderated Network Models

• In this simulation, versions with the correct moderator and all moderators specified appeared consistent, while sequential MNM’s false positives accumulated; pairwise and moderation effects were roughly equally difficult to estimate.The authors attribute a small sensitivity advantage for partially moderated interactions to having more uncorrelated predictors; the simulation used equal effect sizes, although moderation effects may be smaller in practice.Source: §3.4.1 · pp. 21–22

3.4.2 Performance of NCT and FGL

• With the correct moderator specified, NCT’s sensitivity and precision appeared to approach 1 but remained below MNM; FGL had the highest sensitivity for partial moderation, while sequential NCT and FGL had low precision.FGL precision with the correct moderator increased through n = 193 and then decreased; the authors do not recommend FGL for settings similar to this simulation.Source: §3.4.2 · pp. 22–23

3.4.3 Moderated Network Models vs. Sample-split Methods

• In the simulated settings, MNMs were consistent estimators for moderation effects and outperformed the split-sample methods NCT and FGL.The data were generated from an MNM, while NCT and FGL approximate linear moderation with piecewise-constant functions; the authors note that the performance gap may be smaller for a Bernoulli moderator.Source: §3.4.3 · p. 23

4.1 R-Tutorial: Fit Moderated Network Model to Data Set of Mood Variables

• The tutorial fits an MNM to 3,896 observations on five mood variables, using mgm, a package that implements MGMs including GGMs and MNMs.The variables are hostile, lonely, nervous, sleepy, and depressed.Source: §4.1 · pp. 23–24

4.1.1 Fit Moderated Network Model to Data

• The fit uses all five variables as moderators, EBIC with γ = 0.5, scaled predictors, and the AND-rule to combine nodewise regressions.The tutorial notes that cross-validation is an alternative for selecting the regularization parameter.Source: §4.1.1 · pp. 24–25
• 0.4318148 is the reported lonely–depressed pairwise weight, while a 3-way coefficient is interpreted as the moderator-driven change in a pairwise interaction.The output lists nine pairwise interactions and four moderation effects; the nervous–sleepy moderation on depressed has weight 0.0564465.Source: §4.1.1 · pp. 25–26

4.1.2 Visualize Moderated Network Model as Factor Graph

• Factor graphs represent variables as circles, pairwise interactions as squares, and moderation effects as triangles; an alternative uses simple edges for pairwise interactions.The alternative retains the same information except parameter values; conditioning on moderators instead becomes unfeasible as their number grows because the required networks grow exponentially.Source: Figure 6 · p. 27, §4.1.2 · pp. 27–28

4.2 Model Misspecification

• The MNM assumes conditionally Gaussian variables and models each variable’s mean with a regression equation; its possible misspecifications include nonlinear effects and non-Gaussian conditional distributions.Source: §4.2 · p. 28, §4.2.1 · p. 28

4.2.1 Types of Model Misspecification

• The construction implies that population moderation effects should be equal across nodewise regressions, justifying aggregation; with skewed data, estimates may differ, though not invariably.The paper notes that moderation effects were roughly alike across nodewise regressions in an earlier example with skewed data.Source: §4.2.1 · pp. 28–29

4.2.2 Different Moderation Effects across Nodewise Regressions

• ≈0.19 aggregate moderation masks a negative ≈0.09 distressed-node estimate in the opposite direction; Figure 8(b) arrows indicate regression source, not effect direction.Reporting the nodewise conditional distributions avoids misleading aggregate interpretation but leaves the joint distribution unknown, preventing inference on it.Source: §4.2.2 · pp. 31–32, Figure 8 · p. 31

• The paper extends the multivariate Gaussian distribution with moderation, visualizes MNMs using factor graphs, and reports better simulation performance than FGL and NCT.It proposes ℓ1-regularized nodewise regression and provides a reproducible mgm tutorial.Source: §5 · p. 32
• Because no parameter-space constraint guarantees normalizability, a fitted joint may not define a probability distribution, precluding global likelihood-ratio and goodness-of-fit model selection.Conditional distributions remain proper, so conditional inference and prediction are available.Source: §5 · pp. 32–33
• Simulation performance may differ with parameter size or sparsity; moderation effects are often smaller than pairwise effects and therefore harder to detect.The study used equal-sized pairwise and moderation effects; smaller effects may require more data for recovery.Source: §5 · p. 33
• With correlated predictors in skewed data, the ℓ1 estimator may select only a lower- or higher-order term, especially when sample size is small and λ is large.Source: §5 · p. 33
• Allowing a moderation effect to be nonzero while its constituent pairwise effects are zero may be less meaningful; enforcing hierarchy can reduce model fit.The authors say the choice should rest on substantive grounds.Source: §5 · p. 34
• A single λ assumes shared sparsity across pairwise and moderation effects; separate penalties may suit differing sparsity but raise computational cost and instability.Source: §5 · p. 34
• The Gaussian moderation approach could extend to MGMs through nodewise GLMs with appropriate link functions, but a full treatment of interaction types and performance remains future work.Source: §5 · p. 34
• Future work could develop factor-graph centrality measures to identify contextual variables with the strongest influence on psychological or symptom networks.Source: §5 · pp. 34–35

A Mean-centering of Predictors in Moderation Analysis

• Mean-centering the moderator makes the effect of B on A at C = 0 match the unmoderated effect, enabling comparison between models and making the coefficient refer to common moderator values.Source: A Mean-centering of Predictors in… · pp. 39–40, Figure 9 · p. 39

B Joint distribution for p = 3

• For p = 3, each conditional Gaussian mean is linear in the other variables, with pairwise terms and a three-way term representing a moderated pairwise interaction.Source: B Joint distribution for p = 3 · p. 40
• The joint distribution is formed by factorizing the three conditional distributions and combining corresponding parameters by averaging them.Source: B Joint distribution for p = 3 · p. 41
• The construction generalizes to any p, but the parameter constraints that ensure the joint distribution is normalizable are unknown; rejection sampling is used to draw observations.Source: B Joint distribution for p = 3 · p. 41

• The rejection sampler uses Gibbs sampling with a 100-iteration burn-in and rejects chains when any |Xi| exceeds τ = 3.09.The threshold is the 99.9% quantile of a standard normal distribution; chains remaining within [−τ, τ] are treated as satisfying the constraints.Source: C Rejection Sampling · p. 41
• To limit runtime, the authors sampled 10,000 cases from 130 models, selected 100 models with the lowest divergence proportions, and used 1,808 observations per data set.The selected models had rejection proportions between 0.287 and 0.886.Source: C Rejection Sampling · p. 42
• Figure 10 reports slightly downward-biased estimates: βpw = 0.194, βpw(mod) = 0.186, ωmod(pw) = 0.187, and ωmod = 0.189.The estimates were close to the values specified in the Gibbs sampler, and moderation-effect bias appeared to decrease as the proportion of finite samples increased.Source: Figure 10 · p. 42, C Rejection Sampling · p. 42
• The authors note that the slight downward sampling bias also slightly biases performance across all conditions and estimators downward.Source: C Rejection Sampling · p. 43

• The figure is offered to make visible a sensitivity difference previously reported for unmoderated pairwise interactions and full moderations versus partially moderated interactions.Source: D Comparing Sensitivity of Moderated… · p. 43, Figure 11 · p. 43
• Figure 11 compares sensitivity for four parameter types under a correctly specified moderator, sequential moderator search, and simultaneous specification of all moderators.Source: Figure 11 · p. 43
• A simpler simulation isolates interaction types to rule out graph characteristics as the explanation, followed by an account based on the number of uncorrelated neighbors.Source: D Comparing Sensitivity of Moderated… · p. 43

E Simulation with Isolated Interaction Types

• The isolated-graph simulation reproduces the sensitivity difference observed in Appendix D.Figure 12(b) plots sensitivity for four parameter types against the number of observations.Source: Figure 12 · p. 44, E Simulation with Isolated Interaction… · p. 44
• The graph separates pairwise interaction 7–8, fully moderated interaction 1–2–3, and partially moderated interaction 4–5–6 as far as possible.By definition, the pairwise interaction 5–6 and moderation effect 4–5–6 cannot be separated.Source: E Simulation with Isolated Interaction… · pp. 43–44
• With centered, uncorrelated predictors, two nonzero predictors lead EBIC to select a smaller λ, which increases sensitivity relative to one nonzero predictor.This explanation concerns the contrast between the isolated interactions and the 4–5–6 interaction in their nodewise regressions.Source: E Simulation with Isolated Interaction… · p. 44

F Sensitivity as Function of Number of Uncorrelated Neighbors

• Sensitivity to detect neighbors increases with the number of uncorrelated neighbors.The GGM simulation varies the number of uncorrelated neighbors from one to four.Source: F Sensitivity as Function of Number of… · p. 45, Figure 13 · p. 45

G XY is uncorrelated with X and Y if the latter are mean-centered

• For independent, finite-variance X and Y, XY is uncorrelated with X when E[Y] = 0 and with Y when E[X] = 0.The derivation identifies mean-centering as the condition for these zero correlations.Source: G XY is uncorrelated with X and Y if… · pp. 45–46

H Additional Tutorial: Estimate MNM on Iteration 2 of Simulation Study

• The tutorial demonstrates fitting an MNM to 858 observations with mgm and presents factor-graph visualizations.The example data are generated from the model shown in Figure 3.Source: H Additional Tutorial: Estimate MNM on… · p. 46

H.1 Fit Moderated Network Model to Data

• The tutorial configures mgm for 13 continuous variables, includes all variables as moderators, selects λ by EBIC with γ = 0.5, scales predictors, and combines regressions with the AND-rule.The tutorial notes that cross-validation is an alternative for selecting λ.Source: §H.1 · pp. 46–47
• The interaction output separates pairwise interactions at the first level from moderation effects at the second, with four of each estimated in this example.The tutorial uses showInteraction() to inspect individual parameter estimates.Source: §H.1 · pp. 47–48
• For pairwise interaction 4–11, mgm reports a positive weight of 0.103782, interpreted as an approximately 0.104-unit increase in X11 per one-unit increase in X4, holding other variables constant.This is the tutorial's linear-regression-style interpretation of a pairwise parameter.Source: §H.1 · p. 48
• For moderation effect 6–7–13, mgm reports a positive weight of 0.1174669; the X6–X7 interaction is zero at X13 = 0 and increases by about 0.117 per unit of X13.Source: §H.1 · p. 48
• For partially moderated interaction 1–12, the tutorial reports positive weights 0.1269736 for the pairwise term and 0.1467114 for the X13 moderation term.It interprets the pairwise interaction as approximately 0.130 at X13 = 0, increasing by approximately 0.147 for each unit increase in X13.Source: §H.1 · p. 48

H.2 Moderated Network Model as Factor Graph

• FactorGraph() adds a node for each interaction parameter, allowing moderation effects to appear in a visualization.Figure 14(a) uses circles for variables, squares for pairwise interactions, and triangles for moderation effects.Source: §H.2 · p. 49, Figure 14 · p. 49
• Green edges encode positive parameters, red edges negative parameters, and edge width represents absolute parameter size.Source: §H.2 · p. 49, Figure 14 · p. 49
• For larger graphs with many pairwise interactions, PairwiseAsEdge = TRUE replaces pairwise-interaction factors with simple edges while retaining the same information.The resulting display is not a typical factor graph.Source: §H.2 · p. 49, Figure 14 · p. 49

I Varying Moderation Effects across Nodewise Regresssions: A closer Look

• The authors used unregularized linear regressions and conditional scatter plots to examine moderation across the three nodewise regressions.The plots condition on different values of one predictor and show the relationship between the response and the remaining predictor.Source: I Varying Moderation Effects across… · p. 50
• The interaction was positive in regressions on afraid and ashamed but negative in the regression on distressed.These signs matched the ℓ1-regularized results in Figure 8.Source: I Varying Moderation Effects across… · p. 50
• As ashamed increases, the positive afraid–distressed relationship strengthens more than the ashamed–afraid relationship strengthens as distressed increases.The stronger change reflects a larger interaction parameter in the regression on afraid than in the regression on ashamed.Source: I Varying Moderation Effects across… · p. 51, Figure 15 · p. 51
• The ashamed–distressed relationship varies nonlinearly with afraid, approximated by a small negative linear moderation effect.The nonlinear moderation is not exactly canceled out, so the best linear approximation is slightly negative.Source: I Varying Moderation Effects across… · pp. 51–52, Figure 15 · p. 51

J Correlations between lower- and higher-order terms

• For centered independent predictors, cor(X, XY) = 0; this also holds for linearly dependent Gaussian predictors, but not necessarily for linearly dependent skewed predictors.Figure 16 illustrates the four combinations of symmetric versus skewed and dependent versus independent predictors.Source: J Correlations between lower- and… · pp. 52–54, Figure 16 · p. 53
• High correlations between lower- and higher-order predictors can destabilize estimates and lead ℓ1 regularization to select only one of two predictors that are both nonzero in truth.The problem is more severe with strong regularization, which tends to be selected when the sample size is small.Source: J Correlations between lower- and… · p. 54
• Transformations can reduce predictor correlations but make relationships harder to interpret; Ridge avoids ℓ1 selection but yields fully connected networks, while significance tests raise multiple-testing concerns.A detailed investigation of solutions to this model-misspecification problem is left for future research.Source: J Correlations between lower- and… · p. 54

K Simulation Results in Tables

• Table 1 reports sensitivity and precision by sample size for recovering unmoderated pairwise interactions.Precision is missing when no edges were estimated in the 100 simulation iterations.Source: K Simulation Results in Tables · p. 54, Table 1 · p. 55
• The compared methods include settings with the correct moderator, moderators fitted in p = 13 sequential models, or all moderators in one model.The sequential-model estimates are combined.Source: K Simulation Results in Tables · p. 55
• Table 2 reports sensitivity and precision by sample size for recovering moderated pairwise interactions.Precision is missing when no edges were estimated in the 100 simulation iterations.Source: K Simulation Results in Tables · p. 55, Table 2 · p. 55
• Table 3 reports sensitivity and precision by sample size for moderation effects with a pairwise component.Precision is missing when no edges were estimated in the 100 simulation iterations.Source: K Simulation Results in Tables · p. 55, Table 3 · p. 56
• Table 4 reports sensitivity and precision by sample size for moderation effects without a pairwise component.Precision is missing when no edges were estimated in the 100 simulation iterations.Source: K Simulation Results in Tables · p. 56, Table 4 · p. 56