Selection and Estimation for Mixed Graphical Models

1 Introduction

• The task is to reconstruct conditional-dependence edges from observations, including the high-dimensional setting where the number of variables exceeds the sample size.The graph may have p(p −1)/2 candidate edges to estimate from n observations.Source: §1 · p. 2
• When node types are known, parametric methods can be more efficient than flexible semiparametric or nonparametric approaches, motivating a parametric mixed model.The introduction notes that nonparametric methods are often less efficient than parametric counterparts in this setting.Source: §1 · p. 2
• The proposed parametric mixed graphical model allows nodes of different types, with each conditional distribution drawn from the exponential family and node types assumed known.The paper contrasts this setting with earlier approaches in which nodes share one parametric form.Source: §1 · p. 2
• The paper extends beyond mixed conditional Gaussian models and settings limited to two node types, allowing graphs with more than two types and giving specific compatibility requirements for common distributions.The cited parallel work treats exponential-family conditionals but limits the graph to two node types.Source: §1 · pp. 2–3

2.1 Conditionally-Specified Models for Mixed Data

• Conditionals are specified node by node because calculating the joint model's log-partition function is often intractable, but a conditional specification need not define a valid joint distribution.The model uses node potentials and symmetric pairwise edge potentials, with nonedges assigned zero parameters.Source: §2.1 · p. 3
• The model admits Gaussian, Bernoulli, Poisson, and exponential node conditionals within the exponential family, while the mixed class is not closed under marginalization.Integrating Bernoulli nodes from a graph containing Gaussian and Bernoulli nodes can yield a Gaussian mixture outside the exponential family.Source: §2.1 · pp. 3–4

2.2 Compatibility of Conditionally-Specified Models

• Compatibility means a shared generating function exists for the conditionals; strong compatibility additionally requires that function to be a density.For the specified exponential-family conditionals, symmetric edge parameters yield compatibility, and integrability yields strong compatibility.Source: §2.2 · pp. 4–5
• For the four common conditional families, compatibility and strong compatibility impose parameter restrictions, including prohibiting Gaussian–exponential and Gaussian–Poisson edges.For Poisson–Poisson edges, compatibility imposes no restriction, whereas strong compatibility requires negative edge potentials.Source: §2.2 · p. 5, Table 1 · p. 6
• The proposed estimator and theory also apply to other conditional densities of the specified form, and graph structure can be estimated even when compatibility requirements fail.The necessary-and-sufficient compatibility and strong-compatibility conditions for the four listed families are summarized in Table 1.Source: §2.2 · pp. 5–6, Table 1 · p. 6

3.1 Estimation

• Nodewise penalized conditional likelihood estimates each node’s neighbourhood from the nonzero coefficients in its fitted regression.Source: §3.1 · p. 6
• To accommodate variables on different scales, the ℓ1 penalty can be weighted using the empirical standard errors of the corresponding variables.Source: §3.1 · p. 6
• Although true edge parameters are symmetric, nodewise estimates can be asymmetric, and this asymmetry can be exploited for more efficient cross-type edge estimates.Source: §3.1 · p. 7

3.2 Tuning

• BIC selects the tuning parameter, with a separate value chosen for each node type by minimizing BIC summed over nodes of that type.The criterion combines nodewise log-likelihood with a penalty proportional to the number of nonzero coefficients.Source: §3.2 · p. 7

4.1 Neighbourhood Recovery

• Under strong compatibility and assumptions 1–7, neighbourhood selection exactly recovers a node’s true neighbourhood with probability approaching one exponentially in n.The guarantee is meaningful when the graph is sparse, with maximum degree d = o(n).Source: §4.1 · pp. 7, 9, 10
• As degree d increases, minimum edge-strength and sample-size requirements rise while the feasible tuning range shrinks, so the theorem requires d = o(n).Source: §4.1 · p. 9
• For Gaussian nodes, κ2 = 1 and κ3 = 0 relax derivative conditions and yield exact neighbourhood recovery under assumptions 1–5.Source: §4.1 · p. 10

4.2 Combining Neighbourhoods to Estimate the Edge Set

• For Gaussian–non-Gaussian edges, use the Gaussian-side neighbourhood estimate: it needs less sample size, allows broader tuning ranges, and generally has higher recovery probability.Source: §4.2 · p. 11
• When the comparison conditions are not met, there is no clear preference for which neighbourhood to use; same-type endpoints likewise offer no clear reason to favor one estimate.Source: §4.2 · pp. 10, 12
• For different-type non-Gaussian edges, choose the endpoint regression with the highest estimated correct-recovery probability, using consistent estimates of unknown bounds.The rules for choosing a neighbourhood are summarized in Table 2.Source: §4.2 · pp. 11–12, Table 2 · p. 12

• If a node’s conditional density has form (3) and Assumptions 1–3 and 5–8 hold, the estimator exactly recovers its true neighbourhood with the stated probability bound.Source: §5 · p. 13
• Recovery of an edge between two nodes requires form (3) for only one endpoint, allowing relationships beyond those covered by Table 1, such as a Poisson node linked to a real-valued node.Source: §5 · p. 13
• The guarantee covers an edge only when at least one endpoint’s conditional density is exactly form (3); extending it to approximate conditionals is left for future work.Source: §5 · p. 13

6.1 Data Generation

• The simulations generate sparse two-type graphs with same-type and cross-type edges, then sample from node conditionals using a Gibbs sampler.Gaussian-Bernoulli and Poisson-Bernoulli models use m = p/2 nodes of each type; samples are taken 500 iterations apart after a 3000-iteration burn-in.Source: §6.1 · p. 13, Figure 1 · p. 14

6.2 Probability of Successful Neighbourhood Recovery

• Recovery probability rises to 1 beyond the required scaled-sample-size threshold, and Gaussian-node regressions need fewer samples than Bernoulli-node regressions for a given recovery probability.The plots compare recovery probability against n/{3 log(p)} and separate estimates by node type regressed.Source: §6.2 · p. 14, Figure 2 · p. 15

6.3 Comparison to Competing Approaches

• The Gaussian-Bernoulli comparison evaluates seven approaches, including mixed-model methods and methods that ignore mixed node types.The computationally intensive Lee and Hastie method is run on 250 data sets from 50 graphs rather than 2000 data sets from 100 graphs.Source: §6.3 · p. 15
• The proposal generally outperforms the alternatives on the Gaussian-Bernoulli graph, with BIC-selected tuning yielding 52% precision and 95% recall.Gaussian graphical-model neighbourhood selection is closest to the proposal; Ising selection suffers from dichotomizing Gaussian variables, and graphical lasso performs poorly under its Gaussian assumption.Source: §6.3 · p. 16, Figure 3 · p. 16

6.4 Application of Selection Rules for Mixed Graphical Models

• Section 4.2’s selection rules outperform union and intersection rules for Poisson-Bernoulli edges; the graphical random forest leads only when few edges are estimated.Using estimated parameters produces a curve almost identical to using true parameters.Source: §6.4 · p. 17, Figure 4 · p. 17

7 Discussion

• Compatibility and strong-compatibility requirements impose stringent restrictions, especially when modeling unbounded variables.The authors suggest truncation may circumvent some of these restrictions.Source: §7 · p. 17
• The pairwise interaction x_s x_t is a second-order approximation of the true edge potential.The authors describe semi-parametric penalized regressions with nonlinear edge potentials as a way to relax this assumption.Source: §7 · p. 18
• The paper provides R code for replicating its numerical results.The code is available at the stated GitHub repository.Source: §7 · p. 18

A.1 A Proof for Proposition 1

• Conditional densities of the specified form are compatible when edge parameters are symmetric, provided the relevant function is integrable with respect to each node variable.The proof relates this integrability to that of the conditional densities.Source: §A.1 · p. 21
• The proof constructs a log-density ratio and decomposes it into interactions of different orders to recover the joint-density form.It sets variables to zero to isolate node and pairwise interactions.Source: §A.1 · p. 21
• Setting all but the relevant variables to zero shows that third- and higher-order interactions vanish, yielding the pairwise form.The second-order interaction is simplified using the same isolation argument.Source: §A.1 · pp. 21–22

A.2 A Proof for Lemma 1

• For an exponential node, its rate must remain negative: interactions with unbounded Poisson or exponential nodes must be non-positive, interactions with Gaussian nodes must be zero, and Bernoulli interactions are bounded by the baseline rate.The Bernoulli interaction magnitudes must sum to less than −α1s.Source: §A.2 · p. 22
• Gaussian nodes require a negative-definite ΘJJ, while Bernoulli and Poisson conditionals impose no restrictions on their conditional parameters.The proof also states that Bernoulli nodes require no restrictions on edge potentials for a density.Source: §A.2 · pp. 22–23
• The restrictions in Table 1 are sufficient for strong compatibility: Gaussian nodes are isolated from Poisson and exponential nodes, and the remaining conditional densities are integrable under the stated restrictions.The proof uses the negative definiteness of ΘJJ for Gaussian integrability and bounds the joint density by an integrable no-interaction model for Poisson and exponential nodes.Source: §A.2 · pp. 22–23

A.3 A Proof for Theorem 1

• The proof follows the primal-dual witness method, testing the sub-gradient condition of an oracle estimator to establish properties of the ℓ1-penalized estimator.It assumes familiarity with the method and cites Ravikumar et al. (2011) and Yang et al. (2012).Source: §A.3 · pp. 23–24
• The oracle construction sets coefficients for non-neighbours to zero, solves the restricted optimization, and then estimates the remaining dual variables from the sub-gradient condition.The proof verifies that the resulting pair satisfies the sub-gradient condition.Source: §A.3 · p. 24
• The constructed primal-dual pair recovers the true neighbourhood exactly when the required events hold; those events occur with high probability for large sample size.The proof also establishes sign consistency for all edges under the stated minimum-signal condition.Source: §A.3 · pp. 24–26

A.4 A Proof for Lemma 3

• The concentration argument uses the exponential-family moment-generating function and a second-order Taylor expansion.The proof applies these steps to independent samples.Source: §A.4 · p. 26
• A Chernoff bound and the Bonferroni inequality are used to bound the probability of the required events.The argument sets δ = aλn/(8 −4a).Source: §A.4 · p. 27
• The argument requires λn to be large enough relative to √(log(2p)/n) and no larger than a bound determined by κ2, δ2, M, and a.These bounds ensure the parameter v used in the Chernoff argument is at most M/2.Source: §A.4 · p. 27

A.5 A Proof for Lemma 4

• For Lemma 4, convexity of F(u) and a Taylor expansion yield an oracle-estimator error bound of 10λn√(d + 1)/Λ1 and a bound on the residual term.The proof establishes the residual bound under its stated conditions.Source: §A.5 · pp. 27–30

A.6 A Proof for Corollary 1

• For the Gaussian-node corollary, the proof obtains a radius B = 5(d + 1)1/2λn/Λ1 and then uses Lemma 5 to verify the theorem’s conditions.It subsequently drops the requirement of ξ1 in Lemma 5’s condition.Source: §A.6 · pp. 30–31

A.7 Additional Details of Data-Generation Procedure

• The data-generation procedure adjusts the Gaussian-node interaction matrix to be negative definite with eigenvalues no larger than −0.1, then standardizes its diagonal to −1; Poisson–Poisson edge potentials are made negative.Source: §A.7 · p. 31