PaperlayerView exact revision on arXiv

Source-linked AI summary

Graphs for margins of Bayesian networks

Robin J. Evans

arXiv:1408.1809v2math.STstat.OTstat.TH

TL;DR

DAGs with latent variables can induce complex marginal models that no DAG represents, and ordinary mixed graphs do not capture their full range. The paper introduces mDAG hypergraphs and latent projection to represent these margins, with graphical results for equivalence and causal interventions.

  • Problem

    Marginal distributions from DAGs with latent variables are generally complex and not represented by any DAG, while ordinary mixed graphs fail to capture their full range.

    Source: §1 · pp. 1–2

  • Method

    The paper introduces mDAG hypergraphs with hyper bidirected edges and uses latent projection to represent DAG margins.

    Source: §1 · p. 2, §4 · p. 11

  • Results

    mDAGs represent the range of DAG margins, and under suitably defined interventions causally interpreted mDAGs correspond one-to-one with marginal models.

    Source: §1 · p. 2

  • Takeaways & Limitations

    mDAGs provide a framework for representing marginal and intervention structure, but a full characterization of marginal Markov equivalence remains open.

    Source: §8 · p. 27

    Fitting and testing under the marginal Markov property are difficult because an explicit model representation is generally unavailable, and using the nested-model surrogate can lose power by ignoring inequality constraints.

    Source: §8 · p. 27

Abstract

from arXiv · show

Directed acyclic graph (DAG) models, also called Bayesian networks, impose conditional independence constraints on a multivariate probability distribution, and are widely used in probabilistic reasoning, machine learning and causal inference. If latent variables are included in such a model, then the set of possible marginal distributions over the remaining (observed) variables is generally complex, and not represented by any DAG. Larger classes of mixed graphical models, which use multiple edge types, have been introduced to overcome this; however, these classes do not represent all the models which can arise as margins of DAGs. In this paper we show that this is because ordinary mixed graphs are fundamentally insufficiently rich to capture the variety of marginal models. We introduce a new class of hyper-graphs, called mDAGs, and a latent projection operation to obtain an mDAG from the margin of a DAG. We show that each distinct marginal of a DAG model is represented by at least one mDAG, and provide graphical results towards characterizing when two such marginal models are the same. Finally we show that mDAGs correctly capture the marginal structure of causally-interpreted DAGs under interventions on the observed variables.

1 Introduction

DAG models are widely used, but marginalizing latent variables generally produces complex models not faithfully represented by any DAG. The paper introduces mDAGs to represent these margins and their causal intervention structure, while developing partial results on when marginal models are equivalent.

1 Introduction

  • DAG models are widely used in probabilistic reasoning, machine learning, and causal inference because of their modularity, scalability, statistical properties, and causal interpretations.Source: §1 · p. 1
  • Marginalizing latent variables generally yields distributions not faithfully represented by any DAG, and the resulting observed-variable models are complex and poorly understood.Source: §1 · p. 1
  • Existing mixed graphical models do not capture the full range of DAG margins, and no class of ordinary graphs suffices regardless of its number of edge types.Source: §1 · pp. 1–2
  • The paper introduces mDAG hypergraphs with hyper bidirected edges; latent projection represents DAG margins, and each hyper-edge intuitively represents an exogenous latent variable's children.Source: §1 · p. 2
  • mDAGs represent observational DAG margins and intervention structure; under a suitably defined intervention class, causally interpreted mDAGs correspond one-to-one with marginal models.Source: §1 · p. 2
  • Marginal DAG models can be preferable when hidden variables represent arbitrary confounders, since explicit latent-variable models require assumptions that may be implausible or untestable.Source: §1 · pp. 2–3

2 Directed Graphical Models

This section establishes the graph, random-variable, and Markov-property conventions used for DAG models. It relates structural equations to conditional independence through d-separation.

2 Directed Graphical Models

  • A DAG is a finite directed graph without cycles; paths, parents, children, ancestors, ancestral sets, and induced subgraphs describe its structure.Source: §2 · pp. 3–4
  • The structural equation property represents each variable as a measurable function of its parents and an independent error, recursively following the graph.The formulation permits arbitrary variable state-spaces and does not require a joint density; it also allows degenerate relationships such as functional dependence.Source: §2 · p. 4
  • In the potential-outcomes view, the random function for a variable is treated as the object of interest and identified with its function-and-error pair.Source: §2 · p. 4
  • For continuous parent spaces, independently assigning values at every input can produce a nonmeasurable random function, so evaluating it at a random parent may not define a random variable.Source: §2 · pp. 4–5
  • d-separation tests conditional independence: a path is blocked by a conditioned non-collider or by a collider with no conditioned descendant.Source: §2 · p. 5
  • The structural equation property and the global Markov property are equivalent for DAGs.Source: §2 · p. 5

3 Latent Variables

Marginalizing latent variables in a DAG yields models whose constraints can be complex and are not generally represented by a DAG. This section reviews those constraints and shows how to reduce the graphs needed to study such margins.

3 Latent Variables

  • A marginal DAG model collects observed-variable distributions that can be extended to a joint distribution obeying the original DAG.The latent state spaces are unrestricted in principle, making this a non-parametric model that may avoid additional constraints from arbitrary latent-variable assumptions.Source: §3 · p. 6
  • For an ancestral set A, the margin of a DAG model equals the model of the induced subgraph on A.This preserves the original graph structure in this special case.Source: §3 · p. 6
  • Margins can impose constraints beyond conditional independence, including Verma constraints and, for some models, inequalities.For example, the five-vertex DAG's margin over {1, 2, 3, 4} satisfies X1 ⊥⊥ X3 | X2 and a non-parametric constraint.Source: §3 · p. 7, §3.1 · p. 8

3.1 Existing Results

  • Ordinary Markov models use observed-variable conditional independences and form a super-model of the marginal DAG model; nested Markov models add constraints such as Verma constraints.For discrete variables, nested and marginal DAG models coincide up to inequality constraints.Source: §3.1 · p. 8

3.2 Reduction

  • Exogenizing a latent vertex—connecting its parents to its children and removing its incoming edges—preserves the marginal model over observed vertices.The operation is the exogenized DAG r(D, u).Source: §3.2 · p. 9
  • For fixed observed vertices, only finitely many distinct marginal models arise; latent vertices can be taken as exogenous, with nonnested child sets of size at least two.The result does not generally allow every latent variable to be restricted to exactly two children.Source: §3.2 · pp. 9–11

4 mDAGs

mDAGs extend DAGs with bidirected hyperedges to represent the latent dependence structures that arise in margins. Latent projection constructs these graphs, and equal projections guarantee equal marginal models.

4 mDAGs

  • An mDAG is a DAG plus a simplicial complex of bidirected faces, represented visually by directed edges and bidirected hyperedges.Each nontrivial facet can be viewed as a latent variable whose children are the vertices in that facet.Source: §4 · p. 11
  • Allowing bidirected facets larger than pairs makes mDAGs richer than ADMGs; restricting facets to size at most two recovers ADMGs.The facet structure is inclusion-maximal, unlike the factor-node resemblance noted for factor graphs.Source: §4 · pp. 11–12

4.1 Latent Projection

  • Latent projection preserves directed paths through projected-out vertices and adds bidirected faces for observed vertices sharing a hidden common cause.The operation accommodates joint dependence among more than two variables.Source: §4.1 · pp. 12–13
  • The canonical DAG replaces each nontrivial bidirected facet with a latent vertex whose children are exactly that facet’s vertices.This construction links an mDAG to a DAG representation of its latent structure.Source: §4.1 · pp. 13–14
  • Projecting vertices in stages or all at once yields the same mDAG, regardless of projection order.Figure 6 illustrates projecting vertices 1 and 2 in either order.Source: Figure 6 · p. 13, §4.1 · pp. 14–15
  • Two DAGs with the same latent projection onto observed vertices have the same marginal model over those vertices.The result establishes that mDAGs can represent the full class of marginal DAG models, which ordinary non-hyper graphs cannot fully capture.Source: §4.1 · pp. 15–16

5 Markov Properties

The marginal Markov property defines an mDAG’s model through its canonical DAG, with ancestral and district-based properties yielding useful consequences. Weaker ordinary and nested Markov models generally leave out constraints retained by the marginal model.

5 Markov Properties

  • For a DAG margin, the marginal Markov property on its latent projection gives the same model as the original DAG margin.The marginal model of an mDAG is defined as the model of its canonical DAG over the specified vertices.Source: §5 · p. 16
  • For an ancestral set, the marginal model of the induced mDAG equals the marginal model of the original mDAG on that set.The marginal Markov property also yields factorizations over districts, with conditional distributions satisfying the corresponding district-subgraph property.Source: §5 · pp. 16–17

5.1 Weaker Markov Properties

  • In general, the marginal, nested, and ordinary models satisfy Mm(G) ⊆ Mn(G) ⊆ Mo(G), with both inclusions strict in some cases.The nested definition recursively constrains district factors, whereas the ordinary factorization imposes no further structure on them.Source: §5.1 · pp. 17–18
  • The nested model imposes recursive district constraints that the ordinary model does not, and the two models can differ.In the example, the nested property adds X4 ⊥ X1 | X3 while the ordinary property gives no further constraints.Source: §5.1 · p. 18
  • For discrete X2 and X4, the marginal model additionally imposes Bell’s inequality, so it is strictly smaller than the nested model.The nested model therefore omits inequality constraints of the marginal model, though for discrete variables the two are known to have the same dimension.Source: §5.1 · p. 18

6 Markov Equivalent Graphs

The section develops partial criteria for when mDAGs represent the same marginal model, while showing that ordinary graphs cannot capture the full range of marginal models. It illustrates equivalence reductions and their limits across state spaces and graph sizes.

6 Markov Equivalent Graphs

  • Proposition 6.1 preserves the marginal model when a bidirected facet satisfying its conditions is split into C and D and the directed edges from C to D are added.The proof interprets the change as splitting the latent variable for the facet into two independent pieces.Source: §6 · p. 19
  • The reduction can make a graph more DAG-like, with smaller bidirected facets that clarify how the joint distribution factorizes.The example applies the proposition to the graph in Figure 9(a) and its equivalent graph in Figure 9(b).Source: §6 · p. 19, Figure 9 · p. 20
  • For discrete state spaces, different mDAG skeletons imply different marginal models, but this need not hold when a variable is continuous.When X2 is continuous, the three Figure 10 models are saturated despite sharing the skeleton 1 − 2 − 3.Source: Figure 10 · p. 20, §6 · p. 21

6.1 Bidirected Graphs and Connection to ADMGs

  • ADMG latent projection can conflate distinct marginal models: a single latent common cause yields a saturated model, unlike three independent pairwise latent causes.For binary observed variables, the pairwise model cannot assign probability one half to each of the all-zero and all-one outcomes.Source: §6.1 · pp. 21–22, Figure 12 · p. 22
  • Among mDAGs with only bidirected edges, equality of marginal models holds if and only if the graphs are identical.This result contrasts with broader mDAG equivalence, where distinct graphs can represent the same model.Source: §6.1 · p. 22
  • No class of ordinary graphs, even with multiple edge types, can represent all DAG marginal models; hyper-edges are therefore necessary.The paper relates this limitation to the faster growth in the number of marginal models than ordinary graphs.Source: §6.1 · p. 23

6.2 mDAGs on Three Variables

  • Among 48 mDAGs on three unlabelled vertices, there are eight marginal-model equivalence classes; the paper cannot determine saturation for the illustrated four-node graphs.Five of the eight classes are DAG models; the other three include the instrumental variables, unrelated confounding, and pairwise bidirected models.Source: §6.2 · pp. 23–24, Figure 14 · p. 24

7 Causal Models and Interventions

The section extends mDAGs to interventions by defining graph mutilation and showing how it relates to latent projection. It also establishes when interventions distinguish causal mDAGs and notes limits imposed by the intervention setting.

7 Causal Models and Interventions

  • Intervention effects are not identifiable from the observed probability distribution alone; they depend on the full structural generating system.The intervention definition replaces the structural functions and error variables for intervened nodes.Source: §7 · pp. 24–25

7.1 Causal mDAGs

  • Intervening on observed nodes removes incoming directed edges and those nodes from bidirected faces, and mDAG mutilation commutes with latent projection.Thus the mutilated latent projection represents the margin after intervention.Source: §7.1 · pp. 25–26
  • The paper concludes that mDAGs represent both marginal structure and how it changes under interventions on observed variables.This extends the paper’s account of margins of DAG models to their interventional behavior.Source: §7 · p. 24, §7.1 · p. 26
  • Distinct causal mDAGs with different underlying DAGs can be distinguished by some intervention, but distinct mDAGs need not be distinguishable under the considered interventions.The paper gives a pair representing saturated models whose mutilated graphs remain the same after any intervention on their vertices.Source: §7.1 · p. 26, Figure 16 · p. 27
  • More delicate interventions that block individual causal mechanisms would yield a one-to-one correspondence between causal models and underlying bidirected graphs.The remark describes blocking all direct causal links before applying Proposition 6.8.Source: §7.1 · pp. 26–27

8 Discussion

mDAGs offer a framework for representing margins of non-parametric Bayesian networks and their structure under causal interventions, but model equivalence and practical fitting remain unresolved. Nested and latent-variable models can serve as surrogates, with trade-offs in what they capture.

8 Discussion

  • Markov equivalence is only partially characterized for the marginal Markov property, and equivalence for the nested Markov model remains open.The framework represents margins of non-parametric Bayesian networks and their structure under causal interventions.Source: §8 · p. 27
  • Fitting and testing marginal Markov models is difficult because they generally lack an explicit representation, and a general parameterization remains open.The nested model is a useful surrogate, but because it contains the marginal model, a poor nested-model fit rules out a good marginal-model fit while the converse does not hold.Source: §8 · p. 27
  • The nested model is smooth, explicitly parameterized, and dimension-matched to the marginal model in the discrete case, but ignoring inequality constraints may lose power.Inequality constraints can in principle be tested after fitting a larger model; a general parameterization incorporating them remains open.Source: §8 · p. 27
  • A latent-variable model is a second surrogate: if its fit resembles the nested model's under a suitable criterion, the marginal model lies between them.Methods for fitting marginal Markov models would enable powerful searches to distinguish causal models with latent variables.Source: §8 · pp. 27–28

A.1 Proof of Theorem 4.10

The proof establishes that latent projection can be performed in stages: projecting over two latent sets and then over the observed vertices yields the same graph features as projecting over their union.

A.1 Proof of Theorem 4.10

  • If elements of B share hidden common cause A with respect to U1 in H, latent projection implies A shares hidden common cause C with respect to U2 in G, so B shares C with respect to U1 ∪ U2 in G.Source: §A.1 · p. 32
  • Directed edges between observed vertices are preserved when latent projection is performed in stages rather than over both latent sets at once.A directed path through the union of latent sets corresponds to one through the remaining latent set after the first projection.Source: §A.1 · pp. 31–32
  • Hidden common causes among observed vertices are likewise preserved under staged latent projection.The proof identifies first vertices outside the second latent set along paths from the shared cause, then transfers the resulting paths through the projection.Source: §A.1 · pp. 31–32

A.2 Measure Theoretic Results

The measure-theoretic results support constructions used to represent marginal models: they control approximate measurability and construct extensions whose observed margin is preserved. The proof also explains the decomposition of a shared latent variable into independent pieces.

A.2 Measure Theoretic Results

  • The epsilon-measurability condition bounds the mean squared difference between X and its conditional expectation given F by epsilon.This defines the approximation condition used in the measure-theoretic results.Source: §A.2 · p. 32
  • If Xi are each approximately measurable with respect to all but their own independent sigma-algebra, and pairwise squared differences are at most epsilon, then Var Xi is at most k epsilon.The lemma also gives (2 epsilon, F−i,j)-measurability for each Xi when i differs from j.Source: §A.2 · pp. 32–33
  • A measurable X determined by Y and Z can be extended using independent uniform variables to preserve the joint distribution of (X, Y, Z).The proof constructs X from Y and an independent U, then constructs Z from (X, Y) and an independent W.Source: §A.2 · p. 33
  • For P in the marginal model, there exists an extended-model distribution Q whose V-margin is P and whose latent variables split as YB = (YC, YD).Each Xd is measurable with respect to XC, its parents, and the specified additional variables.Source: §A.2 · p. 33
  • The construction decomposes YB into independent pieces: one determines XC given its parents, while the other adds no information once XC and its parents are known.The proof applies Lemma A.3, allowing the relevant variables on which XD can depend as additional inputs.Source: §A.2 · p. 33
Loading 1408.1809v2…