PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Identifiability of Mixed Ordinal and Exponential Family Causal DAGs under Linear Parametric Models

Sambit Mishra, Urbashi Mitra

arXiv:2609.17942v1cs.LGmath.STstat.MEstat.MLstat.TH

TL;DR

Identifying causal directions across mixed ordinal and exponential-family nodes remains unresolved by prior approaches that assume a common family or an exponential-family form in both directions. The paper uses linear parametric models to study these edges and establishes orientation guarantees with stated support and graph conditions, alongside numerical illustrations.

  • Problem

    Prior approaches leave ordinal–exponential-family edges untreated when they require a common family across nodes or an exponential-family representation in both directions.

    Source: §1 · p. 3

  • Method

    The paper models each child’s conditional-distribution parameters as functions of a weighted linear combination of its parent values, allowing conditional families to differ across nodes.

    Source: §2.2 · p. 7

  • Results

    The paper establishes mixed-edge orientation guarantees at every parameter value, and synthetic experiments show orientation error decaying with sample size and identification within a Markov equivalence class.

    Source: §4 · pp. 20–22, §5 · p. 22

  • Takeaways & Limitations

    The guarantee extends to orienting every such mixed edge in a d-node DAG under the stated assumptions.

    Source: §5 · p. 22

    Likelihood-based population optimality does not establish estimator consistency because the KL divergence may not remain bounded away from zero over unbounded competing parameters.

    Source: §3.3 · p. 18

Abstract

from arXiv · show

The problem of identifiability in linear parametric models (LPMs) whose nodes follow either an ordered logit model or a regular one-parameter exponential family is evaluated. The results go beyond classical structural equation models as well as results for nodes with observations from a homogeneous family of distributions. The main result establishes that the orientation of every edge joining an ordinal node to an exponential-family node is identifiable from the joint distribution alone at every parameter value, provided the ordinal node has at least three categories and the exponential-family node at least three points of support, with no restriction on the sufficient statistic. Converses show that both requirements are necessary: the three-category requirement is binding only for affine sufficient statistics, and the three-point requirement is binding under the canonical link. The guarantee extends to orienting every such mixed ordinal-exponential family edge of a given $d$-node undirected skeleton. Numerical experiments illustrate the theoretical results by successfully separating orientations within a Markov equivalence class, which are indistinguishable by conditional independence alone.

1 Introduction

The paper addresses observational identification of causal directions across ordinal and exponential-family nodes, a setting not covered by prior approaches that rely on a common family or an exponential-family representation in both directions. It develops an LPM framework and proves mixed-edge orientation guarantees with converse conditions and numerical validation.

1 Introduction

  • Observational identification matters because causal knowledge enables intervention design and separation of cause from correlation.A DAG represents variables as nodes and causal influence as directed edges.Source: §1 · pp. 1–2

1.2 Related Works

  • Prior identifiability results often use restrictive SEM mechanisms or a single distributional family, while CPCM conditions do not carry over when an ordinal endpoint lacks an exponential-family representation.The mixed ordinal–exponential-family edge is presented as an open identifiability problem.Source: §1 · pp. 2–3

1 Introduction

  • LPMs let parent values shape a child’s conditional-distribution parameters through a weighted linear combination, allowing different conditional families across nodes.The framework accommodates heterogeneous variables without ad hoc transformations.Source: §1 · p. 3
  • Every ordinal–exponential-family edge is distributionally identifiable at every parameter value when the ordinal node has at least three categories and the exponential-family node at least three support points, without restricting the sufficient statistic.The result is stated for the mixed setting introduced in the paper.Source: §1 · p. 3
  • When the exponential-family node has two support points, a forward–reverse parameter construction gives identical joint distributions; with a binary ordinal node, identifiability holds exactly for non-affine sufficient statistics.The three-category requirement is binding only for affine sufficient statistics.Source: §1 · p. 3
  • The multivariate result orients every such edge in a d-node DAG under the Markov property, causal sufficiency, and absence of selection, and identifies the full orientation of a given skeleton.A converse establishes necessity for edges whose endpoints have no neighbors other than each other.Source: §1 · p. 3
  • Synthetic experiments on a canonical three-node Markov equivalence class show orientation error decaying with sample size and successful identification within the class.The experiments illustrate orientations that conditional-independence information alone cannot distinguish.Source: §1 · p. 3

2 Preliminaries

The preliminaries define LPMs as models in which parent values influence a node’s conditional distribution through a weighted linear predictor, then specialize the framework to ordinal and regular one-parameter exponential-family nodes. The ordered-logit conditional is not a one-parameter exponential family, establishing a structural distinction relevant to the later results.

2.1 Causal DAGs

  • A DAG is assumed acyclic, with the joint distribution Markov with respect to the graph; Markov-equivalent DAGs share a skeleton and v-structures.The paper’s identifiability question concerns directions that conditional-independence relations alone may not distinguish.Source: §2.1 · p. 6

2.2 Linear Parametric Models

  • Unlike an SEM’s exogenous-noise mechanism, an LPM lets parent values determine conditional-distribution parameters through a weighted linear combination, while allowing different conditional families across nodes.The conditional family determines the child’s support, dispersion, and tail behavior.Source: §2.1 · p. 7, §2.2 · p. 7

2.3 Ordinal-Exponential LPM

  • The ordinal–exponential LPM partitions nodes into ordinal and exponential-family sets and uses ordered logit for non-root ordinal nodes.Ordinal nodes have finite ordered support; their cutpoints are independent of parent values and weights.Source: §2.3 · p. 8, §2.3.1 · pp. 8–9

2.3.2 Exponential-Family Nodes

  • An exponential-family node has a regular one-parameter conditional family with a minimal sufficient statistic and natural parameter linked monotonically to its parents’ linear predictor.Its support may be discrete or continuous.Source: §2.3.2 · p. 9
  • For at least three ordinal categories and three support points, the ordered-logit conditional indexed by the other node’s values is not a one-parameter exponential family.This mismatch distinguishes the ordinal conditional from the exponential-family representation used for the other node type.Source: §2.3.2 · p. 9

3 Identifiability

This section studies identification of ordinal–exponential-family edge directions in bivariate and multivariate causal DAGs, including settings where orientations can induce the same joint distribution and conditions on category and support counts.

3.1 Identifiability in Bivariate Case

  • With at least three ordinal categories and three exponential-family support points, the two bivariate orientations induce disjoint joint-distribution classes.The guarantee holds for every valid parameter value and imposes no condition on the sufficient statistic beyond minimality; Proposition 2 extends it to cumulative links whose extreme adjacent log-odds ratios are strictly concave and strictly convex.Source: §3.1 · pp. 12–13

3.2 Analysis of Non-Identifiability

  • With exactly two exponential-family support points, affine sufficient statistics and the identity link permit a continuum of matching forward and reverse models.The coincident forward models form a two-dimensional, Lebesgue-null subset of the s-dimensional parameter space for s ≥3, so this establishes overlap between model classes, not typical non-identifiability.Source: §3.2 · pp. 14–15
  • With two ordinal categories and at least three exponential-family support points, edge direction is identifiable if and only if the sufficient statistic is non-affine.For affine statistics, every forward model has a matching reverse model; thus the three-category requirement is necessary for affine statistics but dispensable otherwise.Source: §3.2 · pp. 15–16
  • The bivariate classification depends on node-family specifications rather than parameter values, so edge orientation can be assessed before observing data.The relevant features are category count, support size, and sufficient-statistic shape.Source: §3.2 · p. 16

3.3 Multivariate Identifiability

  • The section turns from isolated bivariate edges to multivariate mixed ordinal–exponential-family DAGs and introduces multivariate edge-direction identifiability.The supplied passage announces this extension but does not state its results.Source: §3.3 · p. 17
  • With nonzero edge weights, monotone-injective links, causal sufficiency, and no selection, an ordinal–exponential edge is identifiable when its ordinal endpoint has at least three categories and its other endpoint at least three support points.The theorem imposes no additional restriction on the sufficient statistic.Source: §3.3 · p. 17
  • Orientation uses direct joint-distribution comparisons, not conditional independencies, so it requires no faithfulness; recovering the given skeleton by conditional-independence testing still requires adjacency faithfulness or a similar condition.The orientation guarantee holds even when path cancellation creates an independence not implied by the graph.Source: §3.3 · p. 18
  • The guarantee is local to each edge’s endpoint conditional laws: neither endpoint must be a source, no topological ordering is needed, and orientation is not propagated from other edges.The result compares graphs that differ on one edge; free root marginals allow the endpoint conditional forms to retain the bivariate properties.Source: §3.3 · p. 18
  • When every skeleton edge joins ordinal and exponential-family nodes and the stated category, support, link, and weight conditions hold, no other acyclic orientation generates the same distribution, even if it reverses multiple edges.The theorem excludes all alternative orientations simultaneously, rather than chaining single-edge reversals.Source: §3.3 · p. 18
  • At every competing orientation and parameter value, the likelihood criterion has strictly positive Kullback–Leibler divergence from the true distribution, making the truth the unique population optimum.This does not establish consistency, which would require divergence to stay bounded away from zero over unbounded competing parameter sets.Source: §3.3 · p. 18
  • For isolated ordinal–exponential edges, identifiability can fail with two-point support plus affine T and canonical link, or with a binary ordinal node plus affine T.These converse cases require the endpoints to have no other neighbors; whether identifiability is restored for non-isolated edges is left open.Source: §3.3 · p. 19

4 Numerical Results and Discussion

Synthetic experiments use a known bipartite skeleton and exhaustive search to test orientation recovery. Orientation errors decrease with sample size, and the chain and fork are distinguishable despite sharing a Markov equivalence class.

4 Numerical Results and Discussion

  • Exhaustive search minimizes BIC over acyclic orientations of the supplied skeleton, with candidate parameters fitted by conditional maximum likelihood.Because the skeleton is fixed, estimates can differ from the truth only in edge direction; any deviation reflects finite-sample noise rather than optimization error.Source: §4 · p. 20
  • The three tested graphs are a chain, a fork, and an inverse fork, with ordinal endpoints and an exponential-family middle node.The chain and fork share a skeleton and have no v-structures; the inverse fork has a v-structure at the middle node.Source: §4 · p. 20
  • Every structure’s orientation error decays with sample size; the inverse fork reaches zero errors fastest, while rates vary by structure and exponential-family choice.The inverse fork reaches the numerical floor for every family except Binomial and Pascal; the chain reaches it only for Poisson and Gamma (β), and the fork for Pascal, Gaussian, and Gamma (α).Source: §4 · pp. 20–21, Figure 2 · p. 21
  • The chain and fork are distinguishable despite sharing a Markov equivalence class, demonstrating orientation recovery beyond conditional-independence distinctions.The inverse fork has a v-structure and belongs to a different equivalence class.Source: §4 · pp. 19–21

5 Conclusions

The paper establishes mixed ordinal–exponential-family edge identifiability under the LPM and extends the guarantee to multivariate DAGs. Its proof uses non-exponential-family structure and incompatible functional constraints, while the conclusion reports experiments showing orientation error decreases with sample size.

5 Conclusions

  • Mixed ordinal–exponential-family orientations are identifiable at every parameter value without restricting the sufficient statistic, with a multivariate extension under stated graph assumptions.The conclusion also reports converses and experiments in which orientation error decreases with sample size.Source: §5 · p. 22

A.1 Proof of Proposition 1

  • With at least three ordinal categories, the ordered-logit conditional cannot be represented as a one-parameter exponential family.The proof uses three distinct support points and strict convexity of a log-ratio curve to contradict the affine pairwise log-ratio structure required by an exponential family.Source: §A.1 · pp. 23–24

A.2 Proof of Lemma 1

  • In the forward model, pairwise conditional log-ratios are affine functions of the sufficient statistic T(y).Their slope and intercept are independent of y.Source: §A.1 · p. 23, §A.2 · p. 25

A.3 Proof of Lemma 2

  • For the reverse model, the two extreme adjacent log-odds-ratios have real-line extensions that are strictly concave and strictly convex, respectively.The closed forms are twice differentiable on the real line, and their strict curvature follows when the reverse edge weight is nonzero.Source: §A.3 · p. 26

A.4 Proofs of Theorem 1

Both proofs assume that opposite edge directions generate the same joint distribution and derive incompatible constraints. One uses log-odds curvature; the other contrasts the conditional families implied by the two directions.

A.4 Proofs of Theorem 1

  • Equal joint distributions force matching conditional distributions and therefore matching log-odds-ratios across the two directions.The proof uses equality of the joint laws as the starting point for both contradictions.Source: §A.4 · pp. 26, 29
  • Eliminating the sufficient statistic yields a function with at most two points at any given value, contradicting its constancy on at least three support points.The contradiction uses the curvature argument and the requirement |Y| ≥3.Source: §A.4 · pp. 27–28
  • Alternatively, the forward conditional of X given Y is a one-parameter exponential family, while the reverse ordered-logit conditional admits no such representation under the theorem’s conditions.Since equal joints require equal conditionals, this mismatch rules out shared joint distributions and identifies the direction.Source: §A.4 · pp. 28–29

A.5 Proof of Proposition 2

The proof rules out a common joint distribution by deriving incompatible constraints on the reverse model’s extreme log-odds-ratios. Their curvature forces a contradiction when the ordinal support has at least three points.

A.5 Proof of Proposition 2

  • A common joint law would imply equal conditionals and equal log-odds-ratios in both directions.The proof begins by summing the joint laws to obtain a common marginal and then dividing by it.Source: §A.5 · p. 30
  • Matching the forward log-odds representation forces the reverse lower and upper extreme log-odds-ratios to be affine in the sufficient statistic.The two extreme contrasts are combined to eliminate that statistic.Source: §A.5 · p. 30
  • The resulting function is strictly concave or strictly convex, so it cannot take the same value at three distinct support points, contradicting |Y| ≥3.The slopes have the same sign, which makes the combined function strictly curved.Source: §A.5 · pp. 30–31
  • Thus the two model classes cannot share a joint distribution, and the edge direction is distributionally identifiable.The contradiction rules out the common-joint assumption.Source: §A.5 · p. 31

A.6 Proof of Theorem 2

The proof constructs forward and reverse parameters with identical joint laws. It matches conditional log-odds increments, selects admissible parameters by support-size parity, and then matches the marginal of Y.

A.6 Proof of Theorem 2

  • The construction matches forward and reverse changes in category log-odds between two support values of Y.With the canonical sufficient statistic, forward increments are proportional to category index differences, and the reverse increments are matched through the cutpoint sequence.Source: §A.6 · p. 32
  • The matched increments yield a two-step recursion for the cutpoint quantities, which separates into even- and odd-indexed chains.The recursion is D_i − D_{i+2} = w∆T, with endpoints D_0 = L and D_s = 0.Source: §A.6 · pp. 32–33
  • For even support size the first cutpoint can vary over a non-empty interval; for odd support size the reverse edge weight can vary over one.In either case the resulting cutpoints remain strictly ordered.Source: §A.6 · pp. 33–34
  • After matching the conditional laws, the construction sets the reverse marginal of Y equal to the forward marginal, making the joint distributions coincide.The admissible parameter interval therefore gives a family of parameter pairs with the same joint law.Source: §A.6 · p. 34

A.7 Proof of Theorem 3

With two ordinal categories, the proof separates affine from non-affine sufficient statistics. Affinity permits matching conditional laws and joint distributions; non-affinity contradicts the equality required by a shared joint law.

A.7 Proof of Theorem 3

  • When the sufficient statistic is affine on its support, the forward log-odds-ratio is affine in y and can be matched by choosing reverse parameters.The construction then matches the marginal of Y as well, yielding a common joint distribution.Source: §A.7 · p. 35
  • If the sufficient statistic is non-affine, three support values provide a test of whether it can be affine in y.A shared joint distribution would force the secant slopes of the sufficient statistic and y to agree across those values.Source: §A.7 · pp. 35–36
  • That forced affine relation contradicts non-affinity, so no shared joint distribution exists and the direction is identifiable.The proof concludes that the two model classes induce disjoint sets of joint distributions.Source: §A.7 · p. 36

A.8 Proof of Theorem 4

The proof rules out a shared joint distribution by reducing a competing orientation to a bivariate comparison. The bivariate models cannot coincide under the stated support conditions, so the edge direction is identifiable.

A.8 Proof of Theorem 4

  • The resulting conditionals define forward and reverse bivariate models with nonzero edge weights and matching joint distributions.The construction uses a shifted exponential-family link and displaced ordered-logit cutpoints.Source: §A.8 · pp. 37–38
  • Because both endpoints meet the three-category and three-support-point conditions, Theorem 1 rules out that shared bivariate distribution.This contradiction establishes that the original edge direction is distributionally identifiable.Source: §A.8 · p. 38
  • Assuming a shared joint law, the proof derives matching conditional log-odds ratios for the two orientations.It then factors each orientation and cancels terms that do not depend on the ordinal endpoint.Source: §A.8 · pp. 37–38

A.9 Proof of Theorem 5

The proof reduces a competing multivariate orientation to a bivariate comparison after selecting a suitable disagreeing edge. The local conditional laws then contradict the bivariate identifiability result.

A.9 Proof of Theorem 5

  • Causal minimality and a graph-comparison result yield a disagreeing edge whose conditioning set avoids descendants of both endpoints.The argument first establishes causal minimality using nonzero edge weights and injective or strictly monotone conditional mechanisms.Source: §A.9 · pp. 38–39
  • Conditioning on the selected set gives forward and reverse factorizations of the endpoint pair into the mixed model forms.The local Markov property preserves the endpoints’ parametric conditionals, with parent contributions shifting the link or cutpoints.Source: §A.9 · p. 39
  • The endpoint models have at least three ordinal categories and three exponential-family support points, so their distributions cannot coincide.Both factorizations come from the same joint law, producing a contradiction and ruling out the competing orientation.Source: §A.9 · p. 39

A.10 Proof of Theorem 6

The appendix proves a converse for isolated mixed edges and describes the scoring, search, and nodewise fitting procedures used in the experiments.

A.10 Proof of Theorem 6

  • For an isolated ordinal–exponential-family edge, two-point support with affine sufficient statistic or a binary ordinal node with affine statistic permits coinciding distributions under both orientations.The proof extends the coinciding pair distributions to the full graph by choosing identical conditionals for all other nodes.Source: §A.10 · p. 40

B.1 Score and Optimization Problem

  • Candidate structures are acyclic orientations of a given bipartite skeleton and are scored by BIC with fitted root-node parameters.The score decomposes into node contributions, and root marginals are fitted within the narrower conventions described for each node type.Source: §B.1 · pp. 40–41

B.2 Search Procedures

  • Exhaustive search returns the exact score minimizer, while greedy search reverses the single edge that most improves the score subject to acyclicity.Greedy search is heuristic and may stop even when a combination of reversals would improve the score.Source: §B.2 · pp. 41–42

Appendix C. Per-Node Conditional MLE

  • Both search procedures fit each candidate DAG’s node conditionals by maximum likelihood and compute a score that decomposes over nodes.The appendix treats exponential-family and ordinal conditionals separately.Source: Appendix C · p. 42

C.1 Exponential-Family Nodes

  • For exponential-family nodes, conditional MLE is convex for the listed families under the canonical link and is solved with L-BFGS-B.For exponential links, the linear predictor is constrained to limit numerical overflow.Source: §C.1 · pp. 42–43

C.2 Ordinal Nodes

  • For ordinal nodes, joint MLE fits weights and cutpoints, using softplus increments to enforce strict cutpoint ordering.The reparameterized objective is optimized over an unconstrained domain with L-BFGS-B and analytical gradients.Source: §C.2 · p. 43

Appendix D. Multi-Node Experiments

In 20-node experiments, orientation error declines across sparse configurations, while dense configurations retain error and show late-range plateaus. The appendix also specifies the parameter settings used to generate the experiments.

Appendix D. Multi-Node Experiments

  • In the dense regime, no mode reaches the error floor, and curves flatten near the end of the sample-size range.The dense model has 119 free parameters versus 45 in the sparse model, or roughly four observations per parameter at N = 500.Source: Appendix D · p. 43, Figure 5 · p. 44
  • At N = 500, most sparse-regime trials orient every edge correctly, though error rates differ across the eight modes.Pascal and Gamma (β) have the lowest errors, while Poisson and the mixed mode have the highest, spanning about one order of magnitude.Source: Appendix D · p. 43, Figure 5 · p. 44
  • The mixed mode lies within the envelope of homogeneous modes in both sparse and dense regimes.The experiments use greedy reversal search on 20-node bipartite skeletons with 10 or 90 edges and 1,000 trials per configuration.Source: Appendix D · pp. 43–44

E.1 Generative Parameters

  • Ordinal cutpoints are sampled once per configuration, and exponential-family hyperparameters are fixed across experiments.The appendix lists the family-specific settings, including Gaussian variance 1.0, Binomial trial count 5, and Pascal parameter 3.Source: §E.1 · p. 44

E.2 Three-Node Experiments

The three-node experiment exhaustively evaluated candidate orientations for a supplied mixed ordinal–exponential skeleton across sample sizes and repeated trials.

E.2 Three-Node Experiments

  • The supplied skeleton X1–X2–X3 was not estimated; the candidate orientation set was enumerated exhaustively.X1 and X3 were ordinal with s = 4, and X2 was exponential-family.Source: §E.2 · p. 45
  • Each configuration used edge weight w = 0.6, 20 sample sizes from 5 to 500, and B = 10000 trials.One configuration was run per family of Table 1.Source: §E.2 · p. 45

E.3 Multi-Node Experiments

The multi-node experiments varied graph density and distribution mode on fixed graph models, while the optimizer settings specified estimation and greedy-search stopping rules.

E.3 Multi-Node Experiments

  • For d = 20, experiments used sparse and dense bipartite graphs with 10 and 90 edges, respectively, and one ground-truth graph per regime.Each regime held the type partition, weights, and cutpoints fixed; trials were data replicates from that model.Source: §E.3 · p. 45
  • Experiments compared eight distribution modes across sample sizes from 5 to 500, using B = 1000 trials per configuration.The modes comprised seven homogeneous families and one mixed mode; ordinal nodes had s ∈ {3, 4}.Source: §E.3 · p. 45

E.4 Optimizer Settings

  • Per-node conditional maximum-likelihood estimation used L-BFGS-B, and greedy search stopped at an improvement threshold or reversal limit.The optimizer used at most 200 iterations; search stopped below ϵs = 10^-6 improvement or after 4|Eu| + 1 reversals.Source: §E.4 · p. 45
Loading 2609.17942v1…