PaperlayerView exact revision on arXiv

Source-linked AI summary

Discrete Score Matching Enables Causal Discovery from Count Data

Euijong Song, Hyewon Park, Gunwoong Park

arXiv:2609.39326v1stat.MLcs.LG

TL;DR

Count-data causal discovery is difficult for score matching because derivatives are unavailable, finite differences may not preserve the sink criterion, and existing identifiable methods rely on specified conditional families. This paper introduces CCS and OCS and estimates them for counts with DISCO’s discrete diffusion. CCS characterizes semiparametric GLM conditionals, bivariate identifiability follows exactly from η-nonlinearity, and DISCO scales to 1,000-node Poisson DAGs.

  • Problem

    Count data lack the derivatives used by score-matching methods, finite differences may not preserve SCORE’s sink criterion, and existing identifiable count-DAG methods rely on specified conditional families.

    Source: §1 · p. 1

  • Method

    CCS conditions curvature on a node’s value for ordering, OCS supports parent recovery, and DISCO estimates both criteria from discrete-diffusion concrete scores.

    Source: §1 · p. 2, §3 · p. 3, §5.2 · p. 5

  • Results

    At d = 1000, DISCO attains median F1 = 0.769 in 57.4 minutes on Poisson DAGs; bivariate direction is identifiable iff η is nonlinear, and zero CCS characterizes semiparametric GLM conditionals.

    Source: §5.1 · p. 5, §7 · p. 9

  • Takeaways & Limitations

    DISCO avoids prespecifying nodewise conditional families for count-DAG discovery, with CCS/OCS recovery supported for nonlinear semiparametric GLM DAGs under the stated assumptions.

    Source: §8 · p. 10

    The class excludes parent-dependent nuisance parameters, such as a negative binomial whose size depends on its parents.

    Source: §4 · p. 4

Abstract

from arXiv · show

Count data pose a challenge for score-matching-based causal discovery: derivatives are unavailable, and simply replacing them with finite differences does not generally suffice for causal discovery. We generalize SCORE's constant-curvature criterion (Rolland et al., 2022) by conditioning on the node's value, yielding the conditional curvature score (CCS) for ordering. We also extend curvature-based parent recovery through the off-diagonal curvature score (OCS), enabling directed acyclic graph (DAG) recovery with both scores constructed from score functions for continuous data and concrete scores for counts. In the bivariate setting, zero CCS exactly characterizes a semiparametric generalized linear model (GLM) conditional form in which the conditional family need not be specified in advance, unlike in classical GLMs. For bivariate semiparametric GLM DAGs under our regularity condition, canonical-parameter nonlinearity is necessary and sufficient for identifiability. In multivariate DAGs, this nonlinearity enables DAG recovery through CCS and OCS. Our framework identifies a new class of semiparametric GLM DAGs that strictly contains the nonlinear Gaussian ANM class identified by SCORE. We introduce DISCO (DIscrete SCOre), a count-DAG recovery algorithm that estimates CCS and OCS using discrete diffusion. Experiments demonstrate accurate DAG recovery across Poisson, negative binomial, binomial, and mixed-family settings, as well as scalability to 1,000-node DAGs on a single GPU.

1 INTRODUCTION

Count-valued data often mix distribution families, while existing count-DAG guarantees commonly rely on prespecified conditional families and finite-difference curvature alone may not identify sinks. The paper introduces CCS and OCS for ordering and parent recovery, establishes identifiability for semiparametric GLM DAGs, and presents DISCO for scalable count-DAG recovery.

1 INTRODUCTION

  • Finite-difference curvature can vary with a sink’s value because its conditional density contributes own-factor curvature, so count-DAG ordering needs a criterion that accommodates it.Existing count-DAG methods often rely on specified conditional families, while sequencing and ecological data can mix families.Source: §1 · p. 1
  • CCS removes own-factor curvature variation by conditioning on the node’s value, while OCS extends curvature-based parent recovery.The scores generalize SCORE’s ordering criterion and extend curvature-based parent selection to counts.Source: §1 · p. 2
  • Zero CCS characterizes a semiparametric GLM conditional form, and canonical-parameter nonlinearity is necessary and sufficient for bivariate identifiability.In multivariate settings, CCS recovers an order and OCS recovers parents; the continuous identifiable class strictly contains SCORE’s nonlinear Gaussian ANMs.Source: §1 · p. 2
  • DISCO estimates joint concrete scores with score-entropy discrete diffusion, then projects them to marginal concrete scores for CCS and OCS.With OCS-based parent selection, a rank transform makes the recovered DAG invariant to strictly increasing coordinate-wise transformations.Source: §1 · p. 2
  • DISCO recovers 1,000-node count DAGs on a single GPU, while existing count-DAG methods’ guarantees can depend on selecting appropriate conditional families.The framework is presented as count-DAG recovery without prespecifying nodewise conditional families.Source: §1 · pp. 1–2

2 BACKGROUND

The background situates score-based causal discovery alongside discrete and count-DAG approaches, then establishes notation for graphs, distributions, and coordinate operators. It highlights that prior count-DAG guarantees often depend on specified conditional structure.

2.1 RELATED WORK

  • Continuous score-based methods use log-density derivatives for ordering and parent recovery, while discrete approaches include concrete score matching and a marginalization-based ordering method.The latter relies on a non-decreasing conditional-randomness assumption; the paper instead states identifiability within its semiparametric GLM class under nonlinearity and regularity conditions.Source: §2.1 · p. 2
  • Count-DAG identifiability has often relied on specified Poisson or QVF structure, and conditionally parametric models still prespecify the conditional family.The paper’s semiparametric GLM class leaves conditional families, links, and canonical-parameter functions unspecified.Source: §2.1 · pp. 2–3

2.2 NOTATION

  • A causal order places each node before its descendants’ causes, and the notation defines sources, sinks, children, parents, and induced subgraphs.The graph is a DAG on d nodes.Source: §2.2 · p. 3
  • Coordinates may be continuous or count-valued under a common density notation, and finite-difference curvature is evaluated only where required shifts remain in support.For a set of nodes, the notation also defines the marginal distribution and density.Source: §2.2 · p. 3

3 CURVATURE SCORES

The section defines CCS for ordering and OCS for parent recovery using diagonal and off-diagonal log-density curvatures. CCS conditions on a node’s value, allowing its curvature to vary with that value while testing for variation attributable to other variables.

3 CURVATURE SCORES

  • CCS measures the conditional variation of a node’s diagonal log-density curvature after fixing that node’s value.Its value is an expectation over the conditional variance, with the same admissible-domain convention for marginal distributions.Source: §3 · p. 3
  • CCS vanishes exactly when the diagonal curvature is almost surely a function of the node alone, unlike SCORE’s requirement that it be constant.For continuous data, the diagonal curvature is a diagonal component of the log-density Hessian; the section establishes when CCS separates sinks from nonsinks.Source: §3 · p. 3
  • OCS supports parent recovery by evaluating off-diagonal curvature under the marginal distribution on a node and its predecessors in a causal order.For count coordinates, the off-diagonal curvature tracks how a log concrete-score entry for a shift in one coordinate changes when another coordinate steps by one level; OCS averages its absolute magnitude.Source: §3 · pp. 3–4

4 SEMIPARAMETRIC GLM DAGS

The section defines semiparametric GLM DAGs through a factorization with arbitrary source marginals and conditional forms that do not require a prespecified family. It defines nonlinearity through the canonical-parameter function and states regularity and model-class boundaries.

4 SEMIPARAMETRIC GLM DAGS

  • A semiparametric GLM DAG factorizes into nodewise conditionals, with arbitrary source marginals and non-source conditional densities in an exponential-family form.The conditional family is determined by the support and base function, while the canonical parameter captures parent dependence.Source: §4 · p. 4
  • The model’s nonlinear condition requires each edge’s canonical-parameter function to be nonlinear in its parent for some fixed values of the other parents.Equivalently, the corresponding second derivative or finite difference is not identically zero.Source: §4 · p. 4
  • Parent-dependent nuisance parameters, such as a negative-binomial size that depends on parents, lie outside the defined class.For fixed-variance Gaussian conditionals, canonical-parameter nonlinearity coincides with SCORE’s mechanism nonlinearity.Source: §4 · p. 4
  • Example conditional distributions illustrate that noncanonical links can yield nonlinear canonical parameters even when the index is affine.The table notes fixed size r for negative binomial and fixed shape α for Gamma examples.Source: Table 1 · p. 4
  • The theory assumes strictly positive densities on product-support interiors and at least three support values for count coordinates.The results are stated under corresponding regularity conditions.Source: §4 · p. 4

5 CURVATURE CHARACTERIZATION AND DAG IDENTIFIABILITY

In the bivariate setting, zero CCS characterizes a semiparametric GLM conditional form, and canonical-parameter nonlinearity determines identifiability. In multivariate DAGs, CCS identifies sinks and OCS recovers parents, yielding identifiability for nonlinear semiparametric GLM DAGs.

5.1 BIVARIATE CHARACTERIZATION AND IDENTIFIABILITY

  • Zero CCS holds exactly for the semiparametric GLM conditional form, which can be derived without prespecifying the conditional family.The derivation uses the fact that the child’s log-density curvature depends only on the child’s value.Source: §5.1 · p. 5
  • Within the bivariate semiparametric GLM class, the direction is identifiable if and only if the canonical-parameter function η is nonlinear.The theorem imposes Condition R1 in the affine case.Source: §5.1 · p. 5

5.2 MULTIVARIATE IDENTIFIABILITY

  • On every ancestral remaining set, a node is a sink exactly when its CCS is zero, enabling sink-based causal ordering under the nonlinearity condition.The CCS conditions on the node’s value, so its own-factor curvature becomes deterministic while residual curvature from children remains variable.Source: §5.2 · p. 5
  • Given a causal order, OCS is positive exactly for parent edges; a nonconstant parent effect suffices, without requiring nonlinearity.The stated curvature relation is evaluated when the child is a sink of the relevant remaining graph.Source: §5.2 · pp. 5–6
  • Nonlinear semiparametric GLM DAGs are identifiable, and this class strictly contains SCORE’s nonlinear Gaussian ANMs.The Gamma example remains identifiable despite nonconstant sink curvature and zero CCS.Source: §5.2 · p. 6

6 DISCO ALGORITHM

DISCO estimates concrete scores on rank-transformed counts, uses CCS to order nodes, and selects parents using OCS or alternative methods. A joint score network and marginal projection support repeated CCS and OCS computation, while the OCS-based procedure has stated consistency and transformation-invariance results.

6.1 CONCRETE-SCORE ESTIMATION

  • DISCO rank-transforms each count variable, estimates joint concrete scores with discrete diffusion, and projects them to marginal scores for ordering and parent selection.The projection network takes the remaining-variable set as input, providing marginal scores for different sets without refitting.Source: §6 · p. 6, §6.1 · p. 6

6.2 CCS AND OCS COMPUTATION

  • CCS estimates average diagonal curvature and measures its within-value variance across observations where the required finite differences are defined.The computation groups observations by the rank value of the node and uses a sample-size-weighted average of within-group variances.Source: §6.2 · p. 7

6.3 DAG RECOVERY

  • Starting with all nodes, DISCO repeatedly removes a minimizer of the CCS ordering criterion, then selects parents using OCS, CAM, or PCM-GAM.OCS-based selection reuses fitted score networks rather than requiring additional regressions or conditional-independence tests.Source: §6 · p. 6, §6.3 · p. 7
  • With OCS-based parent selection, DISCO returns the same graph under strictly increasing coordinate-wise transformations when using the same seed.Under Theorem 8's assumptions, raw-OCS thresholding recovers the graph with probability tending to one when the curvature-estimation error is o(λ_n) and λ_n tends to zero.Source: §6.3 · pp. 7–8
  • The OCS-based implementation has O(nd + d^2) counted operations when training epochs, evaluation sample size, hidden widths, and mask draws are fixed.CAM and PCM-GAM require additional regressions or tests.Source: §6.3 · p. 8

7 NUMERICAL EXPERIMENTS

The experiments evaluate count-DAG recovery across single- and mixed-family settings, compare ordering and parent-selection variants, and test scalability and real-data ordering agreement. DISCO performs especially well at larger sample sizes in the r = 6 and mixed-family settings and scales to 1,000-node Poisson DAGs on one GPU.

Conditional curvature ablation

  • For binomial data, mean F1 is 0.877 with CCS and 0.884 with the constant-curvature criterion, although CCS improves sink-selection accuracy more for Poisson and negative-binomial data.The ordering comparison uses the same fitted networks and d = 50, n = 5000.Source: §7 · p. 8, Table 2 · p. 9

Parent recovery given an estimated order

  • Given the same estimated DISCO order, CAM has higher mean F1 than OCS for NB and Bin, whereas OCS performs best for Poi.Parent-selection comparisons use d = 50 and n = 5000.Source: §7 · p. 8, Table 2 · p. 9

DAG recovery versus sample size

  • At larger sample sizes, DISCO has the highest median F1 under NB r = 6 and mixed families, while ODS (Oracle) is strongest under NB r = 1.Figure 1 compares end-to-end F1 versus sample size for d = 100; the r = 1 case has vanishing own-factor curvature, while DiffAN shows no consistent gains with more data for r = 6 and mixed families.Source: §7 · p. 8, Figure 1 · p. 9

Scalability with graph size

  • At d = 1000, DISCO reaches median F1 = 0.769 in 57.4 minutes on a single GPU, while all baselines exceed the 240-minute time limit.The scalability experiment uses Poisson DAGs with n = 5000; runtimes are not a hardware-matched speed comparison.Source: Table 3 · p. 9, §7 · p. 9

Family-agnostic DAG learning on real MLB data

  • On Lahman batting data, DISCO's mean ordering agreement with seven domain-informed relations is 0.905, compared with 0.333–0.429 across ODS family choices.The partial references do not define a ground-truth DAG, and DISCO requires no family or link input.Source: §7 · pp. 9–10

8 CONCLUSION

The paper concludes that CCS and OCS support score-matching-based causal discovery from counts without prespecifying nodewise conditional families. DISCO estimates these criteria through joint concrete scores and marginal projection, with OCS-based recovery scaling to 1,000-node DAGs in the experiments.

8 CONCLUSION

  • DISCO relaxes the prespecified-family assumption of existing identifiable count-DAG methods, using CCS for ordering and OCS for parent recovery in nonlinear semiparametric GLM DAGs under stated assumptions.The conclusion identifies end-to-end consistency—from finite-noise score estimation and marginal projection through curvature estimation to DAG recovery—as the main remaining theoretical challenge.Source: §8 · p. 10

C.3 PROOF OF THEOREM 8

The proof establishes CCS sink characterization and OCS parent recovery, then uses these results to show DAG identifiability and a strict extension of SCORE’s identifiable class.

C.3 PROOF OF THEOREM 8

  • On every ancestral remaining set, sinks have zero CCS while every nonsink has positive CCS.This separates sinks from nonsinks for causal ordering.Source: §C.3 · p. 17
  • Given a causal order, OCS is finite and positive exactly for parents; linear canonical-parameter dependence with nonzero slope already suffices.For a source, the off-diagonal curvature is zero for every predecessor.Source: §C.3 · p. 18

C.4 PROOF OF COROLLARY 9

  • For nonlinear semiparametric GLM DAGs, the common sink ordering and recovered parent sets imply that the graph is identifiable, without moment conditions.The proof applies the sink rule successively, then recovers each node’s parents.Source: §C.4 · p. 18

D.1 STRICT EXTENSION OF SCORE

  • The identifiable continuous semiparametric GLM class strictly contains SCORE’s nonlinear Gaussian ANMs; a Gamma example has S2 = 0 but Var(H2) > 0.Its nonconstant own-factor curvature makes SCORE’s constant-curvature criterion fail, while CCS identifies the model.Source: §D.1 · p. 18

D.2 LINK–INDEX CHARACTERIZATION

The link–index characterization reduces bivariate identifiability to whether the canonical-parameter function h ◦ f is nonlinear. It also specifies when this condition holds for affine and nonlinear indices.

D.2 LINK–INDEX CHARACTERIZATION

  • In the setting of Theorem 6, X →Y is identifiable exactly when h ◦ f is nonlinear on the interior of the support of X.Here the canonical-parameter function is η = h ◦ f.Source: §D.2 · p. 19
  • For an affine index and a canonical-class link, identifiability is equivalent to a nonlinear index; otherwise, an affine restriction of h can make an affine index unidentifiable.With a nonlinear index, non-identifiability occurs exactly when f = h−1 ◦ a for an affine function a in h(If).Source: §D.2 · p. 19
  • Every unidentifiable case also has a canonical-link representation with an affine index.This is the stated link–index characterization’s final consequence.Source: §D.2 · p. 19

E.1 PROOF OF PROPOSITION 10

  • The proof of Proposition 10 establishes that the zero-extended neighboring entries used by DISCO are integrable.This verifies integrability for the neighboring entries used in the procedure.Source: §E.1 · pp. 19–20

E.2 PROOF OF THEOREM 11

Under the stated assumptions, the proof establishes consistency of CCS ordering and raw-OCS parent selection, and thus recovery of the DAG. It also gives supporting results for score estimation, transformations, and implementation.

E.2 PROOF OF THEOREM 11

  • The grouped CCS estimator and the averaged OCS estimator converge in probability to their population scores under the stated moment and estimation-error conditions.The proof establishes convergence for CCS and OCS before showing order and graph recovery.Source: §E.2 · p. 21
  • Under (13) and λn → 0, each true parent with Oj→i(R) > 0 is selected with probability tending to one; a union bound over T(G), together with the correct-order event, yields Pr{bG = G} → 1.Source: §E.2 · p. 21

E.3 PROOF OF PROPOSITION 12

  • Strictly increasing transformations preserve the count-data CCS and OCS values, and therefore preserve the resulting causal orders and parent sets.The fitted rank transform also produces identical transformed inputs under the stated fixed-seed setup.Source: §E.3 · p. 22

F.1 DENOISING SCORE-ENTROPY OBJECTIVE

  • The denoising score-entropy objective has the corrupted concrete score as its unique population minimizer on modeled entries.Positive generator and time weights do not change the conditional minimizer.Source: §F.1 · p. 22

F.2 VANISHING-NOISE LIMIT

  • As the corruption kernel tends to the identity, the noisy marginal converges pointwise to the original marginal.The stated result also gives convergence of the noisy concrete score for supported states.Source: §F.2 · p. 23

G DISCO IMPLEMENTATION

  • DISCO fits the rank transform and score networks on training folds, then applies them to held-out folds to compute CCS and OCS.This is the specified cross-fitting procedure.Source: G DISCO IMPLEMENTATION · p. 23

G.1 RANK TRANSFORM

  • The rank transform maps observed coordinate values to their ordered ranks and clips or coarsens held-out values onto that rank grid.Values between observed ranks are assigned by order rather than Euclidean distance.Source: §G.1 · p. 23

G.2 JOINT CONCRETE-SCORE TRAINING

The appendix specifies DISCO’s training, projection, curvature estimation, and parent-selection procedures, then evaluates recovery across count families, graph structures, and continuous data. Results include improved ordering and recovery in several settings, with performance varying by family and graph structure.

H.1 EXPERIMENTAL SETUP

  • CCS improves DAG recovery over constant curvature on Poisson and negative-binomial data, while the criteria are similar on binomial data.For binomial data, constant curvature gives slightly higher mean F1 and lower mean SHD; the sink-identification gains from CCS are larger for Poisson and negative binomial than binomial.Source: Table 6 · p. 26, §H.1 · p. 26

H.2.1 INVARIANCE TO STRICTLY INCREASING TRANSFORMATIONS

  • Median Atop = 0.922 and F1 = 0.894 remain unchanged across transformations with the rank transform; without it, median F1 falls from 0.899 on raw counts to 0.301 under x2 +x.The comparison uses Poisson DAGs; DISCO-NORANK omits the rank transform.Source: Table 7 · p. 27, §H.2.1 · p. 27

H.2.3 MIXED-FAMILY DAG RECOVERY

  • The mixed-family evaluation assigns Poisson, negative-binomial, and binomial mechanisms across all six family-order configurations and compares DISCO with four ODS specifications.ODS (Oracle) uses true nodewise conditional families and fixed parameters; DISCO receives no family labels.Source: §H.2.3 · p. 27, Figure 3 · p. 28

H.2.6 SCALE-FREE DAG RECOVERY

  • Recovery varies with graph structure and conditional family; higher Atop on SF3 for Poisson and negative binomial does not imply higher F1.The scale-free results extend evaluation beyond ER graphs but do not establish performance independent of graph structure or family.Source: §H.2.6 · p. 30, Table 10 · p. 30

H.2.7 DAG RECOVERY BEYOND STANDARD COUNT FAMILIES

  • DISCO maintains accurate ordering in both tested non-Poisson COM–Poisson settings, but ν = 0.5 has lower mean F1 than ν = 1.These results support recovery on additional fixed-carrier one-parameter families, not arbitrary count distributions or parent-dependent shape parameters.Source: §H.2.7 · p. 30, Table 11 · p. 30

H.2.8 CONTINUOUS-DATA ORDERING RECOVERY

  • For Gaussian data, DISCO-CONT has higher mean Atop on ER1 while DiffAN leads on ER3; for Gamma data, DISCO-CONT leads on both graph types.The Gamma methods share the same joint network.Source: §H.2.8 · p. 31, Table 12 · p. 31

H.3 REAL DATA: LAHMAN BATTING COUNTS

On Lahman batting counts, DISCO’s q4 ordering agrees with domain-informed partial directions more than the fixed-family ODS variants, while ODS results vary with the supplied family. The analysis is a post-fit diagnostic, not causal validation, and also examines preprocessing sensitivity and projection methods.

H.3 REAL DATA: LAHMAN BATTING COUNTS

  • The q4 cohort contains 1,400 player-season observations and 17 batting-count variables, but quantile compression need not preserve raw-count conditional-independence structure.The q4 representation was selected to increase observations per count level for CCS estimation.Source: §H.3 · p. 31
  • On q4, DISCO respects 6/7, 7/7, and 6/7 R7 relations across three runs (mean Atop = 0.905), versus mean Atop values of 0.429, 0.333, and 0.429 for ODS Poisson, negative-binomial, and binomial specifications.The reference set is incomplete, so non-reference edges are not treated as false causal edges.Source: §H.3 · p. 32, Figure 4 · p. 33
  • ODS order and edge estimates change more under the negative-binomial specification than between Poisson and binomial specifications on q4.Because q4 codes are compressed counts, this is interpreted as sensitivity to supplied ODS specification, not as correctly specified family comparisons.Source: §H.3 · p. 32, Table 13 · p. 32
  • The reported directional pattern persists across examined reference sets and periods, with mean agreement 0.952 on R7 and 0.922 on R17 for 2012–2018.These remain descriptive partial-order diagnostics, not causal validation.Source: §H.3 · p. 34

I.1 MARGINAL PROJECTION METHODS

  • Amortized projection has comparable or higher mean Atop, higher mean F1, lower mean SHD, and shorter projection time than stagewise alternatives in each conditional family.Differences are not uniform across seeds, and computation budgets are not matched.Source: §I.1 · p. 34

I.2 PARENT SELECTION

The parent-selection study compares OCS and alternative selectors under fixed learned orders and examines how threshold choices affect recovery. Its results distinguish population criteria from finite-sample performance and do not provide error-control guarantees.

I.2 PARENT SELECTION

  • For negative-binomial data, relaxing fold agreement alone raises F1 from 0.797 to 0.870; relaxing both threshold requirements increases false positives and lowers F1 relative to one-at-a-time changes.The same OCS estimates can yield different recovery under different selection rules, and the thresholds are not tuned against the true graph.Source: Table 17 · p. 35, §I.2 · p. 35
  • PCM’s population conditional-mean criterion identifies true parents under a valid causal order in the semiparametric GLM model, but this does not guarantee exact finite-sample PCM-GAM recovery.The comparison uses separate ordering and parent-recovery samples; candidate tests use α = 0.05 without multiplicity correction.Source: §I.2 · pp. 35–36
  • With the shared estimated order, OCS outperforms PCM-GAM in all three families and slightly leads CAM for Poisson, while CAM remains strongest for negative binomial and binomial.With the true order, CAM has the highest mean F1 and lowest SHD in every family; the OCS gap reflects learned scores and the practical selector, not the population OCS characterization.Source: Table 18 · p. 36, §I.2 · p. 36
Loading 2609.39326v1…