Source-linked AI summary
Scalable and Versatile Identification for Hierarchical Structural Causal Models: A New Look at Project STAR
Janis Aiad, Aghiles Drali, Aymen El Ouadrhiri, Anass Ettahiri, Yasser Oufqir, Simon Patry, David Cortes, Marianne Clausel, Emilie Devijver
TL;DR
Hierarchical causal inference for nested data needs to distinguish class-level interventions from student-level effects, but flat models cannot represent that structure. This paper builds an operational HSCM pipeline combining symbolic identification with scalable estimation, showing that reliable inference requires hierarchical modeling, numerical estimation, and stability checks.
Problem
Flat causal models lose within-class heterogeneity and cannot distinguish class-level interventions from student-level interventions in nested datasets such as Project STAR.
Method
The pipeline transforms hierarchical graphs, obtains pyAgrum do-calculus estimands, converts them into closed-form HSCM formulas, and estimates them through independently computable AST tasks.
Results
Validation on synthetic benchmarks and STAR shows that flat models recover associations but miss unit-level interventions, while symbolic identification alone is insufficient for reliable hierarchical inference.
Takeaways & Limitations
For class-level interventions with student-level outcomes, the appropriate estimand is hierarchical rather than a flat pooled regression.
Takeaways & Limitations
Factor normalization used to stabilize numerical evaluation changes the weighting and therefore the numerical functional being evaluated.
Abstract
from arXiv · showhide
The STAR (Student-Teacher Achievement Ratio) experiment (1985, Tennessee, USA) is a landmark hierarchical dataset designed to assess the impact of class size on student outcomes, with observations nested within classes. To encode class-level interventions in such hierarchical settings, we develop a complete, scalable, open-source pipeline for Hierarchical Structural Causal Models (HSCM) that bridges symbolic identification and practical estimation. Our approach integrates graph transformations, pyAgrum's do-calculus for automatic identification of causal effects, adaptation of symbolic expression into closed-form HSCM formulas, and numerical estimation from fitted local probability models. A key innovation is our adapted Abstract Syntax Tree (AST), which decomposes pyAgrum's identified formulas into independent density, expectation, and marginalization tasks, enabling parallel and scalable computation. We validate the pipeline on canonical HSCM motifs and benchmark scenarios with known ground truth, then apply it to STAR kindergarten mathematics outcomes. The results show that flat baselines (ignoring hierarchy) recover associations but fail to encode class-level interventions, and that symbolic identification alone is not enough for practical Hierarchical Structural Causal inference; scalable estimation and numerical stability checks are central parts of the scientific object.
1 Introduction
The paper operationalizes hierarchical structural causal inference for Project STAR by combining hierarchical modeling with symbolic identification and scalable numerical estimation. It argues that flat models and symbolic identification alone are insufficient for reliable causal inference in nested data.
- Motivation: Project STAR randomized Tennessee students in 1985 to small, regular, or regular-with-aide classes and measured student mathematics and reading scores.The study assigned students within schools to three class types in a fully randomized design.
- Motivation: Flat analyses fail because aggregation loses within-class heterogeneity, while pooling students can induce latent dependence and confounding.Examples include peer effects, teacher-student interactions, and class composition.
- Method: The paper operationalizes HSCMs through graph collapse, outcome-distribution augmentation, marginalization, pyAgrum identification, and numerical estimation from fitted local probability models.These steps address unresolved operational needs involving local model fitting, computational scaling, and real-world validation.
- Contributions and results: Validation on synthetic benchmarks and STAR kindergarten mathematics shows that hierarchical modeling and symbolic methods must work together for reliable structural causal inference.The pipeline is presented as full, scalable, operational, and available as open-source Python code.
- Contributions and results: Flat models may recover associations but cannot capture unit-level interventions, whereas symbolic identification alone does not provide sufficient practical hierarchical causal inference.The paper illustrates this distinction through its analysis of STAR data and comparison of hierarchical and flat models.
2 Preliminaries: causal inference and hierarchical models
This section defines structural causal models and interventions, then extends them hierarchically by separating unit- and subunit-level mechanisms. HSCMs support unit-level causal targets over distributions of subunit treatments, identified by transforming the hierarchy into a flat graphical model for do-calculus.
- Structural causal models: An SCM consists of exogenous variables, endogenous variables, structural functions, and a strictly positive distribution over mutually independent exogenous variables.Each endogenous variable is determined by its parents and corresponding exogenous inputs.
- Interventions: Hard interventions replace selected structural functions with constants and remove incoming graph edges, whereas soft interventions assign possibly non-constant functions without adding causal connections.These interventions distinguish direct modification of variables from functional changes that preserve the existing causal connections.
- Hierarchical structural causal models: HSCMs partition endogenous variables into unit-level and subunit-level variables, with distributions at both levels and a unit-level intervention alongside classical individual-level intervention.The hierarchical formulation is designed for settings such as classes containing students.
- Hierarchical causal targets: Q-variables summarize subunit-level treatment and outcome mechanisms at the unit level, enabling the target τ(q0 →q1) = E[Y | do(Q(a) = q1)] −E[Y | do(Q(a) = q0)].This target contrasts expected unit-level outcomes under two different distributions of subunit-level treatments.
- Graphical identification: A three-step transformation—collapsing, augmenting, and marginalizing—converts an HSCM into a flat model so standard do-calculus can assess target identifiability.Collapsing replaces subunit mechanisms with Q-nodes, augmenting adds Q(y), and marginalizing projects out variables outside the identification problem.
3 Causal effect estimation in Hierarchical Structural Causal Models
The section presents a complete, scalable pipeline for estimating causal effects in HSCMs by connecting graph transformation, symbolic do-calculus identification, and modular numerical evaluation. Its AST-based implementation supports local and parallel computation while exposing positivity, stability, and computational-complexity considerations.
- Pipeline: The pipeline converts an HSCM into a flat causal graph with Q-variables, applies pyAgrum’s do-calculus identification, and translates the resulting expression back into hierarchical mechanisms.The modeling step uses collapsing, augmenting, and marginalizing transformations; pyAgrum operates on the collapsed HSCM graph rather than the original nested table.
- Numerical evaluation: The numerical backend selects local estimators for AST-derived conditional factors, enabling fitting across units, factors, or Monte Carlo samples in parallel.Supported examples include Bernoulli or multinomial models for discrete variables and Gaussian or two-component Gaussian mixtures for continuous scores.
- AST representation: The adapted AST encodes identified formulas as nested marginalization, product, conditional, and density-factor operations that can be evaluated as independent tasks.Required factors include P(W | Qa), P(Y | Qa, Qz | a, W), P(Qz | a | Qa, W), and P(Qa).
- Numerical stability: Positivity violations or unsupported density nodes can destabilize ratio-based calculations, so the implementation records such cases and normalizes unstable multiplicative factors before evaluation.This stabilization preserves the identified graph structure while preventing an unstable factor from overwhelming the functional.
- Computational cost: Under ideal parallelization, global pipeline complexity scales as O((nmd + BK)/P), with numerical evaluation dominating the comparatively negligible symbolic steps.The dominant numerical work includes fitting local models and evaluating Monte Carlo components.
4 Experimental results over synthetic benchmarks
Synthetic benchmarks validate the pipeline on three canonical hierarchical causal motifs with known ground truth. The hierarchical estimator converges toward the intervention target as sample size increases, while batched GPU evaluation provides large absolute speedups whose scaling declines across the measured range.
- Benchmark designs: Three canonical motifs—confounding, confounding with interference/frontdoor structure, and instrumental-variable design—test graph transformation, identification, AST evaluation, and numerical estimation against known ground truth.These motifs are validation cases rather than restrictions of the pipeline.
- Convergence: With m = 50 sub-units per unit, the hierarchical estimate converges toward the true ATE as n increases from 10 to 200, whereas pooled OLS remains biased.The hierarchical estimator recovers the unit-level intervention encoded by the graph rather than only a pooled association.
- GPU benchmarking: Approximately −0.32 is the fitted average log–log speedup slope across motifs; GPU evaluation remains thousands of times faster in absolute time, but speedup decreases over the measured range.The benchmark uses feasible tensor batch size, which depends on n and m, making throughput hardware-specific rather than a fixed worker-count comparison.
- Convergence: The hierarchical within-unit estimator removes the unit effect and follows the expected 1/√nm decay, while the flat pooled estimator does not approach the intervention target.The contrast reflects the effect of ignoring unit-level latent heterogeneity.
5 Real data analysis: Project STAR
Project STAR is analyzed with a hierarchical intervention that fixes the class-level small-class assignment distribution rather than replacing student-level assignments. The HSCM contrast differs from student-level OLS, while graph selection, class-specific mechanisms, normalization, and diagnostics determine the reported estimand and its numerical stability.
- Data and intervention: STAR randomized kindergarten students within schools to small, regular, or aide-supported regular classes, with student- and school-level records, assignments, teacher information, and outcomes.The public STAR-and-Beyond dataset contains the raw longitudinal Tennessee experiment records.
- Data and intervention: The intervention is class-level do(Qa = qa⋆), fixing the small-class assignment distribution, while Q-quantities summarize class composition and class-specific Gaussian means and variances.Qa represents small-class assignment probability or proportion, Qg gender composition, and Qy or Qb fitted outcome-mechanism means and variances.
- Graph and identification: The ExactBIC graph was selected from PC, FCI, DirectLiNGAM, and ExactBIC analyses of flat variables, without claiming it was the unique recovered causal graph.The reported graph was chosen because it was the most informative for the HSCM analysis.
- Estimated effects: 36.80 points was the ExactBIC HSCM mathematics contrast, compared with approximately 9 to 9.5 mathematics points for the three OLS specifications with school fixed effects or student controls.The HSCM estimate is a class-size contrast rather than another estimate of βA, and the comparison is not a direct replacement of the student-level OLS coefficient.
- Diagnostics and computation: The numerical analysis required normalization and stability checks because estimation uses 322 classes with 10 sampled students per class, and raw factor traces varied on roughly 10^3 scales.The effective sample size is the number of classes, and multi-parent class-level factors can be numerically unstable; with 4 processes, evaluation took 1.9 seconds versus 3.11 seconds sequentially, a speed up of 1.64.
6 Conclusion
The paper presents a scalable pipeline that operationalizes hierarchical structural causal inference by connecting automatic symbolic identification with closed-form formulas and numerical estimation. Experiments and the STAR application validate hierarchical estimands while highlighting limitations in deeper hierarchies and multi-parent Q-density estimation.
- Scalable identification and estimation: The pipeline connects hierarchical graph transformations, pyAgrum do-calculus, closed-form HSCM formulas, and AST evaluation using fitted local probability and response models.This turns HSCM from a formal identification framework into an operational workflow.
- Empirical validation and STAR application: The experiments validate the pipeline on known structures, while STAR shows that class-level interventions with student-level outcomes require a hierarchical estimand.OLS, IV, and non-causal hierarchical summaries remain benchmarks but do not encode interventions on within-class treatment distributions or class-composition mechanisms.
- Limitations and future developments: The implementation is currently two-level, so deeper student–class–school hierarchies are needed for a fully faithful education application.Multi-parent Q-density estimation is the main statistical bottleneck because missing or weakly estimated factors can change the effective estimand rather than merely add numerical noise.