Source-linked AI summary
When Do Learned Blocks Preserve Causal Meaning? Block Identifiability and Constructive Causal Abstractions
Karim Zaghw, Wieland Brendel, Patrik Reizinger
TL;DR
Block-identifiability leaves internal coordinates of multidimensional learned blocks unresolved, raising the question of whether interventions on those blocks retain their causal meaning. The paper uses constructive causal abstractions to characterize the transformations that preserve that meaning. It identifies a maximal class under a transported block-to-concept correspondence and illustrates intervention transfer with approximately preserved effects in a synthetic experiment.
Problem
Block-identifiability can leave internal coordinates of multidimensional learned blocks undetermined, leaving unclear whether interventions on those blocks retain their high-level causal meaning across representations.
Source: §1 · p. 1Method
The paper uses constructive causal abstractions to characterize which representation changes preserve block-to-concept mappings and interventions.
Source: §1 · pp. 1–2Results
Block-diagonal invertible transformations with compatible coordinate permutations are the maximal class guaranteed to preserve every such abstraction, and a synthetic experiment approximately preserves intended intervention effects across independently trained additive autoencoders.
Source: §1 · p. 2Takeaways & Limitations
Under a given block-aligned constructive abstraction, block-identifiability's ambiguity does not destroy the high-level causal interpretation when the alignment is transported with the representation.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Standard identifiability theory assumes one-dimensional latent factors, but learned representations can encode latent information across multidimensional blocks whose internal coordinates are not uniquely determined. When such blocks are used to represent and intervene on higher-level concepts, it is unclear whether their causal meaning survives the transformations left unresolved by block-identifiability. We connect block-identifiability to constructive causal abstraction. We show that block-diagonal invertible transformations followed by compatible coordinate permutations form the maximal class that preserves every constructive causal abstraction built on those blocks. This guarantee assumes that the correspondence between blocks and concepts is updated with the representation. A small synthetic experiment with additive autoencoders shows that high-level interventions can be transferred between independently learned representations while approximately preserving their intended effects.
1 INTRODUCTION
The introduction asks whether block-identifiability leaves higher-level causal meaning intact when concepts span multiple learned coordinates. It frames constructive causal abstractions as a way to answer that question and identifies the transformations that preserve them.
1 INTRODUCTION
- Standard identifiability results often assume one learned coordinate per latent factor and recover factors only up to permutation and separate invertible coordinatewise transformations.Because these transformations do not mix factors, additional assumptions can permit recovery of causal relations.Source: §1 · p. 1
- Concepts may span several learned coordinates, so block-identifiability leaves their internal coordinates undetermined and their causal meaning across representations unresolved.Motivating examples include SAE feature splitting, multidimensional visual concepts, vector-valued object slots, and subspaces recovered up to within-block transformations.Source: §1 · p. 1
- Constructive causal abstractions formalize the question by mapping low-level states and interventions to a concept-level causal model, with each concept computed from a learned-coordinate block.The framework draws on constructive causal abstractions (Beckers and Halpern, 2019; Geiger et al., 2025).Source: §1 · p. 1
- Block-diagonal invertible transformations followed by compatible coordinate permutations form the largest class guaranteed to preserve every abstraction of this form.The block-to-concept correspondence must move with the representation; transformations outside the class fail to preserve at least one abstraction (Thm. 1).Source: §1 · p. 2
- A small synthetic experiment transfers high-level interventions between independently trained additive autoencoders and approximately preserves their intended effects.The interventions are translated into low-level block edits and transferred using fitted blockwise maps.Source: §1 · p. 2
2 BACKGROUND
The background sets up learned representations and concepts as causal models, then defines the mappings needed to compare their states and interventions. It distinguishes consistent causal predictions from the additional block-to-concept alignment required for constructive abstractions.
2 BACKGROUND
- The representation-level SCM L treats learned coordinates as variables, while the high-level SCM H contains human-interpretable concepts that need not coincide with the ground-truth factors.The network maps ground-truth factor settings s to activations o = f(s), and the featurizer maps activations to representation x = g(o).Source: §2 · p. 2
- A deterministic SCM specifies variables and mechanisms; a hard intervention replaces selected mechanisms with constants, and interventions are ordered by extension.A setting is a solution when each variable agrees with its mechanism's output; the empty intervention is the identity.Source: §2 · p. 2
- Exact transformations require low-to-high mapping of interventions and resulting solutions to yield consistent high-level outcomes.The transformation uses a surjective state map and a surjective, order-preserving intervention map.Source: §2 · p. 2
- Bijective translations re-express a causal model's states, mechanisms, and interventions in new coordinates, allowing an abstraction to be carried to the reparameterized model.Transported interventions retain their composition and ordering, and bijective translations compose with other exact transformations.Source: §2 · pp. 2–3
- Figure 1 traces factors through activations and learned blocks to high-level concepts, while emphasizing that ground-truth factors need not equal those concepts.The abstraction maps low-level states and complete-block interventions to high-level states and interventions.Source: §2 · p. 2, Figure 1 · p. 3
- Constructive abstractions add an alignment: a partition assigns low-level variables to high-level concepts, and each concept has a surjective cell map from its assigned block.Variables may also occupy an optional unmatched cell; many-to-one cell maps make low-level values in the same fiber indistinguishable at the high level.Source: §2 · p. 3
3 PRESERVING CONSTRUCTIVE CAUSAL ABSTRACTIONS
The paper asks when reparameterizing learned blocks preserves their causal interpretation. It identifies a maximal class that preserves every block-aligned constructive causal abstraction when the block-to-concept correspondence is transported with the representation.
3.1 Setup
- The setup assigns each learned block to one high-level concept and considers interventions on complete blocks, since a concept may depend on every coordinate in its block.Block-identifiability leaves individual within-block coordinates undetermined.Source: §3.1 · p. 4
3.2 The Maximal Class Preserving Constructiveness
- The representation change must transport both states and interventions, while tracking each block's new cell so the concept correspondence is preserved.The permitted transformations combine within-block reparameterizations with coordinate permutations that move coordinates to transported cells.Source: §3.1 · p. 4, §3.2 · p. 4
- Preserving exactness alone is insufficient: a transformed intervention may cease to be hard, or a high-level concept may depend on coordinates from multiple cells.The XOR example shows overlapping dependencies can prevent concepts from being computed from disjoint cells.Source: §3.2 · pp. 4–5
- Invertible within-block transformations followed by compatible coordinate permutations form the maximal class guaranteed to preserve every block-aligned constructive causal abstraction.The transported cells may be interleaved; transformations outside this class break at least one abstraction, though a particular abstraction may survive.Source: §3.2 · p. 5
4 EXPERIMENTS
The experiments test whether concept interventions translated into block edits can be transported between independently trained additive autoencoders. In a controlled synthetic setting, the edits approximately preserve their intended effects in retained models, but do not establish an exact causal abstraction.
4 EXPERIMENTS
- Synthetic observations came from two factor pairs through a nonlinear additive map, while the high-level model retained s1 and s2 and omitted nuisance variables u1 and u2.Each learned representation had two two-dimensional blocks; only models with reconstruction NMSE at most 0.01 were retained for transport.Source: §4 · p. 6
- Concept-labelled examples fit affine block-to-concept maps, and held-out prediction selected the one-to-one block–concept assignment.Paired codes from the same observations were used to fit affine maps between blocks, with the pairing chosen by lowest held-out code error.Source: §4 · pp. 6–7
- Because a concept map can be many-to-one, a nearest-point rule selects a reference block edit, which is then transferred using fitted within-block maps and a block permutation.The same base observation and exact generator comparison are used to evaluate the reference and recipient outputs.Source: §4 · pp. 6–7
- Every decoded intervention in the reference model and four retained recipient models had NMSE at most 0.012 against the corresponding exact generator comparison.The reference and four of ten additional autoencoders passed the reconstruction threshold; the other six were excluded before steering.Source: §4 · p. 7, Table 1 · p. 7
- The experiment is an approximate illustration and does not establish that the learned models form an exact constructive abstraction.The reported synthetic results can be reproduced from fixed seeds using the supplementary code.Source: §4 · p. 7
5 RELATED WORKS
Related work connects causal abstractions and block-identifiability to concept steering, while distinguishing this paper’s question from prior abstraction discovery and alignment methods. The paper’s theory addresses preservation under representation changes, and its experiment measures approximate rather than guaranteed abstraction.
5 RELATED WORKS
- Constructive causal abstractions relate causal models across levels; distributed alignment search finds subspaces for specified variables, whereas this work studies changes preserving any abstraction with given blocks (Rubenstein et al., 2017; Beckers and Halpern, 2019; Geiger et al., 2025).Other cited approaches seek macro-variables, high-level models, or indistinguishable latent variables (Chalupka et al., 2017; Saulus et al., 2026; Li et al., 2025).Source: §5 · pp. 7–8
- Independent subspace analysis identifies latent subspaces up to permutation and invertible within-subspace transformations; additive decoders have an analogous guarantee under additive structure (Hyvärinen and Hoyer, 2000; Theis, 2006; Lachapelle et al., 2023).These transformations are a special case of the family characterized by the paper’s theorem.Source: §5 · p. 8
- Prior steering work includes one-dimensional directions identified up to permutation and rescaling, while block-sparse featurizers represent concepts as multidimensional manifolds whose coordinates are defined only up to rotation (Joshi et al., 2025; Fel et al., 2026).These settings motivate asking when block-based concept steering retains causal meaning under representation changes.Source: §5 · p. 8
- The maximality theorem concerns exact abstractions; the synthetic experiment measures approximate steering and transport rather than establishing an approximate abstraction guarantee.Approximate abstraction can instead quantify error in reproducing the low-level system (Beckers et al., 2019).Source: §5 · p. 8
6 DISCUSSION
The discussion limits the theory to given deterministic models, block correspondences, and exact abstractions, and the experiment to a controlled approximate setting. It identifies extensions and real-representation evaluation as open problems, while reporting the use and review of generative AI tools.
6 DISCUSSION
- The result assumes deterministic SCMs, a fixed block partition and correspondence, a given exact abstraction, and hard interventions on complete blocks; it does not explain how to learn these assumptions from data.The synthetic experiment also uses concept-labelled observations and retains only accurately reconstructing autoencoders.Source: §6 · p. 8
- Open problems include recovering blocks and abstractions under identifiability assumptions, extending the theorem to stochastic SCMs, soft interventions, and approximate abstractions, and testing real neural representations.Real representations lack the exact generator comparisons used in the synthetic experiment.Source: §6 · p. 8
- Under a given block-aligned constructive abstraction, block-diagonal reparameterizations with compatible coordinate permutations are the maximal class guaranteed to preserve every such abstraction.Transformations outside the class may preserve a particular abstraction but have no universal guarantee; the synthetic experiment illustrates intervention transfer between independently trained representations.Source: §6 · p. 8
AI use statement
- Generative AI tools assisted with experimental code and limited manuscript edits; the authors report reviewing the code, outputs, and edits and taking responsibility for the final work.Source: AI use statement · pp. 8–9
A TERMINOLOGY
The terminology distinguishes low-level blocks, high-level values, and the maps connecting them. It also names the transformations and interventions used to study causal abstractions.
A TERMINOLOGY
- The intervention algebra Ψ comprises hard interventions on zero or more complete blocks.Source: A TERMINOLOGY · p. 11
- An exact transformation is a bijective translation that transports interventions with the coordinate change.The transported intervention is IT = T ◦ I ◦ T^−1.Source: A TERMINOLOGY · p. 11
- A block partition groups low-level variables into B blocks, and each high-level value is computed from one block through a cell map.The cell map π_b is a surjection from the block’s values to the corresponding high-level values.Source: A TERMINOLOGY · p. 11
- The transformation family T consists of block-diagonal reparameterizations followed by compatible coordinate permutations.Compatible permutations preserve matching value domains and may interleave the transported cells.Source: A TERMINOLOGY · p. 11
- Block-identifiability recovers blocks up to permutation and invertible within-block maps; ISA is its linear within-subspace case.Source: A TERMINOLOGY · p. 12
B PROOFS
This section contains the proof of the main theorem and a secondary corollary.
B PROOFS
- The section presents proofs of the main theorem and a secondary corollary.Source: B PROOFS · p. 12
B.1 Universal maximality of block-diagonal invariance
The proof establishes preservation in one direction by transporting the abstraction and interventions, then derives the required transformation form in the converse direction. The converse uses an identity abstraction to retain the information needed to factor the transformation blockwise.
B.1 Universal maximality of block-diagonal invariance
- A block-diagonal reparameterization followed by a compatible permutation transports alignments and interventions while preserving constructiveness.The proof constructs transported cells and establishes exactness for interventions on complete blocks.Source: §B.1 · pp. 12–13
- The converse uses an identity abstraction that packages each complete block as a product-valued macro-variable without losing information.Its bijective translation is exact, placing this abstraction within the family quantified over by the preservation condition.Source: §B.1 · p. 13
- Constructiveness of the transported identity abstraction forces each output block of T^−1 to depend only on its assigned transported input block.The resulting blockwise bijections assemble to T = σ ◦ D; exactness is needed to include the identity abstraction in the quantified family, not for the algebraic factorization itself.Source: §B.1 · p. 13
B.2 Uniqueness up to macro value relabeling
When two constructive abstractions have matching fibers after a block reparameterization, their high-level values are related by a unique blockwise bijection. Their interventions and solution sets correspond under that relabeling; identical transformed cell maps yield identical solution sets.
B.2 Uniqueness up to macro value relabeling
- If the cell maps have matching fibers after a block reparameterization, a unique blockwise bijection relates the abstractions’ high-level values.Source: §B.2 · p. 13
- The bijection transports the abstraction maps and conjugates the induced intervention maps between the two high-level models.Source: §B.2 · p. 13
- When the cell maps are exactly related by the block reparameterization, the relabeling is the identity and the models have the same solution sets under every intervention in ω1(Ψ).Source: §B.2 · p. 13
C CONCEPT-DIRECTED STEERING AND TRANSPORT DETAILS
All retained models pass the predeclared checks, and selected cross-model block maps validate substantially better than alternative pairings. The procedure fits concept and cross-model maps, selects an intervention point in a block’s fiber, and evaluates edits against a shared exact generator observation.
C CONCEPT-DIRECTED STEERING AND TRANSPORT DETAILS
- Each example independently draws s1, u1, s2, u2 from Uniform[−1, 1], lifts each pair to five coordinates, then applies a fixed orthogonal mixing matrix; autoencoders minimize reconstruction MSE without factor labels.Source: C CONCEPT-DIRECTED STEERING AND… · p. 14
- The experiment uses held-out reconstruction and map-validation splits, then evaluates 5,000 interventions per concept in the reference model and four retained recipient models.Cross-model maps are fitted and validated on 5,000 and 2,000 paired, unlabelled observations, respectively.Source: C CONCEPT-DIRECTED STEERING AND… · p. 14
- Affine reference cell maps are selected by one-to-one assignment maximizing validation R2, while recipient block pairings minimize worst-block validation NMSE.The selected within-block maps are fitted using paired codes of the same observations without factor values.Source: C CONCEPT-DIRECTED STEERING AND… · p. 14
- For an intervention, the method chooses the closest point in the affine map’s fiber over the target concept value, which is a representative rather than a unique inverse.The point is selected from the intervened reference block for the base observation.Source: C CONCEPT-DIRECTED STEERING AND… · p. 14
- Evaluation compares edited outputs with one exact generator observation that changes only the intervened concept while preserving its nuisance value and the other factor pair.Predeclared checks include reconstruction NMSE at most 0.01, cross-model map NMSE at most 0.05, and final common-comparison NMSE at most 0.02.Source: C CONCEPT-DIRECTED STEERING AND… · p. 15
- All retained models pass the predeclared checks; selected cross-model maps have validation NMSE 0.001985–0.005839, versus near 1 for alternative block pairings.The reference model reconstructs with NMSE 0.007312, and four of ten additional autoencoders pass the reconstruction gate with NMSE between 0.007762 and 0.007927.Source: C CONCEPT-DIRECTED STEERING AND… · p. 15