Independent mechanism analysis, a new concept?

1 Introduction

• Nonlinear ICA is not identifiable because independent estimated components can remain mixed relative to the true sources.The introduction frames identifiability as recovering the ground truth up to well-defined ambiguities.Source: §1 · p. 1
• Existing routes to identifiability often add supervision or structure, such as auxiliary variables, whereas this work investigates an alternative inspired by causal inference.Here, independence in the ICM principle concerns generative modules chosen independently, not statistical independence of random variables.Source: §1 · p. 1
• IMA constrains the mixing function by requiring source contributions, represented by Jacobian columns, to be orthogonal.The absolute Jacobian determinant should equal the product of the column norms, expressing decoupled local influences.Source: §1 · p. 2, Figure 1 · p. 2
• The paper reports that IMA rules out common nonlinear ICA counterexamples, is consistent with existing identifiability results, and empirically outperforms an unregularized baseline.It also introduces a triangular-Jacobian learning method and a metric for assessing blind source separation.Source: §1 · p. 2

2.1 Independent component analysis (ICA)

• ICA seeks an unmixing that yields independent components, while blind source separation seeks the true sources up to permitted ambiguities.Identifiability requires models producing the same mixture distribution to be equivalent under the chosen ambiguity relation.Source: §2.1 · p. 3
• Nonlinear ICA is not identifiable from independence alone: elementwise transformations preserve independence, and the Darmois construction yields independent uniform components that may remain nonlinearly mixed.The estimated components need not be meaningfully related to the true sources.Source: §2.1 · p. 3
• Rotated-Gaussian measure-preserving automorphisms map sources to an isotropic Gaussian, rotate them, and map them back without changing their distribution, creating further non-equivalent solutions.The construction is a counterexample unless the rotation is a permutation.Source: §2.1 · p. 4
• Auxiliary variables such as environments, timestamps, or labels can support nonlinear ICA identifiability when they render sources conditionally independent.A second noisy view is another setting with related results.Source: §2.1 · p. 4

2.2 Causal inference and the principle of independent causal mechanisms (ICM)

• The ICM principle treats a generative process as autonomous modules that do not share information; its independence is not statistical independence between random variables.For cause and effect, this becomes independence between the cause distribution and the mechanism.Source: §2.2 · p. 4

3 Existing ICM measures are insufficient for nonlinear ICA

• IGCI is insufficient for BSS identifiability because a Darmois solution composed with a rotated-Gaussian automorphism satisfies IGCI without separating the true sources.The proof notes that IGCI is satisfied when the source distribution is uniform, as in the Darmois construction.Source: §3 · p. 5
• Existing ICM criteria target independence between an observed cause distribution and its mechanism, but BSS observes only effects, leaving causes unobserved.The criteria rely on the cause distribution being informative.Source: §3 · p. 5
• An uninformative uniform source distribution makes any mechanism generic under these criteria, allowing spurious solutions to satisfy them.The section also notes that such criteria are generally not invariant to reparameterization of cause variables.Source: §3 · p. 5

4.1 Intuition behind IMA

• IMA treats each source’s partial derivative contribution to the observations as a mechanism and requires these contributions to be independent in a non-statistical sense.The contributions are the columns of the mixing Jacobian.Source: §4.1 · p. 5
• The IMA principle is equivalent to orthogonality of the mixing Jacobian’s columns: the determinant’s volume equals the product of their norms exactly when they are orthogonal.This geometric condition is imposed at every source value.Source: §4.1 · p. 6
• Unlike IGCI’s decoupling of source density and mixing mechanism, IMA decouples individual source influences; both yield additive KL-divergence decompositions under their respective formulations.IMA’s information-geometric interpretation measures influence under soft interventions on individual sources.Source: §4.1 · p. 6

4.2 Definition and useful properties of the IMA contrast

• Local CIMA measures pointwise violation of IMA, is nonnegative, and equals zero exactly when the Jacobian columns are orthogonal; its value is invariant to orthogonal basis changes, permutations, and column rescalings.Global CIMA is the expected local contrast under the source distribution.Source: §4.2 · p. 7
• Global CIMA is nonnegative, vanishes only for orthogonal coordinate transformations almost everywhere, and is invariant to source permutations and invertible element-wise reparameterizations.These invariances match nonlinear ICA’s inherent source ambiguities.Source: §4.2 · p. 7

4.3 Theoretical analysis and justification of CIMA

• When some observed coordinates are dependent, every Darmois solution has positive CIMA and is distinguishable from a zero-CIMA solution; conformal and polar-to-Cartesian mixings can have zero CIMA.The polar-to-Cartesian example is non-conformal, showing that zero-CIMA mixings extend beyond conformal maps.Source: §4.3 · pp. 7–8
• For an orthogonal nontrivial linear mixing with at most one Gaussian source, CIMA is zero for the true mixing and positive for a Darmois solution, without requiring dependent observed coordinates.The result assumes source covariance E[s⊤s] = I.Source: §4.3 · p. 8
• For conformal mixings, applying a rotated-Gaussian measure-preserving automorphism gives positive CIMA unless the rotation maps every non-Gaussian source coordinate to another coordinate.Non-Gaussianity is part of the condition that rules out these alternatives.Source: §4.3 · p. 8

5 Experiments

• Experiments use uniform sources and, unless otherwise specified, conformal Möbius ground-truth mixings; a triangular-Jacobian residual-flow architecture learns Darmois solutions.The reported CIMA-identifiability assumption is crucial: some source distributions permit independent Gaussian components after transformation, causing identifiability to be lost.Source: §5 · pp. 8–9, §5.1 · p. 9

5.1 Numerical evaluation of the CIMA contrast for spurious nonlinear ICA solutions

• Learnt Darmois solutions have positive CIMA across tested dimensions, and applying a rotation to a true solution yields zero CIMA only at permutation rotations; Darmois solutions remain above zero.For n = 2, 1000 random mixings were tested; higher-dimensional evaluations covered n ∈ {2, 3, 5, 10}. With MLP mixings, CIMA rose with depth, but Darmois values were typically higher.Source: §5.1 · p. 9

5.2 Learning nonlinear ICA solutions with CIMA-regularised maximum likelihood

• The learning objective maximizes log likelihood minus λ times CIMA, using a residual flow; evaluation uses KL divergence, nonlinear Amari distance, and MCC.λ = 0 is standard maximum likelihood, while λ > 0 imposes CIMA regularization.Source: §5.2 · p. 9
• CIMA regularization improves nonlinear Amari distance and MCC over λ = 0, while most models achieve good KL fit across λ; λ = 0.5 and λ = 1 show no substantial difference.The comparison covers 50 trained models for each λ ∈ {0.0, 0.5, 1.0} and n ∈ {5, 7}.Source: §5.2 · p. 9, Figure 5 · p. 10

Discussion

• IMA constrains nonlinear mixing through (near-)orthogonality of Jacobian columns, while still allowing strongly nonlinear functions.The approach differs from methods that rely on auxiliary variables or favor near-linear mixing.Source: §6 · p. 10
• The authors do not prove full identifiability, but experiments with CIMA suggest recovery of the BSS equivalence class.They note that identifiability might hold under additional assumptions, such as conformal maps.Source: §6 · p. 10
• IMA addresses a different independence question from traditional cause-mechanism criteria; both IMA and IGCI can hold simultaneously, while IGCI alone is inconsequential for BSS.The paper says an element-wise source transformation can make reconstructed sources uniform without changing CIMA.Source: §6 · p. 10
• The authors conjecture that IMA could connect nonlinear BSS to minimizing algorithmic complexity, assuming algorithmic independence of causal mechanisms.They note that suitable proxies for nonlinear mixing complexity have not yet been addressed.Source: §6 · p. 10
• The authors describe this as the first effort, to their knowledge, to use causal ideas—specifically ICM—for BSS; applying IMA to causal discovery or causal representation learning remains future work.They also identify possible relevance to spatial representations, robot dynamics, and physics problems involving orthogonal reference frames.Source: §6 · p. 10
• IMA rules out a large class of spurious nonlinear ICA solutions, and the proposed regularized maximum-likelihood approach is empirically effective at recovering the true solution.The paper states that the principle holds exactly for orthogonal coordinate transformations.Source: §6 · p. 10

Checklist

• The checklist reports that assumptions and limitations are discussed, full proofs are mainly in the appendices, and code, data, and configuration files are included in the supplement.The checklist also says training details are specified in the configuration files.Source: Checklist · p. 16

Overview

• The appendices cover identifiability and linear ICA, existing ICM criteria and their relation to IMA, full theoretical proofs, a worked CIMA computation, and further experimental results.Additional background on conformal maps and Möbius transformations is also listed.Source: APPENDIX Overview · p. 17

A Additional background on identifiability and linear ICA

• The appendix frames identifiability in terms of the source distribution and mixing function, aligning that formulation with the paper’s causal perspective.The authors say this makes separate constraints on mixing functions and source distributions more natural to express.Source: A Additional background on… · p. 17, §A.1 · pp. 17–18

A.1 Identifiability in terms of equivalence relations

• For nonlinear ICA, source ordering and element-wise transformations are unresolved, so the paper uses an equivalence relation to specify which ambiguities may remain.The paper identifies BSS equivalence as the strongest feasible identifiability notion without further parametric assumptions.Source: §A.1 · p. 17

A.2 Identifiability of linear ICA

• With at most one Gaussian source, linear ICA is identifiable up to source rescaling and permutation.The theorem assumes an orthogonal mixing matrix and non-point-mass source densities.Source: §A.2 · p. 18

A.3 Whitening in the context of linear ICA

• Whitening makes the mixing matrix orthogonal and reduces the search from n^2 parameters to n(n −1)/2, but whiteness alone leaves the sources identifiable only up to an orthogonal transformation.Whitening produces uncorrelated components, which need not be independent.Source: §A.3 · p. 19

B.1 Trace method

• The trace method expresses ICM as a condition between the cause covariance and linear mechanism, but whitening can make that condition trivial, so it cannot constrain linear ICA alone when sources are unobserved.Whitening gives uncorrelatedness, not necessarily independence.Source: §B.1 · p. 20

B.2 Information geometric interpretation of the ICM principle

• In information space, the irregularity of the effect distribution decomposes into cause-distribution irregularity and mechanism irregularity.The mechanism term measures the irregularity of the effect distribution obtained by applying f to a regularized cause distribution.Source: Figure 6 · p. 21, §B.2 · pp. 21–22
• The decomposition is presented as an orthogonality principle linking independence of cause and mechanism to orthogonality of their irregularities in information space.The authors describe the geometric independence interpretation as intuitive, based on independently chosen irregularities.Source: §B.2 · pp. 21–22

B.3 Decoupling of the influences in IMA and comparison with IGCI

• Unlike IGCI’s two-term decomposition of effect-distribution irregularity, IMA’s decomposition has m KL-divergence terms, each tied to one source’s influence on the mechanism.Source: §B.3 · pp. 22, 25
• Under the IMA postulate, the KL measure of a soft intervention decomposes into terms associated with interventions on individual source mechanisms.For a DAG, the argument establishes positivity for the m intervened leaf-node terms; the remaining non-leaf term is additional, and the decomposition has only positive terms when sources are jointly independent.Source: §B.3 · pp. 23, 25

B.4.1 Violations of independence of cause and mechanism

• A whitening mechanism can be fine-tuned to an input covariance matrix, making its singular values and vectors dependent on a property of the cause distribution.The example uses i.i.d. causes with an arbitrary, non-diagonal covariance matrix and a whitening matrix that makes the effects uncorrelated.Source: §B.4.1 · p. 25

B.4.2 Violations of the IMA principle

• Autoregressive mixing can have positive CIMA, while concert-hall acoustics fine-tuned to speaker positions can increase collinearity between Jacobian columns and violate IMA.The autoregressive example corresponds to a setting where each microphone picks up only the first speakers.Source: §B.4.2 · pp. 25–26

• Its dependence on the Jacobian is through J_f(s)^⊤J_f(s), the matrix of column inner products.Source: §C.1 · p. 27
• The local IMA contrast is nonnegative and equals zero exactly when the Jacobian columns are orthogonal.Hadamard’s inequality gives the nonnegativity, with equality exactly for orthogonal columns.Source: §C.1 · p. 27
• The local contrast is invariant to left multiplication by an orthogonal matrix and right multiplication by permutation or diagonal matrices.Source: §C.1 · p. 27

C.2 Proof of Prop. 4.6

• The global contrast is nonnegative and equals zero exactly when the Jacobian columns are orthogonal almost surely under the source distribution.Source: §C.2 · p. 28
• CIMA is invariant under source permutations and invertible element-wise transformations when the mixing function and sources are reparameterized accordingly.Source: §C.2 · pp. 28–29

C.3 Remark on a similar condition to IMA, expressed in terms of the rows of the Jacobian

• Expressing the condition using Jacobian rows would lose the needed reparameterization invariance and would not yield a suitable BSS contrast.Source: §C.3 · p. 29

C.4 Proof of Thm. 4.7

• When some observed components are dependent, a Darmois solution has strictly positive CIMA and can be distinguished from a zero-CIMA solution.The proof uses the triangular Jacobian of the Darmois construction: zero contrast would require a diagonal Jacobian and imply independent observations.Source: §C.4 · pp. 29–30

C.5 Proof of Corollary 4.9

• Under the theorem’s assumptions, a conformal mixing function has zero CIMA and is distinguishable from Darmois solutions.Source: §C.5 · p. 31

C.6 Proof of Corollary 4.11

• For non-trivial orthogonal linear mixing with at most one Gaussian source, CIMA is zero for the mixing matrix and positive for the Darmois solution.Source: §C.6 · p. 31

C.7 Proof of Thm. 4.12

• For conformal f and orthogonal R, CIMA(f ◦ a_R(p_s), p_s) > 0 if at least one non-Gaussian source axis is not mapped to a canonical axis by Rᵀ.This is the condition and conclusion of Theorem 4.12.Source: §C.7 · p. 31
• Assuming zero CIMA would require a matrix expression to be diagonal everywhere, after using CIMA’s invariance to element-wise transformations.Smoothness extends the almost-sure diagonal condition to all z.Source: §C.7 · p. 31
• The non-Gaussian source and the rotated axis condition make an off-diagonal term vary with that source coordinate, contradicting its required constancy at zero.The contradiction rules out the zero-CIMA assumption.Source: §C.7 · p. 32

D Worked out example

• For the polar-to-Cartesian mixing, the true solution has CIMA zero while the Darmois construction has strictly positive CIMA, so the contrast distinguishes them.The off-diagonal-term argument used in the example holds inside the disk of radius R.Source: D Worked out example · pp. 33–35

E.1 Sampling random Möbius transformations.

• Experiments use randomly sampled Möbius transformations, generally with ϵ = 2; ϵ = 0 gives the linear case, and the shift parameter is set to zero.The parameter b is sampled outside the source unit square to avoid a vanishing denominator and strong outliers.Source: §E.1 · p. 36

E.2 How to implement the Darmois construction

• A triangular-Jacobian flow implements the Darmois construction by learning conditional densities and mapping observations to a factorized base distribution.The authors report that this empirical use of triangular-Jacobian flows had not, to their knowledge, been tested before.Source: §E.2 · pp. 36–37

E.3 Generating random MLP mixing functions

• The random MLP mixing setup initializes square weight matrices orthogonally, but this alone does not ensure orthogonal Jacobian columns after element-wise nonlinearities.The experiments use a leaky_tanh invertible nonlinearity.Source: §E.3 · p. 37

E.4 Maximum likelihood with low CIMA

• Optimizing the modified low-CIMA maximum-likelihood objective has O(n^3) computational cost, though the authors found it feasible for moderate dimensions.A 64-layer residual flow trained for 10^5 iterations took about 5.3, 5.7, and 6.3 hours for n = 2, 5, and 7, respectively.Source: §E.4 · p. 38

E.5 Evaluation

• MCC matches reconstructed sources to ground-truth sources by correlation, while nonlinear Amari distance compares Jacobians of the true mixing and learned unmixing.The nonlinear Amari distance is zero when the composed mapping matches the BSS equivalence class.Source: §E.5 · pp. 38, 40
• For 50 nonconformal radial transformations, all final Darmois models have CIMA above zero, with the minimum reported as 0.01.This is presented as empirical confirmation of the theoretical result.Source: §E.5 · pp. 40–41, Figure 12 · p. 40
• For linear mixing, λ = 0.5 and 1.0 nearly perfectly recover sources, unlike λ = 0; for nonlinear mixing, regularization improves MCC and Amari distance, apparently most at λ = 1 and larger n.The reported fits remain comparably good in the linear case; nonlinear cases show higher KL divergence.Source: Figure 8 · p. 39, Figure 9 · p. 39, Figure 10 · p. 40
• Across 50 repetitions, regularized flow models have higher median MCC and lower median nonlinear Amari distance than FastICA, while FastICA beats the unregularized baseline.The comparison uses λ = 0.5 and particularly λ = 1.0 for the regularized method.Source: §E.5 · p. 41, Figure 13 · p. 41, Figure 14 · p. 42

F Additional background on conformal maps and Möbius transformations

• In dimensions n ≥ 3, conformal maps are restricted to similarities or compositions of similarities with sphere inversion.The paper identifies this class with the Möbius transformations used in the experiments.Source: F Additional background on conformal… · p. 43