Source-linked AI summary
Disintegration and Bayesian Inversion via String Diagrams
Kenta Cho, Bart Jacobs
TL;DR
Conditional probability needs general accounts of disintegration and Bayesian inversion, whose existence and behavior become difficult in measure-theoretic settings. The paper formulates both notions with string diagrams and channels, relates them to each other, and studies them for discrete probability, standard Borel spaces, and likelihoods. It uses the resulting framework to prove conditional-probability results and support examples.
Problem
Disintegration and Bayesian inversion are fundamental conditional-probability operations, but measure-theoretic disintegration may fail to exist or be determined only up to negligible sets.
Method
The paper formulates disintegration and Bayesian inversion abstractly with string diagrams, channels, and copying and discarding structures.
Results
The paper relates Bayesian inversion and disintegration in both directions and develops their treatment for discrete probability, standard Borel spaces, and likelihoods.
Takeaways & Limitations
The graphical formulation supports proofs of conditional independence and other conditional-probability results while organizing state updates and inference.
Takeaways & Limitations
The paper restricts its disintegration treatment to causal states and leaves non-probability and non-finite measures for future work.
Abstract
from arXiv · showhide
The notions of disintegration and Bayesian inversion are fundamental in conditional probability theory. They produce channels, as conditional probabilities, from a joint state, or from an already given channel (in opposite direction). These notions exist in the literature, in concrete situations, but are presented here in abstract graphical formulations. The resulting abstract descriptions are used for proving basic results in conditional probability theory. The existence of disintegration and Bayesian inversion is discussed for discrete probability, and also for measure-theoretic probability --- via standard Borel spaces and via likelihoods. Finally, the usefulness of disintegration and Bayesian inversion is illustrated in several examples.
1. Introduction
The paper presents disintegration and Bayesian inversion as abstract channel constructions in string-diagrammatic probability, relating joint states, conditional probabilities, and backward inference. It develops these ideas across discrete and measure-theoretic settings while addressing existence and almost-everywhere issues.
- Applications: Conditional probabilities organize Bayesian networks, and the paper captures conditional independence abstractly through its channel-based approach.The graph structure of Bayesian networks reflects conditional independencies of the underlying joint distribution.
- Core notions: Disintegration extracts a conditional probability channel from a joint state, giving a structural description of conditional-probability equations.The framework represents states as probability distributions and channels as probabilistic maps, with copying and discarding used to express the equations.
- Scope: The paper develops a graphical treatment of disintegration and Bayesian inversion, proves basic conditional-probability results, and illustrates the framework with examples.Its organization includes graphical definitions, naive Bayesian classification, conditional independence, Bayesian networks, and likelihood formulations.
- Measure-theoretic setting: Measure-theoretic disintegration may fail to exist, be determined only up to negligible sets, or lack continuity or computability.The paper addresses the setting through standard Borel spaces and a likelihood-based generalization.
- Core notions: Bayesian inversion turns a channel with a state into a channel in the opposite direction, and the paper relates it to disintegration in both directions.The paper starts from disintegration, while noting that Bayesian inversion could instead be taken as primitive.
2. Graphical language
The paper uses string diagrams as an intuitive language for reasoning about channels in symmetric monoidal categories with copying and discarding. It grounds the framework in discrete and measure-theoretic probability via the Kleisli categories of the distribution and Giry monads.
- Categorical structure: Channels are modeled as arrows in a symmetric monoidal category, with sequential and parallel composition, a unit type, and swap isomorphisms.The framework treats a channel as a stochastic process and a state as a channel with trivial input.
- Graphical language: String diagrams represent types as wires and channels as boxes, making complex channel formulas more convenient and intuitive to reason about.Composition is represented by connecting wires, while parallel composition is represented by juxtaposition.
- Graphical language: The coherence theorem states that well-formed equations of composites follow from the symmetric monoidal axioms exactly when their string diagrams are isomorphic.This justifies diagrammatic reasoning as an equivalent representation of the categorical equations.
- Copying and discarding: Each type has copying and discarding maps forming compatible commutative comonoid structures; categories with this structure are called CD-categories.CD stands for Copy/Discard, and causal arrows are the channels used in the paper’s affine setting.
- Probability models: Discrete probability is represented by the Kleisli category of the distribution monad, where Kleisli maps are channels and states are finitely supported distributions.The category is affine because D(1) is isomorphic to 1.
- Probability models: Measure-theoretic probability is represented by the Kleisli category of the Giry monad, whose Kleisli maps are measurable probability kernels.The Giry Kleisli category is an affine CD-category, with parallel composition given by product measures.
3. Marginalisation, integration and disintegration
The paper represents marginalisation, integration, disintegration, and Bayesian inversion categorically as operations on states and channels. Disintegration reverses integration by extracting conditional channels from joint states, while Bayesian inversion handles the opposite direction and is equivalent to disintegration.
- Marginalisation: Marginals of a joint state are obtained by composing it with projection maps, or equivalently by discarding the other component.
- Integration: A state and a channel integrate into a joint state according to the product rule, with the discrete form ω(x,y)=c(x)(y)·σ(x).
- Disintegration: Disintegration reverses integration by extracting a marginal state and a conditional channel from a joint state.
- Disintegration: In discrete probability, a disintegration always exists and equals conditional probability P(y|x)=P(x,y)/P(x), with arbitrary values allowed when the marginal is zero.
- Disintegration: Measure-theoretic disintegration is ordinary regular conditional probability, but it may not exist without additional assumptions.
- Bayesian inversion: Bayesian inversion disintegrates the joint state generated by a state and channel in the opposite direction, and the two notions are equivalent.
- Existence conditions: For measure-theoretic probability, disintegration exists when the conditioned space is standard Borel, yielding disintegration in the stated subcategory.
4. Example: naive Bayesian classifiers via inversion
The example applies disintegration to weather-table data to extract feature channels, then uses Bayesian inversion to classify whether play occurs. For Sunny, Cold, High humidity, and Windy conditions, the resulting play probability is 0.205, matching the published 20.5% calculation.
- Approach: Naive Bayesian classification assumes independent features and uses disintegration to extract channels before Bayesian inversion produces the classification distribution.The example applies this channel-based construction to a standard weather-and-play dataset.
- Channel construction: The weather table is represented as a uniform distribution over its 14 rows, with channels extracted from the Play column to Outlook, Temperature, Humidity, and Windy.Marginalisation and disintegration select the relevant columns and produce four feature channels.
- Channel construction: The four feature channels are combined into a single channel from Play to the product of Outlook, Temperature, Humidity, and Windy.This tupled channel is the object subsequently inverted.
- Classification: The inversion channel takes Sunny, Cold, High humidity, and true Windy conditions as input to return the probability of play.The query conditions are supplied to the channel obtained by Bayesian inversion with respect to the Play marginal.
5. Almost equality of channels
The paper formalizes almost-everywhere equality for channels and uses it to relate couplings, disintegrations, and Bayesian inversions. In measure-theoretic settings, equality is expressed through agreement of measurable evaluations almost everywhere, with a stronger pointwise-kernel form under countable generation.
- Definition and role: Almost equality is introduced to express the standard notion of equality up to negligible sets in string diagrams and to characterize Bayesian inversion.The construction follows the paper’s affine CD-category setting.
- Discrete and measure-theoretic cases: For discrete kernels, two channels are almost equal under a state exactly when they agree at every input having nonzero state probability.Agreement at zero-probability inputs is not required.
- Discrete and measure-theoretic cases: For measure-theoretic kernels, almost equality means that measurable-set evaluations agree almost everywhere with respect to the input state.The condition is stated for every measurable set in the output space.
- Discrete and measure-theoretic cases: If the output measurable space is countably generated, evaluation-wise almost-everywhere equality is equivalent to equality of the kernels almost everywhere.A countable generating family supports the equivalence.
- Couplings and inversion: When disintegration exists, couplings of two states correspond bijectively to channels modulo almost equality, and the correspondence sends a channel to its Bayesian inversion.This yields the Bayesian inversion correspondence and supports a dagger functor on a suitable quotient.
Strong almost equality
The paper strengthens almost-everywhere equality so that it remains valid after coupling a channel’s input with arbitrary additional systems. Disintegration implies this strengthening, while the measure-theoretic kernel category has it even though it lacks disintegration.
- Definition: Strong almost equality requires two channels to remain equal after composing them with any joint state whose first marginal is the reference state.It is defined as a stability condition under arbitrary extensions of the input.
- Relationship: Strong almost equality implies ordinary almost equality by choosing the reference state itself as the joint state.The paper obtains the implication through a canonical extension.
- Relationship: In an affine CD-category admitting disintegration, ordinary almost equality entails strong almost equality.This property is called equality strengthening.
- Measure-theoretic setting: The category Kℓ(G) does not admit disintegration but does admit equality strengthening.Thus the strengthening result can hold independently of disintegration in this measure-theoretic setting.
6. Conditional independence
The paper extends its string-diagrammatic treatment to multipartite states and formulates conditional independence through disintegrations and almost equality. It proves equivalence with standard conditional-probability formulations and derives the graphoid axioms.
- Multipartite states: Multipartite disintegrations are defined by marginalising and permuting an n-partite state, then disintegrating selected input and output wire groups.The notation treats individual wires and tensor products of wires uniformly.
- Multipartite states: Disintegrations may be nonunique, but the paper treats them as unique up to almost equality, which is sufficient for its purposes.A chosen disintegration is used when notation requires one.
- Conditional independence: Conditional independence is defined with respect to the fixed underlying joint state and agrees with the usual condition P(x, y|z) = P(x|z)P(y|z) in discrete probability.The measure-theoretic formulation uses the corresponding factorisation for almost all conditioning values.
- Equivalent formulations: The graphical formulation makes several conditional-independence characterizations equivalent, including conditional-channel and joint-state factorization forms.The paper presents these equivalences as an abstract graphical contribution.
- Graphoid axioms: The abstract conditional-independence relation satisfies symmetry, decomposition, weak union, and contraction, the standard graphoid axioms.These properties support reasoning about conditional independence in DAGs and Bayesian networks.
7. Beyond causal channels
The section extends the graphical framework beyond causal channels, introducing effects, conditioning, and almost-causal disintegrations. It relates conditioning of joint states to forward and backward inference through extracted channels.
- Beyond causal channels: Non-affine CD-categories add scalars and effects while retaining the earlier arguments in the causal subcategory.The causal subcategory contains the original causal arrows, so the extension preserves the preceding framework there.
- Conditioning: Effects represent observables and condition states by normalization when their validity is invertible.This generalizes conditional probability given an event.
- Discrete and measure-theoretic models: In discrete probability, multisets provide unnormalized distributions whose affine submonad recovers the distribution monad.Causality in the multiset Kleisli category corresponds to total weight one, yielding the distribution Kleisli category.
- Discrete and measure-theoretic models: In measure-theoretic probability, s-finite kernels form a CD-category whose causal subcategory is the Kleisli category of the Giry monad.States are probability measures, and effects are measurable nonnegative functions integrated against those measures.
- Almost causality: Disintegrations of causal states may use non-causal arrows, but the resulting maps are almost causal relative to the relevant marginal.In sfKrn, an almost-causal kernel agrees almost everywhere with a probability kernel, so existence matches existence in the causal subcategory.
- Conditioning and disintegration: Theorem 7.6 identifies crossover influence from conditioning with transformations along channels extracted by disintegration.This gives equivalent state-update routes through effects, disintegrated channels, and their forward or backward transformations.
Disease and mood example
The disease-and-mood example applies three equivalent graphical update procedures to a correlated joint state. A positive disease test lowers the mood probability, and each route yields the same posterior.
- Setup: A disease test with 90% sensitivity and 5% false-positive rate is applied to a joint disease–mood distribution.The disease and mood variables are correlated, so evidence about disease affects the mood marginal.
- Setup: The prior marginals assign mood probability 0.45 and disease probability 0.55.The corresponding complementary probabilities are 0.55 for bad mood and 0.45 for no disease.
- Posterior update: A positive test reduces the mood probability from 0.45 to 0.126.This is the posterior obtained by updating the joint state with the positive-test predicate and then taking the mood marginal.
- Posterior update: Disintegration extracts channels from the joint state, allowing the positive-test predicate to be transformed onto the mood variable.Updating the mood marginal with this transformed predicate produces the same posterior as direct joint-state conditioning.
- Equivalent routes: Updating the disease marginal first and then transforming along the reverse disintegration channel gives the same updated mood.The example therefore demonstrates three equivalent calculation routes for the posterior mood.
8. Disintegration via likelihoods
The section develops Bayesian inversion from likelihood representations in the graphical framework. It proves that an almost-inverse likelihood yields a Bayesian inversion and illustrates the construction in s-finite kernels and a hybrid classification example.
- Likelihood formulation: Likelihood relations represent channels relative to a reference arrow, generalizing density-based Bayesian inversion beyond a fixed reference measure.In sfKrn, the likelihood is a measurable function and the reference arrow is a measure, not necessarily Lebesgue measure.
- Main theorem: Theorem 8.3 states that, under equality strengthening for almost-causal maps, an almost-inverse of the induced effect produces a Bayesian inversion.The resulting channel satisfies the defining Bayesian-inversion equation.
- Measure-theoretic instantiation: In sfKrn, the construction uses a likelihood function and measure satisfying integral normalization, then forms an almost-inverse effect.The resulting inversion is presented as a variant of Bayes’ formula.
- Hybrid example: The hybrid naive-Bayes example combines discrete channels with Gaussian temperature and humidity channels in a single Giry channel.The continuous channels use means and standard deviations computed separately for the two Play outcomes.
- Hybrid example: 0.207 is the computed probability of playing for Sunny, temperature 66, humidity 90%, and true windiness, versus 20.8% in the source calculation.The difference of 0.001 is attributed to intermediate rounding errors.