Theoretical Foundations of Latent Posterior Factors: Formal Guarantees for Multi-Evidence Reasoning

Aliyu Agboola Alege

Source-linked AI summary

Theoretical Foundations of Latent Posterior Factors: Formal Guarantees for Multi-Evidence Reasoning

Aliyu Agboola Alege

arXiv:2603.15674v2Mar 13, 2026cs.AIcs.ITcs.LGstat.ML

TL;DR

The paper addresses reliable confidence and robustness for prediction from multiple evidence items. It studies LPF aggregation and reports strong calibration and accuracy results, while noting scope boundaries in evidence quality and empirical baseline comparisons.

Problem
Reliable confidence matching empirical accuracy is identified as critical for decision-making in multi-evidence prediction.
Method
LPF-SPN aggregates individual evidence items while preserving their calibration within a theoretical bound.
Results
Across eight domains, LPF-SPN achieves 99.3% mean accuracy and 1.5% Expected Calibration Error.
Takeaways & Limitations
LPF provides formally motivated calibration preservation, robust operation under corrupted evidence, and epistemic-aleatoric uncertainty separation.
Takeaways & Limitations
Uniform aggregation achieves lower ECE than LPF at K = 5, while Assumption 2 constrains encoder output variance and evidence quality.

Abstract

from arXiv · show

We present a complete theoretical characterization of Latent Posterior Factors (LPF), a principled framework for aggregating multiple heterogeneous evidence items in probabilistic prediction tasks. Multi-evidence reasoning arises pervasively in high-stakes domains including healthcare diagnosis, financial risk assessment, legal case analysis, and regulatory compliance, yet existing approaches either lack formal guarantees or fail to handle multi-evidence scenarios architecturally. LPF encodes each evidence item into a Gaussian latent posterior via a variational autoencoder, converting posteriors to soft factors through Monte Carlo marginalization, and aggregating factors via exact Sum-Product Network inference (LPF-SPN) or a learned neural aggregator (LPF-Learned). We prove seven formal guarantees spanning the key desiderata for trustworthy AI: Calibration Preservation (ECE <= epsilon + C/sqrt(K_eff)); Monte Carlo Error decaying as O(1/sqrt(M)); a non-vacuous PAC-Bayes bound with train-test gap of 0.0085 at N=4200; operation within 1.12x of the information-theoretic lower bound; graceful degradation as O(epsilon*delta*sqrt(K)) under corruption, maintaining 88% performance with half of evidence adversarially replaced; O(1/sqrt(K)) calibration decay with R^2=0.849; and exact epistemic-aleatoric uncertainty decomposition with error below 0.002%. All theorems are empirically validated on controlled datasets spanning up to 4,200 training examples. Our theoretical framework establishes LPF as a foundation for trustworthy multi-evidence AI in safety-critical applications.

1 Problem Setting and Formal Framework

LPF formulates multi-evidence prediction by encoding evidence into latent posteriors, converting them into weighted soft factors, and aggregating them into calibrated, robust, data-efficient, interpretable predictions.

1.1 Multi-Evidence Prediction Problem: LPF predicts labels for an entity from K associated evidence items in a latent semantic space.The framework represents evidence meanings in Z and seeks a predictive distribution over the entity’s unknown label.
1.2 LPF Architecture: Each evidence item is independently encoded by qϕ(z|ei) into a Gaussian latent posterior whose parameters are produced by a variational autoencoder.The encoder produces posterior means µi and covariances Σi.
1.1 Multi-Evidence Prediction Problem: The target predictive distribution is designed to match empirical accuracy, tolerate corrupted evidence, and achieve target accuracy with minimal K.The problem setting explicitly identifies calibration, robustness, and data efficiency as goals.
1.1 Multi-Evidence Prediction Problem: LPF’s intended interpretation separates epistemic uncertainty from aleatoric uncertainty.This interpretability goal is part of the framework’s stated contribution.
1.2 LPF Architecture: LPF-SPN and LPF-Learned share the encoding, factor-conversion, and weighting stages but differ in aggregation.LPF-SPN performs exact Sum-Product Network marginal inference, whereas LPF-Learned aggregates in latent space before decoding.
1.2 LPF Architecture: Posterior marginalization converts each encoded item into a soft factor, which receives a confidence weight based on posterior uncertainty.The weighting function fconf is monotonically decreasing in posterior uncertainty.
2 Core Assumptions: The theoretical framework assumes conditional evidence independence, bounded encoder variance, a calibrated decoder, and valid SPN marginalization.These assumptions support the probabilistic aggregation guarantees.
2 Core Assumptions: The formal setting also assumes finite evidence support and bounded decoder probability support; the main experiments use Kmax = 5.Bounded support prevents numerical instability in product aggregation, while finite support limits each entity’s evidence count.

3 Core Theorems

LPF is supported by seven theorems covering calibration, approximation, generalization, optimality, robustness, sample complexity, and uncertainty decomposition, with empirical checks for each result.

Theorem 1: SPN Calibration Preservation: Theorem 3.1 bounds aggregated ECE by ϵ + C/√Keff when individual soft factors are ϵ-calibrated.The guarantee holds under Assumptions 1–4; experiments report individual evidence ECE 0.140 and aggregated LPF-SPN ECE 0.185.
Theorem 2: Monte Carlo Error Bounds: Theorem 3.2 establishes Monte Carlo factor error of O(1/√M), where M is the number of samples.The theorem applies to estimating the soft factor from the decoder expectation under the latent posterior.
Theorem 3: Learned Aggregator Generalization Bound: Theorem 3.3 provides a non-vacuous PAC-Bayes generalization bound for the learned aggregator.At N = 4200, the empirical train-test gap is 0.0085 against a theoretical bound of 0.228.
Theorem 4: Information-Theoretic Lower Bound: The information-theoretic verification estimates average posterior entropy at 0.158 bits, implying an 88.7% reduction from marginal entropy.The reported values are H(Y) = 1.399 bits and H̄(Y|E) = 0.158 bits.
Theorem 6: Sample Complexity and Data Efficiency: Theorem 3.6 establishes O(1/√K) calibration decay as evidence count increases, with empirical fit R2 = 0.849.The reported LPF curve plateaus at K ≈ 7; uniform aggregation has lower ECE at K = 5 but lacks LPF’s formal guarantees.
Theorem 7: Uncertainty Quantification Quality: Theorem 3.7 exactly decomposes predictive variance into epistemic and aleatoric components up to Monte Carlo error.Empirical decomposition error is below 0.002%; aleatoric variance remains approximately 0.042 across K, while epistemic variance is non-monotonic.

4 Formal Dependency Structure

The dependency structure links assumptions to lemmas, intermediate results, and seven main theorems, while different theorems rely on different assumption subsets.

The diagram presents logical dependencies from assumptions through lemmas and intermediate results to seven main theorems.Arrows encode these dependencies.
The framework lists six assumptions: conditional independence, bounded encoder variance, calibrated decoding, valid SPN marginalization, finite evidence, and bounded probability support.
Theorem 3.3 is data-dependent and does not directly rely on Assumptions A1–A6.
Theorems 3.6 and 3.7 build on Theorems 3.1, 3.2, and 3.4 rather than on their assumptions alone.

5 Implementation Alignment

The paper maps each theoretical guarantee to an implementation and empirical verification, using dedicated setups, figures, and tables across the seven theorems. Experiments cover calibration, Monte Carlo error, generalization, information-theoretic optimality, robustness, sample complexity, and uncertainty decomposition.

Experiments explicitly connect each theorem to its implementation and empirical verification.
Calibration: 0.058 aggregated ECE for LPF-Learned and 0.185 for LPF-SPN were measured in a 10-bin calibration analysis on 300 test entities.Individual evidence ECE was 0.140, and the calibration result was reported as verified with large margin.
Monte Carlo error: M ∈{4, 8, 16, 32, 64} was tested with 50 trials per configuration; mean errors were 3–10× below worst-case bounds, with error < 0.02 at M = 16.All 95th percentiles remained within theoretical bounds.
Generalization: At N = 4200, the empirical train-test gap was 0.0085 against a theoretical bound of 0.228, with test accuracy of 95.4%.The dedicated dataset used 4,200 training examples and 900 test examples; the bound was verified as non-vacuous across tested sizes.
Information-theoretic optimality: LPF-SPN achieved ECE 0.178, a 0.020 gap from the lower bound, and a 1.12× performance ratio to the achievable bound.The calculation used 100 test companies with full evidence sets.
Sample complexity: The sample-complexity study fitted ECE = 0.245/√K+0.120 with R2 = 0.849 and an empirical plateau at K ≈7.The study evaluated K ∈{1, 2, 3, 5, 7, 10, 15, 20}; uniform aggregation achieved ECE = 0.036 at K = 5 but lacked formal guarantees.
Uncertainty decomposition: Uncertainty decomposition used K ∈{1, 2, 3, 5} and 100 Monte Carlo samples per query, with mean decomposition error < 0.002%.The experiment used 50 test companies.

K Total Variance Epistemic Variance Aleatoric Variance Decomp. Error

The uncertainty analysis separates epistemic and aleatoric variance while documenting how evidence count, disagreement, encoder regularization, and aggregation affect the decomposition. The results support exact decomposition within numerical precision under empirically validated assumptions.

Decomp. Error: Mean decomposition error was < 0.002% for all K, while aleatoric variance remained approximately 0.042 across evidence counts.The reported result confirms exactness within numerical precision.
Epistemic Variance: The encoder’s KL penalty forces Σi ≈0.5I, so low epistemic uncertainty at K = 1 does not represent genuine model confidence.This phase coincides with individual evidence ECE of 0.140.
Epistemic Variance: Epistemic variance rose from 0.034 at K = 1 to 0.123 at K = 3, then decreased to 0.111 from K = 3 to K = 5.The trajectory was attributed to posterior regularization, evidence disagreement, and conflict resolution through weighted aggregation.
Epistemic Variance: Average pairwise KL was 0.317 bits, linking evidence disagreement to increased epistemic uncertainty during K = 1 →3.The passage characterizes conflicting evidence as producing high epistemic uncertainty.
Epistemic Variance: Weighted aggregation using wi = fconf(Σi) reduced epistemic uncertainty from K = 3 to K = 5 by 10%.The reduction was reported as consistent with Theorem 3.1’s prediction.
Assumptions: The calibrated decoder assumption uses individual evidence ECE = 0.140, while temperature scaling would tighten Theorem 3.1 bounds.
Assumptions: All six assumptions were empirically validated, with minor violations such as ρ = 0.12 remaining within tolerance ranges where theoretical bounds remain valid.The main experiments used Kmax = 5, while scaling studies extended to Kmax = 20 and were described as representative of 3–10 sources.
Empirical validation: LPF-SPN achieved 99.3% mean accuracy and 1.5% ECE across eight domains.The domains included compliance, healthcare, finance, legal, academic, materials, construction, and FEVER fact verification.

7 Comparison with Baselines and Related Work

LPF differs from heuristic, neural, ensemble, and evidential approaches by assigning probabilistic semantics to multi-evidence aggregation and providing formal guarantees with exact uncertainty decomposition. Its empirical comparisons show strong accuracy and calibration, while its principal value is formal reliability rather than uniformly lowest ECE.

Comparison with baselines: LPF aggregates evidence-conditioned posteriors within a shared latent space, unlike ensembles that average predictions from independently trained models.
Related work: LPF marginalizes latent explanations z given multiple evidence items, whereas Bayesian Model Averaging marginalizes over model uncertainty.
Comparison with baselines: Heuristic methods such as majority voting, max-pooling, and averaging lack probabilistic semantics, while LPF is derived from first principles with formal guarantees.
Related work: LPF’s learned aggregator has Bayesian justification and exact uncertainty decomposition, unlike attention mechanisms without an explicit probabilistic interpretation.
Theoretical advantages: LPF offers seven formal guarantees covering reliability, calibration, efficiency, interpretability, robustness, generalization, and uncertainty decomposition.
Trade-offs: LPF-SPN’s ECE of 0.185 at K = 5 exceeded LPF-Learned’s 0.058 and the baseline’s 0.036, but LPF-SPN uniquely provided formal calibration guarantees and exact uncertainty decomposition.
Related work: LPF uses one encoder-decoder and reports exact uncertainty decomposition, whereas deep ensembles require K models and approximate uncertainty through variance.

8 Limitations and Future Extensions

The paper identifies assumptions and practical limits affecting LPF’s guarantees, while proposing extensions for dependent evidence, adaptive selection, scalability, and certified robustness. It concludes that LPF’s value lies in formal reliability, uncertainty decomposition, and graceful degradation rather than uniformly superior empirical calibration.

Limitations: Conditional independence is only approximate, with average pairwise correlation ρ = 0.12; future bounds target dependency-aware aggregation.The proposed extension uses Markov Random Fields to model evidence dependencies.
Limitations: Decoder calibration degrades under distribution shift, with individual ECE = 0.140; post-hoc calibration is proposed as future work.The paper distinguishes encoder-variance assumptions from aggregator-complexity assumptions in its guarantees.
Limitations: Few-shot scenarios with N < 100 lack theoretical coverage, while non-vacuous bounds require N ≥ 1.5 × deff = 2002.The reported experiments use N = 4200, satisfying the stated condition.
Limitations: LPF requires O(K · M) decoder calls, reaching 6,400 forward passes for K = 100 and M = 64.Approximate SPN algorithms and distillation are proposed to reduce computation.
Contributions: LPF provides seven formal guarantees spanning reliability, calibration, efficiency, robustness, learnability, and interpretability.These include calibration preservation, Monte Carlo error control, PAC-Bayes generalization, near-optimality, robustness, calibration decay, and uncertainty decomposition.
Implications: Practical guidance recommends LPF-SPN when formal guarantees are essential and LPF-Learned when empirical performance dominates.The framework is presented as a foundation for trustworthy multi-evidence AI in safety-critical applications.

A Supporting Lemmas

The supporting lemmas establish unbiased Monte Carlo estimation, concentration, valid SPN closure, and effective-sample-size control. These results supply the mathematical components used to prove LPF’s calibration and approximation guarantees.

Monte Carlo lemmas: Monte Carlo samples from Gaussian latent posteriors produce an unbiased estimator of the decoder-induced soft factor.The reparameterization uses z(m) = µ + Σ1/2ϵ(m), with independent Gaussian noise samples.
Concentration: Hoeffding’s inequality bounds deviations of independent bounded variables and is used to control Monte Carlo approximation error.The application explicitly connects this lemma to Theorem 3.2.
SPN closure: Weighted sums and normalized products of valid probability distributions remain valid distributions under the stated positivity and normalization conditions.This SPN closure result supports probabilistically valid aggregation.
Effective sample size: Weighted-average concentration scales with effective sample size, capturing the reduction caused by unequal weights.The result is used to obtain calibration bounds for weighted evidence aggregation.

A.5 Lemma 5: Evidence Conflict Lower Bound

The evidence-conflict lemma formalizes an unavoidable calibration cost when evidence items disagree. Its proof connects residual uncertainty and pairwise divergence to an information-theoretic lower bound that LPF approaches.

Evidence conflict: Conflicting evidence imposes an unavoidable calibration error proportional to its average pairwise KL divergence.The lower bound holds for any aggregation method, with the constant depending on |Y|.
Generalization support: Algorithmic stability and PAC-Bayes complexity provide the ingredients for generalization bounds for the learned aggregator.The supporting lemmas use L2-regularized gradient descent, Lipschitz loss, effective dimension, and data-dependent priors.
Calibration proof: SPN aggregation computes a valid probability distribution, after which concentration under conditional independence yields a total calibration bound.The proof transfers individual decoder calibration through soft factors and combines it with a concentration term.
Calibration proof: C ≈ 2.42 for |Y| = 3 and δ = 0.05, while empirical measurements yield the tighter constant Cemp ≈ 2.0.The empirical constant suggests lower evidence variance than the worst-case bound.

B.2 Theorem 2: Monte Carlo Error Bounds

The appendix derives the Monte Carlo error bound from unbiasedness, bounded decoder outputs, Hoeffding concentration, and a union bound. It also records the sample-size condition for the separate learned-aggregator generalization result.

Proof steps: The Monte Carlo estimator is unbiased because each latent sample is independently drawn from q(z|e).This is the first step in bounding factor-approximation error.
Proof steps: Decoder probabilities lie in [0, 1], enabling Hoeffding concentration for each fixed output label.The proof then applies a union bound over all y ∈ Y.
Bound: δ = 2|Y| exp(−2Mϵ2), and solving for ϵ yields factor error that decreases as O(1/√M).The result formalizes the Monte Carlo sample-size dependence.
Generalization distinction: The learned-aggregator generalization theorem depends on algorithmic stability and effective dimension, not encoder variance.Encoder variance affects what is aggregated, whereas effective dimension controls how well the aggregator generalizes.
Generalization condition: N = 4200 > 2002 = 1.5 × 1335 satisfies the non-vacuous-bound condition N ≳ 1.5 · deff.The appendix reports this condition as holding in the experiments.
Optimality proof: LPF’s information-theoretic lower-bound proof combines residual posterior entropy with an additional noise contribution from conflicting evidence.The construction then shows LPF achieves the lower bound up to approximation terms.

B.5 Theorem 5: Robustness to Corruption

LPF's robustness analysis models adversarial evidence replacement and derives a sublinear degradation bound using SPN product stability and variance reduction.

The corruption model replaces a fraction ϵ of K evidence items, with each corrupted soft factor differing by at most δ in L1 norm.
SPN product stability bounds the aggregation change under Assumption 6, which imposes a positive lower bound on each factor value.
Conditional independence reduces the variance of the summed perturbation from K^2 scaling to K scaling.
The resulting robustness degradation scales as O(ϵδ√K), improving over the naive O(ϵδK) bound.The square-root dependence follows from concentration after variance reduction.
Theorem 3.6 derives evidence-count requirements by setting the robustness bound to a target error and solving for Keff.The proof then uses Keff ≤ K to obtain the required condition.