PaperlayerView exact revision on arXiv

Source-linked AI summary

I Know What I Don't Know: Latent Posterior Factor Models for Multi-Evidence Probabilistic Reasoning

Aliyu Agboola Alege

arXiv:2603.15670v2cs.AIcs.LG

TL;DR

The paper addresses multi-evidence reasoning when evidence is noisy or contradictory and existing methods provide limited uncertainty handling or structured aggregation. LPF transforms VAE posteriors into soft likelihood factors and supports both SPN-based and learned aggregation; across eight domains, LPF-SPN reports high accuracy and low calibration error. The framework provides a shared uncertainty representation for comparing structured and learned reasoning, while its current evaluation includes an architectural limitation concerning independence assumptions.

  • Problem

    Existing approaches do not combine neural perception of unstructured evidence with structured probabilistic reasoning under uncertainty for multi-evidence aggregation.

  • Method

    LPF converts VAE posterior uncertainty into soft likelihood factors for SPN inference, with LPF-SPN and LPF-Learned providing structured and end-to-end learned aggregation variants.

  • Results

    Across eight domains, LPF-SPN substantially outperformed the evaluated baselines, reaching 97.8% accuracy and 1.4% ECE in reported results.

  • Takeaways & Limitations

    The dual architectures enable a controlled comparison of structured probabilistic reasoning and learned aggregation under identical evidence encoding.

  • Takeaways & Limitations

    The independence assumption provides a 2–6% advantage broadly, while LPF-Learned models consistency at the cost of interpretability.

Abstract

from arXiv · show

Real-world decision-making, from tax compliance assessment to medical diagnosis, requires aggregating multiple noisy and potentially contradictory evidence sources. Existing approaches either lack explicit uncertainty quantification (neural aggregation methods) or rely on manually engineered discrete predicates (probabilistic logic frameworks), limiting scalability to unstructured data. We introduce Latent Posterior Factors (LPF), a framework that transforms Variational Autoencoder (VAE) latent posteriors into soft likelihood factors for Sum-Product Network (SPN) inference, enabling tractable probabilistic reasoning over unstructured evidence while preserving calibrated uncertainty estimates. We instantiate LPF as LPF-SPN (structured factor-based inference) and LPF-Learned (end-to-end learned aggregation), enabling a principled comparison between explicit probabilistic reasoning and learned aggregation under a shared uncertainty representation. Across eight domains (seven synthetic and the FEVER benchmark), LPF-SPN achieves high accuracy (up to 97.8%), low calibration error (ECE 1.4%), and strong probabilistic fit, substantially outperforming evidential deep learning, LLMs and graph-based baselines over 15 random seeds. Contributions: (1) A framework bridging latent uncertainty representations with structured probabilistic reasoning. (2) Dual architectures enabling controlled comparison of reasoning paradigms. (3) Reproducible training methodology with seed selection. (4) Evaluation against EDL, BERT, R-GCN, and large language model baselines. (5) Cross-domain validation. (6) Formal guarantees in a companion paper.

14 Future Work

The paper frames multi-evidence reasoning as aggregation of noisy, contradictory, variable-quality, and incomplete evidence, then introduces LPF as a bridge from VAE uncertainty to probabilistic reasoning. Its experiments compare structured and learned aggregation, reporting strong accuracy and calibration results across multiple domains and baselines.

  • 14 Future Work: LPF addresses the gap between neural perception of unstructured evidence and structured probabilistic reasoning under uncertainty.The framework converts VAE posteriors into soft factors for downstream reasoning.
  • 14 Future Work: LPF-SPN converts evidence posteriors into soft factors and performs exact marginal inference in a Sum-Product Network.LPF-Learned instead computes quality and consistency scores, learns aggregation weights, and decodes once.
  • 14 Future Work: 97.8% accuracy and 1.4% ECE were reported for LPF-SPN, compared with 56.3% accuracy for EDL-Aggregated and 94.1% accuracy for BERT.The evaluation included eight domains, 15 random seeds, and baselines including EDL, BERT, R-GCN, and large language models.

4 Method: Latent Posterior Factors (LPF)

LPF bridges neural evidence representation and structured probabilistic reasoning by converting VAE posteriors into soft factors for SPN inference, while also supporting learned aggregation. The framework retrieves and encodes evidence, quantifies uncertainty, and preserves provenance for downstream decisions.

  • Problem formulation: The system represents predicates over categorical domains, evidence sets, posterior distributions, top predictions, and provenance metadata.Provenance includes evidence IDs and factor weights.
  • Evidence retrieval: A hybrid evidence index combines exact entity-predicate lookup with optional FAISS semantic reranking to retrieve relevant evidence.Exact lookup prioritizes precision, while semantic search helps surface paraphrases and conceptually related evidence.
  • Latent encoding: The VAE encoder maps 384-dimensional evidence embeddings into lower-dimensional latent posterior distributions that capture semantic content and uncertainty.The encoder uses diagonal Gaussian parameters, balancing uncertainty representation with computational efficiency.
  • Aggregation variants: LPF-SPN performs exact probabilistic reasoning with soft factors, whereas LPF-Learned uses learned aggregation and trades some interpretability for deployment simplicity.LPF-SPN requires explicit SPN structure and can be computationally intensive when aggregating many evidence items.
  • Aggregation variants: LPF-Learned can reduce decoder computation to one call for N evidence items, yielding a 10× speedup when N = 10.The method learns weighting strategies from entity-level supervision without requiring evidence-level labels or hand-crafted aggregation rules.

4. Compute loss and backpropagate

The training procedure learns evidence representations and aggregation behavior while using staged optimization and systematic seed selection. A worked example traces the inference setting, evidence encoding, and uncertainty-derived confidence values.

  • Training objectives: The encoder is trained with reconstruction loss and KL regularization to preserve a structured latent representation that generalizes across evidence.The KL weight is β = 0.01, and training uses Adam with early stopping.
  • Learned aggregation: LPF-Learned computes quality, pairwise consistency, and aggregation weights before combining posterior means in latent space.The aggregated latent code is decoded once, with gradients flowing through the decoder and weighting networks.
  • Staged training: LPF-Learned trains only the aggregator networks using frozen encoder posteriors from the first stage.Freezing the encoder and decoder separates evidence interpretation from evidence combination.
  • Reproducibility: Systematic seed search selects the best validation-accuracy initialization and records accuracy, loss, and convergence information for each seed.The procedure is intended to balance computational cost with statistical rigor.
  • Worked example: In the worked example, LPF processes a tax-compliance query for Company C0001 after the canonical fast path is unavailable.The inference path retrieves five evidence items and encodes each into a 64-dimensional latent posterior.
  • Worked example: Lower posterior variance produces higher confidence scores, reflecting the VAE’s estimate of evidence quality.For example, the illustrative evidence confidence values range from approximately 0.85 to 0.93.

A. Monte Carlo Sampling (Reparameterization Trick)

Monte Carlo decoding samples latent representations from VAE posteriors, decodes each sample, averages the resulting predicate distributions, and applies calibration-aware weighting. The comparison shows distinct confidence and efficiency trade-offs between LPF-SPN and LPF-Learned.

  • Monte Carlo decoding: 16 samples are drawn from the VAE posterior, decoded into predicate distributions, and averaged to estimate the expected distribution.The reparameterization trick supports differentiable sampling and the resulting estimator converges as M →∞.
  • Uncertainty weighting: The credibility weight down-weights uncertain evidence through a sigmoid penalty controlled by α.Higher α more aggressively down-weights uncertain evidence, especially when evidence conflicts.
  • Practical settings: M ∈[16, 32] is recommended for production because accuracy gains beyond M = 32 are below 0.5% while latency increases linearly.With M = 16, the reported standard error is approximately 0.125 and is described as acceptable for normalized probabilistic reasoning.
  • Comparison: LPF-SPN achieved ECE: 0.014 compared with LPF-Learned’s ECE: 0.066 in the reported comparison.The SPN variant’s product of factors compounds agreement, while learned averaging is more conservative.

6 Algorithms

The algorithms implement two LPF variants: LPF-SPN converts latent posteriors into soft factors for tractable SPN inference, while LPF-Learned scores and combines evidence in latent space. The orchestration pipeline retrieves evidence, encodes uncertainty, performs variant-specific inference, and returns confidence with provenance.

  • LPF-SPN: Latent-to-factor conversion samples each posterior, decodes samples into predicate distributions, averages them, and applies temperature scaling.The resulting factor also receives an uncertainty-aware credibility weight.
  • Evidence encoding: Posterior encoding retrieves embeddings when available, otherwise encodes raw evidence, then outputs Gaussian parameters and confidence.Confidence is computed as 1.0 / (1.0 + mean(sigma)), so lower variance yields higher confidence.
  • LPF-Learned: LPF-Learned evaluates posterior quality and pairwise consistency, then uses normalized weights for latent-space aggregation before decoding.Quality, consistency, and weight networks perform the learned aggregation.
  • SPN inference: LPF-SPN attaches soft likelihood factors to SPN variables, computes a marginal query, and normalizes the resulting posterior.The method is intended to provide tractable probabilistic inference over Sum-Product Networks with soft factors.
  • Orchestration: The main orchestrator checks canonical facts, retrieves and encodes evidence, selects SPN or learned inference, computes confidence, and logs provenance.It returns structured results with an audit trail.
  • Complexity: LPF-SPN has complexity O(K ×M ×|domain|) for factor conversion plus O(|V |×|domain|2) for SPN inference, while LPF-Learned has O(K2 × D) consistency cost.These expressions describe the stated computational complexity of the two variants.
  • Supporting procedures: Uncertainty penalties use weight <- 1 / (1 + exp(alpha * mean(sigma))), and probability outputs are normalized with L1 normalization and small epsilon values.The parameter α controls uncertainty-penalty strength, while numerical stability uses log-space computations where appropriate.

7 System Architecture

LPF’s architecture processes retrieved evidence through VAE posteriors and then either converts those posteriors into soft factors for SPN inference or aggregates them neurally. The LPF-SPN path supports exact marginal inference over structured networks, while the learned path operates directly in latent space.

  • System overview: LPF comprises seven core components coordinated by a query handler that routes requests through the appropriate pipeline.The system includes retrieval, encoding, factor conversion or learned aggregation, inference, and provenance components.
  • Evidence and retrieval: The Canonical Database can return fresh authoritative facts immediately, bypassing inference with sub-millisecond response times.Entries typically use a configurable 30-day staleness threshold and include timestamps and confidence scores.
  • Evidence and retrieval: The Evidence Index combines relational entity-predicate lookup with FAISS semantic search to rank evidence by recency, credibility, and relevance.It returns the top-k relevant evidence items for a query.
  • Latent representation: The VAE Encoder maps 384-dimensional Sentence-BERT embeddings to 64-dimensional diagonal-Gaussian posteriors and derives confidence from mean uncertainty.The confidence formula is confidence = 1/(1 + mean(σ)).
  • LPF-SPN inference: LPF-SPN converts each latent posterior into a soft factor by Monte Carlo decoding, averages the resulting distributions, and applies an uncertainty-sensitive credibility weight.The factor converter typically uses M = 16 samples and approximates the posterior-to-potential integral.
  • LPF-Learned inference: LPF-Learned uses quality, consistency, and weight networks to aggregate posteriors in latent space before decoding once.This replaces structured SPN reasoning with learned neural aggregation.
  • LPF-SPN inference: LPF-SPN attaches weighted soft likelihood nodes to predicate-specific SPNs, combining learned evidence factors with hard conditional evidence for exact inference.The SPN uses leaves for priors, product nodes for factorization, and sum nodes for mixtures.

D. Normalization:

The query pipeline normalizes inferred distributions, selects the highest-probability value, and records the result with evidence and factor metadata. LPF-SPN emphasizes calibrated, auditable reasoning, whereas LPF-Learned offers a simpler learned alternative with competitive accuracy.

  • Normalization and output: The normalized LPF-SPN example returns low 0.018, medium 0.052, and high 0.930, selecting high with confidence 0.930.The example reports 3.3 ms execution time.
  • Normalization and output: The LPF-SPN prediction matches the high ground-truth label.The example is marked correct.
  • Normalization and provenance: The inference record stores the posterior distribution, selected value, confidence, evidence chain, factor metadata, and model versions.The recorded example identifies the evidence IDs and associated factor weights.
  • Variant comparison: The learned aggregator forms a weighted latent mean, concatenates a predicate embedding, and decodes the result into a three-class distribution.The pipeline decodes the aggregated latent representation only after weighting the posteriors.
  • Variant comparison: LPF-SPN is presented as the more calibrated and interpretable option, while LPF-Learned trades transparency for simpler end-to-end aggregation.LPF-SPN is associated with high-stakes, auditable use cases; LPF-Learned prioritizes simplicity.
  • Variant comparison: LPF-SPN produces 97.8% accuracy in the reported comparison, while LPF-Learned produces 91.1% accuracy.The corresponding examples report 14.8 ms and 37.4 ms, respectively.

8 Training Methodology

LPF trains its encoder and decoder jointly on evidence-level labels, uses KL regularization and reparameterized latent sampling, and evaluates robustness across random seeds and domains. Results report stable training and broad cross-domain performance, alongside comparisons showing advantages over several baselines.

  • Training procedure: The VAE encoder and conditional decoder are jointly trained on evidence-level inputs and ground-truth labels, avoiding entity-level aggregation during training.Inputs include Sentence-BERT evidence embeddings and a predicate such as compliance_level.
  • Training procedure: The objective combines classification loss with KL regularization using β = 0.01, and samples latent codes through reparameterization.The latent code is z = µ + σ ⊙ϵ with ϵ ∼ N(0, I).
  • Reproducibility: The seed-search protocol reports mean ± standard deviation across seeds to reduce dependence on favorable random initializations.The paper explicitly frames this as protection against cherry-picked results.
  • Training results: 82.3±0.3% training accuracy indicates low-variance optimization, while 5 out of 7 seeds converged before epoch 20.Seed 2024 achieved 86.0% validation accuracy and 0.726 validation loss.
  • Cross-domain evaluation: 83.6% to 99.9% validation accuracy across domains shows variation from legal reasoning to FEVER fact verification.FEVER is reported as the easiest domain and legal as the hardest.
  • Calibration: LPF-SPN reports ECE 1.4% versus BERT’s 12.1%, supporting the paper’s claim of superior calibration.The paper attributes this calibration to its uncertainty-aware design rather than post-hoc correction.
  • Baseline comparisons: 97.8% accuracy for LPF-SPN exceeds R-GCN by 26.3 percentage points and BERT by 3.7 percentage points across seven diverse domains.The reported average improvement over the best baselines is +2.4%.
  • LLM comparison: On FEVER, LPF-SPN reaches 99.7% accuracy and ECE 1.2%, while the compared LLMs report 44.0%–62.0% accuracy and ECE 74–87%.The reported LLM inference times are 1500–3000 ms versus LPF’s 25 ms.

2. Monte Carlo integration: Explicitly marginalizes over latent uncertainty

LPF bridges neural uncertainty representations and structured probabilistic reasoning by converting VAE posteriors into soft factors. Its structured and learned variants support calibrated, auditable multi-evidence inference across domains.

  • Calibration: LPF achieves ECE 1.4% without temperature scaling, while BERT requires tuning to reach ECE 8.9%.The reported calibration gap remains 6× in favor of LPF.
  • Auditability: LPF-SPN provides mathematically explicit factor-level contributions, enabling architectural provenance rather than post-hoc attention-based explanations.The provenance mechanism supports probabilistic reasoning traces and uncertainty decomposition.
  • Architecture: LPF converts continuous VAE posterior distributions into discrete probabilistic factors through Monte Carlo sampling for structured reasoning.The conversion bridges neural representation learning with symbolic probabilistic inference.
  • Cross-domain validation: Across eight domains, LPF reports 94.6% average accuracy, +2.4% improvement over best baselines, and 3.5% average ECE versus 5.0% for EDL.Reported domains include seven synthetic settings and FEVER.
  • Efficiency and data: LPF-SPN achieves 3.3ms inference, 13.6× faster than BERT’s 45ms, while LPF achieves ECE 1.4% with 630 training entities.These results address computational and data-efficiency gaps described for neural aggregation methods.

11 Experimental Design

The evaluation tests LPF against neural, probabilistic, graph, and language-model baselines across a primary compliance task and additional domains. The design also specifies data splits, research questions, and reproducibility controls.

  • Research questions: The experiments ask whether LPF outperforms baselines, which LPF variant is strongest, and whether performance generalizes across domains and degraded evidence.Robustness and cross-domain generalization are identified as accompanying-paper questions.
  • Primary evaluation domain: The primary domain assesses tax compliance risk from multiple evidence sources because it requires uncertainty quantification and auditable reasoning.Each company has five evidence pieces covering different compliance aspects.
  • Data: The compliance dataset contains 4,500 evidence items, split across 630 training, 135 validation, and 135 test companies.The task predicts compliance_level with values low, medium, and high.
  • Cross-domain evaluation: Seven additional domains are used to validate cross-domain generalization beyond compliance.The listed domains include academic, construction, finance, healthcare, legal, and materials settings.

7. FEVER Fact Verification (Real-World Benchmark)

The FEVER evaluation maps claim-verification labels to a three-level compliance predicate and uses a large claim dataset with entity-based stratified splitting. It compares LPF variants with neural, probabilistic, graph, and zero-shot LLM baselines under defined metrics and resource constraints.

  • Dataset and task: FEVER maps SUPPORTS, REFUTES, and NOT ENOUGH INFO to compliance_level values low, medium, and high.The dataset contains 145K training claims, 19K validation claims, and 1,800 test samples.
  • Data splitting: Entity-based stratified splitting prevents evidence leakage, preserves label balance, and avoids temporal leakage in multi-year data.All evidence for an entity appears in only one split.
  • Metrics: The study reports accuracy, Macro F1, Weighted F1, Brier Score, runtime, throughput, confidence statistics, and selective classification.Runtime is defined as average inference time per query.
  • Compared systems: LPF-SPN combines FactorConverter with SPN reasoning, whereas LPF-Learned combines FactorConverter with learned neural aggregation without an SPN.The learned aggregator uses the same encoding hyperparameters and is trained for 30 epochs.
  • Baselines: The evaluation includes VAE-Only, BERT, SPN-Only, EDL-Aggregated, EDL-Individual, R-GCN, zero-shot Groq-hosted LLMs, and an oracle baseline.LLM evaluation is limited to 50 test samples per model for API cost control.
  • Execution: LPF-SPN reports 68 queries/second in the compliance-domain evaluation, with experiments run under deterministic settings for reproducibility.The stated hardware setup does not require a GPU.

12 Results

LPF-SPN delivers strong accuracy, calibration, stability, and latency across multi-evidence benchmarks, generally outperforming neural, evidential, graph, and language-model baselines. Ablations show that evidence quantity, temperature, penalty strength, and top-k selection create distinct accuracy–calibration–runtime trade-offs.

  • Overall performance: 97.8% accuracy with ECE 1.4% and 14.8ms runtime establishes LPF-SPN as the strongest overall configuration.The reported combination links structured probabilistic reasoning with uncertainty-aware factor conversion.
  • Baseline comparisons: 54.8% absolute accuracy separates LPF-SPN from EDL-Aggregated, while LPF-Learned reaches 91.1% accuracy with 6.6% ECE.LPF-Learned remains viable but trails LPF-SPN by 6.7 percentage points in accuracy and 5.2 points in ECE.
  • Baseline comparisons: 98.0% accuracy makes Qwen3-32B competitive, but its ECE is 79.7% and latency is 3008.6ms versus LPF-SPN.BERT is also slower at 134.7ms and has 3.2% ECE, while R-GCN reaches only 15.6% accuracy.
  • Stability: 99.7% mean accuracy and 0.0% accuracy standard deviation across seeds indicate highly stable performance.Calibration varies more than accuracy, with ECE standard deviation 0.1% around a 1.0% mean and a 95% accuracy interval of ±0.1%.
  • Cross-domain performance: LPF-SPN is best or tied-best in 7/8 domains, including 99.7% accuracy on FEVER and 99.3% accuracy with 0.6% ECE in healthcare.The same hyperparameters work across domains, while LPF-Learned achieves 98.0% mean accuracy as an alternative.
  • Ablation studies: 17.7% absolute accuracy separates top_k=1 from top_k=5, after which gains diminish while additional evidence can increase runtime.At top_k=5, accuracy reaches 97.0%; top_k=20 maintains 97.8% with increased runtime, motivating top_k=5 as the accuracy-latency choice.

13 Discussion

LPF’s cross-domain evaluation links its strong performance to preserving probabilistic semantics and explicitly handling multiple uncertain evidence sources. LPF-SPN generally offers better accuracy, calibration, speed, and interpretability, while its advantages depend on domain characteristics and independence assumptions.

  • 94.6% mean accuracy and 3.5% mean ECE across eight domains demonstrate strong overall LPF performance.
  • 97.8% versus 94.8% accuracy and 1.4% versus 30.9% ECE show that removing soft factors degrades both accuracy and calibration.
  • LPF-SPN achieves a 2.4% improvement over the best baselines across seven diverse domains, indicating broad cross-domain generalization.
  • 97.8% versus 91.1% compliance accuracy and 99.3% versus 98.0% mean cross-domain accuracy favor LPF-SPN through exact probabilistic inference.
  • 1.4% versus 6.6% ECE favors LPF-SPN, whose exact SPN marginals provide more reliable calibration than learned aggregation.
  • 14.8ms versus 37.4ms latency favors LPF-SPN despite 80 decoder calls, because cached SPN structures and batch decoding reduce inference overhead.
  • LPF is a poor fit for single-input classification, massive-data settings, sub-1ms latency requirements, or trivial evidence collection.

14 Future Work

Future work extends LPF beyond categorical, passive, text-only, and single-query settings while preserving uncertainty quantification and provenance. Proposed directions include continuous prediction, active evidence selection, explanations, multi-hop reasoning, temporal weighting, multimodal inputs, and interactive use.

  • 14.1.1 Continuous Predicate Outputs: Continuous predicate outputs would replace categorical decoding with Gaussian mixtures and continuous Gaussian SPN leaves.
  • 14.1.1 Continuous Predicate Outputs: Continuous distributions would preserve non-discretized information, uncertainty quantification, and regression capability, but introduce training and evaluation challenges.
  • 14.1.2 Active Evidence Collection: Active learning would select evidence with the highest expected reduction in posterior entropy rather than passively retrieving top-k items.
  • 14.1.2 Active Evidence Collection: 97% accuracy with 3–4 evidence items versus five baseline items is the expected active-selection outcome, reducing retrieval cost by 20–40%.
  • 14.1.3 Contrastive Explanations: Contrastive explanations would identify evidence distinguishing the predicted class from the next-most-likely alternative and present it in natural language.
  • 14.2.1 Multi-Hop Reasoning and Chained Inference: Multi-hop reasoning would recursively resolve dependent predicates, using intermediate predictions as conditioning factors for downstream inference.
  • Multimodal extension would encode text, images, tables, and structured data into a shared latent space, with alignment as the central challenge.
  • Interactive deployment would let users challenge predictions, provide clarifications, and correct interpretations in real time.

A Complete Training Results

The appendix reports comprehensive training results across eight domains, examining seed variability, generalization, convergence, and loss composition. LPF-SPN shows strong cross-domain performance, with domain difficulty and evidence structure affecting accuracy and stability.

  • 99.9% validation accuracy is achieved on FEVER, while Legal is the most challenging domain at 83.6%.
  • 0.0% variance on FEVER contrasts with 0.5% on Materials, indicating domain differences in sensitivity to initialization.
  • Healthcare and Legal show small positive generalization gaps, suggesting slight overfitting, whereas most domains generalize well.
  • Healthcare and Finance reach 100% convergence before the maximum epochs, while FEVER rarely early-stops because of its large dataset.
  • 94–99% of validation loss comes from cross-entropy, while weighted KL divergence contributes 1–6% with β = 0.01.

B.2 Academic Domain

The academic-domain experiments evaluate grant approval prediction and related structured-evidence tasks across model comparisons, uncertainty outputs, and ablations. LPF variants achieve perfect classification on the reported academic comparison and produce highly concentrated predictive distributions.

  • The grant-approval task predicts likely_reject, possible, or likely_accept from investigator, proposal, and institution features.
  • 100% accuracy is achieved by both LPF-SPN and LPF-Learned on the academic model comparison.
  • 0 errors are reported for the grant-approval test set, corresponding to 100% accuracy.
  • The FEVER evaluation contains 5 errors among 1,800 samples, a 0.28% error rate.

B.5 Finance Domain

The finance task assesses credit default risk from borrower credit history, debt ratios, and financial behavior across low-, medium-, and high-risk classes. Reported examples correctly identify both high-risk and low-risk cases with highly concentrated predictive distributions.

  • Credit default risk is classified into low_risk, medium_risk, and high_risk using borrower credit history, debt ratios, and financial behavior.
  • Ablations: Finance-domain ablations examine Monte Carlo sample count, temperature scaling, uncertainty penalty α, and evidence count top_k.
  • Model evaluation: Model comparison and error-count tables are reported for the finance domain using seed 123.
  • Examples: high_risk is predicted with confidence 0.9982 for one correctly classified finance example.Its reported distribution assigns 0.9997289 to high_risk, 0.0002709 to medium_risk, and 2.0e-07 to low_risk.
  • Examples: high_risk is also correctly identified with a distribution of 0.9982, compared with 0.0016 for medium_risk and 0.0002 for low_risk.
  • Examples: A low_risk example is correctly predicted with confidence 1.0000 and probability 0.9999999887.The corresponding probabilities are 1.1e-08 for medium_risk and 5.7e-10 for high_risk.

B.8.3 Best Seed Comparison

The best-seed and ablation results examine model behavior, hyperparameter choices, reproducibility, resource use, and deployment boundaries. Across the supplied evidence, LPF-SPN combines strong performance with low calibration error and explicit operational caveats.

  • Ablation Findings: T = 0.8 and T = 1.0 are most robust across domains, while T > 1.0 degrades calibration without improving accuracy.
  • Ablation Findings: k = 5 is optimal for most domains, following a dramatic improvement from k = 1 and diminishing returns thereafter.
  • Hyperparameter Selection: The configuration (n=4, T=0.8, α=0.1, k=5) works well for 7/8 domains, with compliance benefiting from higher n_samples.
  • Limitations and Risks: LPF may amplify training-data bias, invite over-reliance, or be vulnerable to adversarially crafted high-credibility evidence.
  • Evaluation and Deployment: ECE ranges from 0.6–2.1% across domains, reported as 57–60× better than LLM baselines.
  • Evaluation and Deployment: 14.8ms average inference time is reported as enabling real-time applications.
Loading 2603.15670v2…