PaperlayerView exact revision on arXiv

Source-linked AI summary

Credal and Interval Deep Evidential Classifications

Michele Caprio, Shireen K. Manchingal, Fabio Cuzzolin

arXiv:2512.05526v1cs.LGmath.STstat.MEstat.TH

TL;DR

Reliable classification needs to represent and quantify uncertainty, including its aleatoric and epistemic components. CDEC and IDEC use predictive credal sets and intervals to estimate these uncertainties and support abstention or probabilistic label regions. Experiments report accuracy and calibration improvements, marked OoD-detection gains, and robust classification sets.

  • Problem

    Existing approaches fall short of combining theoretical grounding, fast implementation, and the ability to properly quantify and disentangle aleatoric and epistemic uncertainty.

    Source: §1 · p. 2

  • Method

    CDEC uses a predictive credal set and IDEC an interval of predictive distributions to assess uncertainty and support abstention or label regions with probabilistic guarantees.

    Source: §1 · p. 2

  • Results

    Across datasets and backbones, the approaches improve accuracy and calibration, markedly improve OoD detection across benchmark shifts, and show robust classification sets.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The methods are presented as a way to gauge AU and EU and abstain when excessive uncertainty is detected, with competitive performance in extensive experiments.

    Source: §7 · p. 22

    On CIFAR-100, IDEC's interval expansion for marginal coverage can produce excessively large epistemic uncertainty terms that dominate AU and inflate total uncertainty.

    Source: §6.3 · p. 19

Abstract

from arXiv · show

Uncertainty Quantification (UQ) presents a pivotal challenge in the field of Artificial Intelligence (AI), profoundly impacting decision-making, risk assessment and model reliability. In this paper, we introduce Credal and Interval Deep Evidential Classifications (CDEC and IDEC, respectively) as novel approaches to address UQ in classification tasks. CDEC and IDEC leverage a credal set (closed and convex set of probabilities) and an interval of evidential predictive distributions, respectively, allowing us to avoid overfitting to the training data and to systematically assess both epistemic (reducible) and aleatoric (irreducible) uncertainties. When those surpass acceptable thresholds, CDEC and IDEC have the capability to abstain from classification and flag an excess of epistemic or aleatoric uncertainty, as relevant. Conversely, within acceptable uncertainty bounds, CDEC and IDEC provide a collection of labels with robust probabilistic guarantees. CDEC and IDEC are trained using standard backpropagation and a loss function that draws from the theory of evidence. They overcome the shortcomings of previous efforts, and extend the current evidential deep learning literature. Through extensive experiments on MNIST, CIFAR-10 and CIFAR-100, together with their natural OoD shifts (F-MNIST/K-MNIST, SVHN/Intel, TinyImageNet), we show that CDEC and IDEC achieve competitive predictive accuracy, state-of-the-art OoD detection under epistemic and total uncertainty, and tight, well-calibrated prediction regions that expand reliably under distribution shift. An ablation over ensemble size further demonstrates that CDEC attains stable uncertainty estimates with only a small ensemble.

1. Introduction.

CDEC and IDEC address the difficulty of distinguishing aleatoric from epistemic uncertainty while offering fast, theoretically grounded classification methods. The paper evaluates their predictive performance, uncertainty estimates, and behavior under distribution shift.

1. Introduction.

  • AU is irreducible, whereas EU reflects a lack of knowledge that can be reduced with additional data; excessive AU may warrant a warning or abstention.A single probability measure captures AU but cannot represent the agent's uncertainty about the data-generating process.Source: §1 · pp. 1–2
  • Existing approaches fall short of combining theoretical grounding, fast implementation, and the ability to quantify and disentangle both AU and EU.The cited approaches variously incur computational overhead, focus only on EU, have uncertainty-estimation pitfalls, or rely on measures without strong theoretical justification.Source: §1 · p. 2
  • CDEC and IDEC use predictive credal sets and intervals, respectively, to gauge uncertainty, abstain when total uncertainty is excessive, and otherwise return labels with high-probability coverage.The methods are presented as addressing EU and AU explicitly, with theoretical guarantees for predicted label regions and low computational overhead.Source: §1 · p. 3
  • CDEC and IDEC improve accuracy and calibration across datasets and backbones, markedly improve OoD detection across benchmark shifts, and show robust classification sets in experiments.The ablations report that epistemic uncertainty grows with ensemble diversity, while CDEC maintains high coverage and consistent in-distribution/OoD separation across ensemble sizes.Source: §1 · p. 3

2. Background.

The background establishes the categorical and Dirichlet distributions used to represent class probabilities and their prior uncertainty. It then describes an evidential posterior-predictive procedure based on an encoder and class-specific normalizing flows.

2.1. Dirichlet and Categorical Distributions.

  • A categorical distribution represents class probabilities with a k-dimensional probability vector, while a Dirichlet prior over that vector is updated by class counts to form a posterior.The Dirichlet parameters act as prior pseudo-counts; combining the prior with the categorical likelihood yields a Dirichlet posterior.Source: §2.1 · pp. 3–4

2.2. Posterior via Normalizing Flows.

  • The evidential method of Charpentier, Zügner and Günnemann (2020) derives virtual class counts using an encoder and class-specific normalizing flows to produce a predictive distribution.The method uses a latent representation and flow-assigned probabilities to weight class counts; it is selected for benchmark task performance and uncertainty quality.Source: §2.2 · pp. 4–5
  • For strange inputs, the procedure can produce low probabilities and low posterior counts, a behavior used to detect OoD inputs.The virtual counts need not be integers.Source: §2.2 · pp. 4–5

3. Credal uncertainty modeling.

Credal uncertainty modeling represents uncertainty about the likelihood with a set of plausible distributions, which induces predictive uncertainty. The section also describes entropy-based measures and computable bounds for separating aleatoric and epistemic uncertainty.

3. Credal uncertainty modeling.

  • Different encoder and flow parameters can yield different likelihoods, motivating a set of plausible likelihoods to hedge against likelihood misspecification.The predictive credal set is derived from this likelihood set; the stated assumption is that the true data-generating process belongs to the likelihood set.Source: §3.1 · pp. 5–6

3.1. Likelihood Specification and Credal Sets.

  • CDEC constructs a finite likelihood set from multiple encoders and normalized flows, then uses its convex combinations to obtain a predictive credal set.The stated assumption is that the true likelihood belongs to the likelihood set, in which case the true predictive distribution belongs to the derived predictive set.Source: §3.1 · p. 6

3.2. Aleatoric and Epistemic uncertainties.

  • Upper entropy represents total uncertainty, while the upper- and lower-entropy framework separates AU and EU; the paper gives an operative total-uncertainty upper bound and shows AU can be computed from the predictive-set extremes.The theorem makes AU calculations immediate from the extremes and provides a computable upper bound for total uncertainty.Source: §3.2 · p. 7
  • Other measures can replace upper and lower entropy for AU and EU, provided the chosen total-uncertainty measure is bounded.The paper notes that computing the generalized Hartley measure can be extremely costly even when the label space is moderate.Source: §3.2 · p. 7

4. Credal Deep Evidential Classification.

CDEC represents predictive uncertainty with a credal set and returns a smallest label region with a lower-probability guarantee when uncertainty is acceptable. When total uncertainty is too high, it abstains and distinguishes excess aleatoric from epistemic uncertainty.

4.1. Imprecise Highest Density Regions.

  • An IHDR is the smallest label set whose lower probability is at least 1−γ; in the γ=.1 example, {y1,y2,y3} is the smallest subset reaching 0.9 for all extreme distributions.This lower-probability guarantee holds across the predictive distributions in the credal set.Source: Table 1 · p. 8, §4.1 · p. 8

4.2. Credal Deep Evidential Classification.

  • When total uncertainty is acceptable, CDEC returns a (1−γ)-IHDR; near maximum total uncertainty it abstains, flagging AU when AU is at least half the TU upper bound and EU otherwise.The abstention threshold is user-specified; the method compares upper TU with log2(k).Source: §4.2 · p. 9
  • If the number of encoders and normalizing flows S is at least the number of labels k, CDEC's upper TU is at least log2(k), so it immediately concludes uncertainty is too high.The authors note that TU is sensitive to S and leave choosing its “correct” value for future work.Source: §4.2 · p. 10

4.3. Features of Credal Deep Evidential Classification.

  • CDEC quantifies and disentangles AU and EU, and, when total uncertainty is not too high, returns the smallest label set with probability at least 1−γ.The user chooses γ, which sets the target accuracy level for the label region.Source: §4.3 · p. 10
  • CDEC can use different EDL methods and uncertainty measures, provided the selected total-uncertainty measure is bounded.The passage gives an EDL method addressing zero-evidence regions as one possible adaptation.Source: §4.3 · p. 10
  • A one-class modification returns the least implausible class when total uncertainty is acceptable, but gives up control over the label region's 1−γ accuracy level.The selected class depends on the agnostic prior beliefs and available likelihood evidence.Source: §4.3 · p. 10
  • Unlike the IDM (Bernard, 2005; Walley, 1996), CDEC uses a uniform Dirichlet prior and plausible likelihoods to avoid sensitivity to prior–data conflict.The authors describe this as accounting for possible likelihood misspecification.Source: §4.3 · p. 11

5. Interval Deep Evidential Classification.

IDEC simplifies CDEC’s Algorithm 1 by using intervals of measures in place of credal sets.

5.1. Intervals of Measures.

  • IDEC bounds each class’s predictive measure between ℓ and (1 + d)ℓ, with d representing the designer’s ambiguity about the true predictive distribution.In the classification setting, distributions in the interval are categorical.Source: §5.1 · pp. 11–12
  • For finite label sets, each such interval corresponds to a finitely generated credal set, while IDEC avoids choosing CDEC’s ensemble size S and derives only one categorical predictive distribution.The paper also describes IDEC as faster to implement and says it can select an optimal d⋆.Source: §5.1 · p. 13

5.2. Aleatoric and Epistemic Uncertainties in the Interval Setting.

  • IDEC decomposes total uncertainty into aleatoric and epistemic parts using variance rather than entropy, since its upper measure is not a probability; the properties of these terms are deferred.The authors state that entropy is not well-suited to measuring uncertainty for the positive upper measure.Source: §5.2 · p. 14

5.3. Prediction in the Interval Setting.

  • For a target coverage level, d⋆ minimizes the coverage discrepancy between the precise and interval models, while larger d raises the prediction region’s coverage and cardinality.The authors relate d⋆ to an optimal choice of CDEC’s seed count and seed values for the desired probabilistic guarantee.Source: §5.3 · p. 15

5.4. Algorithm in the Interval Setting.

  • IDEC returns an IHDR when total uncertainty is below threshold; otherwise, it abstains and flags AU or EU according to whether AU accounts for at least half of total uncertainty.The AU-to-total-uncertainty ratio is (1 + d⋆)^-2.Source: §5.4 · p. 16

6. Experiments.

Across benchmark datasets and backbones, CDEC-3 shows strong predictive performance and the strongest out-of-distribution discrimination. Its credal regions expand under distribution shift, while uncertainty estimates stabilize with a small ensemble.

6.1. Implementation.

  • The evaluation spans MNIST, CIFAR-10, and CIFAR-100, their paired OoD shifts, and Conv, VGG16, ResNet18, and ResNet50 backbones.The study examines predictive performance, uncertainty quantification, abstention, and OoD robustness.Source: §6.1 · pp. 16–17

6.2. Predictive performance.

  • Across settings, CDEC-3 consistently outperforms predictive baselines and yields the strongest out-of-distribution discrimination, particularly by epistemic or total uncertainty.Table 2 reports predictive performance and uncertainty results for ResNet-50.Source: §6.2 · p. 18, Table 2 · p. 18, §6.4 · p. 19

6.3. Uncertainty estimation.

  • CDEC-3 produces sharper uncertainty profiles and lower total uncertainty than PostNet and PostNet-3, while IDEC's CIFAR-100 interval expansion inflates epistemic and total uncertainty.CDEC-3 estimates aleatoric uncertainty from minimum entropy across credal-set extremes and epistemic uncertainty from their entropy gap.Source: §6.3 · p. 19

6.4. Out-of-distribution (OoD) detection.

  • CDEC-3 separates in-distribution from shifted data most strongly, especially using epistemic or total uncertainty; IDEC's CIFAR-100 interval inflation can reduce contrast.Credal extremes diverge as inputs move off the training manifold, amplifying epistemic spread.Source: §6.4 · p. 19

6.5. Imprecise Highest Density Region (IHDR) analysis.

  • On in-distribution data, CDEC-3 IHDRs stay near 1 on MNIST and below 7 on CIFAR-10 with near-perfect coverage; IDEC's CIFAR-100 mean sizes range from 36 to 61.CDEC-3 IHDRs expand by +3 to +7 on MNIST and CIFAR-10 and by +23 on CIFAR-100 under distribution shift; IDEC's expansion is smaller.Source: §6.5 · pp. 19–20, Table 3 · p. 20, Figure 1 · p. 20

6.6. The effect of ensemble numbers on CDEC.

  • For CDEC, S ≥3 stabilizes IHDR behavior across datasets, while S = 1 underestimates epistemic uncertainty on CIFAR-10 and CIFAR-100.Coverage stays near 1 on MNIST and CIFAR-100; on CIFAR-10 it is near-perfect for S ∈{3,7,10}.Source: Figure 2 · p. 21, §6.6 · p. 21

7. Conclusion.

CDEC and IDEC quantify aleatoric and epistemic uncertainty for classification and can abstain when uncertainty is excessive. The conclusion identifies known class count as an assumption and outlines extensions to regression, likelihood elicitation, and dependent labels.

7. Conclusion.

  • CDEC and IDEC quantify aleatoric and epistemic uncertainty in classification, supporting abstention when excessive uncertainty is detected.Their theoretical framework draws on credal sets and intervals of measures.Source: §7 · p. 22
  • The methods assume that the number of labels is known; the authors identify relaxing this assumption as future work.Appendix F sketches a high-level approach for the unknown-label setting.Source: §7 · p. 22
  • The authors plan to extend CDEC and IDEC to regression, but note that the categorical-distribution parameter relationship used in Algorithm 1 may not hold generally.They cite Jürgens et al. (2024, Theorem 3) regarding this difficulty.Source: §7 · p. 22
  • Future work aims to improve CDEC's likelihood-elicitation process through profile likelihood or LLM-generated candidate distributions.The proposed LLM approach would use a convex hull of distributions elicited from qualitative descriptions of the data-generating process.Source: §7 · p. 22
  • The authors also plan to relax the assumption that labels are conditionally i.i.d., potentially using a Lazy Naive Credal Classifier-style local training procedure.That procedure would defer training until classification and select nearby training inputs; local training is described as reducing the chance of strong class dependencies.Source: §7 · pp. 22–23

Funding.

The supplementary material acknowledges funding and elaborates on uncertainty concepts, related credal classifiers, operative bounds, and the computational cost of the generalized Hartley measure.

Funding.

  • The first author acknowledges support from ARO Grant ARO-MURI W911NF2010080.Source: Funding · p. 23

APPENDIX B: FURTHER RELATED WORK

  • Credal classifiers generalize classifiers based on Bayesian networks; unlike them, CDEC and IDEC do not require certain independence assumptions or NP-hard optimal-structure searches.The comparison concerns independence between non-descendant, non-parent variables and structure search in Bayesian networks.Source: Appendix B · p. 27

APPENDIX C: GENERAL OPERATIVE BOUNDS FOR EU AND TU

  • The appendix states a theorem on bounds for total and epistemic uncertainties associated with a finitely generated credal set.It defines total, aleatoric, and epistemic uncertainty using entropy quantities; the supplied theorem passage does not include the bounds themselves.Source: Appendix C · p. 27

APPENDIX D: GENERALIZED HARTLEY MEASURE

  • The generalized Hartley measure has desirable uncertainty-quantification properties, but its power-set sums become burdensome when the label set is even moderately sized.The paper gives more than five or six labels as an example of a setting where computation becomes difficult and therefore uses lower and upper entropy in the main sections.Source: Appendix D · p. 28

APPENDIX E: DIFFERENT TYPES OF AMBIGUITY AND DEGREE OF CONSERVATIVENESS OF OUR IHDR

CDEC represents ambiguity through competing likelihood parameterizations, while IDEC places it directly on the predictive distribution. The appendix also relates interval prediction coverage to how conservative the prediction region is.

APPENDIX E: DIFFERENT TYPES OF AMBIGUITY AND DEGREE OF CONSERVATIVENESS OF OUR IHDR

  • CDEC represents likelihood ambiguity through different encoders and normalizing flows, which yield possible Dirichlet posterior parameterizations and a convex posterior-predictive set.Source: Appendix E · p. 28
  • IDEC instead expresses predictive ambiguity by assuming the true posterior-predictive pmf belongs to an interval around the predictive pmf.The interval is I[ℓ,(1 + d)ℓ].Source: Appendix E · p. 28
  • For the interval prediction region, a value in equation (18) closer to γ corresponds to greater conservativeness.The equation quantifies the excess of coverage over the nominal level 1 − γ.Source: Appendix E · p. 28

APPENDIX F: A k-DIMENSIONAL CLASSIFICATION PROBLEM WITH UNKNOWN

The appendix addresses classification when the number of classes is unknown by constructing a credal predictive set from multiple Dirichlet priors. It also proposes dropping categories whose upper predictive probabilities are sufficiently small.

APPENDIX F: A k-DIMENSIONAL CLASSIFICATION PROBLEM WITH UNKNOWN

  • The construction assumes a reasonable maximum category count K and forms a likelihood credal set under categorical i.i.d. labels, then combines multiple Dirichlet priors.The prior credal set is the convex hull of the extended Dirichlet priors.Source: Appendix F · p. 29
  • The predictive credal set's extreme elements can be obtained by a convex-hull algorithm, and their number is at most M · S.Source: Appendix F · p. 29
  • A category can be dropped when its upper predictive probability is bounded by an arbitrarily small ε.The maximum can be computed over the predictive set's extreme elements.Source: Appendix F · p. 29
  • The proposed procedure addresses agnostic prior beliefs, ambiguity about the data-generating process, and uncertainty about the number of categories.The authors plan to compare it with Nonparametric Predictive Inference in future work.Source: Appendix F · p. 29

APPENDIX G: PROOFS

Appendix G supplies proofs for entropy results on finitely generated credal sets and for results on lower probabilities and interval-derived sets. Appendix H outlines the ECE and OoD-detection algorithms.

APPENDIX G: PROOFS

  • Theorem 4.3 follows from the superadditivity of lower probabilities.The proof cites Cerreia-Vioglio, Maccheroni and Marinacci (2015) and Walley (1991).Source: Appendix G · p. 29
  • Proposition 5.2’s proof constructs a finitely generated credal set corresponding to an interval by forming distributions from its lower and upper probabilities.The proof uses 2×|Y| distributions and takes their convex hull.Source: Appendix G · p. 30
  • Theorem G.1 bounds upper and lower entropy using a credal set’s extreme elements, including equality between the set’s entropy and its minimum extreme-point entropy.The proof derives the equality using convex combinations and entropy concavity.Source: Appendix G · p. 30

APPENDIX H: ALGORITHMS

  • The appendix outlines algorithms for computing expected calibration error and OoD-detection metrics from predicted probabilities and labels.The ECE algorithm takes predicted class probabilities, true labels, and a bin count as inputs and returns ECE.Source: Appendix H · p. 31

H.1. Expected Calibration Error (ECE).

The appendix specifies how ECE and OoD-detection metrics are computed, then gives training details and notes extended evaluation across backbone architectures.

H.1. Expected Calibration Error (ECE).

  • ECE weights each confidence bin by its share of samples and aggregates the difference between mean confidence and empirical accuracy.Confidence scores are divided into equal-width bins on [0,1]; the implementation uses a fixed bin count without temperature scaling or post-hoc smoothing.Source: §H.1 · pp. 31–32

H.2. Algorithm for AUROC and AUPRC (OoD detection).

  • OoD detection treats OoD samples as positives and iD samples as negatives, computing AUROC and AUPRC from uncertainty scores or negated maximum confidence.The implementation computes both metrics directly from the scores without additional calibration or smoothing.Source: §H.2 · p. 32

I.1. Training details.

  • Posterior Network training uses three seeds per backbone-dataset pair, shared hyperparameters, Adam at 10^-4, batch size 128, and early stopping.Training uses up to 200 epochs, with checks every two epochs and patience of five checks; the study also tests CDEC ensemble sizes 1, 3, 5, 7, and 10.Source: §I.1 · pp. 32–33

I.2. Adaptability to different backbone architectures.

  • Extended experimental results are reported for all four backbone architectures considered in the study: a lightweight convolutional network (Conv), VGG16, ResNet18, and ResNet50.Source: §I.2 · p. 33

I.2.1. Predictive Performance across all backbones.

Across datasets and backbones, CDEC-3 is reported to achieve best or near-best accuracy and Brier score, while uncertainty analyses show stronger OoD separation than the alternatives. IDEC’s behavior is more mixed, particularly on CIFAR-100.

I.2.1. Predictive Performance across all backbones.

  • Across nearly all dataset-backbone combinations, CDEC-3 yields the best or near-best accuracy and Brier score, with its advantage most pronounced on CIFAR-10 and CIFAR-100.The reported explanation is that restricting aggregation to ensemble posterior-predictive extreme points avoids entropy inflation from uniformly averaging all members, reducing miscalibration and probability-mass spread.Source: §I.2.1 · pp. 33–35
  • IDEC often has the lowest ECE on MNIST, but interval widening on CIFAR-100 can flatten class probabilities and increase Brier error.The reported effect is stronger for deeper backbones.Source: §I.2.1 · p. 35
  • CDEC-3 gives low-variance iD total uncertainty and a clearer upward shift on OoD data, whereas PostNet ensembles have diffuse epistemic uncertainty and total uncertainty that remains correlated with aleatoric uncertainty.IDEC’s uncertainty can be dominated by inflated epistemic terms, especially on CIFAR-100; the paper attributes this to categorical variance scaling with (1+d*)^2.Source: §I.2.1 · p. 36
  • CDEC-3 provides the strongest EU- and TU-based OoD separation across datasets and architectures; IDEC works when d* is moderate but loses iD–OoD contrast on CIFAR-100.Both methods’ uncertainty distributions shift toward higher values for OoD samples, with CDEC producing more compact iD distributions and clearer separation.Source: §I.2.1 · p. 37

I.2.4. Additional IHDR analysis and visualizations.

The IHDR analysis compares prediction-set size and coverage on in-distribution data, then examines size changes under distribution shift and qualitative set examples. CDEC-3 generally produces compact, high-coverage sets and expands more reliably under shift than IDEC.

I.2.4. Additional IHDR analysis and visualizations.

  • CDEC-3 produces compact iD IHDRs—about size 1 on MNIST and typically below 7 on CIFAR-10—while maintaining nearly perfect coverage; IDEC sets are broader, especially on CIFAR-100.Figure 7 displays IHDR cardinality distributions and mean coverage for both methods across the three datasets.Source: §I.2.4 · p. 38, Figure 7 · p. 38
  • CDEC-3 IHDRs expand under distribution shift by +7.3 and +7.5 on MNIST, +3.4–+3.8 on CIFAR-10, and +23.0 on CIFAR-100; IDEC shifts are smaller and can contract.The reported shifts indicate that CDEC-3 distinguishes iD from OoD more reliably in credal space.Source: §I.2.4 · p. 38, Table 10 · p. 39
  • Qualitative IHDR examples show CDEC-3 yields small, semantically coherent sets on CIFAR-10 and remains comparatively compact on CIFAR-100, where both models’ sets grow and IDEC may return dozens of classes.Source: §I.2.4 · p. 39, Figure 8 · p. 39
Loading 2512.05526v1…