PaperlayerView exact revision on arXiv

Source-linked AI summary

Credal Concept Bottleneck Models for Epistemic-Aleatoric Uncertainty Decomposition

Tanmoy Mukherjee, Thomas Bailleux, Pierre Marquis, Zied Bouraoui

arXiv:2604.24170v1cs.AI

TL;DR

CBMs’ point concept probabilities merge epistemic and aleatoric uncertainty, making concept-level uncertainty difficult to interpret and act upon. CREDENCE represents concepts as probability intervals and separates epistemic uncertainty from ensemble-head disagreement and aleatoric uncertainty from predicted ambiguity trained against annotator disagreement when available. Epistemic uncertainty correlated with prediction errors (ρ = 0.287, p < 10−33), while aleatoric uncertainty captured genuine data ambiguity (ρ = 0.785).

  • Problem

    CBMs’ point concept probabilities merge epistemic and aleatoric uncertainty, making concept-level uncertainty difficult to interpret and act upon.

    Source: §1 · pp. 1–2

  • Method

    CREDENCE represents concepts as probability intervals and separates epistemic uncertainty from ensemble-head disagreement and aleatoric uncertainty from predicted ambiguity trained against annotator disagreement when available.

    Source: §3 · p. 3, §3.1 · p. 4

  • Results

    Epistemic uncertainty correlated with prediction errors (ρ = 0.287, p < 10−33), while aleatoric uncertainty captured genuine data ambiguity (ρ = 0.785).

    Source: §5 · p. 9

  • Takeaways & Limitations

    Targeting high-aleatoric concepts for intervention outperformed epistemic targeting across datasets.

    Source: §5 · p. 9

    CREDENCE depends on concept-level annotations, restricting its use in certain domains.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Concept Bottleneck Models (CBMs) predict through human-interpretable concepts, but they typically output point concept probabilities that conflate epistemic uncertainty (reducible model underspecification) with aleatoric uncertainty (irreducible input ambiguity). This makes concept-level uncertainty hard to interpret and, more importantly, hard to act upon. We introduce CREDENCE (Credal Ensemble Concept Estimation), a CBM framework that decomposes concept uncertainty by construction. CREDENCE represents each concept as a credal prediction (a probability interval), derives epistemic uncertainty from disagreement across diverse concept heads, and estimates aleatoric uncertainty via a dedicated ambiguity output trained to match annotator disagreement when available. The resulting signals support prescriptive decisions: automate low-uncertainty cases, prioritize data collection for high-epistemic cases, route high-aleatoric cases to human review, and abstain when both are high. Across several tasks, we show that epistemic uncertainty is positively associated with prediction errors, whereas aleatoric uncertainty closely tracks annotator disagreement, providing guidance beyond error correlation. Our implementation is available at the following link: https://github.com/Tankiit/Credal_Sets/tree/ensemble-credal-cbm

1 Introduction

CREDENCE makes uncertainty in Concept Bottleneck Models actionable by representing concept predictions as probability intervals and structurally separating epistemic uncertainty from aleatoric ambiguity. Its signals support different responses to model limitations and irreducible disagreement.

1 Introduction

  • Standard Concept Bottleneck Models use point probabilities for interpretable concepts, collapsing epistemic and aleatoric effects and making appropriate responses difficult to determine (Koh et al., 2020).Practitioners need to know what attribute is uncertain, not only how much uncertainty a prediction carries.Source: §1 · pp. 1–2
  • High epistemic uncertainty calls for more evidence or model improvements, high aleatoric uncertainty for human oversight or qualified outputs, and high–high cases for abstention or escalation.A single confidence score can conflate a model’s lack of training evidence with genuine ambiguity among competent annotators, though those cases call for different interventions.Source: §1 · pp. 1–2
  • CREDENCE represents concept predictions as probability intervals and derives epistemic uncertainty from ensemble-head disagreement and aleatoric uncertainty from a dedicated ambiguity head trained on annotator disagreement.The framework structurally separates the signals so they do not collapse into one uncertainty measure.Source: §1 · p. 2
  • Across several NLP tasks, epistemic uncertainty aligns with prediction errors, while aleatoric uncertainty tracks annotator disagreement, providing actionable guidance beyond error correlation.The framework supports decisions to trust predictions, abstain, or escalate cases to human review.Source: §1 · p. 2

2 Related Works

Prior NLP interpretability and uncertainty methods leave concept-level uncertainty distinctions and guidance unresolved; this work connects them through a credal, ensemble-based CBM that separates epistemic disagreement from annotator-derived ambiguity for prescriptive routing.

Concept Bottleneck Models

  • Existing CBMs route predictions through interpretable concepts, but their point-probability representations conflate epistemic and aleatoric uncertainty.NLP adaptations include Text Bottleneck Models (Ludan et al., 2024), CLARITY, and CB-LLMs (Sun et al., 2025); probabilistic extensions study stochastic embeddings and intervention under uncertainty (Kim et al., 2023; Collins et al., 2023).Source: §2 · p. 2

Rationales and Local Explanations

  • Rationale extraction, attention explanations, and feature attribution highlight tokens but do not distinguish whether uncertainty reflects model underspecification or linguistic ambiguity.The same diffuse rationale may arise from either epistemic or aleatoric uncertainty.Source: §2 · p. 2

Uncertainty Estimation

  • Uncertainty methods often focus on model outputs rather than intermediate concepts, limiting concept-specific guidance; Xiao and Wang (2019) decompose uncertainty using ensemble disagreement alone.The literature includes MC Dropout, deep ensembles, evidential methods, and uncertainty work on translation, QA, classification, and LLM calibration, while the epistemic-aleatoric distinction has received relatively less attention in NLP.Source: §2 · p. 2

Imprecise Probabilities and Credal Sets

  • Credal sets represent uncertainty about probabilities without committing to a single estimate and have been used for reject-option classification and set-valued prediction.This framework provides concept-level uncertainty signals intended to better separate epistemic and aleatoric effects.Source: §2 · p. 3

Positioning

  • Prior CBMs and NLP uncertainty methods leave concept-level epistemic-aleatoric separation and prescriptions for data collection, human routing, or abstention largely unaddressed.The proposed credal ensemble CBM separates epistemic disagreement from annotator-derived ambiguity and evaluates prescriptive routing rather than error correlation alone.Source: §2 · p. 3

3 Methodology

CREDENCE represents concept predictions as probability intervals and separates epistemic uncertainty from aleatoric uncertainty using structurally different sources. Its architecture aggregates diverse concept-head predictions and uses a dedicated ambiguity prediction to support uncertainty-aware decisions.

3 Methodology

  • CREDENCE replaces each point concept probability with an interval whose span represents uncertainty and whose midpoint is the best estimate.This credal representation expresses a range of probabilities rather than a single value.Source: §3 · p. 3

3.1 CREDENCE Architecture

  • A frozen pretrained encoder feeds multiple independently trained concept heads, implemented as LoRA adapters with different ranks.The heads share a fixed backbone but vary in rank to produce diverse predictions.Source: §3.1 · pp. 3–4
  • The minimum and maximum predictions across heads form conservative concept bounds, with wider intervals indicating greater epistemic uncertainty.Narrow intervals indicate head agreement; wide intervals indicate disagreement.Source: §3.1 · p. 4
  • Mean concept predictions feed a linear classifier, while propagated credal bounds and uncertainty signals support uncertainty-aware routing decisions.The heads, aleatoric head, and classifier are trained jointly with the encoder frozen; annotator labels are not used at test time.Source: §3.1 · p. 4
  • Epistemic uncertainty comes from disagreement across heads, while a dedicated aleatoric head predicts ambiguity and is trained on annotator disagreement when available.Because the signals derive from different parameters, they cannot collapse into a single score.Source: §3.1 · p. 4, Figure 1 · p. 4

4 Experiments

Experiments test whether CREDENCE distinguishes model error risk from human ambiguity and whether that distinction supports targeted actions. Across datasets and model scales, the results show distinct uncertainty signals and actionable correction and routing decisions.

4.1 Main Results

  • On CEBaB, CREDENCE reaches ρale 0.785 vs. best baseline 0.356, while CRED.-RoBERTa reaches ρepi 0.287 vs. CBM+Ensemble’s 0.189.DeBERTa has the strongest encoder epistemic correlations; LoRA-adapted LLMs are competitive but not superior.Source: Table 2 · p. 6, §4.1 · p. 6
  • Epistemic uncertainty is positively associated with errors, while aleatoric uncertainty tracks annotator disagreement on CEBaB; its negative SST-2 error correlation reflects absent disagreement labels, not an inverse relationship.On CEBaB, CREDENCE’s ρale is 0.742–0.785 versus the best baseline’s 0.356, and it increases with concept ambiguity.Source: §4.1 · pp. 6–7
  • Aleatoric-targeted corrections address ambiguity that affects decisions, whereas high-epistemic concepts may be rare edge cases weakly connected to predictions, supporting aleatoric prioritization when annotation budgets are limited.The intervention results show the uncertainty types identify different concept sets; aleatoric-targeted results have ±0.2pp variance and epistemic-targeted results ±2.4pp.Source: §4.1 · p. 7, Table 3 · p. 7
  • Routing distinguishes clear-input learnable errors (DATA, 56.6% ∆Accuracy) from ambiguous cases for human review (REVIEW, 85.7% ∆Accuracy), which aggregate uncertainty would treat alike.The DATA response is to collect training data; REVIEW cases call for human review because people may legitimately disagree.Source: §4.1 · p. 7, Table 4 · p. 8

4.2 Ablation Studies

  • More ensemble heads and diverse head configurations strengthen ρepi while leaving ρale largely stable; identical heads weaken epistemic uncertainty, so diversity matters more than head count.The study reports that five diverse heads outperform fifteen identical ones for exposing model confusion.Source: §4.2 · p. 8, Table 5 · p. 9

4.3 Aleatoric Supervision Modes

  • Removing the aleatoric head collapses ρale from 0.785 to 0.356; without annotator labels, heteroscedastic NLL reaches ρale 0.523 versus the proxy-only baseline’s 0.356, while ρepi remains 0.279–0.287.Supervised BCE is strongest when disagreement labels are available; changing aleatoric supervision does not affect the epistemic signal.Source: §4.2 · p. 8, Table 6 · p. 9, §4.3 · p. 9
  • Using per-annotator UNKNOWN rates as a proxy, CREDENCE’s aleatoric assignments show moderate agreement (κ = 0.47) that increases monotonically with concept ambiguity.The comparison binarizes unknown rates and aleatoric scores at the median and uses Cohen’s κ and Krippendorff’s α.Source: §4.3 · p. 9, Table 7 · p. 9

5 Conclusion

CREDENCE decomposes CBM uncertainty into epistemic and aleatoric components using credal sets, linking the two components to distinct roles in human–AI collaboration.

5 Conclusion

  • Epistemic uncertainty correlated with prediction errors (ρ = 0.287, p < 10−33), while aleatoric uncertainty tracked data ambiguity (ρ = 0.785); high-aleatoric targeting outperformed epistemic targeting across datasets.The intervention comparison is reported in Table 3.Source: §5 · p. 9
  • Epistemic uncertainty signals where models need more data, while aleatoric uncertainty highlights where human judgment is essential.Source: §5 · p. 9
  • Future work will study unsupervised aleatoric estimation without annotator labels, generative settings, and learned aggregation beyond naïve averaging.Source: §5 · p. 9

Limitations · A Notation Reference

CREDENCE is limited by its reliance on concept-level annotations and, with frozen encoders, explicit annotator-disagreement supervision for aleatoric uncertainty. Ensemble inference adds H× forward passes, while moderate epistemic-error correlations suggest boundary mistakes involve both uncertainty types.

Limitations

  • CREDENCE’s dependence on concept-level annotations restricts its use in some domains.Source: Limitations · p. 9
  • Ensemble inference requires H× additional forward passes.Source: Limitations · p. 9
  • Those additional forward passes can be run in parallel.Source: Limitations · p. 9
  • With frozen encoders, estimating aleatoric uncertainty requires explicit supervision derived from annotator disagreement.Source: Limitations · p. 9
  • Epistemic-error correlations are moderate (ρ ≈0.3), indicating that boundary mistakes are influenced by both uncertainty forms.Source: Limitations · p. 9

A Notation Reference

  • Table 8 summarizes the mathematical symbols and notation used throughout CREDENCE to support the methodological descriptions in Section 3.Source: A Notation Reference · p. 12

B Architecture Implementation Details … D.2 Hyperparameter Configuration

CREDENCE uses LoRA ensemble heads with varied ranks to support credal uncertainty, propagates concept intervals through classification, and trains task, concept, and aleatoric objectives. The appendices also describe an alternative uncertainty loss and point to the experiments’ hyperparameter configuration.

B.1 LoRA Head Implementation

  • Each LoRA ensemble head applies low-rank adaptation to a shared base projection layer.Source: §B.1 · p. 12

B.2 Functional Diversity Through Rank Variation

  • Varying LoRA ranks combines regularized, potentially oversimplified low-rank heads with higher-capacity heads that may overfit, helping credal aggregation distinguish agreement from disagreement.Agreement yields narrow credal sets, while disagreement yields wide sets.Source: §B.2 · p. 12

C.1 Exact Interval Propagation

  • Exact interval propagation gives tight logit bounds for a linear classifier over independently varying concept intervals, and monotonicity of the sigmoid gives exact binary-classification probability bounds.Source: §C.1 · pp. 12–13

D.1 Multi-Objective Loss Function

  • The training objective combines task cross-entropy on mean concept predictions, concept-prediction binary cross-entropy across heads and concepts, and binary cross-entropy matching predicted aleatoric uncertainty to annotator disagreement.Source: §D.1 · p. 13
  • When annotator-disagreement labels are unavailable, training can instead use a heteroscedastic regression loss based on predicted variance.The supplied passage identifies the predicted variance for each head and concept.Source: §D.1 · p. 13

D.2 Hyperparameter Configuration

  • Table 10 is identified as the source of the complete experiment-wide hyperparameter configuration and the rationale for each selection.The supplied passage does not include the table’s configuration values.Source: §D.2 · p. 13

D.3 Training Algorithm … F.4 Per-Concept Aleatoric Validation (RQ2)

The sections describe CREDENCE’s training and inference, introduce credal-set concepts and decision criteria, and detail its uncertainty baselines. Per-concept validation reports that aleatoric signals strengthen with concept ambiguity, while epistemic signals capture run-specific model confusion.

D.3 Training Algorithm

  • CREDENCE jointly trains ensemble concept heads, a separate aleatoric head, and a linear concept-to-label classifier while keeping the encoder frozen.Its loss combines task, concept, and aleatoric terms; the aleatoric head is supervised using annotator ambiguity labels.Source: §D.3 · p. 13

D.4 Inference Algorithm

  • At inference, one frozen-encoder pass feeds all concept heads and the aleatoric head; credal bounds are propagated through the linear classifier to produce label confidence intervals.The final label prediction uses mean concept probabilities, and sample-level uncertainties are averaged across concepts.Source: §D.4 · pp. 13–14

E Credal Set Theory Background; E.1 Imprecise Probability; E.2 Credal Sets

  • Imprecise probability represents events with probability intervals, and a credal set is a closed, convex set of probability distributions.For binary events, the credal representation reduces to an interval whose width quantifies total uncertainty.Source: §E.1 · p. 14, §E.2 · pp. 14–15

E.3 Decision Criteria

  • Credal decision criteria include Γ-Maximin, maximality, and E-admissibility, which determine non-dominated labels under concept uncertainty.These criteria respectively use worst-case expected utility, dominance across distributions, or optimality under some distribution in the credal set.Source: §E.3 · p. 15

F Baseline Method Descriptions; F.1 General Uncertainty Quantification Methods; F.3 Baseline Selection Rationale

F.2 Concept-Based Model Baselines; F.3 Baseline Selection Rationale

  • Concept-level uncertainty baselines include CBM with MC Dropout (50 forward passes), CBM with Deep Ensemble (5 independently trained CBMs), and P-CBM (Kim et al., 2023), testing actionable signals without explicit decomposition.CREDENCE uses 5 heads versus 5 ensemble models, and 50 MC passes provide comparable wall-clock inference time.Source: §F.2 · p. 16, §F.3 · p. 16

F.4 Per-Concept Aleatoric Validation (RQ2)

  • Higher rates of “unknown” annotations correspond to stronger aleatoric signals; aleatoric identifies consistently important concepts, while epistemic captures run-specific model confusion.The validation is reported for CEBaB in Table 11.Source: §F.4 · p. 16, Table 11 · p. 16

F.5 Intervention Stability Analysis

Across runs, aleatoric-targeted intervention gains are much more stable than epistemic-targeted gains, reflecting distinct uncertainty signals.

F.5 Intervention Stability Analysis

  • Aleatoric-targeted gains vary by ±0.2pp across runs, versus ±2.4pp for epistemic-targeted gains.Table 12 reports the per-run intervention breakdown.Source: §F.5 · p. 16, Table 12 · p. 16
  • Aleatoric uncertainty identifies concepts with consistent predictive importance, while epistemic uncertainty captures model-specific confusion that varies across runs.Source: §F.5 · p. 16

G Quadrant-Based Routing Analysis … H.3 Deployment Guidelines

The quadrant analysis links epistemic and aleatoric uncertainty to different error and ambiguity patterns, supporting distinct deployment responses. Qualitative examples show how review content and conflicting or idiomatic language map to these quadrants.

G Quadrant-Based Routing Analysis

  • HUMAN REVIEW has the highest ∆Accuracy (85.7%), while COLLECT DATA has the lowest (56.6%), supporting review for disagreement and data collection for fixable errors.High aleatoric uncertainty can reflect legitimate human disagreement even when the model is correct; high epistemic uncertainty identifies learnable errors.Source: G Quadrant-Based Routing Analysis · p. 16

G.1 Cross-Dataset Quadrant Analysis

  • CEBaB shows the clearest separation across quadrants, attributed to its causal concept structure and comprehensive “unknown” annotations.These annotations provide clear training signals for both epistemic and aleatoric uncertainty.Source: §G.1 · p. 16

H Qualitative Analysis and Interpretation Guidelines

  • The qualitative section aims to validate that the decomposition captures meaningful distinctions and to guide practitioners acting on CREDENCE outputs.Source: H Qualitative Analysis and… · p. 16

H.1 Representative Examples by Uncertainty Quadrant

  • The CEBaB analysis sampled 50 examples per quadrant to illustrate the range of linguistic phenomena in each.Table 13 presents extended qualitative examples; its caption defines Uepi as epistemic uncertainty and Uale as aleatoric uncertainty.Source: §H.1 · pp. 16–17, Table 13 · p. 17

H.2 Patterns Across Quadrants

  • High-aleatoric reviews average 4.2 words, versus 12.1 words for low-aleatoric reviews, with shorter text lacking context for aspect-level sentiment.The aleatoric head identifies this as inherent ambiguity.Source: §H.2 · p. 17
  • Conflicting aspect sentiments drive DATA-quadrant epistemic uncertainty, while idioms cluster in high-aleatoric quadrants because cultural context varies across annotators.In DATA cases, individual aspect sentiments are clear and ambiguity lies in aggregation; ABSTAIN cases combine model confusion and inherent ambiguity, and even human experts struggle.Source: §H.2 · pp. 17–18

H.3 Deployment Guidelines

  • TRUST cases are safe to automate, while DATA cases should be prioritized for active learning because additional training data can help.The guidelines characterize TRUST as low epistemic and aleatoric uncertainty, and DATA as high epistemic but low aleatoric uncertainty.Source: §H.3 · p. 18
  • Route REVIEW cases to humans because interpretations may legitimately differ; decline or escalate ABSTAIN cases, where both model and humans struggle.REVIEW has the highest ∆Accuracy (85.7%), but the guideline recommends human review because stakeholders may disagree on the correct label.Source: §H.3 · p. 18

H.4 Failure Mode Analysis … I.6 Loss Weight Sensitivity

The failure analysis distinguishes confident model errors, persistent data-related errors, and legitimate annotator disagreement; the extended ablations show how ensemble and training choices affect uncertainty signals and accuracy.

H.4 Failure Mode Analysis

  • 21.2% of TRUST predictions are errors, driven by negation scope, comparisons, and sentiment requiring world knowledge.Some DATA errors persist because of label noise, conflicting annotation guidelines, or rare constructions. REVIEW reaches 85.7% ΔAccuracy despite high aleatoric uncertainty: many examples have a clear majority label the model predicts correctly, while annotator disagreement supports human review for legitimate minority views.Source: §H.4 · p. 18

I Extended Ablation Studies

  • The extended ablations use CEBaB with a RoBERTa-base encoder unless noted, and the main-paper results appear in Table 5.The studies examine the contributions of CREDENCE components.Source: I Extended Ablation Studies · p. 18

I.1 Ensemble Size

  • As ensemble size increases, epistemic correlation and credal width increase, aleatoric uncertainty stays stable, and ΔAccuracy is largely unaffected.All reported ρ values have p < 0.001.Source: §I.1 · p. 18, Table 14 · p. 18

I.2 Diversity Mechanisms

  • Diversity sources contribute to epistemic correlation, while aleatoric uncertainty remains stable across configurations.Without diversity, heads converge to similar predictions.Source: §I.2 · p. 19

I.3 LoRA Rank Configuration

  • Diverse LoRA rank configurations outperform uniform configurations, while rank ordering has minimal effect.Source: §I.3 · p. 19

I.4 Dropout Spacing Strategy

  • Geometric spacing in keep-probability space produces higher inter-head disagreement than uniform or linear spacing.Source: §I.4 · p. 19

I.5 Aleatoric Supervision

  • Supervised training with annotator disagreement yields the strongest aleatoric correlation, while epistemic correlation remains stable across configurations.Source: §I.5 · p. 19

I.6 Loss Weight Sensitivity

  • Higher λc improves task and concept ΔAccuracy, whereas higher λa improves aleatoric correlation; the default balances these objectives.Source: §I.6 · p. 19

I.7 Cross-Dataset Consistency · I.8 Computational Cost

Key findings replicate across datasets, although SST-2 has negative aleatoric-uncertainty correlation because it lacks annotator-disagreement labels. For computation, CREDENCE shares a frozen encoder across heads and adds only LoRA parameters.

I.7 Cross-Dataset Consistency

  • Findings replicate across datasets, but SST-2 shows negative ρale because it lacks annotator-disagreement labels.Source: §I.7 · p. 19

I.8 Computational Cost

  • CREDENCE shares a frozen encoder across heads and requires only additional LoRA parameters.Source: §I.8 · p. 19
  • The reported ablations cover diversity strategy, LoRA rank, dropout spacing, aleatoric supervision, loss weights, and consistency across datasets.Source: Table 15 · p. 20, Table 16 · p. 20, Table 17 · p. 20
  • The method reports mean pairwise head disagreement as Disagree and mean credal interval as Width.Source: §I.8 · p. 20
  • A geometric dropout-spacing rule sets head-specific d_h values from d_min, d_max, and the number of heads H.The supplied passage gives the formula but no ablation outcomes.Source: §I.8 · p. 20
  • A computational comparison with baselines is listed for CEBaB using RoBERTa-base, but the supplied passage gives no comparison values.Source: Table 21 · p. 20
  • The computational comparison defines total parameters and Train/Infer/Memory relative to Standard CBM.Source: §I.8 · p. 20

Related papers

Loading 2604.24170v1…