PaperlayerView exact revision on arXiv

Source-linked AI summary

Credal Concept Bottleneck Models for Epistemic-Aleatoric Uncertainty Decomposition

Tanmoy Mukherjee, Thomas Bailleux, Pierre Marquis, Zied Bouraoui

arXiv:2604.24170v1cs.AI

TL;DR

Existing concept bottleneck models conflate epistemic and aleatoric uncertainty, making uncertain predictions difficult to interpret and route appropriately. CREDENCE uses credal concept intervals and structurally separate uncertainty signals, finding that epistemic uncertainty correlates with errors while aleatoric uncertainty tracks annotator disagreement.

  • Problem

    Existing models merge epistemic and aleatoric uncertainty into a single confidence score, limiting differentiated responses to uncertain predictions.

  • Method

    CREDENCE represents concept predictions as probability intervals and separates epistemic uncertainty from ensemble disagreement and aleatoric uncertainty from predicted ambiguity.

  • Results

    Epistemic uncertainty correlates with prediction errors (ρ = 0.287), while aleatoric uncertainty tracks annotator disagreement (ρ = 0.785) across evaluated tasks.

  • Takeaways & Limitations

    Epistemic uncertainty identifies cases needing more data, whereas aleatoric uncertainty highlights cases where human judgment is essential.

  • Takeaways & Limitations

    CREDENCE depends on concept-level annotations, and aleatoric estimation with frozen encoders requires supervision from annotator disagreement.

Abstract

from arXiv · show

Concept Bottleneck Models (CBMs) predict through human-interpretable concepts, but they typically output point concept probabilities that conflate epistemic uncertainty (reducible model underspecification) with aleatoric uncertainty (irreducible input ambiguity). This makes concept-level uncertainty hard to interpret and, more importantly, hard to act upon. We introduce CREDENCE (Credal Ensemble Concept Estimation), a CBM framework that decomposes concept uncertainty by construction. CREDENCE represents each concept as a credal prediction (a probability interval), derives epistemic uncertainty from disagreement across diverse concept heads, and estimates aleatoric uncertainty via a dedicated ambiguity output trained to match annotator disagreement when available. The resulting signals support prescriptive decisions: automate low-uncertainty cases, prioritize data collection for high-epistemic cases, route high-aleatoric cases to human review, and abstain when both are high. Across several tasks, we show that epistemic uncertainty is positively associated with prediction errors, whereas aleatoric uncertainty closely tracks annotator disagreement, providing guidance beyond error correlation. Our implementation is available at the following link: https://github.com/Tankiit/Credal_Sets/tree/ensemble-credal-cbm

1 Introduction

The introduction argues that uncertainty in high-stakes NLP must distinguish epistemic from aleatoric sources and expose what concepts are uncertain about. It presents CREDENCE as a credal CBM that separates these components to support targeted interventions such as evidence collection, human oversight, or abstention.

  • Motivation: High-stakes NLP systems need to indicate not only predictions, but also when they are reliable and why they may be unreliable.The motivation includes healthcare, legal analysis, and financial advising.
  • Motivation: Existing models often merge uncertainty types into a single confidence score, encouraging a uniform response to uncertain predictions.Epistemic uncertainty is reducible through additional evidence, whereas aleatoric uncertainty reflects irreducible ambiguity or noise.
  • Problem: Concept-level uncertainty is difficult to interpret and act upon because it can identify risky predictions without specifying the appropriate response.Interpretable attributes such as praise, hedging, or insults provide a basis for making uncertainty actionable.
  • Prescriptive decisions: Different uncertainty types motivate different interventions: collect evidence for high epistemic uncertainty, use human oversight for high aleatoric uncertainty, and abstain or escalate when both are high.The introduction emphasizes that sentiment, toxicity, and emotion tasks are subjective because their ground truth aggregates disputed human judgments.
  • Contributions: CREDENCE represents concepts as probability intervals and structurally separates epistemic ensemble disagreement from aleatoric predicted ambiguity using different parameters.The formulation is designed to support trust, abstention, and escalation to humans while avoiding collapse between uncertainty components.

2 Related Works

Prior work spans concept-based interpretability, uncertainty estimation, and credal sets, but typically does not separate epistemic and aleatoric uncertainty at the concept layer. This work connects these lines through a credal, ensemble-based CBM intended to support concept-level decisions.

  • Concept Bottleneck Models: CBMs route predictions through human-interpretable concepts, enabling inspection and intervention.NLP adaptations include Text Bottleneck Models, CLARITY, and CB-LLMs, while probabilistic extensions introduce stochastic embeddings.
  • Rationales and Local Explanations: Rationale extraction, attention-based explanations, and feature attribution highlight tokens but do not distinguish epistemic from aleatoric uncertainty.The same diffuse rationale may reflect model underspecification or linguistic ambiguity.
  • Uncertainty Estimation: Dominant uncertainty approaches include MC Dropout, deep ensembles, and evidential methods, with applications spanning translation, QA, and classification.The epistemic-aleatoric distinction has been studied in vision, while recent work addresses LLM calibration.
  • Imprecise Probabilities and Credal Sets: Credal sets represent uncertainty about probabilities rather than committing to a single estimate and support classification with reject options and set-valued prediction.This framework is used to obtain concept-level uncertainty.
  • Positioning: Prior CBM work typically represents concepts as point probabilities without separating epistemic and aleatoric uncertainty, while NLP uncertainty work rarely yields concept-level prescriptions.The paper connects these lines with a credal, ensemble-based CBM.

3 Methodology

CREDENCE makes concept-level uncertainty explicit by representing each concept as a probability interval and separating epistemic uncertainty from aleatoric ambiguity. Its pipeline uses diverse concept heads, credal aggregation, a dedicated ambiguity predictor, and uncertainty-aware classification and routing.

  • Credal Concept Predictions: CREDENCE replaces point concept probabilities with credal intervals whose spans represent uncertainty and whose midpoints provide best estimates.The intervals are formed as probability ranges rather than single values.
  • Four-Stage Pipeline: A frozen encoder feeds multiple diverse LoRA concept heads, trained without fine-tuning the backbone.Heads use different LoRA ranks from 4, 8, 16, 32, and 64 to induce prediction diversity.
  • Credal Aggregation: Minimum and maximum predictions across heads form conservative concept bounds, with narrow intervals indicating agreement and wide intervals indicating epistemic disagreement.The aggregation uses ensemble variation to quantify model confusion.
  • Credal Classification and Training: Mean concept predictions support classification, while propagated credal bounds and uncertainty signals enable principled uncertainty-aware routing decisions.The model jointly trains ensemble heads, the aleatoric head, and the classifier while keeping the encoder frozen.
  • Uncertainty Decomposition: Epistemic uncertainty comes from variation across heads, whereas a dedicated aleatoric head predicts inherent ambiguity from annotator disagreement when available.The two signals derive from different parameters, preventing them from collapsing into one score.

4 Experiments

Experiments show that CREDENCE separates epistemic uncertainty, which tracks model errors, from aleatoric uncertainty, which tracks human-perceived ambiguity. This decomposition supports distinct interventions and remains useful even when disagreement labels are unavailable.

  • Decomposition validity: CREDENCE achieves ρale = 0.785 on CEBaB versus 0.356 for the best baseline, while ρepi reaches 0.287 versus 0.189 for a matched CBM+Ensemble.These results establish separation between ambiguity tracking and epistemic-error correlation under the same backbone and five-head budget.
  • Decomposition validity: Epistemic uncertainty rises on incorrect predictions and converges on correct ones, whereas aleatoric uncertainty does not systematically distinguish errors from correct predictions.On SST-2, negative ρale reflects the absence of annotator-disagreement labels rather than an inverse error relationship.
  • Ambiguity detection: On CEBaB, 52.2% of labels are unknown, and CREDENCE’s ρale = 0.742–0.785 exceeds the best baseline value of 0.356 while increasing with concept ambiguity.The per-concept correlation rises from Food at ρ = 0.72 to Ambiance at ρ = 0.81 as unknown annotations increase.
  • Sample routing: The DATA and REVIEW routing quadrants produce ∆Accuracy values of 56.6% and 85.7%, respectively, supporting data collection for clear-input failures and human review for ambiguous inputs.Without decomposition, both cases would receive identical treatment as merely uncertain.
  • Ablations: Increasing ensemble size strengthens ρepi while keeping ρale ≈0.78 constant, and removing the aleatoric head collapses ρale from 0.785 to 0.356 without changing ρepi.Diverse head configurations are needed to expose epistemic disagreement, whereas aleatoric uncertainty requires explicit supervision.
  • Label-scarce settings: Without annotator labels, heteroscedastic NLL reaches ρale = 0.523 versus 0.356 for the proxy-only baseline, while ρepi remains stable at 0.279–0.28.Agreement increases monotonically with concept ambiguity, with overall κ = 0.47 indicating moderate agreement.

5 Conclusion

CREDENCE decomposes CBM uncertainty into epistemic and aleatoric components using credal sets. Epistemic uncertainty correlates with prediction errors, while aleatoric uncertainty captures data ambiguity and better guides intervention targeting across datasets.

  • Contributions: CREDENCE decomposes uncertainty in concept bottleneck models into epistemic and aleatoric components using credal sets.The framework uses credal sets to represent the two uncertainty components.
  • Key findings: ρ = 0.287, p < 10^-33 for epistemic uncertainty’s correlation with prediction errors.Epistemic uncertainty is measured through ensemble disagreement.
  • Key findings: ρ = 0.785 for aleatoric uncertainty’s association with genuine data ambiguity.Aleatoric uncertainty captures ambiguity inherent in the data.
  • Key findings: Targeting high-aleatoric concepts for intervention consistently outperforms epistemic targeting across datasets.The conclusion attributes this result to ambiguous concepts driving prediction outcomes more than confusing concepts.
  • Future work: Future work will estimate aleatoric uncertainty without annotator labels, extend the framework to generative settings, and learn aggregation beyond naïve averaging.These directions aim to broaden applicability and improve integration of diverse ensemble-head outputs.

Limitations · A Notation Reference

CREDENCE is limited by its dependence on concept annotations, added ensemble computation, and supervision requirements for aleatoric uncertainty when encoders are frozen. Its notation is summarized in a dedicated reference supporting the methodological descriptions in Section 3.

  • Limitations: The method depends on concept-level annotations, restricting its use in certain domains.This annotation dependence is identified as a limitation of the approach.
  • Limitations: Ensemble inference adds H× forward passes, although these passes can run in parallel.The computational overhead arises from the ensemble structure.
  • Limitations: With frozen encoders, estimating aleatoric uncertainty requires explicit supervision from annotator disagreement.The supervision requirement applies specifically when encoders are frozen.
  • Limitations: Epistemic-error correlations are moderate, with ρ ≈0.3.This indicates that boundary mistakes are influenced by both epistemic and aleatoric uncertainty.
  • Limitations: Boundary mistakes are influenced by both forms of uncertainty rather than epistemic uncertainty alone.The passage connects this interpretation to the moderate epistemic-error correlation.
  • A Notation Reference: Table 8 provides a comprehensive summary of the mathematical symbols and notation used throughout CREDENCE.The notation reference supports the methodological descriptions in Section 3 of the main paper.

B Architecture Implementation Details … D.2 Hyperparameter Configuration

CREDENCE implements diverse LoRA ensemble heads, propagates their concept intervals exactly through linear classifiers, and trains task, concept, and aleatoric objectives. Rank variation makes head disagreement informative for credal-set width, while hyperparameters are documented for reproducibility.

  • B.1 LoRA Head Implementation: Each LoRA ensemble head applies low-rank adaptation to a shared base projection layer.
  • B.2 Functional Diversity Through Rank Variation: Varying LoRA ranks creates functional diversity: low-rank heads are more regularized, whereas high-rank heads offer greater capacity but may overfit.Low-rank heads capture dominant concept patterns; high-rank heads model finer-grained details.
  • B.2 Functional Diversity Through Rank Variation: Credal aggregation yields narrow sets when heads agree and wide sets when they disagree.
  • C.1 Exact Interval Propagation: For linear classifiers, exact logit interval bounds are obtained by propagating concept-probability intervals through the classifier.The bounds are tight because each concept term can independently attain its extremal value.
  • C.1 Exact Interval Propagation: For binary classification, sigmoid monotonicity converts exact logit bounds into exact probability bounds.The resulting probability interval is [σ(ℓj), σ(ℓj)].
  • D.1 Multi-Objective Loss Function: Training combines task classification loss on mean concept predictions, concept-prediction loss across heads and concepts, and aleatoric loss against annotator disagreement.When annotator-disagreement labels are unavailable, a heteroscedastic regression loss is used instead.
  • D.2 Hyperparameter Configuration: Table 10 documents the complete hyperparameter configuration used across experiments and provides the rationale for each selection.

D.3 Training Algorithm … F.4 Per-Concept Aleatoric Validation (RQ2)

CREDENCE jointly trains diverse concept heads, an aleatoric head, and a classifier, then propagates concept credal bounds to label-level intervals for uncertainty-aware decisions. Its per-concept validation finds that aleatoric signals strengthen with annotation ambiguity, while epistemic uncertainty reflects run-specific model confusion.

  • D.3 Training Algorithm: CREDENCE jointly trains H ensemble concept heads, a separate aleatoric head, and a linear concept-to-label classifier with a combined loss.The encoder remains frozen during training.
  • D.3 Training Algorithm: The task loss uses mean concept predictions, the concept loss promotes diverse and accurate heads, and the aleatoric loss matches annotator ambiguity labels.Experiments fix λc = 1.0 and λa = 0.5.
  • D.4 Inference Algorithm: At inference, one frozen-encoder pass feeds all ensemble heads and the aleatoric head in parallel, producing per-concept uncertainty signals.The procedure computes predictions for every head and concept.
  • D.4 Inference Algorithm: Mean concept probabilities determine the final label, while interval arithmetic propagates credal bounds to label-level confidence intervals and sample uncertainties average across concepts.The inference procedure returns the prediction together with epistemic and aleatoric uncertainty.
  • E Credal Set Theory Background: Credal sets represent imprecise probabilities as closed, convex sets or intervals rather than single event probabilities, with interval width quantifying total uncertainty.For binary events, a credal set reduces to an interval.
  • E.3 Decision Criteria: Γ-Maximin, maximality, and E-admissibility provide decision criteria for selecting non-dominated labels under concept uncertainty.These criteria differ in whether they optimize worst-case utility, avoid domination, or maximize utility for some distribution in the credal set.
  • F Baseline Method Descriptions: Baselines span stochastic dropout, independently trained ensembles, temperature scaling, evidential learning, deterministic CBMs, dropout and ensemble CBMs, and probabilistic CBMs.Standard CBM entropy conflates epistemic and aleatoric uncertainty, while the listed CBM variants do not explicitly decompose them.

F.5 Intervention Stability Analysis

Intervention gains targeted by aleatoric uncertainty are stable across runs, whereas epistemic-targeted gains vary substantially. This difference indicates that aleatoric uncertainty identifies consistently important concepts, while epistemic uncertainty reflects run-dependent model confusion.

  • F.5 Intervention Stability Analysis: Aleatoric-targeted intervention gains remain remarkably stable across runs, varying by only ±0.2pp.Table 12 reports the per-run intervention breakdown and highlights aleatoric stability.
  • F.5 Intervention Stability Analysis: Epistemic-targeted intervention gains vary widely across runs, with variation of ±2.4pp.This greater variability reflects model-specific confusion that changes across runs.
  • F.5 Intervention Stability Analysis: The stability difference suggests that aleatoric uncertainty identifies concepts with consistent predictive importance, whereas epistemic uncertainty captures run-dependent model confusion.The interpretation follows the reported contrast between stable aleatoric and variable epistemic intervention gains.

G Quadrant-Based Routing Analysis … H.3 Deployment Guidelines

CREDENCE’s uncertainty quadrants distinguish automatable, learnable, human-review, and abstention cases, with qualitative analyses linking each quadrant to interpretable linguistic patterns. These distinctions support deployment decisions based on whether model disagreement, input ambiguity, or both drive uncertainty.

  • G Quadrant-Based Routing Analysis: The HUMAN REVIEW quadrant achieves the highest ∆Accuracy at 85.7% despite high aleatoric uncertainty, reflecting cases where humans may legitimately disagree.The model is often correct, but the input supports multiple valid interpretations.
  • G Quadrant-Based Routing Analysis: The COLLECT DATA quadrant has the lowest ∆Accuracy at 56.6%, confirming that epistemic uncertainty identifies fixable model errors.These cases are appropriate targets for additional training data.
  • G.1 Cross-Dataset Quadrant Analysis: CEBaB shows the clearest quadrant separation because its causal concept structure and comprehensive “unknown” annotations provide strong signals for both uncertainty types.The dataset’s annotation design supports distinguishing epistemic from aleatoric uncertainty.
  • H.1 Representative Examples by Uncertainty Quadrant: The qualitative analysis validates semantically meaningful uncertainty distinctions and translates them into concrete deployment guidelines, using 50 examples per quadrant from CEBaB.Table 13 documents representative examples across all four quadrants and their correctness.
  • H.2 Patterns Across Quadrants: High-aleatoric reviews average 4.2 words versus 12.1 words in low-aleatoric quadrants, indicating that shorter inputs lack context for aspect-level sentiment.The aleatoric head identifies this missing context as inherent ambiguity.
  • H.2 Patterns Across Quadrants: Conflicting aspect sentiments produce high epistemic uncertainty because ensemble heads disagree on aggregation, while aleatoric uncertainty remains moderate when individual aspects are clear.Idiomatic expressions instead cluster in high-aleatoric quadrants because cultural context varies across annotators, creating persistent human disagreement.
  • H.3 Deployment Guidelines: TRUST cases are safe for automation, whereas DATA cases should receive active learning because they are clear inputs with learnable model errors.TRUST combines low epistemic and aleatoric uncertainty; DATA combines high epistemic with low aleatoric uncertainty.
  • H.3 Deployment Guidelines: REVIEW cases should route to humans, while ABSTAIN cases should be declined or escalated because both model confusion and human ambiguity are high.ABSTAIN examples are genuinely hard cases, and manual inspection confirms that even human experts struggle with them.

H.4 Failure Mode Analysis … I.6 Loss Weight Sensitivity

Failure analysis attributes high-confidence TRUST errors to linguistic and world-knowledge challenges, while persistent DATA errors reflect noise, conflicting guidelines, or rarity. Ablations show how ensemble design, diversity, supervision, and loss weights affect uncertainty correlations and accuracy.

  • H.4 Failure Mode Analysis: TRUST false positives have a 21.2% error rate and arise from negation scope, comparative constructions, and implicit sentiment requiring world knowledge.These errors occur despite high confidence.
  • H.4 Failure Mode Analysis: Persistent DATA errors remain after intervention because of label noise, conflicting annotation guidelines, and rare constructions with insufficient similar examples.These factors identify opportunities for improving training data and annotation.
  • H.4 Failure Mode Analysis: REVIEW reaches 85.7% ΔAccuracy despite high aleatoric uncertainty because aleatoric measures annotator disagreement, while many examples have a clear majority label.The result supports routing cases to human review for disagreement rather than assuming the model is wrong.
  • I Extended Ablation Studies: The extended ablations isolate contributions of CREDENCE components using CEBaB with a RoBERTa-base encoder, unless otherwise noted.Main paper results appear in Table 5.
  • I.1 Ensemble Size: As ensemble size increases, epistemic correlation and credal width rise, aleatoric uncertainty remains stable, and ΔAccuracy is largely unaffected.All reported ρ values have p < 0.001.
  • I.2 Diversity Mechanisms: Diversity mechanisms contribute to epistemic correlation, whereas aleatoric uncertainty remains stable across configurations.Without diversity, heads converge to similar predictions.
  • I.3 LoRA Rank Configuration: Diverse LoRA rank configurations outperform uniform configurations, while rank ordering has minimal effect; geometric keep-probability spacing increases inter-head disagreement.The spacing result compares geometric spacing with uniform and linear spacing.
  • I.5 Aleatoric Supervision: Annotator-disagreement supervision produces the strongest aleatoric correlation, while epistemic correlation remains stable across configurations; higher λc improves task and concept ΔAccuracy, and higher λa improves aleatoric correlation.The default loss weights balance these objectives.

I.7 Cross-Dataset Consistency · I.8 Computational Cost

Key findings replicate across datasets, although SST-2 yields negative ρale because it lacks annotator-disagreement labels. Computationally, CREDENCE shares a frozen encoder across heads and adds only LoRA parameters, with diversity choices evaluated through dropout and credal-width diagnostics.

  • I.7 Cross-Dataset Consistency: The study explicitly verifies that its key findings replicate across datasets.
  • I.7 Cross-Dataset Consistency: Findings replicate across datasets, with SST-2 showing negative ρale because it lacks annotator-disagreement labels.This dataset-specific limitation affects the aleatoric association rather than the broader replication claim.
  • I.8 Computational Cost: CREDENCE shares a frozen encoder across heads, requiring only additional LoRA parameters.
  • I.8 Computational Cost: The computational analysis includes ablations of diversity strategy, LoRA rank, dropout spacing, aleatoric supervision, loss weights, and cross-dataset consistency.
  • I.8 Computational Cost: Mean pairwise head disagreement and mean credal interval serve as the Disagree and Width diagnostics, respectively.
  • I.8 Computational Cost: Diverse dropout configurations achieve values from .245 to .267, compared with .167 to .182 for uniform dropout.The reported diverse schedules are Linear {0.05, 0.12, 0.20, 0.27, 0.35} (.245), Geometric {0.05, 0.09, 0.15, 0.22, 0.30} (.267), and Wide range {0.02, 0.10, 0.20, 0.35, 0.50} (.254).
  • I.8 Computational Cost: A separate comparison evaluates computational cost against CEBaB and RoBERTa-base, reporting parameters, training, inference, and memory relative to Standard CBM.
Loading 2604.24170v1…