PaperlayerView exact revision on arXiv

Source-linked AI summary

Probabilistic Concept Bottleneck Models

Eunji Kim, Dahuin Jung, Sangha Park, Siwon Kim, Sungroh Yoon

arXiv:2306.01574v1cs.LGcs.AIcs.CV

TL;DR

Deterministic concept predictions can be unreliable when concepts are ambiguous in an image. ProbCBM models this uncertainty with probabilistic concept embeddings, and analyses report improved explanation reliability while deriving class uncertainty from concept uncertainty.

  • Problem

    Concepts can be ambiguous in images, yet CBM predicts them deterministically, making reliable concept explanations a concern.

    Source: §1 · p. 2

  • Method

    ProbCBM maps images to probabilistic concept embeddings and derives class embeddings and predictions from those concept representations.

    Source: §1 · p. 2

  • Results

    Analyses show that ProbCBM reflects concept ambiguity as uncertainty and improve the reliability of explanations, with class uncertainty derived from concept uncertainty.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    Explanations can pair predicted concepts with their uncertainty, and class uncertainty can be explained through concept uncertainty.

    Source: §1 · p. 2

    ProbCBM still requires concept labels, and concept-prediction accuracy is heavily influenced by their human-annotation quality.

    Source: §6 · p. 9

Abstract

from arXiv · show

Interpretable models are designed to make decisions in a human-interpretable manner. Representatively, Concept Bottleneck Models (CBM) follow a two-step process of concept prediction and class prediction based on the predicted concepts. CBM provides explanations with high-level concepts derived from concept predictions; thus, reliable concept predictions are important for trustworthiness. In this study, we address the ambiguity issue that can harm reliability. While the existence of a concept can often be ambiguous in the data, CBM predicts concepts deterministically without considering this ambiguity. To provide a reliable interpretation against this ambiguity, we propose Probabilistic Concept Bottleneck Models (ProbCBM). By leveraging probabilistic concept embeddings, ProbCBM models uncertainty in concept prediction and provides explanations based on the concept and its corresponding uncertainty. This uncertainty enhances the reliability of the explanations. Furthermore, as class uncertainty is derived from concept uncertainty in ProbCBM, we can explain class uncertainty by means of concept uncertainty. Code is publicly available at https://github.com/ejkim47/prob-cbm.

1. Introduction

CBM bases class decisions on concept predictions, but ambiguous visual contexts can make those explanations unreliable. ProbCBM models concept uncertainty with probabilistic embeddings and uses it to explain concept and class predictions.

1. Introduction

  • Ambiguous visual contexts can hide partial concepts, while same-class labels and data augmentation can assign concepts absent from an image.These conditions can make deterministic concept predictions less faithful to the image.Source: §1 · p. 2
  • ProbCBM addresses concept ambiguity with probabilistic embeddings, mapping images to concept distributions and deriving class embeddings from the concept embeddings.The class prediction is derived from concept prediction.Source: §1 · p. 2, Figure 2 · p. 2
  • ProbCBM explains predictions using predicted concepts and their estimated uncertainty, and explains class uncertainty through concept uncertainty.The paper also examines uncertainty across visual contexts, reports increased uncertainty after image transformations, and explores human-model interactions.Source: §1 · p. 2
  • The paper identifies ambiguity in concept prediction as a reliability concern and presents uncertainty modeling as its response.The authors state that, to their knowledge, they pose this issue for the first time.Source: §1 · p. 2

2. Related Work

Related work situates ProbCBM among concept-based interpretable models and probabilistic embedding approaches. It identifies ambiguity in concept prediction as an issue not addressed by the cited CBM research.

2.1. Interpretable Neural Networks

  • CBM predicts concepts before classes using human-defined concepts, while concept-based models make predictions from learned human-interpretable concepts.CBM uses both concept and class labels; other work includes prototypes, post-hoc CBM, and Concept Embedding Models.Source: §2.1 · p. 2
  • Prior CBM work addressed label challenges, accuracy–interpretability trade-offs, and information leakage, but not ambiguity in concept prediction.The passage names PCBM, CEM, and work on leakage as examples.Source: §2.1 · p. 3

2.2. Uncertainty and Probabilistic Embeddings

  • Uncertainty modeling distinguishes model uncertainty from data uncertainty, and probabilistic embeddings primarily represent data uncertainty with distributions over input representations.Prior probabilistic embeddings address face-image ambiguity and cross-modal retrieval; this work extends the approach to concept prediction.Source: §2.2 · p. 3

3. Preliminary: Concept Bottleneck Models

A Concept Bottleneck Model predicts concepts from an input and then predicts its class from those concepts. The concept space is therefore an interpretable bottleneck and supports explanations and concept interventions.

3. Preliminary: Concept Bottleneck Models

  • CBM computes a class prediction by applying a concept predictor to the input and a class predictor to the predicted concepts.The concept space serves as an interpretable bottleneck, and learning the concept mapping requires input-concept data pairs.Source: §3 · p. 3
  • Predicted concepts explain a CBM decision because the class prediction is made according to that concept information.The concepts are intended to align with human understanding.Source: §3 · p. 3
  • Concept intervention changes a predicted concept to alter the classification result, helping people debug incorrect predictions and examine concept–class relationships.Observing the result of an intervention can provide a counterfactual explanation.Source: §3 · p. 3

4. Method

ProbCBM replaces deterministic concept prediction with probabilistic concept embeddings, then predicts classes from those concepts. It estimates uncertainty for both concept and class predictions and trains the concept and class predictors separately.

4. Method

  • ProbCBM uses a concept predictor and a class predictor, matching their embeddings to concept and class anchors to make predictions.The concept predictor generates concept embeddings; the class predictor generates class embeddings from them.Source: §4 · pp. 3–4

4.1. Probabilistic Concept Modeling

  • Each concept is represented by a Gaussian embedding with a mean vector and diagonal covariance, predicted from a shared backbone by a concept-specific module.Concept existence probability is estimated from sampled representations using distances to trainable anchors; proximity to the positive anchor increases the probability.Source: §4.1 · p. 4

4.2. Probabilistic Class Modeling

  • Sampled representations from all concepts are concatenated and projected by a fully connected layer to produce the class representation.Class probabilities are obtained by applying softmax to logits based on distances between class representations and trainable class anchors, with Monte Carlo estimation.Source: §4.2 · p. 4

4.3. Training and Inference

  • The concept predictor uses binary cross-entropy and KL-divergence losses, while the class predictor uses cross-entropy; the predictors are trained separately.During class-predictor training, sampled concept embeddings can be replaced by anchors for ground-truth labels with a predefined probability; inference can use Monte Carlo sampling or concept means.Source: §4.3 · p. 4

4.4. Derivation of Uncertainty

  • Concept uncertainty is the geometric mean of diagonal covariance elements, while class uncertainty is measured by the determinant of the transformed covariance matrix.ProbCBM estimates uncertainty directly from the predicted distributions without sampling.Source: §4.4 · p. 5

4.5. Architecture

  • A shared backbone feeds one probabilistic embedding module per concept, each predicting that concept’s mean and diagonal covariance from the backbone feature map.Each module uses self-attention-based mean and variance heads, preceded by a fully connected layer, batch normalization, and ReLU.Source: §4.5 · p. 5

5. Experiments

Experiments on one synthetic and two real-world datasets evaluate prediction performance and uncertainty estimation across diverse visual contexts and transformations. They also test whether concept intervention can correct predictions and reduce class uncertainty.

5.1.1. DATASET AND EXPERIMENTAL SETTING

  • MNIST digits form four-concept images assigned to 12 classes, and ProbCBM achieves 99.8% concept prediction accuracy and 99.8% classification accuracy.Cutout augmentation simulates ambiguity, and the synthetic dataset has instance-level concept annotations.Source: §5.1.1 · p. 5

5.1.2. UNDERSTANDING ESTIMATED UNCERTAINTY

  • On synthetic images, occluding a digit raises the corresponding concept uncertainty and class uncertainty; the occluded sample’s concept embedding appears as a larger ellipse.In diverse visual contexts, uncertainty is higher when neither member of a paired concept group is present.Source: Figure 5 · p. 5, §5.1.2 · p. 6

5.2.1. DATASETS AND EXPERIMENTAL SETTING

  • CUB uses 112 denoised attributes and AwA2 uses 45 visible attributes as concepts; experiments use pretrained ImageNet-1K ResNet18 and mostly 224 × 224 images.CUB comparisons also use 299 × 299 images.Source: §5.2.1 · p. 6

5.2.2. COMPARISONS

  • ProbCBM outperforms CBM in classification while maintaining comparable concept accuracy, and detects occlusion-driven uncertainty increases better than MC dropout.Against CEM, it has lower class but slightly higher concept performance; CEM has 15.6M parameters versus ProbCBM’s 12.5M and uses joint training to encode additional information.Source: Table 1 · p. 6, Figure 6 · p. 6, §5.2.2 · p. 7
  • ProbCBM’s classification accuracy is higher than or comparable to its deterministic-embedding variant, and mean-only inference differs only slightly from sampling.The model uses probabilistic class embeddings and class anchors for prediction; the table caption identifies the deterministic and no-sampling variants.Source: Table 1 · p. 6, §5.2.2 · p. 7

5.2.3. UNDERSTANDING ESTIMATED UNCERTAINTY

  • Across CUB and AwA2, concept uncertainty varies with visual context, and stronger cutout or hue transformations increase both concept and class uncertainty.In the leg-color embedding visualization, images with invisible legs form larger ellipses between the positive and negative anchors.Source: §5.2.3 · p. 8, Figure 8 · p. 8, Figure 9 · p. 8
  • Groups with greater estimated uncertainty show worse concept and class prediction performance, supporting uncertainty as an estimate of prediction failure probability.The test set is divided into ten equal-sized groups by uncertainty.Source: §5.2.3 · p. 8, Figure 10 · p. 8

5.2.4. CONCEPT INTERVENTION

  • Uncertainty-ranked interventions improve class accuracy faster than random-order interventions, which eventually reach 100%; intervening in more concept groups lowers class uncertainty.Interventions replace sampled concept points with ground-truth anchors; concepts are grouped by similarity, with 28 groups for CUB.Source: §5.2.4 · pp. 8–9, Figure 11 · p. 9

6. Limitations

ProbCBM inherits CBM’s dependence on human-annotated concept labels and remains vulnerable to information leakage. Modeling uncertainty can reduce dependence on label correctness, but leakage remains a reliability issue.

6. Limitations

  • CBM still requires concept labels, and concept-prediction accuracy depends heavily on the quality of human annotations.ProbCBM uses those labels, though introducing uncertainty can reduce its dependency on their correctness.Source: §6 · p. 9
  • Information leakage—using unintended information in task prediction—can harm reliability when that information is not human-interpretable.The authors identify leakage as an additional CBM reliability problem that still needs to be resolved.Source: §6 · p. 9
  • Uncertainty modeling addresses reliability in terms of uncertainty, but does not resolve the separate information-leakage problem.The paper frames leakage as an issue requiring further resolution within CBM.Source: §6 · p. 9

7. Conclusion

The conclusion presents ProbCBM as modeling concept ambiguity with probabilistic embeddings and deriving class uncertainty from probabilistic concept embeddings. Its two-anchor design supports more reliable uncertainty estimates than using only a positive anchor.

7. Conclusion

  • ProbCBM represents concept ambiguity as concept uncertainty with probabilistic embeddings and derives class uncertainty from probabilistic concept embeddings.The authors report analyses showing improved reliability of explanations.Source: §7 · p. 9

A. Usage of two anchors for concept prediction

  • With only a positive anchor, negative samples have much larger σ, making embedding volume an unreliable uncertainty estimate.The positive anchor trains concept-positive samples to be close and concept-negative samples to be far from that anchor.Source: A. Usage of two anchors for concept… · p. 12
  • Separate positive and negative anchors provide proximity and distance targets, allowing probabilistic-embedding σ to represent uncertainty reliably.The approach contrasts with concept prediction using an exclusively positive anchor.Source: A. Usage of two anchors for concept… · p. 12

B. Discussion on data augmentation and ambiguity

Data augmentation can both undermine concept labels and introduce diverse visual contexts that make concept presence ambiguous. ProbCBM models and estimates uncertainty to address this ambiguity and mitigate the adverse effects of diverse contexts on concept prediction.

B. Discussion on data augmentation and ambiguity

  • Augmentation can make concept labels incomplete by removing object parts through cropping or distorting colors through strong color jittering.Concept labels are more susceptible to augmentation than class labels, so the authors call for techniques that preserve their integrity across concept types.Source: B. Discussion on data augmentation and… · p. 12
  • Diverse visual contexts introduced by augmentation can create ambiguity that challenges reliable concept prediction.The discussion notes that label incompleteness and context-driven ambiguity are related but distinct augmentation concerns.Source: B. Discussion on data augmentation and… · p. 12
  • ProbCBM models and estimates uncertainty to mitigate the adverse effects of diverse visual contexts on concept-prediction reliability.The paper presents uncertainty estimation as its approach to concept prediction in the presence of ambiguity.Source: B. Discussion on data augmentation and… · p. 12

C.1. Synthetic dataset based on MNIST dataset

The appendices describe synthetic MNIST-based data construction and implementation choices for CUB and AwA2, including dataset-specific augmentation, resizing, and concept grouping.

C.1. Synthetic dataset based on MNIST dataset

  • Synthetic images combine four randomly ordered MNIST digits at 56 × 56, with one digit dropped at probability 0.5; Figure 13 shows examples.The underlying MNIST splits contain 60,000 training images and 10,000 test images.Source: §C.1 · p. 12, Figure 13 · p. 13
  • The generated synthetic splits contain 68,017 training and 11,244 test images; Table 2 pairs 12 class labels with concepts.The generated samples come from images in the corresponding official MNIST splits.Source: §C.1 · p. 12, Table 2 · p. 12
  • Synthetic-dataset training uses cutout with a size of 7 × 7.Source: §C.1 · p. 12

C.2. CUB

  • CUB training uses color jittering, horizontal flips, and constrained random scaling and cropping; concepts are grouped by shared name prefixes for intervention.Images are resized to 256 × 256 before cropping to 224 × 224, or to 341 × 341 before cropping to 299 × 299.Source: §C.2 · p. 13

C.3. AwA2

  • For AwA2, images are cropped without prior resizing during training, then resized to 224 × 224 for inference; Table 3 lists 45 attributes and their intervention groups.Source: §C.3 · p. 13, Table 3 · p. 13

D. Experimental details

ProbCBM uses dataset-specific embedding dimensions and training schedules, while the other models use a shared ResNet18-based comparison setup. The training details also specify optimizer settings and a sequential concept-then-class training procedure for CBM baselines.

D.1. Details for ProbCBM

  • For real-world datasets, ProbCBM uses Dc=16, Dy=128, 50 concept-predictor epochs, and 20 class-predictor epochs; synthetic settings use Dc=16, Dy=32, and 30 and 20 epochs, respectively.Ns is 50, and pretrained ImageNet-1K weights are fixed for the first five epochs; learnable scaling parameters a and d are initialized to 5 and 10.Source: §D.1 · p. 14
  • ProbCBM uses AdamP with cosine learning-rate scheduling, rates of 10^-3 for pretrained weights and 10^-2 for random weights and learnable parameters, and λKL=5 × 10^-5.Source: §D.1 · p. 14

D.2. Details for other models

  • All models use ResNet18; CBM variants add concept and class fully connected predictors, with MC dropout applied after each convolutional block at rate 0.2.CBM is trained sequentially, and its class predictor receives ground-truth concepts with probability preplace=0.5; data augmentation is shared across models.Source: §D.2 · p. 14

E. Training scheme

The class predictor is trained using sampled concept embeddings and a replacement strategy, and ablations assess replacement probability and embedding dimensions. On CUB, performance is largely stable across these settings, while very small concept embeddings weaken uncertainty-change detection.

E. Training scheme

  • The class-predictor training scheme samples concepts from p(zc|x), applies a Bernoulli replacement strategy, and updates the predictor from class loss until convergence.It concatenates sampled concept embeddings and computes class probabilities; the algorithm follows the training scheme described in Section 4.3.Source: E. Training scheme · p. 14

F.1. Ablation studies

  • The CUB ablations vary replacement probability and concept and class embedding dimensions, assessing classification and concept-prediction performance as well as uncertainty response to occlusion.Dc denotes the concept embedding dimension and Dy the class embedding dimension.Source: §F.1 · p. 15, Figure 15 · p. 15, Figure 16 · p. 15
  • CUB classification performance shows no significant difference across preplace values except at preplace=1, which matches CBM's independent learning strategy.At preplace=1, the class predictor is trained on ground-truth concepts and class labels regardless of concept predictions.Source: §F.1 · p. 15, Figure 14 · p. 15
  • When Dc=4, ProbCBM's ability to detect increased uncertainty diminishes; detection converges as Dc grows beyond a threshold.The authors interpret this as indirect evidence that higher-dimensional concept embeddings positively affect uncertainty estimation.Source: §F.1 · p. 15, Figure 16 · p. 15

F.2. Additional results

Additional results show uncertainty responses to occlusion and image transformations, relate concept uncertainty to class uncertainty, and describe further performance, intervention, and prediction visualizations. The supplementary examples extend these analyses across AwA2, CUB, and synthetic data.

F.2. Additional results

  • On AwA2, ProbCBM detects increased concept and class uncertainty across samples after a centered 64 × 64 gray occlusion.Figure 17 reports the proportions of samples with increased uncertainty, with means and standard deviations over three repetitions.Source: §F.2 · p. 16, Figure 17 · p. 16
  • After image transformations, concept uncertainty increases for most samples, and stronger transformations raise the proportion with significantly increased uncertainty.Figures show example transformed images and violin plots of uncertainty changes.Source: §F.2 · p. 16, Figure 18 · p. 16, Figure 19 · p. 17
  • Images with large concept uncertainties also have large classification uncertainties in the visualized class-embedding space.Figure 20 visualizes the class embeddings in 2D using four images from Figure 9.Source: §F.2 · p. 16, Figure 20 · p. 17
  • AwA2 plots relate performance to median uncertainty for groups of similar-uncertainty samples and show concept intervention in random and uncertainty-based orders.Intervention operates on 15 concept groups; the uncertainty-ordered plots report mean and standard deviation of class uncertainty across test samples.Source: §F.2 · p. 16, Figure 21 · p. 18, Figure 22 · p. 18

F.3. Additional examples

  • Supplementary examples show occlusion-related concept and class uncertainty changes on synthetic data and ProbCBM predictions on CUB and AwA2.P and U denote probability and uncertainty, respectively; occluded image parts are boxed in red.Source: §F.3 · p. 16, Figure 23 · p. 18, Figure 24 · p. 19

Related papers

Loading 2306.01574v1…