Source-linked AI summary
Post-hoc Concept Bottleneck Models
Mert Yuksekgonul, Maggie Wang, James Zou
TL;DR
Conventional CBMs require dense concept annotations and may sacrifice the accuracy of unrestricted models. PCBMs instead retrofit pretrained networks with interpretable concept bottlenecks using concepts from other datasets or language, while residual modeling addresses incomplete concept libraries. They achieve comparable performance to original models in the reported experiments and support global concept-level edits, including user-guided improvements under distribution shift.
Problem
CBMs require dense concept annotations during training and often do not match unrestricted-model accuracy, limiting their practical use.
Method
PCBMs retrofit pretrained models with concept subspaces learned from annotated examples or multimodal natural-language descriptions, then use interpretable predictors and optional residual modeling.
Results
PCBMs achieve comparable performance to original models across reported datasets except CIFAR100, while PCBM-h matches the original model in all scenarios.
Takeaways & Limitations
PCBMs enable global concept-level model edits, and user-guided pruning improves accuracy under distribution shift without target-domain data or model retraining.
Takeaways & Limitations
Editing requires knowledge of the spurious concept in the training domain, which may not be realistic in practice.
Abstract
from arXiv · showhide
Concept Bottleneck Models (CBMs) map the inputs onto a set of interpretable concepts (``the bottleneck'') and use the concepts to make predictions. A concept bottleneck enhances interpretability since it can be investigated to understand what concepts the model "sees" in an input and which of these concepts are deemed important. However, CBMs are restrictive in practice as they require dense concept annotations in the training data to learn the bottleneck. Moreover, CBMs often do not match the accuracy of an unrestricted neural network, reducing the incentive to deploy them in practice. In this work, we address these limitations of CBMs by introducing Post-hoc Concept Bottleneck models (PCBMs). We show that we can turn any neural network into a PCBM without sacrificing model performance while still retaining the interpretability benefits. When concept annotations are not available on the training data, we show that PCBM can transfer concepts from other datasets or from natural language descriptions of concepts via multimodal models. A key benefit of PCBM is that it enables users to quickly debug and update the model to reduce spurious correlations and improve generalization to new distributions. PCBM allows for global model edits, which can be more efficient than previous works on local interventions that fix a specific prediction. Through a model-editing user study, we show that editing PCBMs via concept-level feedback can provide significant performance gains without using data from the target domain or model retraining.
1 INTRODUCTION
Concept bottlenecks make model reasoning inspectable through human-understandable concepts, but conventional CBMs require dense annotations, may reduce accuracy, and lack a clear path for global model editing. PCBMs address these challenges by converting pretrained models into interpretable bottlenecks while leveraging concepts from other sources and recovering performance when concepts are incomplete.
- Concept bottlenecks expose the information used for classification through interpretable predictor weights or rules and concept-prediction errors.
- CBMs require concept labels during training, although real-world training datasets rarely provide such annotations.
- CBMs often fail to match unrestricted-model accuracy when available concepts are insufficient, leaving performance recovery unclear.
- Prior CBM interventions change behavior for a single sample, whereas holistic model editing remains unshown.
- PCBMs convert pretrained models into concept bottlenecks, use concepts from other datasets or natural-language descriptions, and add residual modeling when concepts are insufficient.
- Experiments report comparable performance to black-box models and demonstrate global concept-level interventions for model improvement.
2 POST-HOC CONCEPT BOTTLENECK MODELS
PCBMs build an interpretable predictor on a fixed concept subspace extracted from any pretrained backbone, with concepts learned from examples, multimodal text embeddings, or knowledge-graph relations. A sequential residual predictor can compensate for information missing from the concept bank while preserving the trained concept bottleneck.
- A PCBM first uses any pretrained backbone to produce embeddings, then constructs a concept subspace and trains an interpretable predictor on projected embeddings.The backbone may be a CLIP image encoder or a ResNet up to its penultimate layer.
- Learning the Concept Subspace: CAVs are learned by training a linear SVM on positive and negative concept examples, which need not come from the backbone’s original training data.The vector normal to the SVM boundary represents each concept.
- Leveraging multimodal models to learn concepts: Multimodal models map natural-language concept descriptions into the shared embedding space, avoiding labeled data for constructing the concept subspace.For example, CLIP’s text encoder maps the prompt “stripes” to a concept vector.
- Leveraging multimodal models to learn concepts: ConceptNet supplies task-relevant concepts through hasA, isA, partOf, HasProperty, and MadeOf relations to build the concept subspace.
- Learning the Interpretable Predictor: The interpretable predictor maps fixed concept projections to labels using models such as sparse linear predictors or decision trees.Sparse linear models are used in this work, with regularization controlling model complexity.
- Recovering the original model performance with residual modeling: When the concept bank is insufficient, PCBM-h sequentially fits a residual predictor while keeping the concept subspace and interpretable predictor fixed.The residual is intended to compensate for information missing from the concept bank and recover the original model’s accuracy.
3 EXPERIMENTS
Across five datasets, PCBMs generally retain the original model’s performance while providing concept-based explanations, including with concepts transferred from other datasets or generated from CLIP text representations. PCBM-h recovers performance when the concept bank is insufficient, while CLIP concepts are more reliable for natural-image than specialized medical concepts.
- Performance: PCBMs achieve comparable performance to the original model across five datasets, except CIFAR100, where PCBM-h recovers the original accuracy.The evaluation reports AUROC for HAM10000 and ISIC, mAP for COCO-Stuff, and accuracy for CIFAR and CUB.
- Performance: PCBMs match the original model on HAM10000 and ISIC using only 8 human-interpretable concepts.Original CBMs cannot be trained on most evaluated datasets because their training data lack dense concept labels.
- CLIP concepts: CLIP-generated concept bottlenecks almost recover original accuracy on CIFAR10 and reduce the performance gap on CIFAR100.Concept descriptions are obtained through ConceptNet, while CLIP text embeddings provide the concept subspace.
- CLIP concepts: CLIP represents natural-image concepts effectively but is less reliable for granular or domain-specific concepts such as Blue Whitish Veils.The authors identify this limitation as reducing applicability to specialized medical domains.
- Explanations: PCBM explanations expose class-associated concepts, such as malignant-lesion indicators and Rose-related concepts, consistent with medical knowledge.Figure 2 reports the three largest linear-layer weights for shown classes.
- Explanations: PCBM reveals co-occurrence biases in COCO-Stuff, while its concept weights should not be interpreted as causal effects.Causal estimation would require accounting for interactions between concepts.
4 MODEL EDITING WITH POST-HOC CONCEPT BOTTLENECKS
PCBMs support global concept-level edits that address spurious correlations under distribution shift, including edits guided by users rather than target-domain data or retraining. Across controlled and user-study experiments, simple pruning strategies improve performance and recover substantial fractions of oracle gains.
- Global model editing: PCBMs enable global edits by manipulating concept weights, unlike local interventions that change behavior for only one sample.The editing framework targets entire-model behavior, including adaptation to a new distribution.
- Editing strategies: Prune sets a selected concept weight to zero, while Prune+Normalize rescales remaining weights to preserve the original weight norm.Fine-tuning on the test domain serves as an oracle comparison requiring target-domain samples.
- Controlled experiment: Almost 50% of the accuracy gains from fine-tuning are recovered by simple concept-subspace edits across 10 MetaShift distribution-shift experiments.For table(dog), the dog concept weight is pruned for the table class; PCBM-h gains are lower because residual information can retain the spurious concept.
- Practical boundary: The controlled edit strategy requires knowing the spurious concept in the training domain, which may be unrealistic in practice and motivates human guidance.The method otherwise avoids fine-tuning and target-domain knowledge or data.
- User study: Human-guided pruning improves model accuracy without revealing accuracy feedback during selection, using concepts displayed by their ten most positive weights for the shifted class.The study involved 30 volunteers across 9 scenarios, totaling 270 experiments.
- User study: 3.65±0.39 concepts are pruned in 34.3±6.4s per scenario, and all 30/30 users improve accuracy when averaged over scenarios.Users improve accuracy in all scenarios in 8/30 cases and in at least 6 scenarios in 26/30 cases.
- User study: User pruning reaches around 50% of fine-tuning gains and 80% of greedy-pruning gains while surpassing random pruning.Greedy pruning and retraining are treated as oracles because they require leaked knowledge of the test domain.
5 RELATED WORKS
Related work studies how neural networks encode and use human-understandable concepts and how models can be edited. CBMs provide concept-based interpretability but require concept labels throughout the training dataset, while prior CBM work had not analyzed model edits.
- Concepts: Concept-analysis research examines whether neural networks encode and use human-understandable concepts, including through counterfactual explanations and causal-validity evaluations.Other work automatically discovers concepts used by models.
- Concept-based models: CBMs first predict concepts and then use them to predict targets, providing interpretability but requiring concept labels for the entire training dataset.The passage identifies this annotation requirement as a key limitation.
- Model editing: Prior CBM work discussed interventions but had not analyzed model edits as a model-editing problem.This positions PCBM editing relative to both concept-bottleneck and broader model-editing research.
- Model editing: Model-editing research modifies neural-network information through fine-tuning or separate networks, including work on factual knowledge in language models.These approaches aim to remove or modify information while preserving selected model behavior.
6 LIMITATIONS AND CONCLUSION
The paper presents Post-hoc CBMs as a way to convert any model into a concept bottleneck while retaining performance and interpretability. It also identifies open questions about concept quality, larger-scale tasks, unsupervised concept discovery, and human involvement.
- Conclusion: Post-hoc CBMs convert any model into a CBM while retaining original performance and interpretability benefits.
- Conclusion: Multimodal models provide an interface for using concepts without laborious concept annotation.
- Conclusion: PCBMs support global model interventions in addition to the local interventions available in CBMs.
- Limitations: The concept library must be expressive enough for the task, and the concept dataset may introduce biases.
- Limitations: It remains open whether human-constructed concept bottlenecks can solve larger-scale tasks such as ImageNet-level classification.
ETHICAL STATEMENT
The user study collected no user information and was determined by the institution’s IRB not to involve human subjects under 45 CFR 46.102(f).
- Ethical Statement: The user study collected no user information.
- Ethical Statement: The institution’s IRB determined that the research did not involve human subjects under 45 CFR 46.102(f) and required no review.
A TRAINING DETAILS
The experiments use specified datasets, regularization settings, concept sources, and training procedures. The PCBM-h analysis compares predictions and confidence across PCBM confidence levels.
- Hyperparameters: All models were trained for 10 epochs on a single NVIDIA-Titan Xp GPU, with ElasticNet sparsity ratio α = 0.99.
- Datasets: Metashift uses 50 training and 50 test images per class across five-class classification problems.
- Datasets: The ISIC experiment uses 2000 training images and a held-out test set of 500 images.
- Concept Learning: Visual concepts come from Broden, CUB, or derm7pt, using 50 positive-negative image pairs and a linear SVM to define each concept vector.
- Concept Learning: Multimodal concepts are obtained from ConceptNet relations and represented by CLIP text embeddings.
- Residual Analysis: PCBM-h changes predictions mostly at low PCBM confidence, where the original prediction is likely mistaken.
C COMPARISON TO CBM
On CUB, CBM achieves slightly better performance than PCBM and the original backbone when trained with dense concept annotations. However, varying-annotation experiments show that CBMs require substantially more annotations to reach PCBM accuracy.
- Residual Analysis: PCBM-h changes predictions to fix model mistakes rather than changing predictions indiscriminately.
- Residual Analysis: The effect of PCBM-h on model confidence decreases as the model becomes more confident.
- Comparison to CBM: On CUB, CBM achieves slightly better performance than PCBM and the original backbone when using dense concept annotations.This comparison uses the same concepts, linear predictor, and frozen ResNet18 backbone.
- Comparison to CBM: The CUB comparison costs 112 concept annotations for each training sample.
- Annotation Efficiency: CBM requires a much larger amount of annotation to achieve the same test accuracy as PCBM.For example, 11200 annotations correspond to 100 images when each image has 112 concepts.
D CONTROLLED METASHIFT EXPERIMENTS FOR MODEL EDITING
The Metashift editing experiments use controlled object-recognition tasks to evaluate PCBM model editing, with participants selecting concepts through a user-study interface and reviewing accuracy before and after editing.
- Experimental setup: The experiments use two 5-class Metashift object-recognition tasks with ResNet18 backbones and 100 images per class for bottleneck training.Both tasks use ImageNet-pretrained ResNet18 backbones, Adam optimization, and specified regularization settings.
- User study interface: The model-editing user study begins with a launch page explaining PCBMs and the editing-task instructions.
- User study interface: After nine scenarios, the summary page displays selected concepts alongside unedited-model and edited-model accuracy.
E.2 HUMAN-GUIDED PCBM EDITING: DETAILED PERFORMANCE RESULTS
The detailed user study evaluates human-guided PCBM editing across nine controlled scenarios involving spurious class correlations, reporting shifted-class and overall test accuracy after editing.
- Scenario design: Each of the nine scenarios is a 5-class object-recognition task in which one class has a spurious correlation in the training data.For example, training images for keyboard all contain a cat.
- Scenario design: Table 7 lists the underlying classification classes and the spurious correlation assigned to each user-study scenario.
- Performance evaluation: The reported evaluation includes test accuracy for both the shifted class and the overall classification task.
- Performance evaluation: Accuracy for the pruning strategies is averaged over users and reported as mean ± standard error.
F ANALYSIS ON THE COCO-STUFF BIASES
The COCO-Stuff analysis tests whether PCBM concepts identify dataset co-occurrence biases, finding several aligned contexts while noting that useful concepts must be present in the concept bank.
- Analysis setup: The analysis compares concepts from Singh et al. with the top-five highest-weight concepts identified by a PCBM trained with CLIP concepts.The table also records the frequently co-occurring biased context for each category.
- Bias identification: The PCBM surfaced biased contexts such as table for cup, fruit and fresh vegetables for apple, and traffic circle and intersection for car.
- Bias identification: The analysis also identified potential co-occurrence biases not listed in the comparison, including chimney for clock and toilet for hair drier.
- Limitation: A key limitation is that the desired concepts must exist in the concept bank, motivating automatic concept discovery as future work.