Are Concept Bottleneck Models Effective as Decision-Support Systems?

• CBMs base predictions on human-understandable concepts and let users inspect or modify concept values to examine resulting predictions [Koh et al., 2020].This design is intended to help users detect concept errors and assess prediction robustness.Source: §1 · p. 1, Figure 1 · p. 2
• The work addresses limited evidence on whether CBMs improve decision support, including users’ reliance, confidence, and trust in model predictions.Source: §1 · p. 2
• Two user studies compare support ranging from none to labels, detected concepts, and interactive concept intervention across email-fraud and bird-species classification tasks.The studies involve 705 participants and use tasks differing in familiarity and the objectivity of concept identification.Source: §1 · p. 2
• CBMs, especially their interactive component, can outperform unaided humans and non-interpretable AI support, but benefits require difficult tasks, easily identifiable concepts, and active model interaction.Inaccurate concept detection may also undermine users’ trust in the model.Source: §1 · p. 2
• The paper introduces a cross-dataset paradigm for evaluating collaboration through team accuracy, users’ decision confidence, and trust, and derives practical deployment guidance.Source: §1 · p. 2

Concept Bottleneck Models

• CBMs [Koh et al., 2020] combine a concept encoder that maps inputs to concept activations with a task predictor that maps those activations to labels.The encoder produces interpretable concept probabilities, while the predictor uses them in a linear mapping to generate the final classification.Source: §2 · p. 2
• CBMs can train the concept encoder and task predictor independently, sequentially, or jointly, typically using dense supervision for concepts and labels.Independent training separates the components; sequential training fits the predictor after the encoder; joint training optimizes both together.Source: §2 · pp. 2–3
• At inference, users can inspect detected concepts, override any subset, and receive an updated label prediction from the modified concept vector.The original prediction is y = f(g(x)); after intervention, it becomes ỹ = f(ĉ).Source: §2 · p. 3

3 Present Work

• The paper evaluates CBM-supported binary classification in two studies: fraudulent-versus-legitimate emails and Le Conte’s-versus-Savannah sparrows.The final samples included 363 participants in Study 1 and 342 in Study 2; materials and data are provided in a supplement.Source: §3 · p. 3, §3.2 · p. 6

3.1 Datasets and CBMs specifics

• The CBMs use six human-interpretable concepts and a frozen neural encoder with binary SVM concept classifiers feeding a shallow logistic task predictor.The PhishFuzzer and CUB CBMs achieved 92.3% and 81.4% test accuracy, respectively.Source: §3.1 · pp. 3–4

3.2 Methodology

• Participants are randomly assigned to one of four conditions: no support, label-only, non-interactive concepts, or interactive concepts.Participants receive eight items correctly classified by the model and two incorrectly classified items; the model’s accuracy is 80% in the three AI-supported conditions.Source: §3.2 · p. 4
• The study measures participants’ accuracy and confidence, intervention frequency in the interactive condition, and trust in AI-supported conditions.Analyses use mixed-effects models for accuracy, confidence, and intervention behavior, plus nonparametric tests for trust, with Bonferroni correction for multiple comparisons.Source: §3.2 · pp. 4–5

4 Study 1 – Fuzzy concepts in a familiar task

• Study 1 used a familiar phishing-email task in which participants judged emotional and goal-oriented cues that were not entirely clear-cut or objective.Participants classified 10 emails as legitimate or fraudulent, with items drawn from a pool designed to vary topic diversity and difficulty.Source: §4 · p. 6

4.1 Results

• Participants’ overall classification accuracy was 76% without support, versus 81% with label-only support and 83% with non-interactive or interactive CBMs, showing limited accuracy benefits over label-only support.Source: §4.1 · p. 7
• For incorrectly classified items, participants were more accurate with non-interactive CBM support than label-only support, achieving 68% versus 55%.The interaction between support condition and CBM classification accuracy was significant (p = .002).Source: §4.1 · p. 7
• Participants intervened more often when detected concepts disagreed with annotations, but intervention trials reached 82% accuracy versus 83% without intervention.Interventions averaged 6.87 across 10 items, occurred in 22% versus 9% of mismatched versus matched cases, and did not improve accuracy; 36% never intervened.Source: §4.1 · p. 7
• CBM support did not change classification confidence, while active interaction may have reduced perceived model capability, with trust ratings of 3.5 versus 4.0.Overall confidence differences were nonsignificant, and the trust difference emerged after excluding non-interactive participants in the interactive condition.Source: §4.1 · p. 7

5 Study 2 – Well-defined concepts in an unfamiliar task

• The study used an unfamiliar task classifying birds as one of two sparrow species from readily observable cues such as breast color.The design selected items where bottleneck concepts were clearly visible and manually verified the corresponding annotations.Source: §5 · p. 7, §5.1 · p. 7

5.1 Results

• 82% and 83% accuracy in NIC and IC exceeded 73% with no support, whereas label-only support yielded 79% and did not differ significantly from no support.The result held for correct CBM predictions, with AI-supported accuracy ranging from 85% to 90% versus 74% without support.Source: §5.1 · p. 7
• Interactive users outperformed label-only users after excluding non-interactors, reaching 86% accuracy, primarily by resisting incorrect CBM labels.Accuracy was 65% versus 60% on items with incorrect CBM predictions, while correct-prediction accuracy was 91% versus 89%.Source: §5.1 · p. 8
• Interactive CBM support increased classification confidence relative to no support, with median confidence scores of 4 versus 3.Other pairwise confidence comparisons were not significant.Source: §5.1 · p. 8
• Interactive users who engaged with concepts rated the system as less capable than a novice human than label-only users, with median ratings of 4 versus 5.Overall trust and individual questionnaire-item differences across AI-supported conditions were otherwise not significant.Source: §5.1 · p. 8

Concept Bottleneck Models

• Recent CBM research targets human-aligned concepts, sufficient statistics, interpretable predictors, intervention policies, human involvement, and vision-language concept extraction.These studies primarily address structural components and key properties, whereas this work evaluates CBMs’ efficacy in hybrid decision-making.Source: §6 · p. 8

Concept-based XAI user studies

• Concept-based explanations are generally judged informative, interpretable, and helpful for predicting or critically evaluating outputs, but most studies omit human–AI team performance and user confidence [Buçinca et al., 2020, Senoner et al., 2024].Interaction studies have examined revising incorrect concept predictions [Bontempelli et al., 2023, Michalski et al., 2025], while Das et al. [2023] and Furby et al. [2025] are notable performance-oriented exceptions.Source: §6 · p. 8
• Das et al. [2023] found concept-based explanations improved performance across two game-like tasks, but sequential decisions, training-oriented evaluation, no concept intervention, and small samples limited decision-support relevance.Participants completed unaided sessions before and after supported play, with 15 participants per condition.Source: §6 · p. 8
• Furby et al. [2025] found concept interaction could improve human–AI team performance, particularly at lower model accuracy, but small samples, an atypical task, and problematic sliders limited interpretation.Their studies also lacked a no-support baseline; the expert study additionally omitted label-only and non-interactive concept conditions, preventing effects from being disentangled.Source: §6 · p. 9

Conclusion

• CBMs, particularly their interactive component, improve human–AI team performance beyond unaided humans and non-interpretable AI support.Source: §7 · p. 9

Conditions for effectiveness

• Benefits are more likely when users are uncertain about classifications, concepts are objectively grounded and understandable, and users actively modify concept values.Users confident in their own judgments have less reason to rely on CBM explanations, while fuzzy concepts can reduce trust; approximately one third never modified concepts in the interactive condition.5Source: §7 · p. 9

Limitations

• The studies examined only binary classification, leaving multi-class effectiveness unresolved because lower confidence may help while additional concepts may increase cognitive load.Flexible concept-subset presentation may reduce this burden without sacrificing concept-based reasoning benefits Barker et al. [2023], Schrodi et al. [2025].Source: §7 · p. 9
• Frequently misidentified concepts may undermine trust, cause users to discount predictions, and reduce CBMs’ decision-support benefits.Unreliable concept annotations may substantially compromise CBMs’ potential even when classification performance is good.Source: §7 · p. 9

CBM specifics

• The CBM follows independent training, with the concept extractor and task predictor trained separately, following Koh et al. [2020].Source: CBM specifics · p. 15
• The concept extractor combines a frozen neural encoder with n_c independent binary SVMs whose decision scores represent concept presence or absence.Each SVM uses an RBF kernel, C=1.0, and balanced class weights.Source: CBM specifics · p. 15
• CUB uses CLIP ViT-L14 [Radford et al., 2021], whereas PhishFuzzer uses a pretrained sentence transformer [Reimers and Gurevych, 2019] based on Wang et al. [2020].The encoder is the only architectural difference; subsequent concept extraction is identical across datasets.Source: CBM specifics · p. 15
• The task predictor is bias-free logistic regression over concept vectors encoded in {−1,+1}^n_c, allowing concept absence to provide predictive evidence.This signed representation matches SVM outputs and explicitly represents absent concepts, such as missing fear-related features in legitimate emails.Source: CBM specifics · p. 15
• At test time, concept logits pass through tanh to produce [−1,1]^n_c activations, whose task-predicted positive class is selected at probability ≥0.5.The positive class is legitimate for PhishFuzzer and Savannah sparrow for CUB.Source: CBM specifics · p. 15
• Users can intervene by toggling any concept between active and inactive states, avoiding intermediate confidence values to reduce ambiguity and improve interface consistency.Binary interventions are especially suitable for visually identifiable attributes in Study 2.Source: CBM specifics · p. 15
• Each intervention flips the concept activation’s polarity, while clicking again restores the model’s original predicted value.Thus, users can override discrepant concept predictions without assigning continuous activation levels.Source: CBM specifics · p. 15

PhishFuzzer

• PhishFuzzer retains 1,064 cleaned emails for concept prediction but excludes 513 Spam emails when training the task predictor.This separates concept-predictor data expansion from the binary task-prediction dataset.Source: Datasets · p. 16

CUB

• CUB uses 4,745 images from 198 bird classes for concept prediction, yielding 28,470 annotations, while task prediction uses only Le Conte’s and Savannah Sparrow images.The full 4,796-image, 200-class training set excludes the two task classes for concept prediction and reserves them for binary task classification.Source: Datasets · p. 16

Power analysis and recruitment

• The power analysis required 340 participants—85 per AI-support condition—to detect a small three-way interaction with 82% statistical power.Recruitment continued until each condition had at least 85 eligible participants; inclusion required native UK English speakers with Prolific approval above 98%.Source: A priori power analysis and data… · p. 17

Exclusions

• Study 2 excluded 209 of 551 participants (38%), mainly because 196 failed attention checks, compared with 38 of 401 (9%) in Study 1.Exclusions were especially frequent in No-support (130), versus Label-only (38), Non-interactive concepts (19), and Interactive concepts (9).Source: A priori power analysis and data… · p. 17

Assignment procedure

• After the first 400 randomly assigned participants, the remaining 151 were assigned to specific conditions to ensure at least 85 eligible participants per condition.This adjustment addressed costly recruitment and potential imbalance caused by Study 2’s high, especially No-support, exclusion rate.Source: A priori power analysis and data… · p. 17

Robustness check

• Analyses restricted to the initial 400 fully randomized participants produced the same results as analyses including the adaptively assigned participants.Thus, the post hoc assignment restriction did not change the reported findings.Source: A priori power analysis and data… · p. 17

Trust scale

• The trust scale was adapted from Hoffman et al. [2023], with three items reworded to invert polarity and reduce acquiescence bias; items were randomized.The scale assessed confidence, predictability, reliability, safety, efficiency, wariness, comparative performance, and liking for decision making.Source: Trust scale · p. 18
• Items assessed confidence, reliability, safety, and whether the system could outperform a novice human user.Example statements concerned confidence that the system works well, reliability, safe reliance, and comparative task performance.Source: Trust scale · p. 18
• Additional items assessed unpredictability, inefficiency, wariness, and dislike of using the system for decision making, with each response recorded on a 7-point Likert scale.The scale ranged from “I strongly disagree” to “I strongly agree.”Source: Trust scale · p. 18

Previous experience with AI systems

• Participants reported four levels of prior AI experience, ranging from little or no experience to developing AI systems professionally or personally.The study measured these categories at the end of both studies and reported their distributions in Table S1.Source: Previous experience with AI systems · p. 18

Highest and lowest confidence ratings in Studies 1 and 2

• In Study 1, highest confidence accounted for 25% of responses and lowest confidence for 3%, with neither frequency differing across experimental conditions.Highest-rating frequencies had p = .117, and lowest-rating frequencies had p = .504 across NS, LO, NIC, and IC.Source: Highest and lowest confidence ratings… · p. 19
• In Study 2, the highest confidence rating was less frequent in NS than IC, while the lowest rating was more frequent in NS than NIC and IC.The highest-rating contrast was p = .031; the lowest-rating contrasts were p < .001 for NS versus NIC and p = .004 for NS versus IC.Source: Highest and lowest confidence ratings… · p. 20

Impact of inaccurate concept detection on trust

• Concept-detection errors reduced participants’ likelihood of following correct CBM predictions in NIC, whereas interaction largely mitigated this effect in IC.The interaction was significant, χ2(1) = 4.12, p = .042; NIC had OR = 0.72, whereas IC had OR = 1.12. The analysis was exploratory and likely underpowered.Source: Impact of inaccurate concept detection… · p. 22

Concepts and emails selection

• The studies used six relabeled, human-understandable email concepts and selected a balanced, diverse stimulus pool from PhishFuzzer by difficulty and model correctness.The final pool comprised six misclassified emails and 24 correctly classified emails, with approximately half of the difficulty-balanced items easier and half more challenging.Source: Concepts and emails selection · p. 24

CBM details

• The email CBM encoded messages with all-MiniLM-L6-v2, used six binary linear SVM concept classifiers, and predicted labels with balanced logistic regression.The concept extractor was trained independently per concept, while the label predictor used fit_intercept=False, no penalty, and class balancing.Source: CBM details · p. 25

Instructions

• Participants classified anonymized emails, viewed the AI’s label and detected concepts, rated confidence, and could toggle concepts to examine whether the classification changed.The interface displayed concepts as detected or not detected and updated the model’s classification after interventions.Source: Instructions · pp. 26–27

Concepts and images selection

• Study 2 selected six interpretable sparrow concepts and retained only images in which their corresponding body regions were clearly visible; a pilot found 73% unaided accuracy versus 82% with label-only support.The concepts distinguished Le Conte’s and Savannah sparrows, and their labels were slightly revised for participant interpretability.Source: Concepts and images selection · p. 37

Tables of statistical analyses results

• 100% accuracy was achieved by the logistic-regression task predictor when trained and evaluated using ground-truth concepts, establishing the perfect-concept upper bound for Study 2.The statistical analyses also modeled participants’ accuracy, intervention behavior, confidence, and trust across AI-support conditions and model or concept accuracy.Source: Table 14 · p. 33, Table 18 · p. 35, Concepts and images selection · p. 37

Instructions

• The bird-classification interface showed six detected features alongside the AI’s prediction and let participants alter feature states to examine how classifications changed.Participants were instructed not to blindly trust the system and to adjust features when their own assessment differed from the model’s detected concepts.Source: Instructions · pp. 40–41
• The parkour analyses examined whether model concept-detection accuracy predicted participants’ intervention behavior, both including and excluding non-interactive participants.These relationships were evaluated with mixed-effects logistic regressions, with corresponding result tables and an intervention-proportion figure.Source: Instructions · p. 46, Table 28 · p. 46, Figure 9 · p. 46
• Additional mixed-effects logistic regressions examined how intervening on concept values related to classification accuracy, both for all participants and after excluding non-interactive participants.The analyses used interaction behavior—whether participants intervened at least once during a trial—as a fixed effect.Source: Instructions · p. 47, Table 30 · p. 47, Table 31 · p. 47
• The section also analyzed participants’ confidence across AI-support conditions, separately for all participants and after excluding non-interactive participants.Confidence was modeled with mixed-effects ordinal regressions and illustrated by confidence ratings ranging from 0 to 6.Source: Instructions · pp. 48–49, Table 32 · p. 48, Figure 10 · p. 48
• Self-reported trust was compared across the three AI-support conditions after excluding non-interactive participants.Kruskal–Wallis tests covered the trust index and individual questionnaire items, with adjusted pairwise tests for Item 7.Source: Instructions · p. 50, Table 34 · p. 50, Table 35 · p. 50

Computing infrastructure

• All experiments ran on a Linux server, using CPU-based SVM and logistic-regression training; CLIP and sentence-transformer encoding was the only GPU-dependent step.Hardware and software configurations are documented in Tables 36 and 37.Source: Computing infrastructure · p. 51, Table 36 · p. 51, Table 37 · p. 51