PaperlayerView exact revision on arXiv

Source-linked AI summary

CHOQOLATE: Organizing Concept Bottleneck Latent Spaces with Choquet Integrals

Rémi Kazmierczak, Johanne Cohen, Marianne Clausel

arXiv:2609.37786v2cs.CV

TL;DR

CLIP-based concept bottlenecks can be unfaithful because related concepts are entangled, making individual scores unreliable indicators of their intended meanings. CHOQOLATE uses 2-additive Choquet layers to organize correlated concepts into compact nodes. Across datasets it offers a favorable accuracy–latent-space trade-off, and its post-hoc concept removal performs on par with debiasing methods that require group annotations or retraining, while needing neither.

  • Problem

    CLIP-based concept scores can be unfaithful because related concepts are entangled, so an individual score may not reflect its intended meaning.

    Source: §1 · p. 1

  • Method

    CHOQOLATE uses 2-additive Choquet layers to organize correlated concepts into compact latent nodes with closed-form Shapley attributions.

    Source: §1 · p. 1, §4.1 · p. 4

  • Results

    Across four datasets and metrics, CHOQOLATE offers a favorable accuracy–latent-space-quality trade-off, and CHOQ-PCR performs on par with debiasing methods requiring group annotations or retraining.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    CHOQ-PCR suppresses spurious concepts after training without group annotations or retraining, while Choquet weights provide closed-form Shapley attributions for targeted edits.

    Source: §1 · p. 2, §8 · p. 7

    The quadratic dependence on vocabulary size becomes prohibitive for tens of thousands of concepts, limiting applicability to such large vocabularies.

    Source: §7 · p. 7

Abstract

from arXiv · show

Concept Bottleneck Models (CBMs) built on vision-language models such as CLIP represent a latent space as human-understandable concepts. These representations are unfaithful: related concepts are entangled, so individual scores do not reflect their intended meaning. We propose CHOQOLATE, an interpretable-by-design layer based on 2-additive Choquet integrals, which merges correlated concepts into compact nodes. Across four datasets, CHOQOLATE achieves a favorable accuracy-interpretability trade-off, with weight-sparse and semantically coherent nodes. A closed-form gradient derivation, backed by experiments, explains why Choquet layers drive this organization without explicit supervision. Choquet weights also map directly to Shapley values, which enables test-time intervention. On standard bias-mitigation benchmarks, suppressing spurious concepts after training performs on par with methods that require group annotations or retraining, while needing neither.

1 Introduction

CLIP-based concept bottlenecks can be unfaithful because related concepts are entangled, so individual scores may not reflect their intended meanings. CHOQOLATE reorganizes co-activating concepts into compact nodes with 2-additive Choquet layers, aiming for interpretable and predictive representations without supervising concept-to-node assignments.

1 Introduction

  • CLIP-based CBMs reduce the annotation burden by using image–text similarities, but entangled concepts can make individual scores unfaithful.For example, hair, muzzle, and tail similarities are correlated, so a tail concept may activate on a face-only cat image.Source: §1 · p. 1
  • CHOQOLATE uses two 2-additive Choquet layers to group concepts into latent nodes and map those nodes to class scores, without supervising the grouping.The layers are designed to produce sparse, semantically coherent, attributable, low-dimensional, and informative nodes.Source: §1 · p. 1
  • Across four datasets and metrics, CHOQOLATE offers a favorable trade-off between accuracy and latent-space quality.Its post-hoc concept removal performs on par with debiasing methods requiring group annotations or retraining, while requiring neither.Source: §1 · pp. 1–2
  • CHOQOLATE treats concept redundancy as structure to organize: each node connects to a small group of correlated concepts rather than mixing all inputs indiscriminately.The figure illustrates the correlated concepts spreading along a shared diagonal in the three-concept bottleneck.Source: §1 · p. 1, Figure 1 · p. 2

2 Related Work

Concept Bottleneck Models route predictions through human-understandable concepts, with CLIP similarities removing the need for per-image concept annotations. Related work addresses unfaithful concepts, sparse explanations, and spurious correlations through different representations and interventions; CHOQOLATE instead learns a hierarchy over CLIP similarities and supports post-hoc concept removal.

2.1 Concept Bottleneck Models

  • Concept Bottleneck Models route predictions through human-specified concepts, while text-image models such as CLIP can supply concept scores without explicit per-image annotations.Earlier attribute-based methods were constrained by the need for per-sample concept annotations; prior hierarchical CBMs often specify their structure manually, unlike CHOQOLATE’s learned hierarchy.Source: §2.1 · p. 2

2.2 Interpreting and Debiasing CLIP-based CBMs

  • CLIP-based CBMs face concept unfaithfulness and spurious correlations, while existing interpretation and debiasing approaches use distinct representations or modify model weights.CHOQOLATE learns its hierarchy directly from CLIP similarities without supervision; unlike the Choquet student of Atienza et al. (2024), it is the predictor itself, with no teacher or distillation, and its removal intervention is post hoc.Source: §2.2 · p. 2, Choquet Integrals: From Decision Aid… · p. 3

3 Background

CLIP image–text similarities provide concept scores that can be normalized into a concept bottleneck, where related concepts may co-activate. A 2-additive Choquet integral models pairwise interactions, and its closed-form Shapley values attribute each coordinate’s contribution.

3.1 Concept Bottlenecks from CLIP Similarities

  • A CLIP-based concept bottleneck consists of normalized image–text similarity scores for a fixed vocabulary, with related concepts co-activating because vision-language models entangle them.The image and text encoders remain frozen in this setup, and recovering the correlated concept groups is the stated goal.Source: §3.1 · p. 3

3.2 The 2-Additive Choquet Integral

  • The 2-additive Choquet integral retains individual weights and pairwise interactions; positive min terms encode complementarity, while positive max terms encode substitutability.Its normalized, nonnegative min/max decomposition keeps outputs in [0, 1] and makes the integral monotone in each coordinate.Source: §3.2 · p. 3
  • Each coordinate’s closed-form Shapley value combines its individual weight with half of each pairwise interaction involving that coordinate.This makes the coordinate’s contribution directly computable from the 2-additive measure.Source: §3.2 · p. 3

4 CHOQOLATE

CHOQOLATE replaces a CLIP-based CBM’s classifier with two learned Choquet layers: one forms an interpretable bottleneck of nodes, and the next maps those nodes to class scores. Its Shapley-based intervention removes selected concepts post hoc while preserving the relative contributions of remaining weights within each node.

4.1 Architecture and Training

  • CHOQOLATE keeps CLIP encoders frozen and learns two Choquet layers, with the first mapping concept scores to far fewer interpretable nodes and the second mapping nodes to class scores.Each first-layer node’s concept contributions are represented by Shapley values.Source: §4.1 · p. 4
  • Training minimizes temperature-scaled cross-entropy with an ℓ1 penalty on first-layer interaction weights to encourage nodes with few pairwise interactions.Both layers are trained end to end.Source: §4.1 · p. 4

4.2 CHOQ-PCR: Post-hoc Concept Removal

  • CHOQ-PCR removes a selected concept by zeroing every first-layer weight involving it, thereby removing its pairwise interactions as well as its individual contribution.The remaining weights are rescaled by a common factor to restore normalization, preserving their relative contributions within each node.Source: §4.2 · p. 4

5 Analysis of one Choquet layer

The one-layer analysis explains how Choquet gradients can organize features without explicit clustering supervision. The gradient’s weight and deviation factors shape feature selection, while temperature scaling preserves the error signal needed for those dynamics.

5 Analysis of one Choquet layer

  • For a one-layer Choquet classifier, class scores are inner products between normalized weights and a feature vector.The weights are obtained by applying softmax to the layer’s unnormalized parameters.Source: §5 · p. 4
  • The gradient factorizes into classification error, the feature’s current weight, and its deviation from the integral output.Small weights yield vanishing gradients that freeze features out; updates concentrate on features far from the integral value, anchoring organization.Source: §5 · p. 4
  • Because the inner gradient factors can only shrink the error term, temperature Tce rescales the gradient and changes the predicted probabilities.A small Tce can kill the error term, and thus sparsity pressure, once the model is confident; the experiments use Tce = 0.005.Source: §5 · p. 4

6 Experiments

The experiments assess accuracy and latent organization, test whether the predicted organization emerges during training, and evaluate post-hoc removal of spurious concepts. Across the classification datasets, CHOQOLATE combines competitive accuracy with strong organization quality; its post-hoc intervention performs comparably to the oracle, with practical advantages over methods requiring annotations or fine-tuning.

6 Experiments

  • The experiments are organized around accuracy–interpretability trade-offs, emergence of latent organization, and post-hoc spurious-concept removal.The bias-mitigation case study assumes the spurious concepts are known from the dataset definition and evaluates accuracy and worst-group accuracy on Waterbirds and CelebA.Source: §6 · p. 4, §6.4 · p. 5

6.1 Setup

  • The evaluation covers four classification datasets, two controlled-bias datasets, accuracy and organization metrics, and PCBM, sparse-representation, and other baselines.The setup fixes the latent dimension at 8 and uses a CLIP ViT-L/14@336px backbone; bias mitigation is evaluated on Waterbirds and CelebA.Source: §6.1 · pp. 4–5

6.3 RQ2: Emergence of the Organization

  • On COCO, training shows a two-step organization: an anchor concept first gains attribution, then related concepts gain weight while unrelated concepts are driven to zero.The reported dynamics match the gradient analysis: near-zero weights freeze features, while the deviation factor pushes down concepts unrelated to the anchor and softmax renormalization transfers weight to related ones.Source: §6.3 · p. 5

6.2 RQ1: Accuracy–Interpretability Trade-off

  • CHOQOLATE is competitive in accuracy and improves latent organization quality across datasets, with sparse, coherent concept groupings.It dominates Attribution Gini everywhere and has the best or statistically comparable Node Coherence except on MonumAI; Figure 4 illustrates grouped concepts in its word clouds.Source: §6.2 · p. 5, Figure 3 · p. 6, Figure 4 · p. 6

6.4 RQ3: Post-hoc Bias Mitigation

  • CHOQ-PCR improves over the biased model and performs on par with the oracle, while Group-DRO remains strongest on CelebA worst-group accuracy.CHOQ-PCR is the only compared method described as fine-tuning-free, annotation-free, and test-time; worst-group accuracy variance remains high because the worst group is small.Source: §6.4 · p. 6

7 Limitations

CHOQOLATE depends on redundant concept-similarity scores, and its quadratic parameter cost and monotone aggregation constrain where it is practical and what its explanations can express.

7 Limitations

  • CHOQOLATE relies on strongly correlated concept-similarity scores; another backbone needs an analogous score and a prior check of this redundancy assumption.Node Coherence provides the diagnostic for checking the correlation matrix.Source: §7 · pp. 6–7
  • Each first-layer node has a quadratic number of weights in vocabulary size, becoming prohibitive for tens of thousands of concepts and ruling out settings with far more latent units.The cost remains modest for the paper’s experiments with 15 to 226 concepts.Source: §7 · p. 7
  • Nonnegative weights make explanations presence-only, omitting evidence of absence; this restriction is the price of CHOQ-PCR’s unambiguous edits.The authors report that monotone aggregation suffices for their evaluated tasks and leave negative contributions for future work.Source: §7 · p. 7

8 Conclusion

CHOQOLATE uses 2-additive Choquet integrals to organize concept bottlenecks, with gradient analysis explaining sparse attributions and Shapley-based post-hoc edits supporting bias mitigation. Its training procedure freezes CLIP encoders and learns two Choquet layers end to end.

8 Conclusion

  • CHOQOLATE offers a favorable accuracy–latent-organization trade-off, sparser per-node attributions, and post-hoc spurious-concept suppression on par with debiasing methods requiring group annotations or retraining.The conclusion attributes attribution sparsity to a gradient mechanism under softmax reparametrization: concepts with low current weight progressively freeze out.Source: §8 · p. 7
  • Choquet weights map to Shapley values in closed form, enabling targeted post-hoc edits in the interpretable concept space.CHOQ-PCR suppresses spurious concepts after training.Source: §8 · p. 7

A Weight Parametrization and Training Procedure

  • Softmax parameterization enforces nonnegative weights summing to one by construction, so training uses ordinary gradient descent without projection or a normalization penalty.The unconstrained parameters are transformed into integral weights by softmax.Source: A Weight Parametrization and Training… · p. 11
  • The training procedure freezes the CLIP encoders, computes concept similarities once, and updates only the parameters of the N + C Choquet integrals.Both Choquet layers are trained end to end using cross-entropy, with an ℓ1 penalty on first-layer interaction weights to encourage fewer pairwise interactions per node.Source: A Weight Parametrization and Training… · p. 11

B Dataset Details

The evaluation spans four classification datasets and two controlled-bias datasets, using separate measures of task performance, node organization, and input-information preservation. The appendix reports the latter for transparency rather than as a ranking basis.

B Dataset Details

  • The four classification benchmarks range from three-class Cats/Dogs/Cars and four-class MonumAI to six-class COCO and 200-class CUB, with concept vocabularies from 15 to 226.The tasks cover object, architectural-style, scene-location, and fine-grained bird classification.Source: B Dataset Details · pp. 11–12
  • Bias mitigation is tested on Waterbirds, with 95% training correlation between bird type and background, and CelebA, which targets hair-color/gender correlation.Waterbirds has a balanced test set; CelebA keeps spurious-feature proportions identical across splits.Source: B Dataset Details · p. 12

C Metrics

  • Accuracy measures task performance, while Attribution Gini measures concentration of a node’s Shapley attributions and Node Coherence measures attribution-weighted correlation among its concepts.The two organization metrics are complementary: a node can be sparse but incoherent, or coherent but diffuse.Source: C Metrics · p. 12
  • Worst-group accuracy is the lowest accuracy across groups defined by class label and spurious attribute, isolating performance where the spurious correlation is broken.Unlike average accuracy, it directly measures reliance on the spurious feature.Source: C Metrics · p. 12

D.1 Latent space quality

  • Tables A3 and A4 report accuracy and latent-space quality metrics across four benchmarks, with results averaged over twenty runs and standard deviations.The captions identify the best result per dataset in A3 and per metric column in A4.Source: Table A3 · p. 13, Table A4 · p. 13
  • CKNNA and HSIC measure information shared between the CLIP-score input and learned representation, but high input preservation alone does not establish interpretability or concept organization.The paper excludes them from its main evaluation because copying the input could score highly without providing interpretability benefits.Source: §D.1 · p. 13
  • The authors report CKNNA and HSIC for transparency rather than ranking, noting that CHOQOLATE trades some raw input preservation for sparsity and semantic coherence.Methods close to the identity tend to score highly on these input-preservation metrics.Source: §D.1 · p. 13

D.2 Ablation study

The ablations show how CHOQOLATE’s accuracy and latent-space quality depend on temperature, task difficulty, bottleneck size, and model components. Additional word-cloud comparisons show more salient, semantically grouped explanations than PCBM on COCO.

D.2 Ablation study

  • Temperature controls the interpretability–accuracy trade-off: large Tce sustains sparsity-promoting gradients at an accuracy cost, while small Tce yields dense weights; Tce = 1 also falls into the latter regime.The cited table reports accuracy and latent-space metrics on COCO averaged over five runs, but the supplied passages do not provide its cell values.Source: §D.2 · p. 14, Table A6 · p. 14
  • On CUB-200, accuracy falls from 88.47% at 20 classes to 54.56% at 200, while Attribution Gini rises and Node Coherence declines as class count grows.The authors identify a regime where CHOQOLATE remains effective: a moderate number of classes relative to bottleneck dimensionality.Source: §D.2 · p. 14, Table A7 · p. 15
  • The COCO configuration combining all components with intermediate-layer-only ℓ1 regularization gives the most balanced accuracy–interpretability trade-off.The authors report synergy between second-order terms and ℓ1 regularization: the latter raises interpretability at comparable accuracy.Source: §D.2 · p. 15, Table A8 · p. 16
  • Across COCO bottleneck sizes, CHOQOLATE leads in Attribution Gini and accuracy; it reaches its accuracy plateau at N = 8, while larger N adds little quality and makes explanations less compact.At N = 4, accuracy drops sharply for all methods; from N = 8 onward, baselines need wider bottlenecks to approach CHOQOLATE’s plateau.Source: §D.2 · p. 15

D.3 Additional Global Explanations

  • On COCO, CHOQOLATE produces more salient and semantically grouped word-cloud explanations than PCBM, including nodes grouping outdoor-sports and transportation concepts.The word-cloud concept sizes represent Shapley contributions to each node.Source: §D.3 · pp. 15–16

E Statistical Significance of the Main Results

The statistical tests find CHOQOLATE’s sparsity advantage significant across all four datasets, while coherence gains are more modest. The accompanying proof represents a Choquet layer as a linear map over input-derived features and simplifies its classification-loss gradient.

E Statistical Significance of the Main Results

  • Attribution Gini favors CHOQOLATE over every baseline on all four datasets (p < 0.001; median distance ∆ = 0.83–2.13), while coherence gains are smaller and statistically comparable on MonumAI.Node Coherence improves significantly on Cats/Dogs/Cars and CUB-200, and by a wide margin over PSCBM on COCO.Source: E Statistical Significance of the Main… · p. 17, Table A9 · p. 17

F Proof for a Single Choquet Layer

  • A 2-additive Choquet layer is linear in a feature vector containing input coordinates and pairwise min and max terms, with class-specific weights.The proof treats each class logit as the corresponding Choquet output.Source: F Proof for a Single Choquet Layer · p. 18

F.1 Proof of Proposition 1 of the Main Paper

  • For the cross-entropy gradient, the proof differentiates through each class logit and its weight vector, then simplifies the softmax-reparameterized weight sum using the Kronecker delta.The simplification identifies the first term with j = i and the second with the Choquet output.Source: §F.1 · pp. 18–19

F.2 Proof of Corollary 1

The temperature analysis explains how prediction confidence affects the sparsity-promoting error signal: small temperatures can extinguish it, while large temperatures preserve it at slower convergence. The technical passages also specify experiment hardware and concept counts for the bias benchmarks.

F.2 Proof of Corollary 1

  • Small Tce sharpens predictions and can extinguish the error signal and sparsity pressure once the model becomes confident; large Tce keeps the signal alive at the cost of slower convergence.Because Choquet outputs lie in [0, 1], the passage says useful temperatures sit well below one and trade interpretability against accuracy.Source: §F.2 · p. 19

G Technical details

  • Experiments ran on one NVIDIA GeForce RTX 4070 SUPER GPU with 12 GB VRAM, using PyTorch on Linux.Source: G Technical details · p. 20
  • Waterbirds uses 48 non-spurious and 9 spurious concepts, while CelebA uses 35 non-spurious and 12 spurious concepts.The table defines spurious concepts as attributes correlated with background for Waterbirds and gender for CelebA.Source: Table A2 · p. 22

Related papers

Loading 2609.37786v2…