PaperlayerView exact revision on arXiv

Source-linked AI summary

Identifying Confusion Trends in Concept-based XAI for Multi-Label Classification

Haadia Amjad, Ronald Tetzlaff

arXiv:2608.15731v1cs.CVcs.AI

TL;DR

In multi-label classification, it is difficult to determine whether environmental concepts in explanations reflect valid evidence or dataset bias. This paper applies concept-based explanations to analyze confusion, finding that greater concept distinctiveness aligns with clearer class boundaries and reduced confusion, while environmental concepts reveal learning biases.

  • Problem

    Determining whether environmental concepts in CXAI explanations are valid evidence or reflect dataset bias remains challenging in complex multi-label classification.

  • Method

    The study analyzes concept-level explanations from CRP and CRAFT using models trained on the 20 most frequently annotated MS-COCO labels.

  • Results

    Higher concept distinctiveness aligns with reduced label and conceptual confusion, while environmental concepts expose dataset-induced biases and recurring spurious associations.

  • Takeaways & Limitations

    CRP and CRAFT can support model diagnosis by revealing conceptual ambiguities, learning biases, and patterns relevant to assessing model generalizability.

Abstract

from arXiv · show

Deep Neural Networks (DNNs) deployed in high-risk domains, such as healthcare and autonomous driving, must be not only accurate but also understandable to ensure user trust. In real-world computer vision tasks, these models often operate on complex images containing background noise and are heavily annotated. To make such models explainable, Concept-based Explainable AI (CXAI) methods need to be assessed for their applicability and problem-solving capacity. In this work, we explore CXAI use cases in multi-label classification by training two DNNs, VGG16 and ResNet50, on the 20 most annotated labels in the MS-COCO dataset (Microsoft Common Objects in Context). We apply two CXAI methods, CRP (Concept Relevance Propagation) and CRAFT (Concept Recursive Activation FacTorization), to generate concept-level explanations and investigate the overall evaluations. Our analysis reveals three key findings: (1) CXAI highlights learning weaknesses in DNNs, (2) higher concept distinctiveness reduces label and concept confusion, and (3) environmental concepts expose dataset-induced biases. Our results demonstrate the potential of CXAI to enhance the understanding of model generalizability and to diagnose bias instigated by the dataset.

I. INTRODUCTION · II. RELATED WORK

The paper motivates CXAI for diagnosing confusion, unintended concept associations, and dataset-induced bias in multi-label DNN classification. It builds on related work by using CRP and CRAFT to study how concept-based explanations interact with model confusion.

  • I. INTRODUCTION: Complex, imbalanced multi-label datasets can cause label confusion and wrong associations even in high-performing DNNs, motivating deeper analysis with XAI.Real-world scenes may contain small, clustered, or overlapping objects and multiple annotations.
  • I. INTRODUCTION: CXAI identifies semantically meaningful features and reveals how DNNs internally represent classes, offering more interpretable explanations than saliency maps in complex scenes.The paper distinguishes concept-based explanations from less interpretable saliency maps.
  • I. INTRODUCTION: Environmental concepts can reflect valid context, dataset bias, or model mislearning, because unintended background associations may influence classification.The paper gives associations such as “fingers” with a pen as an example of spurious concept learning.
  • I. INTRODUCTION: The study trains ResNet50 and VGG-16 on the 20 most annotated MS-COCO labels and evaluates well-performing and poor checkpoints with CRP and CRAFT.The methods produce focused visualizations and scores for concept importance and concept relevance, which are compared using concept error and distinctiveness.
  • I. INTRODUCTION: The paper demonstrates that CXAI reveals DNN learning weaknesses, while greater concept distinctiveness is associated with reduced label-prediction and concept-attribution confusion.These findings are stated as two of the paper’s main contributions.
  • II. RELATED WORK: Related CXAI research examines concept selection, concept representation, and concept control as three directions for advancing the field.Lee et al. identify these directions in their review of current CXAI methods.
  • II. RELATED WORK: Prior studies use concept–class relationships to detect model and concept biases and emphasize that dataset choices, even slight variations, affect CXAI meaningfulness.This work extends that foundation by investigating confusion through CRP and CRAFT.
  • II. RELATED WORK: CRP answers what and where questions by locating hidden-layer concepts in input data and measuring each concept’s relevance to a target class.CRP is based on Layer-wise Relevance Propagation and provides concept-level contribution analysis.

III. EXPERIMENTAL SETUP

The experimental setup trains and evaluates DNNs and CXAI methods in a defined workflow, using a 90/10 split of MS-COCO’s training set and focusing on its 20 most frequently annotated labels.

  • Workflow: The workflow includes DNN model training and evaluation, followed by XAI method use and evaluation.The experimental pipeline is illustrated in Figure 1.
  • Dataset: The experiments use MS-COCO’s 2017 object detection subset, containing 80 classes across 118,000 images.The dataset supports object detection, captioning, segmentation, and classification tasks.
  • Dataset: A 90/10 split of the training set produces 106,200 training and 11,800 test images because test labels are unavailable.The split is applied to the MS-COCO training set.
  • Dataset: The experiments focus on the 20 most frequently annotated training labels to ensure sufficient data.These labels are selected from the MS-COCO training set.

B. DNN Models

The study trains ImageNetV2-pretrained ResNet50 and VGG-16 models in PyTorch to support broadly adaptable CXAI evaluation. Two checkpoints per model represent well- and poor-performing scenarios, assessed with multi-label performance, confusion, and label-similarity metrics.

  • Scenario performance: Well-performing models achieved 82.85% Accuracy for ResNet50 and 84.26% Accuracy for VGG-16.The corresponding reported metrics were ResNet50 Recall 85.50, Precision 58.84, and F1 Score 60.84; VGG-16 Recall 86.91, Precision 59.74, and F1 Score 58.84.
  • Scenario performance: Poor-performing models achieved 58.24% Accuracy for ResNet50 and 52.85% Accuracy for VGG-16.The corresponding reported metrics were ResNet50 Recall 77.04, Precision 53.82, and F1 Score 42.92; VGG-16 Recall 74.50, Precision 53.62, and F1 Score 46.12.
  • Evaluation metrics: Evaluation uses accuracy, recall, precision, F1 score, multi-label confusion tensors, Mutual Information, and Jaccard Similarity Coefficients.The confusion tensor accounts for label imbalance, while MI and JSC quantify information sharing or similarity between target and predicted labels.

C. CXAI Methods · D. Explanation of terms (in brief) · 1) Concept Distinctiveness:

The study uses CRAFT and CRP to examine how label confusion is reflected in learned concepts, while measuring concept distinctiveness, concept error, and shared information. Concept distinctiveness quantifies uniqueness from 0 to 1, whereas concept error estimates incorrect concept use against a subjective, class-based ground truth.

  • C. CXAI Methods: The analysis examines confusion across CRAFT and CRP in all four model scenarios without directly comparing their superiority.Their outputs are used to investigate how label confusion appears in learned concepts.
  • C. CXAI Methods: CRAFT outputs overall concept importance, while CRP reports concept relevance for specific target classes.The methods provide different perspectives on concept contributions.
  • C. CXAI Methods: Concept distinctiveness and concept error are computed for both CXAI methods, alongside adapted mutual information for shared information between concepts.These findings are compared with DNN evaluations to support the study’s hypotheses.
  • D. Explanation of terms (in brief): The terminology subsection introduces CXAI concepts and the adaptations used in the analysis.It provides brief explanations of the measures and adaptations applied in the study.
  • D. Explanation of terms (in brief): Concept distinctiveness ranges from 0 to 1 and measures how unique one concept is relative to others.Low distinctiveness indicates overlapping or redundant concepts and may signal learning errors.
  • 1) Concept Distinctiveness:: Concept vectors are directions in activation space that capture distinct features for individual concepts.The notation vCi and vCj denotes the vectors for concepts Ci and Cj, respectively.
  • 1) Concept Distinctiveness:: Concept error measures incorrect or irrelevant concept use during prediction using an approximate binary-classification ground truth.The ground truth includes concepts belonging to the target class and excludes environmental concepts; structured human validation is recommended.

2) Concept Error: · 3) Mutual Information:

Mutual information (MI) measures dependency between variables by quantifying how much information one label provides about another. Applied to concept vectors, MI reveals shared information, dependencies, and potential redundancies in learned features.

  • 3) Mutual Information:: MI quantifies dependency between two variables.It provides a measure of how strongly the variables are related.
  • 3) Mutual Information:: In multi-label classification, MI measures how much information one label provides about another.The measure captures information transfer between labels.
  • 2) Concept Error:: For learned concepts, MI measures the information shared between two concept vectors.This extends dependency analysis from labels to concept representations.
  • 2) Concept Error:: Shared information between concept vectors can reveal dependencies among learned features.MI exposes relationships that may otherwise remain implicit in the representations.
  • 2) Concept Error:: MI can identify potential redundancies in learned features.Concepts with overlapping information may contribute non-independent explanations.
  • 3) Mutual Information:: Concept-level MI therefore supports analysis of dependencies and redundancy in learned representations.The same information-sharing measure connects concept interpretation with feature analysis.

IV. RESULTS

The results section presents findings from case studies that compare the label evaluations described previously.

  • Results: The findings are based on case studies comparing different label evaluations.These evaluations were described in the preceding section.

A. Confusion in Labels Can Be Understood by Their Explanations

CXAI explanations reveal that label confusion arises from recurring visual or contextual co-occurrence and from weak conceptual boundaries. Combining explanations with performance metrics and co-occurrence statistics distinguishes visual similarity from dataset bias or poor internal representations.

  • Confusion patterns: In ResNet50 scenario 1, person is confused with car and chair, while car overlaps with truck and bench.Mutual information links person with handbag and backpack, and car with truck and traffic light, indicating dependencies shaped by recurring visual or contextual co-occurrence.
  • CXAI metrics: In scenario 2, increased concept error and lower distinctiveness coincide with significantly higher confusion scores.The opposite metric pattern suggests that unclear conceptual boundaries increase reliance on misleading contextual aspects.
  • CXAI metrics: Lower concept error and higher distinctiveness for person in scenario 1 align with reduced confusion.The models perform better in scenario 1, where the label exhibits stronger conceptual separation.
  • Interpretation: Combining CXAI explanations with performance metrics and co-occurrence statistics exposes whether confusion reflects visual similarity, dataset bias, or poor internal representations.These analyses clarify the roots of confusion rather than treating it solely as a classification-performance issue.

B. Distinctiveness Reduces Conceptual Confusion

The section shows that greater concept distinctiveness reduces conceptual confusion by helping models separate difficult classes and supporting improved performance. It also indicates that distinctiveness analysis can diagnose model weaknesses and guide dataset curation and feature learning.

  • B. Distinctiveness Reduces Conceptual Confusion: Distinct concepts use unique, class-specific features, whereas overlapping labels produce concepts that capture shared rather than class-specific features.This confusion arises from concept bias, in which irrelevant co-occurring features become associated with a class.
  • B. Distinctiveness Reduces Conceptual Confusion: Concept explanations can reveal concept bias, such as “person” appearing heavily in “tennis racket” explanations for VGG-16.The example comes from scenario 1 and illustrates how co-occurring features can dominate explanations.
  • B. Distinctiveness Reduces Conceptual Confusion: Poor-performing models provide weaker concept-based explanations because they fail to maintain clear distinctions between classes.In scenario 2 of ResNet50, person shows less distinctiveness from unrelated classes.
  • B. Distinctiveness Reduces Conceptual Confusion: Well-performing models separate difficult-to-distinguish classes more actively, as shown by the stronger distinction between car and person in scenario 1 of ResNet50.The comparison connects concept distinctiveness with previously established confusion patterns.
  • B. Distinctiveness Reduces Conceptual Confusion: Increasing concept distinctiveness can improve model performance while helping diagnose weaknesses and guide dataset curation and feature learning.The stated goal is to reduce confusion and improve overall classification accuracy.

C. Environmental Concepts Reveal Dataset Biases

Environmental concepts expose dataset-induced biases that weaken generalization by causing models to rely on frequent contextual co-occurrences rather than robust class representations. Results from MS-COCO and OSDaR23 show that diverse, well-annotated training data is essential for reducing concept bias and improving reliability.

  • Environmental Concepts Reveal Dataset Biases: Environmental concepts often reflect training-dataset patterns, with semantically related classes producing biased explanations that include one another’s contextual concepts.Examples include sports classes such as baseball glove and tennis racket.
  • Environmental Concepts Reveal Dataset Biases: Dining table is associated with chair, person, apple, and cup, indicating reliance on frequent co-occurrences rather than structural generalization.These labels share semantic but not structural similarity, making the associations problematic in deployed systems.
  • Environmental Concepts Reveal Dataset Biases: 95.92% accuracy and 79.93 F1 on OSDaR23 contrast with CXAI evidence of low generalizability for a ResNet50 trained on RGB data.For person, CRP relies heavily on platforms and staircases because these backgrounds frequently co-occur but are unlabeled.
  • Environmental Concepts Reveal Dataset Biases: Environmental concepts can compromise reliability in real-world and high-risk applications by revealing dataset-induced biases that undermine generalization.The paper identifies diverse, well-annotated datasets as necessary for robust and semantically accurate representations.

V. CONCLUSION AND FUTURE WORK

The study shows that confusion in multi-label classification is reflected in concept-based explanations. CXAI reveals how overlapping or spurious environmental concepts contribute to label confusion and learning biases, while greater concept distinctiveness is associated with less conceptual confusion.

  • Confusion in multi-label classification is directly reflected in concept-based explanations.
  • Label confusion often results from overlapping or spurious environmental concepts.These concepts highlight learning biases in the models.
  • Higher concept distinctiveness is inversely related to conceptual confusion.
Loading 2608.15731v1…