PaperlayerView exact revision on arXiv

Source-linked AI summary

A Closer Look at the Explainability of Contrastive Language-Image Pre-training

Yi Li, Hualiang Wang, Yiqun Duan, Jiheng Zhang, Xiaomeng Li

arXiv:2304.05653v2cs.CV

TL;DR

CLIP’s explainability is limited by background-focused, noisy CAMs linked to inconsistent self-attention and redundant category features. The paper proposes CLIP Surgery, which modifies inference architecture and features without fine-tuning, substantially improving explainability and supporting multimodal and open-vocabulary tasks.

  • Problem

    CLIP produces background-focused and noisy explanations, limiting transparent use of the raw model without additional alignment or fine-tuning.

  • Method

    CLIP Surgery performs architecture and feature surgery on raw CLIP during inference without fine-tuning.

  • Results

    The method significantly enhances CLIP’s visual explainability and supports downstream tasks including semantic segmentation, interactive segmentation, and multi-label recognition.

  • Takeaways & Limitations

    CLIP Surgery enables reliable CAM, multimodal visualization, and open-vocabulary applications without extra alignment.

  • Takeaways & Limitations

    Existing alignment methods can produce reasonable visualizations but require further tuning and do not explain raw CLIP.

Abstract

from arXiv · show

Contrastive language-image pre-training (CLIP) is a powerful vision-language model that has shown great benefits for various tasks. However, we have identified some issues with its explainability, which undermine its credibility and limit the capacity for related tasks. Specifically, we find that CLIP tends to focus on background regions rather than foregrounds, with noisy activations at irrelevant positions on the visualization results. These phenomena conflict with conventional explainability methods based on the class attention map (CAM), where the raw model can highlight the local foreground regions using global supervision without alignment. To address these problems, we take a closer look at its architecture and features. Based on thorough analyses, we find the raw self-attentions link to inconsistent semantic regions, resulting in the opposite visualization. Besides, the noisy activations are owing to redundant features among categories. Building on these insights, we propose the CLIP Surgery for reliable CAM, a method that allows surgery-like modifications to the inference architecture and features, without further fine-tuning as classical CAM methods. This approach significantly improves the explainability of CLIP, surpassing existing methods by large margins. Besides, it enables multimodal visualization and extends the capacity of raw CLIP on open-vocabulary tasks without extra alignment. The code is available at https://github.com/xmed-lab/CLIP_Surgery.

1. Introduction

The paper identifies opposite and noisy CLIP visualizations caused by inconsistent self-attention and redundant cross-category features, then proposes CLIP Surgery to improve explainability without fine-tuning.

  • Direct CAM application to CLIP is unsatisfactory because conventional techniques do not work without local alignment.Existing alignment-based approaches require further tuning and do not explain the raw CLIP model.
  • CLIP often prioritizes background regions over foregrounds, producing noisy activations and opposite visualizations in CAM and Grad-CAM.These issues are shown in Figure 1 and also occur in gradient-based CAM methods.
  • Inconsistent semantic relations formed by self-attention contribute to opposite visualizations, while redundant features among categories contribute to noisy activations.The analyses also find that not all layers are equally beneficial or close to final predictions.
  • CLIP Surgery modifies inference architecture and output features through architecture and feature surgery without additional fine-tuning.Architecture surgery reforms self-attention and aggregates beneficial modules, while feature surgery mitigates redundant features.
  • The method surpasses existing CAM methods, including results exceeding 20% mIoU improvement on VCO 2012, and supports open-vocabulary tasks.Supported applications include semantic and interactive segmentation, multi-label recognition, and multimodal visualization.

2. Related Works

Related work includes CAM-based explainability and task-specific alignment methods, whereas CLIP Surgery targets the raw CLIP model without additional alignment or fine-tuning.

  • Existing CAM methods show unsatisfactory results on CLIP, while alignment-based alternatives add masks, bounding boxes, prompt learning, or other supervision.
  • CLIP Surgery is more effective than CAM, Grad-CAM, LRP, Bi-Modal, and gScoreCAM, and is reported to surpass alignment methods in practicalness and flexibility.
  • Text prompts alone perform poorly for interactive segmentation with SAM, motivating combinations with manual points for better results.
  • Unlike task-specific alignment methods, CLIP Surgery uses raw CLIP without further alignment and can provide CAMs for downstream applications.Its CAMs can generate segmentation results or points for text-based SAM.
  • Feature surgery mitigates redundant visual features and improves multi-label recognition, although conventional CAM methods are not responsible for classification improvements.
  • The method supports multimodal visualization and multiple tasks without the additional models, layers, or fine-tuning used by other approaches.

3. Method

CLIP Surgery diagnoses opposite visualizations and noisy activations in CLIP, then modifies its inference architecture and features without fine-tuning to improve explainability.

  • 3.1. Visual Explainability of CLIP: CLIP commonly prioritizes background over foreground regions and produces noisy activations at class-irrelevant positions across ResNets, Vision Transformers, and multiple explainability methods.These phenomena are reported as common rather than isolated cases.
  • 3.2. CLIP Architecture Surgery: Raw self-attention links inconsistent semantic regions because heterogeneous parameters lack local supervision, producing opposite visualizations.The proposed consistent self-attention instead uses homogeneous parameters to link tokens with consistent semantics.
  • 3.2. CLIP Architecture Surgery: Consistent self-attention focuses on foregrounds more reliably than raw self-attention according to qualitative comparisons and the mFSR analysis.The reported mFSR comparison shows most proposed examples focusing on foregrounds at varied thresholds, while few raw-attention samples pass them.
  • 3.2. CLIP Architecture Surgery: Intermediate FFNs can push features toward negative labels, so the method aggregates partial self-attention modules and skips FFNs through dual paths.The last FFN is reported at cosine 0.1231, farther than the cosine of negative labels.
  • 3.3. CLIP Feature Surgery: Similarity maps from positive labels resemble those from an empty string, supporting the diagnosis that redundant category features cause noisy activations.Non-activated redundant features occupy a substantial portion of feature space and appear as noise.

4. Experiments

CLIP Surgery improves CLIP’s visual explainability across datasets, metrics, and backbones without additional training, while extending visualization to multimodal and open-vocabulary tasks.

  • Results of Explainability: 22.11%–35.95% higher mIoU and 47.72%–65.21% higher mSC than CLIP are achieved on average across four datasets and five backbones.CLIP’s mSC is below 0, whereas CLIP Surgery produces results far above 0, correcting the preference for background over foreground.
  • Results of Explainability: The method produces clearer, less noisy visualizations that address opposite visualization using only the original CLIP, without training or complex back-propagation.These improvements apply across ResNet and ViT backbones and avoid the backbone restrictions of several prior methods.
  • Results of Explainability: CLIP Surgery achieves the best performance across all datasets, metrics, and backbones, surpassing prior explainability methods by significant margins.It outperforms ECLIP by up to 15.94% in mIoU and 19.89% in mSC, and RCLIP by 18.42% in mIoU and 23.13% in mSC.
  • Ablation Study: Architecture surgery improves mSC by 47.88%, while feature surgery adds 3.17% and improves multi-label recognition mAP by 5.52%.Using original-path features leaves mAP unchanged, whereas feature surgery significantly enhances it.
  • Results on Open-vocabulary Tasks: CLIP Surgery achieves new state-of-the-art mAP 48.55% on NUS-Wide with ResNet-50 when combined with TaI-DPT, 1.56% above the implemented TaI-DPT.It also improves raw CLIP mAP by 11.61% for ViT-B/16 and 7.24% for ResNet-50.
  • Multimodal Visualization: Multimodal visualization reveals partial-context learning, image-text recognition, redundant text tokens, and frequent activation of the end token.The method provides pixel-level results from text inputs, including one text for all images without interaction on every image.

5. Conclusion

The paper identifies inconsistent self-attention relations and redundant category features as sources of CLIP’s opposite visualizations and noisy activations, then proposes CLIP Surgery to improve explainability and downstream capabilities.

  • 5. Conclusion: Inconsistent semantic regions linked by raw self-attentions produce CLIP’s opposite visualization.
  • 5. Conclusion: CLIP Surgery merges consistent self-attentions through an architecture surgery with a dual-path structure.
  • 5. Conclusion: Redundant features among categories cause noisy activations, which feature surgery mitigates in CLIP’s output features.
  • 5. Conclusion: CLIP Surgery significantly enhances CLIP’s visual explainability for reliable CAM and improves downstream semantic segmentation, interactive segmentation, and multi-label recognition.The method also supports broader applicability and insights into CLIP’s architecture, features, and learning process.

CRediT authorship contribution statement

The authors’ contributions span conceptualization, investigation, methodology, validation, visualization, resources, project administration, supervision, and writing.

  • CRediT authorship contribution statement: Yi Li contributed conceptualization, investigation, methodology, validation, visualization, and writing.
  • CRediT authorship contribution statement: Hualiang Wang and Yiqun Duan contributed conceptualization, investigation, and methodology, with Duan also contributing to investigation.
  • CRediT authorship contribution statement: Jiheng Zhang contributed resources, while Xiaomeng Li contributed conceptualization, investigation, and project administration.
  • CRediT authorship contribution statement: Xiaomeng Li also contributed supervision and writing—review and editing.

Declaration of competing interest

The authors state that the manuscript is unpublished, is not under consideration elsewhere, and was approved by all authors without conflicts of interest.

  • Declaration of competing interest: The manuscript had not been published previously and was not under consideration elsewhere.
  • Declaration of competing interest: All authors approved the manuscript for publication.
  • Declaration of competing interest: The authors declared no conflict of interest.
Loading 2304.05653v2…