PaperlayerView exact revision on arXiv

Source-linked AI summary

The Clever Hans Effect in Unsupervised Learning

Jacob Kauffmann, Jonas Dippel, Lukas Ruff, Wojciech Samek, Klaus-Robert Müller, Grégoire Montavon

arXiv:2408.08041v1cs.LGcs.AIstat.ML

TL;DR

Unsupervised models can achieve accurate predictions for spurious reasons, threatening downstream applications that depend on their representations. The paper applies Explainable AI to expose these Clever Hans effects, finds them widespread and driven largely by model and learning-algorithm biases, and shows that pruning the implicated features improves robustness.

  • Problem

    Unsupervised learning evaluations often emphasize downstream accuracy without checking whether models rely on Clever Hans strategies, creating a research gap for reliable downstream use.

    Source: Discussion · p. 7

  • Method

    The paper uses Explainable AI techniques, including BiLRP and virtual-layer analyses, to identify input features and frequency components supporting similarity and anomaly predictions.

    Source: Clever Hans Effects in Representation… · p. 3, Explanations for Representation Learning · p. 9

  • Results

    CH strategies are prevalent across unsupervised paradigms, and they arise less from data than from inductive biases in models and learning algorithms.

    Source: Discussion · p. 7

  • Takeaways & Limitations

    Pruning spuriously amplified or high-frequency features can improve robustness under difficult data conditions.

    Source: Discussion · p. 7

    The investigation focuses on image data, although the authors state that the analysis could be extended to other modalities and model classes.

    Source: Discussion · p. 7

Abstract

from arXiv · show

Unsupervised learning has become an essential building block of AI systems. The representations it produces, e.g. in foundation models, are critical to a wide variety of downstream applications. It is therefore important to carefully examine unsupervised models to ensure not only that they produce accurate predictions, but also that these predictions are not "right for the wrong reasons", the so-called Clever Hans (CH) effect. Using specially developed Explainable AI techniques, we show for the first time that CH effects are widespread in unsupervised learning. Our empirical findings are enriched by theoretical insights, which interestingly point to inductive biases in the unsupervised learning machine as a primary source of CH effects. Overall, our work sheds light on unexplored risks associated with practical applications of unsupervised learning and suggests ways to make unsupervised learning more robust.

Results

Explainable AI revealed diverse Clever Hans strategies in unsupervised representation learning and anomaly detection, with measurable failures under altered data conditions. Targeted pruning of spuriously used features substantially restored robustness.

Clever Hans Effects in Representation Learning

  • PubMedCLIP achieved 75.2% class-balanced accuracy on GitHub instances, but its downstream classifier had a 40% false positive rate, limiting practical diagnostic use.BiLRP linked this behavior to shared text-like annotations, making it difficult to separate GitHub COVID-positive from COVID-negative cases.Source: Clever Hans Effects in Representation… · p. 3
  • CLIP-based similarities rely on textual annotations, while SimCLR and BarlowTwins rely on human or logo artifacts rather than task-relevant image features.BiLRP found that PubMedCLIP used shared text-like annotations for COVID-19 similarity, CLIP used textual logos, and all models focused on humans for fish images.Source: Clever Hans Effects in Representation… · pp. 3–4, Figure 3 · p. 3

Results

Clever Hans Effects in Representation Learning

  • CLIP accuracy fell from 85.0% to 80.5% on truck data with inserted logos, while SimCLR and BarlowTwins fish accuracy fell from 82.2% and 83.1% to 78.6% and 75.6% under human-only subsets.These subgroup shifts show that identified CH strategies impair uniform prediction and generalization.Source: Clever Hans Effects in Representation… · p. 4

Clever Hans Effects in Anomaly Detection

  • D2Neighbors used actual defect pixels but also irrelevant high-frequency background features, causing F1 to fall from 0.92 to 0.80 after antialiased resizing.Its false negative rate increased from 4% to 26%; the high frequencies supporting predictions disappeared after antialiasing.Source: Clever Hans Effects in Anomaly Detection · pp. 5–6

Alleviating CH in Unsupervised Learning

  • CH mitigation substantially reverses post-deployment degradation, bringing robustified models close to or above original-data performance.The approach modifies the unsupervised model itself: removing text-responsive CLIP features or inserting input blurring to prune high frequencies.Source: Alleviating CH in Unsupervised Learning · p. 6, Table 1 · p. 6

Discussion

The paper argues that Clever Hans strategies are prevalent in unsupervised learning and can undermine downstream reliability. Its analysis links these effects primarily to inductive biases and shows that Explainable AI can help identify and correct faulty features.

Discussion

  • Unsupervised evaluations often emphasize downstream accuracy while overlooking whether models rely on Clever Hans strategies.This leaves the strategy behind apparently high performance insufficiently examined.Source: Discussion · p. 7
  • CH strategies are prevalent across several unsupervised learning paradigms and can cause misdiagnosis or missed manufacturing defects.The paper reports such consequences in medical and industrial use cases.Source: Discussion · p. 7
  • Mitigating CH effects separately for each downstream task may be unsustainable, so the paper argues for addressing them directly in the unsupervised model.Direct mitigation is intended to provide persistent robustness across existing and future applications.Source: Discussion · p. 7
  • The analysis identifies diverse CH effects, including reliance on textual annotations and exposure to noise in unsupervised anomaly models.These examples show that flawed strategies can arise through different mechanisms.Source: Discussion · p. 7
  • Unlike supervised CH effects, unsupervised effects arise less from the data and more from inductive biases in the model and learning algorithm.The paper associates these biases with spurious feature amplification or suppression and missing frequency filtering.Source: Discussion · p. 7
  • Pruning high-frequency or spuriously amplified features revealed by Explainable AI produced systematic improvements on difficult subgroups or post-deployment conditions.This demonstrates that the explanations can guide both diagnosis and correction.Source: Discussion · p. 7
  • The investigation focuses on image data, although the authors state that the analysis could extend to other modalities using Explainable AI methods adapted to modern architectures.Suggested targets include time-series anomaly detection and representations learned by large language models.Source: Discussion · p. 7

Methods

The methods combine multiple unsupervised representation and anomaly-detection models with Explainable AI techniques that expose feature contributions to similarity and anomaly scores. Experiments use public image datasets, model-specific explanation rules, and reproducible implementation resources.

ML Models and Data for Representation Learning

  • The study uses SimCLR, CLIP, BarlowTwins, and PubMedCLIP for representation-learning experiments, alongside supervised counterparts sharing their architectures.The models use ResNet50-based pretrained weights, while the supervised baselines are trained purely with backpropagation.Source: ML Models and Data for Representation… · pp. 7–8
  • Representation-learning data include ImageNet truck and fish subsets plus combined NIH CXR8 and GitHub-COVID X-ray datasets with patient-level train-test splitting.The X-ray images are resized to 224x224 pixels and center-cropped.Source: ML Models and Data for Representation… · p. 8

ML Models and Data for Anomaly Detection

  • Anomaly detection uses D2Neighbors and PatchCore on MVTec industrial-object and texture categories.D2Neighbors operates on image-space distances, whereas PatchCore uses mid-level patch features and a memory bank.Source: ML Models and Data for Anomaly Detection · p. 8
  • D2Neighbors computes anomaly scores with a generalized f-mean interpreted as a soft minimum over distances to training points.Inputs are 224×224 RGB images, and γ is set so average perplexity equals 25% of the training-set size.Source: ML Models and Data for Anomaly Detection · p. 8
  • MVTec experiments simulate deployment shifts by replacing nearest-neighbor resizing with antialiased bilinear interpolation.The latter smooths transitions between adjacent pixels in resized images.Source: ML Models and Data for Anomaly Detection · pp. 8–9

Explanations for Representation Learning

  • BiLRP explains representation similarity by reverse-propagating bilinear dot-product contributions to pairs of lower-layer activations and input features.Its d × d output is rendered as a weighted bipartite graph, with pixel contributions aggregated into patches.Source: Explanations for Representation Learning · p. 9

Explanations for the D2Neighbors Model

  • For D2Neighbors, specialized LRP rules propagate contributions through the distance and soft-min layers, assigning each training point a contribution to an anomaly.Virtual layers and a DCT basis then propagate relevance into pixel-frequency features, whose reductions generate the Fig. 5 heatmaps.Source: Explanations for the D2Neighbors Model · p. 9
  • The D2Neighbors relevance rule includes a small positive ϵ for overlapping inputs, and marginalization over interacting pixels or pixels yields reduced heatmap quantities.These reductions are used to produce the reported visualizations.Source: Explanations for the D2Neighbors Model · p. 9

(Supplementary Notes)

The supplementary analyses examine Clever Hans effects across unsupervised representations, downstream classifiers, and anomaly-detection models, focusing on reliance on spurious artifacts and human features. They assess consequences for subgroup robustness and deployment, including performance under resizing and the mitigating effect of Gaussian blur.

Supplementary Notes

  • Supplementary analyses support that Clever Hans effects are prominent and distinct in unsupervised learning, motivating more robust and reliable downstream models.The notes provide detailed experimental results and analysis supporting the main paper’s claims.Source: (Supplementary Notes) · p. 13

Supplementary Note A. Additional Results for X-Ray Representations

  • PubMedCLIP’s X-ray representation relies on spurious annotations rather than pathology, producing subgroup-dependent similarity and classification behavior.BiLRP identifies annotation artifacts supporting similarity predictions, while subgroup patterns show NIH negatives classified as negative and many Github negatives as positive.Source: (Supplementary Notes) · p. 13
  • PubMedCLIP representations cluster more strongly by data provenance than COVID status, while Explainable AI can identify artifacts without prior knowledge of them.The t-SNE clusters show substantial overlap between Github COVID negatives and positives, whereas the XAI analysis does not require the artifact to be known beforehand.Source: (Supplementary Notes) · p. 14

Analysis of the Downstream Classifier

  • The downstream SVM inherits the representation’s Clever Hans strategy, relying on subgroup-specific artifacts and thereby producing unstable performance and Github false positives.LRP shows positive predictions supported by artifacts and negative predictions supported by distinct upper-right-corner artifacts; mismatched Github-negative artifacts explain their frequent positive misclassification.Source: (Supplementary Notes) · pp. 14–15

Supplementary Note B. Additional Results for Generic Image Models

  • Across generic image models, unsupervised similarity predictions use different spurious cues, whereas a supervised baseline focuses better on task-relevant features.CLIP relies on human faces and textual logos, while SimCLR and BarlowTwins use human bodies and suppress eccentric textual logos; the supervised baseline performs better on task-relevant features.Source: (Supplementary Notes) · p. 16

Analysis of Downstream Classifiers

  • CLIP achieves 86.6% truck accuracy on original images but relies on a logo artifact, while BarlowTwins, SimCLR, and the supervised baseline are largely logo-unresponsive.The original-data accuracies are 85.7% for the supervised baseline, 82.9% for BarlowTwins, and 77.7% for SimCLR.Source: (Supplementary Notes) · p. 18
  • On logo images, SimCLR, BarlowTwins, and the supervised baseline remain stable with a maximum performance drop below 1%, while CLIP becomes the worst performer, dropping almost five percentage points.A confusion matrix shows that 55.1% of CLIP’s errors are misclassifications into the “garbage truck” class.Source: (Supplementary Notes) · pp. 18–19
  • On fish classification, CLIP ranks first at 86.5%, yet all unsupervised models rely on humans rather than fish, whereas the supervised model focuses on the fish.CLIP focuses specifically on the human face; SimCLR and BarlowTwins focus on the body.Source: (Supplementary Notes) · p. 20

Supplementary Note C. Additional Results for Anomaly Detection

  • The anomaly-detection supplement reports Explainable AI analyses and performance before and after Clever Hans mitigation for retained MVTec categories, selected using D2Neighbors F1 scores above 0.9 on original data.The analysis is intended to test whether accurate anomaly predictions use valid strategies or are “right for the wrong reasons.”Source: (Supplementary Notes) · p. 20

Explainable AI analysis

  • D2Neighbors’ explanations reveal Clever Hans behavior: models respond to non-defect high-frequency artifacts, while true anomalies are mostly expressed at mid frequencies.Responses include bottle borders, capsule markings, pill specks, wood stripes, and toothbrush hairs that are typically unrelated to manufacturing defects.Source: (Supplementary Notes) · pp. 20–21, Figure 7 · p. 21

Data-quality shift

  • Antialiased resizing strips high-frequency noise and significantly reduces D2Neighbors’ F1 scores across categories, with especially severe degradation for wood.The effect is linked to spatially widespread high-frequency artifacts in the original wood images.Source: (Supplementary Notes) · pp. 21–22, Table 3 · p. 22

Clever Hans mitigation

  • An 11x11 pre-processing layer improves deployed-test F1 across all tested models and categories, with D2Neighbors ℓ2 recovering from 0.80 to 0.93 and PatchCore increasing from 0.85 to 0.96.Source: (Supplementary Notes) · p. 23

Data-quality shift

  • D2Neighbors’ sensitivity to antialiasing varies by norm configuration: one variant’s F1 score drops from 0.93 to 0.74, while the ℓ4-norm variant generally shows the strongest resilience across categories.For the wood category, the ℓ2- and ℓ4-norm variants perform comparatively better after deployment due to their pixel-sparsity mechanism.Source: (Supplementary Notes) · p. 22
  • PatchCore also degrades after deployment, with F1 falling from 0.92 to 0.85 overall and from 0.89 to 0.64 for toothbrush because antialiasing weakens hair-related anomaly features.This contrasts with D2Neighbors’ category pattern and may reflect amplification or suppression in PatchCore’s feature extractor.Source: (Supplementary Notes) · pp. 22–23

Related papers

Loading 2408.08041v1…