PaperlayerView exact revision on arXiv

Source-linked AI summary

Revisiting Explainable AI through Model-Independent Concept Dictionaries

Thomas Schnake, Doreen Schöppenthau, Alexander Meyer, Jacques Corbeil, Klaus-Robert Müller, Grégoire Montavon

arXiv:2610.10301v1cs.LGstat.ML

TL;DR

Complex AI models motivate XAI, but latent concept methods rely on internal representations that can be difficult to characterize and compare across architectures. DictXAI defines concepts with input-domain dictionaries and attributes predictions to dictionary elements. Its application-grounded evaluations report actionable insights across shortcut artifacts and biomedical signals.

  • Problem

    Latent concept methods rely on internal representations whose units can be polysemantic and whose representations are not comparable across architectures.

    Source: §1 · p. 2

  • Method

    DictXAI projects inputs onto predefined, potentially overcomplete dictionaries and attributes model predictions to the associated dictionary elements.

    Source: §1 · p. 2

  • Results

    Across application-grounded use cases, DictXAI reports advantages over classical XAI and concept-based extensions, including identifying shortcut artifacts and interpreting ECG and microbial spectra.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The authors report human-understandable, actionable insights across data modalities and concept spaces, including removal of offending input-space atoms to block a flawed strategy.

    Source: §1 · p. 3

    The current framework uses fixed-dimensional vectors, limiting out-of-the-box use on variable-sized images, raw text, and non-Euclidean data; overcomplete dictionaries also face dimensionality scaling.

    Source: Limitations and Future Work · pp. 17–18

Abstract

from arXiv · show

Modern applications of AI rely on increasingly complex models. Explainable AI (XAI) has emerged as a set of techniques aimed at improving model transparency. However, existing XAI methods typically assume input features to be inherently interpretable, or they rely on intermediate internal abstractions that are difficult to characterize and highly architecture-specific, hindering consistent use across models. To address these limitations, we propose DictXAI, a method that defines concepts directly in the input domain via a dictionary---a large, potentially overcomplete set of predefined elements, each carrying an interpretable meaning. Technically, DictXAI first computes a sparse code of the input and then attributes the model's prediction to the associated dictionary elements. We demonstrate the actionable nature of DictXAI explanations, showing that they can attribute AI malfunctions (e.g., Clever Hans effects) directly to identifiable artifact patterns in the data, while fostering human-AI alignment on intricate biomedical signals. We further demonstrate our method's ability to operate across a wide variety of dictionaries, including learned image bases, analytically defined waveforms for electrocardiography, and experimentally acquired dictionary elements. Overall, our results show that DictXAI provides more interpretable, actionable, and architecture-agnostic insights than classical XAI or existing concept-based approaches.

1. Introduction

DictXAI defines concepts directly in the input domain through human-inspectable, potentially overcomplete dictionaries, avoiding reliance on architecture-specific hidden representations. Its application-grounded evaluation reports actionable explanations across shortcut mitigation and multiple data modalities.

1. Introduction

  • The introduction frames XAI as addressing the opacity of complex AI models, while explanation quality also requires faithfulness, understandability, and actionability for downstream tasks.The stated motivations include regulatory demands and scientific inquiry.Source: §1 · pp. 1–2
  • DictXAI projects inputs onto a predefined, human-inspectable dictionary and attributes predictions to its elements, with overcompleteness supporting expressive concept extraction in the input domain.The framework avoids complex nonlinear encoders and lets explanations be governed by the original model and explicit dictionary elements.Source: §1 · pp. 2–3
  • Across application-grounded use cases, DictXAI reports advantages over classical XAI and concept-based extensions, including identifying shortcut artifacts and interpreting ECG waveforms and microbial spectra.The authors describe the resulting insights as human-understandable and actionable.Source: §1 · p. 3

2. Related Work

  • Related work is organized around concept-based explanations of internal representations and attribution methods that construct bases directly in the input domain.The paper identifies LRP as a foundation for its explanation framework.Source: §2 · p. 3

2.1. Concepts in Latent Activation Layers

  • Latent concept methods use units, concept-discriminative directions, subspaces, dictionary learning, or sparse autoencoders, but hidden units can be polysemantic and representations differ across architectures.DictXAI instead uses input-defined concepts with explicit semantics and supports comparisons across architectures.Source: §1 · p. 2, §2.1 · pp. 3–4

2.2. Concepts within the Input Domain

  • Input-domain precedents use Fourier or wavelet bases, but their fixed, non-redundant transforms limit use of empirical knowledge, learned dictionaries, and non-orthogonal primitives.DictXAI's overcomplete dictionaries can distinguish correlated variations such as waveform scale shifts and overlapping microbial spectral signatures.Source: §2.2 · p. 4

3. The DictXAI Method

DictXAI represents inputs with sparse combinations of interpretable dictionary atoms, preserves the original model prediction, and attributes that prediction to the atoms. Its dictionary design sets a trade-off between data fit, semantic transparency, and physical grounding, while its properties include concept steerability, fine-grained explanations, and stability guarantees.

3. The DictXAI Method

  • DictXAI first sparsely codes an input, restores exact reconstruction with an instance-specific residual atom, and attributes the model output to dictionary atoms.Attribution options include occlusion or Shapley sampling, Integrated Gradients, and LRP; the formulation remains functionally equivalent to the original model.Source: §3 · pp. 5–6

3.1. Choice of Dictionary

  • Learned dictionaries capture data-specific primitives but lack explicit formulas and can overfit when data are scarce; analytical dictionaries expose interpretable parameters but may require extensive discretization.Empirical constituent libraries ground atoms in real-world entities, but cannot represent unmodeled components; hybrid dictionaries can add learnable atoms to absorb residual variation.Source: §3.1 · pp. 6–8
  • MSE = 0.032 vs. 0.017: the overcomplete Gabor dictionary has almost twice the reconstruction error of the learned baseline despite containing over 12× more elements.Figure 2 contrasts learned atoms with Gabor parameter sweeps and shows class-aggregated coefficient and relevance profiles over spatial position and orientation.Source: Figure 2 · p. 7, §3.1 · p. 8

3.2. Properties of DictXAI

  • Under an L-Lipschitz model gradient, dictionary-level derivatives are (L∥d_j∥_2)-Lipschitz, and attribution stability extends to specified gradient-product and stabilized LRP methods under stated regularity conditions.The active support and sign pattern must remain locally invariant for the coefficient mappings to be locally Lipschitz.Source: §3.2 · p. 9
  • Curated dictionaries let practitioners steer concept vocabularies toward task-specific ontologies, unlike the fixed transforms or unsteerable latent decompositions described as baselines.This is the understandability criterion of concept steerability.Source: §3.2 · p. 9
  • Overcomplete dictionaries support fine-grained input primitives, while learned atoms can capture human-aligned concepts missing from classical analytical families.The paper frames both granularity and discovery as aspects of explanation sufficiency.Source: §3.2 · p. 9
  • DictXAI avoids intermediate-activation hooks; curated dictionaries need no auxiliary training pipeline, while learned dictionaries add unsupervised dictionary learning.The paper reports that both regimes share faithfulness and computational complexity but trade off usability, steerability, and annotation requirements.Source: §3.2 · pp. 8–10
  • Sparse coding once per sample plus one forward–backward pass costs SC + f, avoiding the K sequential backward sweeps required by intermediate-layer concept attribution.DictXAI’s single extended-model pass provides concept attribution and pixel-level grounding without cost scaling linearly with queried concepts.Source: §3.2 · p. 10

4. Evaluation

The evaluation tests input-domain explanations as tools for data-level shortcut removal and for interpreting an ECG model’s temporal strategy. Across these tasks, DictXAI supports targeted interventions and explanations tied to domain-relevant features.

4. Evaluation

  • The evaluation uses task-specific criteria: decorrelated-test accuracy for shortcut removal and independently verifiable failure modes for ECG analysis.For mass spectrometry, the criterion is whether representations can be calibrated for downstream microbial analysis.Source: §4 · pp. 10–11

4.1. Task 1: Data-Level Mitigation of the Clever Hans Effect

  • For contaminated MNIST, DictXAI attributions guide removal of watermark components from training samples before a fresh model is trained and tested on decorrelated data.The three artifacts are a fixed horizontal stripe, a variably located dotted line, and a signature glyph, each inserted into class 6.Source: §4.1 · p. 11
  • Across all three watermark variants, DictXAI leads the ROAR comparison and closely matches the clean-data oracle on decorrelated-test accuracy.Figure 4 plots accuracy against the fraction of concepts removed before fine-tuning.Source: §4.1 · p. 12, Figure 4 · p. 13
  • Peak accuracy requires removing 2.8%, 5.8%, and 1.0% of DictXAI elements for horizontal, dotted, and signature artifacts, versus 7.5%, 9.6%, and 5.7% of LRP pixels.These removal budgets show more compact concept granularity than raw-pixel representations.Source: §4.1 · p. 12
  • DictXAI pruning leaves sanitized digits resembling authentic inputs, and only its retrained model lacks qualitatively visible watermark reliance.Other methods introduce image distortions or leave shortcut reliance visible in retrained-model explanations.Source: Figure 3 · p. 12, §4.1 · p. 13

4.2. Task 2: Mapping an ML Decision Strategy to the Human Expert Domain

  • The ECG task classifies narrow (≤85 ms) versus prolonged (≥105 ms) QRS complexes, using a 300,000-atom waveform dictionary and 150 nonzero coefficients per recording.Borderline cases from 85–105 ms are excluded; the model is trained on 10-second Lead I traces from MIMIC-IV-ECG.Source: §4.2 · pp. 13–14
  • DictXAI attributes class evidence to distinct QRS intervals, and moderate noise shifts the model toward Class 1, supporting an explanation-derived finding of asymmetric reliance on Class 2 features.The noise perturbation revealed a failure mode in which prolonged QRS complexes may be missed and the prediction revert to a normal-conduction baseline.Source: §4.2 · p. 14

B Model's Explanation A ECG Data

The ECG example connects model predictions to QRS morphology and illustrates how noise can expose a model’s reliance on prolonged complexes. In microbial mass spectrometry, DictXAI’s isolate-level explanations help distinguish meaningful shared taxa from spectral background when comparing preprocessing choices.

B Model's Explanation A ECG Data

  • The ECG figure contrasts amplitude-driven LRP explanations with DictXAI’s more discriminative attributions to distinct AMS widths, and shows moderate noise shifting predictions toward the Class 1 baseline.The accompanying analysis describes this as asymmetric reliance on Class 2 features and a vulnerability to missing prolonged QRS complexes.Source: Figure 5 · p. 15, B Model's Explanation A ECG Data · p. 15

4.3. Task 3: Guided Design of Data Representations

  • For microbial mixtures, the dictionary contains 1,104 prototype spectra from 23 isolates, and similarity is modeled as the inner product of their transformed representations.Each isolate is represented by 48 technical replicates; Φ denotes candidate preprocessing transformations.Source: §4.3 · p. 16
  • DictXAI represents mixture similarity with bipartite relevance graphs linking activated dictionary coefficients across the two mixtures.The graphs show whether a candidate representation emphasizes shared taxa or spurious baseline artifacts.Source: §4.3 · p. 16
  • With raw spectra, similarity attribution is diffuse and driven by broad background correlations; after band-pass filtering, relevance concentrates on shared taxa such as Clostridium tertium and DH5a-K12.The comparison supports using DictXAI as an unsupervised diagnostic for representation design without downstream labels or ad-hoc task performance.Source: Figure 6 · p. 16, §4.3 · p. 17

5. Conclusion

DictXAI defines concepts in the input domain through dictionaries and sparse codes, providing granular attributions tailored to domain needs. Its three application-grounded evaluations illustrate this approach across shortcut removal, ECG analysis, and microbial mass spectrometry.

5. Conclusion

  • DictXAI defines concepts directly in the input domain through dictionaries and attributes predictions over sparse codes of dictionary atoms.Dictionary design allows the attribution basis to be tailored to domain needs.Source: §5 · p. 17
  • Input-domain concepts are human-inspectable and mathematically defined, avoiding latent-layer probing and enabling comparisons across distinct architectures.Input-level representations and interventions make concepts actionable at their source.Source: §5 · p. 17
  • The evaluation spans three uses: removing shortcuts in contaminated MNIST, auditing QRS-duration decisions in ECG, and guiding microbial mass-spectrometry preprocessing.The applications use learned image bases, parametric AMS waveforms, and empirical isolate spectra, respectively.Source: §5 · p. 17

Limitations and Future Work

DictXAI is currently limited to fixed-dimensional vector inputs, while its fine-grained input-space concepts motivate future clinical comparisons of similar disease phenotypes.

Limitations and Future Work

  • Fixed-dimensional vector representations restrict DictXAI’s out-of-the-box use on variable-sized images, raw text, and non-Euclidean data.The authors suggest convolutional sparse coding or adaptive dictionary learning as possible extensions.Source: Limitations and Future Work · p. 17
  • Overcomplete dictionaries may require exponentially more atoms as input dimensionality grows to preserve concept granularity.Suggested responses include factored dictionaries, hierarchical sparse representations, and deep dictionary formulations.Source: Limitations and Future Work · pp. 17–18
  • Future work could compare shared and pathology-specific atoms across clinically similar disease phenotypes to support hypothesis generation and differential diagnosis.The proposed extension moves from auditing isolated diagnoses to fine-grained concept mapping.Source: Limitations and Future Work · p. 18

Declaration of Generative AI and AI-assisted technologies in the writing process

The authors report using Gemini 3.8 and Claude 5 to improve formulations during preparation of the work. They reviewed and edited the resulting content and retain full responsibility for the publication.

Declaration of Generative AI and AI-assisted technologies in the writing process

Supplementary Note A. Specification and Training of Image Dictionaries

The supplementary note describes image dictionaries for sparse input representations, including analytically constructed Gabor atoms and learned dictionaries trained with a sparsifying auto-encoder.

Supplementary Note A. Specification and Training of Image Dictionaries

  • Inputs are represented as sparse linear approximations x ≈ Dα, with dictionary D spanning the flattened 28×28 MNIST image vectors.The note describes analytical Gabor atoms and data-learned atoms as construction options.Source: Supplementary Note A. Specification… · p. 23
  • Each Gabor atom uses a cosine carrier under a Gaussian envelope, with rotated coordinates and carrier frequency ν/σ, where ν = 0.4 cycles per envelope width.The dictionary combines the parameter grids in Table 1 with a constant DC atom, giving 10,001 L2-normalized atoms.Source: Supplementary Note A. Specification… · p. 23, Table 1 · p. 23
  • Learned dictionaries are trained by stochastic optimization of a single-layer auto-encoder whose sparsifying and ReLU nonlinearities produce activations for reconstruction.The loss combines reconstruction and activation-balancing sparsity terms.Source: Supplementary Note A. Specification… · pp. 23–24
  • For the Figure 2A dictionary, training adds input noise while reconstructing the noise-free input and applies two random translations of up to 3 pixels per axis.The stored dictionary uses an exponential moving average of weights with decay 0.99.Source: Supplementary Note A. Specification… · p. 24
  • Training uses SGD with momentum 0.9, Gaussian initialization at scale 0.01, and separate warmup and main learning rates.The warmup rate applies during the first 1,000 epochs; Table 2 summarizes learning-dictionary hyperparameters.Source: Supplementary Note A. Specification… · p. 24, Table 2 · p. 24

Supplementary Note B. Experimental Details for Section 4.1

The supplementary note details the MNIST watermark experiments, model training, test-set construction, and baseline settings. It also specifies the fine-tuning and evaluation procedure.

Supplementary Note B. Experimental Details for Section 4.1

  • Three independent watermark variants are injected into all class-6 training images in a 5,000-image MNIST subsample split 90/10 by class.Figure 1 shows each variant’s watermark instances, with an unmodified digit first in each row.Source: Supplementary Note B. Experimental… · p. 24, Figure 1 · p. 25
  • Each variant has a spurious-correlation test set with watermarks on all class-6 images and a decorrelated set with watermarks on 20% of images sampled across classes.The decorrelated set removes the watermark’s class-6 predictive signal.Source: Supplementary Note B. Experimental… · p. 24
  • Five seeded three-layer convolutional networks are trained per variant for 10 epochs using Adam and cross-entropy.The architecture ends with fully connected layers of sizes 128 and 10.Source: Supplementary Note B. Experimental… · p. 25
  • LRP and DFT-LRP use LRP-γ with γ = 0.1 in convolutional and hidden fully connected layers, while Shapley sampling uses 49 non-overlapping 4×4 patches.Shapley sampling uses 10 random permutations per sample and a zero baseline in normalized input space.Source: Supplementary Note B. Experimental… · p. 25
  • WAM uses a db5 wavelet basis at level 3 and coefficient-times-gradient relevance; CartoonX uses the same transform with an optimized rate-distortion mask.CartoonX uses 100 optimization steps, an ℓ1 penalty of 0.6, and 10 Gaussian noise samples per step.Source: Supplementary Note B. Experimental… · pp. 25–26
  • Each of five seeded base models is fine-tuned for 10 epochs on the cleaned spurious-correlation test set, then evaluated on the decorrelated test set.Fine-tuning uses Adam with learning rate 10^-4, batch size 64, and cross-entropy.Source: Supplementary Note B. Experimental… · p. 26

Supplementary Note C. Concept-Filtered Pixel-Wise Heatmaps

The note compares how different attribution methods construct concept-conditioned pixel heatmaps. DictXAI uses its dictionary synthesis representation to produce a heatmap directly, while some transform-based methods have distinct attribution structures.

Supplementary Note D. Experimental Details for Section 4.2

The ECG dictionary approximates each signal with sparse combinations of parameterized atoms. Its oscillatory atoms model QRS complexes, while Gaussian pulses model smoother P and T waves.

Model architecture and training procedure

The classifier is a residual convolutional network modified to use average pooling, which supports canonical relevance propagation. It is trained on labeled single-lead ECG recordings and reaches 95.03% test accuracy.

Loading 2610.10301v1…