Source-linked AI summary
Interpretations are useful: penalizing explanations to align neural networks with prior knowledge
Laura Rieger, Chandan Singh, W. James Murdoch, Bin Yu
TL;DR
Deep neural networks can rely on spurious or biased relationships, while existing explanations often reveal these relationships without providing a way to correct them. CDEP uses contextual decomposition explanations as regularization targets, and experiments show it can remove bias and improve predictive accuracy across toy and real data.
Problem
Deep neural networks may rely on spurious correlations and sensitive-attribute bias, while explanation methods often reveal learned relationships without enabling practitioners to correct them.
Method
CDEP incorporates domain knowledge by directly penalizing feature and feature-interaction importances supplied through contextual decomposition explanations.
Results
CDEP removes bias and improves predictive accuracy across toy and real datasets, including skin-cancer images, misleadingly colored MNIST digits, text classification, and risk prediction.
Takeaways & Limitations
CDEP offers a versatile way to use explanation methods to align neural-network predictions with prior knowledge, with effectiveness depending on the quality of the explanation targets.
Takeaways & Limitations
Future work is needed to extend CDEP to more complex settings and incorporate finer-grained explanations and interaction penalizations.
Abstract
from arXiv · showhide
For an explanation of a deep learning model to be effective, it must provide both insight into a model and suggest a corresponding action in order to achieve some objective. Too often, the litany of proposed explainable deep learning methods stop at the first step, providing practitioners with insight into a model, but no way to act on it. In this paper, we propose contextual decomposition explanation penalization (CDEP), a method which enables practitioners to leverage existing explanation methods in order to increase the predictive accuracy of deep learning models. In particular, when shown that a model has incorrectly assigned importance to some features, CDEP enables practitioners to correct these errors by directly regularizing the provided explanations. Using explanations provided by contextual decomposition (CD) (Murdoch et al., 2018), we demonstrate the ability of our method to increase performance on an array of toy and real datasets.
1. Introduction
CDEP extends neural-network explanations from revealing learned relationships to penalizing incorrect feature importance, enabling models to incorporate prior knowledge while improving prediction across toy and real tasks.
- Deep neural networks can exploit spurious correlations and sensitive-attribute biases, while explanation methods can reveal the relationships they learn.
- CDEP directly penalizes feature or interaction importances so neural networks produce both correct predictions and explanations aligned with prior knowledge.
- CDEP can adapt to differentiable interpretation techniques, handle feature interactions, and apply across arbitrary neural-network architectures.
- In skin-cancer images, CDEP improves classifier prediction by teaching it to ignore spurious confounders in training data.
- CDEP regularizes a color-biased MNIST network toward digit shape and helps mitigate fairness issues in text classification and risk prediction.
3. Methods
The method augments prediction training with explanation supervision, using contextual decomposition scores and user-specified targets to penalize unwanted features or interactions. Its forward-pass formulation supports ordinary backpropagation, efficient fine-tuning, and substantially lower memory use than gradient-based alternatives.
- 3.1. Augmenting the loss function: CDEP augments the standard prediction loss with an explanation error that penalizes explanations inconsistent with user-provided targets.
- 3.1. Augmenting the loss function: Explanation targets are application-dependent; for skin images, assigning zero importance to band-aids teaches the model to ignore them.
- 3.2. Contextual decomposition (CD): Contextual decomposition decomposes logits into β(x), the importance of a selected feature group, and γ(x), the remaining contribution.
- 3.3. CDEP objective function: CDEP substitutes CD scores into the generic objective, applies SoftMax for explanation probabilities, and uses an L1 explanation loss against user targets.
- 3.4. Encoding prior knowledge: Users can specify ground-truth importances for individual features, programmatic rules, or arbitrary feature interactions, although human explanations can be expensive to collect.
- 3.5. Computational considerations: CD attributions are computed during the forward pass, allowing standard single-pass backpropagation instead of optimizing gradients of gradients.
- 3.5. Computational considerations: CDEP supports quick fine-tuning of pretrained networks by freezing early layers and incurs only a small constant memory increase over standard training.
4. Results
CDEP is evaluated on skin-cancer, ColorMNIST, and COMPAS tasks to correct reliance on spurious or sensitive features. Across these experiments, it improves generalization, partially shifts ColorMNIST models toward shape, and reduces and equalizes wrongful-conviction rates while preserving accuracy.
- 4.1. Ignoring spurious signals in skin cancer diagnosis: The ISIC task contains colorful patches in approximately 50% of non-cancerous images, creating a spurious cue for classification.The dataset includes 21,654 diagnosed images after excluding uncertain diagnoses; patches are identified using image segmentation and color filtering.
- 4.1. Ignoring spurious signals in skin cancer diagnosis: CDEP improves ISIC skin-cancer AUC and F1 on both the full test set and the no-patches test set.The comparison averages results over three runs.
- 4.1. Ignoring spurious signals in skin cancer diagnosis: CDEP achieves the best F1 score on ISIC, surpassing both unpenalized models.The comparison includes models trained on data without spurious patches and on the full dataset.
- 4.1. Ignoring spurious signals in skin cancer diagnosis: The RRR comparison did not improve base AUC and severely decreased performance across all considered metrics.The authors attribute this result to gradient penalization being unhelpful for higher-order features and impeding relevant-feature learning.
- 4.2. Combating inductive bias on variants of the MNIST dataset: On ColorMNIST, CDEP raises inverted-color test accuracy to 31.0%, whereas RRR and EG remain at or below the 10% random baseline.The task tests whether models can shift from color to digit shape; increasing CDEP regularization further increases test accuracy.
- 4.3. Fixing bias in COMPAS: On COMPAS, CDEP lowers wrongful-conviction rates for both black and white defendants and brings the rates closer together while test accuracy stays relatively fixed.The regularizer increases importance assigned to race and its interactions, encouraging the model to learn relationships between race and other-feature distributions.
Positive
The SST examples illustrate artificially induced gender bias, while CDEP improves accuracy by ignoring injected spurious signals.
- The SST examples contain movie-review sentences labeled positive or negative.
- Three SST variants inject spurious class signals, including indicator words and gender-linked words.
- CDEP substantially improves predictive accuracy on the unbiased SST test set after biased training.
5. Conclusion
The paper concludes that CDEP aligns neural networks with prior knowledge, handles complex explanations efficiently, and improves accuracy while removing bias across datasets.
- CDEP penalizes complex features and feature interactions to align neural networks with prior knowledge.
- CDEP is more computationally efficient than previous work, enabling use with more complex neural networks.
- Experiments show CDEP removes bias and improves predictive accuracy across toy and real data.
- Future work includes extending CDEP to more complex settings and adding finer-grained explanations and interaction penalizations.
S1. Additional details on MNIST Variants
Additional MNIST analyses describe pixel sampling, compare Expected Gradients with CDEP, and report computational measurements under task-specific conditions.
- ColorMNIST bias correction samples pixels from the distribution of non-zero pixels across the training set.
- Expected Gradients remains at random accuracy, including when RGB-attribution variance is penalized.
- Runtime and memory comparisons were conducted in PyTorch on DecoyMNIST with batch size 64.
- Exact computational ratios depend on network complexity and batch size because constant memory becomes proportionally smaller at larger batches.
- Expected Gradients and RRR require two forward and backward passes, so maximum memory usage was recorded on a single Titan X.
S2. Image segmentation for ISIC skin cancer
For ISIC skin-cancer images, the supplementary procedure segments images and identifies atypical-color regions to create binary patch maps.
- SLIC segments each skin-cancer image, with visually distinct patches usually forming their own segments.
- Segments are filtered using mean RGB and HSV values that differ substantially from typical Caucasian skin tone.
- The filtered segments are exported as binary maps identifying candidate patches.
S3. Additional heatmap examples for ISIC
Additional ISIC examples show that CDEP preserves relevant importance patterns while removing reliance on spurious visual cues, including patches and rulers.
- CDEP regularization leaves importance maps similar for cancerous images and benign images without patches, while causing the regularized network to ignore patches.
- Rulers appeared in roughly 13% of cancerous images versus 5% of benign images, yet networks recognized and exploited this correlation.
- Both networks learned the non-penalized spurious association between rulers and cancer.
- Heatmap visualizations multiply importance maps by the image, making visible regions correspond to features important for classification.
S4. Additional details about the COMPAS task
The COMPAS experiments use a filtered, preprocessed dataset focused on parity between Black and Caucasian groups, with an 80/10/10 data split.
- The COMPAS network has two hidden layers with five neurons each and is trained using SGD with learning rate 0.01, momentum 0.9, and 0.1 dropout.Training continued until loss failed to improve for ten epochs.
- 1042 of 7214 full-dataset samples were excluded because recidivism information was missing.
- Features include categorized age, sex, race, crime severity, prior-felony count, and crime-description categories.
- The analysis focuses on parity between Black and Caucasian groups and excludes youth felony count because positive samples were very sparse.
- The data are split into 80% training, 10% validation, and 10% test sets.
S5. Additional details about SST task
The SST experiments construct three biased variants by associating sentiment with inserted or substituted words, including signals present in only 2% of the dataset.
- Variant 1: Variant 1 inserts “text” into positive sentences and “video” into negative sentences, with both decoy words appearing in 100% of sentences.
- The supplementary figures provide example sentences for the decoy-word, gender, article, and rare gender-word biases.
- Variant 2: Variant 2 replaces “the” and “a” in 27% of sentences so that each substituted word appears exclusively in one sentiment class.
S6. Network architectures and training
The experiments use VGG16 for ISIC, convolutional networks for MNIST, and an LSTM for SST, while explanation penalization adds a tunable training hyperparameter.
- ISIC: The ISIC task uses a pretrained VGG16 network with SGD, learning rate 0.01, and momentum 0.9.Preliminary Adam experiments produced poorer predictive performance.
- MNIST: The MNIST architecture has two convolutional channels with max pooling followed by two fully connected layers.
- Explanation penalization: CDEP introduces λ as an additional hyperparameter, tested across values from 10^-1 to 10^4 and set at a fixed rate in this paper.The authors expect proportional scaling with the normal training loss could produce more stable training.
- SST: The SST model consists of two LSTM layers with 128 hidden units followed by a fully connected layer.