Source-linked AI summary
GradMask: Reduce Overfitting by Regularizing Saliency
Becks Simpson, Francis Dutil, Yoshua Bengio, Joseph Paul Cohen
TL;DR
Generic regularization methods do not specifically penalize incorrect associations between input features and the target class. GradMask penalizes gradient-inferred saliency when it is inconsistent with lesion segmentation. GradMask achieves between 1-4% increase in test accuracy.
Problem
Generic regularization methods do not specifically penalize incorrect associations between input features and the target class.
Source: §1 · p. 2Method
GradMask penalizes gradient-inferred saliency when it is inconsistent with lesion segmentation.
Source: §1 · p. 2Results
GradMask achieves between 1-4% increase in test accuracy.
Source: §2 · p. 3Takeaways & Limitations
The results indicate that GradMask can decrease overfitting on small datasets.
Source: §3 · p. 3On some tasks, unsuitable CNN baseline accuracy hindered improvement; small tumors may also have been eroded by slice resizing.
Source: §3 · p. 3
Abstract
from arXiv · showhide
With too few samples or too many model parameters, overfitting can inhibit the ability to generalise predictions to new data. Within medical imaging, this can occur when features are incorrectly assigned importance such as distinct hospital specific artifacts, leading to poor performance on a new dataset from a different institution without those features, which is undesirable. Most regularization methods do not explicitly penalize the incorrect association of these features to the target class and hence fail to address this issue. We propose a regularization method, GradMask, which penalizes saliency maps inferred from the classifier gradients when they are not consistent with the lesion segmentation. This prevents non-tumor related features to contribute to the classification of unhealthy samples. We demonstrate that this method can improve test accuracy between 1-3% compared to the baseline without GradMask, showing that it has an impact on reducing overfitting.
1. Introduction
Overfitting in medical imaging can make models rely on institution-specific artifacts rather than biologically relevant features. GradMask regularizes gradient-based saliency against lesion segmentation to discourage such incorrect feature attribution.
1. Introduction
- Institution-specific acquisition features can become predictive shortcuts, motivating regularization that penalizes relevance assigned to features doctors would not use.Examples include scan equipment and acquisition-parameter variations rather than biological effects.Source: §1 · p. 1
- Generic regularizers constrain model capacity but do not specifically correct incorrectly assigned feature attribution, leaving room to target spurious correlations directly.Gradient-based attribution can highlight image features the model considers predictive for a class.Source: §1 · p. 2
- GradMask penalizes input-feature attribution when it conflicts with ground-truth segmentation, discouraging use of features outside a lesion for non-healthy predictions.The method uses input gradients to identify discriminative features for a class.Source: §1 · p. 2
- The proposed objective adds a saliency-based penalty to classification loss, with an alternative contrast saliency between healthy and non-healthy classes.The contrast formulation treats input variation affecting both classes as distinct from variation that increases their distinction.Source: §1 · pp. 2–3
2. Experiments
Experiments compared contrast GradMask with a no-GradMask baseline across datasets and training sample sizes. The reported results show increased test accuracy with GradMask.
2. Experiments
- Experiments compared the contrast GradMask method with the architecture without GradMask, varying training samples from 64 to 512; sample sizes of 64–128 first indicated improvement over the base model.Validation AUC served as a proxy for tuning relevant hyperparameters per seed. A 20-trial search with a different seed each was conducted across four datasets to select runs with the best validation AUC.Source: §2 · p. 3
- In the reported experiments, the gradmask method achieved between 1% and 4% increase in test accuracy, with these results shown in Table 1.Source: §2 · p. 3
- Table 1 reports mean test AUC and standard deviation over 20 seeds for GradMask-Contrast versus no GradMask.The supplied table caption gives its scope but no cell values.Source: Table 1 · p. 3
3. Conclusion
The authors conclude that GradMask can decrease overfitting on small datasets, while noting that unsuitable baseline accuracy on some tasks hindered improvements.
3. Conclusion
- GradMask is reported to decrease overfitting on small datasets.On some tasks, unsuitable CNN baseline accuracy hindered improvements; small tumor size and slice resizing may have eroded tumors.Source: §3 · p. 3