Source-linked AI summary
Real Time Image Saliency for Black Box Classifiers
Piotr Dabkowski, Yarin Gal
TL;DR
The paper addresses the difficulty of obtaining interpretable saliency maps quickly and without artifacts for differentiable image classifiers. It trains a masking model to generate maps in one forward pass, reporting real-time operation and stronger evaluation results than competing weakly supervised techniques. The work also introduces a saliency metric and identifies bias in the masking model as an area for further study.
Problem
Saliency detection must reveal classifier-relevant regions, but iterative methods are slow and masking can introduce adversarial artifacts that produce poor explanations.
Method
A trainable masking model takes an image and class selector as inputs and learns to produce salient-region masks in a single forward pass without post-training access to the classifier.
Results
The model produces 100 saliency masks per second, outperforms other weakly supervised techniques on ImageNet localisation, and surpasses popular saliency detectors under the new metric.
Takeaways & Limitations
The method provides real-time saliency maps for differentiable classifiers, while the new crop-based metric evaluates whether compact preserved regions retain the requested class.
Takeaways & Limitations
Iteratively finding masks is slow and can overfit individual images, while the masking model may itself contain biases that affect generated saliency masks.
Abstract
from arXiv · showhide
In this work we develop a fast saliency detection method that can be applied to any differentiable image classifier. We train a masking model to manipulate the scores of the classifier by masking salient parts of the input image. Our model generalises well to unseen images and requires a single forward pass to perform saliency detection, therefore suitable for use in real-time systems. We test our approach on CIFAR-10 and ImageNet datasets and show that the produced saliency maps are easily interpretable, sharp, and free of artifacts. We suggest a new metric for saliency and test our method on the ImageNet object localisation task. We achieve results outperforming other weakly supervised methods.
1 Introduction
Image classifiers can achieve high accuracy while relying on unintended evidence, motivating saliency methods that reveal which image regions drive predictions. The paper proposes a trainable approach that produces accurate saliency maps in one feed-forward pass, making detection much faster than iterative methods.
- 1 Introduction: High classifier accuracy does not guarantee that the model recognizes the intended object rather than a correlated cue such as snow.Saliency detection can test what the classifier is actually using without requiring separate object-only and context-only evaluation images.
- 1 Introduction: Saliency maps identify the image regions most relevant to a classifier for a particular input class.A common strategy searches for the smallest region whose removal substantially reduces the classification score.
- 1 Introduction: Iterative saliency detection often requires hundreds of iterations, making it time consuming.
- 1 Introduction: The proposed model predicts a saliency map for any input image in a single feed-forward pass instead of optimizing each image separately.The authors report orders-of-magnitude speedups, higher-quality masks, and better localisation results than iterative methods.
2 Related work
Prior saliency methods include gradient-based, iterative removal, and optimization-based approaches, each balancing speed, interpretability, or explanation quality. The paper positions its method as a new fast and accurate model-based detector and introduces a saliency metric.
- 2 Related work: Gradient-based methods are fast enough for real-time use but produce limited-quality explanations and are difficult to improve.
- 2 Related work: Iterative patch-removal methods produce interpretable maps but are too time consuming for real-time saliency detection.
- 2 Related work: Optimization-based methods iteratively preserve or isolate activations and salient image regions to maintain or reduce class scores.Cao et al. optimize network activations, while Fong and Vedaldi optimize in image space with the classifier treated as a black box.
- 2 Related work: The paper proposes a fast, accurate, model-agnostic saliency detector and a new metric for evaluating saliency quality.
3 Image Saliency and Introduced Evidence
Saliency maps summarize where a classifier relies on an image, but masking can introduce misleading evidence and artifacts. The paper therefore combines SSR/SDR objectives, artifact-resistant masking, and a crop-based metric for evaluating compact, class-preserving regions.
- Image Saliency: Saliency maps indicate which image parts are relevant to a classifier, but no single obvious metric fully measures their quality.The paper frames saliency as a summarized explanation of where the classifier looks for a particular class.
- Image Saliency: SSR is the smallest region that alone supports confident classification, whereas SDR is the smallest removed region that prevents confident classification.The paper seeks regions that perform well as both sufficient and destroying regions.
- Introduced Evidence: Masking can introduce side-effect evidence, such as a constant green fill suggesting grass and increasing the probability of grass-associated classes.Removing evidence by blurring, constant colour, noise, or cropping can alter what the classifier detects.
- Introduced Evidence: Element-wise masking can create small adversarial artifacts that substantially change classifier predictions, especially when the mask is not smooth.The paper illustrates this with a mask that causes an image previously classified confidently as a tabby cat to lose that classification.
- Fighting the Introduced Evidence: Using a blurred alternative image makes high-frequency artifacts harder to generate, but blur may fail to remove low-frequency evidence in seashore or mountain images.Random noisy constant colour is another proposed alternative, while total variation and mask resizing encourage smoothness.
- A New Saliency Metric: The proposed saliency metric evaluates the tightest rectangular crop containing the salient region by combining crop area with the classifier’s requested-class probability.With ã = max(a, 0.05), lower metric values indicate smaller crops that preserve classification probability; the metric is intended to be negative for good classifier-detector pairs.
- The Saliency Objective: The saliency objective balances mask smoothness, region size, recognition from the preserved region, and low class probability after salient-region removal.The removal term applies the inverted mask, and setting λ4 below 1 can help reduce the removed-region probability.
4 Masking Model
The masking model replaces slow, image-specific mask optimization with a trainable model that predicts class-conditioned saliency masks in one forward pass. Its multiscale architecture and training choices target sharp, precise masks while retaining real-time throughput.
- Motivation: Iterative mask optimization requires more than 100 iterations and can produce blurry, imprecise, image-overfit masks with adversarial artifacts.A large total-variation penalty is needed to prevent artifact formation.
- Masking Model: The trainable masking model takes an image and class selector as inputs and learns to minimize the masking objective without classifier access after training.The model acts as a regularizer that makes adversarial masks harder to learn in practice.
- Architecture: The model adapts U-Net with multiresolution feature maps, using a pretrained ResNet-50 encoder and class-conditioned feature filtering for initial localization.ResNet feature maps are combined with upsampled representations to support mask prediction at multiple resolutions.
- Architecture: A two-channel output, C0 and C1, is converted into the mask Ms using a nonstandard nonlinearity and then bilinearly upsampled fourfold to obtain M.The two-channel formulation improved training compared with sigmoid and tanh nonlinearities.
- Efficiency: The model has complexity comparable to ResNet-50 and processes more than 100 224x224 images per second on a standard GPU.This throughput is reported as sufficient for real-time saliency detection.
- Training: The masking model minimizes the objective while keeping the pretrained ResNet encoder weights fixed during training.Training also varies class selectors and evidence-removal alternatives, including blurred or randomly colored noisy images.
5 Experiments
Experiments on ImageNet and CIFAR-10 evaluate masking models trained with different black-box classifiers using localisation, saliency, and interpretability criteria. The method produces sharp, interpretable masks and achieves competitive or improved localisation results without localisation training data.
- Experimental setup: The experiments assess the technique on ImageNet and CIFAR-10 using multiple metrics, baselines, and black-box classifiers.ImageNet experiments train separate masking models for AlexNet, GoogLeNet, and ResNet-50; CIFAR-10 uses a modified architecture for low-resolution images.
- ImageNet saliency maps: The masking models trained on GoogLeNet and ResNet-50 produce sharp, precise regions, while the AlexNet model produces stronger and slightly larger regions.The authors suggest that AlexNet may need more evidence for successful classification, leading to larger saliency regions.
- Weakly supervised object localisation: 36.7% localisation error was achieved by the model trained on the ResNet-50 black box, compared with 34.3% for a fully supervised VGG approach.The masking model received no localisation training data, and the GoogLeNet model achieved a similar error according to the authors.
- Weakly supervised object localisation: The GoogLeNet model outperforms other approaches, the Center baseline, and iteratively optimised saliency masks on the reported localisation comparison.The centrally placed box remains a relatively strong baseline because many ImageNet images contain a large dominant object near the center.
- Saliency metric: The model achieves a considerably better saliency metric than other saliency approaches and is on par with ground-truth boxes.It also significantly outperforms the Max box and Center box baselines, supporting similar interpretability of its localisation boxes to ground-truth boxes.
- CIFAR-10 evaluation: On CIFAR-10, the produced maps are clearly interpretable, allowing humans to recognise the original objects after masking.The result confirms that the modified masking model works at low resolution and does not need a pre-trained model that could inject its own biases.
6 Conclusion and Future Research
The paper concludes that its fast masking approach supports real-time saliency and strong explanation quality, while identifying extensibility and masking-model bias as future concerns.
- 100 saliency masks per second make the method sufficient for real-time applications.
- The method outperforms other weakly supervised techniques on ImageNet localisation.
- Under the proposed saliency metric, the model outperforms popular saliency detectors and matches ground truth bounding boxes.
- The model-based design can be extended through masking-network architecture changes or objective-function changes targeting desired mask properties.
- Future work includes weakly supervised image segmentation, video saliency, and studying biases embedded in the masking model.
A Appendix
The appendix visualizes the masking model’s ImageNet class embedding and presents saliency masks for varied target classes, including cases where context matters or the target is absent.
- The ImageNet class embedding places closely related objects near one another in the learned t-SNE visualization.Fungi and geographical formations form clusters, while similar dogs occupy similar positions.
- The appendix shows masks generated for multiple selected target classes across varied ImageNet examples.Figures 9–14 provide additional saliency examples with different class selectors.
- The cat is masked when desk is selected because the cat does not contribute to that class.
- The model generates no mask when the selected target class is absent from an image.This is illustrated for Irish setter and street sign selections.
- For ski and bearskin, the classifier apparently requires supporting evidence beyond the corresponding masked objects.The captions explain that recognizing these classes would be difficult from the objects alone.