Source-linked AI summary
Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet
Wieland Brendel, Matthias Bethge
TL;DR
DNN decisions are difficult to interpret, so the paper introduces BagNets, a DNN-based bag-of-local-features architecture using small patches and linear spatial aggregation. BagNets achieve high ImageNet accuracy and exhibit decision-making similarities to common DNNs, while their local evidence remains directly traceable.
Problem
DNN decisions are difficult to understand because input and hidden activations depend on many other parts or units.
Method
BagNets classify images from local image-patch features using a linear classifier and spatial averaging, without using spatial relationships.
Results
BagNets reach high ImageNet accuracy with fairly small patches and show similarities in decision-making behaviour to popular DNNs.
Takeaways & Limitations
The results suggest that many ImageNet-trained DNNs partly follow a bag-of-feature strategy based on local statistical regularities and weak feature interactions.
Takeaways & Limitations
Models relying on local features can fail on cartoon-like images that lack the key local features associated with different objects.
Abstract
from arXiv · showhide
Deep Neural Networks (DNNs) excel on many complex perceptual tasks but it has proven notoriously difficult to understand how they reach their decisions. We here introduce a high-performance DNN architecture on ImageNet whose decisions are considerably easier to explain. Our model, a simple variant of the ResNet-50 architecture called BagNet, classifies an image based on the occurrences of small local image features without taking into account their spatial ordering. This strategy is closely related to the bag-of-feature (BoF) models popular before the onset of deep learning and reaches a surprisingly high accuracy on ImageNet (87.6% top-5 for 33 x 33 px features and Alexnet performance for 17 x 17 px features). The constraint on local features makes it straight-forward to analyse how exactly each part of the image influences the classification. Furthermore, the BagNets behave similar to state-of-the art deep neural networks such as VGG-16, ResNet-152 or DenseNet-169 in terms of feature sensitivity, error distribution and interactions between image parts. This suggests that the improvements of DNNs over previous bag-of-feature classifiers in the last few years is mostly achieved by better fine-tuning rather than by qualitatively different decision strategies.
1 INTRODUCTION
The paper addresses the difficulty of explaining DNN decisions by introducing BagNets, which combine DNN performance and flexibility with BoF interpretability using small local image features.
- DNN decisions are difficult to understand because input and hidden activations have complex dependencies on many other parts or units.
- BoF models are easier to explain because they classify images from local-feature counts rather than spatial relationships.
- BagNets combine DNN performance and flexibility with BoF interpretability while using fairly small image patches on ImageNet.
- The authors propose trading some accuracy for interpretability in applications such as diagnosing failures, domain transfer, attribution benchmarking, and larger vision pipelines.
A B C
Figure 1 presents BagNets’ local-evidence pipeline, ImageNet performance across patch sizes, and similarities between BagNet and VGG-16 decisions.
- BagNets classify images by averaging spatially distributed local-patch class evidence before applying softmax.
- The figure compares Top-5 ImageNet performance across patch sizes and correlates BagNet logits with VGG-16 logits.
- BagNets show similarities to popular DNNs in decision-making behaviour.
2 NETWORK ARCHITECTURE
BagNets use local ResNet-derived features and linear aggregation so each patch’s class evidence can be traced directly to the image-level decision.
- Bag-of-features representations assemble counts of vocabulary elements into a long term vector, analogous to counting words in documents.
- A linear classifier makes a BoF model interpretable because each input part’s influence is independent of the rest.
- BagNet-q extracts 2048-dimensional features from q × q patches, classifies each patch’s evidence, and averages evidence across patches.
- The architecture limits the topmost receptive field to q × q pixels by replacing many 3 × 3 convolutions with 1 × 1 convolutions.
- Linear classification and spatial averaging are interchangeable, enabling exact tracing of local-patch evidence into image-level decisions.
3 RELATED LITERATURE
The related work combines DNN feature extraction with BoF ideas, while BagNets retain local, linear interpretability more directly than several prior approaches.
- Earlier hybrid approaches commonly used DNNs to replace hand-tuned BoF feature extraction for retrieval and geographical scene classification.
- Prior interpretable-DNN methods included pixel labeling and multi-step object, part, and domain detectors, but their local assignments could remain difficult to interpret.
- BagNets restrict CNN representations to small image patches, allowing each patch’s contribution to the final decision to be traced exactly.
- Scattering networks used small receptive fields but did not increase interpretability because a nonlinear classifier followed the local features.
- Region proposal models use the whole image to infer regions, so their extracted features still integrate information beyond the proposed patches.
4 RESULTS
BagNets achieve strong ImageNet accuracy with small local patches and produce interpretable evidence maps, while sharing several decision-making characteristics with conventional DNNs. Their analyses reveal local statistical cues, limited spatial interactions, and similar error distributions across models.
- Accuracy and runtime: 80.5% top-5 performance with 17 × 17 patches matches AlexNet, while 33 × 33 patches reach 87.6%.BagNets are trained directly on ImageNet.
- Accuracy and runtime: 155 images/s inference for BagNets is lower than ResNet-50’s 570 images/s across tested receptive-field sizes.The runtime difference is attributed to BagNets’ reduced downsampling.
- Explaining decisions: Patch-level class evidence yields high-resolution heatmaps showing which image regions contribute most to particular decisions.Class evidence is inferred for each q × q patch and aggregated into image-level evidence.
- Explaining decisions: Top-evidence patches expose predictive local cues, including book-cover text, typewriter keys, fingers holding fish, flamingo beaks, and suit-to-neck transitions.These cues can also explain confusions with visually similar classes or features.
- Explaining decisions: BagNets and VGG-16 can share understandable local-feature-driven errors, such as cucumber edges causing a cleaver-to-granny-smith confusion.Other examples include isolated thimble patches resembling a gas mask and salient letters driving a miniskirt-to-book-jacket prediction.
- Comparing decision-making: 90.1% clean-image accuracy versus 79.4% on texturised images shows VGG-16 remains relatively effective when global shape information is disrupted.The texturisation comparison is used to assess reliance on spatial relationships and global shape integration.
- Comparing decision-making: Error distributions are fairly consistent between BagNet-33 and regular DNNs, while logit correlations with VGG-16 rise from 0.70 to 0.88 across BagNet-9/17/33.The correlation is described as an initial indication rather than a complete measure of shared decision processes.
5 DISCUSSION & OUTLOOK
BagNets show that ImageNet classification can rely on small local features without spatial relationships, while many standard DNNs exhibit related decision properties. This supports using BagNets to study interpretability and to probe whether models learn causal or holistic representations.
- BagNets classify ImageNet images using linear bag-of-local-features representations.
- ImageNet can be solved from small image features without modeling spatial relationships.
- Many common vision models share BagNet-like invariance to spatial relationships and weak interactions between image features.
- VGG-16 is especially close to bag-of-feature models, whereas deeper networks show stronger nonlinear interactions and lower sensitivity to local maskings.
- ImageNet alone does not force DNNs to learn physical and causal representations because local image features suffice for the task.
- BagNets can trade some accuracy for interpretability in diagnostics, attribution benchmarking, and larger computer-vision pipelines.
- DNNs may discover weak statistical regularities more effectively than hand-tuned BoF algorithms without learning substantially different representations.
A APPENDIX
The appendix documents BagNet’s ResNet-derived architecture, training setup, and supplementary attribution visualizations. It also includes additional class and thresholding-related figures.
- The models were trained in PyTorch with Torchvision’s default ImageNet script and specified SGD, batch-size, learning-rate, and crop settings.
- BagNet is nearly equivalent to ResNet-50, replacing most 3 × 3 convolutions with 1 × 1 convolutions and modifying strides.Each ResNet block expands by four, and downsampling uses a stride-2 1 × 1 convolution.
- Figure A.2 compares VGG feature attributions from four attribution methods with BagNet feature attributions.
- Figure A.3 extends Figure 3’s analysis to additional classes.
A.1 EFFECT OF LOGIT THRESHOLDING
The appendix tests whether BagNet-33 depends on exact patch-logit amplitudes by thresholding or binarizing its heatmaps, with performance measured by top-5 accuracy.
- BagNet-33 sensitivity to patch-logit values was tested using thresholding and heatmap binarization.Thresholding clips values below a threshold, while binarization replaces values below it with zero and those above it with one.
- Figure A.4 reports the effect of logit thresholding on model performance using top-5 accuracy.