Source-linked AI summary
Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks
Jie Hu, Li Shen, Samuel Albanie, Gang Sun, Andrea Vedaldi
TL;DR
CNNs may fail to exploit long-range feature context despite theoretically large receptive fields. The paper introduces lightweight gather-excite operators that aggregate broad spatial context and modulate local features, improving performance across architectures, datasets, and tasks while producing lower-class-selectivity intermediate representations.
Problem
CNNs may underuse long-range feature context because their effective receptive fields are smaller than their theoretical receptive fields.
Method
Gather-excite combines a gather operator that aggregates contextual information across large neighborhoods with an excite operator that modulates feature maps using those aggregates.
Results
Gather-excite yields significant improvements across architectures, datasets, and tasks, with ResNet-50 GE variants outperforming the original baseline as spatial extent increases.
Takeaways & Limitations
Gather-excite provides an efficient way to incorporate feature context throughout deep CNN architectures, while producing intermediate representations with lower class selectivity.
Takeaways & Limitations
GE operators add layers, so comparisons with deeper networks do not precisely control for network depth; broader use may require accuracy–storage trade-offs.
Abstract
from arXiv · showhide
While the use of bottom-up local operators in convolutional neural networks (CNNs) matches well some of the statistics of natural images, it may also prevent such models from capturing contextual long-range feature interactions. In this work, we propose a simple, lightweight approach for better context exploitation in CNNs. We do so by introducing a pair of operators: gather, which efficiently aggregates feature responses from a large spatial extent, and excite, which redistributes the pooled information to local features. The operators are cheap, both in terms of number of added parameters and computational complexity, and can be integrated directly in existing architectures to improve their performance. Experiments on several datasets show that gather-excite can bring benefits comparable to increasing the depth of a CNN at a fraction of the cost. For example, we find ResNet-50 with gather-excite operators is able to outperform its 101-layer counterpart on ImageNet with no additional learnable parameters. We also propose a parametric gather-excite operator pair which yields further performance gains, relate it to the recently-introduced Squeeze-and-Excitation Networks, and analyse the effects of these changes to the CNN feature activation statistics.
1 Introduction
CNNs excel at visual tasks but may underuse long-range feature context because effective receptive fields are smaller than their theoretical extent. Gather-excite addresses this by combining large-neighborhood aggregation with feature modulation, improving representations across architectures, datasets, and tasks.
- CNNs are widely used for image classification, object detection, and image segmentation.
- Effective receptive fields are often smaller than theoretical receptive fields, motivating better use of contextual information.
- Gather-excite decomposes context modules into gathering information across large neighborhoods and exciting feature maps conditioned on those aggregates.
- The proposed lightweight gather-excite operators improve performance across architectures, datasets, and tasks with minimal hyperparameter tuning.
- The mechanism produces intermediate representations with lower class selectivity, suggesting greater feature reuse when additional context is available.
2 The Gather-Excite Framework
The Gather-Excite framework efficiently gathers feature responses over large spatial neighborhoods and uses them to modulate local responses. It formalizes the operators and studies how spatial extent and parameterization affect contextual feature processing.
- The framework is motivated by CNNs’ gradual transition from spatial to channel coding as depth, abstraction, receptive fields, and channel count increase.
- Gather-excite pools feature responses over large neighborhoods and uses the resulting context to modulate the original responses.
- The gather operator aggregates responses within each channel over a spatial extent, with global extent when the receptive field covers the full feature map.
- The excite operator rescales and distributes gathered signals to produce a tensor matching the original input dimensions.
3 Models and Experiments
The experiments evaluate parameter-free and parameterised gather-excite designs across architectures, stages, datasets, and tasks. Gather-excite improves accuracy with modest complexity, while parameterisation and broader placement generally strengthen performance.
- Parameter-free pairings: 22.14% top-1 error: global parameter-free gather-excite improves ResNet-50 over the 23.30% baseline and is competitive with ResNet-101 at 22.20%.Increasing spatial extent consistently improves performance, with global average pooling performing best.
- Parameterised pairings: Parameterised gather-excite uses strided depth-wise convolution to learn spatial aggregation while retaining the excite operator.The parameterised design is denoted GE-θ and contrasts with the parameter-free GE-θ− model.
- Parameterised pairings: Additional parameters improve performance over the parameter-free design as spatial extent increases in the parameterised experiments.The parameterised study shows a similar trend to the parameter-free extent experiment.
- Parameterised pairings: Inserting GE operators helps at every stage, with the greatest improvement in the middle and late stages; stage effects can also be combined effectively.GE can be removed from Stage 2 when parameter storage matters, with only a marginal performance cost.
- Parameterised pairings: 21.88% vs 21.87% top-1 error: GE-θ+ approaches ResNet-152 performance at approximately one third of the computational complexity.GE-θ+ combines parameterised gathering with a 1 × 1 convolutional channel-subnetwork excite operator and outperforms the SE and GE-θ models.
- Generalisation: 28.6% mAP versus 27.3% for the baseline: GE-θ improves Faster R-CNN with a ResNet-50 backbone on MS COCO.GE also yields gains on CIFAR-10 and CIFAR-100 and improves ShuffleNet variants at modest theoretical computational complexity, though additional parameters were needed there.
4 Analysis and Discussion
The analysis examines how GE changes learned representations, optimization, and feature reliance relative to ResNet-50. GE exhibits lower class selectivity with depth and lower training and validation error under fixed training.
- Learned representations: GE-θ exhibits less class selectivity than ResNet-50 as depth increases through the fourth stage.The distributions are initially similar but separate at greater depth, becoming distinct by conv4-6-relu.
- Learned representations: The class selectivity analysis uses intermediate representations from the fourth network stage and compares histograms across blocks.The metric is computed for the last layer in each block of that stage.
- Optimization: GE-θ achieves lower training and validation error than the ResNet-50 baseline throughout fixed-schedule ImageNet training.The comparison uses a global extent-ratio GE-θ model trained for 100 epochs.
- Feature importance: GE-θ’s excite mechanism supports feature selection by assigning importance scores that preserve some feature maps while squashing others toward zero.The pruning experiment tests whether these assigned importance scores predict feature contribution.
5 Related Work
The related work positions Gather-Excite within longstanding efforts to incorporate contextual information into visual representations. It connects GE to multiscale convolution, attention, and Squeeze-and-Excitation designs.
- Contextual vision: Context has been used in computer vision because perceptual studies link contextual information to human object-recognition and detection accuracy and efficiency.Earlier automated systems also incorporated context through sophisticated rule-based approaches.
- Contextual vision: Inception architectures use multiscale convolutional modules to aggregate context throughout hierarchies of learned representations.Variants have also appeared in recent automated architecture-search work.
- Attention and lightweight modules: GE builds on Squeeze-and-Excitation networks, which use global embeddings to provide context to a recalibration function.The paper also draws inspiration from work showing that contextual information can be inferred for object localization.
6 Conclusion and Future Work
The paper proposes Gather-Excite as an efficient way to exploit feature context in CNNs and reports effectiveness across datasets and architectures. It identifies semantic segmentation as a future application to investigate.
- Conclusion: Gather-Excite addresses efficient exploitation of feature context in CNNs through the proposed GE framework.The paper evaluates the approach across multiple datasets and model architectures.
- Future work: Future work will investigate whether Gather-Excite operators are useful for semantic segmentation.The authors anticipate that this task may also benefit from efficient use of feature context.
A Appendix
The appendix extends the analysis with fixed-schedule ShuffleNet experiments, pooling comparisons, representation statistics, feature-pruning results, and module diagrams. Average pooling consistently outperforms max pooling in parameter-free GE designs.
- Optimization curves: A fixed 100-epoch schedule enables direct comparison between ShuffleNet and its GE-θ variant.The main results used a longer schedule of approximately 400 epochs to reproduce baseline ShuffleNet performance.
- Pooling method: Average pooling consistently outperforms max pooling for parameter-free GE-θ− designs.Max pooling can hurt performance when used over the full global extent.
- Class selectivity: GE-θ− and SE show weaker but similar class-selectivity trends, with gaps from the ResNet-50 distributions emerging at greater depth.The comparison uses feature histograms from blocks in stage four.
- Feature importance: SE feature-pruning curves broadly match the trends observed for GE-θ.The experiment uses an SE network built on a ResNet-50 backbone.
- Operator diagrams: Figure 9 diagrams several GE variants and shows their integration into residual units.The variants include GE-θ−(E8), GE-θ−, GE-θ(E8), and GE-θ.