Source-linked AI summary
Gated Channel Transformation for Visual Recognition
Zongxin Yang, Linchao Zhu, Yu Wu, Yi Yang
TL;DR
Deep CNNs need efficient and interpretable channel-relationship modeling beyond SE's parameter-heavy, implicitly learned block-level design. GCT uses normalization and lightweight gating to model channel interactions, and experiments report improved performance across image, detection, segmentation, and video recognition tasks with negligible complexity growth.
Problem
SE's fully connected layers implicitly learn channel relationships, limiting module deployment and making interactions difficult to analyze.
Method
GCT uses parameter-free channel normalization, especially ℓ2 normalization, plus lightweight trainable embedding and gating parameters to control channel behavior.
Results
Experiments show that GCT improves deep CNN generalization across visual recognition tasks and datasets, including ImageNet, COCO, and Kinetics, with negligible complexity growth.
Takeaways & Limitations
GCT provides an efficient and interpretable channel-wise transformation that supports both competition and cooperation among channels.
Abstract
from arXiv · showhide
In this work, we propose a generally applicable transformation unit for visual recognition with deep convolutional neural networks. This transformation explicitly models channel relationships with explainable control variables. These variables determine the neuron behaviors of competition or cooperation, and they are jointly optimized with the convolutional weight towards more accurate recognition. In Squeeze-and-Excitation (SE) Networks, the channel relationships are implicitly learned by fully connected layers, and the SE block is integrated at the block-level. We instead introduce a channel normalization layer to reduce the number of parameters and computational complexity. This lightweight layer incorporates a simple l2 normalization, enabling our transformation unit applicable to operator-level without much increase of additional parameters. Extensive experiments demonstrate the effectiveness of our unit with clear margins on many vision tasks, i.e., image classification on ImageNet, object detection and instance segmentation on COCO, video classification on Kinetics.
1 Baidu Research 2 ReLER, University of Technology Sydney
This section identifies the authors’ institutional affiliations: Baidu Research and ReLER at the University of Technology Sydney.
- The listed authors use student.uts.edu.au and uts.edu.au email addresses, indicating affiliation with the University of Technology Sydney.The addresses are associated with named authors in the passage.
- The affiliations include Baidu Research and ReLER, University of Technology Sydney.
1. Introduction
The introduction motivates explicit, efficient modeling of channel relationships in deep CNNs. GCT combines normalization and gating to provide lightweight, interpretable control over competition and cooperation among channels, with improvements across visual recognition tasks.
- SE models channel relationships with fully connected layers, limiting deployment across CNN layers and making channel interactions difficult to analyze.
- GCT replaces fully connected channel modeling with computationally efficient ℓ2 normalization and lightweight channel-wise trainable parameters.
- GCT combines normalization and gating to create either competition or cooperation among channels by changing activation variance.
- The gating design uses a residual 1 + tanh(x) activation to support identity mapping and stabilize training relative to Sigmoid gating.
- Experiments report that GCT improves the generalization capability of deep convolutional networks across visual recognition tasks and datasets.
2. Related Work
Prior work uses gating, attention, and non-local operations to enhance feature representations and capture contextual dependencies. GCT builds on these ideas with channel-wise contextual modeling and reports promising performance with negligible computational growth.
- Gating and attention mechanisms regulate information flow or focus computation on informative feature components across sequence and vision tasks.
- SE-Nets and GE-Nets add lightweight gating to convolutional networks by modeling channel-wise relationships using global or neighboring context.
- C3D and non-local neural networks capture spatiotemporal or long-range dependencies for video classification.
- GCT is built on deep architectures and achieves promising performance improvements while increasing computational complexity negligibly.
- GCT's overview assigns channel embedding before normalization and uses gating parameters to adjust feature channels.
3. Gated Channel Transformation
GCT is a lightweight, explainable channel-transformation unit that combines global context embedding, channel normalization, and gated adaptation. Its O(C) parameter complexity supports deployment across convolutional layers while modeling competition and cooperation among channels.
- Architecture: GCT models channel relationships with trainable embedding, gating, and bias parameters whose total complexity is O(C), versus O(C2) for SE's fully connected layers.The parameters control channel-specific behavior while keeping the transformation lightweight.
- Global Context Embedding: Global context embedding aggregates information from each channel using a trainable weight α and an ℓp-norm instead of global average pooling.The default setting uses the ℓ2-norm; ℓ1 and ℓ2 perform similarly, while ℓ1 can be replaced by GAP for consistently non-negative inputs.
- Channel Normalization: Channel normalization applies ℓ2 normalization across channels to create lightweight competition relationships.Its computational complexity is O(C), lower than the O(C2) complexity of SE's fully connected layers.
- Gating Adaptation: Gated adaptation scales each channel by 1 + tanh(γcŝc + βc), enabling both competition and cooperation among channels.The residual-style gate also permits identity mapping when γ and β are zero.
- Implementation: GCT is implemented and evaluated across multiple deep-learning frameworks, including PaddlePaddle, TensorFlow, and PyTorch.The paper reports similar improvement after introducing GCT across these frameworks.
4. Experiments
Experiments across ImageNet, COCO, and Kinetics show that GCT improves recognition, detection, segmentation, and video classification across diverse architectures. Ablations and analyses indicate that its normalization, gating, and placement choices contribute to efficient channel cooperation and competition.
- ImageNet: GCT improves non-residual networks such as VGG-16 and Inception-v3 and consistently outperforms SE in the reported ImageNet comparisons.The comparison covers both residual and non-residual networks.
- Efficiency: GCT retains efficiency relative to SE, with ResNet-50 inference at 484 FPS versus 525 FPS for SE while enabling convolutional-layer deployment.The reported comparison evaluates inference speed on a GTX 1080 Ti.
- COCO: GCT improves ResNet-50 Mask R-CNN by 2.0 box AP and 1.8 mask AP, exceeding SE improvements of 0.4 box AP and 0.5 mask AP.With ResNet-101, GCT also improves detection AP by 1.9 and segmentation AP by 1.6.
- COCO: BN∗+GCT improves over GN by 0.8 detection AP and 1.1 segmentation AP in ResNet-101, while GCT also works with GN box heads.The short-schedule BN∗+GCT model reaches 37.7 segmentation AP versus 37.2 for the long-schedule GN counterpart.
- Kinetics: GCT consistently improves Kinetics-400 recognition over ResNet-50 and ResNet-101 baselines, although limited memory prevents applying it to every convolutional layer.The experiments apply GCT in the last two convolutional layers of each Res-Block.
- Analysis: Layer analysis suggests GCT reduces channel differences farther from the output for cooperation, but increases them near the output for competition.Correspondingly, GCT reduces feature variance in earlier layers and magnifies it near the network output.
- Ablation Studies: Among normalization choices, ℓ2 normalization performs slightly better than ℓ1, while mean-and-variance normalization reaches 23.7% top-1 error versus the 23.8% ResNet-50 baseline.The ablations also find residual gating activations and placement before convolutional layers effective.
5. Conclusion and Future Work
The paper concludes that GCT improves CNN discriminability by modeling channel relationships with normalization and gating. Extensive experiments support its effectiveness and robustness across architectures and datasets, while future work will examine recurrent networks.
- Conclusion: GCT combines normalization and gating to model both competition and cooperation among channels with negligible parameter complexity.The layer is presented as a mechanism for improving deep CNN discriminability.
- Conclusion and Future Work: Experiments demonstrate GCT effectiveness and robustness across a wide range of modern CNNs and datasets.The paper identifies recurrent networks such as LSTMs as a direction for future study.