PaperlayerView exact revision on arXiv

Source-linked AI summary

RegNet: Self-Regulated Network for Image Classification

Jing Xu, Yu Pan, Xinglin Pan, Steven Hoi, Zhang Yi, Zenglin Xu

arXiv:2101.00590v1eess.IVcs.CV

TL;DR

ResNet shortcuts improve gradient flow but can limit the exploration and reuse of complementary features across blocks. RegNet adds a ConvRNN-based memory regulator to residual networks, and experiments report improved classification performance across three datasets, including parameter-efficient gains and extensions to SE-ResNet.

  • Problem

    ResNet’s additive shortcut mechanism can make consecutive blocks learn less new information and cause reusable information from earlier blocks to be forgotten.

  • Method

    RegNet adds a parallel ConvRNN regulator that encodes spatio-temporal memory and combines hidden states with current features before residual processing.

  • Results

    RegNet significantly improves classification accuracy over ResNet and SE-ResNet on CIFAR-10, CIFAR-100, and ImageNet, while requiring less depth for comparable accuracy.

  • Takeaways & Limitations

    The regulator is an easily implemented extension applicable to ResNet-based architectures, including SE-ResNet, and can improve parameter efficiency.

Abstract

from arXiv · show

The ResNet and its variants have achieved remarkable successes in various computer vision tasks. Despite its success in making gradient flow through building blocks, the simple shortcut connection mechanism limits the ability of re-exploring new potentially complementary features due to the additive function. To address this issue, in this paper, we propose to introduce a regulator module as a memory mechanism to extract complementary features, which are further fed to the ResNet. In particular, the regulator module is composed of convolutional RNNs (e.g., Convolutional LSTMs or Convolutional GRUs), which are shown to be good at extracting Spatio-temporal information. We named the new regulated networks as RegNet. The regulator module can be easily implemented and appended to any ResNet architecture. We also apply the regulator module for improving the Squeeze-and-Excitation ResNet to show the generalization ability of our method. Experimental results on three image classification datasets have demonstrated the promising performance of the proposed architecture compared with the standard ResNet, SE-ResNet, and other state-of-the-art architectures.

I. INTRODUCTION

ResNet shortcuts support optimization but can limit new feature exploration by emphasizing additive residual outputs. RegNet adds a ConvRNN-based regulator that carries memory between blocks to extract complementary features, and improves classification across benchmark datasets.

  • Motivation: ResNet shortcuts help gradients flow through building blocks, but their additive mechanism can cause consecutive outputs to learn less new information.The paper connects similar adjacent feature maps with reduced information gain across blocks.
  • Proposed approach: RegNet places a ConvRNN regulator in parallel with ResNet shortcuts to control memory information passed between building blocks.The regulator encodes spatio-temporal dependencies while constraining parameter growth.
  • Proposed approach: At each block, the ConvRNN combines current features with a hidden state, then convolves their concatenation to produce regulated outputs with complementary information.The paper describes the hidden state as reusable serial information that complements current input features.
  • Generalization: The regulator module is designed as a parallel structure that can be appended to ResNet-based architectures, including SE-ResNet.The paper instantiates the SE-ResNet extension as SE-RegNet.
  • Results: Experiments on CIFAR-10, CIFAR-100, and ImageNet report significantly improved classification accuracy over ResNet and SE-ResNet, while reducing the depth needed for comparable accuracy.The paper also evaluates predictions from consecutive block outputs in ResNet-20 and RegNet-20.

II. RELATED WORK

Related work frames RegNet against residual, densely connected, attention-based, and recurrent architectures. The paper positions ConvRNNs as a separate mechanism for capturing dependencies across ResNet building blocks without reusing all prior feature maps indiscriminately.

  • Residual architectures: ResNet variants achieve strong performance across vision tasks, but their depth issue motivates alternative information-flow mechanisms.The cited variants include Wide ResNet, Inception-ResNet, ResNetXt, and related architectures.
  • Dense connections: DenseNet concatenates features from previous layers to encourage reuse, but this can introduce redundancy and extra computational costs.The paper contrasts this with trade-off architectures such as Dual Path Network and Mixed link Network.
  • Attention modules: SENet, CBAM, and BAM improve ResNet-style models through modules that model attention over intermediate feature maps.The supplied passage specifically identifies SENet’s channel-wise attention mechanism.
  • Convolutional recurrence: ConvLSTM and ConvGRU have been used in vision applications to capture spatio-temporal information and dependencies in image sequences.Examples include rain removal, video super-resolution, compression, detection, and segmentation.
  • Positioning RegNet: RegNet uses ConvRNNs as a separate module to extract spatio-temporal information complementary to ResNet information flow.This separates the recurrent regulator from the residual shortcut pathway.

III. OUR MODEL

The model section revisits residual learning as a response to deep-network degradation and introduces RegNet’s regulated residual-block design. RegNet applies recurrent regulation to the residual architecture rather than replacing its basic building blocks.

  • A. ResNet: Degradation makes traditional networks difficult to converge as depth increases, while ResNet mitigates the problem by fitting residual mappings.The paper contrasts residual mapping with directly fitting the original underlying mapping.
  • A. ResNet: ResNet building blocks use residual mappings instead of directly fitting the original underlying mapping.The residual formulation is illustrated in Fig. 2 alongside the original mapping.

B. ConvRNN and its Variants

RegNet uses convolutional recurrent units to propagate feature information across residual blocks while limiting computational overhead. Its implementation covers basic and bottleneck residual modules and includes grouped-convolution efficiency modifications.

  • ConvRNN and its variants: ConvRNNs replace fully connected operators with convolutions, enabling recurrent processing of spatio-temporal feature information through ConvRNN, ConvLSTM, and ConvGRU variants.The paper adopts these recurrent units for regulating information flow in ResNet.
  • ConvRNN and its variants: In the recurrent formulation, X_t is the current 3D feature map, H_t−1 is the earlier hidden state, and H_t is the current output feature map.The input and output channel counts are both N.
  • Efficient implementation: The ConvRNN convolution receives 2N input channels and produces N output channels, reflecting fusion of the current input with the previous hidden state.The passage defines the channel dimensions of the convolutional operation.
  • Efficient implementation: Grouped 1 × 1 and 3 × 3 convolutions separately fuse channels and capture spatial information, reducing required computation by 18N/11 times with comparable results.The modification avoids directly applying original 3 × 3 convolutions to the full input.
  • RNN-Regulated ResNet: ConvRNN regulators are applied to both non-bottleneck and bottleneck ResNet building blocks for CIFAR and ImageNet settings.The paper describes corresponding regulated modules for the two ResNet block types.

1) RNN-Regulated ResNet Module (RegNet module):

The RegNet module uses ConvLSTM to combine current feature maps with recurrent memory, then reuses the resulting hidden state in subsequent convolutional processing.

  • The module concatenates the current input with the hidden state before the next convolution, enabling feature reuse within the building block.
  • H_t−1 denotes the earlier ConvLSTM output, while H_t denotes the output produced at the current module.
  • The module uses 3 × 3 convolutional kernels for W_t^12 and W_t^34, a 1 × 1 kernel for W_t^23, batch normalization, and concatenation.
  • ConvLSTM receives the current feature map and previous hidden state, automatically determining whether memory information propagates into the output hidden feature map.

2) Bottleneck RNN-Regulated ResNet Module (bottleneck RegNet module):

The bottleneck RegNet module extends the bottleneck ResNet building block with the paper’s recurrently regulated design, while the CIFAR architecture supports multiple RegNet depths.

  • Setting N to 3, 5, or 7 produces 20-, 32-, or 56-layer RegNet architectures for CIFAR-10/100.
  • Table III reports classification error rates on the CIFAR-10 and CIFAR-100 datasets, with best results marked in bold.
  • The bottleneck RegNet module is based on the bottleneck ResNet building block introduced for large images.
  • Its convolutional design uses two 1 × 1 kernels, a 3 × 3 bottleneck kernel, and a 1 × 1 kernel for feature fusion.

IV. EXPERIMENTS

The experiments evaluate the ConvRNN regulator on CIFAR-10, CIFAR-100, and ImageNet using PyTorch and different GPU configurations for small- and large-scale models.

  • The study evaluates the proposed ConvRNN regulator on CIFAR-10, CIFAR-100, and ImageNet benchmark datasets.
  • The algorithms are implemented in PyTorch, with CIFAR models trained on one NVIDIA 1080 Ti GPU and ImageNet models on four.

A. Experiments on CIFAR

The CIFAR experiments instantiate RegNet and SE-RegNet architectures with ConvRNN connections and train them under a specified optimization protocol on 32 × 32 image datasets.

  • The CIFAR-10 and CIFAR-100 datasets contain 32 × 32 RGB images, with 50,000 training images and 10,000 testing images per dataset.
  • Applying ConvRNNs to ResNet and SE-ResNet produces RegNet and SE-RegNet models, respectively.
  • The 20-layer RegNet and SE-RegNet demonstrate the method’s applicability, using three ConvRNN-connected groups of stacked RegNet modules for CIFAR.
  • Training uses SGD with momentum 0.9, weight decay 1e-4, batch size 64, 150 epochs, and a learning rate reduced tenfold at epoch 80.

1) Results on CIFAR:

On CIFAR-10 and CIFAR-100, RegNet variants improve classification accuracy over corresponding ResNets, while requiring fewer parameters or less depth for comparable accuracy.

  • Results on CIFAR:: RegNet and SE-RegNet outperform their original models at the same layer on CIFAR-10 and CIFAR-100.
  • Results on CIFAR:: 1.51% and 2.04% lower error rates are achieved by RegNet-20 with ConvLSTM versus ResNet-20 on CIFAR-10 and CIFAR-100, respectively.
  • Results on CIFAR:: 1.04% and 2.12% lower error rates are achieved by SE-RegNet-20 with ConvLSTM versus SE-ResNet-20 on CIFAR-10 and CIFAR-100, respectively.
  • Parameters Analysis:: A 20-layer RegNet with ConvLSTM reaches 7.28% CIFAR-10 error using 0.32M parameters, versus 7.54% for a 32-layer ResNet using 0.47M.
  • Positions of Feature Reuse:: RegNet(1)(GRU) reduces CIFAR-10 error from 8.38% to 7.52% with 0.006M additional parameters, while lower-layer reuse is more parameter-efficient than higher-layer reuse.

B. Experiments on ImageNet

On ImageNet, RegNet is evaluated against ResNet under computational-overhead constraints and improves both top-1 and top-5 accuracy with modest parameter overhead. RegNet-50 also surpasses deeper ResNet baselines while using fewer parameters and FLOPs.

  • Experiments on ImageNet: The ImageNet experiments use 1.28 million training images, 50,000 validation images, and 1,000 classes, reporting top-1 and top-5 validation errors.
  • Experiments on ImageNet: With 4.7M additional parameters, RegNet improves ResNet-50 by 1.38% in top-1 accuracy and 0.85% in top-5 accuracy.
  • Experiments on ImageNet: RegNet-50 with 31.3M parameters and 5.12G FLOPs surpasses ResNet-50 and ResNet-101, which uses 44.6M parameters and 7.9G FLOPs.

V. CONCLUSIONS

The paper concludes that ConvRNN-based regulators extract complementary features and improve ResNet representation power across three image-classification datasets. Future work targets efficiency, additional ResNet variants, and other vision tasks.

  • V. CONCLUSIONS: The proposed regulator uses Convolutional RNNs to extract complementary features and improve the representation power of ResNets.
  • V. CONCLUSIONS: Future work will improve efficiency, extend the regulator to other ResNet-based architectures, and explore object detection and image super-resolution.
Loading 2101.00590v1…