PaperlayerView exact revision on arXiv

Source-linked AI summary

DualConv: Dual Convolutional Kernels for Lightweight Deep Neural Networks

Jiachen Zhong, Junying Chen, Ajmal Mian

arXiv:2202.07481v1cs.CVcs.LG

TL;DR

Deep CNNs can exceed the memory and computational limits of embedded and mobile devices. The paper proposes DualConv, which combines grouped 3×3 and 1×1 convolutions within existing CNNs, and reports lower cost with preserved or improved accuracy across classification and detection experiments.

  • Problem

    Deep CNN architectures require substantial memory and computation, limiting their deployment on embedded and mobile devices.

  • Method

    DualConv combines 3×3 group convolution with 1×1 pointwise convolution on the same input feature map and replaces standard convolutions in existing CNN architectures.

  • Results

    DualConv generally retains or improves accuracy while reducing computation and parameters, including over 20% reductions for ResNet-50 on CIFAR-100 and a 4.4% mAP improvement for YOLO-V3 on PASCAL VOC.

  • Takeaways & Limitations

    DualConv can make existing standard and lightweight CNN architectures more suitable for mobile and embedded deployment by reducing parameters, computational cost, and inference time.

Abstract

from arXiv · show

CNN architectures are generally heavy on memory and computational requirements which makes them infeasible for embedded systems with limited hardware resources. We propose dual convolutional kernels (DualConv) for constructing lightweight deep neural networks. DualConv combines 3$\times$3 and 1$\times$1 convolutional kernels to process the same input feature map channels simultaneously and exploits the group convolution technique to efficiently arrange convolutional filters. DualConv can be employed in any CNN model such as VGG-16 and ResNet-50 for image classification, YOLO and R-CNN for object detection, or FCN for semantic segmentation. In this paper, we extensively test DualConv for classification since these network architectures form the backbones for many other tasks. We also test DualConv for image detection on YOLO-V3. Experimental results show that, combined with our structural innovations, DualConv significantly reduces the computational cost and number of parameters of deep neural networks while surprisingly achieving slightly higher accuracy than the original models in some cases. We use DualConv to further reduce the number of parameters of the lightweight MobileNetV2 by 54% with only 0.68% drop in accuracy on CIFAR-100 dataset. When the number of parameters is not an issue, DualConv increases the accuracy of MobileNetV1 by 4.11% on the same dataset. Furthermore, DualConv significantly improves the YOLO-V3 object detection speed and improves its accuracy by 4.4% on PASCAL VOC dataset.

I. INTRODUCTION

Deep CNNs increasingly exceed the memory and computational limits of embedded and mobile systems. The paper proposes DualConv as an efficient replacement for standard convolution in existing architectures, reducing cost while retaining accuracy.

  • Embedded and mobile deployment requires CNNs that balance accuracy, computational complexity, and parameter count under limited hardware resources.
  • Model compression reduces parameters and FLOPs, but compression is costly and difficult to apply in practice.
  • Efficient convolutional filters can replace standard filters in existing CNN architectures, avoiding the need to design entirely new networks.
  • DualConv combines 3×3 group convolution with 1×1 pointwise convolution on the same input channels to preserve information while improving efficiency.
  • DualConv replaces standard convolution in VGG-16 and ResNet-50 experiments on CIFAR-10, CIFAR-100, and ImageNet, with reduced computation and parameters and slightly higher accuracy in some cases.

II. RELATED WORK

Prior efficient convolution designs reduce computation by restructuring spatial filtering, channel fusion, or connectivity. Group convolution lowers cost through channel partitioning, but its groups do not share channel information.

  • Efficient convolutional filters aim to reduce computational cost and parameters while replacing standard filters within existing architectures.
  • Depthwise separable convolution factors standard convolution into depthwise filtering followed by 1×1 pointwise channel fusion.
  • Figure 1 compares standard, depthwise separable, group, heterogeneous, and dual convolution using M, N, Di, K × K, G, and 1/P as design parameters.
  • Group convolution divides filters and input channels into G corresponding groups, reducing computation by restricting each group to its associated channels.
  • Group convolution reduces computation but prevents channel information from being shared between different groups.

C. Heterogeneous Convolution

DualConv combines grouped 3×3 filtering with 1×1 pointwise convolution across the same input feature map. Its group structure controls the spatial-kernel proportion and therefore the computation–accuracy trade-off.

  • DualConv combines group convolution and heterogeneous convolution, applying 3×3 and 1×1 operations to selected channels while applying 1×1 convolution across all input channels.
  • The 1×1 path preserves original input information and helps deeper layers extract information more effectively.
  • DualConv addresses GroupConv’s limited channel communication by having each filter group handle the complete input feature map.
  • The group count G controls the proportion of K × K kernels: combined K × K + 1 × 1 kernels occupy 1/G of channels, while the remaining proportion uses 1×1 kernels alone.
  • With K=3, DualConv’s speedup can reach 8 to 9 times when G is large.

B. Comparison with Previous Work

DualConv is compared with depthwise separable, group, and heterogeneous convolutions through computational-cost analysis. Its design retains full input-feature processing while achieving speedups comparable to other efficient convolutions.

  • Comparison with Previous Work: DualConv avoids the extra layers used by depthwise separable convolution while retaining a dual-kernel structure.Depthwise separable convolution uses sequential depthwise and pointwise layers; DualConv does not add additional layers.
  • Comparison with Previous Work: DualConv processes the complete input feature map, unlike GroupConv, whose filters access only 1/G of the input channels.This distinction addresses the limited channel information available to individual GroupConv filters.
  • Comparison with Previous Work: DualConv applies 1×1 convolution across all input channels, unlike HetConv, retaining and fusing original features with only a slight increase in FLOPs and parameters.HetConv does not apply its 1×1 convolution to every input feature-map channel.
  • Comparison with Previous Work: When K=3 and N and P are large, depthwise separable convolution and HetConv reach 8 to 9 times speedup, while GroupConv speedup grows with G.The comparison is derived from the computational formulas for the different convolution types.

IV. EXPERIMENTS AND DISCUSSIONS

DualConv provides an adjustable accuracy–computation trade-off through the number of filter groups G. It generally preserves the original network accuracy and sometimes achieves slightly higher accuracy.

  • IV. EXPERIMENTS AND DISCUSSIONS: Increasing G adjusts the trade-off between network accuracy and computational cost in DualConv.Larger G makes DualConv closer to standard convolution composed of 1×1 kernels.
  • IV. EXPERIMENTS AND DISCUSSIONS: DualConv generally retains the original network accuracy and can achieve slightly higher accuracy in some cases.The reported experiments evaluate this behavior across the proposed dual-convolutional filters.

A. VGG-16 and ResNet-50 on CIFAR-10

DualConv replaces selected standard convolutions in VGG-16 and ResNet-50 while varying the group parameter G. The experiments report substantial reductions in computation and parameters, with accuracy preserved or improved in the highlighted settings.

  • A. VGG-16 and ResNet-50 on CIFAR-10: DualConv replaces the last 12 3×3 convolutions in VGG-16 while keeping each replaced layer’s kernel count unchanged.All replaced VGG-16 layers use the same G value.
  • A. VGG-16 and ResNet-50 on CIFAR-10: DualConv replaces all stride-1 3×3 convolutions in ResNet-50 except the first convolutional layer.This defines the ResNet-50 replacement scope used in the experiments.
  • A. VGG-16 and ResNet-50 on CIFAR-10: When G=4, VGG-16 reduces computations and parameters by over 60% while achieving higher accuracy than standard VGG-16.Accuracy drops slightly as G increases generally, but this setting exceeds the standard model’s accuracy.
  • A. VGG-16 and ResNet-50 on CIFAR-10: When G=8, ResNet-50 with DualConv outperforms standard ResNet-50 while reducing computations and parameters by over 25%.DualConv generally obtains higher accuracy than HetConv in the reported comparisons.
  • A. VGG-16 and ResNet-50 on CIFAR-10: 1×1 convolution across all channels helps DualConv retain and fuse input-feature information without channel shuffle.The paper attributes this information transfer and fusion to pointwise convolution applied to the input feature maps.

B. MobileNetV1 and MobileNetV2 on CIFAR-10

DualConv is evaluated as a replacement for depthwise separable convolutions in MobileNetV1 and inverted residual blocks in MobileNetV2. Results show an accuracy gain for MobileNetV1 and substantial efficiency reductions for MobileNetV2.

  • B. MobileNetV1 and MobileNetV2 on CIFAR-10: DualConv replaces all depthwise separable convolutions in MobileNetV1 and the inverted residual block in MobileNetV2.For MobileNetV2, the replacement keeps convolution stride 1 and adds batch normalization and ReLU6 afterward.
  • B. MobileNetV1 and MobileNetV2 on CIFAR-10: MobileNetV1 accuracy increases by 1.23% with DualConv, despite increases in parameters and computational cost.At G=32, MobileNetV1 still has higher accuracy with similar parameters and computational cost.
  • B. MobileNetV1 and MobileNetV2 on CIFAR-10: When G=32, MobileNetV2 reduces parameters and computational cost by over 60% with only a 1.16% accuracy drop.The result is reported for the DualConv-modified MobileNetV2 network.

C. VGG-16 and ResNet-50 on CIFAR-100

On CIFAR-100, DualConv replaces 3×3 standard convolutions in VGG-16 and ResNet-50, reducing computational cost and parameters while preserving accuracy closely.

  • Experimental setup: The experiments evaluated DualConv-modified VGG-16 and ResNet-50 on CIFAR-100 using replacements for their 3×3 standard convolutions.
  • VGG-16: More than 70% computational-cost reduction was achieved for VGG-16 when G=8, while accuracy improved before declining slightly at larger G values.Increasing G further reduced parameters and computational cost.
  • ResNet-50: ResNet-50 achieved its best DualConv accuracy at G=4, only 0.02% below the original, while reducing parameters and computational cost by more than 20%.

D. MobileNetV1 and MobileNetV2 on CIFAR-100

On CIFAR-100, DualConv improved MobileNetV1 accuracy substantially and reduced MobileNetV2 efficiency costs with only a small accuracy decrease.

  • Experimental setup: The CIFAR-100 experiment tested DualConv in MobileNetV1 and MobileNetV2 using the same experimental settings as the preceding section.
  • MobileNetV1: 4.11% accuracy improvement was obtained for MobileNetV1 at G=4 after replacing its depthwise separable convolutions with DualConv.Even at G=32, accuracy remained above standard MobileNetV1 with similar computational cost.
  • MobileNetV2: MobileNetV2 parameters and computational cost were greatly reduced with DualConv, accompanied by only a 0.68% accuracy drop at G=2 or G=4.

G. Classification Time on ImageNet

The ImageNet timing analysis treats inference speed as a direct efficiency measure and compares CPU runtime across DualConv and alternative convolutional designs.

  • Measurement protocol: Inference time was measured per image on a single Nvidia Tesla V100 GPU and a single-thread Intel Core i7-8700 CPU.Because GPU synchronization overhead obscured clear differences, the discussion focuses on CPU inference time.
  • CPU inference time: 52% and 24% CPU inference-time reductions were measured for DualConv-modified VGG-16 and ResNet-50, respectively.DualConv also reduced parameters and computational cost for both networks.
  • Group-count trade-off: Larger group counts can increase memory-access cost, and DualConv-modified VGG-16 and ResNet-50 were faster at G=16 and G=8 but not G=32.
  • Comparison with alternative filters: At matched G or P values, GroupConv models ran faster than DualConv models, whereas HetConv models ran slower; one example was 92.83ms versus 79.47ms for ResNet-50.

I. YOLO-V3 on PASCAL VOC

DualConv is applied to nearly all YOLO-V3 convolutional layers and evaluated on PASCAL VOC under matched training settings. The modified model reduces computational cost and parameters while improving detection accuracy and inference speed over standard YOLO-V3.

  • Model modification: DualConv modifies all YOLO-V3 convolutional layers except the first, preserving the first layer for low-level information extraction.The remaining layers replace 3×3 standard convolutions with the DualConv design.
  • Experimental setup: YOLO-V3 and DualConv-modified YOLO-V3 are trained from scratch for 100 epochs using 416×416 inputs and the PASCAL VOC 2007+2012 training and validation images.Mean average precision is computed on the PASCAL VOC 2007 test set.
  • Results: Inference time decreases from 26.66 ms to 19.79 ms, a 6.87 ms improvement.The reported result supports faster object detection inference alongside model compression.
  • Comparison scope: The comparison uses identical training settings, whereas ImageNet-pretrained YOLO-V3 results are not directly comparable.Both models are trained from scratch on the same training set with the same number of epochs.
Loading 2202.07481v1…