PaperlayerView exact revision on arXiv

Source-linked AI summary

Advancements in Image Classification using Convolutional Neural Network

Farhana Sultana, A. Sufian, Paramartha Dutta

arXiv:1905.03288v1cs.CVcs.AI

TL;DR

Image-classification research needed architectures that addressed the limitations of conventional CNNs as datasets and visual variation expanded. This paper reviews CNN components and architectures from LeNet-5 through SENet, including model descriptions, training details, and comparative results. It reports that architectural advances such as inception modules, residual blocks, feature reuse, capsules, and squeeze-and-excitation blocks produced differing gains and limitations across datasets, with SENet reaching a 2.25% top-5 error rate on ImageNet.

  • Problem

    The paper addresses the need to understand CNN advancements for image classification, including limitations involving spatial information, viewpoint generalization, and optimization of deeper networks.

  • Method

    The paper reviews CNN components and architectures from LeNet-5 through SENet, describing model designs and training details and comparing their performance.

  • Results

    SENet reduced the ImageNet top-5 error rate to 2.25%, while the review reports improved accuracy from combining inception modules or residual blocks with conventional CNNs.

  • Takeaways & Limitations

    CNN progress involved progressively deeper or structurally enhanced architectures, feature reuse, and channel-wise recalibration, but performance depended on the dataset and model design.

Abstract

from arXiv · show

Convolutional Neural Network (CNN) is the state-of-the-art for image classification task. Here we have briefly discussed different components of CNN. In this paper, We have explained different CNN architectures for image classification. Through this paper, we have shown advancements in CNN from LeNet-5 to latest SENet model. We have discussed the model description and training details of each model. We have also drawn a comparison among those models.

I. INTRODUCTION

Image classification is a foundational computer-vision problem whose traditional approaches were limited by hand-crafted features. The paper reviews CNN advancements and compares architectures from LeNet-5 through newer models.

  • Image classification forms the basis for other computer-vision problems, including localization, segmentation, and object detection.
  • Traditional image-classification methods were bounded by challenges such as hand-crafted feature extraction.
  • CNNs developed from early visual-pattern models and practical systems such as Fukushima’s neocognitron and LeCun’s LeNet-5.
  • Larger labeled datasets, powerful GPUs, and improved regularization enabled CNNs to deliver outstanding image-classification performance.
  • The study reviews CNN architectures, describes their training details, compares models, and concludes with the reported advancements.

A. Convolutional Layer

A typical CNN combines convolution, pooling, and fully connected layers. Convolution extracts reusable local features, pooling downsamples representations, and fully connected layers produce outputs from all preceding neurons.

  • A. Convolutional Layer: The convolutional layer is the central CNN component, applying a filter across image regions to produce an output feature map.Filters are configured using backpropagation.
  • B. Sub-sampling or Pooling Layer: Pooling downsamples convolutional outputs by sub-sampling small regions into single values.Max pooling selects the largest pixel value in each region.
  • B. Sub-sampling or Pooling Layer: Pooling reduces the number of parameters and makes the network invariant to translations in shape, size, and scale.
  • C. Fully-connected Layer (FC Layer): Fully connected layers receive input from all neurons in the preceding layer and compute outputs using individual current-layer neurons.

III. DIFFERENT MODELS OF CNN FOR IMAGE CLASSIFICATION

LeNet-5 established an early CNN architecture for handwritten-digit classification. Its reported MNIST performance illustrates both the model’s effectiveness and the later need for deeper networks as data, image resolution, and class counts increased.

  • A. LeNet-5(1998):: LeNet-5 used seven trainable layers: three convolutional, two average-pooling, one fully connected, and one output layer.Sigmoid nonlinearities preceded pooling, and the output used Euclidean RBF units for ten-digit classification.
  • A. LeNet-5(1998):: The architecture description reports each LeNet-5 layer’s filter size, output feature-map size, and parameter count.
  • A. LeNet-5(1998):: LeNet-5 was trained and tested on MNIST, containing 60k training and 10k test images.Inputs were represented in a 32 × 32 pixel format, with normalized and centered characters.
  • A. LeNet-5(1998):: LeNet-5 achieved a 0.95% test error rate on MNIST, while LeNet-5 with distortion achieved 0.8%.
  • A. LeNet-5(1998):: As dataset size, image resolution, and classification categories increased, deeper CNNs and more powerful GPUs became necessary for training.

B. AlexNet-2012:

AlexNet scaled the LeNet-5 design into a large ImageNet CNN, using eight trainable layers and training techniques intended to accelerate learning and reduce overfitting. Its results indicated that network depth was important to performance.

  • AlexNet comprised 8 trainable layers: 5 convolutional and 3 fully connected.The architecture followed LeNet-5 but was substantially larger.
  • ReLU nonlinearities after convolutional and fully connected layers enabled faster training than comparable networks using tanh units.Local response normalization was applied after the first and second convolutional layers.
  • AlexNet classified ImageNet data containing about 1.2 million training images across 1000 classes for ILSVRC evaluation.The competition reported top-1 and top-5 error rates.
  • Training used 256×256 down-sampled centered images, runtime augmentation with ten random patches, PCA-based RGB shifts, and dropout.These choices were used to reduce overfitting.
  • Removing any middle layer degraded performance, indicating that the result depended on network depth.The authors used purely supervised learning and anticipated benefits from unsupervised pre-training with greater computational capacity.

2) Training Details:

ZFNet retained much of AlexNet’s architecture while modifying early filters and strides to preserve more feature information. Its deconvolutional visualization linked CNN activations to input pixels and exposed hierarchical representations.

  • Architecture: ZFNet reduced the first-layer filter size from 11×11 to 7×7 and used stride-2 convolution in both first and second layers.The changes were intended to retain more information in early-layer features.
  • Visualization: A deconvolutional network mapped higher-layer activations back to input pixels to identify which pixels contributed to feature-map activations.The procedure used unpooling switches, ReLU rectification, and transposed filters.
  • Training Details: ZFNet replaced AlexNet’s sparse connections in layers 3, 4, and 5 with dense connections.The model was trained on ImageNet data following AlexNet’s procedure.
  • Training Details: The model was trained on a single GTX-580 GPU for 12 days over 70 epochs and was also tested across multiple datasets.Experiments included Caltech 101, Caltech-256, and PASCAL-2012.
  • Training Details: Visualization showed hierarchical features, with upper layers requiring more epochs to converge than lower layers.The authors also reported stable outputs under translation and scaling and used occlusion experiments to probe local versus global information.

D. VGGNet

VGGNet deepened the AlexNet-style CNN while using small filters, and its experiments reported better performance for deeper networks with small filters than shallower networks with larger filters.

  • D. VGGNet: VGGNet used 3×3 filters throughout and offered six configurations containing 11 to 19 weighted layers.Configurations included VGG16 and VGG19.
  • D. VGGNet: Three stacked 3×3 convolutional layers matched the effective receptive field of one 7×7 layer while increasing nonlinearity and reducing parameters.Model C also used three 1×1 filters to increase nonlinearity.
  • D. VGGNet: VGGNet training followed AlexNet’s procedure with different input cropping and scaling sizes for training and testing.Pre-initialization helped the deeper, parameter-heavy model converge after 74 epochs.
  • D. VGGNet: A deep network with small filters performed better than a shallower network with larger filters.The authors concluded that network depth was important for visual representation.

1) Training Details:

Later architectures addressed efficiency and optimization challenges through inception modules and residual blocks. The reviewed results emphasized sparse computation, residual learning, and deeper models with controlled complexity.

  • Training Details: GoogLeNet training used distributed learning with asynchronous SGD, momentum 0.9, a constant learning-rate schedule, and seven ensembles.Inputs were sampled at four scales and produced 144 crops per image.
  • GoogLeNet: The inception architecture supported moving toward sparser architectures as a realistic and competent approach.
  • ResNet: Deeper CNNs suffered optimization difficulty because vanishing gradients persisted alongside worse train and test errors, without being caused by overfitting.ResNet addressed this by adding layers that performed identity mapping to a pre-trained shallower model.
  • ResNet: ResNet used stacked residual blocks with 3×3 convolutions, periodically doubled filters, and used stride 2.For depths above 50, bottleneck layers used 1×1, 3×3, and 1×1 convolutions for dimensionality reduction and efficiency.
  • ResNet: ResNet models were evaluated at depths of 34, 50, 101, and 152 layers, with the 152-layer model having lower complexity than VGG networks despite being eight times deeper.

1) Training Details:

The passage describes ResNet’s training setup and reports that greater depth can improve accuracy without degrading performance. DenseNet instead emphasizes dense feature reuse and bottleneck transformations.

  • ResNet: ResNet training used SGD with batch size 128, weight decay 0.0001, momentum 0.9, scheduled learning-rate reductions, weight initialization, and batch normalization.Dropout was not used.
  • ResNet: ResNet experiments showed that deeper networks can be trained without performance degradation and can gain accuracy while remaining easier to optimize.
  • DenseNet: DenseNet connects each layer to all preceding layers by concatenating their outputs, reusing features to strengthen propagation and reduce vanishing gradients.Using small numbers of filters also reduces parameters.
  • DenseNet: DenseNet’s dense-block transformations combine batch normalization, ReLU, and 3 × 3 convolution, with 1×1 bottleneck layers reducing dimensionality.

1) Training Details:

The selected passages cover DenseNet training and results, then introduce CapsNet’s capsule-based design for preserving relationships conventional CNNs may lose. CapsNet replaces scalar features and max-pooling with vector capsules and dynamic routing.

  • DenseNet: DenseNet training used SGD with dataset-dependent batch sizes, an initial learning rate of 0.1, and two tenfold learning-rate reductions.
  • DenseNet: DenseNet used weight decay of 0.0001, Nesterov momentum of 0.9, and dropout of 0.2.
  • DenseNet: DenseNet and DenseNet-BC outperformed previous CNN error rates on C10, C100, and SVHN datasets.A DenseNet twice as deep as ResNet achieved similar ImageNet accuracy with half as many parameters.
  • CapsNet: Conventional CNNs can lose spatial information through subsampling and struggle to generalize to novel viewpoints or affine transformations.
  • CapsNet: CapsNet replaces scalar feature detectors with vector-output capsules and max-pooling with dynamic routing by agreement.
  • CapsNet: The described three-layer CapsNet uses two convolutional layers, primary capsules, and a DigitCaps layer containing one 16D capsule per digit class.

1) Training details:

The passages describe CapsNet’s MNIST experiment and SENet’s squeeze-and-excitation mechanism. They also note a scope limitation for CapsNet and identify SENet’s documented comparison setting.

  • CapsNet: A three-layer CapsNet with three routing iterations and reconstruction regularization achieved a 0.25% test error on MNIST.
  • CapsNet: CapsNet may not perform well on large-scale datasets such as ImageNet and may suffer from vanishing gradients.
  • SENet: SENet introduced squeeze-and-excitation blocks that summarize channel activations, generate per-channel modulation weights, and rescale feature maps.The squeeze function uses average pooling, while excitation uses two fully connected layers with ReLU between them.
  • SENet: SE blocks can be stacked into SENets and incorporated into architectures including VGGNet, GoogLeNet, ResNext, Inception-ResNet, MobileNet, and ShuffleNet.
  • Comparison: Table III compares CNN configurations and identifies DenseNet variants with bottleneck layers and compression using a plus sign.

1) Training Details:

The study compares CNN variants across datasets and reports that adding SE blocks improved original models while increasing training and testing time only slightly. Its conclusion traces architectural progress from larger conventional CNNs to feature reuse, capsules, and channel recalibration.

  • Comparison: SE-augmented and original CNN models were trained and tested on ImageNet, CIFAR-10, and CIFAR-100 to compare speed–accuracy trade-offs.
  • Comparison: SE-enhanced models outperformed their original counterparts with only a small increase in training and testing time.
  • Comparison: The comparison covers different CNN configurations from AlexNet through DenseNet, reporting ImageNet validation top-1 and top-5 errors and test top-5 error.
  • Conclusion: The conclusion states that GoogLeNet and ResNet gained better accuracy by combining inception or residual blocks rather than repeatedly stacking identical building blocks.
  • Conclusion: DenseNet emphasizes feature reuse, while CapsNet performs strongly on MNIST but has not matched earlier CNN performance on high-resolution ImageNet images.
Loading 1905.03288v1…