Source-linked AI summary
Implementation of Training Convolutional Neural Networks
Tianyi Liu, Shuangsang Fang, Yuehui Zhao, Peng Wang, Jun Zhang
TL;DR
Face recognition is challenged by low resolution and pose variation, while lengthy computation can exceed real-time tolerances. The paper implements a CNN face recognizer with cloud parallel computing and reports faster parallel execution alongside a theoretical speedup analysis.
Problem
Low-resolution face images and pose variation degrade recognition performance, while long training and recognition computation can exceed real-time tolerances.
Source: §1 · pp. 2–3Method
The paper implements a CNN for face recognition and uses cloud-based parallel computing, with analysis of forward and back-propagation processes.
Source: §1 · p. 3, §3.1 · p. 3, §5 · p. 9Results
The reported timing was 5317.000000 ms serial versus 2665.000000 ms parallel, with speed-up ratio 1.995122 and speed-up efficiency 0.997561.
Source: §4.3 · p. 9Takeaways & Limitations
The authors identify finer-grained parallelism in additional algorithm modules as future work beyond their coarse-grained strategy.
Source: §5 · p. 9The theoretical analysis assumes serial and parallel methods use the same number of training iterations.
Source: §4.2 · p. 7
Abstract
from arXiv · showhide
Deep learning refers to the shining branch of machine learning that is based on learning levels of representations. Convolutional Neural Networks (CNN) is one kind of deep neural network. It can study concurrently. In this article, we gave a detailed analysis of the process of CNN algorithm both the forward process and back propagation. Then we applied the particular convolutional neural network to implement the typical face recognition problem by java. Then, a parallel strategy was proposed in section4. In addition, by measuring the actual time of forward and backward computing, we analysed the maximal speed up and parallel efficiency theoretically.
1. INTRODUTION
The introduction motivates deep representations and CNNs as tools for face recognition, where pose variation and low resolution degrade performance. It proposes cloud parallel computing to address the long computation time that can prevent real-time use.
1. INTRODUTION
- Deep learning learns levels of representation, while earlier shallow models included SVM, Boosting, and Logistic Regression.The introduction also describes layer-wise pretraining as a way to address deep-network training difficulty, implemented here through unsupervised learning.Source: §1 · pp. 1–3
- CNNs use local receptive fields and shared weights to learn image features while reducing model complexity and training parameters.The introduction describes feature-extraction and feature-map layers, with pooling that reduces resolution.Source: §1 · p. 2
- Low-resolution images and pose variation sharply degrade face-recognition performance and challenge practical monitoring systems.The introduction also notes that pose variation introduces nonlinear factors.Source: §1 · p. 2
- The paper applies a CNN to face recognition and proposes cloud-based concurrent computation because long training and recognition times can exceed real-time tolerances.The authors state that CNN face recognition can overcome pose or resolution influences.Source: §1 · p. 3
2. BACKGROUND AND RELATED WORK
The related-work section surveys CNN applications across image, speech, and visual-recognition tasks, then turns to parallel training as a way to speed ConvNets.
2. BACKGROUND AND RELATED WORK
- LeCun and colleagues designed CNNs for variable two-dimensional shapes and reported outperforming other techniques.Source: §2 · p. 3
- Other cited CNN applications include robust language identification, visual recognition [9], facial-point detection [6], and number-recognition tasks [10, 11].Source: §2 · p. 3
- Facebook AI Group studied parallel multi-GPU training of a standard architecture on ImageNet to speed convergence.Source: §2 · p. 3
3. PRINCIPLE OF CONVELUTIONAL NEURAL NETWORKS
The section describes CNNs as locally connected, weight-sharing networks and outlines their convolution, sampling, and face-recognition architecture. It then presents forward-pass and back-propagation calculations across the network layers.
3.1 Methodology
- CNNs are multilayer perceptrons for two-dimensional image recognition that use local image regions and shared neuron parameters rather than global connections.The methodology identifies convolution and sampling as the two main processes; sampling reduces each neighborhood to a pixel before weighting, bias, and activation.Source: §3.1 · pp. 3–4
- The methodology highlights local receptive fields, shared weights, and subsampling as ways to extract features and reduce training parameters.It also associates subsampling with some robustness to scale, deformation, and displacement.Source: §3.1 · p. 4
3.2 CNN Architecture Design
- The face model preprocesses 96 × 96 grayscale inputs to 32 × 32 and uses seven layers: input, C1, S1, C2, S2, hidden, and output.C1 uses six 5 × 5 kernels to produce six 28 × 28 feature maps from the 32 × 32 input.Source: §3.2 · pp. 4–5
- The architecture continues through S1 and C2, then connects S2's 400 neurons to 170 hidden neurons and 17 output neurons.The described trainable-parameter totals are 5,880 connections for S1, 150 parameters for C2, 32 parameters for S2, 48,120 for H, and 2,907 for F.Source: §3.2 · p. 5
3.1.1 Forward pass
- The forward pass specifies outputs for convolution, subsampling, hidden, and output layers, with prose identifying f as the number of convolution cores in a feature pattern.Source: §3.1.1 · p. 5
3.1.2 Back propagation
- The back-propagation section labels output deviations and weight and bias variations across output, hidden, convolution, and subsampling layers.Source: §3.1.2 · pp. 6–7
4. EXPERIMENTS
The experiments evaluate CNN face recognition on Yale Face Database images and model serial versus parallel training time. The reported timing analysis gives a speed-up ratio of 1.995122 and speed-up efficiency of 0.997561.
4.2 Parallel Strategy and Parallel Efficiency
- Serial execution repeats the forward- and backward-pass work across N images, with a separate weight-and-bias update term.t1 is forward-pass time, t2 is backpropagation time, and t3 is the weight-and-bias update time.Source: §4.2 · pp. 7–8
- The parallel-time model uses N/n image allocation across n nodes, and speed-up efficiency is defined as speed-up ratio/n.The passage defines N as the number of images and n as the number of nodes.Source: §4.2 · p. 8
4.1 Setups
- The experiment uses 136 images from Yale Face Database; the error stabilizes after four iterations, and that constant becomes the threshold for the stopping check.The passage describes a training phase to determine the minimum error, followed by a phase that uses the constant threshold.Source: §4.3 · p. 8
4.3 Results Analysis
- One passage reports 12374.3 milliseconds and attributes algorithm failure to falling into a local optimum; a separate theoretical-analysis passage lists 12874.3 under average/ms.Source: §4.3 · p. 9
- 5317.000000 serial time and 2665.000000 parallel time yield a speed-up ratio of 1.995122 and speed-up efficiency of 0.997561.These are the reported measured times and derived metrics.Source: §4.3 · p. 9
5. CONCLUSION
The conclusion describes a CNN-based face-recognition implementation and parallel computing on a cloud platform. It reports experimental verification of acceleration analysis, while identifying coarse-grained parallelism as an area for future improvement.
5. CONCLUSION
- The work applies a convolutional neural network to face recognition and uses it for parallel computing on a cloud platform.The authors describe this as excavating deep information from a multilayer network during face recognition.Source: §5 · p. 9
- The authors analyze theoretical acceleration and report experimental verification, characterizing the results as good.Source: §5 · p. 9
- The implemented parallelism is coarse-grained, and finer-grained algorithm modules are identified as future work.Source: §5 · p. 9