PaperlayerView exact revision on arXiv

Source-linked AI summary

Object Detection with Deep Learning: A Review

Zhong-Qiu Zhao, Peng Zheng, Shou-tao Xu, Xindong Wu

arXiv:1807.05511v2cs.CV

TL;DR

Object detection requires locating and classifying objects, while traditional pipelines face computational and performance limitations. This review synthesizes deep-learning frameworks across detection tasks, compares representative methods, and finds CNN-based approaches often outperform classic methods, though handcrafted features remain competitive for pedestrian detection.

  • Problem

    Traditional object-detection pipelines require exhaustive region selection, feature extraction, and classification, while their performance can stagnate as complexity increases.

  • Method

    The paper systematically reviews deep-learning detection frameworks across generic, salient-object, face, and pedestrian detection, including experimental comparisons.

  • Results

    CNN-based methods significantly outperform most classic methods for face detection, while handcrafted features remain competitive in pedestrian detection.

  • Takeaways & Limitations

    The review organizes the object-detection landscape and identifies promising future directions across generic and specialized detection tasks.

Abstract

from arXiv · show

Due to object detection's close relationship with video analysis and image understanding, it has attracted much research attention in recent years. Traditional object detection methods are built on handcrafted features and shallow trainable architectures. Their performance easily stagnates by constructing complex ensembles which combine multiple low-level image features with high-level context from object detectors and scene classifiers. With the rapid development in deep learning, more powerful tools, which are able to learn semantic, high-level, deeper features, are introduced to address the problems existing in traditional architectures. These models behave differently in network architecture, training strategy and optimization function, etc. In this paper, we provide a review on deep learning based object detection frameworks. Our review begins with a brief introduction on the history of deep learning and its representative tool, namely Convolutional Neural Network (CNN). Then we focus on typical generic object detection architectures along with some modifications and useful tricks to improve detection performance further. As distinct specific detection tasks exhibit different characteristics, we also briefly survey several specific tasks, including salient object detection, face detection and pedestrian detection. Experimental analyses are also provided to compare various methods and draw some meaningful conclusions. Finally, several promising directions and tasks are provided to serve as guidelines for future work in both object detection and relevant neural network based learning systems.

I. INTRODUCTION · II. A BRIEF OVERVIEW OF DEEP LEARNING

The paper frames object detection as jointly locating objects and classifying their categories, then reviews how deep learning and CNNs address limitations of traditional detection methods. It surveys representative architectures and application-specific tasks, supported by experimental comparisons and analysis.

  • I. INTRODUCTION: Object detection supports complete image understanding and includes subtasks such as face, pedestrian, and skeleton detection.The paper describes it as a fundamental computer vision problem that provides valuable image information.
  • I. INTRODUCTION: Object detection determines where objects are located in an image and which category each object belongs to.Its traditional pipeline comprises informative region selection, feature extraction, and classification.
  • I. INTRODUCTION: Traditional detectors combine manually engineered low-level descriptors with discriminatively trained shallow models, but achieved only small gains during 2010–2012 through ensembles and minor variants.Earlier methods obtained state-of-the-art PASCAL VOC results and real-time embedded systems, yet their improvements stagnated.
  • I. INTRODUCTION: Deep neural networks, especially CNNs, learn more complex and informative object representations through deeper architectures, expressive models, and robust training algorithms.These capabilities reduce the need to manually design features.
  • I. INTRODUCTION: R-CNN and its successors introduced distinct improvements, including joint classification and box-regression optimization, region-proposal subnetworks, and fixed-grid regression.The cited models are Fast R-CNN, Faster R-CNN, and YOLO, respectively.
  • I. INTRODUCTION: The review systematically covers generic, salient, face, and pedestrian object detection, describing representative models, algorithmic characteristics, and relationships across application domains.Generic detection uses bounding box regression, while the surveyed tasks apply CNNs to different detection settings.
  • II. A BRIEF OVERVIEW OF DEEP LEARNING: Unlike an earlier software-tools review focused mainly on implementation, this paper details deep-learning detection models and algorithms and provides experimental comparisons with meaningful analyses.The review also introduces the history of deep learning and the basic CNN architecture before discussing detection approaches.

A. The History: Birth, Decline and Prosperity … 1) R-CNN:

The paper traces deep learning’s revival to data, computation, and architectural advances, then explains how CNNs learn hierarchical features for visual tasks. It introduces generic detection frameworks and presents R-CNN as a region-proposal method that improved accuracy but incurred substantial computational, training, and localization limitations.

  • A. The History: Birth, Decline and Prosperity: Deep learning became popular after 2006, following earlier stagnation caused by overfitting, limited data, limited computation, and modest performance gains.Its revival was associated with a breakthrough in speech recognition.
  • A. The History: Birth, Decline and Prosperity: Large annotated datasets, parallel computing, and advances in network design and training strategies enabled deep models to exploit greater learning capacity.Unsupervised or layerwise pre-training, dropout, and data augmentation supported initialization and training.
  • B. Architecture and Advantages of CNN: CNNs represent deep learning through feature maps, local receptive fields, and transformations that convert pixel intensities into learned internal features.A typical CNN architecture is VGG16.
  • B. Architecture and Advantages of CNN: Alternating convolution and pooling constructs feature hierarchies, while supervised fine-tuning, task-specific output activations, and objective functions adapt CNNs to visual tasks.Fully connected layers can be added before the final task-specific layer.
  • B. Architecture and Advantages of CNN: Compared with traditional shallow models, CNNs automatically learn hierarchical representations from pixels to high-level semantic features and offer greater expressive capability.Their multilevel nonlinear mappings can disentangle hidden factors of the input data.
  • III. GENERIC OBJECT DETECTION: Generic object detection locates and classifies objects in images, represents them with rectangular bounding boxes, and mainly follows region-proposal or regression/classification frameworks.The region-proposal framework first generates candidate regions and then classifies them.
  • A. Region Proposal Based Framework: Region-proposal detection performs a coarse scan before focusing on regions of interest, and Overfeat was an early representative model combining CNNs with sliding windows.It predicts bounding boxes from locations of the topmost feature map after estimating object-category confidences.
  • 1) R-CNN:: R-CNN achieved 53.3% mAP on PASCAL VOC 2012, exceeding the previous DPM HSC result by more than 30% through proposals, CNN features, and class-specific linear SVMs.Its three stages were bottom-up proposal extraction, CNN feature computation for each proposal, and region classification.

2) SPP-net:

SPP-net applies spatial pyramid pooling to reuse convolutional feature maps and convert region proposals of arbitrary sizes into fixed-length vectors. It improves detection accuracy and testing efficiency over R-CNN, but retains a multi-stage pipeline and limits fine-tuning of preceding convolutional layers.

  • SPP-net:: SPP-net incorporates spatial pyramid matching through an SPP layer that partitions features across multiple scales and aggregates them into representations.The pyramid ranges from finer to coarser divisions.
  • SPP-net:: Unlike R-CNN, SPP-net reuses conv5 feature maps to project region proposals of arbitrary sizes into fixed-length feature vectors.This reuse is enabled by feature maps encoding both local response strength and spatial-position relationships.
  • SPP-net:: SPP-net improves accuracy through scale-appropriate region-proposal estimation and increases testing efficiency by sharing computation before the SPP layer.The shared computation is performed once across different proposals rather than repeated for each proposal.
  • SPP-net:: Despite improving accuracy and efficiency over R-CNN, SPP-net retains feature extraction, fine-tuning, SVM training, and bounding-box regression as separate stages.This multi-stage design still requires additional storage space.
  • SPP-net:: The fine-tuning algorithm introduced for SPP-net cannot update convolutional layers preceding the SPP layer.The passage identifies this restriction as a notable drawback of the architecture.

3) Fast R-CNN: · 4) Faster R-CNN:

Fast R-CNN shares convolutional computation across region proposals, jointly trains classification and bounding-box regression, and improves efficiency through hierarchical RoI sampling. Faster R-CNN replaces external proposal generation with an RPN, enabling end-to-end detection at 5 FPS while retaining limitations from alternate training and extreme object scales.

  • 3) Fast R-CNN:: Fast R-CNN jointly optimizes classification and bounding-box regression with a multi-task loss, omitting the localization term for background RoIs.The loss combines classification log loss with a localization term gated by [u ≥1].
  • 3) Fast R-CNN:: Fast R-CNN samples N images followed by R/N RoIs per image, sharing computation and memory among RoIs from the same image during forward and backward passes.Hierarchical mini-batch sampling addresses inefficiency when RoIs originate from different images.
  • 3) Fast R-CNN:: Fast R-CNN processes the whole image convolutionally, applies RoI pooling to each proposal, and feeds fixed-length vectors through fully connected layers into sibling outputs.RoI pooling is a one-level special case of spatial pyramid pooling.
  • 3) Fast R-CNN:: Fast R-CNN trains all network layers in one stage regardless of proposal generation, reducing storage expense while improving accuracy and efficiency.These gains are attributed to its multi-task loss and more reasonable training schemes.
  • 3) Fast R-CNN:: External proposal generation remains an efficiency bottleneck, motivating an RPN that shares full-image convolutional features with the detection network at nearly no additional cost.Selective Search and EdgeBoxes are identified as additional methods used by prevailing detection networks.
  • 4) Faster R-CNN:: Faster R-CNN uses a fully convolutional RPN to predict object bounds and scores at every position, operating on shared convolutional features for arbitrary-sized images.The RPN generates rectangular proposals while sharing preceding layers with the object-detection network.
  • 4) Faster R-CNN:: The RPN slides over feature maps, producing low-dimensional vectors that feed sibling classification and regression layers; Faster R-CNN uses 3 scales and 3 aspect ratios of anchors.The implementation uses an n × n convolution followed by two sibling 1 × 1 convolutions, with ReLU after the n × n layer.
  • 4) Faster R-CNN:: 5 FPS on a GPU was achieved with state-of-the-art PASCAL VOC 2007 and 2012 accuracy, while alternate training was time-consuming and proposals included backgrounds.RPN proposals were also limited for objects with extreme scales.

5) R-FCN:

R-FCN adapts fully convolutional architectures to object detection by resolving the tension between translation variance and classification-oriented translation invariance. Its position-sensitive pooling design shares nearly all layers, achieves state-of-the-art results on PASCAL VOC and Microsoft COCO, and runs at 170ms per image.

  • 5) R-FCN:: R-FCN shares nearly all layers while using powerful classification networks in a fully convolutional detection architecture.This design enables fully convolutional detection without an RoI-wise subnetwork.
  • 5) R-FCN:: R-FCN addresses the challenge that naive fully convolutional detection can be inferior because detection must preserve translation variance while classification favors translation invariance.Shifting an object should be indiscriminative for classification, but translations within a bounding box affect detection.
  • 5) R-FCN:: For each category, R-FCN generates k^2 position-sensitive score maps on a k × k grid and aggregates them with position-sensitive RoI pooling.The pooled k^2 scores are averaged into a C + 1-dimensional vector for category prediction.
  • 5) R-FCN:: R-FCN appends a 4k^2-dimensional convolutional layer to produce class-agnostic bounding boxes after category scores are computed.Softmax responses are computed across categories before bounding-box prediction.
  • 5) R-FCN:: 170ms per image is the reported test speed for R-FCN, which achieves state-of-the-art results on both PASCAL VOC and Microsoft COCO datasets.The result is reported for the fully convolutional R-FCN architecture.
  • 5) R-FCN:: Featurized image pyramids improve scale invariance but rapidly increase training time and memory consumption, motivating alternatives using single input scales.Single-scale training can create train/test inconsistency when image pyramids are used only at test time.

6) FPN: · 7) Mask R-CNN: · 8) Multi-task Learning, Multi-scale Representation and Contextual Modelling:

The reviewed frameworks improve detection and instance-level recognition through feature pyramids, mask-aware multi-task learning, and quantization-free RoI alignment. The paper also surveys multi-task, multi-scale, contextual, and combined representations for addressing small objects, occlusion, local similarity, and varied object scales.

  • 6) FPN:: FPN combines bottom-up and top-down pathways with lateral connections to fuse low-resolution semantic features and high-resolution spatial features.The top-down maps are upsampled, reduced with 1 × 1 convolutions, merged by element-wise addition, and refined with 3×3 convolutions.
  • 6) FPN:: FPN extracts rich semantics across scales, trains end-to-end without sacrificing speed or memory, and applies independently of backbone architecture.It can support different detection stages, including region proposal generation, and other vision tasks such as instance segmentation.
  • 6) FPN:: Mask R-CNN adds a segmentation-mask branch alongside Faster R-CNN classification and bounding-box regression to jointly perform instance segmentation.The mask branch preserves explicit object layout with an m × m mask and contributes an additional loss to the multi-task objective.
  • 7) Mask R-CNN:: RoIAlign replaces RoI pooling’s coarse quantization with bilinear interpolation to preserve per-pixel correspondence for accurate mask prediction.RoI pooling’s misalignment affects pixel-to-pixel mask prediction more strongly than classification.
  • 7) Mask R-CNN:: Mask R-CNN adds a small computational burden while combining complementary tasks, producing a flexible and efficient framework for instance-level recognition.The framework can be generalized to tasks such as human pose estimation with minimal modification.
  • 8) Multi-task Learning, Multi-scale Representation and Contextual Modelling:: Multi-task learning uses correlated tasks, such as object and stuff segmentation, to guide more accurate detection of small objects.The surveyed approaches include StuffNet and multitask network cascades combining region proposals, pixel-level instance segmentation, and regional classification.
  • 8) Multi-task Learning, Multi-scale Representation and Contextual Modelling:: Multi-scale representation combines activations from multiple layers to provide semantic information at different spatial resolutions and address object–receptive-field size inconsistency.MS-CNN uses multiple scale-independent output layers, while other methods exploit scale-dependent pooling and cascaded rejection classifiers.
  • 8) Multi-task Learning, Multi-scale Representation and Contextual Modelling:: Contextual modelling exploits features from or around RoIs across support regions and resolutions, while combined models integrate complementary components for further detection improvements.Examples use segmentation, global context, object parts, semantic segmentation-aware features, and information inside and outside the RoI.

9) Thinking in Deep Learning based Object Detection: · B. Regression/Classification Based Framework

The section reviews factors that improve deep-learning object detection, including hard-example mining, classifier design, subcategory-aware proposals, hierarchical learning, and efficient architectures. It then introduces regression/classification frameworks as faster alternatives to multi-stage region-proposal systems, focusing on YOLO and SSD.

  • 9) Thinking in Deep Learning based Object Detection:: Online hard-example mining (OHEM) automatically selects difficult samples to address the imbalance between annotated objects and background examples.The method leads to more effective and efficient training.
  • 9) Thinking in Deep Learning based Object Detection:: Carefully constructing a deep, convolutional per-region classifier is especially important for object detection with ResNets and GoogLeNets.
  • 9) Thinking in Deep Learning based Object Detection:: A subcategory-aware region proposal network uses object-pose information to guide proposals while jointly optimizing detection and subcategory classification.This addresses scale variation, occlusion, and truncation in 2D detection.
  • 9) Thinking in Deep Learning based Object Detection:: Hierarchical feature learning addresses long-tailed class distributions by clustering visually similar classes and learning deep representations for each group separately.
  • 9) Thinking in Deep Learning based Object Detection:: PVANET follows a deep-and-thin design, combines concatenated ReLU, Inception, and HyperNet blocks, and uses batch normalization, residual connections, and plateau-based learning-rate scheduling.It reduces multi-scale feature-extraction expense and achieves real-time processing at 21 FPS on a Titan X GPU.
  • B. Regression/Classification Based Framework: Region-proposal frameworks use correlated stages that are often trained separately, making component processing a bottleneck for real-time applications.Even Faster R-CNN requires alternative training to obtain shared convolution parameters between its RPN and detection network.
  • B. Regression/Classification Based Framework: One-step global regression/classification frameworks map image pixels directly to bounding-box coordinates and class probabilities, reducing time expense and motivating reviews of YOLO and SSD.Earlier researchers had also modeled object detection as regression or classification before YOLO and SSD.

1) Pioneer Works: · 2) YOLO:

Early deep-learning detectors explored regression, segmentation, iterative refinement, and grid-based proposals, while YOLO unified category confidence and bounding-box prediction over an image grid. YOLO enabled real-time detection but remained limited on small clustered objects, unusual aspect ratios, and coarse spatial features.

  • 1) Pioneer Works:: Szegedy et al. formulated detection as DNN regression by generating a binary mask and inferring bounding boxes, but overlapping objects and direct upsampling remained problematic.
  • 1) Pioneer Works:: Pinheiro et al. combined class-agnostic segmentation and patch-level object likelihood in two CNN branches, sharing most operations for efficient inference.
  • 1) Pioneer Works:: AttentionNet iteratively refined weak directions from top-left and bottom-right image corners into accurate object boxes, though multiple categories made it inefficient.
  • 1) Pioneer Works:: G-CNN iteratively moved and scaled a fixed multi-scale grid toward objects, but struggled with small or highly overlapping targets.
  • 2) YOLO:: YOLO divides an image into an S × S grid, with each cell predicting B bounding boxes, category confidences, and conditional class probabilities for objects centered there.
  • 2) YOLO:: 45 FPS is achieved by YOLO, while Fast YOLO reaches 155 FPS and produces fewer background false positives, enabling cooperation with Fast R-CNN.
  • 2) YOLO:: YOLO struggles with small objects in groups, unusual aspect-ratio configurations, and coarse features caused by repeated downsampling.

3) SSD: · C. Experimental Evaluation · 1) PASCAL VOC 2007/2012:

SSD uses multi-scale default anchor boxes and added feature layers to perform fast single-shot detection, while benchmark comparisons show that architectural advances and training strategies progressively improve accuracy on PASCAL VOC and related datasets.

  • 3) SSD:: SSD discretizes bounding-box outputs with default anchor boxes spanning multiple aspect ratios and scales, then fuses predictions from multiple feature maps for varied object sizes.This single-shot design replaces YOLO’s fixed grids with multi-scale anchor-based predictions.
  • 3) SSD:: Using VGG16, SSD appends feature layers that predict offsets and confidences for default boxes, combines localization and confidence losses, and applies NMS to multi-scale detections.The losses include Smooth L1 localization and Softmax confidence terms.
  • 3) SSD:: SSD with hard-negative mining, data augmentation, and carefully selected anchors outperforms Faster R-CNN in accuracy on PASCAL VOC and COCO while being three times faster.SSD300 runs at 59 FPS and is more accurate and efficient than YOLO, but SSD struggles with small objects.
  • C. Experimental Evaluation: The evaluation compares a broad set of detectors, including R-CNN, Fast and Faster R-CNN, SSD300, SSD512, YOLO, YOLOv2, R-FCN, FPN, and other approaches.The comparison spans PASCAL VOC 2007, PASCAL VOC 2012, and Microsoft COCO.
  • 1) PASCAL VOC 2007/2012:: PASCAL VOC 2007 and 2012 contain 20 categories, with performance reported using category-level Average Precision and 20-category mean Average Precision.Comparative results are presented in Tables II and III.
  • 1) PASCAL VOC 2007/2012:: More powerful backbone CNNs, SPP layers, end-to-end multi-task architectures, and RPNs progressively improve object-detection performance on the evaluated benchmarks.The cited comparisons include R-CNN variants, SPP-net, Fast R-CNN, and Faster R-CNN.
  • 1) PASCAL VOC 2007/2012:: YOLO’s poor high-IoU localization produces weak VOC 2012 results, while Fast R-CNN fusion and YOLOv2 strategies such as anchor boxes, batch normalization, and fine-grained features correct localization errors.R-FCN further improves detection by combining recent techniques in a fully convolutional network.
  • 1) PASCAL VOC 2007/2012:: COCO is more challenging than PASCAL 2012 because it contains less iconic objects across broader scales and imposes stricter localization requirements.COCO evaluation reports AP across different IoU thresholds and object sizes.

2) Microsoft COCO: · 3) Timing Analysis:

On Microsoft COCO, multi-scale representations, feature pyramids, and complementary task information improve detection and localization, while region proposal methods perform better overall. Timing analysis shows that shared features and architectural simplification reduce testing cost, whereas multi-scale processing and deeper networks increase time, creating an accuracy–speed trade-off.

  • 2) Microsoft COCO:: Multi-scale training and testing improve object detection by providing information at different resolutions.R-FCN benefits from multi-scale training and testing, while FPN and DSSD construct feature pyramids for multi-scale representation.
  • 2) Microsoft COCO:: Complementary information from related tasks improves accurate object localization.Mask R-CNN benefits from incorporating instance segmentation as a related task.
  • 2) Microsoft COCO:: Region proposal methods, including Faster R-CNN and R-FCN, perform better overall on Microsoft COCO.
  • 3) Timing Analysis:: Computing CNN features on shared feature maps substantially reduces testing time, with further reductions from unified multi-task learning and removing additional region proposal generation.These changes are described for SPP-net and Faster R-CNN; SVD compression of fully connected layers also helps in PAVNET and FRCN.
  • 3) Timing Analysis:: Multi-scale feature extraction and contextual information increase test time.The timing cost is reported for ION and MR-RCNN&S-RCNN.
  • 3) Timing Analysis:: Deeper networks require more training time, but sharing more layers in fully convolutional layers can reduce this cost.The comparison given is ResNet101 against VGG16, with FRCN illustrating the reduction mechanism.
  • 3) Timing Analysis:: Regression-based models generally achieve real-time processing at the cost of lower accuracy than region proposal methods.Region proposal models can be adapted into real-time systems using techniques such as batch normalization and residual connections, as in PVANET.

IV. SALIENT OBJECT DETECTION · A. Deep learning in Salient Object Detection · B. Experimental Evaluation

The section presents salient object detection as a task for highlighting dominant image regions and reviews CNN-based approaches that integrate semantic, local, global, and multiscale information. Its evaluation uses standard datasets and metrics, showing CNN methods outperform classical approaches while motivating boundary modeling and complementary context.

  • IV. SALIENT OBJECT DETECTION: Salient object detection highlights the most dominant object regions and supports applications including image cropping, segmentation, image retrieval, and object detection.
  • IV. SALIENT OBJECT DETECTION: Bottom-up methods center on local feature contrast, but low-level features cannot capture high-level or multiscale semantic information.
  • A. Deep learning in Salient Object Detection: CNNs are extended to salient object detection because they provide high-level and multiscale representations and have succeeded in related vision tasks.
  • A. Deep learning in Salient Object Detection: Deep learning approaches learn saliency through transferred features, local-global clue integration, multi-context modeling, semantic segmentation, and multiscale feature maps.
  • A. Deep learning in Salient Object Detection: Direct pixel-wise CNN architectures, including shallow networks and deeper deconvoluted VGG-based models, predict salient objects with recurrent and deconvolution networks.
  • B. Experimental Evaluation: Four representative datasets—ECSSD, HKU-IS, PASCALS, and SOD—evaluate state-of-the-art salient object detection methods, including challenging low-contrast and multi-object images.
  • B. Experimental Evaluation: Evaluation uses F-measure and mean absolute error (MAE), with β2 = 0.3 emphasizing precision and MAE emphasizing correctly detected salient objects over non-salient pixels.
  • B. Experimental Evaluation: CNN-based methods outperform classical methods; successful systems combine local and global context, recurrent networks or semantic segmentation, boundary-aware superpixels, multiscale features, and cross-layer connections.

V. FACE DETECTION · A. Deep learning in Face Detection

Face detection supports several face-related applications but differs from generic object detection through its broad scale range and distinctive facial structure. Deep learning approaches address localization, pose variation, multitask learning, and real-time deployment through CNNs, cascades, and architectural adaptations.

  • V. FACE DETECTION: Face detection is an important preprocessing step for face recognition, face synthesis, and facial expression analysis.
  • V. FACE DETECTION: Faces require detection across a much broader scale range than generic objects, from 30–300 points versus 10–1000 points.
  • V. FACE DETECTION: Viola–Jones used cascaded Haar-Like-feature classifiers with AdaBoost, while Felzenszwalb et al. introduced a deformable part model for face detection.
  • A. Deep learning in Face Detection: CNN-based methods improve face localization with a joint IoU loss and support multi-view detection without pose or landmark annotations.Yu et al. jointly predict four box bounds, while Farfade et al.’s Deep Dense Face Detector handles widely varying face orientations.
  • A. Deep learning in Face Detection: Generic detection architectures such as Faster R-CNN have been extended to face detection with task-specific modifications.
  • A. Deep learning in Face Detection: Multitask CNN frameworks jointly learn face detection with landmark localization or 3D face modeling using end-to-end training.DenseBox combines detection and landmark localization, whereas another framework integrates a ConvNet with a fixed 3D mean face model.
  • A. Deep learning in Face Detection: Compact CNN cascades enable real-time frontal face detection on mobile platforms by progressively generating, classifying, and refining candidate positions.
  • A. Deep learning in Face Detection: A cascaded Supervised Transformer Network reduces the effects of large pose variations in face detection.

B. Experimental Evaluation · VI. PEDESTRIAN DETECTION · A. Deep learning in Pedestrian Detection

The review evaluates face-detection methods on FDDB and finds CNN-based approaches generally outperform classic handcrafted-feature methods, while pedestrian detection requires adaptations for small instances and hard backgrounds. Deep pedestrian detectors improve through multimodal, cascaded, multitask, and fusion strategies, but their complexity motivates simpler designs.

  • B. Experimental Evaluation: FDDB evaluation uses discrete and continuous scores, with detections considered positive when IoU exceeds 0.5.The dataset contains 2,845 images and 5,171 elliptically annotated faces.
  • B. Experimental Evaluation: CNN-based face detectors significantly outperform most classic methods, although LDCF+ remains relatively competitive.The comparison includes handcrafted-feature and deep-CNN methods, with ROC curves reported in Figure 11.
  • B. Experimental Evaluation: Most CNN face detectors achieve true positive rates between 60% and 70%, while DeepIR and HR-ER perform substantially better; Joint Cascade remains competitive among classic methods.
  • VI. PEDESTRIAN DETECTION: Pedestrian detection is closely connected to pedestrian tracking, person re-identification, and robot navigation, and historically used boosted forests, handcrafted features, and part-based models.
  • VI. PEDESTRIAN DETECTION: Small pedestrian instances can produce collapsed, plain RoI-pooled features, while hard-background confusion is a primary source of false predictions.
  • A. Deep learning in Pedestrian Detection: Despite strong generic object-detection performance, DCNN pedestrian detectors long failed to surpass the best handcrafted-feature method, motivating adaptations of Faster R-CNN.
  • A. Deep learning in Pedestrian Detection: Pedestrian detectors combine handcrafted and DCNN features, color and thermal imagery, multiple tasks, pedestrian and scene attributes, or parallel soft-rejection classifiers.
  • A. Deep learning in Pedestrian Detection: These approaches can be highly sophisticated, so simplifying architectures is important for reducing software and hardware burden.CompACT-Deep combines handcrafted features with CNN and VGG16 models, while DeepParts uses 45 fine-tuned DCNN models.

B. Experimental Evaluation · VII. PROMISING FUTURE DIRECTIONS AND TASKS · VIII. CONCLUSION

The evaluation compares handcrafted- and CNN-based pedestrian detectors on Caltech, finding complementary strengths between the approaches. The review identifies small-object, scale, cascade, efficiency, multitask, multimodal, and 3D detection as future priorities while concluding that deep learning addresses challenging detection conditions through architectural modifications.

  • B. Experimental Evaluation: The evaluation uses Caltech Pedestrian, containing 250,000 video frames, about 2,300 unique pedestrians, and 350,000 annotated bounding boxes.Labels distinguish clear person identifications, unclear identifications, and large groups of individuals.
  • B. Experimental Evaluation: Evaluated methods span handcrafted-feature detectors and deep CNN-feature methods, with results reported in Table VII.The listed methods include Checkerboards+, LDCF++, SCF+AlexNet, SA-FastRCNN, MS-CNN, DeepParts, CompACT-Deep, RPN+BF, and F-DNN+SS.
  • B. Experimental Evaluation: Handcrafted features remain competitive in pedestrian detection, while CNN methods provide more accurate candidate boxes and multi-level semantic information.Combining handcrafted and CNN features can improve results further through complementary information.
  • VII. PROMISING FUTURE DIRECTIONS AND TASKS: Small-object detection, including cases in COCO and face detection, remains an open issue requiring architectural changes for localization under partial occlusion.The paper identifies small-object detection as a first future direction.
  • VII. PROMISING FUTURE DIRECTIONS AND TASKS: Future detectors should explore multi-task joint optimization and multimodal fusion by exploiting correlations among object-detection sub-tasks and related tasks.The paper gives superpixel semantic segmentation in salient object detection as an example.
  • VII. PROMISING FUTURE DIRECTIONS AND TASKS: Scale robustness requires scale-invariant, multi-scale, or scale-adaptive detectors, supported by stronger backbones, negative-sample mining, reverse connections, and subcategory modeling.Scale variation is especially apparent in face and pedestrian detection.
  • VII. PROMISING FUTURE DIRECTIONS AND TASKS: Cascade architectures reject easy examples early, while future systems must balance speed, memory, and accuracy and can use compact models, improved pretraining, distillation, or hint learning.DSOD is presented as a guideline for training from scratch across different image sources and tasks.
Loading 1807.05511v2…