PaperlayerView exact revision on arXiv

Source-linked AI summary

Recent Advances in Object Detection in the Age of Deep Convolutional Neural Networks

Shivang Agarwal, Jean Ogier Du Terrail, Frédéric Jurie

arXiv:1809.03193v2cs.CV

TL;DR

Object detection matters as a precursor to machines understanding their surroundings, while recent deep-learning methods have improved performance despite more challenging datasets. This survey synthesizes detection methods and challenges, finding that leading approaches cluster around single- and two-stage detectors while research remains active.

  • Problem

    Object detection must locate and classify instances amid substantial variation in size, lighting, appearance, background, and occlusion, and is a precursor to machines understanding their surroundings.

    Source: §1 · p. 4

  • Method

    The paper surveys recent deep-learning and data-driven detection methods, their shortcomings, challenges, and extensions, focusing mainly on bounding-box tasks.

    Source: §1 · p. 4, §1.1.3 · p. 8, §5 · p. 53

  • Results

    Top-performing methods cluster around single-stage approaches such as SSD or YOLO and two-stage approaches in the footsteps of Faster RCNN, while the field remains active.

    Source: §5 · p. 53

  • Takeaways & Limitations

    Modern detectors remain powerless in some domain-adaptation, occlusion, rotation, and small-object cases, with adversarial training and context among the prominent proposed directions.

    Source: §3.3 · p. 43

    The survey covers only evaluation methodologies found in recent literature.

    Source: §1.1.3 · p. 8

Abstract

from arXiv · show

Object detection-the computer vision task dealing with detecting instances of objects of a certain class (e.g., 'car', 'plane', etc.) in images-attracted a lot of attention from the community during the last 5 years. This strong interest can be explained not only by the importance this task has for many applications but also by the phenomenal advances in this area since the arrival of deep convolutional neural networks (DCNN). This article reviews the recent literature on object detection with deep CNN, in a comprehensive way, and provides an in-depth view of these recent advances. The survey covers not only the typical architectures (SSD, YOLO, Faster-RCNN) but also discusses the challenges currently met by the community and goes on to show how the problem of object detection can be extended. This survey also reviews the public datasets and associated state-of-the-art algorithms.

1 Introduction

This survey maps recent advances in object detection, a task that must classify and locate object instances despite substantial variation and real-time constraints. It traces the shift to deep CNN detectors, introduces the task and evaluation metrics, and reviews new architectures, their advances, and limitations.

1.1.1 Problem definition

  • Object detection extends classification by locating and classifying instances while avoiding background and duplicate detections, despite scale variation, occlusion, appearance changes, and limited data.Instances can occupy 0.01% to 0.25% or 80% to 90% of image pixels, and small objects are especially difficult to detect.Source: §1 · p. 4

1 Introduction

  • The survey connects recent data-driven methods to their performance gains and shortcomings, while excluding hand-crafted approaches and related tasks except when they provide detector context.It also examines less-mainstream methods and topics such as interpretability, lifelong detection, few-shot learning, and domain adaptation.Source: §1 · pp. 4–5

1.1.2 Performance evaluation

  • Evaluation matches detections to ground truth under geometric and one-to-one assignment rules; Pascal VOC uses per-category AP at IoU 0.50, while MSCOCO averages AP across 10 IoU thresholds from 0.50 to 0.95.AP summarizes the precision-recall curve; duplicate detections count as false detections.Source: §1.1.2 · pp. 6–7
  • Evaluation conventions also vary by task: DOTA distinguishes horizontal and oriented boxes, KITTI measures 3D-box overlap, and video benchmarks may score frames independently or use tubelet IoU.VeDAI counts detections based on proximity to object centers, regardless of scale.Source: §1.1.2 · p. 8, Object detection in videos · p. 8

1.1.3 Other detection tasks

  • The survey focuses mainly on bounding-box tasks because their literature is extensive, many datasets lack pixel-level labels, and boxes remain useful for distinguishing same-class instances in segmentation pipelines.It notes that richer metrics than AP have been proposed, particularly to characterize localization error.Source: §1.1.3 · pp. 8–9

1.2 From Hand-crafted to Data Driven Detectors

  • Before DCNNs, detectors relied on engineered features and classifiers, including boosting, SVMs, and the Deformable Part Model; DCNNs learned filters from training data but required copious data.Earlier deep CNN work showed improved detection performance over HOG- or SIFT-based systems, and transfer learning helped when data was scarce.Source: §1.2 · p. 9

1.3 Overview of Recent Detectors

  • RPNs replaced non-learned proposal heuristics and enabled end-to-end architectures, while anchor-based fully convolutional designs became common; these detectors are more efficient but can be opaque and data-hungry.Two-stage variants became prominent methods for object detection.Source: §1.3 · p. 10
  • CNN features revived single-stage detection: Redmon et al. showed speed with acceptable performance, and Liu et al. improved speed and accuracy using anchors at different layers.These works prompted substantial literature on single-stage methods; the survey also covers recent work using ideas close to boosting and deformable part-based models, including Dai et al..Source: §1.3 · p. 10
  • Fully convolutional architectures enable training and inference implementation tricks, but their subtle design choices can seem like a dark art to newcomers.The survey says these implementation tricks are discussed at the end of the following section.Source: §1.3 · p. 10
  • The survey reviews object detection and its metrics before presenting recent deep-learning detector designs and the principles behind their architectural ideas.The next section moves directly to recent detector designs.Source: §1.3 · p. 11

2 On the Design of Modern Deep Detectors

This section surveys modern detector design across architectures, training choices, data augmentation, and inference-time refinement, including two-stage detectors, cascades, and parts-based models. It also notes the prominence of architectural research and the convergence of mainstream detectors on fully convolutional pipelines with anchors and multiscale features.

2.1.1 Backbone Networks

  • Backbone accuracy generally tracks detector accuracy for Faster R-CNN and R-FCN but not SSD; larger networks cost more time and data, while multiscale prediction and feature fusion address scale and context.Finer-resolution layers tend to help detect small objects, while coarser layers suit larger objects; top-down networks with lateral connections fuse features across scales.Source: §2.1.1 · pp. 11, 13, 14

2.1.2 Single Stage Detectors

  • Single-stage detectors are often faster but less accurate than two-stage methods; YOLO is fast and uses image-wide context, but struggles with small objects and precise localization, while SSD improves on YOLO with dense multiscale boxes.SSD predicts classes and box locations using default boxes across feature maps, then typically applies Greedy NMS to remove duplicate detections.Source: §2.1.2 · pp. 14–15
  • RetinaNet combines an FPN with single-stage detection and focal loss; RefineDet adds anchor-refinement and object-detection modules, while CornerNet predicts paired corner keypoints without anchors and beat two competitors on COCO.CornerNet groups class-specific corner heat-map predictions into boxes using learned corner embeddings.Source: §2.1.2 · p. 17

2.1.3 Double Stage Detectors

  • Two-stage detectors evolved from selective-search R-CNN and Fast R-CNN to Faster R-CNN, whose shared-backbone RPN learns proposals for a detector that classifies objects and refines their boxes.Fast R-CNN introduced RoI pooling over full-image features, improving on the slower R-CNN approach.Source: §2.1.3 · p. 18
  • R-FCN shares convolutional computation through position-sensitive feature maps to improve inference speed, but can perform slightly worse; two-stage methods achieve strong COCO performance yet are slow and use coarse single-scale features.R-FCN’s position-sensitive pooling avoids repeated RoI-wise subnetworks, while two-stage detectors commonly process about 300 proposals.Source: §2.1.3 · pp. 18–19
  • Two-stage detectors had surpassed the best-performing detection DCNNs, though they could still leave false positives; RoI Align improved Faster R-CNN baselines on COCO.RoI Align uses bilinear interpolation at regularly sampled locations to reduce feature misalignment; Mask R-CNN also added a segmentation-loss branch, which improved detection performance.Source: §2.1.3 · p. 21

2.1.4 Cascades

  • With 3–4 stages and gradually increasing IoU thresholds, cascades improved detector quality by an average of 3 points over non-cascaded versions and featured in at least the first three 2018 COCO challenge teams.The threshold schedule addresses the loss of positive windows and poor-quality later-stage hypotheses caused by raising the IoU threshold too quickly.Source: §2.1.4 · pp. 22–23

2.1.5 Parts-Based Models

  • CNN features made DPM approaches more competitive: DeepPyramidDPM reached and slightly exceeded R-CNN, but later DPM variants fell short of fine-tuned R-CNN.Fast R-CNN combined with DeepID-Net’s deformation-based pooling improved over the state of the art, while deformable R-FCN remained less used than fixed-RoI-pooling counterparts.Source: §2.1.5 · pp. 23–24

2.2.1 Losses

  • Focal loss improved MS COCO performance by 3.2 mAP points over OHEM with a ResNet-50-FPN backbone and 600-pixel image scale.It addresses the imbalance between positive and negative anchors by penalizing easy examples less and hard examples more, whereas OHEM selects hard examples for gradient calculation.Source: §2.2.1 · pp. 25–26
  • Focal loss tackles anchor imbalance, while localization losses can directly optimize IoU; using localization confidence for NMS increased recall on MS COCO [158].The IoU-based loss can be unstable or have zero-gradient regions; hard bounds and a box-fitness term were proposed to stabilize it. Additional segmentation losses can also aid detection.Source: §2.2.1 · pp. 25–26

2.2.2 Hyper-Parameters

  • Increasing batch size from 256 to 8192 reduced training time from 29 hours to 1 hour while maintaining accuracy; batches of 32k brought training below 15 minutes.Training modern detectors often uses small batches because full-image tensors strain GPU memory; learning-rate warmup and linear scaling were also proposed to aid training.Source: §2.2.2 · pp. 26–27

2.2.3 Pre-Training

  • Pre-training on data similar to the detection task is a better starting point than random initialization, but minimizing object-scale differences can improve results while imposing memory and inference costs.A pyramid of images can align instance scales, but very small instances may require impractical upscaling, and multi-scale inference is slower than using each input image once.Source: §2.2.3 · p. 27

2.2.4 Data Augmentation

  • Augmentation strategies such as cropping, flips, and color adjustments expand training diversity, and Taylor and Nitschke [376] found cropping most influential in their benchmark.Cutout and Sample Pairing are described as promising but not widely adopted.Source: §2.2.4 · pp. 27–28
  • Because useful augmentation policies depend on the dataset, proposed automated methods search policies or generate transformed data, but these approaches were developed for classification, not detection.GAN-based augmentation may be less effective for detection because generating scenes with multiple objects in relevant backgrounds is harder than generating images with one dominant object.Source: §2.2.4 · p. 28

2.3 Inference

  • Greedy NMS iteratively keeps the highest-confidence box and suppresses boxes above an IoU threshold, but threshold needs vary by dataset and it can suppress nearby true positives.Sparse-object datasets generally need lower thresholds and crowded datasets higher ones; NMS can also retain poorly localized top-scoring boxes or fail to suppress false positives with insufficient overlap.Source: §2.3 · p. 29
  • Soft-NMS improved COCO performance by 1.1 mAP without extra training, while learned NMS that accounts for duplicate detections gained 0.8 mAP on COCO.Soft-NMS reduces neighboring boxes’ confidence as overlap increases instead of zeroing their scores; the learned approach handles duplicates in its loss and jointly processes neighboring detections.Source: Figure 10 · p. 29, §2.3 · p. 30

2.4 Concluding Remarks

  • Mainstream detectors have converged on fully convolutional pipelines, widely adopted anchors, and multiscale CNN features, while RoI-Pooling and related modules may not persist.Iterative regression of seed boxes shows some promise, and current research increasingly explores information flow, losses, and parameterizations.Source: §2.4 · pp. 30–31

3 Going Forward in Object Detection

This section surveys challenges in object detection, including scale and rotation variance, domain adaptation, localization, occlusion, and small-object detection. It also explores graph networks, adversarial training, and contextual information as possible directions, noting that CNN detectors fail in some challenging cases.

3.1.1 Scale Variance

  • Scale variance spans objects occupying 0.01% to 90% of an image, making a single feature map inadequate; proposed approaches include image pyramids, anchors, multi-scale features, and dilated convolutions.Image pyramids require multiple forward passes, while dense anchors create positive-negative imbalance and must be adapted to dataset object sizes; combining large and small receptive fields can support scale invariance.Source: §3.1.1 · pp. 32–33

3.1.2 Rotational Variance

  • Rotation variance remains incompletely solved: oriented-box and rotated pooling methods improved regular Faster R-CNN from 38.7% to 71.8% with additional tricks on MSRA, but no standard pipeline or evaluation protocol has emerged.Rotated datasets are uncommon, and even sophisticated pipelines are rotation invariant only to a certain extent.Source: §3.1.2 · pp. 33–34

3.1.3 Domain Adaptation

  • 56.1% AP was achieved when a Faster R-CNN trained on COCO was tested on KITTI car images, versus 83.7% using more similar images.Synthetic-data domain randomization with random textures, backgrounds, camera angles, and distractors gained almost one point of AP, but synthetic datasets cannot yet replace real data.Source: §3.1.3 · pp. 35–36

3.1.4 Object Localization

  • Localization is a major source of detection error, especially for small objects and stricter IoU requirements; proposed remedies include hierarchical features, box refinement, IoU-based losses, and weakly supervised localization.Weakly annotated-image methods have produced approximate locations comparable to fully supervised counterparts, and some approaches localize the full object extent rather than only an interior point.Source: §3.1.4 · pp. 36–37

3.1.5 Occlusions

  • Occlusion removes object information and harms localization; methods use object parts and deformable kernels, learned or softened non-maximum suppression, and visible-pixel or video-temporal context, but remain far from human-like inference.Training with occluded examples is constrained by data availability and is not foolproof.Source: §3.1.5 · pp. 37–38

3.1.6 Detecting Small Objects

  • 34.5% mAP on small objects versus 64.9% on large objects was reported by the 2017 COCO challenge winner, an ensemble of four detectors.Small objects are difficult because they carry less information, are easier to confuse with backgrounds, and require more precise localization; expected shape and size can help in domains such as logos or traffic signs.Source: §3.1.6 · p. 38
  • Small-object detection can use attention-guided high-resolution regions [95], shallow networks, context, or GAN-based super-resolution.Naive image upscaling can make images too large for GPU training; shallow-only detectors suit cases where instances are small, while mixed sizes benefit from fine and coarse feature maps. Context can be added to proposals, and GANs can enhance small-object representations.Source: §3.1.6 · pp. 38–39

3.2.1 Graph Networks

  • Graph networks and relational modules offer ways to model dependencies between objects, motivated by detector failures on perturbed COCO validation images.Wang et al. [406] introduced long-distance interactions, while the relation module incorporated object dependencies and adaptive duplicate removal inside a CNN.Source: §3.2.1 · pp. 39, 41

3.2.2 Adversarial Trainings

  • GAN-generated candidate boxes replaced Multiple Instance Learning in a state-of-the-art detector that was faster than all others by two orders of magnitude.Shen et al. used the GAN framework to generate boxes following the distribution of training-image boxes.Source: §3.2.2 · p. 41

3.2.3 Use of Contextual Information

  • Context can come from objects, surroundings, or image-wide cues, but it remains unclear which context method is best or whether detectors should reject unlikely situations.Approaches include spatial relationships, multi-scale features, and local or global image context.Source: §3.2.3 · pp. 42–43

3.3 Concluding Remarks

  • CNN detectors remain impressive but fail on domain adaptation, occlusions, rotations, and small objects; adversarial training and context are prominent proposed avenues beyond these limits.The survey then turns to more specific setups and less traditional problems or environments.Source: §3.3 · p. 43

4 Extending Object Detection

This section surveys extensions of object detection beyond standard 2D images, including video and 3D data and settings with limited annotations or compute, while excluding medical and hyperspectral imagery. It also discusses limitations across learning settings and data modalities and the need for detectors that are reliable in the wild, robust, interpretable, and adaptable beyond benchmark accuracy.

4.1.1 Object Detection in Videos

  • Temporal information improves video-detection mAP by +2.9% to +5.6%; approaches aggregate tubelet features and propagate sparse key-frame features.Video detection also faces motion blur, appearance changes, defocus, pose variation, and computational-efficiency challenges.Source: §4.1.1 · pp. 44–45

4.1.2 Object Detection in 3D Point Clouds

  • 3D detection spans monocular RGB geometry, raw point clouds, voxels, and projected views; multimodal fusion ranks among KITTI’s top methods, and superpixels enable novel-object discovery.The survey notes that 3D point clouds differ substantially from images in structure and information content.Source: §4.1.2 · pp. 45–46

4.2.1 Weakly Supervised Detection

  • Weakly supervised detection commonly uses multiple instance learning over candidate regions, while CNN heat maps and pooling strategies can provide rough object localization.Other approaches use tracked proposals, captions, action descriptions, user feedback, or annotations from related classes as additional supervision.Source: §4.2.1 · pp. 46–47
  • As of the survey’s writing, state-of-the-art WSOD on VOC2007 was 47.6% mAP, still below the 83.1% state-of-the-art with full annotations (without COCO pre-training).Source: §4.2.1 · p. 48

4.2.2 Few-shot Detection

  • Few-shot detectors can learn with three to five boxes per target class; iterative sample generation can make three to four annotations per class competitive with image-level supervision on PASCAL VOC.A separate approach transfers a detector from a source domain to a sparsely labeled target domain through regularization.Source: §4.2.2 · p. 48

4.2.3 Zero-shot Detection

  • Zero-shot detection localizes unseen, semantically related classes without training examples, but protocols vary and unseen objects may be treated as background during training.The section also notes that zero-shot methods can be limited to recognizing a single dominant object rather than all instances in an image.Source: §4.2.3 · p. 49

4.2.4 Fast and Low Power Detection

  • PeleeNet reaches 76.4% mAP on PASCAL VOC2007 and 22.4% mAP on MS COCO at 17.1 fps on iPhone 6s and 23.6 fps on iPhone 8.The section frames fast and low-power detection as a response to speed and hardware constraints, with a general trade-off between speed and performance.Source: §4.2.4 · pp. 49–50

4.3 Towards Versatile Object Detectors

  • Detectors must be assessed beyond limited annotated benchmarks, with robustness to unusual situations and the ability to evolve as key expectations.The survey frames this section as reviewing deep-learning methods against these expectations for behavior in the wild, where annotations are absent.Source: §4.3 · p. 50

4.2.4 Fast and Low Power Detection

  • Only RefineDet [460], DES [467], and STDN [475] were both real-time and above 80% mAP on VOC07; DES and STDN may have benefited from better hardware.Figure 20 compares VOC07 performance with inference speed on a TitanX GPU and indicates the real-time threshold; some related measurements used other devices.Source: §4.2.4 · p. 50, Figure 20 · p. 51

4.3.1 Interpretability and Robustness

  • Driving detectors must withstand physical attacks and weather, yet complex real-world conditions remain an open problem; synthetic data and multisensory inputs may help.The survey highlights changing environments, reflections, and differing traffic signs and rules, and notes that LiDAR and multiple cameras can disambiguate difficult situations.Source: §4.3.1 · p. 52
  • No object-detector interpretability method existed to the authors’ knowledge, despite proposed visualization approaches for classification.The survey describes interpretability as important for understanding and fixing failures, including for legal matters; classification approaches include gradient maps, occlusion analysis, guided backpropagation [355], and perturbation.Source: §4.3.1 · p. 52

4.3.2 Universal Detector, Lifelong Learning

  • Incremental learning can cause catastrophic forgetting as new classes are added; the survey identifies Shmelkov et al. as the only recent object-detection contribution it knew of, using distillation.Lifelong learning and never-ending learning aim to learn novel classes without supervision and improve over time, but remain underdeveloped.Source: §4.3.2 · pp. 52–53

4.4 Concluding Remarks

  • DCNNs remain immature in interpretability and trustworthiness, while object-detection research largely prioritizes static-benchmark performance.The survey notes video temporal-consistency and point-cloud sparsity and memory challenges, while weak supervision and few-image settings show promise through refinements to cost functions and building blocks rather than wholly new architectures.Source: §4.4 · p. 53

5 Conclusions

The survey reviews object detection since 2012, tracing how convolutional neural networks displaced earlier methods. It surveys current model families, active research directions, extensions, datasets, and leading methods.

5 Conclusions

  • Convolutional neural networks revolutionized image object detection, displacing the methods that previously dominated the field.The survey covers developments in the domain since 2012.Source: §5 · p. 53
  • Top-performing detectors cluster around single-stage methods such as SSD or YOLO and two-stage methods following Faster RCNN.The field remains active across graph networks, GANs, context, small objects, domain adaptation, and occlusions.Source: §5 · p. 53
  • The survey also covers extensions to video, 3D point clouds, and weak supervision, and lists public datasets with top-performing methods.The authors present the article as a resource for understanding recent progress and the broader field.Source: §5 · p. 53

A Datasets and Results

After reviewing influential object-detection ideas and literature, the article turns to datasets used to train and evaluate detectors. It discusses major datasets and associated state-of-the-art methods.

A Datasets and Results

  • The article turns to datasets for training and evaluating object detectors after reviewing influential ideas and literature.The passage marks this transition in the article.Source: A Datasets and Results · p. 91
  • Public datasets support measurement and comparison of detector performance and provide examples for learning object models.The passage notes that deep convolutional neural networks benefit from massive amounts of data.Source: A Datasets and Results · p. 91
  • The section discusses the main datasets in recent object-detection literature and presents state-of-the-art methods for each.Source: A Datasets and Results · p. 91

A.1 Classical Datasets with Common Objects

The review begins with datasets of everyday objects captured by consumer cameras, then turns to datasets dedicated to specific detection tasks.

A.1 Classical Datasets with Common Objects

  • Datasets of everyday objects captured by consumer cameras form the first category covered.Source: §A.1 · p. 91
  • The review identifies consumer-camera datasets as the most important category in the domain and one that attracts the largest part of the community.Source: §A.1 · p. 91
  • A subsequent section covers datasets for specific tasks, including face and pedestrian detection.Source: §A.1 · p. 91

A.1.1 Pascal-VOC

Pascal-VOC remains an influential test bed, although its small size has prompted concern about test-set overfitting and increased use of MS COCO. VOC07 and VOC12 results are reported with different training-data conventions and performance ranges.

A.1.1 Pascal-VOC

  • Pascal-VOC introduced 0.5-IoU-based metrics that became a de facto standard, but the dataset's small size has raised test-set overfitting concerns.The 2007 format has 20 classes, and MS COCO is now preferred by some for demonstrating new algorithms.Source: §A.1.1 · pp. 91–92
  • VOC07 results use VOC07+12 training, with reported mAP around 80 points and up to 86.3 AP when MS COCO data are added.The section reports displaying methods above 80 points and notes that results without COCO pretraining appear to plateau around 85 points.Source: §A.1.1 · p. 92
  • VOC12 is described as slightly harder; results use 07++12 training, and COCO pretraining is associated with 83.8 points.The section presents results above 75 points in Table 7.Source: §A.1.1 · p. 92
  • On both VOC splits, leading backbones generally exceed 100 layers, while approaches state of the art using VGG-16.Source: §A.1.1 · p. 92

A.1.2 MS COCO

MS COCO is presented as a challenging, widely used object-detection benchmark, with 80 categories and AP averaged across IoUs from 0.5 to 0.95. Reported leading approaches build on Faster R-CNN and combine techniques including FPN, large-batch training, and GCN.

A.1.2 MS COCO

  • MS COCO contains 80 object categories and 118,000 training images; its AP is calculated similarly to Pascal-VOC but averaged over multiple IoUs.It has become the de facto standard for the object-detection community.Source: §A.1.2 · p. 92
  • 50.5 mAP is reported for the best published single-model method, while its ensemble reaches 52.5.Ensembling Mask R-CNNs is reported at around 50.3 mAP; Deformable R-FCN reaches 48.5 mAP single-model and 50.4 with an ensemble.Source: §A.1.2 · p. 93
  • Faster R-CNN-based methods lead the reported progression, with results incorporating techniques such as FPN, large-batch training, and GCN.The first challenge-winning iteration is reported at 37.3 mAP with ResNet101; a later ensemble reached 41.5 mAP.Source: §A.1.2 · p. 93

A.1.3 ImageNet Detection Task

The ImageNet detection task covers 200 categories, while its classification-and-localization task covers 1,000 categories and asks for five labels and boxes. Reported results include a top 2017 detector with overall AP of 73.13 and a class-specific ensemble approach with 59.30 AP.

A.1.3 ImageNet Detection Task

  • ImageNet's detection task has 200 categories; its separate classification-and-localization task has 1,000 and permits five labels and five boxes.The latter setup avoids penalizing detection of objects present but absent from the ground truth.Source: §A.1.3 · p. 93
  • The 2017 top detector ranked first on 85 categories with overall AP of 73.13.The survey says it knows of no paper describing the approach precisely, though workshop slides were available.Source: §A.1.3 · p. 93
  • Bae et al. [8] selected the best detector for each object class, reporting 59.30 AP and wins in 10 categories.Source: §A.1.3 · p. 93

A.1.4 VisualGenome

Visual Genome pairs bounding boxes with scene graphs across a very large set of object categories, but its focus and evaluation make its status as an object-detection dataset unclear. Reported detection mAP is low, and one approach links detection with scene-graph generation and region captioning.

A.1.4 VisualGenome

  • Visual Genome contains over 100,000 images, more than 17,000 object categories, and an average of 21 objects per image.Its images include bounding boxes and complete scene graphs.Source: §A.1.4 · p. 93
  • Its status as an object-detection dataset is unclear because the paper lacks clear object-detection metrics and focuses on scene graphs and visual relationships.Source: §A.1.4 · p. 93
  • Li et al. [204] report 7.43 mAP by linking object detection, scene-graph generation, and region captioning; Faster R-CNN reports 6.72 mAP.The survey notes that reported mAP is much lower than on previous datasets, with many categories small and hard to detect.Source: §A.1.4 · p. 93

A.1.5 OpenImages

OpenImagesV4 was introduced as a large common-object detection dataset, while the review also situates common-object benchmarks alongside specialized industrial domains. The supplied COCO figure traces rising benchmark performance through 2018.

A.1.5 OpenImages

  • The 2018 COCO challenge’s last entry reached 53% mmAP, with the first two entries estimated at around 55% bbox mmAP.The figure describes a progression from Fast R-CNN, DeepMask, and Faster R-CNN in 2015 to Mask R-CNN-based pipelines and detection-specific backbones in 2018.Source: Figure 22 · p. 94
  • The dataset has 1,743,000 training images and more than 12,000,000 bounding boxes, averaging 7 objects per training image, plus 125,436 test images, including 41,620 for validation.Detection is measured by AP@0.5IoU averaged across classes while accounting for the class hierarchy. At the time of writing, the Inception ResNet Faster R-CNN baseline was 37 mAP, with no other published or non-published results reported.Source: §A.1.5 · p. 95
  • Industrial detection often targets categories and variations absent from VOC and MS-COCO.OpenImagesV4 was described as a common-object dataset spanning up to 500 classes.Source: §A.1.5 · pp. 93, 95

A.2 Specialized datasets

Specialized object-detection subfields and datasets have emerged in response to distinct industry applications.

A.2 Specialized datasets

  • Self-driving applications prompted pedestrian- and traffic-sign detection datasets.These examples are part of the specialized dataset domains described in the review.Source: §A.2 · p. 95
  • Traffic monitoring needs led to aerial-imagery datasets, while reading and translation applications led to text-detection datasets.Source: §A.2 · p. 95
  • Personalized-advertising applications prompted the construction of logo datasets.Source: §A.2 · p. 95

A.2.1 Aerial Imagery

Aerial-imagery detection datasets cover varied settings, from small vehicles to rotated-object benchmarks, and the review reports results alongside important data and comparability constraints.

A.2.1 Aerial Imagery

  • Google Earth has 30 Brussels images containing 1,319 small cars, and reports 94.6 AP as the best CNN result.Source: §A.2.1 · p. 95
  • 83% of F1 on both cars and trucks is reported for on the DLR 3k Munich Dataset, versus 82% for.Both methods provide oriented boxes; the dataset has 20 extra-large images, with 10 for training and 10 for testing.Source: §A.2.1 · p. 95
  • VeDAI comparisons are difficult because recent articles often use different metrics; xView has 60 classes and 1 million instances, with no results yet.VeDAI provides two image resolutions, 512x512 and 1024x1024, and its metric uses ellipse-based distance between ground-truth and detection centers.Source: §A.2.1 · pp. 95–96
  • COWC contains more than 32,000 cars and nearly 60,000 hard-negative patches, but unavailable test annotations prevent proper method testing.OIRDS, with 180 vehicles, is not widely used.Source: §A.2.1 · pp. 95–96
  • 76.2 oriented mAP is reported for the DOTA leaderboard winner, where Mask R-CNN structures are currently best.DOTA has 2,800 images, almost 200,000 instances, 15 categories, and rotated bounding boxes; the passage notes no other published method apart from.Source: §A.2.1 · p. 96
  • VisDrone contains 263 video sequences, 179,264 frames, 10,209 still images, and more than 2.5 million annotated boxes, but no results were available yet.Its images come from 14 Chinese cities and include varied weather, lighting, object density, and occlusion attributes.Source: §A.2.1 · p. 96

A.2.2 Text Detection in Images

The review inventories text-detection datasets and reports method results across scene-text benchmarks, including several datasets with distinct settings and metrics.

A.2.2 Text Detection in Images

  • The reviewed resources range from the 509 scene images in ICDAR 2003 to IIIT 5k-word’s 5,000 words and Synth90K’s 9 million generated images.ICDAR 2003 text is mostly centered and iconic; IIIT 5k-word includes street-scene and born-digital text, while Synth90K uses multiple fonts and a 90,000-word vocabulary.Source: §A.2.2 · pp. 96–97
  • On Street View Text, is among the best-performing methods, with a F-measure of 83%.The dataset contains 350 images and 725 instances.Source: §A.2.2 · p. 96
  • 79% F-measure is reported for [212] on MSRA-TD500, which contains indoor and outdoor natural images with Chinese and English text.The passage also reports F-measures of 77%, 76%, 75%, and 75% for,,, and, respectively.Source: §A.2.2 · p. 96
  • 87% of F measure on ICDAR 2015 is reported for Busta et al., compared with 83.8% for Liao et al. [212].Jiang et al. [159] is reported at 82.54%.Source: §A.2.2 · p. 97
  • The review identifies [212] as the only published result differing from the COCO Text dataset-paper baselines, and says [211] later outperformed.COCO Text has 63,000 images and 173,000 annotations.Source: §A.2.2 · p. 97
  • 67.0% of F measure is reported for Liao et al. [212] on the line-based, mostly Chinese-text RCTW-17 dataset.Source: §A.2.2 · p. 97

A.2.3 Face Detection

Face-detection benchmarks increasingly represent difficult conditions beyond frontal, high-resolution faces, including pose, occlusion, low resolution, blur, and unconstrained scenes.

A.2.3 Face Detection

  • FDDB contains 2,845 images and 5,171 faces spanning occlusion, strong pose changes, low resolution, and out-of-focus faces; Zhang et al. report 98.3% AUR.Najibi et al. report 98.1% AUR.Source: §A.2.3 · p. 97
  • AFLW emphasizes landmark detection and face alignment, with 25,993 faces in 21,997 real-world images and 21 landmarks annotated per face.Other surveyed datasets include AFW, PASCAL Faces, MALF, IJB benchmarks, and cartoon and violent-scene face data.Source: §A.2.3 · pp. 97–98
  • WIDER FACE reports performance by difficulty: Najibi et al. achieve 93.1% AP on easy, 92.1% on medium, and 84.5% on hard.The dataset includes annotations for scale, occlusion, pose, overall difficulty, and events; Zhang et al. report 92.8%, 91.3%, and 84.0% AP on the same sets.Source: §A.2.3 · p. 98
  • UFDD targets a gap between performance on existing challenges and real-world requirements by collecting face images with weather degradation, motion blur, and focus blur.Its authors also analyze detector results and failure cases.Source: §A.2.3 · pp. 97–98

A.2.4 Pedestrian Detection

The surveyed pedestrian benchmarks vary in setting and difficulty, and report detector results using metrics and subsets that differ across datasets.

A.2.4 Pedestrian Detection

  • Figure 23 plots image counts against face counts on a log scale, with bubble size representing average faces per image as an estimate of dataset complexity.The section also lists older and specialized pedestrian datasets, including MIT, GM-ATCI, Daimler DB, CVC-ADAS, USC, and TUD-Brussels.Source: §A.2.4 · pp. 98–100, Figure 23 · p. 99
  • On INRIA, Zhang et al. report 6.4% log average miss rate, ahead of at 6.9%; on difficult ETH, the reported rates are 24.5% and 30.2%, respectively.For ETH, the 24.5% result is from Zhang et al. trained on CityPersons, while is a boosted forest.Source: §A.2.4 · p. 99
  • On Caltech, Wang et al. report 4.0% log average miss rate, Zhang et al. 4.1%, and Mao et al. 5.5%.Caltech evaluation uses Pascal-VOC criteria at 0.5 IoU and averages miss rates at nine FPPI values from 10^-2 to 1.Source: §A.2.4 · pp. 99–100
  • On KITTI moderate, Chen et al. report 67.47 AP using stereo information for object proposals, while Li et al. report 65.01 AP with adapted Fast R-CNN.The moderate set excludes persons less than 25 pixels tall from ranking.Source: §A.2.4 · p. 100
  • CityPersons is described as more diverse and challenging than Caltech; on its reasonable set, Zhang et al. report 11.32% and Wang et al. 11.48% log average miss rate.The adapted Faster R-CNN baseline is 12.97%.Source: §A.2.4 · p. 100
  • On EuroCity's reasonable set, Faster R-CNN reports 8.1%, YOLOv3 8.5%, and R-FCN 12.1%; rankings differ on subsets with small or heavily occluded pedestrians.EuroCity contains 238,300 instances in 47,300 images across 31 cities in 12 European countries.Source: §A.2.4 · p. 100

A.2.5 Logo Detection

This appendix surveys datasets, benchmarks, and reported detector results across logos, traffic signs, other detection data, 3D, and video. It closes by noting that dataset and evaluation choices produce biased views of the world, while the survey provides a partial common information source.

A.2.5 Logo Detection

  • Logo detection pipelines are described as mostly direct applications of Faster R-CNN, while the OpenLogo challenge aggregates and refines seven datasets across 309 classes under an open detection protocol.The challenge assumes only a small proportion of logo classes are annotated; WebLogo-2M instead supplies noisy image-level labels and manually boxed test images.Source: §A.2.5 · pp. 100–101

A.2.6 Traffic Signs Detection

  • On the most challenging surveyed traffic-sign dataset, Tsinghua Tencent 100k, Faster R-CNN-like detectors report 44% precision and 68% recall.The section covers four main traffic-sign benchmarks and Bosch Small Traffic Lights.Source: §A.2.6 · p. 101

A.2.7 Other Datasets

  • The other-datasets subsection includes iNaturalist, with 859,000 images spanning more than 5,000 species and intended to encourage methods for in-the-wild, imbalanced, fine-grained categories.The subsection also introduces datasets usable with modalities discussed in Section 4.1.Source: §A.2.7 · p. 102

A.3 3D Datasets

  • KITTI is a widely used 3D point-cloud detection benchmark; for cars, method reports mAP of 81.94/71.88/66.38 on easy/moderate/hard at 10 fps, while [342] reports 67.72/64.00/63.01 at 50 fps.The benchmark uses 70% 3D box overlap for cars and 50% for pedestrians and cyclists.Source: §A.3 · pp. 102–103

A.4 Video Datasets

  • The video-dataset review includes YouTube-BoundingBoxes, where Real et al. report mAP 59, and ImageNet VID, where reports mAP 79.8 combining detection and tracking.Zhu et al. report 76.3 points with a flow-based approach on ImageNet VID.Source: §A.4 · p. 103

A.5 Concluding Remarks

  • The surveyed datasets each present a biased view through their image selection, category balance, metrics, and evaluation protocols; the survey partially addresses the need for a common dataset-information source.Finding one's way through the growing dataset landscape is described as difficult, especially for older datasets and exhaustive performance lists.Source: §A.5 · pp. 103–104
Loading 1809.03193v2…