PaperlayerView exact revision on arXiv

Source-linked AI summary

Perceptual Generative Adversarial Networks for Small Object Detection

Jianan Li, Xiaodan Liang, Yunchao Wei, Tingfa Xu, Jiashi Feng, Shuicheng Yan

arXiv:1706.05274v2cs.CV

TL;DR

Small objects are difficult to detect because their low-resolution, poor-quality appearance makes discriminative representations hard to learn. Perceptual GAN generates super-resolved representations with adversarial generator–discriminator training, outperforming prior methods on traffic-sign and pedestrian detection, including a 9.48% log-average miss rate.

  • Problem

    Small-object detection remains difficult because low-resolution, poor-quality appearance makes rich discriminative representations hard to learn, while effective solutions remain rare.

  • Method

    Perceptual GAN uses a generator and perceptual discriminator to transform small-object features into super-resolved, large-object-like representations for detection.

  • Results

    Perceptual GAN outperforms prior methods on traffic-sign and pedestrian detection, achieving a lowest log-average miss rate of 9.48% on pedestrian detection.

  • Takeaways & Limitations

    The results support Perceptual GAN as an effective approach for improving small-object detection across traffic-sign, pedestrian, and selected general object categories.

Abstract

from arXiv · show

Detecting small objects is notoriously challenging due to their low resolution and noisy representation. Existing object detection pipelines usually detect small objects through learning representations of all the objects at multiple scales. However, the performance gain of such ad hoc architectures is usually limited to pay off the computational cost. In this work, we address the small object detection problem by developing a single architecture that internally lifts representations of small objects to "super-resolved" ones, achieving similar characteristics as large objects and thus more discriminative for detection. For this purpose, we propose a new Perceptual Generative Adversarial Network (Perceptual GAN) model that improves small object detection through narrowing representation difference of small objects from the large ones. Specifically, its generator learns to transfer perceived poor representations of the small objects to super-resolved ones that are similar enough to real large objects to fool a competing discriminator. Meanwhile its discriminator competes with the generator to identify the generated representation and imposes an additional perceptual requirement - generated representations of small objects must be beneficial for detection purpose - on the generator. Extensive evaluations on the challenging Tsinghua-Tencent 100K and the Caltech benchmark well demonstrate the superiority of Perceptual GAN in detecting small objects, including traffic signs and pedestrians, over well-established state-of-the-arts.

1. Introduction

Small-object detectors struggle because low-resolution representations lack the discriminative features available for large objects. Perceptual GAN addresses this gap by generating super-resolved small-object representations and evaluating them on traffic-sign and pedestrian detection benchmarks.

  • Motivation: Small objects are difficult to detect because their low-resolution representations lack the rich, discriminative features available for large objects.This limitation affects applications including traffic-sign and pedestrian detection.
  • Motivation: Existing solutions enlarge input images or construct multi-scale representations, but small-object detection remains challenging and effective solutions are rare.These approaches enhance resolution or combine features from different network layers.
  • Method: Perceptual GAN transforms poor small-object features into super-resolved representations resembling large-object features through a generator and perceptual discriminator.The method exploits structural correlations between objects at different scales during network learning.
  • Method: The generator learns additive residual representations and is trained to fool the discriminator while improving detection, whereas the discriminator supplies supervision beyond distinguishing fake and real features.The networks are optimized alternately to solve the min-max problem.
  • Results: Perceptual GAN achieves state-of-the-art performance for traffic-sign and pedestrian detection on the Tsinghua-Tencent 100K and Caltech benchmarks.The paper reports large improvements over state-of-the-art methods on both challenging datasets.

2. Related Work

Related work spans traffic-sign and pedestrian detection, where CNN-based and deep-learning methods have recently gained prominence, as well as GAN applications in image generation, translation, representation learning, and super-resolution.

  • Traffic Sign Detection: CNN-based approaches have been widely adopted for traffic sign detection and classification because of their high accuracy.Earlier methods also addressed traffic-sign detection using traditional approaches.
  • Pedestrian Detection: Hand-crafted features such as Integral Channel Features and Aggregated Channel Features remain prominent in pedestrian detectors, while deep learning has greatly boosted performance.The cited examples are associated with Dollár et al.’s pedestrian-detection work.
  • Generative Adversarial Networks: GANs have been applied to image generation, style transfer, inpainting, unsupervised representation learning, and image super-resolution.The related work presents GANs as a framework for learning generative models and notes these application areas.
  • Perceptual GAN: The Perceptual GAN training procedure first trains the discriminator’s perception branch on large objects, then trains the generator on small objects to produce large-object-like representations.The figure describes separate training images containing only large or only small objects.

3. Perceptual GANs

Perceptual GANs transform small-object representations into super-resolved representations using a conditional residual generator. A two-branch discriminator combines adversarial realism with detection-oriented perceptual supervision to guide this transformation.

  • Global Optimization: The generator maps small-object features Fs to super-resolved representations G(Fs) that approximate large-object features Fl.It uses auxiliary low-level feature information and learns the residual between large- and small-object representations.
  • Discriminator Network: The discriminator has adversarial and perception branches that distinguish generated features from large-object features and assess their detection benefit.The perception branch produces classification and bounding-box regression losses, while the adversarial branch evaluates representation realism.
  • Training Procedure: Training first learns convolutional layers and the perception branch from large objects, then trains the generator and adversarial branch using small objects.This staged procedure uses the learned perceptual branch to guide small-object representation super-resolution.
  • Generator Network: The generator is a deep residual network that restores fine-grained details absent from small-object representations.It processes lower-level features through 3 × 3 and 1 × 1 convolutions, followed by B residual blocks.
  • Loss Functions: The discriminator loss is a weighted sum of adversarial and perceptual losses, encouraging generated representations with both large-object similarity and high detection accuracy.The loss is defined as Ldis = w1 × Ldis a + w2 × Ldis p.

4. Experiments · 4.1. Datasets and Evaluation Metrics

The experiments evaluate small-object detection on the Tsinghua-Tencent 100K traffic-sign benchmark and the Caltech pedestrian benchmark. Evaluation uses object-size-specific COCO metrics for traffic signs and log-average Miss Rate for small pedestrians.

  • 4.1.1 Traffic-sign Detection Datasets: Tsinghua-Tencent 100K contains 30,000 traffic-sign instances in 2,048×2,048 images.The benchmark is used for traffic-sign detection.
  • 4.1.1 Traffic-sign Detection Datasets: After excluding classes with fewer than 100 instances, the traffic-sign benchmark retains 45 classes.This follows the dataset protocol in.
  • 4.1.1 Traffic-sign Detection Datasets: Traffic-sign performance is evaluated using the same detection metrics as the Microsoft COCO benchmark.The evaluation reports performance across different object sizes.
  • 4.1.1 Traffic-sign Detection Datasets: Traffic signs are divided into small objects (area < 32×32 pixels), medium objects (32 × 32 < area < 96 × 96), and large objects (area > 96 × 96).The corresponding instance counts are 3270, 3829, and 599, respectively.
  • 4.1.2 Pedestrian Detection Datasets: Caltech provides about 250,000 frames, 350,000 annotated bounding boxes, and 2,300 unique pedestrians.The training data are densely sampled using every 4th frame.
  • 4.1.2 Pedestrian Detection Datasets: Caltech evaluation considers pedestrians over 50 pixels tall with no or partial occlusion, which are often very small.Performance is measured using log-average Miss Rate.

4.2. Implementation Details

The model is initialized from pretrained VGG backbones, with newly added layers using Xavier initialization, and receives resized inputs for traffic-sign and pedestrian detection. Training uses momentum SGD with fixed proposal sampling and runs on a single 12GB NVIDIA GeForce GTX TITAN X GPU.

  • Network initialization: Traffic-sign detection initializes from VGG-CNN-M-1024, while pedestrian detection initializes from VGG-16.Both backbones follow prior implementations cited in the paper.
  • Network initialization: New generator and discriminator convolutional and fully connected layers use Xavier initialization.The initialization applies to parameters of the newly added layers.
  • Optimization and sampling: Training uses SGD with momentum 0.9 and weight decay 0.0005 on a single NVIDIA GeForce GTX TITAN X GPU with 12GB memory.Generator mini-batches contain 128 selected object proposals per training image, with 25% foreground proposals meeting at least 0.5 IoU.

4.4. Ablation Studies

The ablation studies evaluate Perceptual GAN components and show that generated super-resolved representations, adversarial alternative optimization, and lower-level features improve small-object detection. Visualizations further indicate that the generator transfers poor small-object representations toward super-resolved features.

  • Feature enhancement: Feature-enhancement comparisons evaluate generated super-resolved representations against skip pooling, increased input scales, and multi-scale inputs using end-to-end training.All variants are implemented on the base convolutional layers and perceptual branch.
  • Feature visualization: The generator successfully transfers poor small-object representations into super-resolved features that resemble representations of large objects.Figure 6 compares original small- and large-object features with learned residual and generated features.
  • Adversarial optimization: Alternative optimization produces considerable improvements in recall and accuracy for small-object detection compared with end-to-end generator training without the alternating step.The alternating procedure recursively improves the generator and discriminator through adversarial training.
  • Lower-level features: Using higher-layer representations from Conv2 or Conv3 to learn the generator consistently decreases performance relative to using Conv1 features.The generator therefore uses Conv1 features to learn fine-grained details of small objects.

4.5. Discussion on General Small Object Detection

The proposed generator is evaluated for generalization on diverse object categories using a VOC detection pipeline trained end-to-end. On challenging classes with many small instances, it achieves reported AP values of 69.4%, 60.2%, 57.9%, and 41.8%.

  • 4.5. Discussion on General Small Object Detection: The detection pipeline is trained end-to-end with the proposed generator on the union of PASCAL VOC 2007 and VOC 2012 trainval sets, then evaluated on VOC 2007 test data.Evaluation focuses on boat, bottle, chair, and plant, identified as the most challenging classes because small instances are most common.
  • 4.5. Discussion on General Small Object Detection: The method achieves 69.4%, 60.2%, 57.9% and 41.8% in Average Precision (AP) for the evaluated classes.The passage presents these four AP values in the order boat, bottle, chair, and plant.

5. Conclusion

The paper proposes Perceptual GAN for small object detection, generating super-resolved small-object representations through adversarially updated generator and discriminator networks. Its generator learns residual fine-grained details from lower-level layers to make small-object representations approach those of large objects.

  • 5. Conclusion: Perceptual GAN addresses small object detection by generating super-resolved representations that boost detection performance.The approach leverages repeatedly updated generator and discriminator networks.
  • 5. Conclusion: The generator learns residual representations from fine-grained details in lower-level layers to enhance small-object representations toward large-object representations.
Loading 1706.05274v2…