PaperlayerView exact revision on arXiv

Source-linked AI summary

DetNAS: Backbone Search for Object Detection

Yukang Chen, Tong Yang, Xiangyu Zhang, Gaofeng Meng, Xinyu Xiao, Jian Sun

arXiv:1903.10979v4cs.CV

TL;DR

Object detection often uses classification-designed backbones, despite task differences, while NAS backbone search is hindered by ImageNet pre-training requirements and costly candidate training. DetNAS uses a one-shot supernet trained first on ImageNet and then on detection data, followed by evolutionary architecture search. Detection-searched networks consistently outperform ImageNet-searched and hand-crafted alternatives across detectors and datasets, with fewer FLOPs.

  • Problem

    Classification-designed backbones may be sub-optimal for object detection, and NAS backbone search must reconcile target-task rewards with costly ImageNet pre-training.

  • Method

    DetNAS decouples supernet weight training from architecture optimization: ImageNet pre-training, detection fine-tuning, then evolutionary search on the trained supernet.

  • Results

    Detection-searched networks consistently outperform ImageNet-classification-searched networks across FPN and RetinaNet on COCO and VOC; DetNASNet gains 2.9% mmAP over ResNet-50 on COCO FPN with fewer FLOPs.

  • Takeaways & Limitations

    DetNAS provides an efficient detection-guided route to backbones that outperform hand-crafted networks across multiple detectors and datasets.

Abstract

from arXiv · show

Object detectors are usually equipped with backbone networks designed for image classification. It might be sub-optimal because of the gap between the tasks of image classification and object detection. In this work, we present DetNAS to use Neural Architecture Search (NAS) for the design of better backbones for object detection. It is non-trivial because detection training typically needs ImageNet pre-training while NAS systems require accuracies on the target detection task as supervisory signals. Based on the technique of one-shot supernet, which contains all possible networks in the search space, we propose a framework for backbone search on object detection. We train the supernet under the typical detector training schedule: ImageNet pre-training and detection fine-tuning. Then, the architecture search is performed on the trained supernet, using the detection task as the guidance. This framework makes NAS on backbones very efficient. In experiments, we show the effectiveness of DetNAS on various detectors, for instance, one-stage RetinaNet and the two-stage FPN. We empirically find that networks searched on object detection shows consistent superiority compared to those searched on ImageNet classification. The resulting architecture achieves superior performance than hand-crafted networks on COCO with much less FLOPs complexity.

1 Introduction

DetNAS addresses the mismatch between classification-designed backbones and object detection by decoupling supernet training from architecture search. Its detection-guided search produces efficient backbones that outperform hand-crafted and ImageNet-searched alternatives across detectors and datasets.

  • Motivation: Object detection can favor different backbones than ImageNet classification, while handcrafting detection backbones requires expert knowledge and tedious trials.ResNet-101 outperforms DetNet-59 on ImageNet classification but underperforms it on object detection.
  • Motivation: NAS backbone search is difficult because detection requires ImageNet pre-training, whereas NAS needs target-task rewards, making candidate-by-candidate training costly.Training from scratch requires more iterations and breaks down on small datasets such as PASCAL VOC.
  • Method: DetNAS decouples weight training and architecture optimization with a one-shot supernet, making pre-training economical and backbone search feasible.The framework avoids repeatedly pre-training and fine-tuning every candidate architecture.
  • Method: DetNAS uses a three-step pipeline: ImageNet supernet pre-training, detection-dataset fine-tuning, and evolutionary search on the trained supernet.The search is guided by the detection task after supernet fine-tuning.
  • Results: 2.9% better mmAP than ResNet-50 is achieved by DetNASNet on COCO with the FPN detector, while DetNASNet (3.8) exceeds ResNet-101 by 2.0%.These results are reported with much fewer FLOPs for the searched backbones.
  • Results: Detection-searched networks consistently outperform ImageNet-classification-searched networks by more than 3% on VOC and 1% on COCO across FPN and RetinaNet.DetNAS also reports large-margin gains over hand-crafted networks and meaningful structural patterns.

2 Related Work

Prior NAS research primarily targeted image classification, while related work extended NAS to other components or tasks. DetNAS targets the unresolved problem of searching backbones specifically for object detectors.

  • Detection NAS: Object detectors combine a backbone with a head, and prior detection advances often focused on head architecture, loss, and anchor design.FPN and RetinaNet are cited as representative object detectors.
  • NAS on image classification: Image-classification NAS has used reinforcement learning and evolutionary algorithms to determine neural architectures automatically.AmeobaNet showed that a basic evolutionary algorithm could achieve comparable or better results than reinforcement-learning methods.
  • NAS on other tasks: NAS had also been applied to semantic segmentation, including searches for decoder cells and segmentation backbones.Auto-DeepLab used a gradient-based method to search segmentation backbones.
  • Open problem: Before DetNAS, no work had attempted to search neural architectures for object-detector backbones, partly because ImageNet pre-training is costly.The passage identifies costly pre-training as a possible reason for this gap.

3 Detection Backbone Search

DetNAS addresses the difficulty of searching detection backbones by separating supernet weight training from architecture optimization. It pre-trains and fine-tunes a one-shot supernet, then searches its candidate paths with evolutionary optimization under detection guidance.

  • Motivation: DetNAS targets the mismatch between classification-designed backbones and detection-specific objectives while avoiding costly expert-crafted trials.Detection distinguishes locating each object instance from identifying an image’s main object, and backbone choice strongly affects detector performance.
  • Method: The framework decouples supernet weight training from architecture search, allowing candidate detection backbones to inherit shared weights.This avoids individually pre-training and fine-tuning every candidate architecture.
  • Method: DetNAS uses three stages: ImageNet supernet pre-training, detection-dataset supernet fine-tuning, and evolutionary search on the trained supernet.Path-wise training and detection-specific evaluation are used throughout the pipeline.
  • Search Space Design: The search space is based on ShuffleNetv2 blocks, with kernel-size and Xception-block alternatives across large and small configurations.The large space has 40 searchable blocks and approximately 1.2 × 10^24 candidates; the small space has approximately 1.0 × 10^12 candidates.
  • Search Algorithm: Evolutionary search is selected because it can stably satisfy hard FLOPs or inference-speed constraints.Architectures violating the constraint are removed during population-based search.

4 Experimental Settings

The experiments pre-train and fine-tune the supernet under fixed multi-GPU schedules, then reserve validation data for architecture search and held-out test data for final evaluation.

  • Supernet Pre-training: ImageNet supernet pre-training uses 1.28M training images, 8 GPUs, batch size 1024, and 300k iterations.The initial learning rate is 0.5 and decays linearly to zero.
  • Supernet Fine-tuning: Detection fine-tuning uses 8 GPUs with 16 images per minibatch for 90k COCO iterations and 22.5k VOC iterations.Images are resized so the shorter side is 800 pixels.
  • Architecture Search: Architecture search splits each detection dataset into supernet-training and validation subsets, using 5k validation images for both COCO and VOC.The validation subset guides evolutionary search, while remaining images support supernet fine-tuning.
  • Final Evaluation: Final architectures are retrained with the same pre-training and fine-tuning configuration before evaluation on COCO minival or VOC test2007.Performance is reported primarily with COCO mmAP and VOC IoU=.5 metrics under the Detectron “1x” setting.

5 Experimental Results

DetNAS consistently outperforms hand-crafted and ImageNet-searched backbones across detectors and datasets, while revealing detection-specific architectural patterns.

  • Main Results: 40.2% mmAP makes DetNASNet equal to ResNet-101 and superior to ResNet-50 with only 1.3G FLOPs on COCO-FPN.DetNASNet is searched in the large search space and evaluated with the FPN detector.
  • Main Results: DetNAS outperforms the matched ShuffleNetv2-40 baseline by 0.8% mmAP on COCO, separating search effectiveness from search-space effects.The comparison uses identical 40 blocks and 1.3G FLOPs.
  • Main Results: 42.0% mmAP lets DetNASNet (3.8) surpass ResNet-50 by 4.7% and ResNet-101 by 2.0% at identical 3.8G FLOPs to ResNet-50.The enlarged model increases DetNASNet's channels by 1.8 times.
  • Ablation Studies: DetNAS exceeds the matched ShuffleNetv2-20 by more than 1% mmAP on COCO with both FPN and RetinaNet detectors.The ablation search space uses models with similar FLOPs under 300M and evaluates COCO and VOC across FPN and RetinaNet.
  • Ablation Studies: ClsNASNet is best on ImageNet classification but performs disappointingly for object detection, whereas DetNAS and its from-scratch counterpart improve consistently over classification-oriented and hand-crafted baselines.DetNAS also outperforms DetNAS-scratch in all cases, reflecting the importance of ImageNet pre-training.
  • Architecture Analysis: Detection-searched architectures place large-kernel blocks in low-level layers and deep blocks in high-level layers, unlike ClsNASNet's placement of 7×7 blocks in later stages.These structural patterns are reported for the 20-block search space and include architectures searched with FPN and RetinaNet.

6 Conclusion

DetNAS searches object-detector backbones directly, evaluates the approach across detectors and datasets, and reveals architecture-level differences between classification- and detection-searched networks.

  • DetNAS is presented as the first attempt to search backbones in object detectors without a proxy.
  • The framework is evaluated with FPN and RetinaNet on COCO and VOC.
  • DetNAS and ClsNASNet exhibit different architecture-level patterns, providing insights for hand-crafted architecture design.

Supplementary Material

DetNAS trains a one-shot supernet through ImageNet pre-training and detection fine-tuning, then uses evolutionary search to select architectures under detection guidance. The searched networks outperform hand-crafted and from-scratch baselines while retaining practical computational and speed characteristics.

  • DetNAS searches a trained supernet with an evolutionary algorithm after ImageNet pre-training and detection fine-tuning.
  • The search algorithm uses a detection validation set, a small detection-training subset, evolution iterations, and a constraint to produce the best architecture.
  • In “2x” settings, DetNASNet and DetNASNet (3.8) remain superior to ResNet-50, ResNet-101, and ShuffleNetv2-40.
  • DetNASNet processes 5 more frames per second than ResNet-101, while DetNASNet (3.8) is only 2 FPS slower than ResNet-50 and has much better mAP.
Loading 1903.10979v4…