Source-linked AI summary
MMDetection: Open MMLab Detection Toolbox and Benchmark
Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, Dahua Lin
TL;DR
Object detection and instance segmentation need unified, high-quality tooling because complex pipelines and implementation choices can produce different results. MMDetection provides a modular toolbox and benchmark with extensive studies of methods, components, and hyper-parameters to support reimplementation and fair comparison.
Problem
Complex detection and segmentation pipelines lack a unified, high-quality codebase and benchmark, while implementation settings can substantially affect results.
Method
MMDetection uses a modular unified platform with reusable components, training and inference code, broad framework support, and benchmarking studies of methods and hyper-parameters.
Results
The toolbox covers popular detection frameworks, runs basic bounding-box and mask operations on GPUs, and reports extensive ablations of architectures, training strategies, and hyper-parameters.
Takeaways & Limitations
MMDetection offers a flexible basis for reimplementing existing detectors, developing new ones, and making fairer comparisons across methods and settings.
Abstract
from arXiv · showhide
We present MMDetection, an object detection toolbox that contains a rich set of object detection and instance segmentation methods as well as related components and modules. The toolbox started from a codebase of MMDet team who won the detection track of COCO Challenge 2018. It gradually evolves into a unified platform that covers many popular detection methods and contemporary modules. It not only includes training and inference codes, but also provides weights for more than 200 network models. We believe this toolbox is by far the most complete detection toolbox. In this paper, we introduce the various features of this toolbox. In addition, we also conduct a benchmarking study on different methods, components, and their hyper-parameters. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new detectors. Code and models are available at https://github.com/open-mmlab/mmdetection. The project is under active development and we will keep this document updated.
1. Introduction
The introduction presents MMDetection as a PyTorch-based, unified codebase and benchmark for object detection and instance segmentation, motivated by the complexity and implementation sensitivity of these tasks. It emphasizes modularity, broad framework support, GPU efficiency, and empirical guidance through benchmarks and ablations.
- Motivation: MMDetection is built as a PyTorch codebase for object detection and instance segmentation, whose complex pipelines and implementation settings can produce substantially different results.The toolbox aims to provide a high-quality codebase and unified benchmark for these fundamental computer vision tasks.
- Major features: Its modular design lets users construct customized detection frameworks by combining separate components.The framework is decomposed into different modules to support customization.
- Major features: MMDetection supports multiple popular and contemporary detection frameworks out of the box.The introduction points to Section 2 for the full list of supported frameworks.
- Major features: All basic bounding-box and mask operations run on GPUs, and MMDetection’s training speed is faster than or comparable to other codebases.The passage states this as the toolbox’s high-efficiency feature.
- Benchmarking and ablations: The paper reports benchmarks, best practices, and ablation studies covering hyperparameters, architectures, and training strategies to facilitate future research and method comparisons.The remaining sections introduce supported methods and features, present benchmark results, and show ablations on selected baselines.
2. Supported Frameworks
MMDetection provides high-quality implementations spanning popular object detection and instance segmentation methods. Its supported frameworks and features cover more methods and especially more recent methods than other codebases.
- Single-stage detectors: MMDetection includes classic and recent single-stage detectors, including SSD, RetinaNet, GHM, FCOS, and FSAF.These methods span simple single-stage architectures, Focal Loss, gradient harmonization, anchor-free detection, and feature selection.
- Two-stage detection and instance segmentation: It supports two-stage detectors and instance segmentation methods such as Fast R-CNN, Faster R-CNN, R-FCN, Mask R-CNN, Grid R-CNN, Mask Scoring R-CNN, and Double-Head R-CNN.The listed methods include proposal-based, end-to-end, fully convolutional, grid-guided, mask-scoring, and separate classification-localization head designs.
- Multi-stage methods: The toolbox also supports multi-stage methods, including Cascade R-CNN and Hybrid Task Cascade for object detection and instance segmentation.Cascade R-CNN is multi-stage, while Hybrid Task Cascade is multi-stage and multi-branch.
- Components and features: Supported components include mixed-precision training, Soft NMS, OHEM, deformable operators, training from scratch, feature-pyramid and context modules, attention, normalization, HRNet, and guided anchoring.The listed implementations cover FP16 training, hard-sample mining, deformable convolution and RoI pooling, alternative initialization, feature construction, global context, attention, normalization, high-resolution representations, and sparse arbitrary-shaped anchors.
3. Architecture
MMDetection organizes detectors around common components that can be assembled into single-stage and two-stage architectures. It also provides a unified, hook-based training pipeline designed for flexible customization across multiple computer vision tasks.
- Architecture: Backbones transform images into feature maps, while necks connect backbones and heads and refine or reconfigure those feature maps.Examples include ResNet-50 without its final fully connected layer as a backbone and Feature Pyramid Network as a neck.
- Architecture: RoIHeads take RoI features as input and produce RoI-wise predictions for bounding boxes and masks.The described RoIHead includes BBoxHead and MaskHead components.
- Architecture: The component abstractions support both single-stage and two-stage detectors, allowing new methods to be developed by assembling existing and newly created components.The framework illustrates these detector types through the defined abstractions.
- Training pipeline: A unified training pipeline with hooks supports object detection, image classification, and semantic segmentation.The same pipeline is designed to accommodate multiple computer vision tasks.
- Training pipeline: The pipeline iterates through training and optional validation epochs, with hooks enabling user-defined operations before or after specified run, epoch, and iteration stages.The minimum pipeline repeatedly forwards the model, while registered hooks customize behavior at defined timepoints.
4. Benchmarks
MMDetection benchmarks object detection and instance segmentation methods primarily on MS COCO 2017, reporting accuracy, inference speed, memory usage, and mixed-precision training comparisons. The evaluation spans multiple backbones, GPUs, and competing codebases.
- Main results: The main benchmark compares SSD, RetinaNet, Faster R-CNN, Mask R-CNN, Cascade R-CNN, Hybrid Task Cascade, and FCOS across four backbones.The backbones are ResNet-50, ResNet-101, ResNet-101-32x4d, and ResNeXt-101-64x4d; bbox/mask AP and inference speed are reported.
- Memory usage: MMDetection and maskrcnn-benchmark generally use similar and lower actual GPU memory than the other compared codebases.Detectron and SimpleDet report memory using different measurement tools, complicating direct comparisons.
- Inference speed on different GPUs: Inference speed is benchmarked for three models on TITAN X, TITAN Xp, TITAN V, GTX 1080 Ti, RTX 2080 Ti, and V100 GPUs.Differences in other server hardware mean the results provide a basic impression of speed rather than a fully controlled comparison.
- Mixed precision training: MMDetection supports mixed-precision training to reduce GPU memory and speed training while keeping performance almost unchanged.Results are compared with maskrcnn-benchmark and SimpleDet on the same V100 node; Detectron does not yet support mixed precision.
5. Extensive Studies
MMDetection’s extensive studies evaluate components and hyper-parameters under controlled settings to support fair comparisons and identify effective practices. Findings cover regression losses, normalization and batch normalization, training scales, and RPN hyper-parameters.
- Study methodology: Controlled evaluation of components and hyper-parameters is intended to improve fair comparisons across detection methods and settings.The studies use MMDetection to examine important components and hyper-parameters under consistent experimental conditions.
- Regression losses: 0.5% improvement results from increasing Smooth L1 Loss weight, while L1 Loss is 0.6% higher than Smooth L1 without tuning.Balanced L1 Loss achieves 0.3% higher mAP than L1 Loss for end-to-end Faster R-CNN.
- Batch normalization: With a longer learning-rate schedule, trainable BN affine weights outperform fixed weights by about 0.5%, so MMDetection defaults to eval = True and requires grad = True.Under a 1x schedule, fixing affine weights changes AP by only 0.1%.
- Normalization layers: Replacing the 2fc bbox head with 4conv1fc and adding normalization to FPN and bbox/mask heads improves performance by around 1.5%.Simply replacing backbone BN with FrozenBN, SyncBN, or GN yields similar performance, while adding SyncBN or GN to FPN and heads gives no further gain.
- Training scales: Using [640 : 960] instead of [640 : 800] raises bbox AP by 0.4% and mask AP by 0.5%, whereas a minimum scale of 480 performs no better.MMDetection adopts 1333 × 800 as its default training scale, and the experiments use a 2x learning-rate schedule.
- RPN hyper-parameters: Relaxing RPN’s allowed-border rule improves AR from 57.1% to 57.7% by retaining anchors beyond image boundaries.A smaller Smooth L1 beta may slightly improve RPN average recall, while MMDetection retains Detectron settings by default for reference.
A. Detailed Results
This section presents detailed benchmarking results for detection methods on COCO val2017, with Table 10 reporting box and mask mean average precision. It also clarifies backbone notation and the differing convolution strides in pytorch-style and caffe-style ResNet-50 bottleneck blocks.
- Backbone notation: R-50 and R-50 (c) denote pytorch-style and caffe-style ResNet-50 backbones, respectively.The notation distinguishes the two backbone implementations used in the detailed benchmarks.
- Backbone notation: Pytorch-style ResNet uses 1x1 stride-1 followed by 3x3 stride-2 convolution, whereas caffe-style ResNet reverses those stride assignments.The difference occurs in the bottleneck residual block.
- Benchmarking results: Table 10 reports results for different detection methods on COCO val2017 using box mAP (APb) and mask mAP (APm).The table is continued on the following page.