Source-linked AI summary
CBNet: A Novel Composite Backbone Network Architecture for Object Detection
Yudong Liu, Yongtao Wang, Siwei Wang, TingTing Liang, Qijie Zhao, Zhi Tang, Haibin Ling
TL;DR
CNN detectors depend heavily on backbone feature extraction, motivating a stronger backbone without designing a wholly new one. CBNet assembles identical pretrained backbones with stage-wise composite connections, and improves several detectors, reaching 53.3 mAP on COCO with a single model. Its main trade-off is slower inference from increased computation, partially addressed by an accelerated variant.
Problem
CNN detector performance depends strongly on backbone feature extraction, motivating a more powerful backbone built from existing backbones such as ResNet and ResNeXt.
Method
CBNet assembles multiple identical backbones using composite connections that feed preceding high-level stage outputs into succeeding backbones, with the final Lead Backbone producing detector features.
Results
CBNet consistently improves mAP for FPN, Mask R-CNN, and Cascade R-CNN by about 1.5 to 3.0 percent, with Cascade Mask R-CNN reaching 53.3 mAP on COCO using a single model.
Takeaways & Limitations
The architecture improves detection accuracy across several state-of-the-art detectors and also improves instance segmentation results.
Takeaways & Limitations
CBNet slows inference by increasing computation; its accelerated version raises speed from 5.5 fps to 6.9 fps while AP changes from 41.0 to 40.8.
Abstract
from arXiv · showhide
In existing CNN based detectors, the backbone network is a very important component for basic feature extraction, and the performance of the detectors highly depends on it. In this paper, we aim to achieve better detection performance by building a more powerful backbone from existing backbones like ResNet and ResNeXt. Specifically, we propose a novel strategy for assembling multiple identical backbones by composite connections between the adjacent backbones, to form a more powerful backbone named Composite Backbone Network (CBNet). In this way, CBNet iteratively feeds the output features of the previous backbone, namely high-level features, as part of input features to the succeeding backbone, in a stage-by-stage fashion, and finally the feature maps of the last backbone (named Lead Backbone) are used for object detection. We show that CBNet can be very easily integrated into most state-of-the-art detectors and significantly improve their performances. For example, it boosts the mAP of FPN, Mask R-CNN and Cascade R-CNN on the COCO dataset by about 1.5 to 3.0 percent. Meanwhile, experimental results show that the instance segmentation results can also be improved. Specially, by simply integrating the proposed CBNet into the baseline detector Cascade Mask R-CNN, we achieve a new state-of-the-art result on COCO dataset (mAP of 53.3) with single model, which demonstrates great effectiveness of the proposed CBNet architecture. Code will be made available on https://github.com/PKUbahuangliuhe/CBNet.
1 Introduction
CBNet addresses the importance of backbone feature extraction by assembling identical backbones with stage-wise composite connections. Applied to several detectors, it improves detection and instance segmentation, including a single-model COCO result of 53.3 mAP.
- A more representational backbone generally improves the performance of its host object detector.
- CBNet assembles Assistant Backbones and a Lead Backbone, feeding each Assistant Backbone’s stage outputs into the succeeding backbone as inputs.The final Lead Backbone feature maps are used for object detection, while the composite features combine high- and low-level information from multiple backbones.
- 1.5 to 3.0 percent: mAP increases consistently for FPN, Mask R-CNN, and Cascade R-CNN on MS-COCO after applying CBNet.Instance segmentation results also improve.
- 53.3 mAP: Cascade Mask R-CNN with Triple-ResNeXt152 CBNet achieves a new state-of-the-art COCO detection result using a single model.
- The work contributes a backbone-assembly method that improves multiple state-of-the-art detectors and reports a single-model MS-COCO mAP of 53.3.
2 Related work
Prior object detectors use increasingly powerful or task-specific backbones, yet substantial performance headroom remains. CBNet offers an alternative by assembling existing backbones rather than designing and pretraining an entirely new backbone.
- Object detection: State-of-the-art object detection comprises one-stage and two-stage methods, while the best publicly reported COCO mAP in the cited comparison is 52.5 from an ensemble of four detectors.
- Backbone for Object detection: CNN detector backbones extract basic features and are commonly ImageNet-pretrained, but classification-designed backbones may be suboptimal for object detection.DetNet and FishNet are task-specific alternatives that still require ImageNet pretraining.
- Recurrent Convolution Neural Network: Figure 2 compares CBNet with an unfolded recurrent convolutional network, highlighting differences in stage connections and parameter sharing.
- Backbone for Object detection: CBNet builds a more powerful backbone economically by assembling multiple identical existing backbones such as ResNet and ResNeXt.
3 Proposed method
CBNet assembles multiple identical backbones with composite connections so Assistant Backbones enhance a Lead Backbone stage by stage. The architecture supports several composition styles and attaches the Lead Backbone to standard detector components.
- 3.1 Architecture of CBNet: CBNet consists of K identical backbones, including Assistant Backbones and one Lead Backbone, connected to enhance feature extraction.K is at least 2; K=2 is called Dual-Backbone and K=3 Triple-Backbone.
- 3.1 Architecture of CBNet: Assistant Backbone outputs are fused into succeeding backbone inputs stage by stage, while Lead Backbone features are used for detection.The composite connection transforms previous-backbone features before adding them to the succeeding backbone's input feature maps.
- 3.1 Architecture of CBNet: Adjacent Higher-Level Composition feeds higher-level features from an adjacent Assistant Backbone into the corresponding Lead Backbone stage.Its composite connection uses 1×1 convolution, batch normalization, channel reduction, and upsampling.
- 3.2 Variants of CBNet: Other composite styles include Same Level Composition, Adjacent Lower-Level Composition, and Dense Higher-Level Composition.Dense Higher-Level Composition assembles features from all higher-level Assistant Backbone stages before adding them to the Lead Backbone's previous-stage output.
- 3.3 Detection Network with CBNet: CBNet can be applied to off-the-shelf object detectors by attaching the Lead Backbone to functional networks such as RPN and detection heads.The described integration does not require additional modifications to detector network architectures.
4 Experiments
Experiments on MS-COCO show that CBNet improves object detection and instance segmentation across several detector baselines. Additional comparisons indicate that adjacent higher-level composition is effective, while the architecture increases computation and inference cost.
- Detection results: CBNet consistently improves FPN, Mask R-CNN, and Cascade R-CNN baselines by 1.5 to 3 percent mAP on MS-COCO.The experiments compare baseline detectors with Dual-Backbone and Triple-Backbone variants.
- Detection results: 53.3 mAP is achieved on COCO test-dev by Cascade Mask R-CNN equipped with CBNet using a single model.The result requires no baseline improvement beyond replacing the backbone with CBNet.
- Detection results: CBNet also improves baseline performance for instance segmentation, alongside its gains on bounding-box detection.Table 2 reports both object detection and instance segmentation results across four detector types.
- Comparisons of different composite styles: SLC performs worse than the original baseline, whereas adding same-stage features creates redundancy and does not provide more semantic information.The authors attribute the degradation to underutilized parameters and increased training difficulty.
- Comparisons of different composite styles: AHLC is the only tested composite style with optimal detection improvement despite all styles using about twice the parameters of a single backbone.The comparisons include SLC, ALLC, ADLC, and AHLC under the Dual-Backbone architecture.
- Sharing weights for CBNet: Sharing Dual-Backbone weights yields 40.4 mAP versus 39.4 for the baseline, while unshared weights raise this only to 41.0 mAP.These results support the authors’ conclusion that composite architecture, rather than parameter growth, is the dominant source of improvement.
- Number of backbones: Detection mAP increases with the number of backbones and tends to converge at three, motivating Dual-Backbone and Triple-Backbone configurations.The recommendation considers speed and memory cost.
- An accelerated version of CBNet: The Dual-Backbone configuration raises FPN AP by 1.6 percent but reduces speed from 8.1 fps to 5.5 fps.Removing the two early Assistant Backbone stages increases speed to 6.9 fps while changing AP from 41.0 to 40.8.
5 Conclusion
CBNet is a composite backbone architecture that links identical backbones stage by stage, using the final Lead Backbone for detection. It improves detection and instance segmentation performance across several state-of-the-art detectors on COCO.
- CBNet links a series of same-structure backbones through composite connections, with each previous stage feeding the parallel succeeding stage.The final backbone, called the Lead Backbone, supplies feature maps for object detection.
- CBNet increases the COCO detection mAP of FPN, Mask R-CNN, and Cascade R-CNN by about 1.5 to 3 percent.
- CBNet also improves instance segmentation performance when integrated into state-of-the-art detectors.
- A single-model Cascade Mask R-CNN with CBNet achieves a new COCO state-of-the-art mAP of 53.3.