Source-linked AI summary
Scale-Aware Trident Networks for Object Detection
Yanghao Li, Yuntao Chen, Naiyan Wang, Zhaoxiang Zhang
TL;DR
Object detection must handle substantial variation in object scale, while prior approaches do not provide efficient scale-specific features with uniform representational power. TridentNet uses shared-parameter branches with different receptive fields and scale-aware training, with a fast single-branch approximation. It achieves 48.4 mAP on COCO with a ResNet-101 backbone and improves over baseline methods without extra inference parameters or computations.
Problem
Scale variation is a central object-detection challenge, and the work targets limited uniform representational power across scales.
Method
TridentNet uses parallel branches with shared transformation parameters but different receptive fields, while scale-aware training assigns suitable object scales to each branch.
Results
48.4 mAP is achieved on COCO with a single TridentNet model using a ResNet-101 backbone.
Takeaways & Limitations
The fast inference approximation improves over baseline methods without extra parameters or computations.
Abstract
from arXiv · showhide
Scale variation is one of the key challenges in object detection. In this work, we first present a controlled experiment to investigate the effect of receptive fields for scale variation in object detection. Based on the findings from the exploration experiments, we propose a novel Trident Network (TridentNet) aiming to generate scale-specific feature maps with a uniform representational power. We construct a parallel multi-branch architecture in which each branch shares the same transformation parameters but with different receptive fields. Then, we adopt a scale-aware training scheme to specialize each branch by sampling object instances of proper scales for training. As a bonus, a fast approximation version of TridentNet could achieve significant improvements without any additional parameters and computational cost compared with the vanilla detector. On the COCO dataset, our TridentNet with ResNet-101 backbone achieves state-of-the-art single-model results of 48.4 mAP. Codes are available at https://git.io/fj5vR.
1. Introduction
Object detection struggles with objects spanning widely varying scales, while existing image and feature pyramids trade computational efficiency against uniform representational power. TridentNet addresses this trade-off with shared-parameter, scale-specific branches and a fast single-branch approximation.
- Scale variation impedes both one-stage and two-stage detectors, especially for very small or very large objects.
- Image pyramids process multiple image scales independently, whereas feature pyramids use different CNN layers to reduce computation.
- Feature pyramids sacrifice feature consistency across scales, increasing the risk of overfitting and reducing effective training data for each scale.
- TridentNet creates scale-specific feature maps using parallel branches with shared parameters and different receptive fields.
- TridentNet Fast uses one major branch during inference, adding no parameters or computational cost while retaining significant improvements over the vanilla detector.
- 48.4 mAP was achieved on COCO with a single TridentNet model using a ResNet-101 backbone.
2. Related Work
Related work addresses scale variation through image pyramids, multi-level feature representations, and receptive-field expansion. These approaches improve scale handling but involve inference costs, feature transformations, or specialized architectural choices.
- Image-pyramid methods improve detection across small and large objects but increase inference time, including under SNIP and SNIPER.
- Multi-level feature methods combine or predict from features at different spatial resolutions to handle objects of different scales.
- HyperNet and ION require normalization or transformation operators before fusing features from layers with different resolutions.
- Dilated convolution enlarges receptive fields through sparse sampling without additional cost, and this work applies different dilation rates across branches.
3. Investigation of Receptive Field
The controlled study isolates receptive-field effects by varying dilation rates in Faster R-CNN backbones. It finds that suitable receptive fields depend on object scale, motivating scale-adaptive detection branches.
- The study identifies receptive field as an isolated backbone factor whose impact on object detection had not previously been examined in controlled experiments.
- Different dilation rates are used to control receptive-field size by replacing selected backbone convolutions with dilated variants.
- The pilot experiment evaluates Faster R-CNN with ResNet-50 and ResNet-101 backbones on COCO while varying conv4 dilation rates from 1 to 3.
- Increasing receptive fields consistently reduces small-object performance but improves large-object performance on both backbones.
- The most suitable receptive field is strongly correlated with object scale, and larger dilation can still improve large-object detection beyond theoretical coverage.
- These findings motivate adapting receptive fields to object scales in the proposed Trident architecture.
4. Trident Network
TridentNet creates scale-specific feature maps from parallel branches that share transformation parameters but use different dilation rates and receptive fields. Scale-aware training assigns objects to branches by valid scale ranges, while TridentNet Fast reduces inference to one branch.
- Network Structure: TridentNet takes a single-scale image and creates scale-specific feature maps through parallel branches with shared convolution parameters and different dilation rates.The branches preserve a common transformation while providing different receptive fields.
- Network Structure: A trident block replaces convolution blocks with parallel residual blocks whose 3×3 convolutions use different dilation rates.For bottleneck ResNet blocks, the 1×1 convolutions remain part of each parallel residual branch.
- Weight Sharing: Shared weights prevent the multi-branch design from adding parameters and train the same transformation parameters on samples from different scale ranges.The associated RPN and R-CNN heads also share weights across branches.
- Scale-aware Training Scheme: Scale-aware training defines a valid range for each branch and selects only matching ground-truth boxes and proposals during RPN and R-CNN training.Invalid proposals are removed separately for each branch during R-CNN training.
- Inference: During inference, detections from all branches are filtered by their valid ranges and combined with NMS or soft-NMS.This produces the final detection results from multiple branch outputs.
- Inference and Approximation: TridentNet Fast uses only the middle branch during inference, incurring no additional time cost compared with a standard Faster R-CNN detector while suffering only a slight performance drop.The middle branch is used because its valid range covers both large and small objects.
5. Experiments
Experiments on COCO evaluate TridentNet’s components, architectural choices, branch behavior, and performance under standard and enhanced settings. The results support weight sharing, three branches, conv4 placement, and fast inference as effective design choices.
- Implementation Details: The COCO experiments train on trainval35k, evaluate on minival and test-dev, and report AP, AP50/AP75, and scale-specific APs.The standard evaluation separates small, medium, and large objects by size.
- Components of TridentNet: 39.0 AP versus 37.9 AP is achieved by the multi-branch design on ResNet-101, with a 2.3 increase for large objects.On ResNet-101-Deformable, AP rises from 39.9 to 40.5, with a 1.2 increase for large objects.
- Components of TridentNet: 0.8/1.0 AP increases for small objects result from scale-aware training on ResNet-101/ResNet-101-Deformable, but large-object performance drops.The authors conjecture that branch-specific training reduces extreme-scale samples and can cause overfitting.
- Components of TridentNet: Weight sharing consistently improves both base networks, reduces parameters, and alleviates scale-aware training’s overfitting issue by training shared parameters on all object scales.The shared parameters are used by all branches.
- Components of TridentNet: 2.7/1.9 AP increases are achieved by TridentNet on the two base networks, and the method remains compatible with deformable convolution.The results indicate that the proposed structure can be combined with adaptive receptive-field adjustment.
- Ablation Studies: 2.7 to 3.4 AP increases over the single-branch baseline are observed with one to four branches, while four branches do not improve over three.Three branches are selected as the default based on complexity and performance.
- Ablation Studies: Conv4 placement outperforms conv2 and conv3 because their strides are insufficient to create large receptive-field discrepancies between branches.The evaluated total strides are 4, 8, and 16 for conv2, conv3, and conv4.
- Ablation Studies: Branch specialization matches object scale: branch-1 performs well on small objects, branch-2 on medium objects, and branch-3 on large objects.Combining the three branches inherits their individual merits and achieves the best results.
6. Conclusion
The paper presents Trident Network as an object-detection method for building in-network scale-specific feature maps with uniform representational power. Its scale-aware multi-branch design specializes branches for corresponding object scales.
- Trident Network builds in-network scale-specific feature maps with uniform representational power.The conclusion presents this as the paper’s central method.
- Scale-aware training equips each branch of the multi-branch architecture with specialized ability for corresponding object scales.The conclusion describes this as the training scheme used with the architecture.
- Fast inference with the major branch improves over baseline methods without extra parameters or computations.The conclusion identifies this as the fast inference method’s practical benefit.