Source-linked AI summary
RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free
Cheng-Yang Fu, Mykhailo Shvets, Alexander C. Berg
TL;DR
Single-shot detectors are widely used in embedded applications but had fallen behind two-stage detectors in the speed-versus-accuracy trade-off. RetinaMask improves RetinaNet through auxiliary mask prediction, adaptive loss training, and harder examples while keeping inference cost unchanged, and its final model reports better accuracy.
Problem
Single-shot detectors remain important for resource-constrained applications, yet two-stage detectors had surpassed RetinaNet on the speed-versus-accuracy trade-off.
Method
RetinaMask improves RetinaNet training with an instance-mask head, a self-adjusting loss, and more difficult positive examples while leaving the test-time detector structure unchanged.
Results
RetinaMask's final model shows better accuracy without any network architecture change during inference.
Takeaways & Limitations
The training improvements are intended to apply across a wide range of single-shot detector implementations while preserving their inference cost.
Takeaways & Limitations
Evaluation of newer architectures such as NASNET, MobileNet, and ShuffleNet is beyond the scope of this work.
Abstract
from arXiv · showhide
Recently two-stage detectors have surged ahead of single-shot detectors in the accuracy-vs-speed trade-off. Nevertheless single-shot detectors are immensely popular in embedded vision applications. This paper brings single-shot detectors up to the same level as current two-stage techniques. We do this by improving training for the state-of-the-art single-shot detector, RetinaNet, in three ways: integrating instance mask prediction for the first time, making the loss function adaptive and more stable, and including additional hard examples in training. We call the resulting augmented network RetinaMask. The detection component of RetinaMask has the same computational cost as the original RetinaNet, but is more accurate. COCO test-dev results are up to 41.4 mAP for RetinaMask-101 vs 39.1mAP for RetinaNet-101, while the runtime is the same during evaluation. Adding Group Normalization increases the performance of RetinaMask-101 to 41.7 mAP. Code is at:https://github.com/chengyangfu/retinamask
1. Introduction
RetinaMask improves RetinaNet training with instance-mask prediction, an adaptive loss, and harder positive examples, raising single-shot accuracy while preserving inference cost.
- 1. Introduction: Single-shot detectors remain important for embedded vision, self-driving cars, and mobile devices, but two-stage detectors had surpassed RetinaNet in speed-versus-accuracy.The paper targets this gap without changing the inference structure.
- 1. Introduction: Figure 1 compares RetinaMask, RetinaNet, and Mask R-CNN on COCO test-dev using accuracy-versus-inference-time plots across multiple resolutions.RetinaMask models use 400–800 resolutions without multi-scale training augmentation.
- 1. Introduction: The detection component retains exactly the original RetinaNet computational cost at test time despite the additional training tasks.This preserves applicability to embedded systems that favor single-shot detectors.
- 1. Introduction: RetinaMask applies three training improvements—instance-mask prediction, a self-adjusting loss, and more low-overlap positive examples—to improve RetinaNet accuracy.The detector structure remains unchanged during evaluation.
- 1. Introduction: The ablations analyze each contribution, and their combination brings RetinaNet back to state-of-the-art accuracy.The paper attributes the overall boost to the combined training changes.
2. Related Work
The related work surveys two-stage and single-shot detection, general detection improvements, and instance-mask integration, including multitask approaches that add segmentation information to detection.
- 2. Related Work: The review covers two-stage detectors, single-shot detectors, general detection improvements, and instance segmentation.It situates RetinaMask among methods that combine detection with richer visual prediction tasks.
- 2. Related Work: BlitzNet augments SSD with a semantic-segmentation branch, illustrating how additional training information can improve detection accuracy.This provides related precedent for using auxiliary prediction tasks during detector training.
- 2. Related Work: Instance segmentation extends detection by requiring a pixel-level mask for each object in addition to a tight bounding box.COCO established a recognized benchmark for this task beginning in 2015.
3. Model
RetinaMask extends RetinaNet training with best matching, a self-adjusting regression loss, and an instance mask module while retaining the original detector structure at inference.
- 3. Model: RetinaMask adds best matching, Self-Adjusting Smooth L1, and mask prediction to a RetinaNet baseline.The baseline is rebuilt in PyTorch before these modifications are introduced.
- 3.1. Best Matching Policy: Any positive-overlap best-matching anchor can be assigned to previously unmatched ground-truth objects, improving accuracy in the ablation.The regular positive threshold remains 0.5, while the relaxed policy addresses outlier aspect ratios.
- 3.2. Self-Adjusting Smooth L1 Loss: Self-Adjusting Smooth L1 computes its control point from running absolute-loss statistics and clips it to [0, β̂].Running minibatch mean and variance use momentum=0.9, and the control point is μ_R − σ_R^2 before clipping.
- 3.3. Mask Prediction Module: Mask proposals are assigned to FPN layers by detection size: smaller objects use P3, intermediate objects P4, and larger objects P5.Bounding-box prediction uses P3 through P7, whereas mask prediction uses P3 through P5.
- 3.4. Training: Training combines classification, box-regression, and mask losses after selecting top predicted boxes and adding ground-truth boxes to the mask proposals.The final loss is Loss_boxCls + Loss_boxReg + Loss_mask.
4. Experiments
Experiments evaluate RetinaMask through component ablations, qualitative comparisons, and COCO comparisons against RetinaNet and Mask R-CNN. The results show improved detection accuracy with unchanged inference architecture, while speed comparisons require caution because implementations use different hardware and software.
- Ablation Study: Best Matching anchors with any positive overlap produce the best performance and qualitatively reduce duplicate detections by tightening multiple anchors around each ground-truth object.The proposed matching strategy can add positive examples beyond the regular overlap threshold.
- Ablation Study: Self-Adjusting Smooth L1 achieves the best results for every reported metric and remains comparatively robust when its control parameter changes from 0.11 to 1.0.Sharing running statistics across channels performs slightly worse, at 36.4 mAP.
- Ablation Study: Adding mask prediction improves detection by 0.7 mAP with {P3, P4, P5} and by 0.9 mAP with a 1.5x training schedule, although it requires longer training.Using {P2, P3, P4, P5} slightly improves mask prediction but worsens detection.
- Comparison to RetinaNet: RetinaMask improves most tested classes, including hair drier from 0.9 to 7.1 mAP, while toaster decreases from 28.9 to 21.0 mAP with only 9 validation objects.The supplied qualitative figure highlights improved recall for skis and fewer duplicate detections for tie, but also shows a toaster failure case.
- Comparison to RetinaNet: The model’s speed numbers are not directly comparable because RetinaMask and RetinaNet use different GPU architectures and software implementations, and the short-side-400 implementation is unusually slow.The paper attributes the short-side-400 slowdown to library idiosyncrasies while reporting accuracy improvements at other resolutions.
- Comparison to state-of-the-art methods: 2.6 mAP improvement over RetinaNet is reported for the ResNet-101-FPN model on COCO test-dev, while the table also reports a 3.5 mAP improvement over Mask R-CNN.The supplied comparison table summarizes the headline benchmark differences; the cited prose also states that RetinaMask is more accurate across backbone and resolution combinations.
- Comparison with Mask R-CNN on instance mask prediction: RetinaMask remains about 1.2 mAP behind Mask R-CNN on instance mask prediction despite using the same ResNet-101-FPN backbone family.The comparison uses similarly trained models.
5. Conclusion
The conclusion presents RetinaMask as a more accurate single-shot detector obtained through three training components, without changing the inference architecture. It also identifies the Self-Adjusting Smooth L1 loss as potentially applicable beyond detection and instance segmentation.
- 5. Conclusion: RetinaMask combines three training components to improve single-shot detection accuracy without changing the network architecture during inference.The conclusion states that ablations show improvements for each module.
- 5. Conclusion: Self-Adjusting Smooth L1 is presented as applicable beyond object detection and instance segmentation.