Source-linked AI summary
CenterMask : Real-Time Anchor-Free Instance Segmentation
Youngwan Lee, Jongyoul Park
TL;DR
Instance segmentation needs better real-time accuracy, as YOLACT remains below Mask R-CNN despite its speed. CenterMask combines FCOS with a spatial attention-guided mask branch, scale-adaptive RoI assignment, and VoVNetV2; it reports state-of-the-art performance at real-time speed, including 33.4% mask AP and 38.0% box AP at over 35fps for CenterMask-Lite. A remaining limitation is weaker small-object performance than Mask R-CNN with the same ResNet-101 backbone.
Problem
Real-time one-stage instance segmentation, such as YOLACT, has a significant accuracy gap from Mask R-CNN, while anchor-based detectors add sensitivity, imbalance, and computational cost.
Method
CenterMask adds a spatial attention-guided mask branch and scale-adaptive RoI assignment to anchor-free FCOS, while VoVNetV2 uses residual connections and effective Squeeze-Excitation.
Results
CenterMask achieves state-of-the-art performance at real-time speed; CenterMask-Lite reaches 33.4% mask AP and 38.0% box AP at over 35fps on Titan Xp.
Takeaways & Limitations
CenterMask is presented as a baseline for real-time instance segmentation, and VoVNetV2 as a strong, efficient backbone for various vision tasks.
Takeaways & Limitations
With the same ResNet-101 backbone, Mask R-CNN performs better than CenterMask on small objects, possibly because it uses finer P2 features instead of CenterMask's P3 features.
Abstract
from arXiv · showhide
We propose a simple yet efficient anchor-free instance segmentation, called CenterMask, that adds a novel spatial attention-guided mask (SAG-Mask) branch to anchor-free one stage object detector (FCOS) in the same vein with Mask R-CNN. Plugged into the FCOS object detector, the SAG-Mask branch predicts a segmentation mask on each box with the spatial attention map that helps to focus on informative pixels and suppress noise. We also present an improved backbone networks, VoVNetV2, with two effective strategies: (1) residual connection for alleviating the optimization problem of larger VoVNet \cite{lee2019energy} and (2) effective Squeeze-Excitation (eSE) dealing with the channel information loss problem of original SE. With SAG-Mask and VoVNetV2, we deign CenterMask and CenterMask-Lite that are targeted to large and small models, respectively. Using the same ResNet-101-FPN backbone, CenterMask achieves 38.3%, surpassing all previous state-of-the-art methods while at a much faster speed. CenterMask-Lite also outperforms the state-of-the-art by large margins at over 35fps on Titan Xp. We hope that CenterMask and VoVNetV2 can serve as a solid baseline of real-time instance segmentation and backbone network for various vision tasks, respectively. The Code is available at https://github.com/youngwanLEE/CenterMask.
1. Introduction
CenterMask targets the accuracy–speed gap in instance segmentation with an anchor-free, one-stage design built on FCOS. It combines a spatial attention-guided mask branch with scale-adaptive RoI assignment and VoVNetV2 backbone improvements.
- Motivation: CenterMask addresses the significant accuracy gap between real-time YOLACT and Mask R-CNN while targeting both accuracy and speed.YOLACT offers real-time one-stage segmentation, but its accuracy remains below Mask R-CNN.
- CenterMask: CenterMask adds a spatial attention-guided mask branch to FCOS, predicting masks inside detected boxes while focusing on meaningful pixels and suppressing uninformative ones.The SAG-Mask branch follows FCOS box predictions and applies a spatial attention module to the mask features.
- CenterMask: A scale-adaptive RoI assignment function adjusts mask-feature selection using input and RoI scales, addressing fixed assignments designed for two-stage detectors.The proposed assignment is intended for FCOS's P3–P7 feature levels and avoids a hard-coded canonical image size.
- VoVNetV2: VoVNetV2 improves larger VoVNet models with residual connections that ease optimization and effective Squeeze-Excitation that preserves channel information.The eSE module replaces the original SE design's two fully connected layers with one channel-preserving layer.
- Results: CenterMask and CenterMask-Lite target large and small models, respectively, and CenterMask-Lite reaches 33.4% mask AP and 38.0% box AP at over 35fps on Titan Xp.Using the same ResNet-101 backbone, CenterMask is reported to outperform previous state-of-the-art single models at faster speed.
2. CenterMask
CenterMask builds anchor-free instance segmentation on FCOS by combining detection, scale-adaptive RoI assignment, and an attention-guided mask head. VoVNetV2 further improves the backbone with residual connections and effective squeeze-excitation.
- FCOS: FCOS is an anchor-free, proposal-free detector that predicts box offsets, class labels, and centerness at each feature-map location.This avoids predefined anchors and their associated parameter tuning and computation.
- CenterMask architecture: CenterMask detects objects with FCOS and predicts per-pixel masks inside the resulting box regions using a dedicated mask head.Its architecture contains a feature-extraction backbone, FCOS detection head, and mask head.
- Adaptive RoI assignment: CenterMask replaces standard RoI assignment with an adaptive function based on the input-to-RoI area ratio, improving scale matching for one-stage detectors.The method removes the hard-coded canonical size 224 and assigns RoIs across the backbone feature levels according to relative scale.
- Spatial Attention-Guided Mask: SAG-Mask uses spatial attention to emphasize informative pixels and suppress uninformative regions during mask prediction.RoI features are processed by convolutional layers, while the spatial attention map is formed from average- and max-pooled features, concatenation, convolution, and sigmoid normalization.
- VoVNetV2 backbone: VoVNetV2 improves VoVNet with residual connections and eSE, retaining channel dimension in its attention transformation.Residual connections address optimization degradation in deeper stacked OSA modules, while eSE replaces two dimension-reducing fully connected layers with one channel-preserving layer.
3. Experiments
Experiments evaluate CenterMask’s components, feature-level choices, backbones, and comparisons on COCO using accuracy and inference speed. Results show benefits from spatial attention, scale-adaptive RoI assignment, VoVNetV2, and CenterMask’s anchor-free design.
- Ablation study: 0.4% APmask improvement validates the scale-adaptive RoI assignment over the counterpart assignment function.The proposed assignment uses the input/RoI area ratio to adapt pooling scale.
- Ablation study: 33.4% APmask is achieved by the naive mask head, while SAM further improves mask performance and detection performance.SAM focuses prediction on informative pixels and suppresses noise, with refined features also affecting the detection branch.
- Ablation study: 0.7% APmask improvement comes from mask scoring, which recalibrates mask-result ranks but adds approximately 5 ms computation.Unlike SAM, mask scoring does not refine mask-head features or boost detection performance.
- Ablation study: P3∼P5 produces the best mask result among tested feature ranges because higher-resolution feature maps benefit pixel-level prediction.The P7 map may be too small for fine features, such as a 7 × 7 representation.
- Backbone comparison: 1.5× faster runtime is achieved by VoVNetV2-99 while maintaining competitive APmask or higher APbox than ResNeXt-101-32x8d.For smaller models, VoVNetV2-19 exceeds MobileNetV2 by 1.7% APmask/3.3% APbox at comparable speed.
- Comparison with state-of-the-art methods: CenterMask with the same ResNet-101 backbone surpasses counterparts in APmask, APbox, and speed, while CenterMask-Lite consistently exceeds YOLACT above 30 fps.CenterMask also gains 3.6% APmask over RetinaMask and exceeds TensorMask by 1.2% APmask at 5× faster speed.
4. Discussion
On small objects, Mask R-CNN performs better than CenterMask, potentially because its mask branch uses finer P2 feature maps rather than CenterMask’s P3 maps. The authors note that one-stage instance segmentation still has room for improvement.
- Mask R-CNN performs better than CenterMask on small objects.The comparison uses the same ResNet-101 backbone.
- The authors conjecture that Mask R-CNN’s P2 feature maps provide finer object layouts than CenterMask’s P3 feature maps.
- One-stage instance segmentation still has room for improvement through techniques used in Mask R-CNN.
5. Conclusion
The paper proposes CenterMask for real-time anchor-free one-stage instance segmentation and VoVNetV2 as an efficient backbone family. CenterMask achieves state-of-the-art performance at real-time speed, while VoVNetV2 is intended for efficient use across vision tasks.
- CenterMask combines anchor-free one-stage instance segmentation with a spatial attention-guided mask branch.
- VoVNetV2 spans lightweight to larger models to balance speed and accuracy.
- CenterMask achieves state-of-the-art performance at real-time speed.
- The authors propose CenterMask as a baseline for real-time instance segmentation and VoVNetV2 as an efficient backbone for various vision tasks.