Source-linked AI summary
PolarMask: Single Shot Instance Segmentation with Polar Representation
Enze Xie, Peize Sun, Xiaoge Song, Wenhai Wang, Ding Liang, Chunhua Shen, Ping Luo
TL;DR
Instance segmentation must predict both instance locations and semantic masks, while existing pipelines can be complex or slow. PolarMask addresses this with anchor-box-free polar contour modeling through center classification and dense distance regression, achieving 32.9% mask mAP on COCO with single-model, single-scale evaluation.
Problem
Instance segmentation requires predicting each instance’s location and semantic mask, motivating simpler mask prediction modules that can plug into off-the-shelf detectors.
Method
PolarMask represents masks as polar contours and formulates segmentation as parallel instance center classification and dense distance regression, using Polar Centerness and Polar IoU Loss.
Results
32.9% mask mAP is achieved on COCO with single-model and single-scale training and testing, with competitive performance against more complex one-stage methods.
Takeaways & Limitations
PolarMask provides a simple, flexible, and efficient framework intended as a strong baseline for single-shot instance segmentation.
Takeaways & Limitations
Polar representation has corner cases that restrict its theoretical upper bound from reaching 100% AP, and the method assumes uniformly emitted rays for its continuous IoU formulation.
Abstract
from arXiv · showhide
In this paper, we introduce an anchor-box free and single shot instance segmentation method, which is conceptually simple, fully convolutional and can be used as a mask prediction module for instance segmentation, by easily embedding it into most off-the-shelf detection methods. Our method, termed PolarMask, formulates the instance segmentation problem as instance center classification and dense distance regression in a polar coordinate. Moreover, we propose two effective approaches to deal with sampling high-quality center examples and optimization for dense distance regression, respectively, which can significantly improve the performance and simplify the training process. Without any bells and whistles, PolarMask achieves 32.9% in mask mAP with single-model and single-scale training/testing on challenging COCO dataset. For the first time, we demonstrate a much simpler and flexible instance segmentation framework achieving competitive accuracy. We hope that the proposed PolarMask framework can serve as a fundamental and strong baseline for single shot instance segmentation tasks. Code is available at: github.com/xieenze/PolarMask.
1. Introduction
PolarMask presents a simple, anchor-box-free instance segmentation framework that represents masks by contours in polar coordinates. It combines center classification and dense distance regression, with tailored techniques that achieve competitive COCO performance under single-model, single-scale evaluation.
- Instance segmentation requires predicting both each instance’s location and semantic mask, motivating methods that combine detection with segmentation.
- PolarMask represents each mask by a contour in polar coordinates, using an object center, angles, and distances to locate contour points.
- The framework converts instance segmentation into two parallel tasks: instance center classification and dense distance regression.
- PolarMask can be embedded in detectors such as FCOS, RetinaNet, and YOLO with minimal modification.
- Polar Centerness and Polar IoU Loss address center-example sampling and dense distance-regression optimization, respectively.
- 32.9% mask mAP is achieved on COCO with single-model and single-scale training and testing, while the framework remains competitive with more complex one-stage methods.
2. Related Work
Prior instance segmentation methods include two-stage detect-then-segment systems and diverse one-stage approaches. PolarMask differs by directly modeling instances with parallel center classification and contour-distance regression in a simpler framework.
- Two-stage methods typically detect bounding boxes first and then segment within each box, achieving strong performance but often running slowly.
- One-stage methods use varied mechanisms, including energy maps, instance-sensitive score maps, and other fully convolutional designs.
- PolarMask directly models instance segmentation through two parallel branches that classify instance mass-centers and regress ray distances to contours.
- The paper characterizes PolarMask as simpler and more efficient than the related methods discussed.
3. Our Method
PolarMask models instance segmentation in polar coordinates as center classification plus dense distance regression. Its Polar Centerness and Polar IoU Loss address sample quality and regression optimization, while masks are assembled from predicted rays.
- 3.2. Polar Mask Segmentation: PolarMask represents each instance with a candidate center and uniformly angled rays whose predicted lengths reach the contour.The framework therefore formulates segmentation as instance center classification and dense distance regression.
- 3.2. Polar Mask Segmentation: Mass-center sampling is preferred because it is more likely to fall inside an instance than the box center, although donut-shaped cases remain problematic.The method samples approximately 9–16 pixels around each mass-center as positive center examples.
- 3.3. Polar Centerness: Polar Centerness re-weights samples according to ray-length diversity and multiplies with classification scores to down-weight low-quality masks.It is intended for difficult regression examples with highly diverse ray lengths and improves accuracy under stricter localization metrics such as AP75.
- 3.2. Polar Mask Segmentation: Polar Representation has corner cases that prevent its practical upper bound from reaching 100% AP, including rays originating outside masks without intersections.Such rays receive a minimum target value ε, while multiple intersections use the maximum-length intersection.
- 3.2. Polar Mask Segmentation: During inference, contour points are connected sequentially from 0° to assemble the complete contour and mask, followed by confidence filtering and NMS.At most 1k top-scoring predictions per FPN level are assembled before merging levels and applying NMS at a 0.5 threshold.
- 3.4. Polar IoU Loss: Polar IoU Loss treats the rays jointly by deriving mask IoU in polar coordinates from differential angular areas.The loss is differentiable, supports parallel computation, balances classification and regression, and improves performance over smooth-l1 loss.
4. Experiments
Experiments show that PolarMask models masks accurately with polar rays, benefits substantially from Polar IoU Loss and Polar Centerness, and achieves competitive accuracy and speed with a simple pipeline.
- Upper Bound Analysis: Above 90% IoU is reached in the upper-bound analysis as the number of rays increases, showing that Polar Segmentation can model masks precisely.The result uses ground-truth ray distances and evaluates different ray counts.
- Number of Rays: 1.5% AP improvement is obtained with 36 rays over 18 rays, while performance saturates at 72 rays.More rays improve the representational upper bound and AP until additional rays no longer constrain performance.
- Polar IoU Loss vs. Smooth-l1 Loss: 27.7% AP with Polar IoU Loss exceeds the best Smooth-l1 setting by 2.6% AP.Polar IoU Loss achieves this without balancing regression and classification losses, whereas Smooth-l1 reaches 25.1% AP in its best setting.
- Polar IoU Loss vs. Smooth-l1 Loss: Polar IoU Loss produces smoother, more precise contours, whereas Smooth-l1 Loss exhibits systematic artifacts.The comparison attributes the visual difference to whole-object supervision of the contour regression.
- Polar Centerness vs. Centerness: 1.4% AP overall is gained by Polar Centerness, including improvements of 2.3% AP for AP75 and 2.6% AP for APL.The gains are especially pronounced under stricter localization metrics and for large instances.
5. Conclusion
PolarMask is an anchor-box-free, single-shot instance segmentation method that represents masks as contours defined by a center and polar rays, with low computational overhead.
- 5. Conclusion: PolarMask directly predicts instance masks from centers and rays in polar coordinates instead of using bounding boxes and pixelwise mask classification.The framework is designed to remain as simple and clean as single-shot object detectors while adding negligible computing overhead.
- 5. Conclusion: PolarMask is presented as a simple, flexible framework that can serve as a baseline for single-shot instance segmentation.Its design supports direct mask prediction without a box branch.
A.1. Distance Label Generation
Distance labels are generated by relating every contour point to an instance center, then selecting distances for predefined angular rays.
- A.1. Distance Label Generation: Contour points are converted into distance-angle pairs relative to the instance center before ray-specific distance labels are assembled.For each angle, the procedure selects the corresponding distance and uses the maximum when multiple distances are available.
A.2. Computation Complexity and Speed Analysis
The complexity and speed analysis finds that PolarMask adds little computation over FCOS and is faster than TensorMask and Mask R-CNN under the reported settings.
- Computation Complexity: PolarMask without a box branch introduces marginal computation and parameter cost compared with FCOS.The comparison uses 800×1280 inputs and ResNet-50 backbones for all methods.
- Speed Analysis: PolarMask is faster than TensorMask and Mask R-CNN when inference time includes post-processing.Testing uses images with an 800-pixel shorter side and a maximum 1333-pixel longer side.
A.3. Polar Centerness vs. Centerness
Polar Centerness increases the weighting of high-quality positive samples and improves ray-distance regression compared with original Centerness. Figure 9 visualizes these predictions in Cartesian and polar forms.
- Polar Centerness automatically increases high-quality positive-sample weights while decreasing low-quality positive-sample weights.
- Polar Centerness produces more accurate ray-distance regression than original Centerness in the visualized results.
- Figure 9 compares Cartesian Centerness on the left with Polar Centerness on the right.Orange lines show predicted ray distances from centers to contours, while white lines show object contours.
A.4. More Benchmark Results for PolarMask
Additional MS-COCO validation results quantify the effects of multi-scale training and deformable convolution, with the strongest reported configuration reaching 35.9 mAP.
- Table 5 reports results on the MS-COCO validation minival set and defines DCN and ms-train settings.DCN denotes deformable convolution layers; ms-train randomly scales the image short side from 640 to 800.
- 35.9 mAP is achieved by PolarMask with a ResNext-101 backbone, deformable convolution, and multi-scale training.
- Multi-scale training improves final results by 1%-1.5%.
- Deformable convolution boosts performance by at least 2.3% across different backbones.
A.5. Comparison with ESE-Seg
PolarMask differs from ESE-Seg in detection requirements, center-sample selection, and regression design, and it surpasses ESE-Seg by nearly 7.5 AP on COCO 2017 validation.
- PolarMask does not require box detection, whereas ESE-Seg does.
- Polar Centerness selects the best sample among multiple positive samples near the mass center, unlike ESE-Seg’s inner-center rule.
- PolarMask directly regresses ray lengths with Polar IoU Loss, while ESE-Seg regresses fitted coefficients using traditional L2 loss.
- PolarMask surpasses ESE-Seg by nearly 7.5 AP on COCO 2017 validation.