Source-linked AI summary
PolyTransform: Deep Polygon Transformer for Instance Segmentation
Justin Liang, Namdar Homayounfar, Wei-Chiu Ma, Yuwen Xiong, Rui Hu, Raquel Urtasun
TL;DR
Instance segmentation methods can produce over-smoothed masks, while polygon-based annotation methods struggle with large occlusions and disconnected components. PolyTransform converts segmentation masks into polygons and refines their vertices with a deforming network. On Cityscapes, it improves backbone initialization, reaches first place on the test leaderboard, and also improves interactive annotation and annotation speed.
Problem
Instance segmentation must preserve fine-grained object boundaries, while polygon-based annotation methods struggle with large occlusions and disconnected components.
Method
PolyTransform converts segmentation-generated instance masks into polygons and applies a deforming network that predicts vertex displacements to fit object boundaries.
Results
On Cityscapes, PolyTransform improves initialization by 3.0 AP and 10.3 in the boundary metric on validation, beats current state of the art by 3.7 AP on test, and speeds annotation by 35%.
Takeaways & Limitations
The method produces precise, geometry-preserving instance segmentation and improves performance across different initialization networks and interactive annotation settings.
Takeaways & Limitations
The model can fail when initialization is poor or when occlusion is complex or ambiguous, such as with a semi-transparent fence blocking a car.
Abstract
from arXiv · showhide
In this paper, we propose PolyTransform, a novel instance segmentation algorithm that produces precise, geometry-preserving masks by combining the strengths of prevailing segmentation approaches and modern polygon-based methods. In particular, we first exploit a segmentation network to generate instance masks. We then convert the masks into a set of polygons that are then fed to a deforming network that transforms the polygons such that they better fit the object boundaries. Our experiments on the challenging Cityscapes dataset show that our PolyTransform significantly improves the performance of the backbone instance segmentation network and ranks 1st on the Cityscapes test-set leaderboard. We also show impressive gains in the interactive annotation setting. We release the code at https://github.com/uber-research/PolyTransform.
1. Introduction
Instance segmentation must identify individual objects while handling scale, appearance variation, occlusion, and blur, yet common masks often lose fine-grained geometry. PolyTransform refines segmentation-derived polygons to improve object boundaries and supports both instance segmentation and interactive annotation.
- Instance segmentation identifies countable objects and produces a mask for each, unlike semantic segmentation, which assigns only pixel categories.
- Two-stage segmentation methods handle occlusions with object boxes but often produce over-smoothed masks that miss fine-grained details.
- Polygon-based interactive annotation captures object geometry and can accelerate correction, but struggles with large occlusions and disconnected components.
- PolyTransform converts segmentation masks into polygons and uses a deforming network to predict vertex displacements from all polygon vertices.
- 3.0 AP and 10.3 in the boundary metric improve initialization on Cityscapes validation, while the model reaches 1st place and beats current state of the art by 3.7 AP on test.
- Interactive annotation improves the boundary metric by 2.0% over prior state of the art, and polygon outputs reduce crowd-sourced annotation time by 35%.
2. Related Work
Related work spans proposal-based and proposal-free instance segmentation alongside polygon- and curve-based interactive annotation. These approaches trade computational efficiency, proposal dependence, or geometric precision in different ways.
- Proposal-based Instance Segmentation: Proposal-based instance segmentation first generates segment proposals and then uses voting to select which proposal to retain.
- Proposal-based Instance Segmentation: Proposal-based methods improve efficiency through integrated feature pooling but remain relatively slow because they retain a traditional detection-based pipeline.
- Proposal-free Instance Segmentation: Proposal-free methods segment instances without explicit object proposals, using patch-level instance labels with dense CRF or watershed-based energy prediction.
- Interactive Annotation: Interactive annotation treats segmentation as finding polygons or curves that fit object boundaries, building on active contour models and later polygon representations.
3. PolyTransform
PolyTransform combines segmentation-based initialization with polygon deformation to produce precise, geometry-preserving masks. Its deforming network uses multi-scale features and vertex attention to refine polygon boundaries while regularized losses stabilize training.
- Overview: PolyTransform combines prevailing segmentation approaches with polygon-based methods to generate precise, geometry-preserving masks.The model uses segmentation outputs as a starting point and refines their polygon representation.
- Instance Initialization: The instance initialization module generates a mask, uses or fits a bounding box, crops and resizes the image, and extracts deep features.Proposal boxes are used when available; otherwise, a bounding box is fitted to the mask.
- Feature Extraction: An FPN extracts multi-scale features to help polygons capture high-curvature and complex object shapes.The backbone produces feature maps at multiple pyramid levels from the cropped image.
- Vertex Embedding: Vertex embeddings combine upsampled P2–P6 features, coordinate channels, and bilinearly sampled features at the initial polygon vertices.The resulting embedding is an N × (320 + 2) representation for the polygon vertices.
- Deforming Network: A self-attending Transformer models dependencies among neighboring vertices, and a feed-forward network predicts N × 2 offsets that deform the polygon.The offsets are added to the initialization; six Transformer operations are used in the experiments.
- Learning: Training minimizes a weighted sum of a Chamfer Distance loss and an edge-deviation regularizer that prevents overlap and unstable vertex movement.The Chamfer term moves predicted polygon edges toward ground-truth edges, while edge regularization limits sudden shifts.
4. Experiments
Experiments evaluate PolyTransform for instance segmentation and interactive annotation across Cityscapes and a new self-driving dataset, showing consistent boundary and accuracy gains over varied initializations and strong efficiency for polygon refinement. The method also improves qualitative boundary handling, while failures remain under poor initialization or complex occlusion.
- Experimental scope: PolyTransform is evaluated for both instance segmentation and interactive annotation, using Cityscapes and a new self-driving dataset.The experiments include standard segmentation, annotation with provided boxes, qualitative analysis, and comparisons with naive refiners.
- Instance segmentation: 40.1 AP establishes a new Cityscapes test-set state-of-the-art, exceeding PANet by 3.7 AP and 2.8 AP50 points.The result ranks first on the official Cityscapes leaderboard.
- Robustness to initialization: +2.2 AP and +5.8 AF over DWT show improvement across instance-segmentation initializations on the Cityscapes validation set.The paper also reports +1.6 AP and +4.9 AF over a reproduced PANet-strength initialization, with consistent boundary gains.
- Qualitative results and failure modes: Qualitative results show precise boundaries around difficult shapes and handling of some occlusions, but performance can fail with poor initialization or complex, ambiguous occlusion.The failure example involves a semi-transparent fence blocking a car.
- Interactive annotation: Up to 2% better boundary performance in Cityscapes Stretch and 4.9%, 8.3%, and 7.2% gains in Hard show strong interactive-annotation improvements.The Hard-setting gains correspond to mean IoU, F at 1px, and F at 2px, respectively.
- Efficiency: 21 ms per object instance makes PolyTransform 14x faster than Polygon-RNN++ and 1.4x faster than Curve GCN.Processing a full Cityscapes image takes 575 ms on average, with batching and FPN tuning identified as possible speed improvements.
5. Conclusion
PolyTransform combines segmentation networks with polygon deformation to produce geometry-preserving instance masks, improving segmentation performance and interactive annotation efficiency.
- PolyTransform generates instance masks, converts them into polygons, and applies a deforming network to fit object boundaries more precisely.The deforming network predicts vertex displacements using the locations of all vertices.
- 3.0 AP and 10.3 boundary-metric points are gained over initialization on Cityscapes validation, while the method reaches 1st place on the leaderboard.
- PolyTransform speeds up crowd-sourced annotation by 35% and improves the annotation-in-the-loop boundary metric by 2.0%.
- The approach combines prevailing segmentation methods with polygon-based methods and generalizes across various instance segmentation networks.