Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++

Introduction

• Detailed object boundaries matter for autonomous driving, mapping, and medical imaging, yet collecting instance masks typically takes annotators 20–30 seconds per object.Source: §1 · p. 1
• Polygon-RNN [4] predicts object-outline vertices rather than pixel-wise masks, yielding a sparse representation that is easy to edit and typically needs few annotator corrections.Source: §1 · p. 1
• Polygon-RNN++ improves Polygon-RNN with architectural changes, reinforcement-learning training, and a Graph Neural Network [18] that raises polygon output resolution; it also examines noise robustness and out-of-domain generalization.Source: §1 · pp. 1–2
• 10% mean IoU improvement over the original approach on Cityscapes and 50% fewer clicks than [4] in interactive mode establish Polygon-RNN++’s gains.Source: §1 · p. 2
• A Cityscapes-trained model generalized to scene-parsing, aerial, and two medical datasets, outperforming strong pixel-wise baselines; online fine-tuning further sped annotation of out-of-domain data.Source: §1 · p. 2

Interactive annotation

• Earlier interactive methods used scribbles, bounding boxes with GrabCut, or GrabCut combined with CNNs [26], but most label pixels and struggle when foreground and background colors are similar.Source: §2 · p. 2
• Polygon-RNN++ raises polygon prediction resolution from 28 × 28 to 112 × 112, addressing blocky outputs on large objects while extending the approach in [4].Polygon representations are sparse and easy to correct by moving vertices; an RNN captures typical object shapes, including in ambiguous boundary regions.Source: §2 · p. 2

Object instance segmentation

• Most object instance segmentation approaches operate at the pixel level, with many relying on object detection and convolutional networks over box proposals to label objects [17] [30] [41] [39] [23] [1].Source: §2 · p. 2
• Unlike methods that link detected boundary fragments into polygons or combine polygonal superpixels, Polygon-RNN++ uses neural networks to produce polygons for interactive labeling.Source: §2 · p. 2

3. Polygon-RNN++

• Given an annotator-provided bounding box, Polygon-RNN++ extracts a crop enlarged by 15% and decodes polygon vertices one at a time as grid locations.The architecture follows [4], using a CNN for image features and an RNN for vertex decoding.Source: §3 · p. 2
• The redesigned encoder predicts the first vertex, while an attention-based recurrent decoder proposes candidates that a learned evaluator ranks and a graph neural network refines at higher resolution.The graph neural network augments polygons with additional vertices before re-adjusting them.Source: §3 · p. 2
• When a predicted vertex is wrong, an annotator can correct it and feed the correction back to help the model improve its predictions at subsequent steps.Source: §3 · p. 2

3.1. Residual Encoder with Skip Connections

• Reducing ResNet-50 stride and adding dilation increase output feature-map resolution without shrinking receptive fields; the encoder also removes average pooling and fully connected layers.This modification follows and is applied to ResNet-50 [13].Source: §3.1 · p. 2
• Skip connections combine low-level details and high-level semantics by concatenating layer outputs at the highest possible resolution, then applying convolution and max-pooling operations.Unlike the VGG-based design in [4], which downsamples before concatenation, this approach concatenates first; bilinear upsampling aligns outputs with different spatial dimensions. The resulting map is called the skip features.Source: §3.1 · pp. 2–3

3.2. Recurrent Decoder

• The decoder models polygon vertices with a ConvLSTM, which preserves spatial information and reduces parameters, and adds attention and first-vertex prediction in the same network, unlike [4]’s separate networks.Source: §3.2 · p. 3
• The decoder uses two ConvLSTM layers with 3 × 3 kernels and 64 and 16 channels; at D = 28, each output is a one-hot encoding of D × D vertex positions plus an end-of-sequence token.Batch normalization is applied at each time step, with mean and variance estimates not shared across time steps [15].Source: §3.2 · p. 3
• At each step, hidden states produce location-wise attention weights that gate image features, which are concatenated with the two previous vertices and the first vertex before the next recurrent step.Source: §3.2 · p. 3
• A two-layer branch predicts the first vertex: its first layer predicts edges, its second predicts vertices, and the test-time vertex is sampled from the final layer.This avoids treating the first vertex as a separately trained special case, as in [4].Source: §3.2 · p. 3

3.3. Training using Reinforcement Learning

• MLE training [4] over-penalizes valid edge vertices and optimizes a metric unlike IoU; teacher forcing causes exposure bias [27], so MLE is initialization before RL fine-tuning.RL fine-tuning exposes the model to its own predictions while optimizing the non-differentiable IoU metric.Source: §3.3 · p. 3

3.3.1 Problem formulation

• The recurrent decoder is treated as a policy pθ that sequentially selects vertices, with terminal reward equal to polygon-mask IoU against the ground-truth mask.Source: §3.3.1 · pp. 3–4

3.3.2 Self-Critical Training with Policy Gradients

• The model uses REINFORCE with a self-critical baseline [29] set to its greedy test-time reward, and temperature τ = 0.6 controls sampled-vertex randomness.The expected gradient uses a single Monte Carlo sample, which has high variance; the greedy-reward baseline provides context-dependent normalization.Source: §3.3.2 · p. 4

3.4. Evaluator Network

• The evaluator predicts polygon quality as IoU with ground truth from skip features, the final ConvLSTM state, and the predicted polygon.This lets the model choose among candidate polygons when log-probability beam search may retain predictions that follow an occluding object.Source: §3.4 · p. 4
• The evaluator has two 3×3 convolutional layers and a fully connected layer, and is trained separately after RL fine-tuning converges.Training minimizes mean squared error; sampled polygons use τ = 0.3 for vertex diversity, and the evaluator is not used as an RL baseline because self-critical training performed better.Source: §3.4 · p. 4

Inference

• At inference, the model beam-searches polygons from the K top-scoring first vertices, then uses the evaluator to select one; experiments use K = 5.Beam search uses log probability and width B; using the evaluator at every step would make inference impractically slow. The B = K = 1 model runs at 295ms per object instance on a Titan XP.Source: §3.4 · p. 4

Annotator in the Loop

• Following [4], annotators correct vertices sequentially, and each correction is fed back so the model can re-predict the remaining polygon.Source: §3.4 · p. 4

3.5. Upscaling with a Graph Neural Network

• The GGNN upsamples polygons beyond the RNN decoder’s D × D output, with D set to 28, by predicting relative vertex offsets on a graph.The RNN output resolution is limited to 28 × 28 to satisfy memory bounds and keep the output space manageable.Source: §3.5 · pp. 4–5, Figure 5 · p. 5
• Because ground-truth polygons are simplified at their target resolution, higher-resolution objects may contain additional vertices and therefore change polygon topology.Collinear vertices are removed during RNN training to reduce ambiguity in the prediction task.Source: §3.5 · p. 4
• The graph treats predicted vertices as nodes, inserts a midpoint between each consecutive pair, closes the sequence into a cycle, and connects neighbors with three edge types.The GGNN propagates messages over this graph before predicting an output for each node.Source: §3.5 · pp. 4–5, Figure 5 · p. 5
• The GGNN branch uses the 112 × 112 × 256 feature map to extract node observations and classifies each node’s relative displacement in a D′ × D′ grid using cross-entropy loss.Training starts from RNN predictions, corrects predictions that exceed a threshold, and targets relative displacements from corresponding ground-truth vertices.Source: §3.5 · p. 5
• With S = 1 and D′ = 112, the GGNN uses a 256-dimensional GRU hidden state and T = 5 propagation steps; larger D′ did not improve results.During training, vertices are replaced with ground-truth vertices when they deviate by more than 3 cells.Source: §3.5 · p. 5

3.6. Annot. New Domains via Online Fine-Tuning

• The online fine-tuning scenario uses a model trained on one dataset to annotate a novel dataset, with annotators correcting wrong predictions as data arrives.The proposed fine-tuning approach operates in an online fashion.Source: §3.6 · p. 5
• For MLE training on corrected data, targets are smoothed with a Manhattan distance transform truncated at distance 2.Source: §3.6 · p. 5
• For each new-data chunk, the method predicts and corrects annotations, adds sampled previously seen data, then trains with MLE, RL, and evaluator-network steps.The updated model becomes the starting model for the next chunk.Source: §3.6 · p. 5

4.1. In-Domain Annotation

• Polygon-RNN++ outperforms the top performer [4] by almost 10% IoU and achieves the best performance for each Cityscapes class in automatic mode.It also surpasses the reported 78.6% human agreement on cars on average and achieves human-level performance for trucks and buses using car agreement as a proxy.Source: §4.1 · p. 6, Table 1 · p. 6
• At T2=0.7, Polygon-RNN++ achieves over 80% mIoU with 5 clicks per object on average, reducing clicks by more than 50% over [4].At T2=0.8, it remains more accurate than Polygon-RNN at T2=1.0; with bounding-box noise of 5–15%, it outperforms prior baselines using perfect boxes.Source: Figure 7 · p. 6, Figure 6 · p. 6, §4.1 · p. 7

4.1.1 Instance-Level Segmentation

• With FasterRCNN detections, Polygon-RNN++ achieves 22.8% AP and 42.6% AP50 on Cityscapes test; adding semantic post-processing raises these to 25.49% AP and 45.47% AP50.The post-processing applies a logical AND between the predicted class-semantic map and the model prediction.Source: §4.1.1 · p. 7

4.2. Cross-Domain Evaluation

• On KITTI, Polygon-RNN++ outperforms all baselines and reaches the reported 85% human agreement with an average of 2 simulated annotator clicks.It requires an average of 5 fewer clicks than [4] to achieve the same IoU.Source: Figure 8 · p. 6, §4.2 · p. 7, Table 3 · p. 7

4.2.1 Out-of-Domain Imagery

• On out-of-domain datasets, Polygon-RNN++ demonstrates high generalization, while online fine-tuning achieves at least 65% fewer clicks across datasets.Fine-tuning overcomes domain shifts with one chunk of data: 40 Sunnybrook images, 3 ssTEM, 200 ADE, or 20 Aerial; click reduction is almost 100% on Sunnybrook Cardiac MR.Source: Table 2 · p. 7, Figure 9 · p. 7, §4.2.1 · p. 8

4.3. Qualitative Results

• Qualitative results show that reinforcement learning and the evaluator network produce crisper predictions, while the GGNN adds points and builds polygons resembling human annotations.Examples include automatic predictions on unseen datasets using ground-truth boxes without fine-tuning.Source: §4.3 · p. 8, Figure 11 · p. 8, Figure 12 · p. 8

4.4. Interaction with Human Annotators

• In the in-domain human study, annotators using Polygon-RNN++ were 3x faster than manual annotators, with slightly lower IoU agreement.For cross-domain ADE20K annotation without fine-tuning, annotators were still faster with the tool, but less so, and IoU agreement was lower; the authors note potential acceptance of less-than-perfect predictions when more corrections are needed.Source: §4.4 · p. 9, Table 5 · p. 9
• The model predicts one polygon per box, tends to select the more central object, and struggles with occluded or large multi-component objects; it does not handle holes.Interactive annotation would benefit from letting humans add or remove points.Source: §4.4 · p. 9

• Polygon-RNN++ is an object-instance segmentation model designed for interactively annotating segmentation datasets, building on Polygon-RNN [4].Source: §5 · p. 9
• Polygon-RNN++ significantly outperforms Polygon-RNN [4] in both automatic and interactive modes.Source: §5 · p. 9
• The model is robust to noisy annotators and generalizes to novel domains; simple online fine-tuning enables adaptation to out-of-domain datasets.Source: §5 · p. 9

Training with RL

• RL training on Cityscapes increases mean IoU versus the MLE model, while direct IoU optimization decreases average predicted polygon length and the number of polygons with self-intersections.Figure 14 plots mean IoU, average polygon length, and self-intersections during RL training.Source: §6 · pp. 10–11, Figure 14 · p. 11

Interactive mode in Cityscapes

• Most interactive Cityscapes predictions can be corrected with 5 clicks; Figure 15 reports click-count histograms and average IoU and clicks for different T2 and T thresholds.The examples show the automatically predicted polygons, corrected results, and ground truth, with required clicks and achieved IoU.Source: Figure 15 · p. 11, §6 · p. 12

Output Resolution and Sensitivity to T of GGNN

• 224 × 224 output resolution is only 0.02% better than 112 × 112 and takes longer to train, so the paper uses 112 × 112; results are stable across several GGNN propagation-step counts.Table 7 reports automatic-mode performance for different values of T on Cityscapes.Source: §6 · p. 11, Table 7 · p. 12

Automatic Mode in Cityscapes; Extended Evaluation on ADE Val Set

• Cityscapes automatic-mode examples compare predictions with ground truth using bounding boxes, while ADE validation results without fine-tuning show performance for the top and bottom 50 categories with more than 20 instances.The ADE categories are sorted by performance; the Cityscapes model relies on bounding boxes.Source: §6 · pp. 11–12, Figure 16 · p. 13, Figure 17 · p. 14

Evaluator Network

• 2.2% over beam search is reported when the evaluator chooses among K = 5 first-vertex candidates decoded greedily; beam-searching sequences adds 0.3%, and evaluating last-vertex candidates adds 0.2%.Beam search for both the first vertex and full sequence improves only 0.5% over greedy decoding.Source: §6 · p. 11, Table 6 · p. 12

Full-image Instance-Level Segmentation on Cityscapes

• 25.49% AP and 45.47% AP50 are achieved on Cityscapes test after intersecting instance polygons with predicted semantic maps, up from 22.8% AP and 42.6% AP50 with FasterRCNN boxes alone.The model predicts only one polygon per bounding box, so multi-component masks caused by occlusion are penalized by evaluation.Source: §6 · p. 12

Automatic Mode in Out-of-Domain Imagery; Automatic Mode and Online Fine-Tuning

• Cityscapes-trained Polygon-RNN++ is shown in automatic mode on KITTI, general-scene, aerial, and medical datasets without fine-tuning, and online fine-tuning examples show predictions after the last data chunk.Some out-of-domain datasets provide segmentation masks rather than ground-truth polygons; the qualitative fine-tuning examples compare untuned and fine-tuned predictions with ground truth.Source: §6 · p. 12, Figure 21 · p. 17, Figure 22 · p. 18