PaperlayerView exact revision on arXiv

Source-linked AI summary

Mask Encoding for Single Shot Instance Segmentation

Rufeng Zhang, Zhi Tian, Chunhua Shen, Mingyu You, Youliang Yan

arXiv:2003.11712v2cs.CV

TL;DR

Compact mask representation is a challenge for one-stage instance segmentation, motivating the question of whether low-dimensional mask prediction can retain competitive accuracy. MEInst encodes masks as compact fixed-dimensional vectors and regresses them through a branch on a one-stage detector. Its best model reaches 38.2% mask AP on COCO test-dev, and the paper reports competitive accuracy and speed among one-stage methods.

  • Problem

    The paper asks whether masks can be predicted in their intrinsic low-dimensional space while retaining competitive accuracy.

    Source: §1 · pp. 1–2

  • Method

    MEInst encodes each mask as a compact fixed-dimensional vector and adds a detector branch to regress its mask coefficients.

    Source: §1 · p. 2

  • Results

    38.2% mask AP on COCO test-dev is achieved by MEInst’s best model, which also outperforms recent one-stage algorithms.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The framework achieves competitive accuracy and speed among one-stage paradigms, and the authors identify other instance-recognition tasks as a future application to explore.

    Source: §5 · p. 8

    For large objects, the compact representation can struggle to accommodate mask details, so additional detail-encoding modules are needed.

    Source: §4.3 · p. 8

Abstract

from arXiv · show

To date, instance segmentation is dominated by twostage methods, as pioneered by Mask R-CNN. In contrast, one-stage alternatives cannot compete with Mask R-CNN in mask AP, mainly due to the difficulty of compactly representing masks, making the design of one-stage methods very challenging. In this work, we propose a simple singleshot instance segmentation framework, termed mask encoding based instance segmentation (MEInst). Instead of predicting the two-dimensional mask directly, MEInst distills it into a compact and fixed-dimensional representation vector, which allows the instance segmentation task to be incorporated into one-stage bounding-box detectors and results in a simple yet efficient instance segmentation framework. The proposed one-stage MEInst achieves 36.4% in mask AP with single-model (ResNeXt-101-FPN backbone) and single-scale testing on the MS-COCO benchmark. We show that the much simpler and flexible one-stage instance segmentation method, can also achieve competitive performance. This framework can be easily adapted for other instance-level recognition tasks. Code is available at: https://git.io/AdelaiDet

1. Introduction

MEInst addresses the challenge of compact mask representation by encoding each mask as a low-dimensional vector and regressing its coefficients in a one-stage detector. The introduction presents this approach as simple and flexible, with competitive results against other one-stage methods and Mask R-CNN.

1. Introduction

  • Two-stage methods lead COCO rankings but runtime is constrained by instance count, while contour-based methods can produce “hollow decay” on disjointed objects.Contour-based methods represent each instance with a single contour.Source: §1 · p. 1
  • The method asks whether predicting masks in their intrinsic low-dimensional space can retain competitive accuracy, using a learned dictionary and few scalar coefficients.The authors report that the approach is robust to noise and efficient to decode for reconstruction.Source: §1 · pp. 1–2
  • MEInst outperforms recent one-stage algorithms, beats PolarMask [32] at similar computational complexity, and achieves performance comparable to Mask R-CNN.On COCO val2017, it exceeds ESE-Seg [33] by 11.8% in AP50 and 16.5% in AP75.Source: §1 · p. 2
  • MEInst is compatible with most one-stage detectors, including anchor-free frameworks, and the parallel mask branch can also improve bounding-box detection accuracy.The authors demonstrate generality with FCOS and note that anchor-based methods such as YOLO and RetinaNet can be used with minimal modification.Source: §1 · p. 2
  • MEInst encodes each two-dimensional mask as a compact representation vector and adds a mask-coefficient regression branch to a one-stage detector.The compressed vector uses redundancy in the mask to support reconstruction.Source: §1 · p. 2
  • Even the simplest dictionary-learning option, PCA, suffices for mask encoding.Other options mentioned include sparse coding and autoencoders.Source: §1 · p. 2
  • 38.2% mask AP on COCO test-dev is achieved by MEInst’s best model.The introduction describes it as a simple, flexible one-stage method balancing accuracy and speed.Source: §1 · p. 2

2. Related Work

Related work spans two-stage proposal-based methods, segmentation-driven one-stage methods, and approaches that encode masks with prototypes or contours. The section positions MEInst among these alternatives by their mask representations and computational trade-offs.

2. Related Work

  • Two-stage methods detect instance boxes before binary-class segmentation within each box and lead most benchmarks in accuracy.Mask R-CNN introduced ROIAlign, while later work added feature optimization [22] or a MaskIoU branch [15].Source: §2 · p. 2
  • Segmentation-driven one-stage methods first produce pixel-wise classification maps and then cluster them into instances; InstanceCut [16] separates instance-agnostic segmentation and instance-specific boundaries.Source: §2 · p. 2
  • Mask-representation alternatives include TensorMask’s structured 4D tensor [7], YOLACT’s global prototypes and linear coefficients [3], and PolarMask’s parametric representation [32].TensorMask has heavy training and testing computation; BlendMask [4] improves YOLACT in accuracy and speed, while SOLO and SOLOv2 [30] [31] report promising results.Source: §2 · pp. 2–3

3. Our Method

MEInst extends FCOS with a parallel mask-coefficient branch and learns a compact representation for reconstructing instance masks. The method also examines how detection quality and one-stage box behavior relate to mask performance.

3.1. Network Architecture

  • MEInst inherits FCOS’s backbone, feature pyramid, and detection heads, adding a parallel branch that predicts encoded mask coefficients.The detection and mask-vector predictions are made simultaneously; Figure 2 shows the overall architecture.Source: §3.1 · p. 3, Figure 2 · p. 4

3.2. Mask Encoding

  • Because instance masks contain redundant pixels, MEInst flattens class-agnostic masks and compresses them into N-dimensional vectors, with N typically much smaller than H·W.The projection and reconstruction matrices are learned by minimizing training-set reconstruction error using PCA.Source: §3.2 · p. 3
  • Predicted mask vectors are reconstructed after non-maximum suppression on the 100 highest-scoring samples, making the matrix multiplication cost negligible.The mask loss uses L2 distance, while the detection loss combines classification, box-regression, and center-ness losses.Source: §3.2 · p. 3

3.3. Correlation Between Boxes and Masks

  • The reported experiments empirically validate the assumptions, finding that better bounding boxes improve overall performance in the mask branch.Source: §3.3 · p. 4
  • 34.1% vs. 34.2% mask AP shows that FCOS’s stronger detection results did not yield equivalent segmentation gains across the compared pipelines.FCOS had higher APbb across the tested backbones, with reported gains of 0.9%, 2.8%, and 1.8%.Source: §3.3 · p. 4, Table 1 · p. 4, Figure 4 · p. 5

3.2. Mask Encoding

  • The mask-encoding pipeline processes annotations offline and freezes its learned parameters during network training and inference.It reshapes class-agnostic masks, flattens and compresses them, then obtains a transformed matrix for encoding.Source: §3.2 · p. 3, Figure 3 · p. 5

3.3. Correlation Between Boxes and Masks

  • FCOS produces significantly more boxes than Mask R-CNN at the same confidence threshold, and its excess false positives may degrade APbb.The authors hypothesize that insufficient effective receptive field contributes to false-positive boxes and use deformable convolution to enlarge it; dilated convolution and large kernels may also help.Source: §3.3 · p. 5, Figure 4 · p. 5

4. Experiments

On COCO test-dev, MEInst achieves 36.9% mask AP and outperforms most one-stage methods by a large margin. The experiments also examine encoding choices, detector design, and where MEInst compares favorably or unfavorably with Mask R-CNN.

4.1. Ablation Study

  • At 100 components, reconstruction error reaches 2.5%; class-agnostic encoding performs similarly to class-specific encoding while conserving memory.The class-agnostic matrix uses up to C times fewer dimensions.Source: §4.1 · p. 6
  • A 2% improvement from 20 to 60 encoding dimensions gives way to stable performance beyond 60; experiments use N = 60.The authors report that reconstruction matters early but is no longer the main constraint once enough components reconstruct the mask well.Source: §4.1 · p. 6
  • Without explicit encoding, 784-dimensional masks are harder to optimize, lowering AP75 by 1.3% and APL by 2.0%.With the same dimension, mask encoding performs better across all reported metrics.Source: §4.1 · p. 7
  • Among tested losses, l2 performs better than smooth-l1, l1, and cosine similarity.Cosine similarity performs worse, which the authors interpret as evidence that mask encoding has eased redundancy and left vector elements independent.Source: §4.1 · p. 7, Table 4 · p. 7
  • Large-kernel prediction adds 0.7% AP and deformable convolution adds 1.5% AP; MEInst also exceeds FCOS by 0.8% AP in box detection.The reported table captions identify large receptive field and mask learning as the respective comparisons.Source: §4.1 · p. 7, Table 5 · p. 7, Table 6 · p. 7
  • MEInst outperforms contour-based ESE-Seg by a large margin and retains a detail-preserving advantage; at input scale 400, it still performs better at real-time speed.The scale-400 result uses the existing model rather than one trained specifically at that scale.Source: §4.1 · p. 7, Table 7 · p. 7

4.2. Comparison with State-of-the-art Methods

  • 36.9% mask AP on COCO test-dev lets MEInst outperform most one-stage methods by a large margin.The authors report the result without auxiliary semantic-segmentation supervision or other added tricks.Source: Figure 6 · p. 6, Table 8 · p. 8, §4.2 · p. 8

4.3. Advantages and Limitations

  • 21.8% vs. 16.9% for small objects, but 46.7% vs. 53.5% for large objects, comparing MEInst with Mask R-CNN.The authors attribute the large-object disadvantage to compact vectors struggling to capture mask details; they note that additional detail-encoding modules are needed.Source: §4.3 · p. 8

5. Conclusion

MEInst represents instance masks as compact, fixed-dimensional vectors and turns mask prediction into regression. The authors report competitive accuracy and speed among one-stage methods.

5. Conclusion

  • MEInst predicts masks as compact, fixed-dimensional vectors, reformulating mask prediction from spatial-layout binary classification as regression.The authors say this reformulation allows the task to be solved by adding a parallel regression branch to existing one-stage object detectors.Source: §5 · p. 8
  • The framework achieves competitive accuracy and speed among one-stage instance segmentation methods.Source: §5 · p. 8
  • The authors plan to explore other dictionary-learning methods and apply mask encoding to other instance-recognition tasks.Source: §5 · p. 8

Related papers

Loading 2003.11712v2…