Source-linked AI summary
Explicit Shape Encoding for Real-Time Instance Segmentation
Wenqiang Xu, Haiyang Wang, Fubo Qi, Cewu Lu
TL;DR
Instance segmentation is computationally costly when frameworks process each object through an upsampling network. ESE-Seg pairs explicit IR shape encoding and Chebyshev fitting with object detectors, then decodes shapes using tensor operations. On Pascal VOC 2012, it outperforms Mask R-CNN at mAPr@0.5 while running seven times faster with the same ResNet-50 backbone.
Problem
Instance segmentation must predict object location and shape, but per-instance upsampling networks make it less efficient than object detection.
Source: §1 · p. 1Method
ESE-Seg uses an Inner-center Radius contour signature approximated by Chebyshev polynomials, with detectors regressing its coefficients for tensor-based shape decoding.
Source: §1 · pp. 1–2Results
69.3 mAPr at IoU threshold 0.5 is reported on Pascal VOC, better than Mask R-CNN on Pascal VOC 2012 and 7 times faster with the same ResNet-50 backbone.
Source: §1 · p. 2Takeaways & Limitations
ESE-Seg reconstructs multiple instance shapes in one pass, reducing computational consumption without model acceleration techniques.
Source: §1 · p. 2Performance drops substantially at IoU threshold 0.7, which the authors attribute to shape-reconstruction inaccuracy and noise from CNN regression.
Source: §5 · p. 8
Abstract
from arXiv · showhide
In this paper, we propose a novel top-down instance segmentation framework based on explicit shape encoding, named \textbf{ESE-Seg}. It largely reduces the computational consumption of the instance segmentation by explicitly decoding the multiple object shapes with tensor operations, thus performs the instance segmentation at almost the same speed as the object detection. ESE-Seg is based on a novel shape signature Inner-center Radius (IR), Chebyshev polynomial fitting and the strong modern object detectors. ESE-Seg with YOLOv3 outperforms the Mask R-CNN on Pascal VOC 2012 at mAP$^r$@0.5 while 7 times faster.
1. Introduction
ESE-Seg addresses the computational cost of instance segmentation by explicitly encoding object shapes as short vectors that can be decoded alongside bounding boxes. The paper reports better Pascal VOC 2012 performance than Mask R-CNN at a sevenfold speed advantage.
1. Introduction
- Instance segmentation predicts object location and shape, but per-instance upsampling networks make mainstream frameworks computationally expensive compared with object detection.The paper frames short-vector shape regression and simple decoding as a route toward detection-like efficiency.Source: §1 · p. 1
- The Inner-center Radius (IR) signature represents contour points as radii around an inner-center and uses Chebyshev polynomial coefficients as a short shape vector.The coefficients are regressed alongside the 4D bounding-box vector, and shape decoding uses simple tensor operations.Source: §1 · p. 2
- 69.3 mAPr at IoU threshold 0.5 is reported for ESE-Seg, which scores better than Mask R-CNN on Pascal VOC 2012 and is 7 times faster with the same ResNet-50 backbone.The paper also reports 48.7 mAP at IoU threshold 0.5; a YOLOv3-tiny variant reaches approximately 130 fps with 53.2% mAPr@0.5 on Pascal VOC.Source: §1 · p. 2
- ESE-Seg combines explicit shape encoding with modern object detectors and reconstructs multiple instance shapes in one pass, reducing computation without model acceleration techniques.The framework is described as reaching object-detection speed.Source: §1 · pp. 1–2
2. Related Work
Related work contrasts explicit shape encoding with implicit shape decoding and situates ESE-Seg among top-down segmentation and bounding-box-based detectors. It also notes that point-based detectors are outside its compatibility scope.
Explicit v.s Implicit Shape Representation
- Explicit shape representation typically uses contours and can decode all shapes in one pass, whereas implicit representation typically uses masks and requires decoder-network passes for individual instances.The paper also contrasts their training setups: autoencoder and detector inputs differ in object scales and color patterns, while it says this issue does not arise for explicit representation.Source: §2 · p. 2
- Centroid radius, complex coordinates, and cumulative angle are among other contour-sampled shape representations, but only a few such methods can reconstruct shapes.The passage names these as examples of shape-related features sampled along the contour.Source: §2 · p. 2
Object detection
- ESE-Seg is compatible with bounding-box-based detectors, including Faster R-CNN, YOLO, YOLOv3, and RetinaNet, but not point-based detectors.The stated boundary is that point-based detection does not parameterize the bounding box used as the shape in this setting.Source: §2 · pp. 2–3
Instance Segmentation
- Unlike top-down methods that predict instance masks sequentially, ESE-Seg regresses short shape vectors and decodes instances simultaneously, making inference time unaffected by instance count.The paper also says data augmentation and scale normalization methods can be integrated into the system.Source: §2 · p. 3
3. Method
ESE-Seg predicts compact shape descriptors alongside detections, then reconstructs instance contours with explicit shape encoding and tensor operations. The method combines an Inner-center Radius signature, Chebyshev fitting, and object detectors.
3.1. Overview
- The pipeline regresses bounding boxes and Chebyshev shape coefficients, then reconstructs each contour using tensor operations.The overview describes the coefficients as the shape descriptor and the final decoding step as simple tensor operations.Source: §3.1 · p. 3
3.2.1 Inner-center Radius Shape Signature
- The IR signature uses an inner center as the polar origin and samples contour radii by angle; after normalization, it is translation- and scale-invariant.The inner center is the point farthest from the contour, found by distance transform; the implementation samples 360 points at τ = π/180.Source: §3.2.1 · pp. 3–4
3.2.2 Fitting the Signature to Coefficients
- Chebyshev fitting compresses the long, noise-sensitive IR vector into a shorter shape descriptor represented by polynomial coefficients.The truncated approximation uses coefficients c0 through cn as the shape signature vector.Source: §3.2.2 · p. 4
3.2.3 Discussion
- Chebyshev fitting gives lower reconstruction error, lower noise sensitivity, and a better coefficient distribution than polynomial regression and Fourier fitting.The paper also reports that contour-vertex XY sampling drops about 10 mAP, while IR is more accurate than XY at equal vector dimension.Source: §3.2.3 · p. 5
3.3. Regression Under Object Detection Framework
- The detector predicts the inner center and shape vector alongside each bounding box, while retaining the framework’s original box-regression and classification losses.The shape-learning loss is described separately.Source: §3.3 · pp. 5–6
3.4. Decoding Shape Vector to Shape
- Tensor operations decode contours by adding the predicted inner center to the fitted radius along sampled polar directions, with minor GPU cost.The paper states that this decoding lets segmentation run at the same speed as object detection.Source: §3.4 · p. 6
4. Experiment
The experiments compare ESE-Seg with implicit segmentation methods, evaluate explicit shape descriptors and fitting choices, test different base detectors, and inspect qualitative outputs.
4.2. On explicit descriptors
- The experiments compare proposed IR with 2D vertices on Pascal VOC 2012, using bounding boxes as a baseline, and evaluate direct regression versus Chebyshev fitting at signature lengths of 20 and 40 points.Table 2 compares shape-signature choices on Pascal VOC 2012.Source: §4.2 · p. 7, Table 2 · p. 7
4.1. Explicit v.s. Implicit
- On Pascal VOC 2012 without SBD, YOLOv3-Cheby (20) outperforms Mask R-CNN (with ResNet50) at mAPr@0.5 and is close to it on COCO, using reported scores.Source: §4.1 · p. 7
4.2. On explicit descriptors
- The fitting experiments compare polynomial, Fourier, and Chebyshev approximations of IR on Pascal VOC 2012 val.The comparison evaluates neural-network performance on coefficients obtained by these approximation methods.Source: §4.2 · pp. 7–8, Table 3 · p. 8
4.3. On base object detector
- ESE-Seg’s performance is reported as stable across Faster R-CNN, RetinaNet, and YOLOv3-tiny base detectors, with the speed boost of each detector retained.Table 4 compares detectors using the IR shape signature and Chebyshev fitting on Pascal VOC 2012 val.Source: Table 4 · p. 8, §4.3 · p. 8
4.4. Qualitative Results
- Qualitative results show predicted shape vectors capturing contour characteristics rather than producing random noise.The results are shown in Figure 11.Source: §4.4 · p. 8, Figure 11 · p. 8
5. Limitations and Future Works
ESE-Seg has decent performance at IoU 0.5 with minor time-consumption, but its performance drops substantially at higher IoU thresholds such as 0.7.
5. Limitations and Future Works
- At IoU 0.7, reconstruction inaccuracy and CNN regression noise substantially reduce performance, despite decent performance at IoU 0.5.The authors identify improved explicit shape representation and CNN regression training as future research directions for higher-IoU performance.Source: §5 · p. 8