Source-linked AI summary
Poly-YOLO: higher speed, more precise detection and instance segmentation for YOLOv3
Petr Hurtik, Vojtech Molek, Jan Hula, Marek Vajgl, Pavel Vlasanek, Tomas Nejezchleba
TL;DR
YOLOv3 offers high processing speed but has lower precision and suffers from label rewriting and improper anchor distribution. Poly-YOLO addresses these design issues through a single high-resolution output layer and feature aggregation, improving precision while preserving speed and enabling instance segmentation. The authors also identify limitations in polygon-label creation and experimental training resources.
Problem
YOLOv3 is fast for real-time applications but has lower precision, poor detection of big boxes from inappropriate anchor handling, and label rewriting caused by coarse resolution.
Method
Poly-YOLO uses a single high-resolution output layer connected to all anchors, hypercolumn feature aggregation, and squeeze-and-excitation blocks with reduced convolutional filters.
Results
Poly-YOLO increases YOLOv3 detection accuracy by a relative 40% while being slightly faster, and its polygon variant preserves bounding-box accuracy.
Takeaways & Limitations
Poly-YOLO improves YOLOv3 in precision, speed, and instance segmentation through conceptual changes to its detection architecture rather than a backbone swap alone.
Takeaways & Limitations
Polygon-label creation can select the farther of two vertices in one polar cell, adding an incorrect part to strongly non-convex objects.
Abstract
from arXiv · showhide
We present a new version of YOLO with better performance and extended with instance segmentation called Poly-YOLO. Poly-YOLO builds on the original ideas of YOLOv3 and removes two of its weaknesses: a large amount of rewritten labels and inefficient distribution of anchors. Poly-YOLO reduces the issues by aggregating features from a light SE-Darknet-53 backbone with a hypercolumn technique, using stairstep upsampling, and produces a single scale output with high resolution. In comparison with YOLOv3, Poly-YOLO has only 60% of its trainable parameters but improves mAP by a relative 40%. We also present Poly-YOLO lite with fewer parameters and a lower output resolution. It has the same precision as YOLOv3, but it is three times smaller and twice as fast, thus suitable for embedded devices. Finally, Poly-YOLO performs instance segmentation using bounding polygons. The network is trained to detect size-independent polygons defined on a polar grid. Vertices of each polygon are being predicted with their confidence, and therefore Poly-YOLO produces polygons with a varying number of vertices.
1 Problem statement
The paper targets real-time object detection with instance segmentation by addressing YOLOv3’s precision-related weaknesses and extending it with polygon-based contours.
- Problem statement: YOLOv3 offers high processing speed but lower precision than several competing detectors, motivating Poly-YOLO.The identified YOLOv3 drawbacks are inaccurate detection of big boxes from anchor handling and label rewriting caused by coarse resolution.
- Problem statement: Multi-resolution feature maps are aggregated with a hypercolumn, while stairstep upscaling unifies their resolutions with preserved computation speed.The paper reports slightly lower loss than direct upscaling for stairstep upscaling.
- Problem statement: Poly-YOLO extends detection to instance segmentation using bounding polygons whose maximal vertex count can be adjusted for the required precision.Polygons are defined on a polar grid with relative coordinates, supporting size-independent shapes and a dynamic number of vertices.
2 Current state and related work
The related work distinguishes one-stage and two-stage detection and surveys instance-segmentation approaches, while positioning Poly-YOLO’s polygons as a distinct representation.
- Object detection: Two-stage detectors first propose regions of interest and then regress boxes and classify within them, whereas one-stage detectors predict boxes and classes simultaneously.Two-stage methods are generally more precise but slower than one-stage methods.
- One-stage detectors: YOLO and SSD are established one-stage detectors that divide images into grids and commonly use predefined anchor dimensions.Dense one-stage predictions create many negative examples, motivating methods such as focal loss.
- Instance segmentation: Mask R-CNN extends Faster R-CNN with a fully convolutional mask-prediction head, while RetinaMask applies the same principle to RetinaNet.Instance-segmentation methods are categorized as top-down, bottom-up, or direct approaches.
- Poly-YOLO: Poly-YOLO produces a dynamic number of polygon vertices according to object shape complexity, rather than fixing one polygon size for every instance.The polygon size is selected according to the particular bounding box.
3 Fast and precise object detection with Poly-YOLO
Poly-YOLO revisits YOLOv3 to address label rewriting and imbalanced anchor distribution while preserving fast detection. It uses a lighter architecture with a single high-resolution output, hypercolumn aggregation, stairstep upsampling, and SE blocks, achieving higher precision with fewer parameters.
- YOLOv3 limitations: YOLOv3 is fast but suffers from label rewriting and imbalanced anchor distribution across output scales.These issues can leave objects undetected or assign boxes to unsuitable scales.
- Poly-YOLO architecture: Hypercolumn aggregation combines multi-resolution feature maps, while stairstep upsampling reduces loss slightly compared with direct upscaling at identical computation time.Stairstep interpolation progressively changes resolution using the lowest available upscale factor, two.
- Poly-YOLO architecture: The backbone adds squeeze-and-excitation blocks and reduces convolutional filters to 75% of the original number to offset computation costs.The lighter neck and head together have 37.1M parameters versus 61.5M for YOLOv3, while Poly-YOLO achieves higher precision.
- Design rationale: Poly-YOLO focuses on conceptual head changes that can accommodate modern backbones without retaining YOLOv3’s label-rewriting and anchor-distribution issues.The authors distinguish these head-design problems from limitations that would be solved by merely swapping the backbone.
4 Instance segmentation with Poly-YOLO
Poly-YOLO extends YOLO with instance segmentation by predicting size-independent bounding polygons on a polar grid alongside bounding boxes. Its architecture and loss are modified to support variable polygon vertices while preserving box detection.
- Polygon representation: Poly-YOLO adds polygon detection to YOLO without recurrent processing, allowing bounding polygons to contain a varying number of vertices.The extension is intended to provide tighter object boundaries than rectangular boxes.
- Polygon representation: The polygon representation augments each box with polar-grid vertices, where each vertex contains distance, angle, and confidence values.Relative polar coordinates support shape descriptions that are less dependent on object size.
- Polar grid: Each polar cell predicts one vertex’s polar coordinates, while its confidence indicates whether a vertex is present in that cell.The box center serves as the polar origin, and predicted distances can be scaled by the box diagonal.
- Architecture and loss: Integrating polygon detection changes the output layer and adds a polygon-specific loss to the YOLOv3-derived losses.The added loss covers distance, angle, and vertex-confidence prediction across grid cells and anchors.
- Architecture and loss: Polygon integration increases the total number of network parameters by 0.83% and does not affect processing speed, but training requires more VRAM.The higher-resolution output tensor may reduce the maximum training batch size.
- Joint detection: The network simultaneously detects bounding boxes and polygons, combining rectangular localization with a tighter object boundary.The authors motivate this combination because polygon vertices delimit the same object as the bounding box.
5 Benchmarks
The benchmarks evaluate bounding-box detection and instance segmentation across Simulator, Cityscapes, and IDD. Poly-YOLO improves YOLOv3’s detection accuracy, while comparisons with RetinaNet and Mask R-CNN expose speed–precision trade-offs and evaluation limitations.
- Experimental setup: The evaluation covers bounding-box detection and instance segmentation on Simulator, Cityscapes, and IDD datasets.Inference time is measured on an RTX2080Ti, while training uses several mid-tier graphics cards.
- Datasets and preprocessing: Cityscapes supplies both bounding-box and pixel-level labels, requiring polygon extraction before Poly-YOLO can train on segmentation targets.The paper describes preprocessing because Poly-YOLO requires polygonal rather than pixel-wise representations.
- Bounding-box results: Poly-YOLO raises YOLOv3 detection accuracy by a relative average 40% while being slightly faster at inference.Poly-YOLO with bounding polygons preserves bounding-box detection accuracy, with improvements in four of six cases.
- Bounding-box results: Poly-YOLO lite slightly outperforms YOLOv3 in detection accuracy and runs twice as fast.This variant uses a lower-output-resolution design intended to reduce computational cost.
- Comparisons: RetinaNet achieves higher precision than Poly-YOLO but is slower in two of three cases.On Simulator, Poly-YOLO has higher AP50 but lower AP75 when both use the same resolution.
- Evaluation caveats: Speed comparisons are limited by framework and operating-system differences, especially because RetinaNet uses an optimized PyTorch/Linux implementation.YOLOv3 and Mask R-CNN use the same TensorFlow/Windows environment as Poly-YOLO, whereas RetinaNet does not.
- Comparisons: Mask R-CNN has slightly better box accuracy but lower masking accuracy and substantially lower processing speed than Poly-YOLO.The authors report that this speed difference makes Mask R-CNN unsuitable for real-time image or video processing in their comparison.
- Evaluation caveats: Absolute scores may remain below their best possible values because hardware limits batch size and training uses dynamic learning-rate control with early stopping.The authors did not run enormous numbers of iterations with a fixed learning-rate decrease.
6 Discussion
The discussion examines architectural choices, polygon resolution, anchor count, practical polar-coordinate use, and label-creation limitations in Poly-YOLO.
- Architectural variants: SE-Standard places the squeeze-and-excitation block after the convolutions and achieves the best result among the tested variants.The comparison is based on training-loss curves computed as the mean from five runs.
- Polygon resolution: Higher polygon-vertex limits preserve more information for complicated objects but increase output-tensor parameters, potentially reducing batch sizes and increasing training time.The appropriate setting depends on the user and available hardware.
- Anchor count: Anchor count affects detection precision because each label is assigned to the anchor with maximum intersection-over-union during preprocessing.YOLOv2 uses five anchors in one output layer, whereas YOLOv3 uses nine anchors across three output layers.
- Practical application: Polar-coordinate intervals let intelligent headlamps illuminate or dim selected object regions without additional labels, while remaining size- and aspect-ratio-independent.The approach is described for classes such as cars, bikers, pedestrians, and vans, and does not affect training or inference speed.
- Limitations: A label-creation limitation occurs when two vertices share a polar cell: retaining the farther vertex can add an incorrect part to strongly non-convex objects.The network is trained correctly, but predictions follow the imprecise labels; objects whose vertices occupy distinct polar cells can still be handled correctly.
7 Summary
Poly-YOLO improves YOLOv3 through a redesigned neck and head, reducing model size while increasing accuracy and adding real-time instance segmentation with dynamic polygons.
- Detection improvements: Poly-YOLO improves YOLOv3 by addressing label rewriting and incorrect anchor distribution with a new neck and head.The neck combines hypercolumn features with stairstep upsampling, while the head processes one high-resolution output tensor.
- Detection improvements: 60% of YOLOv3's parameters yields a relative 40% accuracy improvement.The paper attributes the precision gain to removing YOLO's label-rewriting and anchor-distribution issues.
- Instance segmentation: Poly-YOLO adds instance segmentation by detecting bounding polygons with a dynamic number of vertices for each object.The polygon representation learns size-independent shapes and supports real-time operation on mid-tier graphics cards.