PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Object Localization with Fitness NMS and Bounded IoU Loss

Lachlan Tychsen-Smith, Lars Petersson

arXiv:1711.00164v3cs.CV

TL;DR

Many detectors rely on a single matching IoU and therefore may select sufficiently accurate rather than best-fitting boxes. The paper proposes Fitness NMS, Bounded IoU Loss, and RoI-clustering analyses, reporting improved high-localization MAP and competitive MSCOCO MAP–inference-time results.

  • Problem

    Common Non-Max Suppression designs rely on a single matching IoU, while better-fitting boxes can improve localization of object position, scale, and instance count.

  • Method

    Fitness NMS changes scoring to select boxes maximizing estimated IoU, while Bounded IoU Loss trains tighter clusters using IoU upper bounds.

  • Results

    Fitness NMS significantly improves MAP at high localization accuracies, with Joint Fitness NMS improving Ωtest = 0.75 without measurable loss at Ωtest = 0.5.

  • Takeaways & Limitations

    The modifications, combined with RoI clustering, produce highly competitive MSCOCO MAP versus inference-time results for a two-stage detector.

  • Takeaways & Limitations

    Joint Fitness NMS incurs a minor evaluation-rate loss and may be less suitable for single-stage detectors because they require more softmax outputs.

Abstract

from arXiv · show

We demonstrate that many detection methods are designed to identify only a sufficently accurate bounding box, rather than the best available one. To address this issue we propose a simple and fast modification to the existing methods called Fitness NMS. This method is tested with the DeNet model and obtains a significantly improved MAP at greater localization accuracies without a loss in evaluation rate, and can be used in conjunction with Soft NMS for additional improvements. Next we derive a novel bounding box regression loss based on a set of IoU upper bounds that better matches the goal of IoU maximization while still providing good convergence properties. Following these novelties we investigate RoI clustering schemes for improving evaluation rates for the DeNet wide model variants and provide an analysis of localization performance at various input image dimensions. We obtain a MAP of 33.6%@79Hz and 41.8%@5Hz for MSCOCO and a Titan X (Maxwell). Source code available from: https://github.com/lachlants/denet

1. Introduction

The paper frames object detection as jointly classifying instances and localizing their bounding boxes, then targets NMS designs that may retain sufficiently accurate rather than best-fitting boxes. It proposes modifying NMS scoring and similarity functions while preserving the overall algorithm.

  • Object detection jointly localizes instance bounding boxes and classifies their contents.
  • Bounding box regression updates each RoI toward a better match with the nearest true instance.Prior improvements include multiple regression stages, better RoI anchors, higher image resolution, and improved sampling-region localization.
  • When multiple predicted boxes exceed IoU 0.5 with the ground truth, standard designs do not directly discriminate between them, whereas Fitness NMS selects the blue box.
  • Standard NMS clusters detections using IoU overlap and retains the box with the highest class confidence.Its behavior is determined by same(.), score(.), and the threshold λ_nms.
  • Fitness NMS modifies score(.) to favor boxes estimated to maximize ground-truth IoU, while Bounded IoU Loss modifies training inputs to same(.) for tighter clusters.The paper also analyzes RoI clustering and input image sizes.

2. Experimental Setup

The experiments define the localization and clustering quantities used throughout the paper and evaluate models on Pascal VOC and MSCOCO with standardized Titan X timing. Matching IoU controls localization requirements, while clustering IoU controls NMS suppression.

  • A sampling Region-of-Interest (RoI) is a bounding box generated before classification in two-stage detectors.
  • Intersection-over-Union (IoU) is the intersection area divided by the union area of two bounding boxes.
  • Matching IoU (Ω) specifies the overlap required before a generated box and ground-truth box count as the same detection.Pascal VOC uses Ωtest = 0.5, whereas MSCOCO uses Ωtest ∈[0.5, 0.95].
  • Clustering IoU (λnms) specifies the overlap between detections required before NMS removes one, with DeNet defaulting to λnms = 0.5.
  • Validation uses combined Pascal VOC 2007 and 2012 trainval data with Pascal VOC 2007 test, while MSCOCO experiments train on trainval and evaluate on test-dev.Timing uses an Nvidia Titan X (Maxwell), cuDNN v5.1, and batch size 8 unless stated otherwise.

3. Fitness Non-Max Suppression

The paper identifies that standard NMS can discard better-localized boxes because it ranks detections by class probability rather than estimated IoU. Fitness NMS adds localization fitness to selection, improving fine-localization performance while preserving coarse-localization behavior and largely maintaining evaluation rate.

  • 3.1. Matching IoUs: Many detectors optimize for satisfying a single matching IoU instead of selecting the bounding box with maximum IoU.Changing the training matching IoU improved performance at corresponding test thresholds, while training near the center of the MSCOCO range improved MAP@[0.5:0.95] by 1.2%.
  • 3.2. Detection Clustering: Standard NMS ranks boxes by class probability and can discard high-IoU boxes during deduplication.At Ωtest = 0.9, recall dropped from near 40% to below 20% as detections were culled, despite the Without NMS result indicating that suitable boxes were identified earlier.
  • 3.3. Novel Fitness NMS Method: Fitness NMS modifies NMS scoring and clustering inputs to favor detections with higher estimated IoU and class probability.Fitness is represented as a discrete variable over F values, with F = 5 in this paper; boxes with ρj below 0.5 receive the null class.
  • 3.3. Novel Fitness NMS Method: Fitness NMS improved MSCOCO MAP@[0.5:0.95] by 1.9–2.7% across evaluated DeNet variants.Joint Fitness NMS avoided measurable loss at Ωtest = 0.5 and improved performance at Ωtest = 0.75; Joint Fitness NMS incurred a minor evaluation-rate loss from its larger softmax.
  • 3.3.1 Applying Soft-NMS: Fitness NMS can be combined with Soft NMS, yielding an additional 0.7–1.3% MAP improvement.The results indicate that the two methods address tangential problems and can therefore be applied together.

4. Bounding Box Regression

Bounding box regression improves localization but also changes NMS clustering; the paper introduces Bounded IoU Loss to target IoU maximization with suitable convergence properties.

  • The proposed Bounded IoU Loss uses IoU upper bounds to better match IoU maximization while retaining smoothness and robustness for gradient descent.
  • Bounding box regression improves localization across two-stage and single-stage detectors by updating each RoI toward its nearest ground-truth instance.
  • Bounding box regression can improve recall partly by making RoIs from the same instance more likely to satisfy NMS’s IoU comparison.
  • The upper bound for one free parameter estimates maximum attainable IoU while the other box parameters equal their ground-truth values.
  • The proposed positional cost is considerably larger across the operating range, while its width cost is nearly identical to the original R-CNN cost.
  • The Bounded IoU Loss consistently improves MAP across categories and matching thresholds, particularly for DeNet-34 variants.

5. RoI Clustering

Wide DeNet models reduce evaluation rate because they generate many candidate boxes. RoI clustering restores speed by reducing classifications while preserving similar MAP.

  • 67M candidate boxes force wide DeNet models to increase RoIs from 576 to 2304, causing a near 50% evaluation-rate reduction for DeNet-34 wide.
  • Corner clustering and standard NMS clustering reduce the number of RoI classifications and improve evaluation rate by 30% to 80%.
  • Both clustering methods improve MAP@[0.5:0.95] over no clustering, with similar results; standard NMS is slightly slower because of increased CPU load.
  • Corner clustering performs slightly better at high matching IoU and worse at low matching IoUs relative to standard NMS.

6. Input Image Scaling

Larger input images trade evaluation rate for localization accuracy. The paper evaluates DeNet-101 wide across input dimensions without retraining.

  • The 512-pixel input setting improves evaluation rate but costs object localization accuracy relative to larger detector input sizes.
  • DeNet-101 wide is tested from 384 to 1536 pixels with Fitness NMS, Bounded IoU Loss, and corner clustering, without retraining.

7. Conclusion

The paper proposes Fitness NMS and Bounded IoU Loss, combines them with RoI clustering, and reports competitive MSCOCO MAP versus inference time.

  • Fitness NMS addresses a flaw in common detector designs by improving MAP at high localization accuracies without reducing evaluation rate.
  • Bounded IoU Loss better suits IoU maximization while retaining convergence properties suitable for gradient descent.
  • Combining the modifications with RoI clustering yields highly competitive MSCOCO MAP versus inference time.
Loading 1711.00164v3…