PaperlayerView exact revision on arXiv

Source-linked AI summary

LocNet: Improving Localization Accuracy for Object Detection

Spyros Gidaris, Nikos Komodakis

arXiv:1511.07763v2cs.CVcs.LGcs.NE

TL;DR

Object detectors are often evaluated at IoU 0.5, although higher localization accuracy is needed in applications and is rewarded by COCO’s metric. LocNet estimates object boundaries from row- and column-wise probabilities, and the authors report improved localization results over bounding-box regression.

  • Problem

    Detection is often judged at IoU 0.5, while applications may require higher localization accuracy and COCO evaluates average precision across IoU thresholds from 0.5 to 1.0.

    Source: §1 · p. 1

  • Method

    LocNet predicts row- and column-wise probabilities related to box boundaries and uses them to infer an object’s location within a search region.

    Source: §1 · p. 2, §6 · p. 10

  • Results

    LocNet significantly improves mAP over bounding-box regression at IoU 0.7 and COCO-style metrics, while reporting 78.4% mAP on VOC2007 and 74.78% on VOC2012 at IoU ≥0.5.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The localization module can be incorporated into existing state-of-the-art detectors, and the detection approach achieves high mAP with sliding-window candidate boxes.

    Source: §1 · p. 2, §6 · p. 10

    The paper does not explore an advanced probabilistic model that predicts K > 1 boxes per search region.

    Source: §3.1.2 · p. 4

Abstract

from arXiv · show

We propose a novel object localization methodology with the purpose of boosting the localization accuracy of state-of-the-art object detection systems. Our model, given a search region, aims at returning the bounding box of an object of interest inside this region. To accomplish its goal, it relies on assigning conditional probabilities to each row and column of this region, where these probabilities provide useful information regarding the location of the boundaries of the object inside the search region and allow the accurate inference of the object bounding box under a simple probabilistic framework. For implementing our localization model, we make use of a convolutional neural network architecture that is properly adapted for this task, called LocNet. We show experimentally that LocNet achieves a very significant improvement on the mAP for high IoU thresholds on PASCAL VOC2007 test set and that it can be very easily coupled with recent state-of-the-art object detection systems, helping them to boost their performance. Finally, we demonstrate that our detection approach can achieve high detection accuracy even when it is given as input a set of sliding windows, thus proving that it is independent of box proposal methods.

1. Introduction

LocNet targets localization errors that matter at higher IoU thresholds by using row- and column-wise probabilities to infer object boundaries within a search region. The authors report gains over bounding-box regression and show that the module can be incorporated into existing detection systems.

1. Introduction

  • The adapted CNN architecture reduces top-layer parameters, and the localization module can be incorporated into existing state-of-the-art detectors to improve localization performance.The paper also describes LocNet as scalable with respect to the number of object categories.Source: §1 · p. 2
  • LocNet focuses on post-recognition localization, complementing proposal methods that generate candidate boxes before category-specific recognition.The paper contrasts this with bounding-box regression, which predicts ground-truth box coordinates from a loosely localized candidate.Source: §1 · p. 2
  • The method assigns probabilities to each row and column of a search region, representing whether each is inside the object box or one of its four borders.The model uses these probabilities to infer the object boundaries rather than directly regressing box coordinates.Source: Figure 1 · p. 1, §1 · p. 2
  • LocNet significantly improves mAP over bounding-box regression at IoU 0.7 and COCO-style metrics, while reporting 78.4% mAP on VOC2007 and 74.78% on VOC2012 at IoU ≥0.5.The authors describe the VOC2007 and VOC2012 traditional-metric results as state of the art at the time of writing.Source: §1 · p. 2
  • The authors report high detection accuracy with sliding-window inputs, indicating that their approach can operate independently of box proposal methods if extra computational cost is ignored..Source: §1 · p. 2

2. Object Detection Methodology

The method combines recognition and localization models in an iterative detection pipeline that refines candidate boxes, then merges and post-processes detections.

2. Object Detection Methodology

  • The initial candidate boxes can come from dense sliding windows or category-agnostic bounding-box proposals.Source: §2 · p. 3
  • Each round scores current candidate boxes for likelihood of localization on an object, then generates better-localized candidates for the next round.The recognition and localization models are implemented as convolutional neural networks.Source: §2 · p. 3
  • Candidate boxes generated across iterations are merged, then non-maximum suppression and bounding-box voting produce the final detections.Source: §2 · p. 3
  • LocNet represents localization evidence with row- and column-wise in-out probabilities and conditional probabilities for each possible box boundary.Source: Figure 2 · p. 3

3. Localization model

LocNet predicts row- and column-wise probabilities within a search region rather than directly regressing box coordinates. These distributions support probabilistic box inference and yield faster convergence and higher localization accuracy than bounding-box regression in the reported comparison.

3. Localization model

  • Given a candidate box, the localization workflow enlarges it into a search region and returns a tighter box around an object of interest.Source: §3 · p. 3

3.1. Model predictions

  • For each search-region row and column, the model predicts either inside-box probabilities or probabilities of being one of four borders; opposite-side border probabilities are assumed independent.A row or column counts as inside when any part of its corresponding region lies within the box.Source: §3.1 · pp. 3–4

3.1.1 Bounding box inference

  • Three inference variants maximize the likelihood of in-out elements, borders, or both; ground-truth boxes are projected to integer output coordinates.The combined variant uses both probability distributions.Source: §3.1.1 · p. 4

3.1.2 Discussion

  • Multimodal border distributions may indicate adjacent instances, though predicting K > 1 boxes per region remains unexplored; training requires box annotations rather than pixel masks.The paper notes that a more advanced multi-box probabilistic model could be a future addition.Source: §3.1.2 · pp. 4–5
  • In-Out ML converges faster to higher mAR than bounding-box regression, while retained probability modes can correct an initially wrong border.Figure 3 illustrates the retained correct left-border mode at 5k iterations and recovery of the correct border by 50k.Source: §3.1.2 · p. 4, Figure 4 · p. 5, Figure 3 · p. 5

3.2. LocNet network architecture

  • LocNet crops and pools search-region activations, then uses dimension-specific branches to emit category-specific probability vectors with reduced prediction-layer parameters.The X and Y branches aggregate features across the perpendicular dimension before predicting their assigned probabilities.Source: §3.2 · pp. 5–6, §3.1.2 · p. 5

3.3. Training

  • Training maps category-conditioned search regions to target probabilities with summed binary-logistic losses; balanced border weights are used, while Gaussian-target MSE did not improve performance.The loss weighting balances border and non-border elements, which occur at different frequencies.Source: §3.3 · p. 6

4. Implementation details

LocNet training and evaluation use a shared image scale and fine-tuning setup, with localization samples constructed from proposals and trained using specified optimization settings.

4. Implementation details

  • All models are fine-tuned from conv4_1 upward and use images resized so the smallest dimension is 600 pixels.Training samples use selective-search and Edge Box proposals.Source: §4 · p. 6
  • Localization samples use proposals with IoU of at least 0.4, enlarged 1.8× into search regions and paired with the highest-IoU ground-truth box per category.The output resolution M is 28; training uses SGD with batches of 128 candidate boxes from two images, weight decay 0.00005, and learning rate 0.001.Source: §4 · p. 7
  • Training runs for up to 150k iterations, reducing the learning rate by 10× every 60k iterations, and takes about 1.5 days on one NVIDIA Titan Black GPU.The mini-batch contains 128 training candidate boxes.Source: §4 · p. 7

5. Experimental results

The experiments test localization and detection on VOC2007, then examine sliding-window proposals, preliminary COCO results, and qualitative examples. Across tested recognition models, the proposed localizers improve localization-focused detection metrics over bounding-box regression.

5. Experimental results

  • Models are trained on VOC2007+2012 trainval and tested on VOC2007 test, using Edge Box, Selective Search, or sliding windows and Fast R-CNN or Reduced-MR-CNN recognition.The study uses CNN-based bounding-box regression as its baseline.Source: §5 · p. 7

5.1. Localization performance

  • Above IoU 0.65, the proposed localizers achieve higher recall than bounding-box regression, with the improvement increasing over more iterations and average recall about 6 points higher.Recall is averaged across classes; the initial candidate boxes and recognition model are held constant when comparing localization models.Source: §5.1 · p. 7, Figure 7 · p. 8

5.2. Detection performance

  • The proposed localizers improve VOC2007 mAP by an average of 7 points for IoU ≥0.7 and COCO-style mAP, and by around 0.7 points for IoU ≥0.5 versus bounding-box regression.Gains become clear around IoU 0.65 and widen through 0.9; results use four pipeline iterations. On VOC2012, the reported mAP is 74.8% for IoU ≥0.5, described as state of the art at the time of writing.Source: §5.2 · pp. 7, 9, Figure 8 · p. 8, Table 2 · p. 9

5.3. Sliding windows as initial set of candidate boxes

  • With 10k sliding windows per image, the full pipeline reaches mAP close to proposal-based approaches, including at IoU ≥0.7 and COCO-style mAP.Recognition-only scoring performs much worse with sliding windows, while localization and rescoring reduce that gap; the localization models’ advantage over regression is greater in this setting.Source: §5.3 · pp. 9–10, Table 4 · p. 10

5.4. Preliminary results on COCO

  • On COCO, Combined ML improves mAP by 5 points for COCO-style evaluation, 8 points at IoU ≥0.75, and 1.4 points at IoU ≥0.5.These are preliminary results from 320k training iterations without further parameter tuning; the COCO Fast R-CNN recognition model uses a softmax classifier.Source: §5.4 · p. 10, Table 5 · p. 10

5.5. Qualitative results

  • The qualitative comparison shows candidate boxes, ground-truth boxes, and predictions from bounding-box regression and three proposed localization models.Figure 10 uses yellow for candidates, green for ground truth, and blue for predictions.Source: §5.5 · p. 10

6. Conclusion

The paper concludes that row- and column-conditioned probabilities support accurate bounding-box inference, and that LocNet improves detection performance, including with sliding-window candidates.

6. Conclusion

  • The method assigns localization probabilities to rows and columns of a search region, using them to infer object boundaries accurately.The probabilities provide information about the object’s location within the region.Source: §6 · p. 10
  • On VOC2007, LocNet outperforms CNN-based bounding-box regression across evaluation metrics, with significant gains on metrics that reward good localization.The localization module can be plugged into existing state-of-the-art detectors to boost performance.Source: §6 · p. 10
  • The detection approach achieves high mAP even when its initial candidates are sliding windows.Figure 10 compares regression and In-Out ML, Borders ML, and Combined ML predictions against candidate and ground-truth boxes.Source: §6 · p. 10, Figure 10 · p. 11

A. Bounding box regression model

The appendix describes the CNN-based bounding-box regression baseline as four scalar functions that transform a candidate box toward a category-specific ground-truth box.

A. Bounding box regression model

  • Four CNN-based scalar functions, f_x, f_y, f_w, and f_h, predict geometric-transformation coefficients from candidate box B and category c.The resulting transformation is intended to map B to a ground-truth box for an object of category c.Source: A. Bounding box regression model · p. 13
  • The candidate box is represented by its top-left coordinates and width and height, and the predicted box is obtained from the predicted transformation coefficients.The passage introduces predicted target values T = {t_x, t_y, t_w, t_h}, but the supplied text does not specify their definitions.Source: A. Bounding box regression model · p. 13
  • The regression CNN adopts the architecture proposed in and is trained with the sum of Euclidean distances between target and predicted values.Its final fully connected layer is initialized from a Gaussian distribution with standard deviation 0.01; other training details follow those of the proposed localization models.Source: A. Bounding box regression model · p. 13

B. Training the localization models

The section reports a training benefit for Fast R-CNN recognition and box regression, while the proposed localization models are not yet trained jointly with recognition. It also identifies the evaluation setup reported in Table 6.

B. Training the localization models

  • Table 6 reports VOC2007 test mAP using the Reduced-MR-CNN recognition model and Edge Box initial boxes.The caption gives the evaluation scope but no result values.Source: Table 6 · p. 13
  • Joint training improves the accuracy of both the Fast R-CNN recognition model and its bounding box regression model.The two models share convolutional layers during training.Source: B. Training the localization models · p. 13
  • The proposed Borders ML, In-Out ML, and Combined ML models are not currently trained simultaneously with recognition.The authors expect joint training could further improve overall performance and plan to explore it.Source: B. Training the localization models · p. 13

C. Region activation maps size in bounding box regression model

The authors tested larger region activation maps for bounding box regression but found negligible difference from the 7x7 setting. LocNet uses 14x14 maps with far fewer parameters than the enlarged regression model.

C. Region activation maps size in bounding box regression model

  • The authors enlarged regression maps from 7x7 to 14x14 to try to improve VGG16-based box localization.They initialized fc6 by bilinearly up-sampling its ImageNet-pretrained 7x7 weights for the larger map.Source: C. Region activation maps size in… · p. 13
  • The 14x14 regression model had around 422M parameters, and its difference from the 7x7 case was negligible.The authors therefore did not use this model.Source: C. Region activation maps size in… · p. 13
  • LocNet's 14x14x512 region activation maps use 26M parameters for InOut and 34M for Borders, almost 15 times fewer than the 14x14 regression case.The LocNet extra layers required no explicit initialization care.Source: C. Region activation maps size in… · p. 13

D. Recognition models

The recognition-model implementations adapt MR-CNN to reduce its region and parameter load, and modify Fast R-CNN's recognition and regression branches. The reported Fast R-CNN changes improved detection performance.

D. Recognition models

  • The reduced MR-CNN implementation keeps six of ten regions, omits semantic-segmentation-aware CNN features, and reduces parameters in the region adaptation modules.The model is based on the MR-CNN system.Source: D. Recognition models · pp. 13–14
  • Reduced MR-CNN decomposes fc6 and fc7 into smaller fully connected layers, applies truncated SVD, copies the resulting parameters across region modules, and continues training.The fc6 and fc7 decompositions use intermediate dimensions 1024 and 256, respectively.Source: D. Recognition models · p. 14
  • The reimplemented Fast R-CNN separates recognition and regression branches after conv5_3, enlarges regression inputs by α = 1.3, and trains linear SVMs on recognition features.The authors report that their implementation changes improved detection performance.Source: D. Recognition models · p. 14

E. Object detection pipeline

After the first scoring pass, the detection pipeline prunes low-confidence candidate boxes and applies non-maximum suppression. Category-agnostic initial boxes can be processed across categories with shared computation.

E. Object detection pipeline

  • After the first scoring pass, the pipeline thresholds candidates to about 18 boxes per image and category to reduce later computation.The candidate boxes are pruned separately for each category.Source: E. Object detection pipeline · p. 14
  • The pruning step uses non-maximum suppression at IoU 0.95, or 0.85 when sliding windows generate the initial boxes.Source: E. Object detection pipeline · p. 14
  • Because category-agnostic proposal boxes are the same across categories, recognition computation on them can be shared during the first iteration.Source: E. Object detection pipeline · p. 14
Loading 1511.07763v2…