PaperlayerView exact revision on arXiv

Source-linked AI summary

Hierarchical Object Detection with Deep Reinforcement Learning

Miriam Bellver, Xavier Giro-i-Nieto, Ferran Marques, Jordi Torres

arXiv:1611.03718v2cs.CVcs.LG

TL;DR

The paper addresses object detection with many candidate regions by guiding a top-down search through a predefined hierarchy using deep reinforcement learning. It compares overlapping versus non-overlapping candidates and region-specific versus shared convolutional features, finding better results for overlap and independently extracted features, while predefined regions limit recall.

  • Problem

    Object detection commonly analyzes many region proposals, motivating a method that can search hierarchically with far fewer visited regions.

    Source: §1 · p. 1, §3 · p. 3

  • Method

    A deep reinforcement-learning agent selects among five predefined subregions while comparing overlapping and non-overlapping hierarchies and two feature-extraction strategies.

    Source: §3.1 · p. 3

  • Results

    The overlapping strategy performs better, and Image-Zooms outperforms Pool45-Crops; qualitative results show successful localization in a few steps.

    Source: §4.2 · pp. 6–7, §4.1 · p. 6

  • Takeaways & Limitations

    Few proposals from an appropriate hierarchy can support detection, making region-specific feature extraction worth considering when the agent visits relatively few regions.

    Source: §5 · p. 8

    Predefined regions clearly limit recall because many ground-truth objects do not fit the considered proposals.

    Source: §5 · p. 8

Abstract

from arXiv · show

We present a method for performing hierarchical object detection in images guided by a deep reinforcement learning agent. The key idea is to focus on those parts of the image that contain richer information and zoom on them. We train an intelligent agent that, given an image window, is capable of deciding where to focus the attention among five different predefined region candidates (smaller windows). This procedure is iterated providing a hierarchical image analysis.We compare two different candidate proposal strategies to guide the object search: with and without overlap. Moreover, our work compares two different strategies to extract features from a convolutional neural network for each region proposal: a first one that computes new feature maps for each region proposal, and a second one that computes the feature maps for the whole image to later generate crops for each region proposal. Experiments indicate better results for the overlapping candidate proposal strategy and a loss of performance for the cropped image features due to the loss of spatial resolution. We argue that, while this loss seems unavoidable when working with large amounts of object candidates, the much more reduced amount of region proposals generated by our reinforcement learning agent allows considering to extract features for each location without sharing convolutional computation among regions.

1 Introduction

The paper proposes top-down hierarchical scanning guided by reinforcement learning, focusing sequentially on informative image regions. It compares region-overlap designs and region-specific versus shared convolutional features.

  • The method replaces independent sliding-window analysis with a top-down hierarchy that exploits relationships among image regions.Source: §1 · pp. 1–2
  • A reinforcement-learning agent selects among five predefined regions—four quadrants and a center—and stops when it detects an object.The hierarchy supports either overlapping or non-overlapping candidates, with detections constrained to predefined regions.Source: §1 · p. 2
  • Image-Zooms computes features separately for visited regions, whereas Pool45-Crops reuses whole-image feature maps for region crops.The reduced number of agent-selected regions makes higher-resolution region-specific descriptors computationally practical.Source: §1 · p. 2
  • The paper reports better results for overlapping candidates and for region-specific features, while arguing that the latter's computational overhead is minor.Source: §1 · p. 2

2 Related Work

Prior object-detection systems use many proposals, shared convolutional features, anchors, or iterative box refinement. This work instead combines reinforcement learning with a fixed top-down hierarchy and a smaller set of analyzed regions.

  • Traditional proposal and sliding-window methods process many regions independently, making proposal extraction and feature computation expensive.Source: §2 · p. 2
  • CNN-based detectors reduce feature cost by sharing full-image convolutional computation across region proposals.Source: §2 · p. 2
  • Anchor-based methods facilitate bounding-box prediction, while refinement methods iteratively adjust or deform predictions toward target objects.Source: §2 · pp. 2–3
  • Unlike Active Object Localization, this approach imposes a fixed hierarchy in which each action zooms into a predicted region of interest.Source: §2 · p. 3
  • Compared with hierarchical subdivision that analyzes multiple selected subregions, this method analyzes fewer regions and can extract high-quality descriptors for each.Source: §2 · p. 3

3 Hierarchical Object Detection Model

The model casts object detection as a Markov decision process in which an agent navigates a predefined image hierarchy. It uses IoU-based rewards, deep Q-learning, and two visual-feature extraction variants.

3 Hierarchical Object Detection Model

  • The agent treats detection as sequential decision-making over image regions, interacting with the image to find objects in few steps.Source: §3 · p. 3

3.1 MDP formulation

  • Each state combines the current region descriptor with a 24-dimensional memory vector encoding the previous four actions.Source: §3.1 · p. 3
  • The hierarchy offers five movement regions—four quarters and a central overlap—plus a terminal action, with overlapped and non-overlapped quarter variants.Source: §3.1 · p. 3, Figure 1 · p. 4, Figure 2 · p. 4
  • Movement rewards favor regions with higher IoU than the previous region, while termination is rewarded when IoU reaches τ = 0.5.The reward magnitude parameter is η = 3.Source: §3.1 · pp. 3–4

3.2 Q-learning

  • Q-learning selects actions through a Bellman update, with the Q-function approximated by a deep Q-network.Source: §3.2 · p. 4

3.3 Model

  • Image-Zooms computes Pool5 descriptors from independently resized regions, whereas Pool45-Crops reuses whole-image Pool4 or Pool5 maps through ROI pooling.The selected feature-map depth depends on region scale: deeper maps serve large objects and shallower maps smaller ones.Source: §3.3 · p. 5
  • Both feature variants produce 7x7 maps for a common network whose output represents six possible agent actions.Source: §3.3 · p. 5

3.4 Training

  • Training uses ε-greedy exploration, forced termination when IoU exceeds 0.5, experience replay, and γ = 0.90 discounting.The agent can change target objects during exploration by focusing on the highest-overlap object in the current region.Source: §3.4 · pp. 5–6

4 Experiments

Experiments evaluate the agent on PASCAL VOC aeroplane images, combining qualitative searches, precision–recall analyses, model comparisons, and region-count measurements. The results favor overlapping regions and Image-Zooms features, while most detectable objects are found within a few steps.

4 Experiments

  • On PASCAL VOC, the experiments train on 2007/2012 trainval data and test on the 2007 test set for the aeroplane class.Source: §4 · p. 6

4.1 Qualitative results

  • Image-Zooms with overlapping regions successfully zooms toward objects, including small instances, and often selects their bounding boxes within two or three steps.Source: §4.1 · p. 6, Figure 7 · p. 8

4.2 Precision-Recall curves

  • More than half of ground-truth objects cannot be detected even by the best upper bound because they do not fit the predefined region proposals.Source: §4.2 · p. 6
  • The overlapping-region scheme achieves higher precision and recall than the non-overlapping scheme for both upper-bound and random agents.Source: §4.2 · pp. 6–7, Figure 5 · p. 7
  • Image-Zooms outperforms Pool45-Crops, which the authors associate with lower spatial resolution after ROI pooling in the cropped-feature model.Source: §4.2 · p. 7, Figure 6 · p. 7
  • Image-Zooms performance improves across training epochs in the plotted precision–recall curves.Source: §4.2 · p. 7, Figure 7 · p. 8

4.3 Number of regions analyzed per object

  • Most objects are found in one step, and fewer than three steps nearly cover all objects detectable by the agent.Source: §4.3 · p. 8, Figure 8 · p. 8

5 Conclusions

The paper presents deep reinforcement learning for top-down exploration of a hierarchy of image regions in object detection. Experiments show that few proposals can suffice with an appropriate hierarchy, while predefined regions and cropped convolutional features impose limitations.

  • Deep reinforcement learning guides top-down exploration through a hierarchy of regions for object detection.Source: §5 · p. 8
  • Objects can be detected with very few proposals from an appropriate hierarchy, but predefined regions clearly limit recall.Source: §5 · p. 8
  • Cropped convolutional features have limitations, especially for small objects.Source: §5 · p. 8
  • The work is publicly available for reproducibility and extension.Source: §5 · p. 8
Loading 1611.03718v2…