Source-linked AI summary
Dynamic Zoom-in Network for Fast Object Detection in Large Images
Mingfei Gao, Ruichi Yu, Ang Li, Vlad I. Morariu, Larry S. Davis
TL;DR
High-resolution images make CNN detection costly in time and GPU memory, while downsampling can obscure small objects. The paper uses reinforcement learning to guide coarse-to-fine zooms with R-net and Q-net. It reports fewer processed pixels and lower detection time with negligible accuracy loss on Caltech, and larger efficiency reductions on high-resolution YFCC100M data.
Problem
Applying CNN detectors directly to high-resolution images requires substantial processing time and GPU memory, while downsampling can make small objects too small to detect.
Source: §1 · p. 1Method
The method runs coarse detection on a downsampled image, then uses R-net-predicted accuracy gains and Q-net sequential selection to choose higher-resolution regions.
Source: §1 · p. 2, §3.1 · p. 3Results
Over 50% fewer processed pixels and 25% lower average detection time on Caltech with negligible accuracy drop; on high-resolution YFCC100M, about 70% fewer pixels and over 50% lower average detection time.
Source: §1 · p. 2Takeaways & Limitations
Abstract
from arXiv · showhide
We introduce a generic framework that reduces the computational cost of object detection while retaining accuracy for scenarios where objects with varied sizes appear in high resolution images. Detection progresses in a coarse-to-fine manner, first on a down-sampled version of the image and then on a sequence of higher resolution regions identified as likely to improve the detection accuracy. Built upon reinforcement learning, our approach consists of a model (R-net) that uses coarse detection results to predict the potential accuracy gain for analyzing a region at a higher resolution and another model (Q-net) that sequentially selects regions to zoom in. Experiments on the Caltech Pedestrians dataset show that our approach reduces the number of processed pixels by over 50% without a drop in detection accuracy. The merits of our approach become more significant on a high resolution test set collected from YFCC100M dataset, where our approach maintains high detection performance while reducing the number of processed pixels by about 70% and the detection time by over 50%.
1. Introduction
High-resolution images make CNN detection costly in processing time and GPU memory, while downsampling can make small objects too small to detect. The paper proposes sequential, reinforcement-learning-guided zooms, reporting reduced processing with negligible accuracy loss.
1. Introduction
- High-resolution images demand substantial processing time and produce convolution maps too large for current GPU memory.The introduction contrasts these images with standard, relatively low-resolution vision datasets.Source: §1 · p. 1
- Existing options have trade-offs: architecture simplification may not generalize, tiling remains costly, downsampling can hide small objects, and proposal-based recall requires many proposals.The cited proposal burden is especially severe for small objects in large images.Source: §1 · pp. 1–2
- The method first detects objects in a downsampled image, then uses reinforcement learning to select higher-resolution regions based on predicted accuracy gains and prior zooms.R-net predicts region-level gains; Q-net selects zoom locations and scales.Source: §1 · p. 2
- Over 50% fewer processed pixels and 25% lower average detection time on Caltech, with negligible accuracy drop; on YFCC100M, about 70% fewer pixels and over 50% lower average detection time.The YFCC100M dataset contains pedestrians of varied sizes.Source: §1 · p. 2
2. Related work
Prior work improves detector efficiency through architectural changes or sequential region search, while contextual cues and reinforcement learning offer additional strategies. The paper situates its approach among these lines of work, emphasizing computation-aware search.
2. Related work
- Detector improvements such as feature sharing and single-shot designs aid efficiency, but single-shot detectors generalize less well to large images with variable-size objects and incur rising costs as images grow.The cited methods perform well on relatively low-resolution datasets with generally large objects.Source: §2 · p. 2
- Sequential search can avoid processing the whole image, but prior approaches often prioritize accuracy over computation; cited methods add about 5 seconds per object class or over 15% detection-time overhead.These overhead examples are reported for Alexe et al. and Zhang et al., respectively.Source: §2 · p. 2
- Scene-segmentation context has been explored for localization, but this paper uses coarse detections alone to guide search and leaves added context for future work.Another prior coarse-to-fine method speeds detection without sequentially selecting promising regions.Source: §2 · p. 2
- Prior reinforcement-learning search methods can require many steps or run slowly, and the cited active detector does not explicitly price selection cost.The paper follows a prior cost-sensitive approach that combines accuracy and cost in the reward.Source: §2 · p. 2
3. Dynamic zoom-in network
The network uses coarse detection to guide sequential high-resolution zoom-ins, learning both where accuracy may improve and which region to inspect next. Its state and action selection combine predicted gains with cost and prior zoom history.
3. Dynamic zoom-in network
- Coarse detection on a down-sampled image guides sequential high-resolution analysis of promising regions, with reinforcement learning modeling long-term accuracy under limited cost.The framework uses an R-net to estimate potential gain and a Q-net to select regions to inspect.Source: §3 · pp. 2–3
3.1. Problem formulation
- The MDP treats each zoom region as an action and represents state with predicted gains plus analyzed-region history; its reward trades accuracy improvement against added pixel cost.The Q-net learns a long-term cost-aware reward from immediate rewards.Source: §3.1 · pp. 3–4
3.2. Zoom-in accuracy gain regression network
- R-net learns coarse-to-fine detection correlations from paired proposals and estimates proposal gains; its AG map zeros regions already analyzed.Coarse and high-resolution proposals are matched when their intersection over union exceeds 0.5; high-resolution detection does not always improve accuracy because detectors handle different object-size ranges.Source: §3.2 · p. 4
3.3. Zoom-in Q function learning network
- Q-net maps the AG state to long-term action rewards for zoom regions of different sizes, with actions from all maps competing in one learned action set.The Q-net is trained to learn a long-term cost-aware reward function.Source: §3.3 · pp. 4–5
- Because candidate zoom windows are sparsely sampled, the Refine module locally adjusts the Q-net’s selected window toward a better location on the accuracy-gain map.The refinement searches a local area around the Q-net output.Source: §3.3 · p. 5
4. Experiments
Experiments evaluate the framework on CPD and a 100-image, high-resolution Web Pedestrian dataset collected from YFCC100M, which contains densely distributed pedestrians of varied sizes.
4.1. Baseline methods
- Comparisons use Fine-detection-all as the high-accuracy, high-cost reference, alongside all-coarse without zooming, greedy search, entropy-based gains, and SSD/YOLOv2.The entropy baseline replaces the R-net output while holding the rest of the pipeline fixed.Source: §4.1 · p. 5
4.2. Variants of our framework
- Variants isolate region refinement (Qnet*), a fully connected Q-net (Qnet-FC), and an R-net reward without an explicit cost term (Rnet*).Qnet-FC uses a fixed number of outputs, with window sizes proportionally increased for WP.Source: §4.2 · p. 6
4.3. Evaluation metric
- Aperc measures AP as a percentage of Fine-detection-all, while Pperc and Tperc measure processed pixels and detection time as percentages of that baseline..Source: §4.3 · p. 6
4.4. Implementation details
- Experiments use images downsampled by a factor of two, apply fine detection only to zoomed regions, and use Faster R-CNN coarse and fine detectors.Q-net samples two window sizes with half-window strides; refinement locally shifts windows, and action selection stops when summed AG-map values fall below 0.1.Source: §4.4 · p. 6
4.5. Qualitative results
- Refinement repairs windows that split pedestrians; Q-net may accept near-term suboptimal zooms for fewer long-term steps, while R-net discourages zooms where fine detection worsens results.Sparse sampling can leave optimal regions uncovered, especially when windows are small relative to the image.Source: §4.5 · p. 6, Figure 3 · p. 6, Figure 4 · p. 7
4.6. Quantitative evaluation
- Over 50% fewer processed pixels with comparable or better accuracy on CPD; on WP, the best variant reduces pixels by over 60% while retaining 97% of Fine-detection-all accuracy.The framework improves Coarse-detection-all accuracy by about 35% on CPD; detection time falls by 25% on CPD and 50% on WP.Source: §4.6 · p. 7, Figure 6 · p. 7, Table 2 · p. 8
4.7. Ablation analysis
- Refinement improves accuracy at fixed costs, especially on WP; fully connected Q-net uses CPD driving-view priors but drops significantly on WP, while the cost term matters most on CPD at low pixel budgets.Without explicit cost penalization, the method often chooses the largest zoom regions; on WP, relatively small windows make larger zooms beneficial, and the unpenalized variant performs better than other variants.Source: §4.7 · p. 8
5. Conclusion
The paper proposes a dynamic zoom-in network that speeds detection in large images without changing the underlying detector. It first predicts the accuracy gain of zooming into regions, then selects regions sequentially for fine detection; experiments report effectiveness on two pedestrian datasets.
- The dynamic zoom-in network speeds object detection in large images without manipulating the underlying detector’s structure.Source: §5 · p. 8
- Images are downsampled first, and R-net predicts the accuracy gain of zooming into a region before Q-net selects regions for fine detection.Source: §5 · p. 8
- Experiments show the method is effective on the Caltech Pedestrian Detection dataset and a high-resolution pedestrian dataset.Source: §5 · p. 8