Source-linked AI summary
DenseBox: Unifying Landmark Localization with End to End Object Detection
Lichao Huang, Yi Yang, Yafeng Deng, Yinan Yu
TL;DR
Object detection needs a unified approach that can handle challenging objects without the limitations of proposal-based pipelines. DenseBox uses a one-stage fully convolutional network that directly predicts boxes and class confidences, optionally incorporating landmark localization. The paper reports impressive results on face and car detection, while noting that the original system requires several seconds per image.
Problem
R-CNN-style detection uses proposal generation and classification stages, which are difficult to optimize jointly and can struggle with small objects such as faces and far-away cars.
Method
DenseBox is an end-to-end one-stage FCN that directly predicts bounding boxes and class confidences, using hard-negative mining and joint landmark-localization learning.
Results
DenseBox achieves impressive performance on MALF face detection and KITTI car detection, and landmark information further improves detection accuracy.
Takeaways & Limitations
A single fully convolutional detector can handle multiple object scales and heavy occlusion while avoiding proposal generation.
Takeaways & Limitations
The original DenseBox requires several seconds to process one image.
Abstract
from arXiv · showhide
How can a single fully convolutional neural network (FCN) perform on object detection? We introduce DenseBox, a unified end-to-end FCN framework that directly predicts bounding boxes and object class confidences through all locations and scales of an image. Our contribution is two-fold. First, we show that a single FCN, if designed and optimized carefully, can detect multiple different objects extremely accurately and efficiently. Second, we show that when incorporating with landmark localization during multi-task learning, DenseBox further improves object detection accuray. We present experimental results on public benchmark datasets including MALF face detection and KITTI car detection, that indicate our DenseBox is the state-of-the-art system for detecting challenging objects such as faces and cars.
1 Introduction
DenseBox asks whether a single one-stage FCN can perform object detection without proposal generation while handling small and occluded objects. It combines end-to-end detection with hard-negative mining and landmark localization, and reports strong results across face and car benchmarks.
- Motivation: R-CNN improves detection accuracy but struggles with small objects because candidate regions have low resolution and limited context.Its proposal and classification stages also cannot be optimized jointly for end-to-end training.
- Research question: DenseBox investigates how well a one-stage FCN can perform object detection without proposal generation.The detector is designed for end-to-end optimization during training.
- Method: DenseBox is carefully designed to detect objects at small scales and under heavy occlusion.The training procedure uses hard-negative mining to improve detection performance.
- Method: Landmark localization is integrated through joint multi-task learning to further improve object detection accuracy.The authors annotate keypoints for the KITTI car detection dataset to support this evaluation.
- Results: Experiments on MALF face detection and KITTI car detection indicate state-of-the-art performance for faces and cars.The claim concerns a single fully convolutional network designed and optimized for multiple scales and heavy occlusion.
2 Related Work
Object detection evolved from handcrafted sliding-window systems and early neural-network detectors toward CNN-based methods that predict boxes directly. DenseBox fits this progression as a fully convolutional pipeline using image pyramids, convolutional processing, upsampling, and post-processing.
- Pre-CNN detection: Before CNNs, object detectors used handcrafted features extracted at image locations and scales, combined with deformable object models and classifiers.Examples include HOG, SIFT, Fisher Vector, pictorial structures, and deformable part-based models.
- Early neural detectors: Early neural-network face detectors applied convolutional networks to image pyramids in a sliding-window fashion.These systems targeted faces with specific sizes or upright frontal configurations.
- CNN-based detection: Later CNN methods predicted bounding boxes or region proposals directly, including OverFeat, MultiBox, and YOLO.Their outputs reduced reliance on traditional sliding-window processing or supplied proposals for later classification.
- Proposal-based detection: Many state-of-the-art systems rely on R-CNN-style proposal generation followed by region classification, despite newer joint one- or two-stage variants.The related work notes that general proposal methods can produce inferior performance in some detection tasks.
- DenseBox pipeline: DenseBox processes an image pyramid through convolution, pooling, upsampling, and output convolutions before converting feature maps into boxes and applying non-maximum suppression.The pipeline is fully convolutional except for non-maximum suppression.
3 DenseBox for Detection
DenseBox uses a fully convolutional network to predict object confidence and bounding-box geometry at every output location, avoiding region proposals. Its training combines segmentation-like ground-truth maps, multi-scale evaluation, hard-negative mining, masked losses, and a VGG-19-derived architecture.
- DenseBox Detection: Each output pixel predicts an object confidence score and four distances to a bounding box’s corners, then thresholded boxes undergo non-maximum suppression.The network produces a 5-channel output map at one-quarter the input resolution.
- Training and Evaluation: The model trains at a single scale but is applied at multiple scales during evaluation.Large patches containing objects and sufficient background reduce computation spent convolving over background.
- Ground Truth Generation: Training uses 240 × 240 patches resized from object-centered crops, producing 60 × 60 five-channel ground-truth maps.The positive confidence region is a radius-scaled filled circle, while four channels encode distances to the nearest box corners.
- Model Design: DenseBox is initialized from the first 12 convolutional layers of VGG 19 and adds four 1 × 1 convolutional layers for confidence and box regression.The architecture has 16 convolutional layers, with the final 1 × 1 layers acting as sliding-window fully connected layers.
- Multi-Task Training: Hard-negative mining selects badly predicted negative samples through online bootstrapping to improve learning efficiency and reduce noise.Positive and randomly cropped patches are used to explore object-centered and broader negative samples.
- Multi-Task Training: Masked multi-task losses suppress regression on negative samples and ignore gray-zone pixels near positive regions.The regression target is normalized by standard object height, with λloc = 3 reported to work across experiments.
4 Experiments
DenseBox is evaluated on MALF face detection and KITTI car detection, including comparisons with and without landmark localization and against other methods. The experiments show strong detection performance, while landmark benefits vary with annotation coverage and quality.
- DenseBox evaluation covers MALF face detection and KITTI car detection, with comparisons between landmark-assisted and no-landmark models.The experiments also compare DenseBox with current state-of-the-art systems.
- MALF contains 5,000 diverse Internet images, while KITTI includes many small and occluded cars that make detection difficult.KITTI training data contains 51,867 objects, including 28,742 cars.
- KITTI Car Detection Task: 27% landmark annotation on KITTI still improves detection, with the landmark model averaging 0.9% higher average precision.The improvement is smaller than for face detection, where landmark annotations cover all faces and include more landmarks.
- KITTI Car Detection Task: 85.74% average precision is reported for moderate cars, slightly exceeding DeepInsight’s result.DenseBox also defeats Regionlets and spCov by a large margin, while later submissions surpass it.
- Figure 7 shows confidence scores above detections and illustrates strong performance in complex scenes, alongside missed objects and false alarms.The examples cover both the MALF detection set and the KITTI car detection set.
5 Conclusion
DenseBox is presented as a unified end-to-end detection pipeline whose performance can be improved with landmark information. The paper reports impressive face and car detection performance but identifies processing speed as its key problem.
- DenseBox is a unified end-to-end detection pipeline for object detection.
- Landmark information can boost DenseBox performance.
- DenseBox achieves impressive performance on face and car detection tasks, including situations where proposal generation might fail.
- Several seconds per image is the key speed problem for the original DenseBox.The authors state that a later version addresses this issue.