Source-linked AI summary
Deep convolutional neural networks for pedestrian detection
Denis Tomè, Federico Monti, Luca Baroffio, Luca Bondi, Marco Tagliasacchi, Stefano Tubaro
TL;DR
Pedestrian detection remains an open and important challenge for automotive, surveillance, and robotics applications. This paper adapts a general-purpose convolutional network, optimizes the detection pipeline, and reports accuracy near the state of the art with low computational time. A lightweight implementation is also tested in real time on an NVIDIA Jetson TK1.
Problem
Pedestrian detection remains an open accuracy challenge despite its importance for automotive, surveillance, and robotics applications.
Method
The system adapts a general-purpose convolutional network to pedestrian detection and optimizes most stages of its detection pipeline.
Results
The optimized system outperforms traditional handcrafted-feature and deep-learning approaches while approaching state-of-the-art detection accuracy.
Takeaways & Limitations
The lightweight version supports real-time pedestrian detection on modern hardware, including validation on an NVIDIA Jetson TK1 development board.
Takeaways & Limitations
The approach depends partly on ImageNet-pretrained models because pedestrian-detection datasets of comparable annotated scale are not publicly available.
Abstract
from arXiv · showhide
Pedestrian detection is a popular research topic due to its paramount importance for a number of applications, especially in the fields of automotive, surveillance and robotics. Despite the significant improvements, pedestrian detection is still an open challenge that calls for more and more accurate algorithms. In the last few years, deep learning and in particular convolutional neural networks emerged as the state of the art in terms of accuracy for a number of computer vision tasks such as image classification, object detection and segmentation, often outperforming the previous gold standards by a large margin. In this paper, we propose a pedestrian detection system based on deep learning, adapting a general-purpose convolutional network to the task at hand. By thoroughly analyzing and optimizing each step of the detection pipeline we propose an architecture that outperforms traditional methods, achieving a task accuracy close to that of state-of-the-art approaches, while requiring a low computational time. Finally, we tested the system on an NVIDIA Jetson TK1, a 192-core platform that is envisioned to be a forerunner computational brain of future self-driving cars.
1. Introduction
Pedestrian detection remains a challenging computer-vision problem, motivating a carefully optimized deep-learning pipeline that targets both accuracy and computational efficiency.
- Motivation: Pedestrian detection is difficult despite decades of research and is important for applications requiring rapid visual interpretation.The paper emphasizes automotive, surveillance, robotics, and time-sensitive scene analysis.
- Motivation: Deep-learning performance can depend strongly on small parameter choices, so the paper thoroughly analyzes the existing pedestrian-detection pipeline.The work builds upon Hosang et al. and dissects each pipeline stage.
- Contributions: The paper optimizes most pipeline stages and proposes novel solutions intended to significantly improve detection accuracy.These optimizations form the central technical contribution.
- Contributions: The proposed system approaches state-of-the-art detection accuracy while outperforming traditional handcrafted-feature methods and deep-learning approaches.The comparison is reported as a paper-level performance claim.
- Contributions: A lightweight version runs in real time on modern hardware and is validated on an NVIDIA Jetson TK1 platform.The Jetson TK1 is presented as a compact GPU-based platform relevant to future automotive systems.
2. Background on pedestrian detection and Convolutional Neural Networks
Pedestrian detectors commonly transform raw images into candidate regions, features, and classifications. Convolutional neural networks instead learn visual features and a classifier directly from pixel-level content.
- Pedestrian detection pipeline: A common pedestrian-detection pipeline converts a scene into pedestrian bounding boxes through region proposal, feature extraction, and region classification.The pipeline starts from raw pixels and ends with bounding boxes of varying sizes.
- Region proposal: Region proposal analyzes the full frame to generate candidate regions and can reduce computation by filtering likely negative regions before later stages.Pedestrian-specific proposals aim to discard negatives while preserving positives.
- Feature extraction: Feature extraction maps each candidate region to a compact feature vector representing its visual characteristics.Earlier methods use visual cues such as intensity contrast, pooled gradients, or channel transformations.
- Region classification: Classification uses each region’s feature vector to assign a binary label indicating whether it contains a pedestrian.Historical classifiers include AdaBoost and Support Vector Machines.
- Convolutional Neural Networks: CNNs learn filters and a classifier from pixel-level image content through stacked operations including filtering, normalization, activation, and pooling.This contrasts with handcrafted features designed from human intuitions about image structure.
3. Optimizing deep convolutional networks for pedestrian detection
The paper optimizes a deep convolutional pedestrian-detection pipeline by refining region proposals, training data, preprocessing, and classifier inputs. These changes target both accuracy and computational efficiency.
- Region proposal: Candidate-region selection can reduce the windows passed to later stages by up to three orders of magnitude.This reduction is especially important because convolutional feature extraction is computationally intensive.
- Region proposal: The pipeline evaluates sliding windows, Selective Search, and LDCF as alternative region-proposal strategies.Sliding windows provide 100 percent recall but many regions; Selective Search reduces processing burden, while LDCF confidence thresholds trade precision against recall.
- Network training: The system fine-tunes ImageNet-trained convolutional networks using annotated positive and negative pedestrian regions.The training procedure adapts both network weights and the classifier to pedestrian detection.
- Data preprocessing and augmentation: Padding expands imprecise proposals so ground-truth pedestrian boxes are fully contained, and random crops simulate proposal uncertainty during training.The padding reference is obtained from the mean of the measured padding distribution.
- Data preprocessing and augmentation: Negative-sample decorrelation selects diverse training regions using quantized color histograms and distances between cumulative color distributions.The procedure repeatedly removes the region with the highest average distance until K negative regions are selected.
- Region proposal scores: Region-proposal scores can be supplied as an additional feature to the final classifier, using proposal and neural-network scores in parallel.This differs from the serial design in which only regions above a score threshold proceed to later stages.
4. Experimental evaluation and results
The experiments evaluate a CNN-based pedestrian-detection pipeline on the Caltech dataset, using standardized miss-rate/FPPI metrics and progressively optimized region proposals and network fine-tuning. The final DeepPed system combines LDCF proposals with a fine-tuned CNN, achieving strong detection accuracy and substantially lower desktop processing time than CCF.
- Datasets: The evaluation uses the Caltech Pedestrian Dataset, whose standard protocol separates six training sessions from five test sessions and resamples correlated test frames.The dataset contains approximately 10 hours of urban-traffic video, with 640 × 480 resolution at 30fps; approximately 250k frames are annotated with 350k bounding boxes.
- Evaluation Metrics: Performance is measured by miss rate and false positives per image, with particular interest in miss rate at 0.1 FPPI as a real-world operating condition.A detection matches ground truth when intersection-over-union exceeds 50%; miss rate is the ratio of false negatives to positive examples.
- Experimental Setup: The experiments compare sliding windows, Selective Search, and LDCF region proposals with AlexNet-Pedestrian and original AlexNet, using fixed-size 227 × 227 input regions.Sliding windows use a minimum height of 50 pixels, maximum height of 100 pixels, and an approximately 2:1 aspect ratio; LDCF acts as a coarse negative-region filter.
- Region Proposals: Selective Search cannot achieve miss rate below 0.5 because its proposals fail to cover half the positive bounding boxes, whereas sliding windows provide higher recall but overwhelm the classifier.LDCF offers good region-proposal performance, although the initial CNN configuration does not detect pedestrians effectively.
- Optimization: Fine-tuning adapts the object-detection networks to pedestrian detection through six-fold cross-validation, and the final DeepPed system combines LDCF with AlexNet-Pedestrian or GoogLeNet-Pedestrian.The final system also incorporates the proposed pipeline optimizations and evaluates positive and negative training regions in depth.
- Results: 0.197 miss rate: GoogLeNet-Pedestrian slightly outperforms AlexNet-Pedestrian at 0.199, while DeepPed beats several traditional and sophisticated competing algorithms.On the desktop setup, DeepPed processes a 640 × 480 frame in 530ms, compared with more than 3 seconds for CCF.
5. Conclusions
The paper presents a CNN-based pedestrian detection system that outperforms handcrafted- and learned-feature alternatives at reasonable computational complexity. A lightweight version detects pedestrians in real time on modern hardware and was tested on a smart-car development board.
- The proposed pedestrian detection system outperforms alternatives based on handcrafted and learned features.
- The system achieves this performance at a reasonable computational complexity.
- A lightweight version detects pedestrians in real time on modern hardware.
- The authors tested the system on a development board envisioned as the computational brain of smart cars.
- Future work targets LDCF optimization and an entirely GPU-based pipeline to avoid expensive memory copies and improve performance.