Source-linked AI summary
Strengthening the Effectiveness of Pedestrian Detection with Spatially Pooled Features
Sakrapee Paisitkriangkrai, Chunhua Shen, Anton van den Hengel
TL;DR
Pedestrian detection remains challenging because target appearance, lighting, pose, and occlusion vary across applications. The paper introduces spatially pooled low-level features, combines them with existing features, and optimizes performance in the practically important false-positive range. The resulting detector outperforms prior reported results across four major pedestrian benchmarks, while the improvements increase computational complexity.
Problem
Pedestrian detection is difficult because target appearance, lighting, pose, and occlusion vary, despite its importance in surveillance, robotics, and human-computer interaction.
Method
The paper extracts low-level visual features with spatial pooling, combines them with existing features, and optimizes the log-average miss rate over the practical false-positive range.
Results
The combined features with pAUC struct outperform previous best results on four major pedestrian detection benchmarks, including an INRIA miss rate of 11.1%.
Takeaways & Limitations
The approach outperforms all competitors on the standard benchmark datasets and shows that feature choice and implementation details are crucial for detection performance.
Takeaways & Limitations
The improvement increases computational complexity through additional feature-extraction and related processing costs.
Abstract
from arXiv · showhide
We propose a simple yet effective approach to the problem of pedestrian detection which outperforms the current state-of-the-art. Our new features are built on the basis of low-level visual features and spatial pooling. Incorporating spatial pooling improves the translational invariance and thus the robustness of the detection process. We then directly optimise the partial area under the ROC curve (\pAUC) measure, which concentrates detection performance in the range of most practical importance. The combination of these factors leads to a pedestrian detector which outperforms all competitors on all of the standard benchmark datasets. We advance state-of-the-art results by lowering the average miss rate from $13\%$ to $11\%$ on the INRIA benchmark, $41\%$ to $37\%$ on the ETH benchmark, $51\%$ to $42\%$ on the TUD-Brussels benchmark and $36\%$ to $29\%$ on the Caltech-USA benchmark.
1 Introduction
Pedestrian detection is difficult because target appearance, lighting, pose, and occlusion vary, motivating robust feature and classifier design. The paper combines spatially pooled low-level features, multiple feature types, and practical-range metric optimization to improve benchmark performance.
- Pedestrian detection supports surveillance, robotics, and human-computer interaction but is challenged by variation in appearance, lighting, pose, and occlusion.
- Combining multiple features can significantly improve pedestrian detection performance, according to a recent literature survey.
- Spatial pooling creates a new low-level feature type, while combining it with channel features further improves detection performance.
- The detector outperforms all reported pedestrian detectors, including LatSVM, ConvNet, and DBN-Mut.
- The paper optimizes log-average miss rate over the practical range of 0.01 to 1 FPPI rather than the complete false-positive range.
- Experiments report state-of-the-art performance on the INRIA, ETH, TUD-Brussels, and Caltech-USA pedestrian benchmarks.
2 Our approach
The approach combines hand-crafted low-level descriptors with spatial pooling and boosted decision trees, while tuning training choices to improve robustness and detection performance.
- Detector framework: The framework combines spatially pooled covariance and LBP features with LUV colour features and builds on a sliding-window boosted detector.The paper also describes bootstrapping hard negatives and modifications to the baseline detector framework.
- Spatially pooled features: The covariance descriptor uses a 9-dimensional feature image and stores 45 values from each symmetric 9 × 9 covariance matrix.The input includes pixel locations, intensity derivatives, gradient magnitude, and two edge-orientation features.
- Spatially pooled features: Spatial pooling summarizes descriptors over local regions, improving invariance to translation and deformation while discarding irrelevant detail.The method applies max-pooling to covariance and LBP descriptors extracted from image patches.
- Spatially pooled features: Multi-scale patches at 8 × 8, 16 × 16 and 32 × 32 pixels expand feature richness and capture human body parts at different scales.The implementation uses 4 × 4 pixel pooling regions and a 4-pixel pooling stride.
- Detector design: Decision trees deeper than stumps improve performance, especially on ETH, and depth three is selected for good generalization and faster training than depth four.Decision stumps can underfit by failing to separate pedestrian from non-pedestrian patches.
3 Experiments
Experiments evaluate the proposed detectors across benchmark datasets, feature combinations, pAUC struct tuning, and computational cost. The results show improved detection performance, including state-of-the-art benchmark results, alongside increased computational complexity.
- Experimental setup: Two detectors are trained: an INRIA-trained model for INRIA, ETH, and TUD-Brussels, and a Caltech-USA-trained model for Caltech-USA.Evaluation uses the reasonable subset, with pedestrians at least 50 pixels tall and at least 65% visible.
- Improved covariance descriptor: 136-channel sp-Cov features use modified covariance descriptors and decision-tree weak learners, avoiding the computationally expensive Riemannian covariance-distance approach.The feature set combines low-level image statistics, covariance terms, and LUV colour channels.
- Feature comparison: sp-Cov performs better than M+O+LUV+LBP on INRIA, worse on ETH, and comparably on TUD-Brussels; combining sp-Cov and sp-LBP with M+O+LUV performs best.These comparisons use log-average miss rates for feature combinations.
- pAUC struct tuning: 20.7% is the best cross-validation miss rate after pAUC struct post-tuning with C = 24 and β = 0.7, improving 0.6% over the 21.3% untuned detector.The pAUC struct score re-ranks predicted patches over the 0.01–1 FPPI range.
- State-of-the-art comparison: 11.1% miss rate is achieved on INRIA with combined features and pAUC struct, and the combined detector outperforms previous best results on INRIA, ETH, TUD-Brussels, and Caltech-USA.The paper reports 11.2% without pAUC struct and 11.1% with it on INRIA; selected regions commonly include human contours and body areas.
- Computational cost: The detector runs at approximately 0.126 frames per second without global normalization and 0.119 frames per second with it on Caltech data using a single-core 2.70GHz Intel Xeon CPU.Additional costs include extracting sp-Cov and sp-LBP, recomputing confidence scores, and performing global normalization.
4 Conclusion
The paper combines spatially pooled low-level visual features with pAUC-oriented weak-learner optimization to improve pedestrian detection. This combination outperforms competing detectors across standard benchmark datasets, while experiments show that feature choice and implementation details are crucial.
- Spatially pooled low-level visual features and pAUC-oriented weak-learner optimization together produce a strong pedestrian detector.The pAUC optimization targets detection performance in the range of greatest practical importance.
- The combined approach outperforms all competitors on the standard pedestrian detection benchmark datasets.
- Feature selection and implementation details are crucial for achieving the best detection performance.