Source-linked AI summary
Pedestrian Detection with Unsupervised Multi-Stage Feature Learning
Pierre Sermanet, Koray Kavukcuoglu, Soumith Chintala, Yann LeCun
TL;DR
Pedestrian detection must handle highly variable appearances, and the paper addresses this challenge with a convolutional network that learns hierarchical features using unsupervised sparse-coding pre-training and multi-stage connections. The resulting model achieves state-of-the-art or competitive performance on major pedestrian datasets, while its higher-level features are difficult to visualize directly in input space.
Problem
Pedestrian detection is important for surveillance, automotive safety, and robotics, but varying pose, occlusion, clothing, lighting, and backgrounds make it challenging.
Method
The model learns hierarchical features with convolutional sparse coding, jointly learns the dictionary and sparse representations, and combines high- and low-resolution features through multi-stage connections.
Results
The model achieves state-of-the-art or competitive results on most measures across publicly available pedestrian datasets; unsupervised learning and multi-stage features together reach a 10.55% error rate.
Takeaways & Limitations
Generic feature-learning algorithms can produce successful pedestrian-detection feature extractors, reducing reliance on hand-designed features.
Takeaways & Limitations
Higher-level features cannot generally be mapped back into input space for visualization after absolute-value rectification and local contrast normalization.
Abstract
from arXiv · showhide
Pedestrian detection is a problem of considerable practical interest. Adding to the list of successful applications of deep learning methods to vision, we report state-of-the-art and competitive results on all major pedestrian datasets with a convolutional network model. The model uses a few new twists, such as multi-stage features, connections that skip layers to integrate global shape information with local distinctive motif information, and an unsupervised method based on convolutional sparse coding to pre-train the filters at each stage.
1. Introduction
Pedestrian detection is challenging because pedestrian appearance varies widely, while existing systems rely heavily on hand-crafted low-level features. This paper applies a convolutional network with unsupervised pre-training and multi-stage feature connections to achieve competitive benchmark performance.
- Pedestrian appearance varies with pose, occlusion, clothing, lighting, and background, making detection difficult.
- Existing state-of-the-art systems combine hand-crafted features with trainable classifiers, while mid-level feature design remains difficult without learning.
- The model uses unsupervised convolutional sparse auto-encoders to pre-train features and supervised end-to-end training to fine-tune them.
- Multi-stage features with layer-skipping connections combine global shape detectors with local motif detectors at output stages.
- The paper reports state-of-the-art or competitive results across major pedestrian detection benchmarks.
2. Learning Feature Hierarchies
The paper learns hierarchical feature representations with convolutional sparse coding, then fine-tunes the complete system using labels. Multi-stage processing combines representations across levels and improves accuracy across tasks, especially pedestrian detection and traffic-sign classification.
- Generic feature learning algorithms can produce successful feature extractors that achieve state-of-the-art results.
- The model trains a generic unsupervised feature hierarchy layer by layer, using each previous layer’s representation, followed by supervised end-to-end updates.The complete architecture is fine-tuned with labeled data after layer-wise training.
- Convolutional sparse coding learns both sparse representations and the dictionary, using alternating optimization because the joint problem is non-convex but convex in either variable when the other is fixed.The method uses a predictor function resembling a single-layer ConvNet and FISTA for inference.
- The convolutional predictor applies learned filters, gains, biases, convolution, and a tanh nonlinearity to produce feature maps.For grayscale inputs, the output feature maps have spatial size p−m+1 × p−m+1.
- Each stage derives the next representation through prediction, absolute-value rectification, local contrast normalization, and average down-sampling.These transformations are applied before training the successive layer.
- Multi-stage features improve accuracy across tasks, with the greatest improvements for pedestrian detection and traffic-sign classification and minimal gains for house-number classification.The paper attributes larger gains to more multi-scale and textured objects, while house numbers are less complex.
3. Experiments
The system is evaluated across five standard pedestrian datasets while training only on INRIA, with experiments isolating unsupervised learning and multi-stage features. On fixed-INRIA large-pedestrian evaluation, combining both techniques gives the strongest reported error rate among the ConvNet variants.
- Evaluation setup: Five standard pedestrian datasets are evaluated, but all model training uses the INRIA dataset.The study names separate ConvNet variants for unsupervised and fully supervised training, with and without multi-stage features.
- Evaluation setup: Evaluation uses DET curves relating false positives per image to miss rate, ranked by continuous AUC over the [0, 1] FPPI range.The evaluation also ranks algorithms by error rate at 1 FPPI.
- Results: 17.29% error results from multi-stage features without unsupervised learning, while ConvNet-U-MS reaches a 10.55% error rate.The combined variant outperforms either individually modified ConvNet variant in the reported comparison.
- Results: The ConvNet achieves state-of-the-art performance with some margin on ETH, Caltech and TudBrussels large-pedestrian measures, while remaining close behind leading methods on INRIA and Daimler.On reasonable measures, performance is competitive for INRIA, Daimler and ETH but poor on Caltech.
4. Discussion
The paper introduces a hierarchical feature-learning model that learns features at all levels instead of hand-designing low-level features. Combining high- and low-resolution information and learning from color channels provides reported performance benefits across publicly available pedestrian datasets.
- Discussion: The model learns all feature levels in a hierarchy rather than relying on hand-designed low-level features.This contrasts the proposed feature-learning model with popular models whose low-level features are manually designed.
- Discussion: Combining high- and low-resolution features and learning from input color channels provide clear performance benefits on INRIA.The paper reports state-of-the-art or competitive results on most measures across publicly available datasets.
- Discussion: Small-scale pedestrian measures remain an improvement target, with future work proposed using multiple scale models that rely less on high-resolution details.The paper attributes poor Caltech reasonable-measure performance to sensitivity to resolution loss and suggests multiple-resolution training.
- Discussion: Although computational speed was not reported as a focus, the model was used at near-real-time speed in a haptic belt system with parallel hardware.The paper expects optimized GPU-based parallel computing and speed-oriented models to improve computational performance.
5. Evidence for using the proposed continuous Area Under Curve measure
Continuous AUC avoids discrete evaluation artifacts by integrating across the full curve and can change model rankings.
- 10.36% rather than 0% AUC is assigned to ConvNet-F under continuous evaluation on INRIA medium scale.The discrete method uses only a few benchmark points and incorrectly reports zero.
- Several models are re-ranked when continuous AUC replaces the discrete computation on the INRIA medium scale experiment.
- Several models, including Convnet-U-MS and VeryFast, are re-ranked on the ETH large scale experiment under continuous AUC.
6. Evidence for using the proposed fixed INRIA dataset
Fixing the INRIA dataset changes the comparative ordering of several pedestrian detection algorithms.
- ChnFtrs, MLS, and ConvNet-F-MS each advance by one rank when the fixed INRIA dataset replaces the original.
7. All results with the continuous AUC measure
The continuous AUC results evaluate systems over the full [0,1] DET-curve range, where smaller values indicate greater accuracy and fewer false positives.
- Continuous AUC is computed over the [0,1] range from DET curves relating false positives per image to miss rate.
- Smaller continuous AUC percentages indicate more accurate systems with greater reduction of false positives.
8. All results with the original discrete AUC measure
The discrete-AUC results use the alternative benchmark computation rather than the paper’s proposed continuous measure.
- Table 4 reports all experiments using discrete AUC instead of the proposed continuous AUC.