PaperlayerView exact revision on arXiv

Source-linked AI summary

Taking a Deeper Look at Pedestrians

Jan Hosang, Mohamed Omran, Rodrigo Benenson, Bernt Schiele

arXiv:1501.05790v1cs.CV

TL;DR

Pedestrian detection has historically lagged behind other computer-vision applications of convolutional neural networks. This paper systematically studies straightforward convnets, their architectures, proposals, and training data, finding competitive or top benchmark performance on Caltech and KITTI. The work also identifies a scope boundary: the gains are promising but do not match the large improvements seen on Pascal VOC and ImageNet.

  • Problem

    The paper asks whether convolutional neural networks can transfer their broader computer-vision success to pedestrian detection, where they historically underperformed other detectors.

  • Method

    The authors systematically explore small and large convnet architectures, proposal methods, training choices, external data, and surrogate-task pre-training without custom part or occlusion modeling.

  • Results

    The paper reports competitive or top convnet performance on Caltech and KITTI, including best-known convnet results on both challenging datasets.

  • Takeaways & Limitations

    Vanilla convnets can achieve strong pedestrian-detection performance, including without pre-training, across model sizes from 10^5 to 10^7 parameters.

  • Takeaways & Limitations

    On pedestrian detection, the authors do not observe the large improvement seen on datasets such as Pascal VOC and ImageNet.

Abstract

from arXiv · show

In this paper we study the use of convolutional neural networks (convnets) for the task of pedestrian detection. Despite their recent diverse successes, convnets historically underperform compared to other pedestrian detectors. We deliberately omit explicitly modelling the problem into the network (e.g. parts or occlusion modelling) and show that we can reach competitive performance without bells and whistles. In a wide range of experiments we analyse small and big convnets, their architectural choices, parameters, and the influence of different training data, including pre-training on surrogate tasks. We present the best convnet detectors on the Caltech and KITTI dataset. On Caltech our convnets reach top performance both for the Caltech1x and Caltech10x training setup. Using additional data at training time our strongest convnet model is competitive even to detectors that use additional data (optical flow) at test time.

1. Introduction

The paper revisits whether convolutional neural networks can transfer their success to pedestrian detection, where prior neural approaches used specialized designs and lagged leading detectors. It evaluates straightforward convnets and reports competitive results on established benchmarks.

  • Pedestrian detection is a canonical object-detection task with applications in car safety, surveillance, and robotics.
  • The paper revisits straightforward convnets and tests whether their broader computer-vision success transfers to pedestrian detection.
  • Prior neural-network pedestrian detectors used handcrafted features, part modeling, or occlusion modeling, while leading methods relied on Adaboost-trained decision trees.
  • The experiments cover small and large vanilla convnets, training details, network parameters, proposal methods, and different training data.
  • Using RGB alone can achieve good performance, while more sophisticated inputs systematically improve detection quality, making the opposition between handcrafted features and convnets an illusion.
  • The authors report best-known convnet performance on Caltech and first convnet results on KITTI, including gains from extended and external training data.

2. Training data

The study varies the amount and source of training data, using Caltech in standard and expanded forms, KITTI, and large surrogate-task datasets for pre-training.

  • Training-data volume is treated as important for achieving good convnet performance.
  • The standard Caltech Reasonable training set contains 4 250 frames and approximately 2·10^3 annotated pedestrians, while its test set contains 4 024 frames and approximately 1·10^3 pedestrians.
  • Caltech validation experiments use the first five training videos for validation training and the sixth for validation testing.
  • Caltech10x increases training data tenfold by sampling one out of three frames, yielding approximately 2·10^4 annotated pedestrians from 42 782 frames.
  • KITTI contains 4 445 training pedestrians across 7 481 frames, including 4 024 pedestrians taller than 40 pixels, and has 7 518 test frames.
  • ImageNet and Places provide surrogate pre-training tasks with 1.2·10^6 and 2.5·10^6 annotated training images, respectively.

3. From decision forests to neural networks

The paper shows how a decision forest can be represented exactly as a deep neural network, linking proposal-based pedestrian detection to a neural-network formulation.

  • The proposal method can be converted into a convnet, making the overall detector a cascade of two neural networks.
  • SquaresChnFtrs pools and thresholds information from feature channels at tree nodes, which are implemented with fully connected layers, biases, and sigmoid nonlinearities.
  • The resulting network exactly reproduces SquaresChnFtrs outputs on the same inputs, although its parameters were learned by Adaboost rather than back-propagation.
  • CifarNet is illustrated as a small network with approximately 10^5 parameters.

4. Vanilla convolutional networks

The authors start from established convnet architectures and systematically examine proposals, inputs, window size, training labels, and architectural choices for pedestrian detection. A vanilla CifarNet already outperforms prior convnet detectors, while several design changes provide little or inconsistent benefit.

  • 4. Vanilla convolutional networks: The exploration starts from known effective architectures, using CifarNet and later AlexNet as baselines for pedestrian detection.CifarNet is a small CIFAR-10 network; AlexNet has 600 times more parameters and targets ILSVRC2012 classification.
  • 4.1. How to use CifarNet?: SquaresChnFtrs supplies the default proposals because it was the best available Caltech pedestrian detector with source code.Class-specific proposals reduce the number of proposals by three orders of magnitude compared with generic proposals.
  • 4.1. How to use CifarNet?: Ground-truth annotations are used as positive examples, while proposals below a separate IoU threshold are negative and intermediate cases are ignored.The authors report that using ground-truth annotations as positives is beneficial, without significant jitter.
  • 4.1. How to use CifarNet?: A 128×64-pixel model window works best for CifarNet, despite the network being designed for 32×32-pixel inputs.Alternative stretching, cropping, and context-border variants did not show a clear advantage in the reported experiments.
  • 4.2. How far can we get with the CifarNet?: 30.7% MR versus 37.9% MR for SDN: vanilla CifarNet immediately improves over the previously best known convnet on the Caltech test set.Using the same detection proposals as JointDeep, CifarNet also improves over both JointDeep and SDN.
  • 4.3.1 Number and size of convolutional filters: Large convolutional filters hurt quality, while changing filter counts has less impact and no configuration shows a clear overall advantage.The authors retain the simple CifarNet configuration with 32-32-64 filters and 5×5-pixel filters for subsequent experiments; architectural alternatives likewise do not improve over the original three convolutional layers.
  • 4.3.1 Number and size of convolutional filters: HOG+L provides a small advantage over RGB, but the experiments use raw RGB to maintain direct comparability with larger networks.The paper notes that more sophisticated inputs systematically improve detection quality, while good performance is still reachable with RGB alone.

5. Large convolutional network

The large AlexNet, evaluated through R-CNN, achieves strong pedestrian-detection performance through ImageNet pre-training, Caltech fine-tuning, and additional training data. Its gains persist across proposal methods and extend to KITTI.

  • Method: R-CNN wraps ImageNet-pre-trained AlexNet, fine-tunes all layers for pedestrian detection, replaces softmax with a linear SVM, and uses SquaresChnFtrs proposals.This is the default AlexNet detection setup unless otherwise specified.
  • Pre-training: 25.9% MR is achieved on Caltech after fine-tuning all AlexNet layers, compared with 39.8% MR when only the top-layer SVM is trained.The result supports the effectiveness of the R-CNN training recipe.
  • Caltech10x: 23.3% MR makes the Caltech10x AlexNet setup the best reported single-frame detector on Caltech without optical flow.The additional data primarily contributes more pedestrian positives, despite correlation across training samples.
  • Training from scratch: 32.4% MR is obtained by training AlexNet from scratch on Caltech, improving over SquaresChnFtrs at 34.8% MR and SDN at 39.8% MR.Using Caltech10x further lowers the miss rate to 27.5% MR, although ImageNet pre-training reaches 23.3% MR.
  • Proposal methods: AlexNet performance is stable across ACF and SquaresChnFtrs proposals, and stronger proposals do not improve results beyond roughly 22% MR.The proposal-method comparison covers detectors with an approximately 10 percentage-point performance gap.
  • KITTI: 46.9% AP on KITTI improves over the SquaresChnFtrs proposal method's 44.4% AP, providing the first published convnet results on that benchmark.The network is pre-trained on ImageNet and fine-tuned using KITTI training data.

6. Small or big convnet?

The comparison between small CifarNet and large AlexNet shows that capacity interacts with training-data volume, while AlexNet's detection cost makes proposal quality and speed important.

  • Performance comparison: 30.7% MR on Caltech1x makes CifarNet 2 percentage points better than AlexNet, while Caltech10x results are 28.4% MR for CifarNet and 27.1% MR for AlexNet.The comparison uses models trained only on Caltech1x and Caltech10x.
  • Capacity and data: Higher-capacity AlexNet benefits more from increased training data, whereas lower-capacity CifarNet saturates earlier.The reported trend is based on the differing Caltech1x-to-Caltech10x improvements.
  • Timing: 3ms per proposal window makes AlexNet too slow for sliding-window detection, while fewer than 100 proposals enable roughly 300ms per image.SquaresChnFtrs takes about 2 seconds per image in the authors' experience, so proposal generation dominates runtime.

7. Takeaways

The paper shows that both small and large off-the-shelf convnets can achieve competitive pedestrian-detection performance without specialized architectural modeling. It reports leading Caltech results and encouraging first convnet results on KITTI.

  • 7. Takeaways: CifarNet and AlexNet reach competitive performance despite using off-the-shelf architectures rather than specialized pedestrian-detection designs.The paper presents these as evidence that involved architectures are not required for competitive performance.
  • 7. Takeaways: The two Caltech-only networks outperform all previously published convnets trained on Caltech data.This conclusion covers both the small CifarNet and the large AlexNet.
  • 7. Takeaways: CifarNet is the best single-frame detector trained on Caltech1x, while AlexNet is the best single-frame detector trained on Caltech10x.These rankings are stated for the respective training-data setups.
  • 7. Takeaways: ImageNet-pre-trained AlexNet achieves competitive Caltech results without using additional information at test time such as optical flow.Figure 6 compares the result with published Caltech methods, including methods using optical flow.
  • 7. Takeaways: AlexNet improves over the proposal method on KITTI, establishing the first published convnet results on that pedestrian-detection benchmark.The paper describes these results as encouraging for further convnet-based KITTI performance.

8. Conclusion

The paper provides systematic evidence that vanilla convnets can perform competitively for pedestrian detection without custom designs, across training regimes and model sizes. It reports best-known convnet results on Caltech and KITTI, while noting smaller gains than in some other mature benchmarks.

  • Using the same proposals and training data as previous approaches, vanilla convnets outperform previous pedestrian-detection convnets.
  • Pre-training on surrogate tasks enables top performance, while competitive results remain possible without pre-training across 105 to 107 parameters.The experiments also identify which parameters are most critical for top performance.
  • The paper reports the best-known convnet results on both the Caltech and KITTI datasets.
  • Convnet best practices transfer to pedestrian detection, but the paper does not observe the large improvements reported on Pascal VOC and ImageNet.

A. CifarNet training, the devil is in the details

CifarNet training depends on several implementation choices, including optimization, regularization, and weight initialization. The reported setup uses SGD with scheduled learning-rate reduction, L2 regularization, and layer-specific Gaussian initialization.

  • Training neural networks is sensitive to learning rate, initialization, regularization, and batch size, whose effects are difficult to isolate and setup-dependent.
  • CifarNet uses SGD with learning rate 0.005, momentum 0.9, and batch size 128, then reduces the learning rate tenfold after 60 epochs.Training continues for 10 additional epochs; further reduction did not improve classification accuracy.
  • The training objective combines softmax classification loss with L2 weight regularization, weighted 0.005 for all layers except the softmax layer.The softmax weights receive regularization weight 1.
  • Weights are initialized from Gaussian distributions with standard deviation 0.01, except the first layer, which uses 0.0001.

B. Grid search around CifarNet

Grid searches around CifarNet show that moderate architectural changes produce relatively small fluctuations, whereas extreme filter sizes, widths, or parameter settings degrade performance. Proposal-negative and SVM parameters are generally insensitive near their defaults but can cause substantial degradation when far from optimal.

  • Grid search around CifarNet: Using filter sizes or network widths that are too small or too large hurts quality, while middle-range configurations show only small miss-rate fluctuations.Each parameter set was evaluated using four networks and averaged results with mean and standard deviation.
  • Grid search around CifarNet: Replacing mean-pooling with max-pooling hurts performance except on the first layer.
  • Grid search around CifarNet: No parameter set close to the default CifarNet configuration has a clear advantage, while moving too far from it causes classification accuracy to plunge.
  • Grid search around CifarNet: For proposal-negative selection and SVM regularization, the defaults are IoU < 0.5 and C = 10^-3.
  • Grid search around CifarNet: Near-optimal proposal and SVM parameters vary performance by 1 ∼2 percentage points, but far-from-optimal C causes 10 percentage-point degradation.
  • Grid search around CifarNet: Caltech and KITTI pedestrian-height distributions are somewhat dissimilar despite visual similarity, and models trained on each dataset do not transfer well across datasets.

E. Proposals statistics

Proposal analysis emphasizes a low proposal count that preserves recall while reducing false positives after re-scoring. On Caltech, roughly three proposals per image can exceed 90% recall above IoU 0.5, while overlap statistics reveal a non-maximum-suppression artefact for some detectors.

  • Proposals statistics: Proposal evaluation examines detector statistics on the Caltech test set, complementing the main paper’s table 9.
  • Proposals statistics: Keeping few proposals per image reduces the false-positive rate after re-scoring, unlike class-agnostic proposal practice that favors more windows for recall.Proposals are filtered using a detection-score threshold.
  • Proposals statistics: Recall higher than 90% is achievable with ∼3 proposals per image for IoU thresholds above 0.5.The low average reflects that most Caltech test frames contain no pedestrian.
  • Proposals statistics: SpatialPooling and LDCF show a bump near 0.5 IoU among false positives, attributed to their non-maximum-suppression method.Because these detectors assign very low scores to such windows, re-scoring must do likewise to improve quality.
  • Proposals statistics: A useful proposal method should combine high recall at high ground-truth overlap with false positives having low overlap.Overlapping false positives may contain partial persons that confuse the re-scoring classifier.
  • Proposals statistics: AlexNet reaches top detection quality by improving over LDCF, SquaresChnFtrs, and Katamari.
Loading 1501.05790v1…