Source-linked AI summary
Filtered Channel Features for Pedestrian Detection
Shanshan Zhang, Rodrigo Benenson, Bernt Schiele
TL;DR
The paper asks which ingredients and filter choices matter for pedestrian detection, building on the observation that strong detectors use an intermediate filtering layer. It unifies these designs as filtered channel features, systematically evaluates filter families, and reports top Caltech and KITTI performance with HOG+LUV, with optical flow yielding the best known Caltech result.
Problem
Despite steady progress, the key ingredients for good pedestrian detection remain unclear, including the effectiveness of parts, components, and feature learning.
Method
The paper unifies intermediate linear filtering and boosted decision forests as filtered channel features, then systematically explores hand-crafted, random, and learned filter banks.
Results
17.1% MR on Caltech, corresponding to 93% recall at 1 false positive per image, is reported when optical flow is added; HOG+LUV alone reaches top performance on Caltech and KITTI.
Takeaways & Limitations
Filtered channel features show that HOG+LUV remains competitive for pedestrian detection, while optical flow can further improve the Caltech result.
Takeaways & Limitations
Effective methods for context and optical flow across datasets have yet to be shown, and the authors could not reproduce 2Ped or SDt on KITTI.
Abstract
from arXiv · showhide
This paper starts from the observation that multiple top performing pedestrian detectors can be modelled by using an intermediate layer filtering low-level features in combination with a boosted decision forest. Based on this observation we propose a unifying framework and experimentally explore different filter families. We report extensive results enabling a systematic analysis. Using filtered channel features we obtain top performance on the challenging Caltech and KITTI datasets, while using only HOG+LUV as low-level features. When adding optical flow features we further improve detection quality and report the best known results on the Caltech dataset, reaching 93% recall at 1 FPPI.
1. Introduction
The paper addresses uncertainty about which ingredients drive pedestrian-detection quality by unifying several top-performing methods and systematically exploring filtered feature channels. It reports strong results with HOG+LUV features alone on Caltech and KITTI, with further improvement from optical flow.
- Pedestrian detection has improved substantially, but the key ingredients for good performance—including parts, components, and feature learning—remain unclear.
- The proposed framework models an intermediate linear filtering layer between low-level feature maps and classification, unifying multiple top-performing pedestrian detectors.
- Systematic experiments explore different filter banks and show that an appropriate filter bank can reach top detection quality.
- HOG+LUV features alone reach top performance on the challenging Caltech and KITTI datasets, matching methods using optical flow and additional features.
- The paper provides extensive experiments summarizing more than 65 trained models to analyze filtered integral channels.
- The authors additionally report the best known results on Caltech while using HOG+LUV features only.
2. Filtered channel features
Filtered channel features reinterpret pooled channel responses as convolutional filtering followed by single-value reads, with a boosted decision forest classifying the resulting features. The framework encompasses several detector designs and is evaluated with HOG+LUV channels on Caltech and KITTI.
- The architecture transforms an image into feature channels, pools rectangular regions, and feeds the resulting vector to a decision forest trained with Adaboost.
- Sum-pooling over rectangular regions is equivalent to convolution with a filter bank followed by reading one response value, enabling arbitrary filter banks.
- ACF, ChnFtrs, and SquaresChnFtrs correspond respectively to one, tens of thousands, and 16 filters in the framework.
- InformedHaar uses filters and read locations based on a human shape template, whereas LDCF uses PCA bases learned from training patches.
- The detectors use HOG+LUV feature channels, while the filtering-and-tree approach is treated as orthogonal to adding LBP or covariance features.
- Caltech evaluation uses log-average miss-rate for the reasonable setup, while KITTI evaluation uses average precision for the moderate setup.
3. Filter bank families
The paper compares hand-crafted, random, and data-driven filter banks within the filtered-channel framework. These families vary in prior structure, sampling, and whether filters are learned from background or pedestrian patches.
- The experiments compare filter banks designed from prior knowledge with data-driven filters learned from training data.
- InformedFilters: InformedFilters relaxes InformedHaar’s fixed filter positions and adds square pooling, producing 212 filters per feature-channel set.
- Checkerboards: Checkerboards matches InformedFilters’ size coverage without a human-shape prior, using square, gradient, and checkerboard patterns.
- RandomFilters: RandomFilters sample filter sizes uniformly and assign binary ±1 cell values uniformly; ternary values produced a small quality decrease.
- The framework treats feature channels as transformation outputs, filters as convolutional operators, and features as response-map entries passed selectively to the decision forest.
- Data-driven filters: LDCF uses PCA eigenvectors as filters, while PcaForeground learns separate filters from background and pedestrian patches.
4. How many filters?
The experiments examine how filter-bank size and filter-family design affect pedestrian detection quality. Around 50 filters often performs best, while filter-family differences are generally modest.
- A larger filter bank provides a richer view of fixed channel features, motivating analysis of filter-count and filter-family trade-offs.
- Figure 3 summarizes detection quality versus filters per channel across approximately 30 trained models.
- InformedFilters: Allowing Adaboost to choose pooling locations creates a meaningful gap between InformedHaar-Ours and similarly sized InformedFilters banks, 209 versus 212 filters.
- InformedFilters: Approximately 50 filters outperform approximately 200 for the available Caltech training data, while performance degrades below 50 filters across methods.
- RandomFilters: Random filters remain surprisingly close to other filter families, suggesting that expanding feature channels through filtering matters more than selecting perfect filters.
- LDCF/PcaForeground: LDCF under-performs when increasing its filters from 4 to 8 on the standard Caltech training set, while PcaForeground improves only marginally over LDCF8.
- Takeaways: Overall, approximately 50 filters are a Caltech sweet spot, with no flagrant difference between filter types.
5. Additional training data
The paper tests whether additional training data supports higher-capacity filtered-channel models and adjusts tree depth and boosting accordingly. Deeper trees and larger training sets improve detection when model capacity is sufficient.
- Increasing the number of filters also increases the features available to Adaboost, so Caltech10x is used to test whether models are data-starved.
- Deeper trees significantly improve detection over level-2 trees even with the original training data.
- Increasing training-data volume improves detection only when decision trees are deep enough.
- Level-4 decision trees are selected as a balance between increased detection quality and reasonable training times.
- With Caltech10x, deeper trees, additional training data, and Realboost provide a significant detection-quality boost over the Caltech1x configuration.
- The Caltech10x training results show a clear overall gain from increasing the training set, and RandomFilters can outperform LDCF-Ours with enough filters.
6. Add-ons
The paper evaluates context and optical flow as add-ons to its core filtered-channel detector. These cues provide limited additional benefit from a strong starting point, while more sophisticated extraction may be needed for further progress.
- Context and optical flow are added to evaluate complementarity, compare with existing methods, and report the best possible detection quality.
- Context: The 2Ped context re-scoring method yields less than 0.5 pp in these experiments, with gains disappearing at the Checkerboards performance level.
- Optical flow: Optical flow is supplied as two additional unfiltered channels derived from differences between weakly stabilized video frames.
- Optical flow: 1.4 pp is the gain provided by SDt over the stronger Checkerboards results, after which the authors name the detector All-in-one.
- Filtered-channel features are strong enough to erode existing context and flow features, although both remain complementary cues.
- The authors could not reproduce 2Ped or SDt results on KITTI, leaving effective cross-dataset methods for context and optical flow unresolved.
7. Test set results
On Caltech, filtered channel features improve detection over prior methods, while optical flow further reaches the best reported result. On KITTI, the approach also performs competitively using monocular images.
- Caltech test set: The InformedHaar-Ours baseline is the best reported result when trained with Caltech1x.The baselines show an approximately 10 pp gap between ACF/InformedHaar and ACF/InformedHaar-Ours.
- Caltech test set: The results show that proper validation of training parameters, including model size and negative samples, matters for baseline performance.The comparison reports an approximately 10 pp gap between ACF/InformedHaar and ACF/InformedHaar-Ours.
- Caltech test set: 18.5% MR with Checkerboards is lower than LDCF’s 24.8% MR when both are considered with Caltech10x training.RandomFilters reaches the same result but requires training and merging multiple models.
- Caltech test set: 17.1% MR with optical flow improves the previous best optical-flow method by approximately 5 pp on Caltech.This is reported as the best result on the challenging Caltech dataset.
- KITTI test set: 54.0% AP on KITTI is just 1 pp below the best known result using monocular images.Competing methods use additional LBP and covariance features, while the reported model does not.
8. Conclusion
The paper unifies several pedestrian detectors under filtered channel features and systematically explores their filter banks. It reports competitive HOG+LUV-only results on Caltech and KITTI, with optical flow setting a new Caltech state of the art.
- Conclusion: ACF, (Squares)ChnFtrs, InformedHaar, and LDCF fit within the filtered channel features detector framework.The framework places these seemingly disconnected methods under one umbrella.
- Conclusion: Systematic exploration of filter banks provides important improvements for pedestrian detection.
- Conclusion: Competitive results on Caltech and KITTI can be obtained using only HOG+LUV features.The conclusion states that these features have not yet saturated.
- Conclusion: 17.1% MR with optical flow sets the new Caltech state of the art, corresponding to 93% recall at 1 false positive per image.
A. Learned model
The learned models focus on similar informative pedestrian regions across strong and weak models, while filtered channels extract more discriminative information at those locations. Filter usage is similar across filter-bank families.
- Spatial distribution: Strong models and the weaker Roerei model focus on similar spatial regions of pedestrians.The strong models reach approximately 18% MR, while the weaker model reaches approximately 46% MR.
- Spatial distribution: Filtered channels extract more discriminative information at the same informative locations rather than changing which areas matter.
- Spatial distribution: Diagonal oriented channels focus on the left and right shoulders across all three models.
- Channel usage: The U colour channel is mainly used around the face, while luminance and gradient magnitude channels are used across the body.Head, feet, and upper torso areas provide most detection clues.
- Filter usage: Filter usage distributions are similar across different filter-bank families.Figure 8 shows the ten most and least frequently used filters independently of feature channel.