Source-linked AI summary
Learning Complexity-Aware Cascades for Deep Pedestrian Detection
Zhaowei Cai, Mohammad Saberian, Nuno Vasconcelos
TL;DR
Pedestrian sliding-window detection makes sophisticated features computationally difficult, while prior cascades did not explicitly optimize accuracy and variable feature complexity together. The paper introduces CompACT, a Lagrangian boosting method that places complex features later in cascades, enabling unified handcrafted-and-CNN detectors. These cascades achieve state-of-the-art pedestrian detection on Caltech and KITTI at fairly fast speeds, while CNN complexity requires a trigger-complexity approximation.
Problem
Sliding-window detection is computationally expensive, and prior cascades struggled to accommodate deep features while explicitly balancing detection accuracy and variable feature complexity.
Method
CompACT learns cascades by optimizing classification risk under a complexity constraint, using boosting to select weak learners and pushing higher-complexity features to later stages.
Results
CompACT cascades achieve state-of-the-art detection rates on Caltech and KITTI and integrate handcrafted and CNN features within a unified detector.
Takeaways & Limitations
Complexity-aware cascade learning expands cascade feature pools to deep CNN responses and generalizes object-proposal-plus-CNN designs by integrating CNNs into cascade stages.
Takeaways & Limitations
CNN features are assigned a trigger complexity because their network structure makes individual-feature computation inefficient and direct CPU-operation comparisons inappropriate.
Abstract
from arXiv · showhide
The design of complexity-aware cascaded detectors, combining features of very different complexities, is considered. A new cascade design procedure is introduced, by formulating cascade learning as the Lagrangian optimization of a risk that accounts for both accuracy and complexity. A boosting algorithm, denoted as complexity aware cascade training (CompACT), is then derived to solve this optimization. CompACT cascades are shown to seek an optimal trade-off between accuracy and complexity by pushing features of higher complexity to the later cascade stages, where only a few difficult candidate patches remain to be classified. This enables the use of features of vastly different complexities in a single detector. In result, the feature pool can be expanded to features previously impractical for cascade design, such as the responses of a deep convolutional neural network (CNN). This is demonstrated through the design of a pedestrian detector with a pool of features whose complexities span orders of magnitude. The resulting cascade generalizes the combination of a CNN with an object proposal mechanism: rather than a pre-processing stage, CompACT cascades seamlessly integrate CNNs in their stages. This enables state of the art performance on the Caltech and KITTI datasets, at fairly fast speeds.
1. Introduction
The paper targets efficient pedestrian detection by learning cascades that explicitly balance detection accuracy against computational complexity. CompACT enables unified cascades combining inexpensive handcrafted features with substantially more complex CNN features, achieving state-of-the-art results at fast speeds.
- Motivation: Sliding-window pedestrian detection evaluates nearly one million windows per 640×480 image, making sophisticated classifiers difficult to deploy in real time.Cascade detectors address this by rejecting most background patches with a few simple stages.
- Limitations of prior cascades: Earlier cascade designs relied on heuristics or assumed equivalent feature complexity, preventing explicit optimization of the accuracy-complexity trade-off.This assumption reduces learning to selecting accuracy-maximizing features.
- Limitations of prior cascades: Deep features are difficult to accommodate in cascades because sliding-window CNN computation is intractable, motivating object-proposal systems that separate proposal generation from classification.The paper identifies this two-stage design as suboptimal in accuracy-speed trade-offs.
- Contribution: CompACT formulates cascade learning as Lagrangian optimization of a risk penalizing both detection errors and implementation complexity.The method introduces complexity risk, complexity margins, and complexity losses, then derives a boosting algorithm to minimize the Lagrangian.
- Contribution: CompACT explicitly accounts for variable feature complexity, integrates handcrafted and CNN features in one detector, and achieves state-of-the-art Caltech and KITTI detection at faster speeds than close competitors.The resulting cascade generalizes object-proposal-plus-CNN architectures by integrating CNN stages with stages of other complexities.
2. Related Works
Prior pedestrian detectors developed efficient handcrafted feature channels and increasingly diverse representations, while deep models remained heavy and comparatively weaker for pedestrian detection. This paper positions complexity-aware cascades as a way to integrate proposal-like filtering and deep classification within one optimized detector.
- Cascade design: Cascade designs historically used heuristics to obtain accuracy-complexity trade-offs, while newer work began incorporating complexity terms into optimization objectives.The paper frames its approach as explicit variable-complexity cascade learning.
- Handcrafted features: Pedestrian detection features evolved from Haar-like and integral-channel representations to ACF, LBP, covariance, filtered, and decorrelated channels.These methods pursued computational efficiency or improved discrimination through alternative channel constructions.
- Deep models: Deep convolutional classifiers achieved strong general-object-detection results but had not excelled on pedestrian detection, where handcrafted features still dominated benchmarks.Transferred RCNN improved over earlier deep pedestrian detectors but remained significantly behind the state of the art.
- Deep models: Deep models are too heavy for sliding-window detection, so object proposals pre-select promising patches as a simple cascade mechanism.The paper instead studies seamless integration of proposal and classification stages under a joint accuracy-complexity criterion.
3. Complexity-Aware Cascade Training
CompACT extends boosting-based cascade learning by optimizing classification risk under a complexity constraint. Its design focuses complexity penalties on surviving examples, encouraging early rejection of negatives and progressively permitting more complex weak learners in later stages.
- Complexity-aware objective: CompACT introduces a boosting algorithm for cascade learning under a criterion that jointly accounts for classification risk and complexity risk.The complexity-aware objective is expressed as constrained optimization and equivalently as Lagrangian minimization.
- Complexity-aware objective: Complexity is modeled through a complexity margin that assigns positive complexity to positives and negative complexity to negatives.The measure Ω(F(x)) reflects evaluation time, such as machine operations or another empirical complexity measure.
- Cascade representation: A cascaded classifier applies thresholded stages sequentially, while average per-stage complexity accounts for which examples survive to each stage.The survivor indicator r_k(x) is one only when an example passes all preceding stages.
- CompACT optimization: Each boosting iteration selects a weak learner using the Lagrangian derivative, combining classification-risk and complexity-risk derivatives before updating the cascade predictor.The complexity derivative is approximated with a differentiable substitute for the Heaviside step function, and the update step can be found by line search.
- Properties: Only active examples that survive the current cascade contribute to the complexity term, so later-stage learning focuses on the remaining candidates.This matches the cascade intuition that examples rejected earlier should not affect later-stage complexity.
- Properties: As the active set shrinks, complexity pressure weakens across cascade stages, allowing increasingly complex weak learners and asymptotically approaching cascaded AdaBoost.Early iterations favor lower-complexity learners, whereas later stages process fewer examples and incur less complexity penalty.
4. Pedestrian Detection
The proposed pedestrian detector combines feature families spanning widely different computational costs, including ACF, handcrafted derived features, and CNN-based features. CompACT evaluates complex features selectively within cascade stages, while large CNN models can be incorporated at the final stage.
- 4.1. Feature Pools of Variable Complexity: CompACT weighs accuracy against computation, enabling seamless combination of multiple feature subsets.This allows low-complexity features to be pre-computed while more expensive features are evaluated as needed.
- 4.1. Feature Pools of Variable Complexity: CNN features are evaluated on demand, but the network structure makes computing individual features inefficient.When CNN features are needed for an image window, computing the fifth convolutional layer jointly is more efficient than computing each feature separately.
- 4.2. Embedding Large CNN Models: Large CNN models were restricted to the final cascade stage because embedding them throughout iterative boosting was impractical.The implementation considered Alex and VGG models adapted to the canonical 128×64 pedestrian template.
- 4.2. Embedding Large CNN Models: VGG-Net is approximately 8 times slower than Alex-Net, while modified VGG-Net is only twice as slow.The modified architecture uses a stride of 2 in Alex-Net's first convolutional layer and supports the canonical pedestrian template size.
5. Experiments
Experiments show that CompACT learns cascades balancing accuracy and complexity, selecting cheaper features early and reserving expensive CNN features for later stages. Across Caltech and KITTI, embedded CNN variants achieve strong accuracy while retaining competitive speed.
- Cascade Configuration: ACF features were selected exclusively during the first 200 stages and rarely after stage 500, while later stages favored more discriminative features.The learned configuration demonstrates stage-dependent feature selection based on feature complexity and discrimination.
- Cascade Comparison: Single-feature cascades perform poorly relative to CompACT, while CNN features improve accuracy at a ten-fold complexity increase over ACF.CNNCB obtains the best single-feature detection results but requires substantially more computation for only a marginal gain over CNN.
- Cascade Comparison: CompACT achieves the best accuracy-complexity trade-off compared with manually staged and unconstrained boosting cascades.Manual cascades have low complexity but poor accuracy, whereas unconstrained boosting is more accurate but significantly more complex.
- Large CNN Models: Embedding a large CNN as the final cascade stage achieved the best results in all evaluated cases.The large CNN was computed only on windows selected by the preceding CompACT cascade.
- State-of-the-Art Comparison: On Caltech, CompACT reached state-of-the-art performance at 4 fps, while CompACT-Deep improved by 7 points over the state of the art and ran at 2 fps.CompACT-Deep also exceeded the best deep pedestrian detector by 11 points and was faster than competing detectors.
- State-of-the-Art Comparison: On KITTI, CompACT was the fastest state-of-the-art detector, while CompACT-Deep exceeded R-CNN by more than 8 points.Regionlets was the only competitive approach, but its reported classification time excluded proposal generation, which can take several seconds.
6. Conclusion
CompACT learns detector cascades by optimizing classification risk under a complexity constraint, placing expensive features later and integrating diverse feature families, including deep CNNs. Applied to pedestrian detection, it achieves state-of-the-art rates on Caltech and KITTI at faster speeds than competing methods.
- CompACT optimizes classification risk under a complexity constraint to learn complexity-aware detector cascades.
- The learned cascades push high-complexity features to later stages and integrate multiple feature families in one design.
- CompACT extends cascaded detection to features such as deep CNNs and generalizes the combination of object proposals with CNNs.
- Pedestrian detectors learned with CompACT achieve state-of-the-art detection rates on Caltech and KITTI at much faster speeds than competing methods.