PaperlayerView exact revision on arXiv

Source-linked AI summary

SINet: A Scale-insensitive Convolutional Neural Network for Fast Vehicle Detection

Xiaowei Hu, Xuemiao Xu, Yongjie Xiao, Hao Chen, Shengfeng He, Jing Qin, Pheng-Ann Heng

arXiv:1804.00433v2cs.CV

TL;DR

CNN vehicle detectors are sensitive to scale, while traffic scenes commonly contain vehicles with widely varying sizes. SINet combines context-aware RoI pooling and a multi-branch decision network, reporting state-of-the-art accuracy and speed, up to 37 FPS, on KITTI and a new highway dataset.

  • Problem

    CNN detectors struggle with large vehicle-scale variance because RoI pooling can destroy small-object structure and a single network may inadequately represent differently scaled vehicles.

  • Method

    SINet uses context-aware RoI pooling to preserve small-object structure and context, then applies a multi-branch decision network to handle objects at different scales.

  • Results

    The method reports state-of-the-art detection accuracy and speed on KITTI and the LSVH highway dataset, with real-time detection up to 37 FPS.

  • Takeaways & Limitations

    The lightweight techniques address scale sensitivity without extra computational effort and support fast vehicle detection across large scale variance.

Abstract

from arXiv · show

Vision-based vehicle detection approaches achieve incredible success in recent years with the development of deep convolutional neural network (CNN). However, existing CNN based algorithms suffer from the problem that the convolutional features are scale-sensitive in object detection task but it is common that traffic images and videos contain vehicles with a large variance of scales. In this paper, we delve into the source of scale sensitivity, and reveal two key issues: 1) existing RoI pooling destroys the structure of small scale objects, 2) the large intra-class distance for a large variance of scales exceeds the representation capability of a single network. Based on these findings, we present a scale-insensitive convolutional neural network (SINet) for fast detecting vehicles with a large variance of scales. First, we present a context-aware RoI pooling to maintain the contextual information and original structure of small scale objects. Second, we present a multi-branch decision network to minimize the intra-class distance of features. These lightweight techniques bring zero extra time complexity but prominent detection accuracy improvement. The proposed techniques can be equipped with any deep network architectures and keep them trained end-to-end. Our SINet achieves state-of-the-art performance in terms of accuracy and speed (up to 37 FPS) on the KITTI benchmark and a new highway dataset, which contains a large variance of scales and extremely small objects.

I. INTRODUCTION

Vehicle detection is important for intelligent transportation systems, but CNN detectors struggle with vehicles spanning large scale ranges while fast detection remains necessary. SINet addresses these barriers with context-aware RoI pooling and a multi-branch decision network.

  • CNN-based vehicle detection is important for autonomous driving, surveillance, and intelligent traffic management systems.
  • Existing multi-resolution and multi-scale feature methods address scale variation but introduce computational overhead that limits fast detection.
  • Small-region features and traditional RoI pooling can lose or distort small vehicles’ original structures by replicating feature values.
  • Large intra-class distances between differently scaled vehicles make representation with one set of network weights difficult.
  • SINet combines context-aware RoI pooling, which preserves small-object structure and context, with a multi-branch decision network for scale variation.
  • Up to 37 FPS is achieved on 256×846 images, while the method reports state-of-the-art accuracy and speed on KITTI and a new highway dataset.
  • The highway dataset contains 14,388 labelled images covering different roads, times, weather conditions, and traffic states.

II. RELATED WORKS ON VEHICLE DETECTION

Vehicle detection research progressed from motion and hand-crafted-feature methods to deep CNN features, but scale variation remains difficult for accurate real-time detection.

  • Early motion-based methods model background or temporal motion but require complex post-processing and cannot distinguish fine-grained moving-object categories well.
  • Hand-crafted approaches describe image regions with features such as HOG, SURF, Gabor, and Haar-like descriptors, followed by classifiers including SVM and Adaboost.
  • Deep CNN features outperform many hand-crafted vehicle detectors on benchmarks, yet large scale variance still hinders accurate real-time detection.

III. WHY CURRENT CNNS ARE SCALE-SENSITIVE

CNN detectors are scale-sensitive because fixed-size RoI pooling can damage small-object structure, while differently scaled objects may be far apart in feature space.

  • CNN-based detectors use either two-stage proposal pipelines or faster end-to-end detection pipelines.
  • Two-stage detectors use RoI pooling to convert proposals into fixed-size feature vectors for classification networks.
  • When a proposal is smaller than the preset pooling grid, RoI pooling replicates values, potentially destroying the original structure of small objects.

B. Intra-class Distance Caused by Scale Variations

Scale variation creates a large intra-class distance between vehicles, challenging a single network's representation. Existing image- and feature-pyramid solutions either incur computational cost or lose spatial or semantic information for small objects.

  • Intra-class distance: Large and small vehicles can have substantially different feature representations, making shared weights difficult to use accurately across scales.The paper identifies this intra-class distance as a key source of scale sensitivity.
  • Existing solutions: Image pyramids fit multiple scales but introduce large computational costs that prohibit real-time detection.They process multiple input resolutions to cover different object sizes.
  • Existing solutions: Feature pyramids assign shallow high-resolution layers to small objects and deep low-resolution layers to large objects, but shallow layers often lack semantic information.This can make small objects difficult to distinguish accurately.
  • Existing solutions: Concatenating multi-layer features still leaves small objects with insufficient deep-layer spatial information because of down-sampling.The resulting deep representations remain difficult for detecting small objects.
  • Existing solutions: Upsampling deep features with high-resolution maps can preserve information for small objects, but processing the entire feature map requires extra memory and computation.The paper considers this cost unacceptable for real-time vehicle detection.
  • Proposed direction: SINet addresses scale sensitivity internally with context-aware pooling and a multi-branch decision network while adding zero extra computational cost.The two techniques target small-object information loss and scale-related feature variation.

IV. SCALE-INSENSITIVE NETWORK

SINet generates proposals from convolutional feature maps and processes them with context-aware RoI pooling designed to retain information from small proposals. The pooling adapts its operation to proposal size before extracting fixed-size discriminative features and remains trainable through back propagation.

  • Network architecture: SINet takes an image, generates convolutional feature maps, and obtains object proposals from them using region proposal networks.The proposals are candidate regions likely to contain objects.
  • Context-aware RoI pooling: Context-aware RoI pooling adjusts proposals to a specified size without sacrificing important contextual information.This is intended to preserve the original structures of small-scale objects.
  • Context-aware RoI pooling: For proposals larger than the target size, CARoI pooling uses maximum values within each sub-window as in standard RoI pooling.This is the first of three size-dependent cases.
  • Context-aware RoI pooling: For smaller proposals, CARoI pooling applies a bilinear-kernel deconvolution to enlarge them while retaining their surrounding context.The enlarged proposal then provides features for fixed-size pooling.
  • Context-aware RoI pooling: The deconvolution kernel depends on proposal scale: it matches the ratio of target pooled size to proposal size for small proposals and equals one for large proposals.Afterward, maximum feature values in each sub-window represent the proposal.
  • Back-propagation: CARoI pooling is trained end-to-end because loss derivatives are propagated through the pooling operation to its input features.The backward pass routes gradients through the selected maximum positions and deconvolution operation.

C. Multi-branch Decision Network

The multi-branch decision network assigns proposals of different sizes to branches that detect objects with more similar scales. Shared upstream convolutional features reduce duplication while branch-specific layers perform classification and bounding-box regression.

  • Branch design: Proposals with different sizes are split into branches, each detecting objects with similar sizes to reduce scale variance.The branches share features extracted by preceding convolutional layers.
  • Branch design: Each branch contains one convolutional layer and one fully connected layer followed by classification and bounding-box-regression classifiers.The two classifiers handle object category prediction and localization, respectively.
  • Branch assignment: The number of branches is chosen empirically from the dataset's scale distribution and available computational resources.The two-branch example separates large and small proposals using the median training-set object scale.
  • Branch assignment: During training, a dynamically varied threshold lets proposals near the median scale enter either branch, while testing uses the median threshold.The threshold variation is simulated with a Gaussian model centered at the median scale.

D. Implementation Details

The implementation tests the proposed modules on PVA and VGG networks, connects an MS-CNN RPN to the decision network, and trains the full system end to end. Inference combines branch outputs using soft-NMS.

  • Network architecture: Context-aware RoI pooling and the multi-branch decision network are designed to work with arbitrary CNN architectures and are tested with PVA and VGG.The CARoI kernel sizes are 6 × 6 for PVA and 7 × 7 for VGG.
  • Network architecture: SINet uses the MS-CNN proposal extraction network to obtain high-quality proposals from multiple CNN layers before the multi-branch decision network.The whole network is trained end to end.
  • Training strategies: Training first optimizes the RPN with a learning rate of 0.0001 for 10k iterations, then trains the whole network with a learning rate of 0.0005.The latter schedule reduces the learning rate by 0.1 at 40k and 70k iterations.
  • Inference: Inference combines small- and large-object outputs from multiple branches and refines them with soft-NMS.Soft-NMS averages coordinates from several highly overlapping boxes with relatively high confidence.

V. EXPERIMENTS

SINet is evaluated on KITTI and the newly constructed LSVH highway dataset, which contains vehicles across varied scales, scenes, conditions, and resolutions.

  • Datasets: Experiments use the KITTI benchmark and a newly constructed large scale variance highway dataset, LSVH.The evaluation runs on a single NVIDIA TITAN X GPU and eight CPUs.
  • Datasets: KITTI contains 7,481 training images and 7,518 testing images with easy, moderate, and hard vehicle-detection difficulty levels.Difficulty levels reflect vehicle size, occlusion, and truncation.
  • Datasets: LSVH contains 16 videos recorded across different scenes, times, weather conditions, and resolutions.Vehicles are categorized as car, bus, or van under sparse and crowded scenes; unrecognizable regions are marked “don’t care” and ignored.
  • Datasets: LSVH defines crowded scenes as those averaging more than 15 vehicles per frame; otherwise, scenes are sparse.The dataset includes car, bus, and van categories, while “don’t care” regions are excluded from training and evaluation.
  • Evaluation metrics: Performance is evaluated with average precision and intersection over union, using an IoU threshold of 0.7 on both datasets.KITTI is evaluated across all three difficulty levels, while LSVH is evaluated by vehicle category and scene type.

B. Comparison with the State-of-the-arts

SINet is compared with published detectors on KITTI and LSVH, achieving strong accuracy and efficiency across benchmark and varied-scale highway settings.

  • KITTI comparison: SINet achieves the highest accuracy on the KITTI moderate case and the fastest speed among the compared non-one-stage methods.The comparison includes 18 published methods.
  • KITTI comparison: SINet requires only 1/14 of Deep3DBox’s computation among the compared two-stage deep-learning detectors.The paper contrasts this efficiency with one-stage detectors that are faster but reported to have much lower accuracy.
  • LSVH comparison: Both SINet variants outperform the MS-CNN baseline and Faster R-CNN on LSVH in detection accuracy and efficiency.SINet also surpasses YOLO and YOLOv2 by a significant accuracy margin.
  • LSVH comparison: SINet performs well on crowded LSVH scenes and detects vehicles across orientations, scales, truncation levels, blur, rain, and occlusion.The visualized results particularly indicate strong detection of small vehicles with large scale variance.
  • Image resolution sensitivity: SINet remains robust across input image resolutions, whereas MS-CNN loses accuracy at small resolutions and incurs greater computational overhead at larger resolutions.The resolution experiments use the KITTI training set.

D. Ablation Analysis

Ablation results show that CARoI pooling and the multi-branch decision network improve detection accuracy without reducing efficiency, while additional branches offer limited gains.

  • Component ablation: CARoI pooling dramatically improves accuracy over MS-CNN baselines without introducing extra time.The gains are particularly significant for the moderate and hard categories.
  • Component ablation: The multi-branch decision network with two branches further improves accuracy while maintaining efficiency.The ablation compares models using PVA and VGG backbones within the MS-CNN framework.
  • Component ablation: Soft-NMS contributes to performance on the hard category, which contains many occluded and truncated vehicles.This result is reported for the fourth and ninth ablation rows.
  • Component ablation: Increasing the number of decision branches further provides limited performance gains while increasing network parameters and memory usage.The authors therefore use a two-branch decision network in the remaining experiments.

E. Vehicle Scale Analysis.

On LSVH, SINet improves detection across vehicle scales and scenes, with especially strong gains for small vehicles and crowded conditions.

  • Vehicle scale analysis: Improvements are more significant for small vehicles because traditional RoI pooling introduces artifacts and distortions that CARoI pooling avoids.The result is reported relative to both PVA- and VGG-based baselines.
  • Vehicle scale analysis: SINet shows dramatic improvement in crowded scenes, especially for small and medium vehicles.The experiments use the LSVH testing set under the strategy-1 sunny condition with 768 × 1344 input images.
  • Conclusion: The conclusion states that SINet achieves state-of-the-art accuracy and speed on KITTI and LSVH with zero extra computational effort from its two techniques.The paper identifies evaluation on more challenging datasets and integration into intelligent transportation systems as further investigations.
Loading 1804.00433v2…