PaperlayerView exact revision on arXiv

Source-linked AI summary

VSSA-NET: Vertical Spatial Sequence Attention Network for Traffic Sign Detection

Yuan Yuan, Zhitong Xiong, Qi Wang

arXiv:1905.01583v1cs.CV

TL;DR

Traffic sign detection remains difficult for small signs and visually similar objects in complex scenes. The paper proposes an end-to-end detector combining multi-resolution features with vertical spatial sequence attention, and reports effectiveness across traffic-sign and general object-detection evaluations.

  • Problem

    Small traffic signs are difficult to detect, while visually similar non-traffic objects can cause false detections without contextual information.

  • Method

    The method combines densely connected deconvolution-based multi-resolution features with vertical spatial sequence attention for context-aware sequence classification and regression.

  • Results

    99.18% average precision and 94.42% average recall are reported on the STS dataset, while OPTTSR mAP increases from 48.54% to 55.43%.

  • Takeaways & Limitations

    The experiments show that the proposed framework can improve traffic-sign detection and reduce false positive detections in complex scenes.

Abstract

from arXiv · show

Although traffic sign detection has been studied for years and great progress has been made with the rise of deep learning technique, there are still many problems remaining to be addressed. For complicated real-world traffic scenes, there are two main challenges. Firstly, traffic signs are usually small size objects, which makes it more difficult to detect than large ones; Secondly, it is hard to distinguish false targets which resemble real traffic signs in complex street scenes without context information. To handle these problems, we propose a novel end-to-end deep learning method for traffic sign detection in complex environments. Our contributions are as follows: 1) We propose a multi-resolution feature fusion network architecture which exploits densely connected deconvolution layers with skip connections, and can learn more effective features for the small size object; 2) We frame the traffic sign detection as a spatial sequence classification and regression task, and propose a vertical spatial sequence attention (VSSA) module to gain more context information for better detection performance. To comprehensively evaluate the proposed method, we do experiments on several traffic sign datasets as well as the general object detection dataset and the results have shown the effectiveness of our proposed method.

I. INTRODUCTION

Traffic sign detection is challenging because signs are small and visually similar non-traffic objects occur in complex scenes. The paper addresses these issues with multi-resolution features and vertical spatial context modeling.

  • Challenges: Small signs occupy less than 1% of images, reducing usable appearance information and making detection more difficult.The paper links small size to the need for earlier detection in driver-assistance applications.
  • Challenges: Complex scenes contain billboards and unrelated signs that can resemble traffic signs, causing false detections without spatial context.Hand-crafted features are described as insufficient for distinguishing these subtle differences.
  • Contributions: The method uses densely connected deconvolution layers with skip connections to produce multi-resolution feature maps for small-object detection.This is presented as the first contribution and as a dedicated architectural response to the small-size problem.
  • Contributions: Traffic sign detection is formulated as region-sequence classification and regression, using attention to encode contextual information.The sequence model focuses on local spatial patterns while suppressing irrelevant background noise.
  • Evaluation: The proposed method is evaluated on two traffic sign datasets and one general object detection dataset.The paper describes these experiments as a comprehensive evaluation of the method.

II. RELATED WORK

Prior traffic-sign detectors use color, shape, machine-learning, or deep-learning features, but complex scenes expose robustness and small-object limitations. Related work increasingly combines multi-resolution features with spatial context.

  • Color- and shape-based methods: Color- and shape-based methods exploit traffic signs’ regular shapes and distinctive colors but are sensitive to illumination, shadows, and weather.These methods are therefore commonly used as preprocessing or postprocessing stages.
  • Machine learning based methods: Machine-learning detectors can work for rigid objects, but shallow features such as those in ACF are limited in complicated traffic scenes.The related work contrasts their effectiveness in simple scenes with their weaker representation under complexity.
  • Deep learning based methods: Deep-learning detection frameworks include region-proposal-based and single-stage methods, with R-CNN-family systems achieving strong benchmark performance.The passage identifies the two-stage and single-stage distinction as a major organizational division.
  • Small-size object detection methods: Small-object methods improve detection by using super-resolved features, multiple resolutions, higher-resolution feature maps, or fused features.The cited approaches target representation quality at small object scales.
  • Spatial context based methods: Spatial-context methods encode directional or object-to-object context, addressing a limitation left relatively neglected by feature-focused approaches.Spatial RNNs are specifically described as improving small-size object performance.

III. OUR METHOD

The proposed detector combines a MobileNet backbone, a multi-resolution feature-learning module, and a vertical spatial sequence attention module. It builds semantic multi-resolution maps before encoding context along image columns.

  • III. OUR METHOD: The framework uses MobileNet for time efficiency and combines multi-resolution feature learning with vertical spatial sequence attention.The two modules respectively target feature fusion and traffic-sign classification using vertical context.
  • III. OUR METHOD: The multi-resolution module combines different semantic-level features with densely connected deconvolution layers to construct multi-resolution feature maps.The training pipeline first transforms inputs to multiple scales before building these maps.
  • III. OUR METHOD: The VSSA module explicitly encodes vertical spatial context to improve traffic-sign classification accuracy.The spatial sequence is encoded along each image column after multi-resolution feature construction.

A. Multi-Resolution Conv-Deconv Network

The multi-resolution conv-deconv module combines semantic features at multiple resolutions to improve traffic-sign detection, especially for small objects.

  • A. Multi-Resolution Conv-Deconv Network: The design targets small objects by combining higher-resolution location information with deeper semantic representations.Higher-resolution features improve localization, while deeper layers provide higher-level semantics.
  • A. Multi-Resolution Conv-Deconv Network: Densely connected deconvolution layers combine multi-level features while preserving higher-resolution localization details.Skip connections concatenate features across layers, with deconvolution aligning lower-resolution maps before fusion.
  • A. Multi-Resolution Conv-Deconv Network: The architecture adds three convolution layers to MobileNet and uses DS4 and DS6 high-resolution outputs in deeper deconvolution layers.MobileNet provides the backbone, while extra layers support multi-resolution feature learning.
  • A. Multi-Resolution Conv-Deconv Network: The final feature maps are DS13, DCD1, and DCD2 at 5 × 5, 10 × 10, and 19 × 19 resolutions.L2 normalization is applied after concatenating DCD1 and DCD2.

B. Vertical Spatial Sequence Attention Model

The VSSA module models traffic signs as vertical spatial sequences, using recurrent encoding and attention decoding to incorporate contextual information.

  • B. Vertical Spatial Sequence Attention Model: The module is designed to add context modeling to the multi-resolution feature maps before detection.The VSSA formulation treats detection as a spatial sequence classification and regression task.
  • B. Vertical Spatial Sequence Attention Model: Vertical capsules group neighboring feature vectors along image columns, allowing LSTM encoding to capture top-to-bottom spatial context.A capsule is a 3 × 1 × 1024 tensor in the example, and features are processed sequentially from top to bottom.
  • B. Vertical Spatial Sequence Attention Model: An attention decoder assigns weights over encoded hidden states so each decoding step can use information from the complete vertical sequence.The attention weights are generated sequentially from decoder hidden states.
  • B. Vertical Spatial Sequence Attention Model: The attended hidden state is concatenated with the decoder state to predict the detection label and bounding box.The VSSA module is applied to selected feature maps to reduce computational complexity.

C. Multi-Task Training

The detector is trained jointly for classification and bounding-box regression using a weighted combination of classification and Smooth L1 losses.

  • C. Multi-Task Training: The concatenated sequence features are projected by a 1 × 1 convolution to produce classification labels and bounding-box shifts.The outputs are generated for every feature point.
  • C. Multi-Task Training: The total loss combines classification log loss with Smooth L1 regression loss for bounding boxes.The regression term is weighted by α, set to 0.1 in this work.
  • C. Multi-Task Training: The multi-task objective uses ground-truth labels and bounding boxes to supervise both prediction tasks.The label and box targets are denoted by y_i and d_i, respectively.

IV. EXPERIMENTS

Experiments evaluate the proposed network on two traffic-sign datasets and Pascal VOC, using multi-resolution detection settings and visual and tabular comparisons.

  • IV. EXPERIMENTS: The evaluation covers STS, OPTTSR, and Pascal VOC datasets with differing scene complexity and object scales.STS contains highway and city scenes, OPTTSR contains challenging driving conditions, and Pascal VOC provides general object detection data.
  • IV. EXPERIMENTS: The experiments use SSD-based detection with 5 × 5, 10 × 10, and 19 × 19 feature maps, followed by non-maximum suppression.Multi-scale training uses image scales of 0.75, 1, and 1.25 times the original size.
  • IV. EXPERIMENTS: The proposed method is evaluated against existing detection architectures on the OPTTSR dataset.Table III reports the performance comparison for challenging traffic-sign images.

B. STS dataset

On STS, the proposed method combines multi-resolution feature learning with sequence modeling and achieves strong detection performance against other methods.

  • B. STS dataset: 99.18% average precision and 94.42% average recall were achieved across all traffic sign classes, attaining state-of-the-art results.Evaluation used an IoU threshold of 0.5 and considered signs larger than 50 pixels.
  • B. STS dataset: The evaluation uses SSD with 300 × 300 input, training on Part0 of Set1 and testing on Part0 of Set2.The dataset comparison follows the setup used by the compared traffic sign detection methods.
  • B. STS dataset: The experiments separately evaluate the Multi-Resolution architecture and sequence modeling to assess their contributions to traffic sign detection.

C. OPTTSR dataset

On OPTTSR, the proposed multi-resolution and vertical-context method improves detection, especially for small and difficult signs, while retaining real-time operation at 300 × 300 input.

  • C. OPTTSR dataset: OPTTSR contains 8,725 fully annotated images spanning seven Chinese traffic-sign classes and challenging conditions including rain, illumination variation, and occlusion.Annotated sign sizes range from 10 to 400 pixels, and mAP is used for evaluation.
  • C. OPTTSR dataset: Vertical spatial modeling outperformed horizontal modeling for 6 of 7 sign types, with especially clear gains for construction and supplemental signs.The comparison supports the paper’s assumption that vertical context is more effective for traffic sign detection.
  • C. OPTTSR dataset: 55.43% mAP was obtained with the proposed two modules, improving over the 48.54% baseline mainly through better small-sign detection.The VSSA module encodes additional context to reduce mistakes from visually similar objects such as billboards.
  • C. OPTTSR dataset: 21 FPS was achieved by the proposed vertical-context model with 300 × 300 input, while outperforming DSSD 321 running at 9.8 FPS.Using 500 × 500 input further improves performance at the cost of inference speed.

D. Pascal voc dataset

On Pascal VOC 2007, the full method improves detection over baseline methods, with gains concentrated in objects possessing vertical spatial structure and comparable directional-context mAP overall.

  • D. Pascal voc dataset: The full proposed method achieves the best detection performance on Pascal VOC 2007 among the evaluated methods.The comparison uses 5,011 training images and 4,952 evaluation images after 70,000 SSD-based training iterations.
  • D. Pascal voc dataset: The reported 2.1% improvement is mainly caused by several object classes, including chairs whose natural vertical arrangement favors vertical spatial context.Qualitative results also indicate stronger performance for objects with vertical spatial properties.
  • D. Pascal voc dataset: The proposed context models outperform SSD300+VGG and DSSD321, while DSSD513 with ResNet101 performs better but requires more computation.DSSD513 uses a 513 × 513 input and ResNet101 backbone, making it more time-consuming.
  • D. Pascal voc dataset: Horizontal and vertical context models achieve similar overall mAP, but vertical context helps boats and bottles while horizontal context helps dogs and sofas.The directional comparison is presented through the Pascal VOC context-model figures.

V. CONCLUSION

The paper formulates traffic sign detection as sequence classification and regression using a unified end-to-end framework with multi-resolution feature learning and vertical spatial sequence attention.

  • V. CONCLUSION: The unified framework contains a Multi-Resolution feature learning module and a vertical spatial sequence attention module.The Multi-Resolution module concatenates multi-layer features through densely connected deconvolution layers.
  • V. CONCLUSION: Traffic sign detection is treated as a sequence classification and regression task within the proposed end-to-end framework.
Loading 1905.01583v1…