Source-linked AI summary
Vehicle Re-identification Using Quadruple Directional Deep Learning Features
Jianqing Zhu, Huanqiang Zeng, Jingchang Huang, Shengcai Liao, Zhen Lei, Canhui Cai, LiXin Zheng
TL;DR
Vehicle re-identification must match vehicles across changing viewpoints and other adverse imaging conditions, while practical systems may lack densely sampled views. QD-DLF addresses this by extracting and concatenating horizontal, vertical, diagonal, and anti-diagonal directional features, and experiments on VeRi and VehicleID report improvements over compared methods.
Problem
Vehicle re-identification is challenged by viewpoint variation and other adverse factors, while densely sampled camera viewpoints are difficult to obtain in practical surveillance systems.
Method
QD-DLF uses a shared convolutional architecture with horizontal, vertical, diagonal, and anti-diagonal average pooling layers whose normalized directional maps are concatenated.
Results
QD-DLF achieves the highest MAP of 61.83% and rank-1 identification rate of 88.50% among compared methods on VeRi, and outperforms compared deep methods across VehicleID Test800, Test1600, and Test2400.
Takeaways & Limitations
Fusing multiple directional deep features improves vehicle-image representation and supports robustness to viewpoint variations within the evaluated VeRi and VehicleID benchmarks.
Abstract
from arXiv · showhide
In order to resist the adverse effect of viewpoint variations for improving vehicle re-identification performance, we design quadruple directional deep learning networks to extract quadruple directional deep learning features (QD-DLF) of vehicle images. The quadruple directional deep learning networks are with similar overall architecture, including the same basic deep learning architecture but different directional feature pooling layers. Specifically, the same basic deep learning architecture is a shortly and densely connected convolutional neural network to extract basic feature maps of an input square vehicle image in the first stage. Then, the quadruple directional deep learning networks utilize different directional pooling layers, i.e., horizontal average pooling (HAP) layer, vertical average pooling (VAP) layer, diagonal average pooling (DAP) layer and anti-diagonal average pooling (AAP) layer, to compress the basic feature maps into horizontal, vertical, diagonal and anti-diagonal directional feature maps, respectively. Finally, these directional feature maps are spatially normalized and concatenated together as a quadruple directional deep learning feature for vehicle re-identification. Extensive experiments on both VeRi and VehicleID databases show that the proposed QD-DLF approach outperforms multiple state-of-the-art vehicle re-identification methods.
I. INTRODUCTION
Vehicle re-identification matches images of the same vehicle across cameras but is challenged by viewpoint variation and other adverse imaging conditions. The paper focuses on QD-DLF, which fuses directional features to improve viewpoint robustness.
- I. INTRODUCTION: Vehicle re-identification matches the same vehicle across different cameras for video surveillance, but viewpoint variation, illumination change, blur, occlusion, and low resolution make it difficult.The paper identifies viewpoint variation as the most crucial and frequently encountered challenge.
- I. INTRODUCTION: The paper proposes quadruple directional deep learning features (QD-DLF) by fusing features learned through four directional average pooling layers.The stated novelty is the first attempt to fuse quadruple directional deep features for viewpoint robustness.
- I. INTRODUCTION: VeRi and VehicleID provide large benchmark databases for developing and evaluating vehicle re-identification methods.
A. Feature Representation for Vehicle Re-identification
Vehicle re-identification methods use handcrafted or deep feature representations, while existing holistic deep features do not specifically address viewpoint variation. The paper motivates directional features as a way to improve viewpoint robustness without extra vehicle information.
- A. Feature Representation for Vehicle Re-identification: Existing vehicle re-identification features include handcrafted representations such as LOMO and BOW-CN and deep representations based on networks including AlexNet, VGGNet, and GoogLeNet.
- A. Feature Representation for Vehicle Re-identification: Most existing deep features are holistic and do not provide a specific solution for viewpoint variation, a crucial vehicle re-identification challenge.
- A. Feature Representation for Vehicle Re-identification: ABLN models transformations across continuous vehicle viewpoints and infers a global representation from one visible view, whereas densely sampled viewpoints are difficult to obtain in practical surveillance systems.
- A. Feature Representation for Vehicle Re-identification: The proposed QD-DLF targets viewpoint-robust representation without using extra vehicle information or introducing extra computational load.
III. VEHICLE RE-IDENTIFICATION USING QUADRUPLE DIRECTIONAL DEEP LEARNING FEATURES
The approach uses four directional deep feature-learning networks that share a basic architecture but apply horizontal, vertical, diagonal, and anti-diagonal average pooling to represent vehicle images from different directions.
- A. Quadruple Deep Feature Learning Networks: Four networks share a common shortly and densely connected convolutional architecture, then use direction-specific average pooling layers for horizontal, vertical, diagonal, and anti-diagonal features.The networks are HDFLN, VDFLN, DDFLN, and ADFLN, each followed by spatial normalization.
- 2) Quadruple Directional Average Pooling Layers:: Horizontal average pooling averages each row into one point, producing a horizontal feature map with dimensions d×1×c.For example, h1 averages f1, f2, f3, and f4.
- 2) Quadruple Directional Average Pooling Layers:: Vertical average pooling averages each column into one point and transposes its output during testing to match the horizontal feature-map dimensions.The resulting feature Q has dimensions 1×d×c before transposition.
- 2) Quadruple Directional Average Pooling Layers:: Diagonal and anti-diagonal pooling average feature-map elements along their respective directions to capture complementary directional structure.The supplied examples average f9 and f14 for a diagonal output and f4, f7, f10, and f13 for an anti-diagonal output.
- 2) Quadruple Directional Average Pooling Layers:: Average pooling maps a d×d input feature map to a 1×d output by averaging each pooling window, with gradients propagated through the chain rule.The matrix form uses Xi as the i-th column vector and J as the overall objective function.
3) Spatial Normalization Layer:
Spatial normalization follows each directional average-pooling layer and scales feature-map dimensions into a unified range.
- 3) Spatial Normalization Layer:: Spatial normalization follows every directional average-pooling layer and makes each feature dimension distribute within [0, 1).This is intended to prevent any single dimension from becoming too predominant.
- 3) Spatial Normalization Layer:: The normalization output Z is computed from input P using neighboring feature values, with Nj denoting the neighborhood size.The backward propagation of the normalization layer is also formulated in the method.
- 3) Spatial Normalization Layer:: The proposed networks combine the basic feature-learning architecture, directional pooling, and spatial normalization into horizontal, vertical, diagonal, and anti-diagonal branches.The branches are HDFLN, VDFLN, DDFLN, and ADFLN.
B. Objective Function
The objective function uses softmax classification with L2 regularization to train the proposed vehicle re-identification model.
- Softmax classification defines the training objective for the proposed method.The formulation uses a projection matrix to predict vehicle class labels from deep learning features.
- The objective includes an L2 regularization term weighted by α.α controls the contribution of the regularization item, while K and C denote the numbers of training samples and classes.
IV. EXPERIMENT AND ANALYSIS
The experiments compare QD-DLF with state-of-the-art methods on VeRi and VehicleID using Euclidean feature distances and CMC and MAP evaluation criteria.
- QD-DLF is evaluated against multiple state-of-the-art methods on the VeRi and VehicleID databases.
- CMC measures identification accuracy across candidate-list sizes, whereas MAP summarizes precision and recall across queries.
- Euclidean distance measures similarity between vehicle pairs represented by quadruple deep learning features.
A. Training Configuration
Training uses standard hardware and software settings, with 128 × 128 image inputs, augmentation, and database-specific regularization weights.
- The implementation uses Matconvnet, CUDA 8.0, CUDNN V5.1, MATLAB 2014, and Visual Studio 2012.Experiments run on an Intel Xeon E3-1505 M v5 workstation with an NVIDIA Titan X GPU and 128 GB DDR3 memory.
- Images are resized to 128 × 128 pixels and augmented by horizontal mirroring and small random rotations.Rotations are sampled within [−3°, 0°] and [0°, 3°].
- Weights are initialized from N(0, 0.01), biases are initialized to zero, and α is set to 0.005 on VeRi.
B. Databases
The study evaluates QD-DLF on VeRi and VehicleID and reports stronger VeRi performance than compared multimodal, single-modal, and handcrafted approaches.
- 1) Comparison on VeRi:: 61.83% MAP and 88.50% rank-1 identification are the highest VeRi results among the compared methods.These values are reported for QD-DLF in Table II.
- 1) Comparison on VeRi:: QD-DLF exceeds the compared multimodal methods on VeRi in MAP, rank-1, and rank-5 rates except for PROVID’s slightly higher rank-5 rate.QD-DLF nevertheless achieves higher MAP and rank-1 identification than PROVID.
- 1) Comparison on VeRi:: QD-DLF outperforms the compared single-modal deep learning methods on VeRi, while NuFACT reaches 48.47% MAP, 76.76% rank-1, and 91.42% rank-5.
- 1) Comparison on VeRi:: SCCN-Ft+CLBL-8-Ft and ABLN-Ft-16 do not show clear superiority on VeRi despite explicitly addressing viewpoint variation.The passage attributes this limitation to VeRi vehicles not being densely captured across viewpoints for their training requirements.
- 1) Comparison on VeRi:: QD-DLF performs better than the compared handcrafted representations BOW-CN, LOMO, and BOW-SFIT.
2) Comparison on VehicleID:
On VehicleID, QD-DLF outperforms the compared deep-learning and traditional methods, while diagonal features outperform horizontal and vertical alternatives. Fusing more directional features consistently improves performance, with QD-DLF best among the evaluated fusions.
- QD-DLF outperforms all compared deep learning methods on the Test800, Test1600, and Test2400 VehicleID subsets.
- D-DLF and A-DLF outperform H-DLF and V-DLF on VeRi and VehicleID.
- The authors attribute diagonal-feature advantages to vehicle-image symmetry and pooling that incorporates more diverse local feature regions.
- QD-DLF achieves the best performance among the evaluated feature fusions, exceeding DAH-DLF, DA-DLF, and D-DLF.The reported ordering is QD-DLF > DAH-DLF > DA-DLF > D-DLF.
4) Comparison on Directional and Deep Holistic Learning Features:
The study compares directional features with holistic features produced by replacing directional pooling with a full connection layer. Directional features are more effective than the best tested holistic configuration, and QD-DLF provides the strongest reported results.
- QD-DLF is 20.84% higher in MAP and 8.35% higher in rank-1 identification rate than F-DLF-256 on VeRi.
- F-DLF-256 has the highest MAP among F-DLF-128, F-DLF-256, F-DLF-512, and F-DLF-1024, but remains inferior to V-DLF.
- The results indicate that directional deep learning features represent vehicle images more effectively than the tested holistic learning features.
5) Running Time Analysis:
The running-time analysis compares feature extraction time across the proposed directional features, QD-DLF, and several state-of-the-art methods. Single directional features are much faster than DenseNet121, while QD-DLF remains slightly faster than DenseNet121 despite its larger model.
- Each single directional feature requires about 17% of DenseNet121’s feature extraction time.
- The four single directional features have similar extraction times, slightly slower than VGG-CNN-M-1024 and comparable to GoogLeNet.
- QD-DLF’s feature extraction time is about five times that of each single directional feature, rather than the intuitive fourfold increase.The difference is attributed to lower CPU-GPU communalization efficiency in the larger quadruple model.
- QD-DLF is 2.448ms faster than DenseNet121 for feature extraction.