Source-linked AI summary
Stripe-based and Attribute-aware Network: A Two-Branch Deep Model for Vehicle Re-identification
Jingjing Qian, Wei Jiang, Hao Luo, Hongyan Yu
TL;DR
Vehicle Re-ID must distinguish different vehicles with nearly identical appearances, a challenge that matters for surveillance applications. SAN combines stripe-based part features with attribute-supervised global features in a single descriptor. The reported comparisons show stronger results than the compared methods on VehicleID and VeRi.
Problem
Vehicle Re-ID must distinguish different instances with inconspicuous differences, while global features can miss small but discriminative regions.
Source: §I · p. 1Method
Results
SAN reports the highest Rank-1 rates among compared methods on VehicleID’s three test sets and higher accuracy than the cited single-modal methods on VeRi.
Source: 1) Comparison on VehicleID · p. 8, 2) Comparison on VeRi · p. 8Takeaways & Limitations
The authors report that SAN resists adverse horizontal viewpoint variations and improves vehicle Re-ID performance.
Source: §V · p. 8
Abstract
from arXiv · showhide
Vehicle re-identification (Re-ID) has been attracting increasing interest in the field of computer vision due to the growing utilization of surveillance cameras in public security. However, vehicle Re-ID still suffers a similarity challenge despite the efforts made to solve this problem. This challenge involves distinguishing different instances with nearly identical appearances. In this paper, we propose a novel two-branch stripe-based and attribute-aware deep convolutional neural network (SAN) to learn the efficient feature embedding for vehicle Re-ID task. The two-branch neural network, consisting of stripe-based branch and attribute-aware branches, can adaptively extract the discriminative features from the visual appearance of vehicles. A horizontal average pooling and dimension-reduced convolutional layers are inserted into the stripe-based branch to achieve part-level features. Meanwhile, the attribute-aware branch extracts the global feature under the supervision of vehicle attribute labels to separate the similar vehicle identities with different attribute annotations. Finally, the part-level and global features are concatenated together to form the final descriptor of the input image for vehicle Re-ID. The final descriptor not only can separate vehicles with different attributes but also distinguish vehicle identities with the same attributes. The extensive experiments on both VehicleID and VeRi databases show that the proposed SAN method outperforms other state-of-the-art vehicle Re-ID approaches.
I. INTRODUCTION
Vehicle Re-ID must distinguish different vehicles whose appearances are nearly identical, including vehicles sharing a model or differing in model or type. SAN combines part-level features with attribute supervision, and the paper reports state-of-the-art results on VehicleID and VeRi.
Motivation
- Appearance-based Re-ID remains challenging because different vehicle instances can have inconspicuous differences, despite its importance in surveillance applications.License plates may be faked or unrecognizable at low resolution, motivating appearance-based identification.Source: §I · p. 1
- Global features can miss small discriminative regions, while metric-learning approaches may ignore hierarchical multigrain relationships among vehicles.Examples of local cues include inspection marks on windscreens and personalized decorations.Source: §I · pp. 1–2
- Vehicles can remain difficult to distinguish even when they share a model, belong to different models, or have different types.Part-level cues and model or type annotations are presented as useful complementary information.Source: Figure 1 · p. 1, §I · p. 2
Contributions
- SAN combines part-level and global representations in a two-branch network, using vehicle attribute labels to distinguish identities with different annotations.The attribute-aware branch uses model labels on VehicleID and type labels on VeRi.Source: §I · p. 2
- VehicleID Rank-1 accuracy is 79.7%, 78.4%, and 75.6% across its three test sets, while VeRi mAP and Rank-1 accuracy are 72.5% and 93.3%, respectively.The authors report these as state-of-the-art results on the two datasets.Source: §I · p. 2
II. RELATED WORKS
Prior Re-ID work includes local cues and attributes for people, while vehicle research has explored datasets, spatial-temporal information, and fine-grained part features. The cited vehicle studies use key points, viewpoint attention, and discriminative regions such as windscreens and lights.
A. Re-identification (Re-ID)
- Person Re-ID work has used local body-part patches and attributes, alongside practices such as hard triplet mining and identity-classification pretraining.Triplet mining and ID classification have also been adopted in vehicle Re-ID.Source: A. Re-identification (Re-ID) · p. 2
B. Vehicle Re-ID
- Vehicle Re-ID research includes VehicleID and VeRi-776 datasets, spatial-temporal methods, attribute-aware ranking, and viewpoint-aware multi-view inference.The cited datasets include identity labels, while VehicleID is described as containing 26,267 vehicles and over 222,000 images.Source: B. Vehicle Re-ID · p. 2
C. Discriminative part-level features
- Vehicle part-feature methods use key points or viewpoint attention to localize regions, with windscreens, lights, and vehicle brands identified as discriminative cues.The cited approaches also include directional part features and attention over windscreen and car-head regions.Source: C. Discriminative part-level features · pp. 2–3
D. Attribute-embedded Re-ID
Earlier attribute-aware vehicle Re-ID work uses attributes for recognition, ROI features, retrieval filtering, or result refinement. SAN’s overview combines a part-feature branch with a global feature branch supervised by vehicle attributes.
D. Attribute-embedded Re-ID
- Prior attribute-aware methods use traits such as model and color, attribute-guided networks, structural-attribute ROIs, or attribute-based retrieval refinement.Some approaches combine attribute recognition with verification and triplet losses, while others use metadata during re-ranking.Source: D. Attribute-embedded Re-ID · p. 3
A. SAN
- SAN downsamples the backbone feature map into column vectors for dimension-reduced part-level features and extracts a global feature under vehicle-attribute supervision.The overall method uses separate stripe-based and attribute-aware branches.Source: A. SAN · pp. 3–4
1) Backbone Network:
SAN uses ResNet-50 with increased spatial resolution to retain feature granularity. Its stripe branch pools horizontal bands and reduces their dimensions to form part-level features.
1) Backbone Network:
- ResNet-50 is the selected backbone, and its final spatial down-sampling operation is set to 1 to obtain a higher-resolution feature map.The paper says SAN can conceptually use any deep convolutional image-classification network as its backbone.Source: 1) Backbone Network · p. 4
2) Stripe-based Branch:
- The stripe branch divides the feature map into horizontal stripes, averages each stripe into a part vector, and reduces each vector to 512 dimensions.The paper motivates vertical partitioning by vehicle parts such as the ceiling, windshield, header panel, and wheels; dimension reduction is used to limit classifier computation.Source: 2) Stripe-based Branch · p. 4
- For average pooling, the gradient for each input row in a pooled pair is the output gradient scaled by 1/(2m).The matrix-form backward propagation distributes the pooled output gradient across the two rows used to form each stripe average.Source: 2) Stripe-based Branch · p. 4
3) Attribute-aware Branch:
The attribute-aware branch uses global features and vehicle attribute labels to distinguish similar vehicles. The branch also addresses missing model annotations, while horizontal average pooling is described as robust to horizontal viewpoint changes.
3) Attribute-aware Branch:
- The attribute-aware branch treats vehicle attributes as a multi-label classification problem because ID, model, and type are not absolutely mutually exclusive.Global average pooling obtains the input image's global features, followed by fully connected prediction parts.Source: 3) Attribute-aware Branch · pp. 4–5
- DenseNet121 predicts soft model labels for VehicleID training images without model annotations.It is trained on images with identity and model annotations, supervised to predict both.Source: 3) Attribute-aware Branch · p. 5
- Horizontal average pooling averages each two-row feature-map segment into a stripe spanning the input image's width, which the authors say increases robustness to horizontal viewpoint changes.For a 16 × 16 × 2048 feature map, it produces an 8 × 1 × 2048 map.Source: 3) Attribute-aware Branch · p. 5
B. Loss Function
The loss combines stripe-level identity supervision with identity and model/type supervision in the attribute-aware branch. Its model/type component uses annotated labels or model soft-labels.
B. Loss Function
- Softmax predicts the vehicle identity or model/type, and the model/type loss uses either annotated model images or images with model soft-labels.The passage distinguishes images with model annotation from images with model soft-labels.Source: B. Loss Function · p. 5
- The total loss combines each stripe loss with the identity and model/type losses: L_i + LID + LModel.The branch losses are cross-entropy losses, with L_i assigned to stripe i.Source: B. Loss Function · p. 5
- The loss formulations use prediction probabilities and ground-truth distributions for stripe identity, vehicle identity, and model/type labels.For the model/type branch, the passage distinguishes the ground-truth label distribution from the ground-truth soft-label distribution.Source: B. Loss Function · p. 5
IV. EXPERIMENTS
The experiments use VehicleID and VeRi to assess SAN through ablations and comparisons with other vehicle Re-ID methods. The supplied experiment passages identify ablation tables and CMC curves but do not report their result values.
IV. EXPERIMENTS
- The experiments compare vehicle Re-ID methods on VehicleID and VeRi, using Euclidean distance to measure similarity from part-level and global features.The authors describe ablations to assess each SAN branch and comparisons with state-of-the-art methods.Source: §IV · p. 5
- Table I presents detailed ablation results for VehicleID and VeRi, with check marks identifying components used in each model.Its caption distinguishes attribute-aware ID-loss-only and model-loss-only variants.Source: Table I · p. 6
- Figure 4 presents CMC curves for the ablation study on VehicleID Test800, Test1600, and Test2400, and VeRi.The caption identifies the dataset subsets, but the supplied passage does not provide curve values.Source: Figure 4 · p. 6
A. Datasets and Evaluation Protocols
The evaluation uses VehicleID and VeRi, which differ in scale, camera setting, and attribute annotations. CMC and mAP are used to compare vehicle Re-ID performance.
A. Datasets and Evaluation Protocols
- VehicleID contains 221,763 daytime images of 26,267 vehicles, with identity labels and model annotations for some images; its tests use three gallery sizes.The training subset has 110,178 images; Test800, Test1600, and Test2400 contain 800, 1,600, and 2,400 gallery images, respectively.Source: A. Datasets and Evaluation Protocols · p. 6
- VeRi contains over 50,000 images of 776 vehicles from 20 cameras, with identity and type annotations.Its test evaluation uses a query set of 1,678 images and a gallery of 11,579 images.Source: A. Datasets and Evaluation Protocols · p. 6
- CMC estimates whether the correct match appears among the top K results, while mAP considers both precision and recall.Both metrics are adopted for comparison with existing methods.Source: A. Datasets and Evaluation Protocols · p. 6
B. Ablation Studies
The ablations evaluate attribute-aware and stripe-based branches separately, then combine them in SAN. Both branches add discriminative information, with attributes emphasizing vehicle-head cues and stripes capturing local identity details.
1) Effectiveness of the Attribute-aware Branch:
- The attribute-aware branch improves Rank-1 accuracy by 8.1%, 6.8%, and 5.7% across VehicleID’s three test sets, and improves VeRi Rank-1 and mAP by 1.1% and 5.7%.The comparison is against the baseline with only LID.Source: 1) Effectiveness of the… · pp. 6–7
- Visualizations show attribute-aware features emphasizing car-head cues, stripe-based features revealing local traces, and SAN enhancing discriminative details in both regions.The attribute-aware branch also emphasizes global vehicle shape, while stripe features can expose inspection marks and use traces.Source: 1) Effectiveness of the… · p. 7, Figure 5 · p. 7
2) Effectiveness of the Stripe-based Branch:
- Increasing stripe count q improves retrieval accuracy, with q = 8 reaching the highest reported accuracy and promoting part-level feature discrimination most.The stripe branch uses horizontal average pooling and dimension-reduced convolution to extract vertical part-level features.Source: 2) Effectiveness of the Stripe-based… · p. 7
- With q = 8, the stripe-based branch improves Rank-1 accuracy by 8.1%, 8.2%, and 11.2% on VehicleID, and improves VeRi Rank-1 and mAP by 4.0% and 9.1%.These comparisons use the baseline with only LID; the reported improvement is larger on the more arduous test sets.Source: 2) Effectiveness of the Stripe-based… · p. 7
- SAN with q = 8 reaches 79.7%, 78.4%, and 75.6% Rank-1 accuracy on VehicleID’s three test sets, and 72.5% mAP and 93.3% Rank-1 accuracy on VeRi.The final model combines the attribute-aware and stripe-based branches.Source: 2) Effectiveness of the Stripe-based… · pp. 7–8
C. Comparison with State-of-the-art Methods
On VehicleID, SAN is reported to outperform the compared deep learning-based methods across all three test sets. It also leads the reported Rank-1 and Rank-5 identification rates.
1) Comparison on VehicleID:
- SAN achieves the highest VehicleID Rank-1 rates—79.7%, 78.4%, and 75.6%—and Rank-5 rates—94.3%, 91.3%, and 88.3%—across Test800, Test1600, and Test2400.These are the reported highest rates among compared methods.Source: 1) Comparison on VehicleID · p. 8
- Deep learning-based methods outperform the traditional BOW-CN and LOMO methods on VehicleID.Source: 1) Comparison on VehicleID · p. 8
- SAN outperforms every compared deep learning-based method, including PN, across all three VehicleID test sets.PN is identified as the top deep learning-based method before SAN in the comparison.Source: 1) Comparison on VehicleID · p. 8
2) Comparison on VeRi:
On VeRi, SAN achieved the highest reported mAP, Rank-1, and Rank-5 rates among compared methods. The conclusion attributes its approach to combining stripe-based part-level features with attribute-supervised global features.
2) Comparison on VeRi:
- 72.5% mAP, 93.3% Rank-1 identification rate, and 97.1% Rank-5 identification rate were the highest among compared methods on VeRi.Table III presents the comparison.Source: 2) Comparison on VeRi · p. 8, Table III · p. 8
- SAN consistently outperformed the handcrafted LOMO and BOW-CN representations on mAP, Rank-1, and Rank-5.Source: 2) Comparison on VeRi · p. 8
- SAN outperformed the compared single-modal deep-learning methods; QD-DLF reported 61.82% mAP, 88.50% Rank-1 ID rate, and 94.46% Rank-5 ID rate.Source: 2) Comparison on VeRi · p. 8
- SAN also outperformed the multi-modal methods PROVID and Siamese-CNN+Path-LSTM on mAP, Rank-1, and Rank-5.Source: 2) Comparison on VeRi · p. 8
V. CONCLUSION
- The conclusion describes SAN as combining stripe-derived part-level features and attribute-supervised global features, and reports resistance to horizontal viewpoint variations.The two feature types are concatenated into the final descriptor.Source: §V · p. 8