A Benchmark for Vehicle Attribute Classification in Cross-Domain Surveillance Scenarios

Sergio M. Silva; Otavio T. Remer; Gabriel E. Lima; Lucas Wojcik; Rayson Laroca; David Menotti

Source-linked AI summary

A Benchmark for Vehicle Attribute Classification in Cross-Domain Surveillance Scenarios

Sergio M. Silva, Otavio T. Remer, Gabriel E. Lima, Lucas Wojcik, Rayson Laroca, David Menotti

arXiv:2609.01584v1Sep 1, 2026cs.CV

TL;DR

Vehicle-attribute models often degrade when surveillance conditions differ in viewpoint, occlusion, illumination, and sensing. The paper introduces UVIB with unified operational labels and evaluates representative architectures across domain-transfer protocols, finding that domain shift affects performance more than architecture choice, especially for VMMR suitability and color clarity.

Problem
Vehicle-attribute models trained under controlled conditions can degrade in surveillance scenarios with varying viewpoint, occlusion, illumination, and sensor characteristics.
Method
UVIB combines 84,835 images from seven public Brazilian datasets with unified labels for orientation, VMMR suitability, and color clarity, evaluated across acquisition-domain protocols.
Results
Domain shift affects performance more than architecture choice, with substantial cross-domain degradation concentrated in VMMR suitability and color clarity.
Takeaways & Limitations
Benchmarks and protocols should explicitly measure operational robustness beyond standard in-domain accuracy.
Takeaways & Limitations
VMMR suitability remains constrained by class imbalance and ambiguous occlusions, while color clarity is sensitive to illumination and sensor modality.

Abstract

from arXiv · show

Vehicle attribute analysis is a key component of Intelligent Transportation Systems (ITS), supporting applications such as vehicle identification, traffic monitoring, and forensic investigation. However, models trained under controlled conditions often degrade in real surveillance scenarios due to changes in viewpoint, occlusion, illumination, and sensor characteristics. This paper introduces Unconstrained Vehicle Identification Benchmark (UVIB), a benchmark for evaluating three operational vehicle-analysis tasks: front/rear orientation, occlusion-related suitability for Vehicle Make and Model Recognition (VMMR), and color clarity. The benchmark contains 84,835 vehicle images from seven public Brazilian datasets, grouped into surveillance and general acquisition domains, with unified binary annotations that were not jointly available in the original sources. Four representative architectures, EfficientNetV2-S, ResNet-50, ViT/B-16, and YOLO11s-cls, are evaluated under mixed-domain, cross-domain, and cross-dataset protocols. The results show that domain shift has a stronger impact than architecture choice, with substantial degradation in cross-domain settings, especially for VMMR suitability and color clarity. While orientation generalizes more reliably, VMMR suitability remains affected by class imbalance and ambiguous occlusions, and color clarity is highly sensitive to illumination and sensor modality. These findings highlight the need for benchmarks and evaluation protocols that explicitly measure operational robustness beyond standard in-domain accuracy. The proposed benchmark is publicly available at https://github.com/UFPR-IPASP-PR/uvib-vehicle-attributes/.

I. Introduction

Vehicle-attribute models can degrade under surveillance conditions, while existing datasets rarely provide unified labels for key operational decisions. UVIB addresses this gap with standardized annotations and evaluation protocols across acquisition domains.

Surveillance conditions vary in viewpoint, occlusion, illumination, and sensor characteristics, degrading models trained under controlled settings.
UVIB evaluates orientation, occlusion-related VMMR suitability, and color clarity as operational decisions affecting downstream vehicle analysis.
Existing vehicle datasets rarely provide unified labels for these three factors, limiting direct comparison and isolation of dataset bias or domain shift.
UVIB contains 84,835 images from seven public Brazilian datasets, with jointly unavailable annotations for the three operational tasks.
The paper provides standardized baselines for four representative architectures under four protocols, including strict cross-domain evaluation.
Domain shift leaves VMMR suitability and color clarity as the main bottlenecks for robust deployment.

II. Related Work

Vehicle attribute analysis supports multiple ITS applications, but surveillance imagery can make the visual cues underlying these tasks unreliable.

Vehicle attribute analysis supports VMMR, VCR, ALPR, and vehicle re-identification applications in ITS.
Viewpoint changes, occlusions, and color variations can make surveillance-image cues unreliable for vehicle analysis.

A. Vehicle Attribute Recognition in Challenging Scenarios

Prior work identifies acquisition conditions and dataset-specific biases as major challenges for vehicle attribute recognition. UVIB complements these datasets by pairing unified operational labels with protocols that separate mixed-domain performance from cross-domain transfer.

A. Vehicle Attribute Recognition in Challenging Scenarios: Orientation changes which vehicle cues are discriminative, with frontal views emphasizing grilles, headlights, and emblems and rear views emphasizing taillights, trunks, and plates.
A. Vehicle Attribute Recognition in Challenging Scenarios: Occlusion handling becomes an operational question of whether visible vehicle structure remains sufficient for downstream VMMR.
A. Vehicle Attribute Recognition in Challenging Scenarios: Illumination, exposure, reflections, and sensor modality can make vehicle-color labels unreliable even when the vehicle is visible.
B. Dataset Bias and Cross-Domain Generalization: Models may exploit acquisition-specific shortcuts rather than the visual concepts they are expected to learn.
B. Dataset Bias and Cross-Domain Generalization: Cross-dataset evaluations report performance drops across ALPR, VMMR, and VCR when training and testing sources differ.
B. Dataset Bias and Cross-Domain Generalization: UVIB pairs unified annotations for three tasks with protocols that distinguish mixed-domain performance from cross-domain transfer.

III. The UVIB Benchmark

UVIB aggregates vehicle images from seven Brazilian datasets and organizes them by acquisition domain to support explicit domain-transfer analysis. The benchmark’s source datasets span fixed surveillance imagery and more heterogeneous general acquisition settings.

UVIB contains 84,835 vehicle images aggregated from seven public Brazilian datasets.
The Surveillance Domain contains 57,798 images from fixed traffic-monitoring cameras, while the General Domain contains 27,037 images from heterogeneous acquisition settings.
The Surveillance Domain includes Vehicle-Rear, LPLCv2, and UFPR-VeSV, whereas the General Domain includes UFOP, SSIG-SegPlate, UFPR-ALPR, and RodoSol-ALPR.
The seven datasets are grouped by acquisition domain and displayed in chronological publication order in Fig. 2.
The benchmark’s vehicle-cropping process is followed by formal definitions and annotation criteria.

A. Preprocessing and Data Extraction

Vehicle crops are obtained through dataset-specific extraction strategies, using existing crops or bounding boxes when available and automated YOLO-based extraction otherwise.

UFPR-VeSV required no additional extraction because it already provides cropped vehicle instances.
Vehicle-Rear, SSIG-SegPlate, and UFPR-ALPR use their provided vehicle bounding boxes directly.
LPLCv2, RodoSol-ALPR, and UFOP require an automated YOLO-based extraction pipeline.
Multiple vehicles in source images can make the final number of vehicle crops exceed the number of original images.

B. Annotation Framework

UVIB assigns independent binary labels for orientation, VMMR suitability, and color clarity, with manual annotation and auditing procedures supporting the dataset-wide labels.

Each vehicle crop receives independent binary annotations for orientation, VMMR suitability, and color clarity.
VMMR suitability marks whether a crop preserves enough visible structure for make and model analysis, with severe occlusion or missing key regions indicating unsuitability.
Color clarity marks whether chromatic information is reliable, while non-color denotes degraded image evidence rather than a physical vehicle-color category.
Manual color-clarity annotation produced Cohen’s Kappa values of 0.69 for UFPR-VeSV and 0.62 for the multi-dataset subset.
A multi-stage auditing protocol reconciled disagreements, reviewed borderline cases, and performed an exhaustive consistency sweep across all 84,835 images.

IV. Experimental Setup

The experiments treat the three UVIB tasks as independent binary classification problems and compare four representative architectures under standardized fine-tuning conditions.

The three UVIB tasks are evaluated independently as binary classification problems.
The benchmark compares EfficientNetV2-S, ResNet-50, ViT/B-16, and YOLO11s-cls, spanning convolutional, residual, transformer, and deployment-oriented architectures.
All images are resized to 224 × 224 pixels, classification heads become two-output layers, and all network parameters are fine-tuned.
Training uses Adam with a learning rate of 10^-4, batch size 32, up to 100 epochs, and early stopping after 12 stagnant validation epochs.
Focal Loss replaces cross-entropy for VMMR Suitability to emphasize harder cases in its highly imbalanced classification task.

A. Evaluation Protocols and Cross-Domain Splits

Four protocols separate mixed-domain evaluation from cross-domain and cross-dataset transfer, using withheld target data to assess generalization under dataset shift.

The benchmark defines four protocols to distinguish mixed-domain performance from stricter transfer settings.
S2G trains on 57,798 surveillance images and tests on 27,037 general-domain images, while G2S reverses the training and testing domains.
All-Datasets pools all sources and uses stratified train, validation, and test splits to measure performance with both acquisition domains represented during training.
CDS combines surveillance and general datasets in both phases to distinguish domain-level variation from dataset-specific bias.
Cross-domain protocols use a 60/40 source-domain training-validation split, full target-domain testing, and no vehicle-identity overlap between domains.
Target datasets are completely withheld during training and validation for the cross-domain and CDS evaluations.

B. Statistical Rigor and Evaluation Metrics

The evaluation uses repeated seeded runs and reports metrics independently before aggregation. Macro F1 is the primary comparison metric because class imbalance can make Accuracy misleading.

Each experiment is repeated with three random seeds while keeping protocol splits fixed.Seeds control data shuffling and initialization of replaced classification heads, estimating variability from stochastic training factors.
Accuracy, per-class F1, and Macro F1 are computed independently from each run before reporting means and standard deviations.
Macro F1 is the primary comparison metric for imbalanced tasks because it weights both classes equally.Accuracy summarizes correctness, while per-class F1 exposes class-specific performance.

V. Results and Discussion

Across architectures, evaluation protocol matters more than model choice: cross-domain and dataset-specific shifts reduce performance, particularly for VMMR Suitability and Color Clarity, while Orientation transfers more reliably.

Overall protocol effects: Cross-domain evaluation causes substantial degradation, especially for VMMR Suitability and Color Clarity, whereas mixed-domain models perform well.
Overall protocol effects: The cross-domain drop cannot be explained by training-set size alone; S2G declines despite training on the larger Surveillance Domain.
Overall protocol effects: Models learn acquisition-specific cues that transfer unreliably to unseen cameras, illumination patterns, and viewpoint distributions.
Overall protocol effects: EfficientNetV2 Macro F1 in CDS is 96.3%, 87.8%, and 82.2% across the three tasks, demonstrating dataset-specific bias despite mixed acquisition domains.CDS performs between one-way domain drops and the All-Datasets baseline.
Overall protocol effects: Cross-domain Macro F1 variability is ±1.4 for S2G, ±1.78 for G2S, and ±1.93 for CDS, versus 0.35 for All-Datasets.The All-Datasets setting minimizes camera-specific variance through joint optimization across datasets.
Task-specific findings: Orientation is the most transferable task, although front views are slightly weaker and errors cluster in ambiguous near-lateral viewpoints.
Task-specific findings: VMMR Suitability is highly imbalanced: cross-domain Macro F1 ranges from 56.6% to 75.9% while Accuracy remains 82.9% to 96.8%.YOLO11s-cls in S2G reaches only 21.3% Unsuitable F1, showing how Accuracy can be dominated by Suitable examples.
Task-specific findings: Color Clarity is sensitive to acquisition bias, with reflections, shadows, night lighting, and infrared-like captures degrading chromatic evidence.

VI. Conclusions

UVIB establishes a public benchmark for three operational vehicle-analysis tasks and evaluates robustness across surveillance and general-domain imagery. The results identify domain shift, particularly for VMMR Suitability and Color Clarity, as a persistent challenge while suggesting model and dataset directions for future systems.

UVIB contains 84,835 images from seven public Brazilian datasets with new annotations for Orientation, VMMR Suitability, and Color Clarity.
Domain shift affects reliability more than model family, with persistent cross-domain limitations especially for VMMR Suitability and Color Clarity.Mixed-domain training performs strongly, while cross-dataset testing provides an intermediate evaluation setting.
EfficientNetV2-S is a strong default baseline, ResNet-50 remains competitive, and YOLO11s-cls is mainly suitable when efficiency outweighs minority-class robustness.
Future extensions include richer occlusion and color annotations, downstream integration, and learning strategies targeting rare-condition coverage and cross-domain degradation.Suggested approaches include multi-task learning, synthetic data generation, and targeted augmentation.