Source-linked AI summary
Evaluating 2D and 3D-Aware Vision Foundation Models for Vehicle Attribute Recognition
Alexandre V. Delazeri, Gabriel E. Lima, Eduil Nascimento, Rayson Laroca, David Menotti
TL;DR
Vehicle recognition supports intelligent transportation applications when ALPR is unreliable because occlusion, viewpoint variation, and poor image quality can make plates illegible. The paper benchmarks 2D and 3D-aware foundation models as frozen feature extractors and finds that 2D self-supervised features are strongest overall, while Depth Anything v2 is more pose-invariant for vehicle type classification.
Problem
Vehicle recognition remains important when real-world surveillance conditions make ALPR unreliable, creating a need for highly discriminative and generalizable feature representations.
Method
The study benchmarks 14 2D and 3D-aware vision foundation models across vehicle type, make, and model recognition using linear probing on frozen representations, followed by non-linear, few-shot, and OOD evaluations.
Results
2D self-supervised models provide the most effective features overall, with DINOv3 achieving 97.7% Ma-Acc for vehicle type and over 93.0% for make and model recognition, while Depth Anything v2 is more pose-invariant for type classification.
Takeaways & Limitations
Current 2D self-supervised representations are strongest for these classification tasks, whereas 3D-aware models require dedicated adaptation strategies to become viable for vehicle recognition.
Takeaways & Limitations
The benchmark does not strictly isolate geometric priors from confounding variables such as differences in pre-training data, model capacity, and architecture.
Abstract
from arXiv · showhide
Vehicle attribute recognition is an important task in intelligent transportation systems, particularly when Automatic License Plate Recognition (ALPR) is unavailable or unreliable. Although vision foundation models have shown strong transferability across domains, their effectiveness for fine-grained vehicle classification remains underexplored. Moreover, given the inherently three-dimensional structure of vehicles, it is unclear whether emerging 3D-aware foundation models offer advantages over standard 2D architectures. This paper presents an empirical benchmark of 14 state-of-the-art 2D and 3D-aware vision foundation models. Using the challenging real-world UFPR-VeSV dataset, we evaluate these models as frozen feature extractors via linear probing for vehicle type, make, and model recognition. We further stress-test the best-performing models under few-shot learning and Out-of-Distribution (OOD) domain shifts. Our results show that standard 2D self-supervised models, particularly DINOv3, substantially outperform 3D-aware models in fine-grained tasks, achieving over 93% Macro-Accuracy for make and model recognition. However, the 3D-aware Depth Anything v2 exhibits stronger invariance to viewing angles in vehicle type classification. These findings motivate hybrid approaches that combine 2D and 3D priors for robust vehicle recognition. Our code is publicly available at https://github.com/UFPR-IPASPPR/3D-Vision-Benchmark/.
I. Introduction
Vehicle attribute recognition complements unreliable ALPR by using global appearance, but remains difficult under surveillance variability and class imbalance. This paper benchmarks whether 2D or 3D-aware foundation-model representations better address these challenges.
- Vehicle recognition supports traffic monitoring, parking management, and forensic analysis in intelligent transportation systems.
- Vehicle attribute recognition uses global vehicle appearance to complement ALPR when occlusion, viewpoint variation, or poor quality make plates illegible.
- Low interclass variance, high intraclass variance, and severe class imbalance make reliable real-world attribute recognition challenging.
- The study asks whether 3D-aware representations provide advantages over standard 2D foundation models for vehicle attribute recognition.
- The benchmark evaluates 5 standard 2D and 9 3D-aware models as frozen feature extractors across vehicle type, make, and model recognition.It also tests top-performing models using few-shot learning, OOD generalization, and non-linear probing.
II. Related Work
Vehicle attribute recognition has progressed from handcrafted features to deep learning and foundation models, but existing evaluations rarely use frozen representations or compare 3D-aware models. This work addresses both gaps with a direct benchmark under real-world surveillance conditions.
- Earlier vehicle attribute methods used handcrafted features and conventional classifiers, while modern approaches primarily use CNNs and ViTs with attention, multiscale, or metric-learning enhancements.
- Explicit 3D approaches used 3D bounding boxes or aligned object models to learn viewpoint-invariant features, but their annotation requirements limited practicality.
- Recent foundation-model studies often fine-tune on target datasets, thereby not assessing transfer from frozen representations.
- Existing evaluations are predominantly limited to standard 2D architectures, leaving the potential of 3D-aware foundation models unexplored.
- This work directly compares state-of-the-art 2D and emerging 3D-aware models as frozen feature extractors under real-world surveillance conditions.
III. Background
Vision foundation models learn transferable representations through self-supervised pretraining, with 2D models emphasizing appearance and 3D-aware models emphasizing geometry and multi-view consistency. Their objectives range from cross-view completion and depth prediction to native 3D representation learning.
- Foundation models are pretrained on massive datasets, often through self-supervision, and can support downstream classification through frozen backbones and linear heads.
- Standard 2D models use masked image modeling or view-matching objectives that primarily capture appearance-based cues.
- 3D-aware models encourage geometric consistency through cross-view learning, which reconstructs masked patches from paired viewpoints and learns viewpoint-invariant representations.
- Depth-supervised and multi-view reconstruction models learn spatial hierarchies, pixel correspondences, depth, and scene structure from 2D images.
- Native 3D models use point clouds, meshes, gaussian splats, or voxels to preserve physical spatial relationships, although scalability is constrained by limited data.
IV. Methodology
The methodology benchmarks frozen 2D and 3D-aware representations on UFPR-VeSV using linear probing, then examines separability, data efficiency, and OOD robustness. Evaluation averages Micro-Accuracy and class-balanced Macro-Accuracy across ten official splits.
- Fourteen foundation models are evaluated across vehicle type, make, and model recognition using linear probing on frozen feature representations.
- The study further applies non-linear probing, few-shot data-efficiency tests, and OOD generalization to the top-performing models from each paradigm.
- UFPR-VeSV contains 24,945 real-world traffic images covering 14 types, 26 makes, and 136 models with a severe long-tailed distribution.
- The dataset includes adverse surveillance conditions and viewpoint annotations supporting orientation-based analyses.
- Results are averaged across 10 official stratified train-validation-test splits using Micro-Accuracy and Macro-Accuracy.Macro-Accuracy weights classes equally so majority classes cannot mask poor minority-class generalization.
- The implementation provides publicly available code and checkpoint specifications for reproducibility.
A. Selected Models and Benchmark Setup
The benchmark prioritizes comparable model capacity and evaluates frozen representations before task-specific classifiers. Features are extracted from model latents using architecture-specific token pooling, then assessed with linear and non-linear probes.
- ViT-Large backbones isolate pre-training objectives from architectural capacity, while DINOv2-based 3D-aware models enable direct comparison with a 2D baseline.
- Frozen feature extraction avoids fine-tuning effects that could mask representational shortcomings.For 3D reconstruction models, embeddings are taken before spatial regression or generation modules.
- Feature construction concatenates CLS and GAP tokens for standard Vision Transformers, with specialized-token replacement or global pooling for other architectures.
- Probes use native feature spaces and inverse-frequency weighted cross-entropy to address class imbalance.Optimization uses Adam with batch size 32, learning rate 10^-4, weight decay 10^-4, up to 100 epochs, and 5-epoch early stopping patience.
- Non-linear probing replaces the linear classifier with a 512-neuron, one-hidden-layer multilayer perceptron using 0.2 dropout.All other training hyperparameters remain identical to the linear baseline.
B. Representation Robustness Analysis
The robustness analysis tests leading 2D and 3D-aware models under limited data and domain shift for vehicle type recognition. The OOD set uses unseen vehicle models assigned to the original type classes.
- Robustness evaluation targets data efficiency and OOD generalization under operational constraints including domain shifts and scarce annotations.The analysis is restricted to vehicle type recognition for a unified scope.
- Training subsets retain 1%, 5%, 10%, or 25% of instances across 10 official UFPR-VeSV splits, while preserving at least one sample per class.Probes are trained from scratch on each subset and evaluated on the original test sets.
- The curated OOD set contains 200 unseen vehicle images captured with similar surveillance infrastructure but representing a different data distribution.Its distribution approximates the original UFPR-VeSV type-class distribution.
- The OOD samples present unseen vehicle models that map directly to the original UFPR-VeSV type classes.
V. Results and Discussion
The benchmark finds that DINOv3 provides the strongest overall frozen representations, while 3D-aware models are more competitive for coarse vehicle type recognition and pose variation. Data-efficiency, OOD, non-linear probing, and error analyses further distinguish the models’ robustness and failure modes.
- Linear probing: 97.7% Ma-Acc for vehicle type and over 93.0% for make and model recognition made DINOv3 the strongest linear-probing model across the evaluated tasks.DINOv3 also consistently outperformed Depth Anything v2 across the 10 official splits, with p < 0.002.
- 3D-aware models: 94.5% and 95.0% Ma-Acc for Depth Anything v1 and v2, respectively, made depth-based models competitive for coarse vehicle type classification.Their performance advantage did not extend to fine-grained make and model recognition.
- 3D-aware models: 3D reconstruction and segmentation architectures struggled with fine-grained recognition, possibly because spatial compression discards high-frequency details or reconstruction objectives entangle features.These explanations are presented as possibilities rather than established causes.
- Robustness and errors: Depth Anything v2 showed stronger pose invariance than DINOv3 for vehicle type classification, with no significant effect from pose at p ≈0.19 versus p < 0.01 for DINOv3.Error analysis also found DINOv3 confusing tractor-trucks with standard trucks, while Depth Anything v2 confused minibuses with buses.
- Non-linear probing: Non-linear probing improved Depth Anything v2 and slightly degraded DINOv3, yet DINOv3 still heavily outperformed it on make and model recognition.The results suggest DINOv3’s features are robust and linearly separable in the studied scenario.
- Few-shot efficiency: 91.8% Ma-Acc for DINOv3 on vehicle type recognition with 25% of the data showed stronger few-shot efficiency than many 3D-aware architectures.Depth Anything v2 degraded more sharply under extreme data scarcity.
VI. Conclusions
The benchmark finds that standard 2D self-supervised models currently provide the most effective frozen features for vehicle attribute recognition, while 3D-aware models underperform. The authors recommend adaptation, controlled comparisons, and hybrid 2D–3D designs as directions for improving robustness.
- Standard 2D self-supervised architectures currently provide the most effective features for vehicle attribute recognition.
- 3D-aware models underperformed because reconstruction and depth objectives may abstract away high-frequency details needed for fine-grained vehicle identities.
- 3D-aware models did not show the expected advantage under Out-of-Distribution domain shifts.
- The observed performance gap is an empirical snapshot of current models, not definitive evidence that 2D representations are inherently superior.
- Future work should investigate parameter-efficient fine-tuning, controlled pre-training comparisons, data-scale effects, and hybrid architectures combining 2D semantics with 3D spatial consistency.