Source-linked AI summary
VeriCam: A Verification Baseline for the Classification of Unknown Data
Lucas Wojcik, Gabriel E. Lima, Sergio M. Silva, Eduil Nascimento, David Menotti
TL;DR
VeriCam addresses zero-shot classification when the number of classes is unknown and existing methods struggle to separate subtly different classes. It reframes inference as pairwise verification, builds a similarity graph, and clusters unseen data; on LPLCv2, cross-device verification reaches 93.41 F1-score and Leiden clustering reaches 80.14 V-Measure.
Problem
Standard classification models require a fixed, known number of classes, while existing methods lack a clear way to separate similar but subtly different classes.
Method
VeriCam uses a verification network to estimate whether image pairs share a latent class, constructs a pairwise-similarity graph, and applies naive or Leiden clustering to identify classes.
Results
93.41 F1-score and 80.14 V-Measure are reported for cross-device verification and Leiden clustering, respectively, while cross-device verification performance is lower than within-device performance.
Takeaways & Limitations
Device recognition is an important step toward fair, unbiased ALPR benchmarks because device contamination can skew intra-dataset testing results.
Takeaways & Limitations
The approach often fails on lesser-represented classes and similar classes, leaving noise in zero-shot clustering and requiring further research.
Abstract
from arXiv · showhide
The advent of foundation models have enabled a new era in zero-shot classification. Yet, key challenges persist. Despite their impressive generalization power that leverages the immense pre-training knowledge, both foundation models for image and text as well as vision-text hybrids lack the representational power needed for fine-grained, minutiae-based class separation that some real-world tasks require. To address the current gaps in the literature, we propose VeriCam, a pipeline designed to learn highly specialized features that enable classification of unknown classes in unseen data. VeriCam works by leveraging the representation power of image models trained for the verification task, where the model develops an intricate feature space that incorporates fine-grained details. By training a model to discriminate between pairs of images from the same and different classes, a relational graph is constructed, representing the class relationships between data points. We then present two approaches for graph clustering: a naive algorithm and a specific setup for the Leiden graph clustering algorithm. The pipeline is validated on the LPLCv2 dataset, which comprises real-world traffic surveillance images. We show that the dataset carries an inherent capture device bias that is posed as a generalization challenge for downstream License Plate recognition tasks such as OCR. As such, we dynamically identify capture devices with a label-agnostic approach, enabling the construction of a fair and unbiased benchmark. In the cross-device scenario, our pipeline reaches an F1-Score of 93.45 in the verification baseline and a V-Measure score of 80.13 in the clustering step. All code is publicly available at https://github.com/lmlwojcik/VeriCam
I. INTRODUCTION
VeriCam addresses unknown-class classification by using verification-based representations and relational clustering rather than a fixed label space. The paper applies this approach to capture-device recognition for fairer ALPR evaluation.
- Standard classification models cannot handle an unknown and unconstrained number of classes because their output neurons encode predefined classes.
- Zero-shot and OOD approaches target unseen data but face limitations including lower accuracy, complexity, or insufficient domain-specific discrimination.
- Capture-device contamination can skew intra-dataset ALPR experiments, making device recognition important for constructing fair and unbiased benchmarks.
- VeriCam reframes zero-shot inference as pairwise verification, estimating whether two samples share a latent class and clustering the resulting graph.
- The contributions include a verification-network method for zero-shot classification and two experimentally validated algorithms.
II. RELATED WORK
Related work extends OOD detection toward zero-shot classification, but existing methods often struggle with domain-specific and subtly different classes. VeriCam addresses this gap through local relational comparisons instead of globally defined knowledge.
- OOD methods detect samples outside known classes, while zero-shot classification additionally aims to recognize and classify unseen classes.
- Text-based zero-shot methods do not transfer directly to images because domain-specific data requires specialized visual features.
- Image zero-shot approaches using synthetic data or text descriptions remain limited by the general knowledge of their underlying models.
- Existing methods lack a clear way to separate similar but subtly different classes that require highly specialized knowledge.
- VeriCam uses local proximity features and relational similarity rather than globally defined knowledge or crisp labels.
III. METHODOLOGY
The methodology learns feature descriptors from local instance relationships, using verification-inspired comparisons to distinguish similar and different classes. These relationships support robust feature representation for unknown-class classification.
- The method compares instances to group highly similar examples and separate dissimilar ones, drawing on verification-based facial-recognition methods.
A. Proposed Pipeline
VeriCam trains a verification model to produce pairwise similarity and converts those relations into a graph for clustering. It evaluates both a naive threshold-based procedure and Leiden clustering for label-agnostic class discovery.
- The pipeline trains a ViT-based feature descriptor to determine whether two images belong to the same class, then performs graph clustering.
- Triplet loss optimizes cosine distance between feature vectors, where distance ranges from 0 for equal vectors to 2 for completely different vectors.
- The affinity matrix forms an undirected graph whose vertices are instances and whose edge weights encode cosine similarity, after which clustering yields class labels.
- The naive algorithm initializes known classes and assigns each instance by mean similarity, creating a new class when all class similarities fall below a threshold.
- Leiden clustering with the Constant Potts Model is used because the dataset contains more than six thousand test images and spectral clustering is prohibitively expensive.
B. The Dataset
LPLCv2 provides a large camera-annotated traffic-surveillance dataset for dynamically identifying capture devices, but its device classes are highly imbalanced.
- LPLCv2 contains 37,099 images, including 34,760 annotated with camera IDs defined by installation location.Each device is identified through the scene shown in its images.
- The working dataset includes 33,668 images from devices associated with at least ten images.This filtering ensures that every selected device class is sufficiently represented.
- Approximately 50% of the dataset, or 16,644 images, comes from 15% of the classes, corresponding to 90 devices.The selected data span 612 devices overall, revealing substantial class imbalance.
IV. EXPERIMENTS
The experiments evaluate verification and graph-based clustering under intra-device and cross-device splits, with OCR used to investigate device bias in a downstream task.
- Experimental scenarios: Intra-device splits use a 60/20/20 image partition so validation and testing contain unseen images from known devices.Images from each device are divided among training, validation, and testing.
- Experimental scenarios: Cross-device splits assign all images from each of 612 devices exclusively to training, validation, or testing, making evaluation occur on unknown devices.The same partitioning schema is applied at device level rather than image level.
- Evaluation procedure: Verification uses Triplet loss, followed by evaluation on 50,000 test-image pairs with a 0.5 genuine-to-impostor ratio using binary accuracy and F1-score.The pairs form a static baseline for testing the verification model.
- Evaluation procedure: Clustering is evaluated with V-measure at β = 1.0 because the metric is independent of label IDs and measures zero-shot labeling efficiency.The algorithms are evaluated with training labels available as prior knowledge.
- Downstream OCR evaluation: A PARSeq-tiny OCR model is trained on the same intra-device and cross-device splits to investigate potential device bias in a real-world OCR task.Both pretrained and from-scratch versions are trained using standard parameters.
A. Training Parameters
The verification model uses standardized 224×224 inputs and a ViT-b16 trained from scratch with triplet loss, while clustering uses fixed naive and Leiden settings.
- Verification: Images are resized to 224×224 while preserving aspect ratio, centered, and padded with gray pixels.The verification model is trained on a NVIDIA RTX 6000 GPU with dynamically generated training and validation triplets.
- Verification: The verification model uses a ViT-b16 architecture trained from scratch with triplet loss.Verification hyperparameters are listed in Table II.
- Clustering: The naive clustering algorithm assigns images to known clusters using a threshold of 0.6.
- Clustering: Leiden clustering uses the Constant Potts Model quality function with resolution parameter 0.8.The setup is intended for tightly knit communities expected at high verification accuracy.
V. RESULTS
The experiments show device bias across verification, unknown-class clustering, and OCR, with prior knowledge helping known-device classification but harming cross-device performance. Leiden clustering reaches 80.14 V-Measure, while lower recall leaves classes fragmented and cross-device OCR accuracy declines.
- Verification baseline: Cross-device verification performance is significantly lower, exposing capture-device bias and limited generalization between dataset partitions.The authors attribute this to non-homogeneous data, where training on one partition does not necessarily generalize to another.
- Verification baseline: Cross-device verification has lower Recall because many genuine image pairs are rejected as false negatives.This indicates that distinguishing genuine pairs is more difficult across devices.
- Naive algorithm: A priori knowledge improves intra-device performance but worsens cross-device efficacy because cross-device distributions fall outside the training range.The results nevertheless indicate that known information is often desirable for robust algorithm execution.
- Leiden algorithm: 80.14 V-Measure is achieved by the Leiden clustering approach, whose performance is tied to verification efficiency and remains improvable through better verification.The clustering step is intended to filter noise from erroneous verification relationships.
- Leiden algorithm: Lower recall produces good homogeneity but low completeness because some classes are split into multiple chunks despite little within-class contamination.The paper identifies this fragmentation as the main limitation of the proposed method.
- OCR results: Cross-device whole-plate OCR accuracy drops from 95.29 to 93.41, a decrease of 1.88 percentage points, especially affecting illegible low-resolution plates.The OCR results are stratified by plate-wise legibility, with illegible plates most impacted.
VI. CONCLUSION
VeriCam presents zero-shot classification through verification-based graph clustering, with initial results on LPLCv2 but remaining challenges in class separation and performance. Future work targets test-time correction, improved inter-class separation, and broader validation.
- VI. CONCLUSION: VeriCam uses verification-based graph clustering for automatic detection of new classes, combining a naive algorithm with modular Leiden-based community detection.The approach is demonstrated on the LPLCv2 dataset.
- VI. CONCLUSION: The method struggles with lesser represented classes and similar classes, while its zero-shot performance remains unsatisfactory.Verification errors between similar classes introduce noise into the zero-shot classification step.
- VI. CONCLUSION: Future work will use test-time information to correct earlier mistakes, improve separation between similar classes, and validate the approach on additional datasets.The proposed extensions rely partly on high presumed within-class cohesion at strong verification performance levels.