PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Benchmarks: Using VLMs to Reveal Systematic Classification Failures Under Real World Conditions

Dieuwertje Alblas, Alma M. Liezenga, Jan Erik van Woerden, Fedor Taggenbrock, Dalia Aljawaheri, Klamer Schutte

arXiv:2609.11126v1cs.CVcs.AIcs.ET

TL;DR

Reliable V&V of classification models is difficult because manual inspection of erroneous samples is laborious and defence data pose domain and environmental challenges. The paper proposes VLM-based error slice discovery using semantically structured embeddings and generative descriptions. It identifies artificially introduced perturbations in a non-military dataset and clusters military images by surroundings, but current results support analyst assistance rather than fully automated V&V.

  • Problem

    Manual inspection makes systematic failure discovery laborious, while defence domains are underrepresented in VLM training data and contain less diverse surroundings.

  • Method

    The paper introduces VLM-based error slice discovery that uses shared image-text embeddings, residual class embeddings, clustering, and generative VLM descriptions.

  • Results

    The method identified deliberately introduced systematic vulnerabilities in dog-breed data and formed meaningful environment-related clusters, while military clusters showed overlapping descriptions and video-related grouping.

  • Takeaways & Limitations

    VLM-based ESD could accelerate V&V by helping human analysts discover, prioritize, and investigate systematic failure modes rather than replacing them.

  • Takeaways & Limitations

    Military datasets had limited environmental variability, which could steer clustering toward videos or environments and restrict clusters beyond those characteristics.

Abstract

from arXiv · show

Verification and validation (V&V) of classification models is crucial to enable a wide range of sensor processing applications. Currently, the V&V process relies on time-consuming manual inspection of erroneous samples to find meaningful patterns. This work explores the use of Vision Language Models (VLMs) to speed up this laborious process. VLMs are trained to embed images into a semantically meaningful vector representation, from which human-interpretable systematic errors can be distilled. Deploying such VLM-based methods in a defence context introduces two major challenges: (1) the defence domain is underrepresented in the training data of VLMs, and (2) surroundings and context are less diverse than for other domains. This study provides an initial assessment of the suitability of VLM-based methods for V&V of defence applications. We propose a VLM-based error slice detection (ESD) method that independently groups and labels systematic errors made by a classification model. We demonstrate that this method is able to identify operationally-relevant artificially added perturbations in a non-military dataset. In a military context, our method clusters and describes images based on their surroundings, but also exhibits overlap between cluster descriptions. We further investigate the difference in embedding variation between our military and non-military dataset, which remains a topic of interest. Although the results do not yet warrant fully automated V&V through VLM-based ESD, they show that VLMs could be used to accelerate V&V processes in the future.

1. INTRODUCTION

AI vision models are difficult to validate comprehensively because their decision boundaries and operational inputs are complex. VLMs may reduce the labor of finding systematic failures, but defence applications face domain-representation and environmental-diversity challenges.

  • Problem: Black-box decision boundaries, unbounded visual inputs, and weak out-of-distribution generalization complicate systematic failure identification.These factors limit confidence in model behaviour under previously unseen but operationally relevant conditions.
  • Existing V&V: V&V requires understanding which environmental factors, visual attributes, or conditions are associated with model failures on finite test samples.Traditional methods designed for explicitly defined rule-based systems are not directly sufficient for models with complex implicit boundaries.
  • Existing V&V: Manual inspection of saliency maps makes semantic categorization of large numbers of failure cases labor-intensive and difficult to scale.Saliency methods provide qualitative insight into individual decisions, but operators must inspect images and interpret highlighted regions.
  • VLM opportunity: Contrastive VLMs map images and text into a semantically structured shared latent space, enabling distances to represent semantic similarity.Generative VLMs extend this foundation toward automatic interpretation of image semantics.
  • Defence challenges: Defence-oriented ESD faces unreliable embeddings from underrepresented training data and reduced environmental diversity compared with academic datasets.Operational military data also tends to have lower quality and narrower environmental coverage.

2. MATERIALS & METHODS

The paper evaluates VLMs for defence-oriented AI V&V and introduces an automatic error slice discovery method inspired by Domino.

  • Contribution: The study introduces a VLM-based automatic error slice discovery method for evaluating AI models in a defence context.The method leverages VLMs’ semantically structured latent representations to investigate systematic errors.

2.1 VLM preliminaries

VLMs connect images and semantic inputs in a shared embedding space whose geometry reflects semantic similarity. The method uses this structure to separate contextual information from dominant object content.

  • VLM spaces: A contrastive VLM maps image space and semantic space into a shared d-dimensional embedding space.The dataset consists of n images assigned to one of k classes.
  • ESD rationale: The ESD overview applies the classifier, embeds test images, subtracts class means, clusters residuals, and prompts a generative VLM to describe cluster similarities.The subtraction is intended to shift attention from the main object toward surrounding context.
  • Embedding geometry: Distances in the joint embedding space are structured to reflect semantic similarity, although individual embedding points are not human-interpretable.Images and text remain human-interpretable, but their distances are less straightforward to interpret directly.
  • Embedding geometry: VLM embeddings combine objects and environmental elements but tend to be dominated by the largest or most salient object.Local linearity enables embedding manipulation to disentangle objects from contextual elements.

2.2 VLM-based Error Slice Discovery

The proposed ESD pipeline turns classifier outcomes and VLM embeddings into interpretable clusters of semantically related test samples. Residual embeddings emphasize surroundings, and a generative VLM labels the resulting clusters.

  • Pipeline: The pipeline embeds classified test images, subtracts their class-level average embeddings, clusters the residuals, and generates cluster descriptions.Subtracting the class average is intended to shift representations toward surrounding environments rather than the main object.
  • Cluster labeling: Generative VLMs describe each cluster from a collage of multiple images using five ranked words.This replaces Domino’s selection of a textual description from a corpus based on embedding similarity.
  • Interpretation: Clusters may contain both correctly and wrongly classified samples, allowing environmental patterns to be related to model performance.The resulting descriptions are intended to be human-interpretable summaries of semantically related test samples.
  • Datasets: The datasets span dog breeds and military vehicles, with public sources alongside more challenging vehicle data extracted from video-frame bounding boxes.The military-vehicle datasets provide a contrasting domain and data setting for assessing the method.

2.3 Data

The study uses four datasets spanning dog breeds and military vehicles, with variation in VLM-domain representation and operational difficulty. The military datasets range from high-quality centered imagery to low-resolution, adverse-condition recordings.

  • 2.3 Data: Four datasets cover dog breeds and military vehicles, enabling comparison across domains with different VLM training-data representation and environmental variation.Dog images are expected to be more represented and more varied in surroundings than military-vehicle images.
  • 2.3 Data: The dog-breed dataset contains 8,040 web images across 93 breeds, with high image quality, centered subjects, varied environments, and small inter-class differences.
  • 2.3 Data: The public military-vehicle dataset contains approximately 50 high-resolution images per class across diverse vehicle categories, mostly with centered objects.
  • 2.3 Data: The Ukraine vehicle dataset uses 289 cropped video-frame samples with manually labelled functional classes, low resolution, and challenging operational conditions.
  • 2.3 Data: The STORE dataset contains 1,813 cropped bounding boxes from 80 videos recorded from air-to-ground and ground-to-ground perspectives under varying climates, rainfall, snow, and occlusion.
  • 2.3 Data: Artificial operational challenges are simulated with motion blur, random snow, slice blackout, bars blackout, and random rain at severities of 20%, 40%, 60%, and 80%.

2.4 Evaluation

The evaluation combines quantitative cluster assessment with qualitative inspection of generated descriptions and cluster image content. It uses silhouette score alongside checks for semantic overlap.

  • 2.4 Evaluation: Clusters are evaluated quantitatively with silhouette score and qualitatively through generated descriptions and the semantic content of their images.Silhouette score ranges from -1 to 1, with higher values indicating better separation.

2.5 Implementation details

Implementation covers the experimental setup, classification models, VLMs, datasets, and clustering pipeline. The Ukraine evaluation showed accuracy differences across videos, while Figure 3 illustrates perturbations used in the experiments.

  • 2.5 Implementation details: The implementation-details section specifies the models, algorithms, and datasets used in the experiments.
  • 2.5 Implementation details: Figure 3 depicts perturbations and occlusions applied at increasing severities of 20%, 40%, 60%, and 80%.
  • 2.5 Implementation details: The Ukraine vehicle evaluation produced accuracies of 81% and 88% for videos 1 and 3, but only 45% for video 2.
  • 2.5 Implementation details: The ESD pipeline uses CLIP PE-Core-L-14-336 for image and class-level embeddings and GPT-5 mini for cluster descriptions.
  • 2.5 Implementation details: The dog-breed experiment fine-tunes a classifier on training data, uses validation data for class-level embeddings, and reserves the test split for full-pipeline validation.
  • 2.5 Implementation details: Residual embeddings are reduced to 10 dimensions with UMAP before HDBSCAN clustering using a minimum cluster size of 5 and minimum sample size of 25.

3. EXPERIMENTS & RESULTS

Experiments show that VLM-based error slice discovery forms semantically coherent clusters and detects systematic perturbations in dog-breed data, while military-data clusters are strongly shaped by video and dataset context. Embedding analyses further show dataset-specific variation patterns and support the role of mean-class subtraction in exposing contextual structure.

  • 3.1 Dog breed classification: The dog-breed experiment produced nine semantically coherent clusters with a silhouette score of 0.67, although 469 images (53%) remained unassigned.Cluster descriptions agreed with qualitative inspection, with attribute agreement ranging from 79% to 100%.
  • 3.1.1 Cluster quality: Cluster-level performance exposed operationally relevant failures, with cluster 6 performing below the model’s 90% average under clutter, low resolution, blur, and varied lighting.Performance was better for dogs in snow or grass.
  • 3.1.2 Systematic perturbations: The method detected systematic perturbations: snow and rain grouped together, motion blur and slide blackout grouped together, and bars blackout formed a separate cluster.Transform severity did not affect cluster assignment, indicating sensitivity to subtle alterations.
  • 3.2.1 Cluster quality: Military vehicle dataset 2B yielded tight clusters with a silhouette score of 0.83, but descriptions overlapped because clustering largely followed the three source videos.Video 2 nearly formed cluster 0, while videos 1 and 3 were distributed across multiple clusters.
  • 3.2.1 Cluster quality: Combining military datasets produced 22 clusters, most dominated by a single dataset, indicating that clustering was largely driven by dataset-specific characteristics rather than clearly distinct vehicle contexts.The resulting differences between clusters were often subtle.
  • 3.3 Dataset analysis: PCA showed that over 90% of dataset 2B variation was explained by 30 components, versus 99 for all military datasets and 140 for dog breeds, while residual embeddings required more components.Pairwise-distance analysis likewise found greater residual variation, but no clear overall variance difference between dog-breed and combined military datasets.

4. DISCUSSION

The VLM-based ESD method identified systematic vulnerabilities in dog-breed data but produced fewer novel insights for military-vehicle data. Its usefulness is constrained by limited environmental variation, incomplete clustering coverage, subjective evaluation, and limited ablation evidence.

  • The method successfully identified deliberately introduced systematic vulnerabilities in the dog-breed dataset, while military datasets yielded fewer novel insights.For military data, cluster descriptions overlapped and clusters corresponded to individual videos.
  • Most dog-breed samples were not assigned to a cluster, reducing analysis coverage and motivating alternative clustering approaches.The discussion attributes this possibly to the clustering algorithm.
  • 4.1 Limitations: Limited environmental variability in military datasets can steer clustering toward videos or environments rather than other characteristics.Dataset 2B contained images from three videos, while 2C covered only two locations.
  • 4.1 Limitations: The silhouette score indicates cluster cohesion but is not representative of cluster usefulness, so qualitative assessment remains necessary and subject to author bias.The study therefore combines quantitative and qualitative evaluation.
  • 4.1 Limitations: The military-domain and residual-embedding analyses do not constitute a full ablation study of the factors affecting method performance.Unexamined factors include the selected VLM, domain knowledge, and average and residual embedding calculations.
  • 4.2 Future work: Future work should investigate VLM concept purity, spurious correlations, and iterative residual subtraction to reveal subtler variation and error modes.The individual contributions of training-data underrepresentation and limited surroundings remain unresolved.
  • 4.2 Future work: Using a different VLM for cluster descriptions than for cluster formation may affect how the underlying embedding concepts are distilled.Dictionary learning is suggested as one alternative for transferring from embedding space to semantic space.
  • 4.2 Future work: AI-assisted workflows could combine VLM scalability with human expertise rather than relying solely on full automation.The proposed direction supports, prioritizes, and steers human analysts during V&V.

5. CONCLUSION

Current VLM-based ESD methods do not yet support fully automated AI-model V&V. They may nevertheless help human analysts discover, prioritize, and investigate systematic errors, with effectiveness depending on domain and semantic demands.

  • Current VLM-based ESD methods are not yet suitable to fully automate AI-model V&V.
  • VLM-based methods may accelerate V&V by helping human analysts identify and characterize potential failure modes, especially in third-party models.
  • Semantically structured VLM embeddings can characterize training and test data, identify underrepresented scenarios, and reveal coverage gaps.
  • The most promising role of VLMs is supporting human analysts in discovering, prioritizing, and investigating systematic model errors.Effectiveness depends on the target domain and the level of semantic interpretation required.

APPENDIX A. PROMPT FOR CLUSTER DESCRIPTION GENERATION

The appendix specifies the prompt used to generate concise cluster descriptions from sampled images, focusing on visual factors that may cause classification errors.

  • The prompt asks a generative VLM to describe image similarities that might make a classification model misclassify samples.It directs attention to environment, object position, lighting, backgrounds, angle, perturbations, augmentations, and occlusions while excluding dog breed or class.
Loading 2609.11126v1…