PaperlayerView exact revision on arXiv

Source-linked AI summary

DASH: Detection and Assessment of Systematic Hallucinations of VLMs

Maximilian Augustin, Yannic Neuhaus, Matthias Hein

arXiv:2503.23573v1cs.CVcs.AIcs.LG

TL;DR

Existing benchmarks use curated datasets and do not systematically assess the images that trigger VLM object hallucinations, limiting their fit to open-world use. DASH combines text- and image-based queries to find real-image clusters that trigger false-positive hallucinations. Across three models, DASH-OPT finds more images and clusters than DASH-LLM, and initial mitigation experiments suggest DASH could contribute to future VLM training pipelines.

  • Problem

    Existing benchmarks use curated datasets and do not systematically assess which images trigger object hallucinations, limiting their reflection of open-world VLM use.

    Source: §1 · p. 1

  • Method

    DASH uses LLM-generated text queries and optimized diffusion-generated images to retrieve real images from ReLAION-5B and identify semantically similar images that trigger false-positive hallucinations.

    Source: §3 · p. 2, §1 · p. 2, §3.2 · p. 3

  • Results

    Across PaliGemma and two LLaVA-NeXT models, DASH-OPT finds more images and clusters than DASH-LLM.

    Source: §4.1 · p. 5

  • Takeaways & Limitations

    Initial mitigation experiments suggest DASH could be a valuable addition to future VLM training pipelines.

    Source: §5 · p. 8

    DASH-LLM's text-query generation is agnostic to the specific VLM being used.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Vision-language models (VLMs) are prone to object hallucinations, where they erroneously indicate the presenceof certain objects in an image. Existing benchmarks quantify hallucinations using relatively small, labeled datasets. However, this approach is i) insufficient to assess hallucinations that arise in open-world settings, where VLMs are widely used, and ii) inadequate for detecting systematic errors in VLMs. We propose DASH (Detection and Assessment of Systematic Hallucinations), an automatic, large-scale pipeline designed to identify systematic hallucinations of VLMs on real-world images in an open-world setting. A key component is DASH-OPT for image-based retrieval, where we optimize over the ''natural image manifold'' to generate images that mislead the VLM. The output of DASH consists of clusters of real and semantically similar images for which the VLM hallucinates an object. We apply DASH to PaliGemma and two LLaVA-NeXT models across 380 object classes and, in total, find more than 19k clusters with 950k images. We study the transfer of the identified systematic hallucinations to other VLMs and show that fine-tuning PaliGemma with the model-specific images obtained with DASH mitigates object hallucinations. Code and data are available at https://YanNeu.github.io/DASH.

1. Introduction

VLMs can falsely report objects in images, while existing benchmarks rely on small curated datasets and do not systematically reveal the image contexts that trigger errors. DASH targets open-world false-positive hallucinations with automated retrieval and clustering.

1. Introduction

  • Existing benchmarks such as POPE [24] and AMBER [51] use curated datasets such as MSCOCO [25], limiting their assessment of object hallucinations.They do not systematically characterize the image types that trigger hallucinations, and their small object-class coverage does not reflect open-world VLM use.Source: §1 · p. 1
  • The paper focuses on false-positive hallucinations: a VLM answers “Yes” that an object is present when it is absent.The authors distinguish this from false-negative errors, which they say existing detection benchmarks can check.Source: §1 · p. 1
  • DASH is a fully automated, human-label-free pipeline that identifies systematic false-positive hallucinations by clustering semantically similar images that trigger them.Its components include LLM-generated text queries and image-based queries optimized to induce hallucinations; the authors report more than 19k clusters comprising over 950k images.Source: §1 · p. 2

2. Related Work

Prior work benchmarks hallucinations in captions and factual image questions, but the authors characterize several benchmarks as limited in scale or object coverage. Related work also includes mitigation methods and guided image generation for model debugging.

Hallucinations of VLMs: Benchmarks and Mitigation

  • CHAIR [41] benchmarks hallucinations in short captions, while POPE [24] targets factual yes/no errors but covers only 80 MSCOCO objects.AMBER [51] includes existential hallucinations, while HALO-QUEST [53] and MMVP [50] cover multiple modalities but are described as too small for sufficient statistics on specific errors.Source: §2 · p. 2
  • Hallucination mitigation work includes visual contrastive decoding [21], robust instruction tuning [26], LURE [62], and M-HalDetect [13].The passage characterizes the mitigation literature as limited; M-HalDetect is described as a type I detection and mitigation technique.Source: §2 · p. 2

Spurious Correlations/Hallucinations in Image Classification

  • Image-classification research studies spurious correlations, including Spurious ImageNet [34], a hallucination benchmark for 100 ImageNet classes.The passage gives a cow-on-a-beach example in which the absence of grass is associated with failure to recognize a cow.Source: §2 · p. 2

Debugging of ML Models using Guided Image Generation

  • Guided image generation has been used to probe model errors, including deceptive CLIP images [61], rare-subgroup classifier errors [32], and optimization-guided debugging with DiG-IN.Zhang et al. [61] report partial transfer of generated-image findings to VQA-type tasks in miniGPT4 and LLaVA-1.5.Source: §2 · p. 2

3. DASH: Detection and Assessment of Systematic Hallucinations

DASH seeks systematic false-positive hallucinations by finding semantically similar real images that trigger them, searching web-scale ReLAION-5B rather than relying on small benchmark datasets. It combines text- and image-based query generation with retrieval, verification, and clustering.

3. DASH: Detection and Assessment of Systematic Hallucinations

  • DASH is a fully automatic, label-free pipeline that searches ReLAION-5B for clusters of semantically similar real images that trigger object false-positive hallucinations.Its open-world search addresses the limits of small benchmarks and avoids exhaustive testing of every image–object pair in ReLAION-5B.Source: §3 · pp. 2–3, Figure 2 · p. 3

3.1. DASH-LLM

  • DASH-LLM uses Llama 3.1-70B with a carefully crafted system prompt to generate 50 text prompts per object, targeting false recognition from spurious features while omitting object names and parts.The approach is entirely agnostic to the specific VLM being used.Source: §3.1 · p. 3

3.2. DASH-OPT

  • DASH-OPT optimizes diffusion conditioning to produce natural-manifold images that elicit a VLM “yes” while discouraging object-detector confidence; it yields more clusters and higher success than DASH-LLM.It initializes from the same 50 LLM prompts, then steers images away from those that fail to fool the VLM or contain the object.Source: §3.2 · pp. 3–4

3.3. Exploration, Exploitation and Clustering

  • Exploration filters retrieved images for detector-flagged object presence and VLM false positives; exploitation retrieves semantically similar neighbors, filters near-duplicates, and clusters successful images.Exploration aims for diverse candidates, whereas exploitation aims for high VLM acceptance and systematic errors; both stages use ReLAION-5B retrieval.Source: §3.3 · p. 4

4. Experiments

Across 380 object categories, the experiments evaluate DASH on three VLMs, compare its retrieval and transfer behavior, test the DASH-B benchmark, and assess fine-tuning PaliGemma with DASH images.

4. Experiments

  • The evaluation covers 380 object categories from several datasets and tests PaliGemma and two LLaVA-NeXT variants using ReLAION-5B retrieval.The categories include COCO, Objects365, ImageNet, and frequency-based OpenImages subsets.Source: §4 · pp. 4–5

4.1. Results of DASH

  • DASH-OPT retrieves more images and identifies more hallucination clusters than DASH-LLM across all three evaluated VLMs.Image counts for PaliGemma, LLaVA-NeXT Vicuna, and LLaVA-NeXT Mistral are 221.7K, 252.0K, and 133.8K for DASH-OPT versus 99.3K, 162.4K, and 78.5K for DASH-LLM; cluster counts are 3895/4632/3229 versus 1892/3632/2001.Source: Table 1 · p. 5, §4.1 · p. 5

4.2. Transfer to unseen VLMs

  • Transfer rates vary with model components: LLaVA-NeXT’s LLM backbone and Prismatic’s vision encoder have larger effects than Qwen2-VL’s LLM size.For Prismatic, CLIP averages a 60% transfer rate versus 43% for SigLIP; for Qwen2-VL, the rates are 18% and 19% for the two sizes.Source: §4.2 · p. 7

4.3. DASH-B: Object Hallucination Benchmark

  • On DASH-B, models have a lower true negative rate than on POPE, indicating that DASH-B is less saturated for measuring false-positive hallucinations.The reported TNR is 48.6% ± 10.1% on DASH-B, compared with 96.0% ± 1.2% on POPE.Source: §4.3 · pp. 7–8

4.4. Fine-Tuning on DASH

  • Fine-tuning PaliGemma with DASH images improves accuracy on DASH-B, AMBER Existence, and R-Bench, while accuracy decreases slightly on POPE.DASH-B accuracy improves by 11.6%; the authors also report captioning and VQA drops and say curriculum integration is out of scope.Source: Table 4 · p. 8, §4.4 · p. 8

5. Conclusion and Limitations

DASH identifies systematic hallucinations in open-world images, where such errors persist despite the limits of curated benchmarks. The paper also outlines supplementary analyses and query-generation details.

5. Conclusion and Limitations

  • DASH is presented as an effective automatic pipeline, but exhaustive coverage is impossible and underrepresented images in ReLAION-5B can leave too few similar examples to establish a systematic hallucination.A conservative object-detector threshold may also limit assessment of the most advanced VLMs.Source: §5 · p. 8

A. Overview

  • The appendices cover query creation, DASH-OPT optimization, retrieval, object-frequency effects, model results, transfer, benchmark construction, mitigation, and prompt generalization.They also include analyses of detector performance, COCO annotation errors, and a possible reverse-task application.Source: A. Overview · p. 13

B. DASH-LLM Prompt

  • DASH-LLM queries are initialized with 50 Llama-generated prompts, then refined with a follow-up prompt to remove object references or duplicates.The same initial queries seed image-query generation in DASH-OPT.Source: B. DASH-LLM Prompt · p. 13

C. DASH-OPT Optimization

DASH-OPT finds more systematic hallucinations for rare objects, while its optimized image queries can help uncover rarer failures among frequent objects. The section also illustrates the generated queries and describes optimization and retrieval choices.

C. DASH-OPT Optimization

  • DASH-OPT updates the initial latent and both SDXL text encodings for 25 Adam steps, selecting the generated image with the lowest loss because optimization is highly non-convex.The optimization takes around 50 seconds for PaliGemma and one minute for LLaVA-NeXT models on an NVIDIA A100 GPU with 80GB memory.Source: C. DASH-OPT Optimization · p. 13

D. Retrieval process, exploration and exploitation

  • DASH removes near-duplicates above a DreamSim similarity score of 0.9, then merges retrieval pre-clusters using average-linkage DreamSim distances with a maximum merge threshold of 0.6.The ReLAION-5B index uses OpenCLIP ViT-H.Source: D. Retrieval process, exploration and… · p. 14

E. Impact of object occurence frequency on object hallucinations

  • DASH finds an average of 506 images per very rare object, and optimized queries yield significantly more images per object than DASH-LLM for frequent-object splits.The rare-object vulnerability trend also holds for LLaVA-NeXT Vicuna and Mistral.Source: E. Impact of object occurence… · p. 14

F.1. Additional qualitative examples

The additional examples show substantial type-II hallucinations across all three VLMs. Their image clusters include both contextually related associations and more unexpected semantic or visual confusions.

Additional qualitative examples

  • All three VLMs show substantial type-II hallucinations, with clusters ranging from semantic associations to objects or scenes unrelated to the queried class.Examples include academic imagery for “Academic Gown,” winter gear for “Dogsled,” and wild cats for “Agama.”Source: §F.1 · pp. 14–15
  • PaliGemma’s DASH-LLM clusters link “Barracouta” to Minecraft harbors and “Fireboat” to police water cannons, while DASH-OPT surfaces bath bombs for “Bathtub” and oranges for “Puck.”The examples range from contextual association to confusion based on shape or related items.Source: §F.1 · p. 14
  • LLaVA-NeXT Vicuna’s DASH-OPT clusters associate “Strawberry” with festive wreaths and “Beehive” with portraits, including women with beehive hairstyles.The strawberry association is attributed as a possibility to the red-and-green palette shared with decorations.Source: §F.1 · pp. 14–15
  • LLaVA-NeXT Mistral’s DASH-LLM clusters connect “Dumbbell” to exercise balls, while DASH-OPT includes shape-based confusion such as water bottles for “Horizontal Bar” and color-based confusion such as blue furniture for “Bluehead.”Other clusters include contextual links such as “Shallot” with kitchens and “Bulletproof Vest” with surveillance rooms.Source: §F.1 · p. 15

F.2. All Clusters Visualizations

The visualizations expose systematic clusters that can reflect semantic and contextual confusion, including scenes far removed from an object’s appearance. Comparisons with reference datasets and exploration diversity further illustrate the open-world search’s findings.

F.2. All Clusters Visualizations

  • Ptarmigan clusters include mountain landscapes, unrelated objects, and abstract scenes; place names such as Ptarmigan Peak and Ptarmigan Ridge may explain some mountain associations.DASH-OPT also finds unusual cases including auroras and surreal artwork.Source: §F.2 · p. 15
  • For Baumkuchen, DASH-LLM finds no images of the cake, while DASH-OPT reveals clusters of 111 fountain-pen images and 66 wooden-kitchen-utensil images.The cluster of eight cake images contains no Baumkuchen.Source: §F.2 · p. 15

F.3. DASH vs Reference Datasets

  • Neither COCO’s 80K samples nor Objects365’s 1.7M samples contain nearest neighbors that trigger PaliGemma’s systematic errors found by DASH.DASH examples include colorful Wellington boots detected as “apple” and Baobab trees as “sausage.”Source: §F.3 · p. 15

F.4. Larger exploration range of DASH-OPT over DASH-LLM

  • DASH-OPT achieves greater image diversity than DASH-LLM across PaliGemma, LLaVA-NeXT Vicuna, and LLaVA-NeXT Mistral.Source: §F.4 · p. 16

G. Object Detector: False Negative Rate vs False Positive Rate

The detector’s low threshold supports conservative filtering, but human review finds ambiguous labels and missed objects in specific categories and image conditions.

G. Object Detector: False Negative Rate vs False Positive Rate

  • 5.2% of manually reviewed images contained the object and 7.8% were ambiguous, in a 10-image-per-object audit of DASH-OPT on PaliGemma.The audit was intended to verify the automatic pipeline, especially its conservative detector threshold.Source: G. Object Detector: False Negative… · p. 16

G. Object Detector: False Rate vs False Positive Rate

  • Human reviewers found ambiguity from low resolution and visually similar alternatives, including barn versus house, Kai yang versus kebab, and airplane versus train interiors.For cowry, reviewers could not guarantee that small sea snails were absent even when they could not identify any.Source: G. Object Detector: False Negative… · p. 16

G. Object Detector: False Negative Rate vs False Positive Rate

  • Objects365 labels can themselves make presence judgments uncertain: “Glasses” means eyewear and “Soccer” means a soccer ball, not the sport.The label definitions can make images containing glass objects or soccer referees difficult to classify as yes or no.Source: G. Object Detector: False Negative… · p. 16
  • Detector failures included small objects, missed visible instances, truck or van misclassification, and distribution shifts such as plush mice or colored eggs.All cited failure-case images had detector confidence below 0.1 and were not rejected by the pipeline.Source: G. Object Detector: False Negative… · p. 16
  • The authors conclude that the detector generally performs well but struggles with particular categories and object variations.The cited limitations include ambiguity in object definitions and difficulty detecting certain variations.Source: G. Object Detector: False Negative… · p. 16

H. Effect of COCO annotation errors on POPE

The section revisits POPE false positives, describes model and benchmark choices, and compares newer models while building DASH-B from transferred, human-checked retrieval results.

H. Effect of COCO annotation errors on POPE

  • 35 of PaliGemma’s 137 POPE false positives (25.5%) actually contained the object, while 31 (22.6%) were ambiguous, indicating substantial label noise among the remaining false positives.The images were re-annotated as yes, no, or ambiguous; the authors conclude POPE is saturated.Source: H. Effect of COCO annotation errors on… · p. 16, Figure 25 · p. 34

I.1. VLM Models

  • The transfer experiments use official model checkpoints, with Transformers for all models except Prismatic, which uses its official implementation.The authors note that Qwen2-VL uses a custom 675m-parameter ViT, not SigLIP as stated in an earlier table.Source: §I.1 · p. 35

I.2. True positive rate

  • True positive rate is computed from 100 official validation images per class for ImageNet, COCO, and Objects365 by counting correct “yes” responses.Source: §I.2 · p. 35

I.3. Qwen2-VL vs Llama 3.2-VL

  • Qwen2-VL and Llama 3.2-Vision disagree on some object-absent images, and each model hallucinates on examples where the other does not.The examples show that the models differ in their vulnerabilities despite being described as robust to hallucinations.Source: §I.3 · p. 35

J.1. Image Selection

  • DASH-B combines retrieval images from all three source models, filters for transfer to Qwen2-72B and Llama 3.2-VL-11B, and has two human labelers verify object absence for 70 selected objects.The image count is limited to between 3 and 50 per object; positive samples are also added and human-checked.Source: §J.1 · p. 35, J. DASH-B · p. 35

J.2. Metrics

This section specifies DASH-B evaluation metrics and the data construction for a small PaliGemma fine-tuning experiment.

J.2. Metrics

  • DASH-B reports accuracy, TNR, TPR, and their harmonic mean; accuracy alone gives a trivial always-yes or always-no model 50%.The authors report no significant effect from HM except for LLaVA-NeXT-Vicuna; source-model results are biased because those models created the benchmark.Source: §J.2 · p. 35, Table 6 · p. 37

K.1. Data

  • PaliGemma-3B is fine-tuned with LoRA on DASH retrieval results, using 200 negative and 400 positive samples per object.Positive samples are included to preserve object recognition; the experiment evaluates systematic hallucinations and other tasks.Source: K. Fine-tuning on DASH · p. 38, §K.1 · p. 38
  • Negative training images come from PaliGemma’s DASH-LLM and DASH-OPT results, with validation clusters held out and training images filtered by “no” responses from Llama 3.2-VL and Qwen2-VL.Source: §K.1 · p. 38
  • Positive images are retrieved from ReLAION using Llama 3.2 prompts, then filtered by the detector and Llama 3.2’s “Yes” response.Source: §K.1 · p. 38

K.2. Results

Fine-tuning PaliGemma on DASH images improves accuracy on unseen hallucination clusters, with smaller gains on related hallucination benchmarks and trade-offs on other tasks and positive recognition.

K.2. Results

  • +77.7% on unseen-cluster Validation accuracy is reported for fine-tuned PaliGemma-3B versus PaliGemma-3B.The validation metric is the ratio of correct “no” responses on the validation set.Source: §K.2 · p. 38
  • Amber score increases by +0.2, Amber Existence accuracy by +3%, and R-Bench accuracy by +1.1% after fine-tuning.Source: §K.2 · p. 38
  • TextVQA accuracy decreases by −1.4%, VQAv2 accuracy by −0.9%, COCO CiDER by −1.3, and Flickr30k CiDER by +0.1%.Source: §K.2 · p. 38
  • TPR-ICO decreases by −5.1%, including a −12.1% TPR drop on Objects365; the authors suggest a mismatch in object visibility between retrieved positives and Objects365 images.Replacing retrieved Objects365 positives with original Objects365 training images improves TPR on all three datasets but yields smaller hallucination-task gains.Source: §K.2 · p. 38

L. Reverse Task

The reverse task applies DASH-LLM to cases where an object is visible but the VLM answers “No,” while prompt-transfer results vary by model and prompt similarity. The accompanying fine-tuning results report improved robustness, with a slight cost on more general VQA and captioning tasks.

L. Reverse Task

  • The reverse task uses DASH-LLM to find cases where a VLM answers “No” despite the object being visible, adjusting the prompt and object-detector threshold.Figure 29 shows example clusters in which LLaVA-NeXT Vicuna answers “no” to “Can you see a OBJ in this image?” despite the visible object.Source: L. Reverse Task · p. 38, Figure 27 · p. 40, Figure 28 · p. 41
  • The reverse-task experiment is a proof of concept: the object detector performs worse in this direction and should be replaced for larger-scale experiments.The authors say DASH’s benefits are more pronounced for false-positive hallucinations because images containing an object are much less numerous than images without it.Source: L. Reverse Task · p. 38
  • Fine-tuning on DASH results improves robustness on unseen clusters and related hallucination benchmarks, while slightly worsening general VQA and captioning performance.Replacing retrieved Objects365 positives with original training images also improves average TPR-ICO.Source: Table 8 · p. 39

M. Transfer across prompts

  • Transfer rates are stable for LLaVA-NeXT Vicuna and Mistral, but PaliGemma transfers less well to similar prompts and robustness does not generalize to less similar prompts.The experiments evaluate 10 prompts against each source model’s DASH-LLM and DASH-OPT subsets; the default prompt is “Can a object be seen in the image?”.Source: M. Transfer across prompts · p. 43, Table 9 · p. 44

Related papers

Loading 2503.23573v1…