Source-linked AI summary
Woodpecker: Hallucination Correction for Multimodal Large Language Models
Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, Enhong Chen
TL;DR
MLLMs often produce text inconsistent with images, including nonexistent objects and incorrect attributes. Woodpecker is a training-free, post-correction framework that diagnoses responses, constructs visual claims, and revises hallucinations with evidence. On POPE, it raises MiniGPT-4/mPLUG-Owl accuracy from 54.67%/62% to 85.33%/86.33%.
Problem
MLLMs generate descriptions inconsistent with images, including nonexistent objects and inaccurate attributes, which obstructs practical application.
Method
Woodpecker uses five training-free stages to diagnose MLLM responses, build image-specific visual knowledge, and correct hallucinations with bounding-box evidence.
Results
MiniGPT-4/mPLUG-Owl accuracy rises from 54.67%/62% to 85.33%/86.33% on POPE.
Takeaways & Limitations
Woodpecker provides a correction-based paradigm that can integrate with different MLLMs and expose intermediate outputs for interpretability.
Takeaways & Limitations
Automatic evaluation of open answers may be inaccurate or biased because translating image content into captions and bounding boxes loses information.
Abstract
from arXiv · showhide
Hallucination is a big shadow hanging over the rapidly evolving Multimodal Large Language Models (MLLMs), referring to the phenomenon that the generated text is inconsistent with the image content. In order to mitigate hallucinations, existing studies mainly resort to an instruction-tuning manner that requires retraining the models with specific data. In this paper, we pave a different way, introducing a training-free method named Woodpecker. Like a woodpecker heals trees, it picks out and corrects hallucinations from the generated text. Concretely, Woodpecker consists of five stages: key concept extraction, question formulation, visual knowledge validation, visual claim generation, and hallucination correction. Implemented in a post-remedy manner, Woodpecker can easily serve different MLLMs, while being interpretable by accessing intermediate outputs of the five stages. We evaluate Woodpecker both quantitatively and qualitatively and show the huge potential of this new paradigm. On the POPE benchmark, our method obtains a 30.66%/24.33% improvement in accuracy over the baseline MiniGPT-4/mPLUG-Owl. The source code is released at https://github.com/BradyFU/Woodpecker.
1. Introduction
MLLMs can generate image-inconsistent descriptions, creating object- and attribute-level hallucinations that hinder practical reliability. Woodpecker addresses this with a training-free, interpretable post-correction pipeline and reports substantial benchmark gains.
- MLLM hallucinations include nonexistent objects and inaccurate object attributes, obstructing practical application.
- Existing mitigation methods commonly use instruction tuning, which can reduce detail or require iterative generation and substantial data and computation.
- Woodpecker directly corrects generated text without retraining and adds bounding boxes as visual evidence for fact-checking.
- Woodpecker diagnoses responses through key concept extraction, question formulation, visual knowledge validation, visual claim generation, and hallucination correction.
- MiniGPT-4/mPLUG-Owl accuracy rises from 54.67%/62% to 85.33%/86.33% on POPE.
- The framework is designed so each step is clear and transparent, providing interpretability.
2. Related Work
Prior MLLM hallucination research emphasizes evaluation or mitigation, with mitigation methods primarily changing data collection or training procedures. Related work also uses external knowledge and LLM reasoning to improve factuality or support multimodal tasks.
- Prior hallucination research mainly addresses evaluation or mitigation, using classifiers or answer comparisons to detect hallucinations.
- Mitigation methods such as LRV-Instruction and VIGC modify answer length or generate iteratively to reduce hallucinations while managing detail.
- Knowledge augmentation studies use predefined or internet-sourced knowledge as evidence to refine possibly false claims and improve factuality.
- LLM-aided visual reasoning assigns language models roles including task dispatcher, reasoner, or language refiner for vision and multimodal tasks.
3. Method
Woodpecker decomposes hallucination correction into five subtasks that build image-specific visual knowledge from an MLLM response, then use it to revise the response with evidence.
- The method separates diagnosis and correction into key concept extraction, question formulation, visual knowledge validation, visual claim generation, and hallucination correction.
- 3.1. Key Concept Extraction: Key concept extraction identifies main objects in generated sentences as centers for subsequent hallucination diagnosis.
- 3.2. Question Formulation: Question formulation asks object-level existence and count questions plus attribute-level questions about actions, positions, colors, and relationships.
- 3.3. Visual Knowledge Validation: An open-set object detector validates object existence and counts, while a pretrained VQA model answers image-conditioned attribute questions.
- 3.4. Visual Claim Generation: Question-answer pairs are organized into object-level and attribute-level visual claims, including counts, attributes, and interactions.
- 3.5. Hallucination Correction: An LLM combines the visual knowledge base with the original response, corrects hallucinations, and attaches bounding boxes after object references.
4. Experiment
Woodpecker is evaluated across POPE, MME, and LLaVA-QA90 using object- and attribute-level hallucination tests, open-ended response evaluation, and module analyses. Across these settings, the framework generally improves hallucination correction and response quality, while detector and VQA modules contribute complementary gains.
- Experimental Settings: POPE evaluates object existence with balanced Yes-or-No questions under random, popular, and adversarial negative-sampling settings.Accuracy, precision, recall, and f1-score are reported; the baselines include mPLUG-Owl, LLaVA, MiniGPT-4, and Otter.
- Experimental Settings: MME measures object-level existence and counting plus attribute-level position and color hallucinations using Yes-or-No subsets and partition scores.The reported score is the sum of accuracy and accuracy+ following the official implementation.
- Experimental Settings: LLaVA-QA90 evaluates open-ended image descriptions on accuracy and detailedness, with GPT-4V directly rating responses against the input image.This setup avoids relying on text-only evaluation from captions and bounding boxes, which can lose image information.
- Experimental Results: 30.66% and 24.33% relative accuracy gains are obtained for MiniGPT-4 and mPLUG-Owl on POPE, respectively.Across random, popular, and adversarial settings, Woodpecker improves most metrics; in the adversarial setting, mPLUG-Owl rises from 56.33% to 81%.
- Experimental Results: +65 to +101.66 score gains are achieved on MME object-level evaluation, while attribute-level color performance also improves substantially.Position gains are smaller, possibly because of weaker BLIP-2 position reasoning and limited LLM comprehension of bounding-box relations.
- Experimental Results: GPT-4V evaluation finds consistent gains in both response accuracy and detailedness after correction, with bounding boxes contributing additional response detail.The framework analysis further reports 79.2% accuracy for the default model while omission and mis-correction rates remain relatively low.
5. Conclusion
The paper proposes a correction-based, training-free framework for mitigating MLLM hallucinations and evaluates it across three benchmarks under varied settings. The authors position the approach as broadly integrable and potentially useful for future hallucination research.
- Woodpecker is presented as the first correction-based framework for mitigating hallucinations in MLLMs.
- The training-free approach incorporates multiple off-the-shelf models and can be integrated into different MLLMs.
- Experiments cover three benchmarks under different settings, including direct and automatic assessment using GPT-4V.
- The authors hope the work will stimulate new approaches to addressing hallucinations in MLLMs.
A. Prompt Templates
The appendix provides prompt templates for the LLM-driven stages of Woodpecker and for GPT-4V-aided evaluation. Each template specifies the inputs used to guide its corresponding task.
- A. Prompt Templates: The prompt-template appendix covers key concept extraction, question formulation, hallucination correction, and GPT-4V-aided evaluation.
- A. Prompt Templates: The templates use in-context examples to instruct the LLM on task requirements.
- A.2. Question Formulation: The question formulation template takes an input sentence and entities produced by key concept extraction.
- A.3. Hallucination Correction: The hallucination correction template uses formatted input information and the original response as its inputs.
- A.4. GPT-4V-aided Evaluation: The GPT-4V-aided evaluation template compares an original response with its corrected response.
B. GPT-4V-aided Evaluation Case
The evaluation case illustrates how GPT-4V assesses original and Woodpecker-corrected MLLM responses. GPT-4V provides both scores and reasons for its judgments.
- Figure 7 presents an example of GPT-4V-aided evaluation comparing an MLLM response with a Woodpecker-corrected response.
- “Assistant 1” and “Assistant 2” correspond to the MLLM and MLLM with Woodpecker, respectively.
- GPT-4V supplies respective scores and reasons for judging the two responses.