PaperlayerView exact revision on arXiv

Source-linked AI summary

Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision

Seongyun Lee, Sue Hyun Park, Yongrae Jo, Minjoon Seo

arXiv:2311.07362v4cs.CLcs.CV

TL;DR

Large multimodal models can answer in ways that do not align with visual input, including when image grounding is poor. VOLCANO generates visual natural-language feedback to revise its responses. The authors report improvements across hallucination and multimodal-understanding benchmarks, while noting a substantial execution-time cost.

  • Problem

    Multimodal models can produce responses misaligned with visual information, and inaccurate vision-encoder grounding may leave them relying on parametric knowledge.

    Source: §1 · pp. 1–2

  • Method

    VOLCANO generates natural-language feedback grounded in visual information and uses it to iteratively revise an initial response.

    Source: §1 · p. 2

  • Results

    VOLCANO improves across multimodal hallucination and understanding benchmarks, with a reported 24.9% enhancement over prior methods specialized for hallucination mitigation.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Qualitative analysis indicates that image-grounded feedback can provide richer visual information and help reduce multimodal hallucination.

    Source: §6 · p. 8

    VOLCANO takes around 2–3 times longer than the base model on average, requiring 5.8 seconds per response versus 2.7 seconds for LLaVA-1.5.

    Source: Limitations · p. 8

Abstract

from arXiv · show

Large multimodal models suffer from multimodal hallucination, where they provide incorrect responses misaligned with the given visual information. Recent works have conjectured that one of the reasons behind multimodal hallucination is due to the vision encoder failing to ground on the image properly. To mitigate this issue, we propose a novel approach that leverages self-feedback as visual cues. Building on this approach, we introduce Volcano, a multimodal self-feedback guided revision model. Volcano generates natural language feedback to its initial response based on the provided visual information and utilizes this feedback to self-revise its initial response. Volcano effectively reduces multimodal hallucination and achieves state-of-the-art on MMHal-Bench, POPE, and GAVIE. It also improves on general multimodal abilities and outperforms previous models on MM-Vet and MMBench. Through qualitative analysis, we show that Volcano's feedback is properly grounded on the image than the initial response. This indicates that Volcano can provide itself with richer visual information through feedback generation, leading to self-correct hallucinations. We publicly release our model, data, and code at https://github.com/kaistAI/Volcano}{github.com/kaistAI/Volcano

1 Introduction

Large multimodal models can produce answers misaligned with images, including when visual features are poorly grounded. VOLCANO uses natural-language self-feedback to revise answers and reports improvements on hallucination and multimodal-understanding benchmarks.

1 Introduction

  • Multimodal hallucinations are responses misaligned with visual information; inaccurate encoder grounding can leave models relying on parametric knowledge and generating guesses.Prior work also reports that models attend more to preceding tokens than image features when generating misaligned responses.Source: §1 · pp. 1–2
  • VOLCANO generates natural-language feedback from visual information and iteratively revises its initial response until it determines that no further improvement is needed.Training data for feedback and revision is collected using proprietary LLMs.Source: §1 · p. 2
  • Qualitative analysis finds that VOLCANO’s feedback is image-grounded and carries fine-grained visual information that can guide improvement even when the encoder initially fails to ground properly.The feedback attends to image features with higher intensity and coverage than the initial response.Source: §1 · p. 2
  • VOLCANO achieves state-of-the-art results on multimodal hallucination and understanding benchmarks, with consistent hallucination-benchmark improvements and a reported 24.9% enhancement over specialized prior methods.The reported enhancement is stated for comparison with previous methods specialized in mitigating multimodal hallucination.Source: §1 · p. 2
  • The authors release VOLCANO in 7B and 13B sizes, together with training and inference data and code.Source: §1 · p. 2

2 Related work

Related work characterizes multimodal hallucination, reviews language-model self-correction, and contrasts prior mitigation strategies with VOLCANO’s feedback-guided revision approach.

2.1 Multimodal hallucination

  • Multimodal hallucination research has mainly examined object hallucinations, while holistic scene or environment misrepresentations have only recently begun to be documented.Proposed sources of grounding failure include imprecise visual features, language-pattern preferences, and autoregressive generation.Source: §2.1 · p. 2

2.2 Self-correcting from feedback

  • Natural-language feedback supplies more information than scalar preference or fine-grained feedback, and prior work has used it for language-model self-correction.The authors claim to be the first to improve multimodal models through self-feedback-guided refinement.Source: §2.2 · pp. 2–3

2.3 Mitigating multimodal hallucination

  • Prior mitigation includes data changes, reward-model training, and post-hoc revision; VOLCANO instead trains one model to self-revise using feedback that revisits visual features and pinpoints hallucinated elements.The approach does not require reward-model training or extra revision modules.Source: §2.3 · p. 3

3 VOLCANO

VOLCANO uses one multimodal model for sequential response generation, feedback, revision, and selection. Its training data pairs initial responses with feedback and revisions produced using text descriptions of image content.

3 VOLCANO

  • VOLCANO is a single LMM that generates initial responses, feedback, revisions, and acceptance decisions in a sequential critique–revision–decide procedure.The overall process takes an image and question and generates an improved response based on self-feedback.Source: §3 · p. 3, Figure 2 · p. 4

3.1 Iterative self-revision

  • The iterative loop retains a revision only when the model judges it better than the current best response, then repeats up to a preset maximum or stops early.The model randomizes the order of the two responses during comparison to avoid position effects.Source: §3.1 · p. 3

3.2 Data collection

  • Training data starts with open-source LMM responses; a proprietary LLM generates feedback and revisions using text-formatted image details, captions, the question, initial response, and gold answer.Prompts direct the proprietary LLM to focus on image details when generating feedback, since the gold answer is unavailable at inference.Source: §3.2 · pp. 3–4

3.3 Implementation details

  • The data uses the first turn of LLaVA-SFT-127k, while VOLCANO is fine-tuned with llava-1.5-mix665k and uses LLaVA-1.5 7B or 13B backbones.The proprietary feedback-generation model is gpt-3.5-turbo.Source: §3.3 · p. 4

4 Experiments

VOLCANO is evaluated on hallucination and general multimodal-understanding benchmarks, where it reports strong performance against existing models. Ablations examine the roles of feedback, revision, decision-making, and iteration count.

4.1 Benchmarks

  • The evaluation uses POPE, GAVIE, and MMHal-Bench for hallucination mitigation, and MM-Vet and MMBench for general multimodal performance.POPE and GAVIE assess object-level hallucinations; MM-Vet covers 16 tasks and 218 instances, while MMBench has 4,377 questions.Source: §4.1 · pp. 4–5

4.2 Baselines

  • Baselines include general-purpose LMMs and hallucination correctors LURE and Woodpecker, which use separate models or components for correction.Woodpecker uses GPT-3.5-turbo as corrector, Grounding DINO as object detector, and BLIP-2-FlanT5-XXL for VQA.Source: §4.2 · p. 5

4.3 Main results

  • VOLCANO consistently outperforms the base model, LLaVA-1.5, and other existing LMMs on multimodal hallucination benchmarks, including proprietary-LLM and conventional-metric evaluations.GAVIE results also indicate improved instruction following; Table 1 reports MMHal-Bench, POPE, and GAVIE results.Source: §4.3 · p. 5, Table 1 · p. 5
  • VOLCANO outperforms revision methods LURE and Woodpecker, and comparisons suggest natural-language feedback is more effective than scalar feedback for reducing hallucination.The authors argue that conveying visual features directly to the corrector is critical; Table 2 compares recent hallucination-reduction methods.Source: §4.3 · pp. 5–6, Table 2 · p. 5
  • VOLCANO achieves superior performance to existing LMMs on MM-Vet and MMBench, with its 13B model scoring about twice as high as LLaVA-1.5 13B on the math score.Table 3 reports results on multimodal-understanding benchmarks.Source: §4.3 · p. 6, Table 3 · p. 6

4.4 Ablation studies

  • Using only the initial prediction still beats the LLaVA-1.5 7B base, while omitting the decision stage reduces performance relative to making a decision after revision.The authors attribute a role to the decision stage in preventing unnecessary revisions; Table 4 defines the ablation variants.Source: §4.4 · p. 6, Table 4 · p. 6
  • Increasing the maximum number of revision iterations decreases hallucination rate but increases inference time.Table 5 reports the iteration ablation.Source: §4.4 · p. 6, Table 5 · p. 6

5 Qualitative analysis

The qualitative analysis finds that feedback attends more strongly to image features than initial responses and covers salient visual regions more fully. A pot-color example illustrates how feedback can correct an initial error by attending to image regions relevant to the color terms.

5.1 Amount of visual information

  • Manual inspection found that initial responses often identify objects but misread attributes or relationships, whereas feedback tends to describe image contents more comprehensively.The analysis examines VOLCANO 7B on MMHal-Bench instances where the revised answer is selected as final.Source: §5.1 · p. 7, §5 · p. 7
  • Feedback attends more strongly to image features than initial responses, with increased concentration across wider image areas even though feedback also takes the initial response as input.Attention is averaged across instances where VOLCANO revision enhances the initial response; Figure 5 compares initial-response and feedback attention.Source: §5.1 · p. 7, Figure 5 · p. 7

5.2 Coverage of visual information

  • For the pot-color example, feedback corrects “red” to “silver” while attending across the pot and berries; token heatmaps focus on the image regions corresponding to “silver” and “red.”Figure 4 visualizes attention over image features for all output tokens or selected tokens.Source: Figure 4 · p. 7, §5.2 · p. 8
  • The analysis suggests that feedback's increased coverage of salient features provides fine-grained visual information that can guide revision even when the vision encoder fails to ground properly.The authors connect this capability to reducing multimodal hallucination.Source: §5.2 · p. 8

6 Conclusion

Volcano uses its own feedback as a visual signal to refine image-misaligned responses. The paper reports state-of-the-art results on a multimodal hallucination benchmark and improved multimodal-understanding performance over baseline models.

6 Conclusion

  • Volcano reports state-of-the-art results on a multimodal hallucination benchmark and improved performance over baseline models on multimodal-understanding benchmarks.The conclusion also reports that its feedback is well grounded on the image.Source: §6 · p. 8
  • Volcano uses feedback as a visual signal to direct refinement of responses that do not accurately reflect the image.Source: §6 · p. 8
  • Qualitative analysis finds that Volcano's image-grounded feedback provides rich visual information that helps reduce multimodal hallucination.Source: §6 · p. 8

Limitations

Volcano's self-feedback revision takes multiple model calls and is less time-efficient than directly generating a response. The authors limit revision to three iterations to reduce execution time.

Limitations

  • Volcano takes around 2 to 3 times longer than the base model to generate a response.The reported averages are 5.8 seconds for Volcano and 2.7 seconds for LLaVA-1.5.Source: Limitations · p. 8
  • Multiple model calls make Volcano less time-efficient than directly generating a response.Source: Limitations · p. 8
  • The authors limit revision to 3 iterations as a strategy to reduce overall execution time.Source: Limitations · p. 8

A Full results on benchmarks

The appendix presents detailed results on benchmarks designed to assess large multimodal models from multiple perspectives, across varied subtasks and question types.

A Full results on benchmarks

A.1 Multimodal hallucination benchmarks

The multimodal hallucination benchmarks categorize questions by different types: MMHal-Bench has eight categories, while POPE has three.

A.1 Multimodal hallucination benchmarks

  • MMHal-Bench questions are categorized as Attribute, Adversarial, Comparison, Counting, Relation, Environment, Holistic, or Other.Source: §A.1 · p. 11
  • POPE evaluates random, popular, and adversarial question types.Source: §A.1 · p. 11
  • MMHal-Bench has eight question types, whereas POPE evaluates three.Source: §A.1 · p. 11

A.2 Multimodal understanding benchmarks

The appendix describes the multimodal benchmarks, inference prompts, feedback-generation prompt design, and computation used for Volcano.

A.2 Multimodal understanding benchmarks

  • MM-Vet evaluates six LMM capabilities, while MMBench evaluation follows prior work on its L-2 tasks: CP, FP-S, FP-C, AR, RR, and LR.The six MM-Vet capabilities are Recognition, OCR, Knowledge, Language generation, Spatial awareness, and Math.Source: §A.2 · p. 11

B.1 Prompts for inference at each stage

B.2 Prompt for generating multimodal feedback

  • Feedback prompts give image-blind LLMs object information and a gold caption, omit object detection when objects are unavailable, and accept synonyms or paraphrases as correct.The prompt design avoids propagating detector errors and avoids labeling sufficiently aligned alternative expressions as hallucinations.Source: §B.2 · p. 11

C Computation

  • Training used 8 GPUs for 15 hours for VOLCANO 7B and took 30 hours for VOLCANO 13B; full evaluation takes about 2–3 times longer than response-only baselines.The stated hardware was an NVIDIA A100-SXM4-80GB GPU and an AMD EPYC 7513 32-Core Processor.Source: C Computation · p. 11

D Hyperparameters

The appendix reports training hyperparameters and decoding settings, alongside captions identifying prompt figures and benchmark-results tables.

D Hyperparameters

  • Training used batch size 128, learning rate 2e-5, one epoch, maximum length 2048, no weight decay, and cosine scheduling with a 0.03 warmup ratio.Gradient checkpointing and DeepSpeed zero stage 3 were also used.Source: D Hyperparameters · p. 11
  • Self-revision was limited to three iterations, and response generation used greedy decoding following LLaVA-1.5.Source: D Hyperparameters · p. 11
  • Tables 6–9 are identified as results on MMHal-Bench, Pope, MM-Vet, and MMBench, respectively.Source: Table 6 · p. 12, Table 7 · p. 12, Table 8 · p. 12
  • Figure 6 is labeled as the multimodal-feedback generation prompt, while Figure 7 presents prompts for inference at each stage.Source: Figure 6 · p. 13, Figure 7 · p. 14

Related papers

Loading 2311.07362v4…