PaperlayerView exact revision on arXiv

Source-linked AI summary

Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning

Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, Lijuan Wang

arXiv:2306.14565v4cs.CVcs.AIcs.CEcs.CLcs.MM

TL;DR

Large multi-modal models hallucinate image-inconsistent content and may fail to follow human instructions, while existing instruction data and evaluation methods are limited. The paper introduces the 400k-example LRV-Instruction dataset with positive and multi-level negative instructions, plus GAVIE for flexible evaluation. Existing LMMs hallucinate especially on existent-object and knowledge manipulations, whereas finetuning MiniGPT4 and mPLUG-Owl on LRV-Instruction reduces hallucination and improves benchmark performance.

  • Problem

    Current LMMs can produce descriptions conflicting with images or instructions, while instruction-tuning data often lacks diversity and is dominated by positive or few-template examples.

  • Method

    The paper builds LRV-Instruction with 400k GPT4-generated instructions across 16 tasks and three negative semantic levels, and proposes GAVIE for groundtruth-free evaluation.

  • Results

    Existing LMMs seriously hallucinate on negative instructions, especially existent-object and knowledge manipulations; LRV-Instruction finetuning reduces hallucination and improves evaluation and public-benchmark performance.

  • Takeaways & Limitations

    Balanced positive and negative training data supports more robust visual instruction tuning across the paper’s evaluated models and benchmarks.

Abstract

from arXiv · show

Despite the promising progress in multi-modal tasks, current large multi-modal models (LMMs) are prone to hallucinating inconsistent descriptions with respect to the associated image and human instructions. This paper addresses this issue by introducing the first large and diverse visual instruction tuning dataset, named Large-scale Robust Visual (LRV)-Instruction. Our dataset comprises 400k visual instructions generated by GPT4, covering 16 vision-and-language tasks with open-ended instructions and answers. Unlike existing studies that primarily focus on positive instruction samples, we design LRV-Instruction to include both positive and negative instructions for more robust visual instruction tuning. Our negative instructions are designed at three semantic levels: (i) Nonexistent Object Manipulation, (ii) Existent Object Manipulation and (iii) Knowledge Manipulation. To efficiently measure the hallucination generated by LMMs, we propose GPT4-Assisted Visual Instruction Evaluation (GAVIE), a stable approach to evaluate visual instruction tuning like human experts. GAVIE does not require human-annotated groundtruth answers and can adapt to diverse instruction formats. We conduct comprehensive experiments to investigate the hallucination of LMMs. Our results demonstrate existing LMMs exhibit significant hallucinations when presented with our negative instructions, particularly Existent Object and Knowledge Manipulation instructions. Moreover, we successfully mitigate hallucination by finetuning MiniGPT4 and mPLUG-Owl on LRV-Instruction while improving performance on several public datasets compared to state-of-the-art methods. Additionally, we observed that a balanced ratio of positive and negative instances in the training data leads to a more robust model. Code and data are available at https://github.com/FuxiaoLiu/LRV-Instruction.

1 INTRODUCTION

Current large multi-modal models can hallucinate image-inconsistent content and fail to follow human instructions, partly reflecting limited and unbalanced instruction-tuning data. The paper introduces LRV-Instruction and GAVIE to study and mitigate these problems.

  • LMMs may describe nonexistent objects or activities and generate lengthy descriptions that do not follow the human instruction.
  • Existing instruction-tuning data often lacks diversity, with some models relying on only a few templates or positive instructions.
  • LRV-Instruction contains 400k GPT4-generated visual instructions spanning 16 vision-language tasks with open-ended instructions and answers.
  • Its negative instructions target nonexistent objects, existent-object manipulations, and knowledge manipulations alongside positive instructions.
  • GAVIE evaluates instruction-following and visual hallucination without human-annotated groundtruth answers and adapts to diverse instruction formats.
  • Five evaluated LMMs seriously hallucinate on negative instructions, especially existent-object and knowledge manipulations.
  • Finetuning MiniGPT4 and mPLUG-Owl on LRV-Instruction reduces hallucination and improves performance on the evaluation set and public benchmarks.

2 RELATED WORKS

Prior vision-language models use diverse architectures and instruction-tuning strategies, but hallucination evaluation remains difficult for open-ended outputs and often depends on human annotations.

  • Vision-language models increasingly use large language models as decoders, while instruction-tuned systems combine multiple vision-language datasets.
  • Existing image-captioning metrics such as CIDEr and SPICE do not appropriately penalize hallucination.
  • CHAIR requires complex human-crafted parsing rules, limiting its stability for hallucination evaluation.
  • Binary yes-or-no evaluation handles hallucination less effectively for open-ended outputs and depends on human-annotated groundtruth answers.

3 LRV-INSTRUCTION

LRV-Instruction is generated with GPT4 from detailed visual representations and combines diverse positive and negative instructions across 16 vision-language tasks. Quality control and dataset statistics characterize its coverage and composition.

  • 3 LRV-INSTRUCTION: GPT4 generates open-ended positive and negative visual instructions in different linguistic styles, reducing reliance on manually written annotations.
  • Positive Visual Instruction Generation: The positive-instruction pipeline selects 16 vision-language tasks and represents images using size, bounding boxes, and dense captions.
  • Positive Visual Instruction Generation: Chart and news images broaden visual diversity, with GPT4 generating question-answer pairs from captions describing chart patterns and named entities.
  • Negative Visual Instruction Generation: Negative instructions manipulate nonexistent objects, activities, attributes, and interactions, as well as existing objects with incorrect attributes and caption knowledge.
  • Negative Visual Instruction Generation: The generation prompt supplies bounding-box coordinates, object sizes, and dense captions so GPT4 can represent image content.
  • Quality Control: Quality control removes overly long answers, unnecessary content, inaccurate task labels, and facial-expression instructions, while manually checking 1,000 evaluation instances.
  • 3.1 DATA STATISTICS: Figure 4 categorizes dataset statistics using colors for existent-object, nonexistent-object, and knowledge manipulation.
  • 3.1 DATA STATISTICS: LRV-Instruction covers more vision-language tasks than existing datasets and includes positive and multi-level negative instructions with diverse, physically plausible examples.

4 VISUAL INSTRUCTION TUNING

The visual instruction-tuning experiments use MiniGPT4 and mPLUG-Owl as backbones, with parameter-efficient finetuning configurations for each model.

  • MiniGPT4 uses a Vision Transformer image encoder, Vicuna text decoder, and Q-Former to connect visual and textual representations.

5 GPT4-ASSISTED VISUAL INSTRUCTION EVALUATION

GAVIE evaluates multimodal instruction following and hallucination using GPT4, while LRV-Instruction supports broad robustness experiments across negative-instruction types and public benchmarks. Finetuning on LRV-Instruction reduces hallucination, improves evaluation performance, and benefits from balanced positive-negative training data.

  • 5 GPT4-ASSISTED VISUAL INSTRUCTION EVALUATION: GAVIE compares image content, human instructions, and model responses, scoring accuracy and relevancy without requiring human-annotated groundtruth answers.GPT4 receives dense captions with bounding boxes and scores responses on a 0–10 scale.
  • 5 GPT4-ASSISTED VISUAL INSTRUCTION EVALUATION: The experiments evaluate publicly available LMMs on MME, POPE, AMBER, and GQA, covering perception, cognition, hallucination, and open-ended visual question answering.The implementation compares zero-shot performance across recently released LMMs and public benchmarks.
  • 6.2 MAIN RESULTS: Existing LMMs perform poorly on open-ended negative instructions, whereas LRV-Instruction-finetuned models improve robustness and can match or surpass 13B counterparts.Both finetuned LMMs outperform their original versions in zero-shot evaluations.
  • 6.2 MAIN RESULTS: LRV-Instruction-finetuned mPLUG-Owl achieves on-par performance with InstructBLIP in generic visual question answering.This result is reported on the GQA benchmark.
  • 6.2 MAIN RESULTS: On the authors’ evaluation set, the proposed model outperforms existing LMM baselines by a large margin, which the authors attribute to dataset composition and prompt design.InstructBLIP performs better than other baselines, while LLaVA’s longer synthetic answers may include irrelevant information.
  • 6.3 DETAILED ANALYSIS: GAVIE rankings roughly align with human evaluation: experts rank the proposed model first, InstructBLIP second, and MMGPT worst.Three NLP experts rated model outputs using accuracy and relevancy criteria.
  • 6.3 DETAILED ANALYSIS: GAVIE’s standard deviation ranges from 0.65 to 2.46 across repeated evaluations, while scores remain within the same grade level.The authors characterize this as evidence of evaluation stability despite variation in individual scores.
  • 6.3 DETAILED ANALYSIS: Existent Object Manipulation and Knowledge Manipulation are harder for baselines than Nonexistent Object Manipulation, while the proposed model improves across all semantic levels.Existent-object cases require fine-grained attribute understanding, and knowledge manipulation challenges models lacking specific knowledge.

7 CONCLUSION

The paper constructs LRV-Instruction and proposes GAVIE to address hallucination in large multimodal models. Experiments validate robust visual instruction tuning, while future work targets stronger vision encoders and additional model biases.

  • 7 CONCLUSION: LRV-Instruction contains 400k visual instructions across 16 vision-language tasks, combining positive and negative instructions across semantic levels and styles.The dataset is designed to support robust visual instruction tuning.
  • 7 CONCLUSION: GAVIE evaluates visual instruction tuning without human-labeled groundtruth answers and adapts to different instruction formats.The paper presents it as a flexible evaluation approach for visual hallucination.
  • 7 CONCLUSION: Future directions include replacing current vision encoders with more powerful visual models and investigating other LMM biases.These directions aim to develop more robust multimodal models.

A.1 GAVIE EVALUATION

GAVIE uses visual representations and instruction-response pairs to have GPT4 score multimodal outputs for accuracy and relevancy. The evaluation is designed to reflect image content and instruction following.

  • A.1 GAVIE EVALUATION: GAVIE supplies GPT4 with bounding boxes and dense captions as visual input, alongside human instructions and model responses.GPT4 is prompted to act as a smart teacher and score answers from 0 to 10.
  • A.1 GAVIE EVALUATION: The evaluation uses accuracy and relevancy criteria to judge whether responses match the image and follow the instruction.Accuracy concerns image content, while relevancy concerns instruction adherence.

A.1.1 GPT4-Assisted Visual Instruction Evaluation (GAVIE) VS. HUMAN EVALUATION

The human evaluation asks three NLP experts to rate multimodal responses for accuracy and relevancy using four ordered quality levels. Their rankings broadly agree with GAVIE’s results.

  • A.1.1 GPT4-Assisted Visual Instruction Evaluation (GAVIE) VS. HUMAN EVALUATION: Three NLP experts independently assess 40 randomly selected image-instruction instances, each containing responses from four LMMs.The questionnaire is shuffled for each expert and takes about 20 minutes on average.
  • A.1.1 GPT4-Assisted Visual Instruction Evaluation (GAVIE) VS. HUMAN EVALUATION: Experts rate each answer for accuracy to image content and relevancy to the instruction.The rubric defines four levels: Very Poor, Poor, Good, and Excellent.
  • A.1.1 GPT4-Assisted Visual Instruction Evaluation (GAVIE) VS. HUMAN EVALUATION: GAVIE scores roughly align with expert ratings, with the proposed model ranked first, InstructBLIP second, and MMGPT last.The human ranking matches the ordering observed in GAVIE evaluation.

A.1.2 STABILITY OF GPT4-Assisted Visual Instruction Evaluation (GAVIE)

GAVIE shows stable evaluation across repeated runs: score variation remains limited, and repeated outputs stay within the same qualitative grade levels.

  • The Standard Deviation of GAVIE scores ranges from 0.65 to 2.46 across the evaluation instances.
  • Although ACCURACY and RELEVANCY scores can vary between runs, they remain within the same grade level.
  • RELEVANCY and ACCURACY each use four grade levels spanning scores from 0–2 through 9–10.RELEVANCY grades range from seldom relevant to completely relevant; ACCURACY grades range from mostly or completely wrong to completely accurate.
  • GAVIE is evaluated by running it five times on randomly selected evaluation instances, using Mean and Standard Deviation to assess stability.

A.2.1 Do LMMs perform better on Positive or Negative Instructions?

The evaluation compares LMM behavior on positive and negative instructions, across instruction formats and public hallucination benchmarks. Results show stronger performance on positive or short instructions for existing models, while LRV-Instruction improves robustness across negative cases and related benchmarks.

  • A.2.1 Do LMMs perform better on Positive or Negative Instructions?: Baseline LMMs perform better on positive than negative instructions, whereas the finetuned model performs best on both sets.MMGPT performs poorly on both sets, and LLaVA shows the most severe degradation on negative instructions.
  • A.2.1 Do LMMs perform better on Positive or Negative Instructions?: LMMs score slightly higher on interrogative than declarative instructions, while longer instructions are more difficult than shorter ones.The difference between interrogative and declarative formats is relatively small; longer instructions contain more information.
  • A.3.1 POSITIVE INSTANCE GENERATION BASED ON VISUAL GENOME DATASET: Positive instances are generated from Visual Genome image sizes, bounding boxes, and dense captions, with GPT4 producing examples in declarative and interrogative formats.Prompts also address highly overlapping bounding boxes and request short answers.
  • A.3.3 NEGATIVE INSTANCE GENERATION - NONEXISTENT/EXISTENT OBJECT MANIPULATION: Negative instances manipulate nonexistent or existent objects and knowledge in captions, including nonexistent attributes, relationships, named entities, and events.Knowledge manipulation includes named entities and events, while GPT4 converts manipulated outputs into interrogative questions.
  • A.3.5 PROMPT DESIGN FOR EVALUATING KNOWLEDGE HALLUCINATION: GAVIE evaluates instruction-following with Relevancy and visual hallucination with Accuracy, without requiring human-annotated groundtruth answers.Its evaluation prompt compares reference answers with model predictions and treats contradictions as incorrect.
  • A.4 MORE DATASET STATISTIC: Knowledge manipulations mainly cover events, numbers, dates, persons, places, and other categories, while examples expose errors in color recognition and cloud detection.The appendix also reports evaluation figures for positive and negative instructions, instruction lengths, and MME perception and cognition.
Loading 2306.14565v4…