PaperlayerView exact revision on arXiv

Source-linked AI summary

EFUF: Efficient Fine-grained Unlearning Framework for Mitigating Hallucinations in Multimodal Large Language Models

Shangyu Xing, Fei Zhao, Zhen Wu, Tuo An, Weihao Chen, Chunhui Li, Jianbing Zhang, Xinyu Dai

arXiv:2402.09801v3cs.CLcs.CV

TL;DR

MLLMs can describe objects absent from their images, while existing alignment methods require costly paired annotations and substantial computation. EFUF uses unlearning without paired data; experiments report reduced hallucinations while retaining general model performance.

  • Problem

    MLLM object hallucinations produce descriptions of objects absent from images, while alignment methods may require costly paired annotations and substantial finetuning computation.

    Source: §1 · pp. 1–2

  • Method

    EFUF uses unlearning, including gradient ascent on negative samples, to mitigate hallucinations without paired data.

    Source: §1 · p. 2

  • Results

    EFUF effectively reduces multimodal hallucinations while retaining the model's general performance across experiments on different baselines.

    Source: §7 · p. 9

  • Takeaways & Limitations

    CLIP-derived relevance scores can support setting thresholds to separate positive and negative samples for unlearning.

    Source: §3.2 · p. 4

    The investigation primarily addresses object hallucinations, not other varieties such as incorrect object attributes or positions.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Multimodal large language models (MLLMs) have attracted increasing attention in the past few years, but they may still generate descriptions that include objects not present in the corresponding images, a phenomenon known as object hallucination. To eliminate hallucinations, existing methods manually annotate paired responses with and without hallucinations, and then employ various alignment algorithms to improve the alignment capability between images and text. However, they not only demand considerable computation resources during the finetuning stage but also require expensive human annotation to construct paired data needed by the alignment algorithms. To address these issues, we borrow the idea of unlearning and propose an efficient fine-grained unlearning framework (EFUF), which can eliminate hallucinations without the need for paired data. Extensive experiments show that our method consistently reduces hallucinations while preserving the generation quality with modest computational overhead. Our code and datasets will be publicly available.

1 Introduction

The introduction frames object hallucination as a reliability problem and identifies data and computation costs in alignment-based remedies. EFUF applies unlearning without paired data, using CLIP relevance scores to curate samples.

1 Introduction

  • MLLM object hallucinations include descriptions of objects absent from images, which can spread misinformation and undermine user trust.The example given is a caption mentioning “landing gear” that does not appear in the image.Source: §1 · p. 1
  • Existing finetuning-based alignment methods require paired hallucinated and non-hallucinated responses, often annotated by humans, and substantial computational resources.Preference alignment may require multiple models to operate simultaneously.Source: §1 · p. 2
  • EFUF uses unlearning to mitigate hallucinations without paired data and with lower finetuning resource demands than traditional alignment methods.The framework applies gradient ascent to negative samples and avoids simultaneous operation of multiple models for preference alignment.Source: §1 · p. 2
  • CLIP text-image relevance scores help distinguish real from hallucinated objects and curate samples for EFUF without manual annotation.The authors first assess CLIP's capability, then use its scores to curate the dataset.Source: §1 · p. 2
  • The authors present EFUF as compatible with existing MLLMs and report experiments across multiple models validating its effectiveness.They describe obtaining positive and negative examples separately in a cost-effective and reliable manner.Source: §1 · p. 2

2 Related Work

Related work covers inference-time and finetuning approaches to hallucination mitigation, alongside prior unlearning methods. EFUF applies unlearning selectively to hallucinated objects in MLLMs.

2.1 Hallucination Mitigation for MLLM

2.2 Unlearning

  • Prior LLM unlearning uses gradient ascent to induce forgetting, with methods also seeking to limit damage to general performance.Yao et al. (2023) use random mismatch and KL-divergence restrictions for positive samples.Source: §2.2 · p. 3
  • EFUF extends unlearning to MLLMs by targeting hallucinated objects rather than entire responses, aiming to correct inaccuracies without diminishing other capabilities.The authors describe this as the first attempt to adopt unlearning to MLLMs.Source: §2.2 · p. 3

3 Preliminary Experiment

The preliminary experiment tests whether CLIP text-image relevance distinguishes hallucinated from non-hallucinated objects. The reported score patterns support using relevance thresholds to separate samples for unlearning.

3.1 Hallucinated v.s. Non-Hallucinated

  • The experiment tests whether text-image congruence indicates hallucination, using CLIP similarities for objects in 200 manually annotated MiniGPT and LLaVA captions.Objects are labeled hallucinated or non-hallucinated according to whether they appear in the image.Source: §3.1 · p. 3, §3 · p. 3
  • For object-level relevance scores, images are divided into patches and sliding windows identify the best matching region for each object.The resulting scores are grouped into hallucinated and non-hallucinated sets for comparison.Source: §3.1 · p. 4

3.2 Results and Analysis

  • A two-sample t-test rejects the null hypothesis that hallucinated and non-hallucinated objects have identical mean relevance scores.The paper presents the statistics and significance test in Table 1.Source: §3.2 · p. 4, Table 1 · p. 4
  • Above 32, only 0.6% and 1.6% of objects were hallucinated; below 23, only 2.3% and 1.7% were not, for MiniGPT and LLaVA respectively.The Figure 2 density plots compare image-relevance scores for hallucinated and non-hallucinated objects from the two models.Source: §3.2 · p. 4, Figure 2 · p. 4

4 Multimodal Hallucination Mitigation

EFUF builds samples from CLIP-based image relevance scores and applies unlearning to mitigate hallucinations. Its fine-grained dataset and three-loss training strategy target incorrect objects while retaining correct-object and full-sentence capabilities.

4.1 Overview

  • EFUF has two stages: CLIP-based sample segregation constructs the dataset, then unlearning is applied to the model.The overview describes dataset construction followed by the unlearning process.Source: §4.1 · pp. 4–5

4.2 Dataset Formation

  • The dataset combines positive and negative object subsentences with complete sentences selected using the mean image-relevance score of their referenced objects.Positive and negative samples are filtered using predetermined image-relevance thresholds; for an object subsentence, text following that subsentence is truncated.Source: §4.2 · pp. 5–6

4.3 Unlearning for MLLM

  • EFUF uses gradient ascent on hallucinated-object subsentences, a positive loss on non-hallucinated subsentences, and a sentence loss to retain full-sentence comprehension.The method introduces three losses tailored to the constructed datasets and combines them using weights.Source: §4.3 · p. 6
  • Training concurrently samples from the three datasets, computes their losses individually, and aggregates them into a final loss.The paper states this training strategy mitigates hallucinations while preserving proficiency in processing extensive sentences.Source: §4.3 · p. 6

5 Experiments

Experiments use MSCOCO images and assess both hallucination and generation quality. EFUF is integrated into four pretrained and instruction-finetuned MLLMs for evaluation.

5.1 Experimental Settings

  • MSCOCO images are used for the experiments, with annotations reserved exclusively for evaluation.The assessment covers hallucination using CHAIR, MHumanEval, and POPE, and generation quality using BLEU, GPT-4 judgments of informativeness, and GPT-2 perplexity for fluency.Source: §5.1 · pp. 6–7

5.2 Baselines

  • EFUF is integrated into MiniGPT4, mPLUG-owl, LLaVA, and ShareGPT4V, which serve as the evaluated base models.The paper describes these models as pretrained on multimodal datasets and subsequently finetuned on high-quality instructions.Source: §5.2 · p. 7

6 Results and Analysis

EFUF reduces hallucination rates while improving generation quality, and its ablations indicate that fine-grained unlearning and sentence loss both matter. Comparisons report competitive hallucination mitigation and VQA/reasoning performance, with lower training costs than other finetuning-based methods.

6.1 Main Results

  • Across four MLLMs, EFUF reduces hallucination rates by approximately 15% and 5% on ChairS and ChairI, 18% and 8% on HumanS and HumanI, and 1% on POPE.Source: §6.1 · p. 7
  • Across four models, EFUF improves BLEU-1 by 4%, BLEU-2 by 3%, BLEU-4 by 2%, informativeness by 1%, and fluency by 1%.The authors attribute accurate object generation to unlearning and enhanced fluency to sentence loss.Source: §6.1 · p. 7

6.2 Ablation Study

  • Fine-grained unlearning and complete EFUF substantially outperform vanilla unlearning and sentence-loss-only configurations, whose hallucination-rate and BLEU gains are described as trivial.The comparison includes vanilla unlearning, fine-grained unlearning, sentence-loss-only, and EFUF.Source: §6.2 · pp. 7–8, Table 3 · p. 8
  • Fine-grained unlearning alone slightly lowers hallucination rates but sacrifices informativeness and fluency; sentence loss helps preserve readable, comprehensive generation in EFUF.The fine-grained-only responses show increased perplexity and fragmented, incoherent sentences, while EFUF and sentence-loss-only do not exhibit these flaws.Source: §6.2 · p. 8

6.3 Comparison with Other Methods

  • On LLaVA, EFUF has comparable or superior hallucination-rate and generation-quality performance to competing methods, with minimal data-construction and training-resource costs.Its generation-quality improvements are reported as on par with RLHF-based methods.Source: §6.3 · pp. 8–9, Table 4 · p. 8
  • Across MME, GQA, ScienceQA, and QBench, EFUF shows modest performance fluctuation relative to other mitigation strategies and does not negatively affect VQA or reasoning capabilities.Source: Table 5 · p. 8, §6.3 · p. 9

6.4 Training Cost

  • MiniGPT4 EFUF training takes 3 A100 GPU hours, compared with 20 for RLHF-based finetuning, 8–16 for DPO, and around 10 for contrastive learning.The authors also report lower dataset-construction costs because EFUF avoids costly human annotations.Source: §6.4 · p. 9, Figure 4 · p. 9

6.5 Additional Analyses

  • Supplementary analyses examine EFUF alongside existing mitigation strategies, vary hyperparameter weights, and quantitatively compare generated text across methods.Source: §6.5 · p. 9

7 Conclusion

The paper uses text-image similarity to identify multimodal hallucinations and proposes an unlearning framework for mitigating them. Experiments report reduced hallucinations while retaining general model performance.

  • Text-image similarity helps identify multimodal hallucinations, and EFUF uses CLIP-derived image-relevance scores to curate samples before applying three losses for unlearning.Source: §7 · p. 9

Limitations

The study primarily evaluates object hallucinations defined by whether depicted objects appear in images, and identifies alternative similarity measures and other hallucination types as future work. Implementation uses fixed training settings, while MSCOCO images and prompts support annotation-free dataset construction.

Limitations

  • The authors identify alternative ways to assess text-image relevance beyond CLIP as an avenue for further dataset-construction research.Source: Limitations · p. 9
  • The study primarily addresses object hallucinations—whether an object is present in its corresponding image—leaving attributes and object positioning for future work.Source: Limitations · p. 10

A.1 Implementation Details

  • Training tunes only each model’s multimodal mapping layers for one epoch, using AdamW and an NVIDIA A100 GPU.The reported settings include unlearning-loss weight 0.3, sentence-loss weight 0.2, learning rate 1e-5, and weight decay 0.05.Source: §A.1 · p. 12

A.2 Dataset

  • For dataset construction, the approach uses MSCOCO images and associated prompts without requiring annotations, and selects 3,200 non-training images for validation and testing.Source: §A.2 · p. 12

A.3.1 Metrics on Hallucination Rate

The evaluation measures hallucination rates with CHAIR, human review, and POPE, and assesses generation quality through informativeness, similarity to human captions, and fluency.

A.3.1 Metrics on Hallucination Rate

  • CHAIR measures the ratio of nonexistent objects mentioned to all objects mentioned, with CHAIRS operating at sentence level and CHAIRI at instance level.A hallucinated response contains at least one hallucinated object.Source: §A.3.1 · p. 12
  • MHumanEval uses expert review of 100 randomly selected responses to assess hallucinations at both object and response levels.It addresses CHAIR’s limitation to predefined object categories.Source: §A.3.1 · p. 12
  • POPE probes whether models recognize image objects, including randomly introduced nonexistent objects, and reports F1 scores.An automated segmentation tool delineates objects before the model is queried.Source: §A.3.1 · p. 12

A.3.2 Metrics on Generation Quality

  • Informativeness is scored by GPT-4 comparing COCO object annotations, a ground-truth caption, and the generated caption for coverage of significant image details.Source: §A.3.2 · p. 13
  • BLEU measures the linguistic similarity between generated captions and expert-written ground-truth captions as an indicator of consistency with human responses.Source: §A.3.2 · p. 13
  • Fluency is assessed by perplexity under pretrained GPT-2, with lower perplexity indicating higher fluency.Source: §A.3.2 · p. 13

B EFUF is beneficial to other hallucination mitigation methods

EFUF can be added to MLLMs already enhanced with RLHF or instruction tuning; experiments report reduced hallucination rates without compromising generated-text quality.

  • Adding EFUF to RLHF- or instruction-tuned models reduced hallucination rates without compromising generated-text quality.The tested baselines were LLaVA-RLHF and LRV.Source: B EFUF is beneficial to other… · p. 13

C Effects of different weight

The weight experiments identify settings that balance hallucination reduction and generation quality, while the MiniGPT4 case study contrasts EFUF with baseline and sentence-loss-only outputs.

C Effects of different weight

  • λ1 = 0.3 was identified as the balance point; increasing it improved hallucination reduction and generation quality up to 0.2, while larger weights continued reducing hallucinations but degraded quality.The quality decline was especially apparent in informativeness and fluency at higher weights.Source: C Effects of different weight · p. 13, Table 7 · p. 14
  • λ2 = 0.2 was identified as optimal: 0.1 reduced fluency, while 0.3 increased the hallucination rate.The authors attribute the increase at 0.3 to sentence loss prioritizing whole-sentence patterns over unlearning hallucinated content.Source: C Effects of different weight · p. 13, Table 7 · p. 14

D Case Study

  • EFUF discouraged low-relevance objects in the MiniGPT4 case study, unlike the baseline and sentence-loss-only model, which did not fully correct hallucinated objects.The baseline generated nonexistent “large windows” and “bookshelves”; sentence-loss-only finetuning lacked an explicit signal to exclude inaccurate objects.Source: D Case Study · p. 14, Figure 5 · p. 15

Related papers

Loading 2402.09801v3…