PaperlayerView exact revision on arXiv

Source-linked AI summary

FINER: MLLMs Hallucinate under Fine-grained Negative Queries

Rui Xiao, Sanghwan Kim, Yongqin Xian, Zeynep Akata, Stephan Alaniz

arXiv:2603.17662v1cs.CVcs.AI

TL;DR

MLLMs are poorly tested on fine-grained negative queries that require rejecting subtle errors. FINER introduces targeted benchmarks and FINER-Tuning, which consistently reduces hallucinations and improves paired accuracy across FINER and broader benchmarks, with gains up to 24.2%.

  • Problem

    Existing hallucination benchmarks largely test coarse mismatches, leaving MLLMs’ ability to reject fine-grained errors involving multiple objects, attributes, and relations unclear.

  • Method

    FINER introduces two benchmarks spanning four fine-grained negative-query settings, while FINER-Tuning applies preference optimization to FINER-inspired positive and negative data.

  • Results

    FINER-Tuning consistently reduces hallucinations and improves paired accuracy across FINER, other hallucination, and general multimodal benchmarks, with gains up to 24.2%.

  • Takeaways & Limitations

    Fine-grained negative queries expose persistent hallucination vulnerabilities, while FINER-Tuning improves models across diverse backbones, training regimes, and benchmark types.

  • Takeaways & Limitations

    The supplied limitation evidence is an acknowledgments passage and does not specify a substantive limitation.

Abstract

from arXiv · show

Multimodal large language models (MLLMs) struggle with hallucinations, particularly with fine-grained queries, a challenge underrepresented by existing benchmarks that focus on coarse image-related questions. We introduce FIne-grained NEgative queRies (FINER), alongside two benchmarks: FINER-CompreCap and FINER-DOCCI. Using FINER, we analyze hallucinations across four settings: multi-object, multi-attribute, multi-relation, and ``what'' questions. Our benchmarks reveal that MLLMs hallucinate when fine-grained mismatches co-occur with genuinely present elements in the image. To address this, we propose FINER-Tuning, leveraging Direct Preference Optimization (DPO) on FINER-inspired data. Finetuning four frontier MLLMs with FINER-Tuning yields up to 24.2\% gains (InternVL3.5-14B) on hallucinations from our benchmarks, while simultaneously improving performance on eight existing hallucination suites, and enhancing general multimodal capabilities across six benchmarks. Code, benchmark, and models are available at \href{https://explainableml.github.io/finer-project/}{https://explainableml.github.io/finer-project/}.

1. Introduction

FINER investigates whether MLLMs can reject fine-grained negative queries, exposing severe hallucination brittleness as queries combine multiple objects, attributes, and relations. It introduces FINER-CompreCap and FINER-DOCCI, organizing evaluation into four fine-grained settings.

  • Motivation: MLLMs’ visual perception and instruction-following progress enables sophisticated image question answering, but their ability to detect nuanced errors in detailed image queries remains unclear.The introduction contrasts reasonable coarse-question handling with uncertainty about fine-grained understanding.
  • Motivation: Existing hallucination probes are largely coarse, testing single-object presence or limited single-object attributes and relations rather than fine-grained mismatches involving multiple elements.This motivates testing whether MLLMs can reject nuanced mistakes involving multiple objects, attributes, and relations.
  • Motivation: Question granularity is increased across seven levels by injecting one fine-grained contradiction—NEG OBJ, NEG ATTR, or NEG REL—while keeping the remaining description visually consistent.Each image-query pair requires a “Yes” or “No” answer, with the correct answer always “No”; evaluation samples 320 FINER-COMPRECAP and 1,687 FINER-DOCCI items.
  • Motivation: ∼80% to ∼20%: INTERNVL3.5-14B accuracy falls across levels 1 to 5-7 on FINER-COMPRECAP, revealing increasing false-positive hallucinations under finer negations.On FINER-DOCCI, accuracy likewise drops from ∼58% at level 1 to ∼15% by levels 6-7.
  • Contributions: FINER-CompreCap and FINER-DOCCI group fine-grained negative queries into four settings: Multi-obj, Multi-attr, Multi-rel, and Wh.The first three settings target existence and binding, according to the supplied passage.

2. FINER Benchmarks

FINER benchmarks test whether MLLMs detect missing or incorrect objects, attributes, and relations under fine-grained negative queries. They use scene-graph-based negative generation, validated annotations, and paired multiple-choice evaluation requiring correct answers to both positive and negative variants.

  • Benchmark goal: FINER composes negative questions over multiple objects, attributes, and relations to test detection of subtle scene mismatches.The benchmarks evaluate whether models can identify missing or incorrect components even when the perturbations are semantically plausible.
  • Benchmark datasets: FINER-CompreCap uses human-annotated scene graphs from COCO, while FINER-DOCCI uses 5K images with long human-written captions for larger-scale questions.CompreCap provides explicit scene graphs but is limited to COCO; DOCCI requires scene-graph extraction from captions.
  • Negative generation: For DOCCI, Gemini-2.0-Flash extracts scene-graph-like annotations from captions, followed by Qwen2.5VL-72B filtering and human verification on sampled data.These validation steps are especially important for reducing incorrect relation annotations.
  • Negative generation: Each positive scene graph yields four negative counterparts per object, attribute, and relation, generated with Qwen3-14B or Gemini-2.0-Flash and filtered by Qwen2.5-VL-72B.The generators are used for FINER-CompreCap and FINER-DOCCI respectively, while the discriminator reduces the risk that negatives are actually present in the image.
  • Evaluation: Paired MCQs require models to answer both each negative question and its corresponding positive question correctly, reducing bias toward either answer type.Each MCQ contains one correct answer and four distractors, and the paired metric requires success on both variants to guard against false positives and false negatives.

3. Training with FINER (FINER-Tuning)

FINER-Tuning addresses MLLM vulnerabilities with Direct Preference Optimization on minimally edited, semantically precise fine-grained contradictions. Its training data is generated from Pixmo captions across four evaluation scenarios and organized into preference pairs ranking correct responses above incorrect ones.

  • Motivation and approach: FINER-Tuning applies DPO to fine-grained negative queries involving objects, attributes, and relations, using minimally edited semantic contradictions.Examples contrast a car with a yellow bumper against a car with a chrome bumper.
  • Training-data generation: The pipeline uses Pixmo captions and Phi-4-14B to extract positive phrases for Multi-obj, Multi-attr, Multi-rel, and Wh scenarios.The training pipeline avoids the LLMs used to construct the benchmarks.
  • Training-data generation: Negative phrases are created by randomly replacing one selected object, attribute, relation, or Wh instance in a positive phrase.The resulting negative phrase preserves the selected scenario while introducing a contradiction.
  • DPO training: DPO training pairs positive and negative questions with accepted and rejected responses, ranking correct answers above their opposites.Accepted responses begin with “Yes” for positive questions and “No” for negative questions, then mention the correct image features.

4. Experiments

Experiments show that FINER-Tuning reduces fine-grained hallucinations on FINER benchmarks, generalizes to broader hallucination evaluations, and avoids sacrificing general multimodal capabilities. Ablations further identify the benefits of combining positive and negative queries and training across query subsets.

  • Other hallucination benchmarks: FINER-Tuning reduces hallucinations across both discriminative and generative benchmarks, whereas RLAIF-V’s gains are less consistent on discriminative tasks.RLAIF-V degrades performance relative to OmniLMM on DASH, POPE, RePOPE, and HallusionBench; FINER-Tuning provides broader improvements.
  • General MLLM capabilities: FINER-Tuning improves general multimodal performance rather than incurring an alignment tax, including a 1.4% improvement for InternVL3.5-14B across six benchmarks.The evaluation covers MMStar, TextVQA, ChartQA, MMVP, NaturalBench, and V∗Bench.
  • Qualitative analysis: Qualitative examples show FINER-Tuning removing a spurious necklace, identifying fine color details, correcting a relation hallucination, and changing an incorrect Wh-question answer.The examples are drawn from FINER-COMPRECAP, with additional qualitative results in the supplement.
  • Training-strategy ablation: 36.7%: SFT with both query types reduces Multi-obj performance relative to the baseline, while DPO with both positive and negative queries outperforms DPO using only negative queries.The ablation uses InternVL-3.5-8B variants and shows that both query types are valuable for FINER-Tuning.
  • Training-subset ablation: Training on Multi-Obj, Multi-Rel, or Wh subsets produces the best scores on corresponding tests and also improves other settings.Subset experiments use InternVL-3.5-8B while keeping the total training samples fixed at 160k.

5. Related Works

Prior hallucination benchmarks probe object, relation, attribute, and false-premise errors, but often rely on MSCOCO or limited scales. Hallucination-aware tuning uses negative instructions, contrastive pairs, misleading contexts, uncertainty reasoning, and preference learning.

  • Hallucination Benchmarks: Existing benchmarks probe object, relation, and attribute hallucinations through yes-or-no questions, categorical subsets, and other question-based evaluations.POPE uses yes-or-no questions; Amber categorizes object, relation, and attribute hallucinations; MMHalBench evaluates eight question types.
  • Hallucination Benchmarks: Many benchmarks rely on MSCOCO annotations, while DASH retrieves challenging images from LAION-5B and MMHalBench remains limited in scale.RePOPE corrects POPE annotation errors, and CRPE focuses on relation hallucinations.
  • Hallucination Benchmarks: HaloQuest includes false-premise questions, but FINER targets fine-grained attribute mismatches in existing objects rather than primarily nonexistent objects.This distinction concerns the Wh setting described in the passage.
  • Hallucination-aware Fine-tuning: Prior hallucination-aware tuning adds negative instructions, contrastive correct-versus-hallucinated pairs, misleading contexts, uncertainty tokens, retrospective reasoning, and preference learning.The cited approaches include LRV-Instruct, HALVA, PerturboLLaVA, REVERSE, and OPA-DPO.

6. Conclusion and Limitation

The paper introduces FINER to expose fine-grained-negation hallucinations in MLLMs and reports that frontier models remain vulnerable across its benchmarks. It also proposes FINER-Tuning, a model-agnostic method for improving responses to fine-grained negative queries.

  • Conclusion: FINER is a suite of fine-grained negative queries designed to reveal how MLLMs fail under precise negations.The suite evaluates hallucinations in FINER-CompreCap and FINER-DOCCI.
  • Conclusion: Systematic evaluation across all four FINER settings shows that even frontier MLLMs remain vulnerable to FINER-induced hallucinations.The evaluation covers FINER-CompreCap and FINER-DOCCI.
  • Conclusion: FINER-Tuning is a simple, model-agnostic recipe that aligns models to react correctly to fine-grained negative queries.The paper applies the approach across diverse backbones and training regimes.

Supplementary Material … B.2. SG Extraction Pipeline for FINER-DOCCI

FINER extends hallucination evaluation to fine-grained negative queries and constructs benchmarks from scene graphs derived from human annotations or dense captions. For FINER-DOCCI, a conservative multimodal and human validation pipeline prioritizes precision, with relations achieving 85.1% verification accuracy versus 82.8% for FINER-COMPRECAP.

  • A.1. Hallucination benchmarks: Existing benchmarks cover object hallucination, yes-or-no detection, modality conflicts, fine-grained alignment, and false-premise settings with differing scopes.FINER’s false premises concern fine-grained attributes of existing objects, whereas HaloQuest mainly focuses on non-existent objects.
  • A.2. Hallucination-aware Fine-tuning: FINER targets hallucinations under fine-grained negative queries involving object, attribute, and relation mismatches.It differs from prior work that mainly detects or corrects hallucinations in responses and often focuses on the LLaVA family.
  • B. FINER Benchmark Details: FINER-COMPRECAP uses human-annotated positive scene graphs, whereas FINER-DOCCI derives positive scene graphs from dense captions before applying shared negative-generation, filtering, and question-conversion pipelines.FINER-DOCCI is designed for larger-scale open-set validation, while FINER-COMPRECAP emphasizes per-sample annotation fidelity.
  • B.1. Positive SG for FINER-COMPRECAP: CompreCap provides 560 human-annotated images with scene graphs containing objects, attributes, and relations, whose attributes are rewritten into standardized phrases and relations parsed structurally.Qwen3-14B rewrites attribute sentences without changing their original meaning.
  • B.2. SG Extraction Pipeline for FINER-DOCCI: DOCCI contains 5,000 images paired with detailed human-annotated captions, from which Gemini-2.0-Flash extracts positive scene-graph information.The extraction approach avoids directly asking an LLM to summarize a full scene graph because that process is brittle and error-prone.
  • B.2. SG Extraction Pipeline for FINER-DOCCI: Candidate FINER-DOCCI relations undergo visual-textual validation using Qwen2.5-VL-72B and Gemini, with jointly rejected relations discarded and additional human filtering.The complete process filters out 1,771 relations.
  • B.2. SG Extraction Pipeline for FINER-DOCCI: The FINER-DOCCI pipeline keeps relations supported by both captions and images, adding human checks to prioritize precision over recall.This conservative design aims to improve scene-graph reliability despite known challenges in LLM-based extraction.
  • B.2. SG Extraction Pipeline for FINER-DOCCI: 85.1% versus 82.8%: InternVL3.5-8B verifies FINER-DOCCI relations slightly more accurately than FINER-COMPRECAP relations.The higher FINER-DOCCI relation accuracy may reflect more detailed annotations that provide more information for verification.

B.3. Negatives Generation Pipeline. · B.4. MCQ Design

The pipeline generates four text-space negatives per scene-graph object, attribute, and relation, then filters likely false negatives using MLLM discrimination, entropy, regeneration, and human checks. These negatives support fixed-template yes/no and Wh MCQs whose positive and corrupted variants are tightly grounded in scene graphs.

  • B.3. Negatives Generation Pipeline.: B.3. Negatives Generation Pipeline.: Four negative counterparts are generated for every object, attribute, and relation using LLMs operating purely in text space.FINER-DOCCI uses Gemini-2.0-Flash, while FINER-COMPRECAP uses Qwen3-14B; text-space generation avoids directly inheriting vision-model biases.
  • B.3. Negatives Generation Pipeline.: B.3. Negatives Generation Pipeline.: False negatives are detected by querying Qwen2.5-VL-72B on five-choice questions and treating low-entropy misclassifications as likely present entities.Low entropy is informative but imperfect because the MLLM can also be confidently wrong.
  • B.3. Negatives Generation Pipeline.: B.3. Negatives Generation Pipeline.: H(p) = 0.0119 identifies a visible “ground” negative in one example, whereas the same entropy value accompanies a confidently wrong “is behind the” prediction in another.The latter negative is replaced with a new proposal such as “is on top of the”.
  • B.3. Negatives Generation Pipeline.: B.3. Negatives Generation Pipeline.: Candidates with H(p) < θ are regenerated, rechecked by Qwen2.5-VL-72B, and periodically inspected by human annotators to reduce systematic filtering errors.Thresholds are selected by sorting misclassified examples by entropy and incrementally validating samples around candidate thresholds.
  • B.3. Negatives Generation Pipeline.: B.3. Negatives Generation Pipeline.: FINER-DOCCI achieves close to 90% classification accuracy for generated objects and attributes in the model-based quality assessment.The benchmark prioritizes large-scale evaluation of rich human-described semantics rather than a small, fully human-curated set.
  • B.4. MCQ Design: B.4. MCQ Design: Yes/no MCQs use a fixed template with one positive phrase and four corrupted alternatives, with answer choices randomly shuffled.Positive MCQs affirm the true configuration, while negative MCQs query a randomly selected corrupted phrase and make the true phrase the correction answer.
  • B.4. MCQ Design: B.4. MCQ Design: Multi-object and multi-relation MCQs replace one object, attribute, or relation at a time while keeping the remaining scene-graph content unchanged.The same positive/negative construction is applied to q±multi-obj and q±multi-rel, using attributes and relations as basic units where appropriate.

C. Training Details

FINER-Tuning constructs preference data from the first 24 Pixmo-caption shards, with each image yielding up to eight preference tuples. Training uses model-specific sample limits, a 1 × 10^-4 SFT learning rate for InternVL3.5-8B, and H100 GPUs.

  • Training data examples: The training pipeline applies to the first 24 Pixmo-caption shards, and each image can produce up to eight preference tuples.These tuples provide the preference data used to train FINER-Tuning.
  • Fine-tuning configuration: DPO training uses 40K examples for LLaVA-1.6, 120K for Qwen2.5-VL, and 160K for the InternVL3.5 series.The sample counts are limited because excessive training can drive validation loss near zero without improving performance and may degrade downstream results.
  • Fine-tuning configuration: SFT fine-tunes InternVL3.5-8B on 160K examples with a learning rate of 1 × 10^-4.This setting is reported for the SFT experiments in Table 4.

D. Evaluation Details … E.2. Ablation: Training Data Filtering

The paper specifies evaluation protocols for FINER, hallucination, and general-capability benchmarks, then reports additional analyses of positional bias and training-data filtering. These experiments examine decoding consistency, benchmark-specific metrics, position-dependent errors, and removal of out-of-scope images.

  • D. Evaluation Details: FINER benchmarks use greedy decoding at temperature 0, no sampling, and a maximum of 3 output tokens for multiple-choice evaluation.Models are instructed to answer with a single capital letter, and paired accuracy Accpaired is computed.
  • D. Evaluation Details: HallusionBench, AMBER, and CRPE R are evaluated with VLMEvalKit defaults, while DASH, POPE, and RePOPE follow official yes/no protocols.The reported metrics include all accuracy for HallusionBench and averaged accuracy for CRPE R.
  • D. Evaluation Details: MMHal-Bench and HaloQuest use released pipelines with replacement judges and temperature 0, reporting their prescribed score and hallucination metrics.GPT-4.1-mini judges MMHal-Bench, Gemini-2.0-Flash judges HaloQuest, and both generative evaluations use temperature 0 for reproducibility.
  • D. Evaluation Details: General capabilities are evaluated on six benchmarks with VLMEvalKit defaults, using overall accuracy except for NaturalBench group accuracy (G ACC).The six benchmarks are MMStar, TextVQA, ChartQA, MMVP, NaturalBench, and V∗.
  • E. Additional Experiments: Additional experiments cover positional bias, training-data filtering, qualitative FINER-DOCCI results, per-subset results, extended method comparisons, and random-guess analysis.These analyses are presented in Sections E.1 through E.6.
  • E.1. Positional bias study: FINER-Tuning consistently improves accuracy across negated positions, but positional curves remain non-flat, indicating persistent bias.InternVL3.5-8B peaks when negating the middle entity, whereas InternVL3.5-14B peaks when negating the third entity; the authors relate this to sequence structure and leave deeper investigation open.
  • E.2. Ablation: Training Data Filtering: Pixmo-caption images are filtered by classifying them as natural images, screenshot ui, chart graph, or document text because some fall outside FINER’s natural-image scope.Phi-4-14B is run over the long captions to perform this four-way classification before filtering.

E.3. Qualitative Results

Qualitative FINER-DOCCI examples span four fine-grained settings and focus on challenging negative multiple-choice questions. They show that models often miss subtle concepts or hallucinate incorrect attributes and relations, while FINER-Tuning improves fine-grained identification in an example.

  • Qualitative setup: The FINER-DOCCI examples cover Multi-obj, Multi-attr, Multi-rel, and Wh settings, visualizing negative MCQs because they are more challenging than positive counterparts.Some positive MCQs appear in human-study examples.
  • Multi-object: In Multi-obj, only Gemini-2.5-Flash and FINER-Tuning-tuned InternVL3.5-14B reliably identify the fine-grained concept “macbook”.This example contrasts a frontier model with the FINER-Tuning-tuned model on fine-grained concept identification.
  • Multi-attribute: Multi-attr questions target subtle details, including “the white note on the back driver’s side window” and “the cat with perked-up ears”.These examples test whether models recognize fine-grained attributes rather than only broad objects.
  • Multi-relation: In Multi-rel, Qwen2.5-VL-7B hallucinates that the dog is “behind the fence” even though it is clearly in front of the fence.The example demonstrates a relation-level mismatch between the model’s answer and the image.

E.4. Per-subset results … E.7. MCQ Version of the Motivational Study

The supplementary results detail FINER-Tuning’s per-subset and cross-method performance, then clarify MCQ baselines and the motivation for reformulating the motivational study. FINER-Tuning improves AMBER for LLaVA-1.6 and compares favorably with competing hallucination-reduction methods, while the MCQ setting exposes structured random-guess behavior.

  • E.4. Per-subset results: 20.1% absolute improvement on AMBER is achieved by LLaVA-1.6 with FINER-Tuning in the per-subset evaluation.The results further demonstrate FINER-Tuning’s effectiveness on AMBER.
  • E.4. Per-subset results: FINER-Tuning can further improve captioning capabilities when the base models are sufficiently strong, according to results on AMBER.This observation is reported for 7B, InternVL-3.5-8B, and InternVL3.5-14B models.
  • E.5. Comparing with more methods: The comparison fine-tunes LLaVA-1.5-7B with 40K FINER-Tuning examples and evaluates POPE, AMBER, MMHal-Bench, and HaloQuest against REVERSE, DoLA, and HA-DPO.The benchmarks include discriminative and generative hallucination evaluations.
  • E.5. Comparing with more methods: Using 40K Phi-4-14B-curated training samples, FINER-Tuning matches or surpasses REVERSE on discriminative tasks and improves HaloQuest by 6.3%, but trails on MMHal-Bench.It is comparable to HALVA and HA-DPO on discriminative benchmarks while substantially outperforming them on generative benchmarks.
  • E.6. Smarter random guess baselines: The uniform random-guess baseline is 4%, corresponding to independently selecting one of five answer options for both positive and negative questions.The probability is expressed as (1/5)^2.
  • E.6. Smarter random guess baselines: The polarity-aware random guesser first chooses Yes or No uniformly, then uniformly selects among the four No options when it chooses No.This baseline reflects the structured answer space of Multi-obj, Multi-attr, and Multi-rel MCQs.
  • E.6. Smarter random guess baselines: The paired accuracy of this polarity-aware baseline is 0.0625, calculated as 0.5 × (0.5 × 0.25).For the negative MCQ, the probability is 0.5 × 0.25.
  • E.7. MCQ Version of the Motivational Study: Because prior benchmarks use yes/no probing while FINER uses MCQs, the motivational study is additionally reformulated with the MCQ protocol to reduce confusion between evaluation settings.Yes/no probing is standard in DASH, POPE, and AMBER for false-positive hallucinations; Figure 12 presents the MCQ study and FP/FN rates by granularity.

F. Human Study · G. Templates

The human study finds over 90% paired accuracy across all FINER settings, while revealing carelessness and ambiguity as residual error sources and motivating MCQ-based evaluation. FINER-Tuning data are constructed with LLM-generated positive/negative phrases and preference tuples, using randomized templates for OBJ/ATTR/REL and dedicated generation for WH questions.

  • F. Human Study: Humans achieve over 90% paired accuracy across all settings in FINER-COMPRECAP and FINER-DOCCI.The study evaluates 20 MCQs per subset across eight subsets, totaling 160 MCQs.
  • F. Human Study: Human errors mainly reflect carelessness or ambiguity in subjective visual attributes.Examples include selecting an incorrect window-related option through oversight and confusing bald ears with floppy ears.
  • F. Human Study: MCQ evaluation reduces ambiguity by requiring humans and models to choose among multiple descriptions rather than answer simple yes/no questions.Subjectivity, ambiguity, and annotation errors nevertheless remain unavoidable despite filtering, verification, and MCQ design.
  • F. Human Study: The human study is limited to 20 MCQs per subset because FINER is highly text-intensive and requires substantial reading time.The authors treat this limited scale as a limitation because larger studies may introduce reading burden and additional noise.
  • G. Templates: For OBJ, ATTR, and REL, the construction pipeline extracts positive phrases, generates corresponding negative phrases, and composes preference tuples with five templates.The same LLM is prompted for phrase generation before template-based composition.
  • G. Templates: One of five templates is randomly selected for each example to avoid overfitting to a fixed pattern and remain consistent with FINER benchmarks.Templates use placeholders {X}, {Y}, and {Z1}, . . . , {Z4} filled with generated phrases.
  • G. Templates: WH preference tuples are generated directly by the LLM rather than through fixed templates.The LLM creates positive and partially conflicting negative questions, then symmetrizes the pair by assigning each accepted response as the other question’s rejected response.
  • G. Templates: Gemini-2.0-Flash relation extraction is prompted to inspect object pairs and extract all directed relations explicitly stated in the description.The prompt assumes a natural-language image description and a numbered catalog of main objects with all attributes.
Loading 2603.17662v1…