PaperlayerView exact revision on arXiv

Source-linked AI summary

FPCO-Dialog: A Multi-Turn False-Premise Benchmark for Correction and Cooperation in Vision-Language Models

Jiayuan Ma, Yuqi Lu, Weiyang Guo, Chenrui Wang, Junyi Shu, Xuebo Liu, Min Zhang, Jing Li

arXiv:2609.03331v1cs.CL

TL;DR

Existing evaluations rarely isolate how VLMs handle the same visually grounded false premise across multiple dialogue turns. FPCO-Dialog introduces a controlled, stratified benchmark and finds persistent cross-model differences, turn-wise dynamics, and false-premise-type variation.

  • Problem

    Existing evaluations rarely isolate VLM responses when the same visually grounded false premise persists across dialogue turns.

  • Method

    FPCO-Dialog evaluates 20 VLMs using a ten-turn protocol, CorrTP@K, and two independent detectors on 1,080 stratified images.

  • Results

    The benchmark reveals substantial persistent cross-model differences in aggregate correction tendency, model-specific turn-wise dynamics, and systematic false-premise-type variation.

  • Takeaways & Limitations

    Correction behavior varies across models and false-premise types, with identity errors corrected most often and location errors frequently handled through cooperation under the current substitution distribution.

  • Takeaways & Limitations

    The benchmark covers only three false-premise classes, four object categories, and English questions, while detector judgments and substitution choices can affect results.

Abstract

from arXiv · show

Vision-language models (VLMs) are increasingly deployed in multi-turn settings where users may describe visual content with incorrect assumptions. Yet existing evaluations rarely isolate how models respond when the same visually grounded false premise persists across dialogue turns. We introduce FPCO-Dialog, a benchmark for evaluating correction and cooperation behavior in VLMs under repeated false premises. FPCO-Dialog contains 1,080 images and 10,800 question turns, stratified by visual complexity, object category, and false-premise class, and uses a 10-turn protocol in which a correct dialogue prefix is followed by repeated false-premise referring expressions. We evaluate 20 commercial and open-source VLMs with a model-agnostic protocol and CorrTP@K, a correction-rate metric over false-premise turns, scored by two independent detectors. FPCO-Dialog reveals substantial and persistent cross-model differences in aggregate correction tendency, model-specific turn-wise dynamics, and systematic variation across false-premise types under the benchmark's substitution distribution. The dataset, evaluation protocol, model outputs, detector labels, and code are available.

1 Introduction

Existing VLM evaluations rarely isolate responses to repeated visually grounded false premises across dialogue turns. FPCO-Dialog introduces a controlled benchmark to measure correction–cooperation behavior and compares 20 models.

  • The benchmark measures whether VLMs explicitly correct a false premise or cooperate with the user’s main question without repairing it.
  • Repeated false premises are difficult to study because existing evaluations mostly use independent questions or naturally collected dialogue histories.
  • FPCO-Dialog uses a fixed ten-turn dialogue with a premise-correct prefix followed by repeated false-premise referring expressions.The benchmark covers identity, attribute, and location errors across 1,080 stratified images.
  • The study evaluates 20 models with a model-agnostic framework including CorrTP@K, dual-detector scoring, model outputs, detector labels, and code.
  • The results show substantial cross-model differences and systematic variation across false-premise types.

2 Related Work

Prior work evaluates multimodal competence, dialogue grounding, hallucination, and false-premise answering, but does not provide the same controlled repeated-false-premise comparison. FPCO-Dialog addresses this gap by varying premise class alongside visual and object conditions.

  • VLM benchmarks have expanded from visual question answering to integrated reasoning, expert multimodal problem solving, hallucination diagnosis, and preference-based open-ended evaluation.
  • Related QA and VQA tasks study unanswerable questions, abstention, hallucination, and false premises, typically framing the problem as unsupported-answer detection.
  • FPCO-Dialog structures dialogues with three premise-correct turns followed by repeated false-premise turns.
  • Its stratification crosses single-target clean, multi-target separated, and multi-target overlapping images with person, vehicle, animal, and food categories.

3 FPCO-Dialog Dataset

FPCO-Dialog is a manually curated, stratified dataset for testing VLM responses to persistent visual inconsistencies. Each image is paired with a ten-turn sequence that changes one target description into a repeated false premise.

  • Each instance contains an image, target entity, correct and modified target descriptions, and a ten-turn question sequence processed with preceding dialogue context.
  • The first three turns use the correct description, while turns 4–10 repeat the same modified referring expression.This schedule tests whether correction behavior persists, changes, or is suppressed across turns.
  • Images are selected from MS COCO and Open Images and organized for comparisons across visual complexity, object category, and false-premise class.The authors define the design and constraints, while LLMs assist with question generation and rewriting.
  • The dataset contains 1,080 images across three visual-complexity levels, four object categories, and three false-premise classes.
  • Identity errors replace target identity, attribute errors change a visible property, and location errors change spatial description or relation.
  • Substitutions follow a constrained minimal-edit policy that changes one word or short phrase and introduces only one assigned-class error.

4 Experiment Setup

FPCO-Dialog evaluates 20 VLMs with a shared ten-turn, model-agnostic protocol and detector-based scoring to measure correction behavior under repeated false premises.

  • Evaluated models: 20 commercial/API-based and open-source VLMs from five model families are evaluated, producing 10,800 responses per model and 216,000 responses overall.The evaluation is inference-only and includes Gemini, GPT, Qwen, InternVL, and LLaVA models.
  • Evaluation protocol: Each model receives the image, current question, and preceding dialogue context at every turn, generating responses sequentially across the ten-turn dialogue.The same benchmark instances, turn order, and false-premise schedule are used for all models.
  • Evaluation protocol: The protocol withholds false-premise annotations, target descriptions, and detector labels from models to isolate behavior under identical benchmark access.Any VLM supporting image-conditioned multi-turn responses can be evaluated with the same dialogue and scoring pipeline.
  • Dual-detector scoring: Two independent detectors score whether responses explicitly identify or clearly repair the false premise, while responses that accept or ignore it are not corrections.The main reported label is the arithmetic mean of the two binary detector labels.
  • Metrics: CorrTP@K measures the proportion of false-premise response items up to turn K that receive an explicit correction or clear premise repair.Higher CorrTP@K indicates stronger explicit premise-correction tendency, not general correctness, helpfulness, safety, or pragmatic appropriateness.
  • Metrics: TurnCorr captures non-cumulative correction at each false-premise turn, while CorrFP@K checks correction on premise-correct turns for indiscriminate over-correction.Definitions, formulas, and full model-level results are provided in Appendix F.

5 Results

FPCO-Dialog reveals persistent differences in models’ aggregate correction tendencies, strong variation by false-premise type, and clearer interaction patterns between visual complexity and premise type than complexity alone.

  • Evaluation: Detector labels were averaged from GPT-5.4 and Gemini-3.1-Pro-Preview, with strong agreement against expert annotations on 360 stratified responses.The full validation protocol and results are reported in Appendix C.
  • Correction Behavior across Models: Representative models show well-separated CorrTP@K trajectories, with cross-model differences remaining relatively stable from K = 4 to K = 10.Non-cumulative TurnCorr@K additionally reveals model-specific stability or decay after the first false-premise turn.
  • False-premise Type Matters: Identity errors have average CorrTP@10 of approximately 0.56, compared with approximately 0.41 for attribute errors and 0.15 for location errors.This ordering is descriptive under the benchmark’s current substitution distribution; location premises are often handled through cooperation.
  • Visual Complexity and Interaction Effects: Average CorrTP@10 varies modestly by visual complexity: approximately 0.39 for single-target clean, 0.38 for multi-target separated, and 0.35 for multi-target overlapping images.The complexity-only effect is weaker than the interaction patterns obtained by crossing complexity with false-premise type.
  • Visual Complexity and Interaction Effects: Across complexity settings, identity correction remains comparatively high, attribute correction declines, and location correction remains low.The reported identity values are approximately 0.55, 0.56, and 0.58; attribute values decline from approximately 0.50 to 0.39 and 0.34; location values are approximately 0.12, 0.18, and 0.14.
  • Detector Consistency: The two detectors produce similar model-family trends and the same overall ordering across false-premise classes, despite disagreements on borderline responses.The main score therefore uses their arithmetic mean.

6 Case Studies

Case studies show that models adopt different interaction strategies under matched visual and dialogue conditions, and that correction can coexist with answering the user’s question.

  • Matched Model Strategies: Under the same identity false premise, GPT-5.4 answers under the modified referring expression while Qwen3.6-Plus explicitly corrects it.These matched examples distinguish interaction strategy from task accuracy.
  • Correction and Cooperation: Correcting a false premise need not be refusal: models can repair an animal or color description and still answer the main question.CorrTP@K therefore measures premise-correction behavior rather than willingness to help.
  • Location Premises: Location false premises are often handled cooperatively, with models answering the requested visual question without explicitly challenging the spatial phrase.The examples support the aggregate pattern that spatial mismatches may be treated as non-central to the main question.
  • Attribute Premises: The same red–green train mismatch elicits correction from GPT-5.4 but accommodation from GPT-4o, illustrating model-dependent attribute behavior.This complements the aggregate intermediate regime for attribute errors.

7 Conclusion

FPCO-Dialog provides controlled measurement of VLM correction–cooperation behavior under repeated visual false premises. Across 20 VLMs, it finds persistent model differences and systematic premise-type variation, motivating evaluation of responses to conflicts between referring expressions and visual evidence.

  • Benchmark Scope: FPCO-Dialog fixes the dialogue protocol while varying false-premise class, visual complexity, and object category for controlled model comparison.The benchmark evaluates responses to repeated visually grounded false premises.
  • Main Findings: Evaluating 20 commercial/API-based and open-source VLMs reveals substantial persistent cross-model differences and model-specific turn-wise dynamics.These findings cover both aggregate correction tendency and within-dialogue behavior.
  • Main Findings: Correction varies systematically by premise type: identity errors are corrected most often, attribute errors are intermediate, and location errors are frequently handled through cooperation.The ordering is reported under the benchmark’s current substitution distribution.

Limitations

The benchmark’s fixed schedule and English-only coverage constrain generalization, while detector interpretation and substitution properties limit how its correction scores should be read.

  • Scope: The fixed schedule covers only three false-premise classes, four object categories, and English questions, leaving broader linguistic, cultural, and domain coverage for future work.It does not cover all open-ended false-premise interactions.
  • Interpretation: Class-level differences may depend on lexical, semantic, and contextual substitution properties, and the identity–attribute difference is not robust to lexical variation.CorrTP also measures explicit premise repair rather than answer quality or pragmatic necessity.
  • Measurement: Scoring uses two VLM-based detectors, so borderline cases may depend on detector interpretation, and evaluated behavior may change as systems and checkpoints are updated.

Ethical Considerations

FPCO-Dialog uses licensed source images and author-generated benchmark components, while synthetic false premises avoid sensitive-attribute inference. Automated scoring aggregates two detector judgments to reduce detector-specific bias.

  • FPCO-Dialog uses images from MS COCO and Open Images, with source images remaining under their original licenses.
  • The authors release metadata, questions, false-premise modifications, the evaluation protocol, model outputs, detector labels, and code.
  • The dataset contains synthetic false premises, avoids sensitive-attribute inference, and excludes names, unique identifiers, and offensive descriptions from author-generated content.
  • LLMs assist with question generation, false-premise rewriting, evaluation detection, and writing, but authors define and verify the benchmark and findings.
  • CorrTP@K averages binary labels from GPT-5.4 and Gemini-3.1-Pro-Preview to reduce potential single-detector bias.
  • The detectors achieve exact agreement of 0.973 and Cohen’s κ of 0.942 overall, with lower but substantial agreement for location cases at κ = 0.809.

C Human Validation of LLM as Detectors

Human validation tests whether automated detectors apply FPCO-Dialog’s correction criterion consistently. Agreement is strong between experts and detectors, supporting an automated and scalable scoring pipeline, while self-scoring checks leave model rankings unchanged.

  • Human validation: 360 responses were sampled across 20 models, three false-premise classes, and turns 4–10 for stratified human validation.The design samples six responses per model, class, and false-premise turn range.
  • Human validation: Annotators label a response as corrective when it identifies or repairs the false premise, and non-corrective when it accepts or ignores it.
  • Agreement: Human annotators reach strong overall agreement at Cohen’s κ = 0.859, while human–detector agreement ranges from 0.832 to 0.855.
  • Agreement: The two automated detectors achieve Cohen’s κ = 0.960 on the human-validation subset and remain above 0.80 across all false-premise classes.
  • Interpretation: Human validation supports detector judgments while preserving an automated, scalable benchmark evaluation pipeline.
  • Self-scoring check: Leave-one-judge-out scoring leaves the overall 20-model ranking unchanged, indicating no material effect from detector self-scoring.GPT-5.4 changes from 0.2939 to 0.2886, and Gemini-3.1-Pro-Preview from 0.5880 to 0.5906.
  • Extended metrics: CorrTP@K aggregates correction responses through turn K, whereas TurnCorr@K isolates the current false-premise turn.
  • Extended metrics: CorrFP@K checks whether models correct premise-correct turns, complementing CorrTP@K with a false-positive correction measure.

G Lexical Sensitivity Analysis

A matched exploratory study tests whether class-level correction differences depend on lexical substitutions. Location premises consistently receive lower correction rates, but the identity–attribute ordering is sensitive to lexical variation.

  • Study design: The sensitivity study holds image, target, dialogue prefix, and Q4 intent fixed while comparing two alternative substitution sets across three models.
  • Findings: Both substitution sets produce lower correction rates for location premises than for identity and attribute premises.
  • Findings: 73.1% of 1,000 random A/B selections retain the full Identity > Attribute > Location ordering.
  • Findings: The separation of location from identity and attribute is comparatively robust, whereas the identity–attribute difference is sensitive to lexical variation.
  • Caveat: The main class ordering is treated as descriptive under the current substitution distribution rather than as a class-only causal effect.
  • Caveat: The exploratory analysis does not fully control semantic or perceptual salience.

H Protocol Robustness

Protocol-robustness analyses examine sequence length, sampling, and implementation choices. The class ordering persists across cutoffs and stratified sample sizes, while turn-level analyses reveal model-specific within-dialogue dynamics.

  • Protocol scope: The benchmark does not claim that its three-turn prefix, seven false-premise turns, or current dataset size are uniquely optimal.
  • Sequence robustness: Identity > Attribute > Location appears from K = 4 and remains unchanged through K = 10 under truncated false-premise sequences.
  • Turn-level dynamics: For 10 of 20 evaluated models, TurnCorr decreases by more than five percentage points from Q4 to Q10.
  • Sampling robustness: Across 1,000 repetitions at each of 10, 15, 20, 25, and 30 images per cell, Identity > Attribute > Location is retained in every repetition.
  • Sampling robustness: The same ordering holds under stratified sampling without replacement, indicating stability to substantially smaller stratified samples.
  • Construction: Question generation selects one visible target and creates ten questions that share its target description.
  • Construction: False-premise rewriting minimally modifies the target description and reuses it consistently from turns K = 4 to K = 10.
  • Inference: During inference, models receive the image, current question, and prior dialogue history without benchmark-side false-premise annotations or scoring metadata.
Loading 2609.03331v1…