PaperlayerView exact revision on arXiv

Source-linked AI summary

Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination

Chufan Shi, Cheng Yang, Yaokang Wu, Linghao Jin, Bo Shui, Taylor Berg-Kirkpatrick, Xuezhe Ma

arXiv:2605.15864v2cs.CVcs.CL

TL;DR

VLMs can say they will re-examine an image, but whether they actually use new visual evidence is an open question. The paper probes this with image swaps after reasoning and finds models often miss the change; explicit user intervention, unlike self-reflection, restores visual grounding.

  • Problem

    The paper asks whether VLM self-reflection genuinely re-attends to images or reproduces learned linguistic patterns, a question relevant to model optimization and trustworthiness.

    Source: §1 · p. 1

  • Method

    VisualSwap retains a model’s reasoning while replacing the image, and VS-Bench supplies 800 similar but answer-divergent image pairs from four benchmarks.

    Source: §1 · p. 1

  • Results

    Across 15 models, image swaps cause substantial accuracy drops of up to 60%, while self-reflection mobilizes less visual attention than explicit multi-turn user instructions.

    Source: §1 · p. 2

  • Takeaways & Limitations

    In these evaluations, self-generated reflection does not reliably restore visual grounding, whereas explicit user intervention does.

    Source: §1 · p. 2, §7 · p. 9

    The evaluation assumes each image pair’s question remains semantically valid and naturally applicable to both images.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Vision-Language Models (VLMs) often produce self-reflective statements like "let me check the figure again" during reasoning. Do such statements trigger genuine visual re-examination, or are they merely learned textual patterns? We investigate this via VisualSwap, an image-swap probing framework: after a model reasons over an image, we replace it with a visually similar but semantically different one and test whether the model notices. We introduce VS-Bench, 800 image pairs curated from MathVista, MathVerse, MathVision, and MMMU-Pro. Experiments on Qwen3-VL, Kimi-VL, and ERNIE-VL reveal a striking failure: models overwhelmingly miss the swap, with accuracy dropping by up to 60%. Counterintuitively, thinking models are nearly 3x more vulnerable than their instructed counterparts, and scaling offers no mitigation. Multi-turn user instructions restore visual grounding, but self-generated reflective statements during continuous generation do not. Attention analysis explains why: user instructions substantially elevate attention to visual tokens, whereas self-reflection does not. Current VLMs tend to say rather than actually see when claiming to perform visual re-examination. Our code and dataset are available at the project page: https://visualswap.github.io

1. Introduction

The paper asks whether VLMs genuinely revisit visual evidence when they generate self-reflective statements or merely reproduce learned linguistic patterns. VisualSwap tests this by replacing an image after reasoning; experiments find that models often miss the change, with self-reflection failing to mobilize visual attention as effectively as explicit user instructions.

Motivation

  • The motivating question is whether statements such as “let me check the figure again” reflect genuine re-attention or learned linguistic patterns.The authors frame this as relevant to model optimization and trustworthiness.Source: §1 · p. 1

Approach

  • VisualSwap probes re-examination by retaining prior reasoning while replacing the image and prompting the model to reflect.The paper also introduces VS-Bench, an 800-pair benchmark with visual changes that alter answers while preserving question validity.Source: §1 · pp. 1–2

Experiments

  • Across 15 models, image swaps expose persistent reliance on the initial visual context, with accuracy drops reaching 60%.Thinking models are more vulnerable than instruct counterparts, and larger models do not consistently mitigate the failure.Source: §1 · p. 2

Mechanism and intervention

  • Self-reflective statements elicit insufficient visual attention, while explicit multi-turn user instructions substantially elevate attention and restore grounding.Extended reasoning contexts exacerbate the tendency to declare re-examination without executing it.Source: §1 · p. 2

2. VISUALSWAP: A Diagnostic Framework for Visual Re-examination

VISUALSWAP tests whether a VLM uses current visual evidence after generating reasoning from an earlier image. Its protocol swaps the image while retaining the reasoning context, and its metrics compare ordinary accuracy with performance under this re-examination probe.

2.1. Overview

  • VISUALSWAP first generates reasoning from an original image and question, then substitutes an alternative image while retaining the reasoning context and prompting re-examination.The design tests whether responses follow current visual evidence or prior linguistic reasoning.Source: §2.1 · pp. 2–3, §2.3 · p. 3

2.2. Problem Formulation

  • Each evaluation instance pairs visually similar images whose answer-critical differences produce distinct ground-truth answers to the same question.The formulation denotes the original and alternative images as Ia and Ib, and their answers as Aa and Ab.Source: §2.2 · p. 3

2.3. Evaluation Protocol

  • The probe appends a reflection prompt to reasoning generated from Ia, replaces it with Ib, and asks the model to continue; successful re-examination requires detecting the discrepancy and revising the response.Source: §2.3 · p. 3

2.4. Evaluation Metrics

  • Base accuracy measures standard inference on Ib, probe accuracy measures final accuracy under VISUALSWAP, and degradation is the gap between them.A substantial positive gap indicates insufficient incorporation of current visual evidence during reflective revision.Source: §2.4 · p. 3

3. VS-BENCH: A Benchmark for Visual Re-examination

VS-BENCH is an 800-pair benchmark designed to evaluate whether VLMs re-examine visual input during self-reflective reasoning. It uses similar but answer-divergent image pairs drawn from four benchmarks, with annotation and verification procedures intended to support controlled, detectable comparisons.

3. VS-BENCH: A Benchmark for Visual Re-examination

  • VS-BENCH comprises 800 image pairs and corresponding questions and answers for testing visual re-examination, a capability distinct from static visual understanding.Source: §3 · p. 3

3.1. Design Principles

  • Pairs preserve question validity and strong visual similarity while differing in details that lead to distinct answers.This combination makes prior reasoning superficially plausible while leaving a specific visual discrepancy to detect.Source: §3.1 · p. 3

3.2. Data Sources

  • The benchmark draws samples from MathVista, MathVerse, MathVision, and MMMU-Pro, which cover visual mathematical reasoning and expert-level multimodal tasks.Source: §3.2 · p. 4

3.3. Data Annotation

  • Human-in-the-loop annotation identifies answer-critical elements, specifies answer-changing edits, and iteratively refines generated alternative images.The process selects 200 high-quality pairs from each of the four sources, totaling 800 pairs.Source: §3.3 · p. 4

3.4. Quality Verification

  • CLIP similarity of 0.95, SSIM of 0.86, and LPIPS of 0.14 indicate visually similar pairs with answer differences detectable by human observers.The three metrics assess semantic alignment, structural consistency, and perceptual distance, respectively.Source: §3.4 · p. 4, Table 1 · p. 4

4. Experiments

Across the evaluated VLMs, swapping the image during reflection sharply reduces accuracy. Thinking variants degrade more than instructed counterparts, and larger scale does not resolve the failure.

4.1. Experimental Setup

  • The evaluation spans recent VLM families and scales, comparing single-turn baseline accuracy with continued generation after image replacement for probe accuracy.Closed-source models were excluded because they did not support the probing mechanism; experiments used official chat templates and default generation configurations.Source: §4.1 · p. 4

4.2. Main Results

  • 88.8% to 34.1%: Qwen3-VL-235B-Thinking accuracy plummets after the image swap, reflecting a broader failure across models and benchmarks.Table 2 reports base accuracy (AccBase), probe accuracy (AccProbe), and degradation (∆).Source: §4.2 · pp. 4–5
  • 17.9% for Qwen3-VL-32B-Instruct versus 48.3% for its Thinking variant, while ERNIE-4.5-VL drops 34.3% versus 60.3%, respectively.Thinking variants show substantially larger degradation than instruction-tuned counterparts across model families.Source: §4.2 · p. 5
  • ∆=39.4% for Qwen3-VL-8B-Thinking versus ∆=54.6% for Qwen3-VL-235B-A22B-Thinking, showing that increased scale does not mitigate degradation.Increased capacity improves baseline accuracy but does not address the limitation in visual re-examination during continuous decoding.Source: §4.2 · p. 5

5. Discussion

Discussion experiments show that models can solve swapped images independently and recover visual grounding when users explicitly request re-examination, but continuous self-reflection does not reliably redirect attention. Longer retained reasoning worsens probe accuracy, while stronger image-token attention partly mitigates the failure.

5.1. Models Can Solve Both Images Independently

  • Baseline accuracy differs by only 0.1%–6.3% between Ia and Ib, versus 20%–55% degradation in Probe, supporting that Ib is not inherently harder.Models that answer Ib correctly in isolation can fail to deploy that capability after the swap during continuous generation.Source: §5.1 · p. 5, Table 3 · p. 7

5.2. Explicit User Instruction Restores Visual Grounding

  • Qwen3-VL-235B-A22B-Thinking recovers from 34.1% under Probe to 85.4% under Multi-turn (+51.3%), nearly matching its 88.8% baseline.The authors attribute the recovery to explicit external instruction unlocking retained visual-recognition capability.Source: §5.2 · p. 5, Table 4 · p. 7

5.3. Attention Patterns Explain the Gap

  • At Layer 55, Qwen3-VL-235B-Thinking’s visual-attention increase is ∆=1.07 in Probe versus ∆=2.21 in Multi-turn; Qwen3-VL-8B-Thinking shows ∆=2.44 versus ∆=4.70.The authors link the gap to autonomous attentional-control failure and increasing decoupling from visual tokens during extended chain-of-thought, especially for Thinking models.Source: §5.3 · pp. 6–7

5.4. Impact of Reasoning Context Length

  • Probe accuracy declines monotonically as retained reasoning increases; Qwen3-VL-235B-Thinking falls from 88.8% at 0% context to 34.1% at 100% context, while Multi-turn restores 85.4%.The decline is particularly severe for Thinking models, and the figure reports probe accuracy versus retained reasoning length.Source: §5.4 · p. 7, Table 4 · p. 7, Figure 4 · p. 8

5.5. Robustness to Prompt Variations

  • Across 10 semantically equivalent prompt variants, probe degradation remains consistent, with standard deviations σ ≤1.1 across models.Qwen3-VL-235B-Thinking has a reported bound of 36.5 ± 4.9.Source: §5.5 · p. 8, Table 6 · p. 8

5.6. Probing Setup Reflects Natural Behavior

  • Reflective triggers occur in 87%–97% of natural generations, and swapping at natural reflection points yields accuracy at or below standard probing across variants.The authors use standardized prompt injection in the main experiments to enable fair cross-model comparison because natural reflection wording varies too widely to enumerate exhaustively.Source: §5.6 · p. 8, Table 7 · p. 8, Table 8 · p. 9

5.7. Amplifying Visual Attention Mitigates the Illusion

  • A training-free 2× image-token attention intervention improves the Thinking variant’s accuracy by +18.2%, partially restoring grounding without parameter updates or additional training.The authors describe this as proof-of-concept.Source: §5.7 · p. 9, Table 9 · p. 9

6. Related Work

Related work traces VLMs from cross-modal architectures toward LLM-backed systems, and describes efforts to add reasoning and self-reflection. Whether those reflections genuinely re-examine visual content remains an open question addressed by VISUALSWAP.

Vision-Language Models

  • Conventional VLMs combine unimodal encoders with cross-modal fusion, while modern systems use LLM backbones and modality alignment to support multimodal understanding and generation.The cited work also notes limits in conventional VLM generalization to open-domain generation and complex reasoning.Source: §6 · p. 9

Reasoning in VLMs

  • VLM reasoning research ranges from reasoning-chain instruction tuning to reinforcement learning, enabling long-horizon textual reasoning that can include self-reflection.The passage frames genuine visual re-examination during these reflective behaviors as unexplored.Source: §6 · p. 9
  • VISUALSWAP addresses this open question by replacing visual inputs to test whether VLM reflections are grounded in visual perception.The passage identifies uncertainty about whether reflective behavior genuinely re-examines visual content.Source: §6 · p. 9

7. Conclusion

The paper concludes that VLMs can claim to re-check images while remaining anchored to textual reasoning. Explicit user intervention restores visual grounding, indicating a control failure rather than absent visual capability.

7. Conclusion

  • Current VLMs may claim to check images again yet remain blinded by textual reasoning inertia during continuous decoding.The paper calls this the “illusion of visual re-examination.”Source: §7 · p. 9
  • Explicit user intervention restores visual grounding to near-baseline levels, while self-reflection fails to mobilize visual attention.The conclusion characterizes this as a failure of control, not capability.Source: §7 · p. 9
  • The authors argue that current chain-of-thought paradigms prioritize textual coherence over visual intake and call for mechanisms enabling genuine autonomous attentional control.Source: §7 · p. 9

Impact Statement

This material presents the paper’s societal motivation, qualitative examples of failed and successful re-examination, and experimental setup and baseline checks. The examples show how prior reasoning can persist despite a changed image, while baseline evaluations find the swapped images remain within model capability.

Impact Statement

  • The work highlights a potential risk of unwarranted trust in stated verification in safety-critical settings such as medical diagnostics and autonomous navigation.It aims to support reliability and transparency by identifying reasoning inertia; the authors foresee no negative societal consequences from this diagnostic research.Source: Impact Statement · p. 10

A. Case Study

  • The case study uses representative examples from geometry, chart understanding, synthetic-scene VQA, and function plots to illustrate re-examination mechanisms and failures.Source: A. Case Study · p. 13

A.1. The Illusion of Re-examination

  • Examples show textual inertia overriding changed angles, bar heights, object colors or counts, and function-plot relationships during reflection.The model continues prior reasoning despite visible changes in the swapped image.Source: §A.1 · p. 13

A.2. Evidence of Genuine Grounding

  • Other examples show models detecting changed angles, bars, objects, and function shapes when they successfully re-examine the image.The cases demonstrate visual re-examination capability despite its frequent suppression during autoregressive generation.Source: §A.2 · p. 13

B. Models

C. Experimental Setup

  • Inference uses vLLM on NVIDIA H200 GPUs at temperature τ = 0.1, with 10 Qwen3-VL sampling runs showing accuracy standard deviation below 0.02.The passage also specifies VLMEvalKit for evaluation and distinct prompts for self-reflection and user intervention.Source: C. Experimental Setup · p. 14

D.1. Detailed Baseline Results

  • Across MathVista, MathVerse, MathVision, and MMMU-Pro, baseline performance on original and swapped images is comparable, indicating the swaps are within model capability.Qwen3-VL-235B-A22B-Thinking scores 93.5% on MathVista with the original image and 91.0% with the swapped image; the authors attribute Probe-setting drops to reasoning inertia.Source: §D.1 · p. 14

D.2. Detailed Multi-turn Results

The detailed results show that separate user turns recover performance after self-reflection probes fail, while longer retained reasoning context worsens probe accuracy. Additional analyses isolate the role of meaningful content and turn structure, and verify that image differences are detectable by people.

D.2. Detailed Multi-turn Results

  • Across benchmarks, Multi-turn interaction recovers performance to near-baseline after Probe failures; Qwen3-VL-235B-Thinking rises from 22.5% to 97.0% on MathVerse.The comparison covers Standard Baseline, Probe, and Multi-turn accuracy across all four benchmarks.Source: §D.2 · pp. 14–15, Table 12 · p. 15

E. Reflection Prompt Variants

F. Implementation Protocol for Image Swapping

  • Image swaps use a full re-prefill with the swapped image; Probe and Multi-turn differ in turn boundaries while retaining the same image and textual context.In Probe, reasoning continues within one assistant response; Multi-turn closes that response before a new user instruction and subsequent generation.Source: F. Implementation Protocol for Image… · pp. 15–16

D.3. Detailed Context Length Ablation

  • Probe accuracy declines monotonically in almost every context-length condition; Qwen3-VL-235B-Thinking on MathVision falls from 74.0% at 0% context to 31.0% at 100%.The degradation is more severe for Thinking than Instruct models across tasks.Source: §D.3 · p. 15, Table 13 · p. 16

H. Decomposition of User Instruction Effects

  • Only Multi-turn combines semantically meaningful content with turn boundaries to restore performance; unusual meaningful probes help Thinking models partially, while random strings degenerate and system tokens alone have negligible effect.Qwen3-VL-235B-A22B-Thinking improves from 34.1% to 47.8% with the high-perplexity meaningful probe, but no Probe-style intervention approaches Multi-turn performance.Source: Table 15 · p. 17, H. Decomposition of User Instruction… · p. 17

G. Human Baseline for Image Pair Distinguishability

  • All five volunteers identified the answer-critical visual differences in their assigned 50 image pairs, yielding a 100% success rate.The pairs were sampled across the four data sources, and volunteers considered the question when identifying differences.Source: G. Human Baseline for Image Pair… · p. 17

I. Stratified Analysis by Initial Correctness

The stratified analysis separates examples initially solved correctly from those initially answered incorrectly. After image swaps, initially correct examples often become incorrect, while Thinking models that remain wrong typically repeat their original error.

I. Stratified Analysis by Initial Correctness

  • The analysis stratifies Probe outcomes by whether the model initially answered the original image correctly.For initially incorrect examples, it further separates correct answers on the swapped image, repeated errors, and different errors.Source: §I · pp. 17–18
  • Most initially correct examples flip to incorrect after the swap, with Thinking models showing the highest flip rate.Qwen3-VL-235B-A22B-Thinking flips 447 of 675 initially correct examples (66.2%), indicating degradation on examples it had initially solved.Source: Table 16 · p. 18, §I · p. 18
  • Among persistently incorrect cases, Thinking models overwhelmingly repeat the same error rather than making a new one.Qwen3-VL-235B-A22B-Thinking repeats its error in 72 of 80 cases (90%); the passage contrasts this with a more balanced same-error/new-error split for Instruct models.Source: §I · p. 18, Table 16 · p. 18

J. Control Experiment with Visually Distinct Images

A control swaps in an unrelated image to test whether visual similarity explains failures to notice swaps. Thinking models under Probe still often miss the change, while Multi-turn detection is much higher, though detection does not establish task-level correctness.

J. Control Experiment with Visually Distinct Images

  • The control replaces the original with an unrelated image and measures whether the model notices the change and refuses or revises its answer.Because the unrelated image has no meaningful answer for the original query, the measure is detection rate rather than task accuracy.Source: J. Control Experiment with Visually… · p. 18
  • Detection rate requires noticing the image change, whereas task accuracy also requires attending to the new image and correctly solving the problem.Near-perfect Multi-turn detection on unrelated images therefore does not establish correct task-level answers under those conditions.Source: J. Control Experiment with Visually… · pp. 18–19
  • Even with completely unrelated images, Thinking models under Probe often fail to detect the change, while the Probe–Multi-turn gap persists.Qwen3-VL-235B-A22B-Thinking has a 35.6% detection rate under Probe, compared with 98.3% under Multi-turn; detection rates measure noticing a change, not solving the task.Source: J. Control Experiment with Visually… · p. 18, Table 17 · p. 19

K. Evaluation on Closed-Source Models

On Gemini 3 Flash Preview, Multi-turn accuracy remains close to baseline, supporting generalization of the recovery effect to a closed-source model. API constraints prevent testing whether self-generated reflection re-grounds vision during continuous generation.

K. Evaluation on Closed-Source Models

  • The main experiments focus on open-source VLMs because Probe requires inserting content inside an assistant response and closed-source APIs do not support it.The mechanistic analyses also require internal attention weights, and some models return thought summaries rather than raw reasoning chains.Source: K. Evaluation on Closed-Source Models · p. 19
  • For Gemini, the returned thought summary serves as a proxy for the initial reasoning before a new user turn supplies the swapped image and re-examination instruction.The API evaluation omits Probe because it is not implementable through the API.Source: K. Evaluation on Closed-Source Models · p. 19
  • Gemini 3 Flash Preview retains 86.1% accuracy under Multi-turn, close to its 88.5% baseline.This supports generalization of Multi-turn recovery to this closed-source model, but the Probe failure cannot be tested through APIs that disallow mid-response intervention.Source: K. Evaluation on Closed-Source Models · p. 19

Related papers

Loading 2605.15864v2…