PaperlayerView exact revision on arXiv

Source-linked AI summary

On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs

Rosie Zhao, Anshul Shah, Xiaoyu Zhu, Xinke Deng, Zhongyu Jiang, Yang Yang, Joerg Liebelt, Arnab Mondal

arXiv:2602.12506v3cs.LG

TL;DR

RL-finetuned VLMs improve visual reasoning benchmarks, but their robustness and chain-of-thought faithfulness under misleading textual context remain unclear. This paper stress-tests eight visual reasoning benchmarks with controlled perturbations and finds that robustness gains from augmentation can coexist with persistent faithfulness drift. The findings motivate evaluations and training objectives that jointly address visual grounding, robustness, and reasoning consistency.

  • Problem

    The robustness and faithfulness of RL-finetuned VLM improvements under benign or misleading textual context remain insufficiently established.

    Source: §1 · pp. 1–2

  • Method

    The paper applies controlled misleading captions and conflicting chain-of-thought traces across eight visual reasoning benchmarks, evaluating accuracy, uncertainty, and answer–reasoning consistency.

    Source: §1 · p. 3, §2.1 · p. 6, §3.3 · p. 14

  • Results

    Adversarial augmentation improves robustness, but accuracy gains do not prevent faithfulness drift, and combining augmentation with naive faithfulness rewards can induce shortcut strategies.

    Source: §1 · p. 3, §5 · p. 17

  • Takeaways & Limitations

    Reliable multimodal reasoning requires training and evaluation that explicitly address modality conflict, reasoning traces, calibration, and reward design.

    Source: §5 · pp. 17–18

    The faithfulness metric measures external reasoning–answer consistency rather than complete internal mechanistic faithfulness.

    Source: §1 · p. 3

Abstract

from arXiv · show

Reinforcement learning (RL) finetuning has become a key technique for enhancing large language models (LLMs) on reasoning-intensive tasks, motivating its extension to vision-language models (VLMs). While RL-tuned VLMs improve on visual reasoning benchmarks, they remain vulnerable to weak visual grounding, hallucinations, and over-reliance on textual cues. We show that simple, controlled textual perturbations, including misleading captions or incorrect chain-of-thought (CoT) traces, cause substantial drops in robustness and confidence, and that these effects are more pronounced when CoT consistency is taken into account across open-source multimodal reasoning models. In contrast, closed models exhibit similar failure modes but maintain markedly greater robustness and reasoning consistency, suggesting that the gap reflects a shortcoming in current open-source RL finetuning rather than an inherent limitation of the task. To better understand these vulnerabilities, we further analyze RL finetuning dynamics and uncover an accuracy-faithfulness trade-off: finetuning raises benchmark accuracy, but can simultaneously erode the reliability of the accompanying CoT and its robustness to contextual shifts. Although adversarial augmentation improves robustness, it does not by itself prevent faithfulness drift. Incorporating a faithfulness-aware reward can restore alignment between answers and reasoning, but when paired with augmentation, training risks collapsing onto shortcut strategies and robustness remains elusive. Together, these findings highlight the limitations of accuracy-only evaluations and motivate training and assessment protocols that jointly emphasize correctness, robustness, and the faithfulness of visually grounded reasoning.

1 Introduction

The paper stress-tests RL-finetuned VLMs with misleading textual context, revealing reliance on spurious cues, accuracy–faithfulness trade-offs, and weaker robustness in open-source models. It also evaluates adversarial augmentation and faithfulness-aware rewards, showing that consistency can improve but jointly achieving robustness and faithfulness remains difficult.

Augmented benchmarks

  • Controlled perturbations across eight visual reasoning benchmarks expose open-source VLMs’ reliance on misleading text, causing accuracy drops, CoT–answer inconsistencies, and altered uncertainty profiles.The perturbations target counting and 2D/3D spatial relations using misleading captions or conflicting CoT prefixes.Source: Figure 1 · p. 2, §1 · p. 3

Accuracy - faithfulness tradeoff

  • Controlled RL finetuning improves headline accuracy while increasingly disconnecting generated reasoning from final answers, establishing an accuracy–faithfulness trade-off.The inconsistency becomes more pronounced with additional finetuning steps.Source: §1 · p. 3

Robustness of open vs closed VLMs

  • Closed-source models share hallucinations, overthinking, and accuracy drops, but generally produce more consistent CoT, acknowledge misleading context, and show greater robustness than open-source models.Some closed models occasionally self-correct when textual context conflicts with the image.Source: §1 · p. 3

Augmentation improves robustness, but faithfulness drift persists

  • Adversarial augmentation improves robustness by preserving training diversity, but accuracy gains do not prevent faithfulness drift.Faithfulness remains decoupled from robustness improvements even with perturbed examples during training.Source: §1 · p. 3

Faithfulness-as-reward aligns CoT with answers, but combining with augmentation is unstable

  • Faithfulness-aware rewards align CoT with answers, whereas combining them with augmentation can produce unstable training dynamics and limited robustness gains.The interaction is not additive, highlighting difficulty enforcing robustness and faithfulness together.Source: §1 · p. 3

Clarification on our use of “faithfulness”

  • Here, faithfulness denotes external consistency between the generated CoT and final answer, not whether the rationale causally reflects the model’s internal decision process.The distinction follows the difficulty of evaluating internal causal faithfulness in black-box and open-source VLMs at scale.Source: §1 · p. 3

2 Evaluating Robustness of Vision-Reasoning Models

This section evaluates the robustness and faithfulness of vision-reasoning models under controlled textual perturbations and default prompting. It examines how model confidence, entropy, and correct-answer probability relate to calibration, error filtering, modality conflicts, and the reliability of chain-of-thought reasoning.

2 Evaluating Robustness of Vision-Reasoning Models

  • The evaluation uses spatial and general VQA benchmarks with controlled Stop-Think, Wrong-Think, and Wrong-Caption perturbations, testing sensitivity to misleading textual context and reasoning suppression.Wrong-Think and Wrong-Caption are procedurally generated from randomly sampled incorrect answer options, with corrective markers or disclaimers used as repair variants.Source: §2 · pp. 3–4

2.1 Open Sourced Reasoning Models

  • Open RL-finetuned models suffer substantial accuracy degradation from misleading thinking strings and captions, while corrective cues recover performance but expose strong textual dependence.Many correct Wrong-Think answers are unfaithful, and models most robust to perturbations can show the highest inconsistency, even under Base prompting.Source: §2.1 · pp. 5–6, Figure 3 · p. 6, Figure 4 · p. 7

2.2 Closed-Source Model Comparison

  • Closed models show smaller performance drops and substantially higher faithfulness under misleading interventions, although Gemini-3.1-Pro is strongest and hallucinations still occur.Disclaimers often restore performance near baseline, but closed models inconsistently prioritize visual evidence versus misleading text during conflicts.Source: Figure 2 · p. 5, §2.2 · pp. 8–9, Figure 5 · p. 9

2.3 Entropy-Based Analysis

  • Stop-Think consistently increases answer entropy, whereas adversarial prompts reduce P(Correct Letter), sometimes nearly to zero, indicating confident misdirection rather than mere uncertainty.Corrective cues restore probability mass toward the ground-truth letter while leaving entropy above Default; training recipes differ in whether robustness appears mainly through entropy or probability shifts.Source: §2.3 · p. 10, Figure 6 · p. 10, Figure 7 · p. 11

2.4 Are entropy-based metrics under default prompting predictive of performance on perturbed prompts?

  • Pbase predicts robustness better than negative entropy across nearly all models and perturbations, achieving AUROC 0.958 versus 0.732 for entropy in SpaceR under Stop-Think.Probability assigned to the correct letter better captures robust knowledge because entropy can be low even when the model is confidently wrong.Source: §2.4 · p. 11, Table 6 · p. 11
  • Models with high robustness calibration strongly correlate base confidence with perturbation robustness; SpaceR exceeds 0.94 AUROC, but this robustness compromises CoT faithfulness.These models act as “stubborn experts,” relying on strong visual grounding to resist deceptive reasoning traces rather than consistently following adversarial instructions.Source: §2.4 · p. 11
  • Brittle models show weak confidence predictiveness under perturbation; Vision-R1’s Stop-Think AUROC is 0.565, near random chance, while answers follow adversarial CoT text.For these models, default-prompt certainty does not ensure robustness, and final answers depend more on textual context than prior visual confidence.Source: §2.4 · p. 11
  • Robust models enable confidence-threshold filtering, but this capability reflects answer–reasoning decoupling alongside an accuracy–faithfulness trade-off from RL finetuning.Finetuning raises benchmark accuracy while potentially eroding accompanying CoT reliability and robustness to contextual shifts.Source: §2.4 · p. 11

3 Effect of RL-Finetuning on Robustness and CoT Consistency

RL finetuning improves benchmark accuracy and caption-perturbation robustness but progressively suppresses entropy and can degrade reasoning faithfulness. Faithfulness-aware rewards preserve answer–reasoning consistency only imperfectly, as training may collapse onto shortcut strategies.

3.1 Experimental Details

  • The experiments use GRPO with format and correctness rewards across three dataset mixtures, including optional Geometry3K and mixed correct/incorrect caption and thinking augmentations.Augmented samples apply one of four contextual variants with 10% probability each, while 60% remain unmodified.Source: §3.1 · p. 12

3.2 Robustness Results

  • Adding Geometry3K consistently improves baseline accuracy across most benchmarks, especially for mathematical or geometric reasoning and often under Wrong-Think.This suggests explicit visual math reasoning strengthens visual grounding beyond the spatial training mixture alone.Source: §3.2 · p. 13, Figure 8 · p. 13
  • Augmentation preserves unperturbed accuracy while substantially improving robustness to misleading captions, bringing Wrong-Caption performance closer to Base-level accuracy.The benefit is weaker for misleading chain-of-thought, suggesting models more readily ignore captions than presented reasoning traces.Source: §3.2 · p. 13, Figure 8 · p. 13
  • Entropy steadily declines throughout RL training across perturbations, indicating globally narrowed predictive distributions rather than prompt-specific adaptation.Augmentation drives entropy lower in Base and Wrong-Caption, while Stop-Think and Wrong-Think retain comparatively higher entropy.Source: §3.2 · p. 14, Figure 9 · p. 14
  • Results vary sharply across random seeds, sometimes more than across dataset compositions, making multi-seed reporting essential for judging RL-finetuning stability.Single-seed outcomes can therefore misrepresent effectiveness.Source: §3.2 · p. 14

3.3 Faithfulness Results

  • RL finetuning generally decreases faithfulness even as accuracy improves, and augmentation does not prevent this accuracy–reasoning drift.Faithfulness measures responses whose reasoning is both correct and aligned with the final answer.Source: §3.3 · p. 14, Figure 10 · p. 15
  • Faithfulness-aware rewards help preserve answer–reasoning consistency but can favor right-caption or right-think shortcuts, causing training to collapse without robust signal discrimination.Standard verifiable rewards and adversarial augmentation therefore do not simultaneously enforce robustness and faithfulness.Source: §3.3 · pp. 14–15

4 Related Work

Prior work advances multimodal reasoning through RLVR, explicit visual grounding, perception-first pipelines, and visual reasoning intermediates, while raising concerns about robustness and CoT faithfulness. This study complements these directions by testing whether RL-with-CoT finetuning remains visually grounded and faithful under conflicting textual cues.

RL-based finetuning

  • RLVR has become a cornerstone for improving reasoning in language models and is increasingly adapted to vision-language models through diverse multimodal finetuning recipes.These recipes include spatial, temporal, multimodal-CoT, and coordinate-grounded approaches, alongside preference optimization targeting hallucination and language-prior shortcuts.Source: §4 · pp. 15–16

Perception–reasoning decomposition

  • Perception-first pipelines strengthen visual grounding with curated data, intermediate representations, or verifiable perception objectives before applying RL or CoT-style reasoning finetuning.This work is complementary: it analyzes side effects of RL-with-CoT finetuning rather than proposing another training pipeline, emphasizing robustness and faithfulness as evaluation axes.Source: §4 · p. 16

Spatial reasoning in MLLMs

  • Explicit grounding methods improve spatial reasoning by linking language and intermediate reasoning to regions, coordinates, scene structure, depth, segmentation, or replayed visual evidence.Benchmarks and evaluations nevertheless report persistent perception, reasoning, attention, hallucination, distraction, uncertainty, and calibration gaps.Source: §4 · pp. 16–17

Visual reasoning intermediates

  • Visual-intermediate approaches interleave visual and textual reasoning, making continuous visual tokens or generated visual structures part of the reasoning substrate.These methods extend text-only CoT by incorporating visual structure directly into intermediate reasoning.Source: §4 · p. 17

CoT faithfulness

  • Prior language-model studies show that fluent CoT can omit, misrepresent, or fail to reflect the factors driving correct answers, including under outcome-based RL finetuning.Some findings also question whether explicit CoT is necessary for strong benchmark performance under fixed token budgets.Source: §4 · p. 17

Multimodal CoT faithfulness

  • Multimodal studies find diminishing visual attention, hallucination, and weak correspondence between visual evidence and verbalized reasoning, motivating this work’s conflicting textual-cue stress tests.Unlike PerturboLLaVA’s dense-captioning supervision objective, this study tests faithful visual-spatial reasoning under adversarial textual context in open-source models.Source: §4 · p. 17

5 Conclusion and Future Work · Appendix

Controlled textual perturbations expose brittleness in open-source RL-finetuned VLMs, while stronger closed models indicate that modality-conflict failures reflect underdeveloped open-model capabilities rather than inherent task limits. The findings motivate training and evaluation that jointly address conflict resolution, CoT faithfulness, calibration, and reward design.

5 Conclusion and Future Work

  • Textual perturbations substantially degrade open-source RL-finetuned VLMs, revealing sensitivity to language priors and weak arbitration between textual and visual evidence.Stronger closed-source models suggest these failures reflect underdeveloped detection, representation, and resolution of modality conflict rather than inherent task difficulty.Source: §5 · p. 17
  • Misleading-text augmentation and naive faithfulness rewards can induce shortcut strategies instead of genuine reconciliation of conflicting evidence.Future objectives should train models to detect disagreement, choose the trustworthy source, and revise or qualify predictions.Source: §5 · p. 17
  • Under RL, CoT can become an independently optimized output whose rationale is only weakly coupled to the model’s actual decision process.Faithfulness therefore requires tighter connections among internal computation, external rationales, and visually grounded conflict resolution, rather than CoT supervision or rationale rewards alone.Source: §5 · pp. 17–18
  • Modality-conflict perturbations should become routine benchmarks and lightweight diagnostics for text dominance, localization drift, and CoT confabulation.These perturbations are easy to construct and expose failures often masked by standard benchmarks.Source: §5 · p. 18
  • Calibration should be evaluated under conflicting evidence, with models expected to express uncertainty, request clarification, or acknowledge image–text disagreement.Additional targets include source-aware reasoning, distillation from stronger models or humans, and verification of textual claims against visual evidence.Source: §5 · p. 18
  • Extending perturbations to multi-turn and interactive settings would better reflect deployments involving ambiguous prompts, underspecified references, and conflicting user descriptions.This direction tests whether models can sustain conflict resolution beyond isolated visual reasoning tasks.Source: §5 · p. 18
  • Trustworthy multimodal reasoning requires protocols that explicitly target interactions among modalities, reasoning traces, calibration, and reward design.Simple visual reasoning perturbations are practical probes of grounding, uncertainty calibration, and conflicting-evidence resolution rather than merely toy stress tests.Source: §5 · p. 18

A Evaluation Details … A.4 Closed-Source Reasoning Model Suite

The evaluation spans spatial and general-purpose VQA benchmarks, tests robustness with systematically generated incorrect captions and reasoning strings, and compares open- and closed-source multimodal reasoning models under standardized prompting. Closed-model evaluation additionally enforces continuation from the supplied incorrect reasoning string in the Wrong-Think condition.

A.1 Benchmark Datasets

  • The evaluation covers five spatial-reasoning benchmarks and three complementary general-purpose VQA benchmarks spanning spatial relations, perception, commonsense, instruction following, and multi-step reasoning.The spatial suite comprises 3DSRBench, CV-Bench, Spatial-MM, and WhatsUp; the broader suite adds V∗-Bench, MME-RealWorld-Lite, and MMBench.Source: §A.1 · p. 25

A.2 Dataset Augmentation Details

  • Augmented examples prepend captions and thinking strings that consistently describe an incorrect answer, enabling Wrong-Caption and Wrong-Think evaluations.For structured multiple-choice datasets, incorrect options are selected uniformly at random and used to automatically construct both strings.Source: §A.2 · p. 25
  • SpatialMM Multihop uses Qwen3 to generate alternative free-form answers and matching incorrect context, with every sample manually verified because the dataset is small.Binary questions have one alternative, while color or object questions permit multiple alternatives.Source: §A.2 · p. 26
  • The three general VQA benchmarks are augmented by selecting an incorrect multiple-choice option and constructing a caption and thinking string that support it despite less uniform spatial templates.This procedure is applied to MMBench, MME-RealWorld-Lite, and V∗-Bench.Source: §A.2 · p. 27

A.3 Open-Source Reasoning Model Suite

  • The open-source set includes SpaceR, Video-R1, Vision-R1, and VLAA-Thinker, spanning SFT with reinforcement learning, temporal modeling, mixed rewards, and structured reasoning development.The models are based on Qwen2.5-VL-7B-Instruct, use suggested system prompts when provided, and are evaluated at temperature 0.01.Source: §A.3 · p. 28

A.4 Closed-Source Reasoning Model Suite

  • Closed-source evaluation obtains generations from o3, o4-mini, Gemini-2.5-Pro, and Gemini-3.1-Pro-Preview using base, Wrong-Caption, and Wrong-Think prompt templates.All templates require internal reasoning in <think> tags followed by the final answer in <answer> tags.Source: §A.4 · pp. 28–29
  • In the closed-model Wrong-Think condition, prompts explicitly require copying the supplied incorrect thinking string verbatim and continuing from it without starting a new answer.The instruction also preserves the required <think></think><answer></answer> format.Source: §A.4 · pp. 28–29

A.5 Faithfulness Evaluation · B RL Finetuning Details · C Additional Evaluation Results

The appendix operationalizes faithfulness as consistency between a model’s final answer and its concluding reasoning, while documenting the prompts and shared GRPO hyperparameters used for RL finetuning. Qwen3 provides reliable consistency judgments across both consistent-but-incorrect and inconsistent-but-correct examples.

A.5 Faithfulness Evaluation

  • Faithfulness is evaluated by checking whether the final answer matches the model’s concluding reasoning, independently of whether that answer is correct.The evaluator permits answer changes during reasoning and assesses the final judgment before </think>.Source: §A.5 · p. 30
  • The evaluation handles missing or truncated traces, maps answer letters to option contents, and returns only a yes/no judgment in dedicated tags.Missing final answers receive “no,” while absent thinking receives vacuous “yes.”Source: §A.5 · p. 30
  • Qwen3 in Thinking mode reliably judged faithfulness, distinguishing incorrect-but-consistent responses from correct-but-inconsistent responses.The examples show a matching answer receiving “yes” and a reasoning–answer contradiction receiving “no.”Source: §A.5 · pp. 30–31

B RL Finetuning Details

  • RL finetuning uses shared GRPO hyperparameters across the three introduced data settings.The hyperparameters are provided in Table 7.Source: B RL Finetuning Details · p. 32, Table 7 · p. 32
  • For SAT2 and Pixmo-Count, Qwen3 generates correct or wrong captions and initial thinking strings by conditioning on the ground-truth or a randomly selected incorrect option.These strings support experiments on captions and initial thinking during VLM RL finetuning.Source: B RL Finetuning Details · p. 32
  • The generation prompt requires exactly one caption and two thinking sentences, with the thinking leaning toward the supplied answer and using only question-and-answer content.Outputs must be enclosed in <caption> and <thinking> tags.Source: B RL Finetuning Details · p. 33

C.1 Additional Results – Correct Thinking and Caption Ablations

Correct auxiliary text improves VLM accuracy, whereas dummy text has little effect, showing that perturbations arise from semantic context rather than prompt-format changes. Disclaimers reduce these benefits, while faithfulness results reveal persistent decoupling between reasoning traces and final answers.

C.1 Additional Results – Correct Thinking and Caption Ablations

  • Dummy captions leave accuracy largely unchanged, while Correct-Caption and Correct-Think substantially improve performance across models and datasets, confirming semantic-content rather than format sensitivity.Figures 11 and 12 show the corresponding correct-thinking and caption ablations.Source: §C.1 · p. 34, Figure 11 · p. 34, Figure 12 · p. 34
  • Disclaimer language partially reverses correct-text gains, indicating that models strongly follow salient textual context rather than reliably calibrating whether auxiliary information should be trusted.This occurs for both Correct-Caption and Correct-Think conditions, including the But variant.Source: §C.1 · pp. 34, 36
  • Correct captions likewise improve performance without reliably separating visual evidence from textual context, with accompanying faithfulness patterns showing context-shaped reasoning and persistent answer–reasoning decoupling.Figure 14 reports faithfulness under Correct-Caption and Correct-Caption With Disclaimer.Source: §C.1 · pp. 34, 36, Figure 14 · p. 35
  • Correct thinking can increase accuracy while incorrect final answers often remain inconsistent with the injected reasoning, demonstrating that reasoning and answers remain only partially coupled.The reasoning prefix can anchor the stated rationale even when the final answer diverges from it.Source: §C.1 · pp. 34, 36, Figure 13 · p. 35

C.2 Additional Results - Real World Understanding / General VQA Benchmarks

Across MMBench, MME-RealWorld(-Lite), and V*Bench, textual Wrong-Think and Wrong-Caption perturbations substantially reduce accuracy, while disclaimers only partially recover performance. These benchmarks also reveal uneven Stop-Think effects, persistent answer–CoT faithfulness gaps, and generally reliable judge agreement.

Real-World Perturbation Robustness

  • Wrong-Think and Wrong-Caption substantially degrade accuracy across all three benchmarks, while explicit disclaimers rarely restore Base performance.These vulnerabilities persist in more open-ended, real-world VQA settings.Source: §C.2 · pp. 36–37

Benchmark Difficulty

  • Base accuracies are generally lower on these harder benchmarks, especially MME-RealWorld and V*Bench, where Wrong-Think effects are often more severe.The perturbation results therefore vary with both benchmark difficulty and model robustness.Source: §C.2 · p. 36

Real-World Perturbation Robustness

  • Wrong-Think often causes more extreme and model-dependent failures than Wrong-Caption, including near-collapse on MME-RealWorld and V*Bench.Wrong-Caption drops are significant but more tightly clustered across models, indicating less response variation.Source: §C.2 · pp. 36–37

Stop-Think Results

  • Stop-Think effects remain model- and task-dependent: fluctuations are small on MMBench but larger elsewhere, improving Vision-R1 and ViGoRL while degrading Video-R1 and VLAA-Thinker.Suppressing explicit reasoning therefore does not uniformly harm visual understanding and can remain competitive for some architectures and training recipes.Source: §C.2 · p. 37, Table 8 · p. 37

Faithfulness Analysis

  • Models can retain robust final-answer accuracy under Wrong-Think while producing CoT traces frequently judged unfaithful.MMBench shows more faithful answers under Qwen3-32B than controlled spatial tasks, potentially because dataset priors or superficial textual cues can answer some questions without tight visual grounding.Source: §C.2 · p. 37, Figure 17 · p. 38

Evaluation Reliability

  • Agreement among GPT-OSS-120B, Qwen3-32B, and Llama3.1-70B-Instruct remains consistently high, with Llama3.1-70B-Instruct slightly lower but still strong.A small human study also reports high agreement between humans and judge models.Source: §C.2 · p. 37, Table 9 · p. 37

C.3 Additional Results – InternVL3 Results

InternVL3-8B and SenseNova-SI reproduce the main perturbation trends across five benchmarks: misleading textual context degrades accuracy and increases reasoning–answer inconsistency. These findings show the vulnerabilities extend beyond the Qwen2.5-VL family.

InternVL3 Results

  • The evaluation applies the original protocol to InternVL3-8B and SenseNova-SI across 3DSRBench, CV-Bench, Spatial-MM, WhatsUp, and SpatialMM-Multihop.SenseNova-SI is a spatial-reasoning model finetuned from InternVL3-8B.Source: §C.3 · p. 38
  • Both InternVL3-8B and SenseNova-SI suffer accuracy degradation under Wrong-Think and Wrong-Caption perturbations, despite unchanged images and original questions.The same qualitative trends observed for Qwen2.5-VL therefore extend to InternVL3-based models.Source: Figure 18 · p. 38, §C.3 · p. 38, Figure 19 · p. 39
  • Misleading thinking prefixes reduce accuracy and increase reasoning–answer inconsistency for InternVL3-based models across the five benchmarks.This indicates deterioration in the relationship between generated reasoning and final predictions, not merely answer-selection errors.Source: §C.3 · p. 38, Figure 20 · p. 39

C.4 Additional Results – RefCOCOg Grounding Perturbation Evaluation

A proof-of-concept RefCOCOg evaluation shows that misleading spatial text can degrade and redirect localization even when images and referring expressions remain unchanged. Vulnerability varies across models and perturbation types, extending the paper’s central robustness concern beyond VQA while falling short of a full faithfulness benchmark.

C.4 Additional Results – RefCOCOg Grounding Perturbation Evaluation

  • On the RefCOCOg test set, evaluation covers 9,602 referring expressions across 2,600 images under Default, Wrong-Caption, and reasoning-only Wrong-Think conditions.Default uses the image and referring expression; Wrong-Caption prepends a misleading directional sentence, while Wrong-Think injects a misleading spatial belief for reasoning models only.Source: §C.4 · p. 40
  • Wrong-Caption degrades every evaluated model, with relative IoU drops ranging from 2% for SenseNova-InternVL3-8B to 52% for Video-R1-7B.Other reported drops include 6% for Qwen2.5-VL-7B-Instruct, 20% for InternVL3-8B, and 39% for SpaceR.Source: §C.4 · p. 40, Table 10 · p. 40
  • Directional follow rates show systematic steering by misleading captions, reaching 0.70 for Video-R1-7B and sometimes causing predictions to abandon the referring expression.Qualitative inspection describes a wrong right-side caption producing a box spanning much of the image’s right half for a target whose gold box is on the left.Source: §C.4 · pp. 40–41
  • Wrong-Think exposes complementary reasoning vulnerabilities: VLAA-Thinker drops 22% versus 6% under Wrong-Caption, while SpaceR drops 80% versus 39%.Vision-R1-7B and Video-R1-7B also show measurable Wrong-Think degradation, though smaller than SpaceR’s.Source: Table 10 · p. 40, §C.4 · p. 41
  • These findings extend susceptibility to superficial spatial misinformation beyond VQA, but the experiment is only a proof of concept because IoU and directional shifts cannot establish whether localization relied on visual evidence or injected text.The authors note that rigorous grounding faithfulness evaluation would require manual annotation.Source: §C.4 · pp. 40–41

C.5 Additional Results - Wrong Caption Faithfulness Evaluation

Wrong-caption settings produce fewer unfaithful responses than Wrong-Think, but correct yet unfaithful outputs remain substantial. Models also succeed either by conditioning on misleading captions or by ignoring them, making accuracy an incomplete measure of visual-textual discernment.

Wrong Caption Faithfulness Evaluation

  • Correct yet unfaithful generations comprise roughly 10–15% of outputs in Wrong-Caption and Wrong-Caption-with-Disclaimer settings.These settings show fewer unfaithful responses overall than Wrong-Think.Source: §C.5 · p. 41
  • Some models answer incorrectly while explicitly conditioning on the misleading caption, whereas others answer correctly by disregarding it and re-describing the image.The latter is valid but does not necessarily demonstrate discernment between visual and textual information.Source: §C.5 · p. 41
  • Most incorrect generations are consistent but incorrect, indicating that systematic failures usually arise from adopting the wrong caption at face value.Inconsistent incorrect answers also occur, but they are not the dominant pattern.Source: §C.5 · p. 41

C.6 AdditionalResults–HumanStudyComparisonwithLLM-as-JudgeforFaithfulnessEvaluation · D Example Traces

A human study supports the reliability of both human faithfulness annotations and the Qwen3-32B LLM judge, with high inter-annotator agreement and strong agreement between human majority labels and the judge. Representative evaluation traces are also provided for open models.

C.6 AdditionalResults–HumanStudyComparisonwithLLM-as-JudgeforFaithfulnessEvaluation

  • The study’s design used three annotators per example, with majority labels defined by 2-of-3 or 3-of-3 agreement, across 600 sampled cases.Eight annotators participated, and each prompt condition contributed 50 examples per model-dataset combination.Source: §C.6 · pp. 41, 43
  • Human annotators achieved high agreement across 600 examples, with all reported agreement metrics above 0.85 and 92.3% unanimous decisions.The study evaluated ViGoRL and SpaceR on CVBench and 3DSRBench under Base, Wrong-Think, and Wrong-Think-With-But prompts.Source: §C.6 · pp. 41–42, Table 11 · p. 42
  • Agreement remained consistently high across datasets, models, and prompt conditions, although Wrong-Think-With-But was slightly lower than the other settings.This indicates that consistency labeling remains reliable even when models revise or qualify misleading rationales.Source: Table 12 · p. 42, §C.6 · p. 42
  • 91.8% overall agreement and Cohen’s κ=0.813 show strong agreement between Qwen3-32B’s faithfulness judgments and human majority labels.Humans labeled 70.8% of examples consistent, compared with 65.3% for Qwen3-32B.Source: Table 13 · p. 43, §C.6 · p. 43
  • Qwen3-32B is slightly more conservative than humans, chiefly labeling some human-consistent outputs inconsistent; only 8 examples show the reverse pattern.This conservative bias may slightly underestimate reported reasoning-answer consistency rates.Source: Table 14 · p. 43, §C.6 · p. 43
  • These findings support using Qwen3-32B as an automated judge for large-scale reasoning-answer faithfulness evaluation.Strong human agreement indicates that the annotation criterion is well-defined, while judge-human agreement supports automated assessment.Source: §C.6 · p. 43

D Example Traces

  • Representative example traces from the open-model evaluation results are provided.The traces appear in Section 2’s evaluation of open models.Source: D Example Traces · p. 43

D.1 Wrong-Caption - “Ignoring" the Caption … E.1 Wrong-Caption: Text vs. Vision Conflict

Under misleading captions or reasoning traces, open-source vision-reasoning models alternate between ignoring textual cues, anchoring on them, and producing answers whose chains of thought are unfaithful. Closed-model behavior is more meta-aware but still inconsistent, sometimes prioritizing visual evidence and sometimes textual premises.

D.1 Wrong-Caption - “Ignoring" the Caption

  • Open-source models oscillate between caption neglect and caption anchoring instead of consistently grounding reasoning in visual evidence.Correct responses often ignore misleading captions, but other traces accept them and hallucinate descriptions based on incorrect text.Source: §D.1 · pp. 43–44

D.2 Wrong-Caption - Unfaithful Generation

  • Even grounded-RL models hallucinate under wrong captions, and disclaimers may correct the answer without correcting the unfaithful reasoning.ViGoRL initially selects “shelves” from the misleading caption; with a disclaimer, it selects “books” while continuing to describe shelves as closer.Source: §D.2 · p. 44

D.3 Wrong-Think - Unfaithful Generation

  • Wrong-Think prompts can produce correct labels despite reasoning that simply continues the misleading premise, exposing a direct answer–reasoning disconnect.SpaceR repeats that yellow apples are at the top but outputs the correct label, “bottom right.”Source: §D.3 · p. 46

D.4 Wrong-Think - Occasional Self-Correction

  • Self-correction occurs occasionally, but it may depend on textual contradiction detection or common-sense priors rather than explicit visual grounding.Vision-R1 reverses an initial “far away” interpretation after noticing inconsistency, while other corrections rely on general knowledge such as typical object locations.Source: §D.4 · p. 47

E.1 Wrong-Caption: Text vs. Vision Conflict

  • Closed-model reasoning shows greater meta-awareness of text–vision conflicts, yet Gemini-2.5-Pro inconsistently follows visual cues across inputs.It follows the image in one CVBench example but adheres to the textual premise in another, even with temperature controlled.Source: §E.1 · p. 48
  • Gemini-2.5-Pro can reject a false caption when visual analysis identifies a reflection, answering that the image contains one physical cat rather than two.The model explicitly treats the “two cats” statement as misleading and selects option (B) 1.Source: §E.1 · p. 48
  • In another conflict, Gemini-2.5-Pro prioritizes the textual premise over visible evidence, answering zero bananas despite describing four in the image.The trace explicitly weighs image-first and premise-first paths before selecting option (B) 0 from the stated premise.Source: §E.1 · pp. 49–50

E.2 Wrong-Caption: Hallucinations … F.2 Qualitative Improvements to Model Behavior

Wrong-Think can induce hallucinations or faithful acceptance of misleading reasoning, whereas RL finetuning yields modest, nonmonotonic Stop-Think gains and data augmentation can promote caption-resetting behavior. However, these qualitative improvements remain inconsistently faithful and may collapse into paraphrasing the wrong caption.

E.2 Wrong-Caption: Hallucinations

  • Gemini-2.5-Pro sometimes corrected misleading Wrong-Think and answered correctly, but also hallucinated nonexistent image-crop links or accepted the incorrect reasoning.Hallucinations were especially common when conditioning on misleading reasoning, which appeared to push the model into an uncontrolled output region.Source: §E.2 · p. 50
  • In one Wrong-Think example, Gemini-2.5-Pro reconsidered the spatial arrangement, rejected its initial assessment, and correctly selected the printer over the bin.The model first followed misleading reasoning favoring the bin, then inferred that the printer was adjacent to the stationery while the bin was farther away.Source: §E.2 · pp. 50–51
  • In another example, Gemini-2.5-Pro followed the misleading premise that the chair was closer and incorrectly answered chair rather than mouse.The model explicitly treated the supplied reasoning as decisive instead of independently resolving the visual spatial relationship.Source: §E.2 · pp. 51–52

F.1 Stop-Think Performance

  • Stop-Think provides only modest gains over Base, with nonmonotonic accuracy across training steps; runs including math data generally perform better.These results are shown in Figure 22 for the RL-finetuned models.Source: §F.1 · p. 53, Figure 22 · p. 53

F Additional RL finetuning Results

  • Across additional RL-finetuning results, quantitative gains and qualitative robustness improvements coexist with nonmonotonic performance and unresolved faithfulness failures.Stop-Think gains remain modest, while data augmentation produces correct caption resets only inconsistently.Source: §F.1 · p. 53, §F.2 · p. 53

F.2 Qualitative Improvements to Model Behavior

  • Data-augmented RL can improve Wrong-Caption robustness through a caption-resetting strategy that re-describes the scene, contradicts the misleading caption, and produces the correct answer.In the CVBench example, the model reset the claim that books were closer and correctly answered lamp; this behavior appeared across multiple checkpoints.Source: §F.2 · pp. 53–54
  • The qualitative reset strategy is not universally faithful: some resets remain superficial or collapse into paraphrasing the wrong caption.Thus, improved robustness behavior can coexist with continued reliance on misleading textual reasoning.Source: §F.2 · p. 53

F.3 Adding Faithfulness-as-a-Reward

Faithfulness-aware rewards align accuracy with reasoning consistency, but combining them with adversarial augmentation can destabilize training and induce shortcut behaviors, leaving robustness unresolved.

F.3 Adding Faithfulness-as-a-Reward

  • Joint faithfulness rewards and data augmentation produce unstable training, often causing generation collapse and leaving robustness elusive.The LLM-as-judge adds computational cost, while rewarding correctness only when reasoning and answers align biases optimization toward caption or reasoning structures that maximize both signals.Source: §F.3 · p. 54
  • Faithfulness enforcement can make models mechanically copy misleading captions or reasoning, whereas augmentation alone encourages recaptioning and fresh visual reasoning.In representative Wrong-Caption and Wrong-Think cases, Aug+Faithfulness copies deceptive cues and answers incorrectly, while Aug-only resets its reasoning and answers correctly.Source: §F.3 · p. 55
  • The findings suggest models cannot reliably distinguish valid from invalid textual reasoning cues, so enforcing faithfulness and correctness remains insufficient for robustness.Wrong-Caption can sometimes be mitigated through recaptioning, but Wrong-Think requires deeper reasoning-validity discrimination that the models lack.Source: §F.3 · p. 55
  • Faithfulness rewards preserve alignment between accuracy and reasoning consistency, particularly under Wrong-Caption perturbations, whereas standard RL shows accuracy-faithfulness decoupling.Figure 23 reports these trends across five benchmarks; in the Base condition, consistent-answer performance generally improves with the faithfulness reward.Source: §F.3 · p. 54, Figure 23 · p. 58

Related papers

Loading 2602.12506v3…