PaperlayerView exact revision on arXiv

Source-linked AI summary

RECAP: Looking Once Is Not Enough for Vision-Language Reasoning

Zhaolu Kang, Tailong Luo, Chenxin Li, Yingjie He, Zhenyu Yu, Fengyu Zhou, Jiachen Qian, Lei Wei, Shuang Chen, Jiachen Li, Eric Hanchen Jiang, Rongchao Zhang, Zhengtao Yao, Hoi Leong Lee, Guansu Wang, Kaiyue Zhou

arXiv:2610.30834v1cs.CV

TL;DR

Long-CoT VLMs can drift from image evidence, while final-answer rewards do not identify which intermediate steps should return to the image. RECAP uses detached visual-revisit traces to shape credit assignment, improving HallusionBench accuracy over GRPO across three backbones while preserving reasoning and text-only capability.

  • Problem

    Final-answer reinforcement-learning rewards indicate whether a rollout ended correctly but not which intermediate tokens should re-ground in image evidence.

    Source: §1 · p. 1

  • Method

    RECAP uses detached visual-revisit traces for revisit-conditioned advantage estimation, dynamic discounting, and gated value prediction without changing rollout sampling or inference-time computation.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    RECAP improves HallusionBench accuracy over GRPO by +3.8, +3.5, and +4.1 points on Qwen2.5-VL-7B, InternVL3-8B, and LLaVA-OneVision-7B, while preserving reasoning and text-only capability.

    Source: §4.2 · p. 7

  • Takeaways & Limitations

    Across three backbones, revisit peaks are predictive of correctness and supported by masking and injection interventions as a temporal grounding signal for long-chain reasoning.

    Source: §5 · p. 9

    RECAP relies on accessible, meaningful cross-modal attention traces, so benefits may be smaller for short answers, text-only tasks, or architectures where attention is a weak proxy for grounding.

    Source: §5 · p. 9

Abstract

from arXiv · show

Long chain-of-thought reasoning improves vision-language models (VLMs), but it also exposes a temporal grounding failure: as generation unfolds, models may drift from image evidence while reinforcement learning still provides only a final-answer reward. We study this gap through visual revisit, a temporally localized reactivation of image evidence during reasoning. Across three VLM backbones, revisit peaks are predictive of correctness and causally linked to performance: masking them degrades accuracy more than masking non-peaks, while injecting revisit-like peaks into failed rollouts partially restores correct answers. Control analyses show that this signal is not explained by raw attention strength, uncertainty, logit margin, or hidden-state norm. We propose RECAP, a GRPO-compatible training method that turns detached revisit traces into a credit-assignment signal. RECAP uses revisit-conditioned advantage estimation, dynamic discounting, and gated value prediction to propagate final rewards through visually important reasoning steps, without extra rewards, additional rollouts, or inference-time changes. Across Qwen2.5-VL-7B, InternVL3-8B, and LLaVA-OV-7B, RECAP consistently improves over GRPO and VPPO, yielding $+6.1$--$+6.5$ HallusionBench gains over the SFT base while preserving reasoning and text-only performance. Results on larger LoRA-tuned models, Chinese zero-shot benchmarks, visual perturbations, and human evaluation further demonstrate robust improvements in image-grounded reasoning.

1 Introduction

Long-chain vision-language reasoning can drift from image evidence, while final-answer rewards do not identify which intermediate steps should return to the image. The paper studies visual revisit as a temporal grounding signal and proposes RECAP to use revisit traces for credit assignment.

1 Introduction

  • Final-answer rewards indicate whether a rollout was correct but not which intermediate tokens should revisit the image or rely on language-internal reasoning.When later decisions again depend on visual evidence, failing to re-ground can produce fluent but visually unsupported conclusions [7].Source: §1 · p. 1
  • Correct rollouts show recurring visual-revisit peaks, and masking those peaks hurts accuracy more than masking random non-peaks; injecting peaks partially recovers failed rollouts.Control analyses distinguish revisit from raw attention mass, token entropy, logit margin, and hidden-state norm; the causal effect is concentrated in a small set of heads.Source: Figure 1 · p. 2, §1 · p. 2
  • RECAP uses detached revisit traces for advantage estimation, dynamic discounting, and gated value prediction without extra rewards, additional rollouts, or inference-time changes.The paper reports improvements on larger LoRA-tuned models, Chinese zero-shot benchmarks, visual perturbations, human evaluation, and composition with VPPO.Source: §1 · p. 2

2 Related Work

Prior work improves long-CoT VLM training and visual grounding through reward updates, static corrections, or decoding interventions. This paper instead focuses on when image evidence should re-enter a reasoning trajectory and how to credit those moments.

RL post-training for long-CoT VLM reasoning

  • GRPO and multimodal variants make verifiable-reward post-training practical, while VPPO, VAPO, TVC, λ-GRPO, and TreeAdv modify visual updates or credit assignment without explicitly modeling when reasoning returns to the image.RECAP targets this temporal grounding gap with detached revisit traces.Source: §2 · p. 3

Visual hallucination and grounding during generation

  • Grounding mitigations address fluent visual drift through inference-time interventions or preference/data-level supervision, but mostly treat grounding as static correction or decoding-time intervention.This work asks whether revisit moments in a model's trajectory predict and causally support correctness.Source: §2 · p. 3

Revisit-aware credit assignment and cross-modal mechanisms

  • Because raw attention varies with token type, position, and layer, RECAP defines revisit as a local rise above a trajectory-specific baseline and uses it to shape credit assignment.Dynamic discounting can propagate credit retroactively through future revisit peaks; gated value prediction addresses a return-residual bias–variance trade-off.Source: §2 · p. 3

3 The RECAP Framework

RECAP retains GRPO's rollout and policy-update structure while using detached visual-revisit traces to shape estimation, reward propagation, and value prediction. The framework also defines a revisit-aware diagnostic and provides analyses of dynamic discounting and oracle value conditioning.

3.1 Preliminaries

  • GRPO samples groups of chain-of-thought rollouts, scores them with a terminal verifier reward, and assigns each token the same group-normalized advantage; RECAP replaces that uniform advantage with a revisit-conditioned surrogate.It retains the sampling protocol, clipped policy-ratio update, and KL anchor.Source: §3.1 · p. 3

3.2 The Revisit Signal

  • Visual revisit is a local positive jump in cross-modal attention after recent language-internal reasoning, rather than a measure of absolute attention mass.The bounded, normalized signal is detached and computed after ordinary-policy sampling, so it changes training-time advantage construction rather than the sampling distribution.Source: §3.2 · p. 4

3.3 RECAP Objective

  • RECAP uses trace-augmented estimator inputs, revisit-conditioned dynamic discounts, and revisit-gated value prediction; future revisit peaks can retroactively increase credit for earlier decisions.The revisit trace modulates discounting and value prediction, not the reward itself.Source: §3.3 · pp. 4–5

3.4 The RAP Metric

  • RAP measures the return-weighted concentration of revisit signal using a fixed, method-independent return; it is bounded and insensitive to revisit spikes at zero-return positions.The paper validates it as a training diagnostic and reports that it rises early and plateaus higher under RECAP than under GRPO or VPPO.Source: §3.4 · p. 5

3.5 Proposition 1: A Toy Separation from Same-Step Reweighting

  • Unlike same-step local reweighting, revisit-conditioned dynamic discounting can let a future revisit peak alter earlier action credit, producing a preference that positive local reweighting cannot induce in the stated construction.The proposition establishes this separation for a finite-horizon MDP and specified revisit-pattern distribution and parameters.Source: §3.5 · p. 6

3.6 Theorem 1: Revisit-Gated Return-Residual Variance

  • Under the stated oracle conditions, conditioning value prediction on current revisit cannot increase return-residual variance; the reduction depends on revisit information about returns.The analysis is not a policy-gradient optimality guarantee, and the paper describes post-hoc revisit conditioning as a surrogate with a bias–variance trade-off.Source: §3.6 · p. 6

4 Experiments

The experiments evaluate RECAP against GRPO and other baselines across three VLM backbones, then test its components, causal grounding signal, and generalization. The results report improved HallusionBench performance while preserving reasoning and text-only capability, alongside supporting intervention and robustness analyses.

4.1 Setup

  • The evaluation uses a shared two-stage SFT-then-RL pipeline, with 8 seeds for headline results and 5 for ablations.The three primary backbones use 50K filtered long-CoT SFT examples, followed by RL on 30K disjoint verifiable-answer questions.Source: §4.1 · pp. 6–7

4.2 Main Results

  • RECAP improves HallusionBench accuracy over GRPO by +3.8, +3.5, and +4.1 points on Qwen2.5-VL-7B, InternVL3-8B, and LLaVA-OneVision-7B, while preserving reasoning and text-only capability.The result is reported in Table 1 across three backbones.Source: Table 1 · p. 7, §4.2 · p. 7

4.3 Ablations and Signal Characterization

  • Removing dynamic discount causes the largest ablation drop; gated value prediction follows, and replacing revisit with entropy, logit margin, or hidden-state norm underperforms.Using revisit only as a local advantage reweight is also weaker than using it as a dynamic discount.Source: §4.3 · pp. 7–8

4.4 Causal and Mechanistic Evidence

  • Top revisit positions are more decision-relevant than random non-peaks, and the top-10 heads carry more than 40% of revisit mass across backbones.Zeroing those heads reproduces most of the accuracy drop from masking cross-modal attention.Source: §4.4 · p. 8, Figure 3 · p. 9

4.5 Generalization, Robustness, and Diagnostics

  • In human preference comparisons, RECAP wins over 60% of image-groundedness comparisons against GRPO and VPPO, and the LLM-as-judge ranking agrees with the human majority on 72% of items.The study covers 100 HallusionBench and MathVista questions with non-identical chains of thought.Source: §4.5 · pp. 8–9
  • English-only-trained RECAP consistently improves over GRPO zero-shot on Chinese CMMMU, MMMU, and MathVista-CN, with smaller gains than on English hallucination benchmarks.The gains occur without translation-specific tuning or Chinese RL data.Source: §4.5 · p. 9
  • Across Gaussian noise, JPEG compression, patch dropout, off-topic patches, and question reformulations, RECAP keeps a larger margin than GRPO and VPPO; strong patch dropout is hardest.Under strong patch dropout, the margin does not invert.Source: §4.5 · p. 9
  • RAP rises early during Stage-2 RL and plateaus higher under RECAP than under GRPO or VPPO, supporting its use as a method-agnostic training diagnostic.Cross-method scatter plots support RAP as a diagnostic rather than a RECAP-specific objective trace.Source: §4.5 · p. 9

5 Conclusion

The conclusion presents visual revisit as a temporal grounding signal and RECAP as a way to use its traces for credit assignment. The accompanying proof setup distinguishes dynamic discounting from same-step local reweighting, while the stated limitations bound RECAP's applicability.

5 Conclusion

  • RECAP uses detached revisit traces for credit assignment and improves visual grounding over GRPO and VPPO while largely preserving reasoning and text-only capability.The trace is extracted from the same forward pass and discarded at inference.Source: §5 · p. 9
  • RECAP relies on accessible, meaningful cross-modal attention traces, and its benefits may be smaller for short answers, text-only tasks, or architectures where attention is a weak grounding proxy.The conclusion also notes that salient distractors or corrupted visual regions can mislead revisit.Source: §5 · p. 9

A Proof of Proposition 1

  • The proposition isolates retroactive credit coupling from dynamic discounting; it does not claim dominance over all trajectory-dependent reweighting schemes.The stated separation is between dynamic discounting and same-step local reweighting.Source: A Proof of Proposition 1 · p. 11

A.1 Setup

  • The proof models a finite-horizon discrete-action MDP whose state encodes the prompt, image, and generated-token history, with a deterministic revisit sequence computed from cross-modal attention.For the proof, revisit is treated as an exogenous function of the trajectory.Source: §A.1 · p. 11

A.2 Horizon-3 counterexample

The horizon-3 counterexample separates same-step revisit reweighting from dynamic discounting: a future revisit peak can change credit for an earlier decision. The discussion limits this result to local reweighting, since trajectory-dependent reweighting can simulate the discount.

A.2 Horizon-3 counterexample

  • In the horizon-3 construction, a+ defers reward cR to the terminal step while a− receives R at step 2, and revisit peaks only at step 3.The sole decision is at step 1.Source: §A.2 · p. 12
  • Because revisit at the decision step is zero, any positive same-step reweighting preserves the static-discount preference between a+ and a−.The proof states that multiplying by w(0) cannot change the preference sign.Source: §A.2 · p. 12
  • In the constructed example, local reweighting prefers a− while bounded dynamic discount prefers a+, with all discounts remaining below one.The dynamic discount incorporates the later revisit peak into the earlier decision's return.Source: §A.2 · p. 13

A.3 Discussion

  • A later revisit peak retroactively changes credit for earlier decisions through discount products, which same-step local reweighting cannot reproduce in this example.Trajectory-dependent reweighting can simulate dynamic discounting, but that is outside the proposition's local-reweighting comparison.Source: §A.3 · p. 13
  • Horizon 3 is the smallest clean setting because the decision, intervening revisit peak, and delayed reward occupy distinct positions; analogous examples extend to longer horizons.For longer horizons, peaks can lie at any step strictly between the decision and reward.Source: §A.3 · p. 13

B Proof of Theorem 1 and Bias–Variance Decomposition

The appendix establishes when revisit conditioning can reduce return-residual variance and clarifies the bias trade-off of using a trajectory-dependent value predictor. It also details how the revisit signal and RAP are constructed and what their diagnostic properties do—and do not—establish.

B.2 Proof of Theorem 1

  • Conditioning on the revisit trace reduces expected return-residual variance, with a strictly positive reduction when the trace varies and changes the conditional return mean.The variance identity itself holds without Assumption 1; that assumption is used only to make the reduction nontrivial.Source: §B.2 · p. 14

B.3 Bias induced by trajectory-dependent value predictors

  • Post-hoc revisit dependence on the sampled token can make RECAP’s policy update biased relative to the vanilla return gradient, so it optimizes a revisit-conditioned surrogate.The bias vanishes for prefix-measurable predictors and is small when post-hoc variation is small.Source: §B.3 · p. 15

C.1 Cross-modal attention aggregation

  • Cross-modal attention is averaged over heads in the middle-layer range, where profiling identifies the desired image-looking signal; max pooling adds variance without improving accuracy.The default uses the middle third of layers, and the layer-range sensitivity sweep finds it stable relative to nearby choices.Source: §C.1 · p. 15

C.2 Why ReLU above local baseline?

  • Revisit measures a positive jump above a sliding local baseline, separating returns to image evidence from absolute attention levels and downward turn-aways.The text associates downward dips chiefly with arithmetic or syntactic transitions, which are not predictive of grounded reasoning.Source: §C.2 · p. 16

C.3 Stability of normalization

  • Per-rollout min–max rescaling makes revisit distributions comparable across questions, while temperature controls their peakedness; the tested η = 6 keeps most values between 0.1 and 0.9.The revisit trace is stop-gradient, separating forward-pass revisit behavior from backward-pass policy improvement.Source: §C.3 · p. 16

C.4 Properties of RAP

  • RAP can rise from revisits aligned with positive evaluation return even when those revisits are not causally responsible for the return.Causal grounding is tested separately through masking and injection interventions.Source: §C.4 · p. 17
  • RAP depends on the rollout distribution, forward-pass revisit signal, and fixed evaluation return—not the training objective—so it is comparable across methods.The appendix reports empirical correlation with HallusionBench accuracy on checkpoints from methods other than RECAP.Source: §C.4 · p. 17

D Log-Domain Implementation of Discounted Returns

The appendix uses a backward log-domain recursion to propagate discounted terminal returns while avoiding underflow on long reasoning trajectories. It also records the training protocol and configurations used for the reported experiments.

D Log-Domain Implementation of Discounted Returns

  • Naive discount products can underflow on 200–1024-token reasoning traces, eliminating late-trajectory return contributions and biasing surrogate advantages toward earlier tokens.For γbase in [0.95, 0.99], the longest-horizon product can be about 4 × 10^-23.Source: D Log-Domain Implementation of… · p. 17
  • The recursion places the raw terminal verifier score at EOS, sets intermediate entries to zero, and propagates only non-negative returns; format penalties reduce positive credit but are not propagated as negative returns.The revisit signal itself is not treated as reward.Source: D Log-Domain Implementation of… · p. 17
  • On full-length rollouts, the log-domain recursion preserves non-degenerate advantages along the trajectory, while the naive computation zeros advantages past about token 700; truncated-rollout errors are at most 1 × 10^-5 in fp32 and 1 × 10^-4 in bf16.The recursion matches naive linear-domain computation within 10^-5 relative error on rollouts where that computation does not underflow, verified on at least 10^4 rollouts.Source: D Log-Domain Implementation of… · p. 18

E.1 Stage 1 SFT

  • Stage 1 uses backbone-specific supervised fine-tuning settings before Stage 2 RL, with the per-backbone configurations summarized in Table 4.The examples are approximately 50K long-CoT samples produced by sampling base checkpoints on verifiable-answer questions and filtering for correctness.Source: §E.1 · p. 18, Table 4 · p. 19

E.2 Stage 2 RL

  • Stage 2 uses a shared GRPO setup, while method-specific RECAP settings and baseline choices are documented separately.The protocol uses group size M = 8, clipping ε = 0.2, and a KL coefficient of 0.04 against the SFT checkpoint.Source: §E.2 · p. 19, Table 5 · p. 19

E.3 72B LoRA configuration

  • The Qwen2.5-VL-72B experiment uses rank-64 LoRA adapters on attention projections, with trainable parameters comprising approximately 0.6% of the base model.The run uses ZeRO-3 across at least 32 GPUs, and its scores are reported in Table 6.Source: §E.3 · p. 19

E.4 Compute summary

The appendix reports RECAP’s step-time and compute costs, the sensitivity of its revisit-signal settings, and cross-method RAP training curves. The default signal configuration is robust to nearby choices, while all-layer pooling is the notable degradation case.

E.4 Compute summary

  • The reported compute totals approximately 780 GPU-days at H20-equivalent, with eight seeds for headline runs and five for ablations.The estimate is approximately 320–400 H100-days.Source: §E.4 · p. 19
  • RECAP adds about 5% wall-clock time per optimization step over GRPO on Qwen2.5-VL-7B, with no inference-time overhead.The measured overhead comprises 3.8% for attention aggregation and baseline computation, 0.9% for the log-domain recursion, and 0.3% for the auxiliary value head.Source: §E.4 · p. 19

E.5 Signal-pipeline sensitivity sweep

  • The default revisit-signal setting stays within ±0.6 points across the hyperparameter grid, while all-layer pooling significantly underperforms as top-layer noise dilutes the estimate.The default is within 0.2 points of the best per-backbone tuned setting; the full grid has a 3.1-point max–min spread.Source: §E.5 · p. 20, Table 7 · p. 20

F.1 Training curves

  • RAP rises early and plateaus higher under RECAP than under GRPO, VPPO, or the SFT reference.The appendix describes RAP as a method-agnostic training diagnostic rather than a RECAP-specific objective trace.Source: §F.1 · p. 20

F.2 Full scatter data

RAP tracks HallusionBench accuracy across training methods and has stronger associations with visually grounded benchmarks than reasoning-dominated ones. The appendix localizes revisit signals to a small, reproducible set of heads and reports limits on RAP’s informativeness and cross-backbone alignment.

F.2 Full scatter data

  • RAP correlates with HallusionBench accuracy across checkpoints from all four methods, supporting its use as a method-agnostic diagnostic.The correlation is highest for RECAP but remains substantial for the other methods.Source: §F.2 · p. 20, Table 8 · p. 22

F.3 Per-benchmark breakdown

  • RAP correlations are strongest on visually resolvable but linguistically tempting benchmarks and weaker, though positive, on reasoning-dominated MathVista and MMMU.This pattern bounds RAP’s scope to grounded reasoning rather than reasoning capability generally.Source: §F.3 · p. 21, Table 9 · p. 22

F.4 When does RAP fail?

  • RAP becomes uninformative when revisit is uniformly zero or saturated; the zero-revisit regime was not observed in the reported training runs.Without stop-gradient, revisit saturation occurs within about 500 optimization steps and HallusionBench accuracy collapses.Source: §F.4 · p. 21

G.1 Per-backbone head heatmaps

  • The top 10 heads carry at least 53% of revisit signal while covering at most 4% of heads, and their outputs predict near-future correctness better than matched random heads.The probe AUC is at least 0.66 across backbones (pooled mean 0.67), versus at most 0.55 for random heads.Source: §G.1 · p. 22

G.2 Top-10 heads catalogue

  • On Qwen2.5-VL-7B, the top revisit head contributes 9.3% of revisit-peak attention, with a 1.8-point HallusionBench mask drop and probe AUC 0.83.The catalogue orders each backbone’s top-10 heads by revisit contribution.Source: §G.2 · p. 22, Table 11 · p. 24

G.3 Pre/post-training shift

  • The top-10 head share rises from 34% for the SFT base to 54% after RECAP, while GRPO on the same data shifts it by only +4 points.The reported shift redistributes existing revisit attention onto fewer heads rather than recruiting new heads.Source: Figure 6 · p. 23, §G.3 · p. 23, Table 12 · p. 24

G.4 Cross-backbone alignment

  • Top revisit heads cluster at normalized layer positions 0.4–0.65 and head positions 0.1–0.5 across backbones, but the authors do not claim a universal subspace.The positional agreement is reported at the ±0.05 level and is presented as suggestive, warranting further work.Source: §G.4 · p. 23

H.1 Recruitment

  • The human-study pool comprised 10 annotators selected for English proficiency, multimodal QA experience, and no prior employment with the authors’ institutions.The appendix reports four women and six men, with ages 22–45.Source: §H.1 · p. 24

H.2 Task

Annotators independently ranked anonymized GRPO, VPPO, and RECAP reasoning chains for groundedness and coherence, and rated trustworthiness. The study reports significant groundedness gains for RECAP over both baselines, with an independent LLM-judge evaluation providing a second comparison.

H.2 Task

  • Annotators ranked three anonymized chains per sample for image-groundedness and coherence, and rated each chain’s trustworthiness.The ground-truth answer was masked, and annotators were instructed not to use it to infer groundedness.Source: §H.2 · p. 24

H.3 Quality control

  • Annotators passed a calibration threshold of at least 0.70 pairwise agreement; none were excluded under the post-hoc κ = 0.40 agreement rule.Calibration responses were not included in reported data.Source: §H.3 · p. 24

H.4 Statistics

  • RECAP has the highest image-groundedness win rate, with significant gains over GRPO and VPPO; the LLM-judge ranking agrees with the human majority on 72% of items.Reported inter-annotator agreement is κ = 0.55 for groundedness, κ = 0.48 for coherence, and α = 0.51 for trustworthiness.Source: §H.4 · pp. 24–25, Table 14 · p. 25

H.5 Sample interface

  • The interface displayed the image and question above three numbered, anonymized chains, with randomized chain order and answer masking enabled by default.Groundedness and coherence used ranking widgets; trustworthiness used a per-chain 1–5 slider.Source: §H.5 · p. 25

H.6 LLM-as-judge triangulation

  • All three LLM judges rank RECAP highest, scoring it 4.05–4.12 versus at most 3.51 for VPPO and 3.29 for no-RL TVC.The judges’ inter-agreement is α = 0.63.Source: §H.6 · p. 25

I Cross-Lingual Evaluation: Protocol and Quality Control

The cross-lingual evaluation uses three Chinese benchmarks, including a reviewed translated subset of MathVista, and compares English-only RECAP training with zero-shot Chinese performance. The reported gain ratios indicate that Chinese gains track English gains across most backbone-benchmark pairs; the appendix also specifies image and question perturbations.

I.1 Datasets

  • The Chinese evaluation spans CMMMU, MMMU, and a 1,000-sample translated MathVista-CN subset.The appendix identifies these as spanning hallucination, reasoning, and grounding axes covered by the English suite.Source: §I.1 · p. 26

I.2 Translation procedure

  • Two native Chinese reviewers scored all 1,000 MathVista-CN translations, with a mean fidelity score of 4.6/5; four low-scoring samples were discarded and re-translated.Reported numbers use the reviewed 1,000-sample subset, not the remaining unreviewed translations.Source: §I.2 · p. 26

I.3 Per-benchmark scores

  • Table 16 reports the full four-backbone by three-benchmark comparison for cross-lingual evaluation.The table caption states that RECAP is trained on English data and evaluated zero-shot on Chinese benchmarks, with consistent gains across backbones.Source: Table 16 · p. 26, §I.3 · p. 26

I.4 English–Chinese gain ratio

  • Across 12 backbone-benchmark cells, the median Chinese-to-English gain ratio is 0.85, and 10 of 12 cells reach at least 0.8.Both cells below 0.8 are on MMMU; the authors interpret the pattern as supporting language-agnosticism.Source: §I.4 · p. 27

J.1 Image-side perturbations

  • The appendix evaluates image-side perturbations at inference time while keeping the question and model unchanged.The perturbation families are Gaussian noise, JPEG compression, random patch dropout, and insertion of a non-occluding distractor.Source: §J.1 · p. 27

J.2 Text-side perturbations

  • Each HallusionBench question receives five GPT-4 reformulations, with a manually reviewed 20% sample and an 8% mean rejection rate.Reformulations changing the answer or introducing ambiguity are rejected and regenerated.Source: §J.2 · p. 27

J.3 Per-perturbation accuracy curves

RECAP remains more robust than the compared methods under image perturbations and question reformulations, but its benefits are bounded: short reasoning, misleading saliency, and mistimed injection are important failure regimes.

J.3 Per-perturbation accuracy curves

  • RECAP retained a positive accuracy gap over GRPO across every image-perturbation grid cell, with smaller drops than VPPO and an advantage under stronger perturbations.The gap narrows at the strongest patch-dropout setting but does not invert.Source: Figure 7 · p. 28, §J.3 · p. 28
  • RECAP’s reformulation standard deviation is ≤1.0 pt on every tested backbone, while GRPO and VPPO exceed 2.5 pt.Accuracy is averaged over five meaning-preserving reformulations per question.Source: §J.3 · p. 28, Table 18 · p. 28

K Failure Cases

  • The failure analysis covers designated benchmark strata and is not presented as exhaustive.It catalogs regimes where RECAP does not improve over GRPO or underperforms.Source: K Failure Cases · p. 28

K.1 Where RECAP does not help

  • On ≤30-token questions, RECAP matches GRPO within 0.3 pt, where no extended language-internal stretch supports a meaningful revisit.The authors suggest a short-rollout fallback when target latency demands short answers.Source: §K.1 · p. 29
  • On text-only questions, checkpoints match the SFT base on MMLU/GSM8K to within 0.5 pt.With a placeholder image, the revisit signal is near-zero and conditioning is minimal.Source: §K.1 · p. 29
  • Misleading salient objects account for ∼15% of RECAP’s remaining HallusionBench errors because the revisit signal cannot distinguish relevance from saliency.The model can over-attend to an answer-irrelevant object and produce a confident wrong answer.Source: §K.1 · p. 29

K.2 Peak-injection counterexamples

  • Misaligned injection into a non-candidate position, such as arithmetic, drops accuracy by ∼0.5–1.5 pt; injection has no measurable effect when RAP ≥0.7.The authors bound the useful scope to long-CoT chains with low RAP and say it does not generally rescue arithmetic, pure-text, or misleading-saliency failures.Source: §K.2 · p. 29

L Reproducibility Statement

  • A Docker image reproduces the Qwen2.5-VL-7B main-table row on a 16-GPU H20 node in an estimated ∼30 hours.A CPU-only image reproduces analysis figures from cached artifacts in ≤30 minutes.Source: L Reproducibility Statement · p. 30

Related papers

Loading 2610.30834v1…