PaperlayerView exact revision on arXiv

Source-linked AI summary

Visual sensitivity is not claim retractability: persistence-aware credit assignment for multimodal reinforcement learning

Zhongan Bi, Kepeng Lin, Xuanang Gao, Yuhan Sun, Lianrun Zhang

arXiv:2609.36572v3cs.AI

TL;DR

Correct answers can contain unsupported visual claims that receive positive outcome-level credit, and visual sensitivity does not ensure those claims become retractable. The paper diagnoses this mismatch with fixed-rollout image interventions and introduces PACG to gate credit for persistent claims. PACG improves benchmark accuracy across DAPO and VPPO while reducing unsupported-claim persistence.

  • Problem

    Outcome-level rewards and image-dependence signals do not determine whether individual visual claims are supported, so unsupported claims in correct trajectories can inherit positive credit.

    Source: §1 · p. 2, §3 · p. 4

  • Method

    A fixed-rollout counterfactual diagnostic separates prediction sensitivity to image intervention from claim persistence, and PACG attenuates positive credit for unusually persistent visual claims.

    Source: §1 · p. 2, §4.2 · p. 5

  • Results

    PACG improves DAPO on all nine benchmarks and further improves VPPO, while reducing unsupported-claim persistence and preserving or improving EFS and final-answer accuracy.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Visual sensitivity and claim retractability are complementary dimensions of multimodal credit assignment.

    Source: §1 · pp. 2–3

    NoisyRollout is trained using Geometry3K training data, so its Geometry3K score is excluded from that column’s ranking.

    Source: §5.2 · p. 7

Abstract

from arXiv · show

Reinforcement Learning with Verifiable Rewards (RLVR) has been extended to Large Vision-Language Models (LVLMs), and perception-aware methods further encourage policies to rely on visual evidence. Yet relying on the image does not guarantee that visual claims are supported by it. Before RL training, 27.81% of the correctly answered responses of Qwen2.5-VL-7B on four multimodal reasoning benchmarks contain at least one direct visual claim that the image does not support. Since outcome-level RL rewards each response as a whole, these claims inherit the positive credit of the correct answer. We introduce a fixed-rollout counterfactual diagnostic that re-scores the same response under an intervened image to separate Evidence-Function Sensitivity (EFS), how strongly the model's predictions change, from claim persistence, whether the model keeps supporting the same claim rather than retracting it. The diagnostic reveals Sensitivity-Persistence Decoupling (SPD): under DAPO and VPPO, EFS increases and claims become more retractable overall, yet unsupported claims become significantly more persistent, whereas GRPO raises EFS without this deterioration. We therefore propose Persistence-Aware Credit Gating (PACG), which attenuates positive credit for unusually persistent visual claims and leaves all other credit unchanged. It requires no supported/unsupported labels and adds no inference cost. On Qwen2.5-VL-7B, PACG raises the nine-benchmark average over three seeds from 58.1% to 59.9% with DAPO and from 59.8% to 60.9% with VPPO, while making unsupported claims more retractable. The gains extend to a larger model, a newer backbone, and the accuracy of HallusionBench also improves consistently. These results suggest that visual sensitivity and claim retractability are complementary dimensions of multimodal credit assignment.

1 INTRODUCTION

Outcome-level rewards can credit unsupported visual claims embedded in correct answers, and visual sensitivity alone does not ensure those claims are retractable. The paper diagnoses this mismatch and introduces a credit gate that improves benchmark performance while reducing unsupported-claim persistence.

1 INTRODUCTION

  • Among 8,521 answer-correct Base trajectories, 2,370 (27.81%) contained at least one unsupported direct visual claim, and each such claim inherited positive trajectory-level credit despite lacking visual support.Direct visual claims are minimal response spans asserting information read directly from the image, such as counts, OCR values, or geometric properties.Source: §1 · p. 2
  • DAPO and VPPO increase EFS and overall retractability while making unsupported claims more persistent; GRPO raises EFS without the same deterioration.The fixed-rollout diagnostic separates Evidence-Function Sensitivity from persistence, which is calibrated against residual tokens from the same rollout.Source: §1 · p. 2
  • PACG improves DAPO on all nine benchmarks and further improves VPPO, reducing unsupported-claim persistence while preserving or improving EFS and accuracy.Both gains are significant across three seeds, and HallusionBench accuracy also improves consistently.Source: §1 · pp. 2–3
  • PACG attenuates positive token credit for unusually persistent routed visual claims, while leaving negative advantages and non-routed tokens unchanged.Training requires no supported/unsupported labels, and PACG adds no inference-time cost.Source: §1 · pp. 2–3

2 RELATED WORK

Prior multimodal RL methods allocate or refine credit using outcomes, visual dependence, or process-level supervision. PACG instead targets whether an individual visual claim loses support when its evidence is disrupted.

2 RELATED WORK

  • Outcome-level multimodal RL methods often broadcast one answer-level advantage across response tokens, allowing unsupported claims in correct trajectories to inherit positive credit.The paper reports that this persistence shift worsens under DAPO and VPPO but not under GRPO.Source: §2 · p. 3
  • Perception-aware methods shape where and how strongly policies rely on images, while PACG asks whether a specific claim loses support after evidence disruption.PACG can be composed with perception-aware methods such as VPPO.Source: §2 · p. 3
  • Prior counterfactual approaches analyze modality effects or alter decoding and preference data; this work uses image intervention during RL training to create token-level retractability credit without inference-time cost.The paper contrasts its training mechanism with approaches that evaluate a fixed model or modify decoding and preference data.Source: §2 · p. 3

3 PRELIMINARIES

The preliminaries identify a gap in token-level credit: outcome correctness and image dependence do not establish whether the image supports a visual claim. PACG is designed to add that missing signal.

3 PRELIMINARIES

  • Outcome-driven GRPO and DAPO broadcast sequence advantage to every response token, so unsupported visual claims in correct trajectories receive positive credit.The objective provides no signal to distinguish unsupported claims from supported ones.Source: §3 · p. 4
  • VPPO reshapes token credit using image-perturbation dependency scores, but credit still does not establish whether the image supports the claim.Both outcome-driven and perception-aware credit depend on answer correctness and/or token image dependence.Source: §3 · p. 4
  • PACG supplies a support-related signal without requiring support labels during training.Its overview routes direct visual claims, measures their persistence under image intervention, and gates positive token credit.Source: §3 · p. 4, Figure 2 · p. 4

4 METHOD

PACG routes direct visual claims, measures whether they retract when image evidence is disrupted, and gates positive credit for claims that persist. It modifies only routed positive credit, leaving other credit unchanged.

4.1 DIRECT VISUAL CLAIM ROUTING

  • Direct visual claims are spans asserting information read from the image; incorrect assertions are routed if presented as image evidence, while derived reasoning and question paraphrases are excluded.Supportedness is annotated only for analysis, never used in training.Source: §4.1 · pp. 4–5

4.2 FIXED-ROLLOUT COUNTERFACTUAL MEASUREMENT

  • EFS measures changes in claim-token predictive distributions under image intervention, but its unsigned value cannot distinguish retraction from increased support.The same sampled response is teacher-forced under original and degraded images, isolating changes attributable to visual context; corrected persistence subtracts mean likelihood shift on residual tokens.Source: §4.2 · p. 5
  • Corrected persistence indicates retraction when a claim loses more support than the response’s residual tokens, while values near or above zero indicate persistence.The mismatch between EFS and corrected persistence is termed Sensitivity–Persistence Decoupling.Source: §4.2 · p. 5

4 METHOD

  • PACG scales down positive credit for unusually persistent routed claims while leaving negative advantages and non-routed tokens unchanged.Claims that retract by a sufficient margin retain full positive credit; the gate uses a frozen rollout policy and stop-gradient.Source: §4.3 · p. 5, §4.4 · p. 6

4.3 PERSISTENCE-AWARE CREDIT GATING

  • Claims retracting by at least margin m keep full positive credit; more persistent claims are smoothly attenuated toward floor gmin, retaining some credit.The gate strength is controlled by α.Source: §4.3 · p. 5

4.4 PACG POLICY OPTIMIZATION

  • PACG applies the minimum gate across routed claims covering each token and substitutes gated for original parent credit without changing rewards, rollout generation, normalization, or clipping.A free-generation audit found a modest drop in direct-claim rate alongside longer responses, rather than a collapse of visual content.Source: §4.4 · pp. 5–6

5 EXPERIMENTS

PACG improves benchmark accuracy while the experiments reveal that increased visual sensitivity can coexist with greater persistence of unsupported claims. Ablations identify the gate components and margin choice, while mechanism analyses show that PACG mitigates this decoupling for both parent optimizers.

5.2 MAIN RESULTS

  • DAPO+PACG raises the nine-benchmark average from 58.1 to 59.9, while VPPO+PACG reaches the best 7B average of 60.9.Both average gains are statistically significant across three training seeds; DAPO+PACG improves all nine benchmarks and VPPO+PACG improves eight.Source: §5.2 · p. 6, Table 1 · p. 7
  • DAPO+PACG raises average accuracy from 66.3 to 67.2 on Qwen2.5-VL-32B, and from 51.3 to 53.5 on Qwen3-VL-2B-Thinking.VPPO+PACG gives the best aggregate on Qwen3-VL-2B-Thinking at 54.5.Source: §5.2 · p. 6, Table 1 · p. 7
  • PACG improves all three HallusionBench metrics for both parents, and VPPO+PACG is best in every column.The largest VPPO gain is 0.86 points on qAcc; fAcc also rises for both parents.Source: §5.2 · pp. 6–7, Table 2 · p. 6
  • PACG closely tracks GRPO’s reward trajectory and raises late-stage reward for DAPO and VPPO.Figure 3 shows trailing 10-step means for one representative seed per method, with parent runs dashed and PACG compositions solid.Source: §5.2 · p. 7, Figure 3 · p. 8

5.3 ABLATION STUDY

  • Removing direct-claim routing collapses accuracy to 38.0; removing residual calibration or positive-only gating loses 7.4 and 6.1 points on Clever-Count and weakens unsupported-claim retractability.The component ablations use DAPO as parent and report point estimates across accuracy and unsupported corrected persistence.Source: §5.3 · p. 7, Table 3 · p. 8
  • At m = 0.10, positive-credit retention drops to 65% and EFS to 0.28, with accuracy slightly below the m = 0.05 result despite stronger retraction.The zero margin retains 95% of positive credit and matches ungated DAPO on MMK12; m = 0.05 is the chosen operating point.Source: §5.3 · p. 8, Table 4 · p. 8

5.4 SENSITIVITY–PERSISTENCE DECOUPLING AND ITS MITIGATION

  • DAPO and VPPO increase EFS and reduce overall persistence, yet raise unsupported persistence by 0.0651 and 0.0831, respectively.GRPO increases EFS the most without a resolved deterioration; DAPO and VPPO apply roughly three times GRPO’s effective positive update mass to routed claims, which may contribute to the difference.Source: §5.4 · pp. 8–9, Figure 4 · p. 9
  • PACG lowers unsupported persistence by 0.0151 for DAPO and 0.0819 for VPPO while raising EFS and accuracy.VPPO+PACG returns to the Base persistence level, whereas DAPO+PACG recovers part of the gap; the phase trajectory shows DAPO drifting toward weaker retractability while PACG avoids this drift.Source: §5.4 · p. 9, Table 5 · p. 9, Figure 5 · p. 10

6 CONCLUSION

The paper identifies a credit-assignment failure in multimodal RL: greater visual sensitivity does not ensure unsupported claims become retractable. It introduces PACG and reports improvements in retractability and accuracy across the evaluated settings.

6 CONCLUSION

  • Sensitivity–Persistence Decoupling is an overlooked credit-assignment failure in multimodal RL: DAPO and VPPO can increase image sensitivity while making unsupported claims more persistent.The paper distinguishes whether a policy responds to image changes from whether it retracts a claim unsupported by the image.Source: §6 · p. 10
  • PACG tests whether direct visual claims retract when evidence is disrupted and selectively attenuates positive credit for claims that do not.It uses no supported/unsupported labels and adds no inference-time cost.Source: §6 · p. 10
  • PACG improves unsupported-claim retractability and accuracy across nine multimodal reasoning benchmarks for both optimizer types, with transfer to a larger model, newer backbone, and HallusionBench.The paper concludes that visual sensitivity and claim retractability are distinct dimensions of multimodal credit assignment.Source: §6 · p. 10

A.1 SHARED TRAINING RECIPE

The appendix specifies the shared training recipe, optimizer compositions, PACG gate, systems setup, and rollout procedure. Its routing and residual-control discussion frames the residual tokens as a practical within-trajectory reference rather than a pure non-visual baseline.

A.1 SHARED TRAINING RECIPE

  • The backbone recipes use an entropy penalty of 0.1, with backbone-specific training budgets and response limits; Qwen3-VL-2B-Thinking is evaluated as the additional backbone.Qwen2.5-VL runs use two epochs, while Qwen3-VL-2B-Thinking uses 200 steps; the vision tower remains trainable and rewards are binary answer correctness.Source: §A.1 · p. 17, Table 6 · p. 17

A.2 PARENT OPTIMIZERS AND REGULARIZATION

  • PACG preserves the parent optimizer’s reward and loss configuration, changing only routed positive token credits; all configurations use frozen-reference KL regularization.DAPO uses dynamic sampling and asymmetric clipping, while plain GRPO does not inherit DAPO’s dynamic filtering.Source: §A.2 · p. 17

A.3 PACG CONFIGURATION

  • PACG gates routed direct visual claims using a retraction margin of 0.05, attenuation strength of 10, and minimum gate of 0.30.Negative advantages and non-routed tokens remain unchanged, and supported/unsupported labels do not assign gates.Source: §A.3 · p. 17

A.4 COMPOSITION WITH VPPO

  • With VPPO, PACG is applied to base credits using the same positive-only rule; VPPO’s visual-dependency score is distinct from frozen-reference KL regularization.VPPO retains its visual-token selection fraction of 0.4 and minimum trajectory-level advantage scaling of 0.9.Source: §A.4 · p. 18

A.5 SYSTEMS AND EVALUATION

  • Training uses a VERL/EasyR1-derived pipeline with FSDP, BF16 parameters, gradient checkpointing, and vLLM rollout generation.Evaluation uses eight stochastic responses at temperature 1.0 with the same answer extraction and verification pipeline across matched models.Source: §A.5 · p. 18

B FULL PACG TRAINING PROCEDURE

  • Algorithm 1 preserves rollout generation, reward computation, and the parent policy objective while modifying eligible positive token credits.For each rollout, it routes claims, constructs a residual set and counterfactual image, teacher-forces the fixed response under both images, then applies gates before the base clipped update.Source: B FULL PACG TRAINING PROCEDURE · p. 18

B.1 ROUTING AND RESIDUAL CALIBRATION

  • The router identifies spans asserting direct image information, while residual tokens provide a within-response reference for calibrating trajectory-wide likelihood drift.The residual set is not assumed to be strictly vision-independent, and image-level degradation need not remove exactly the evidence relevant to an individual claim.Source: §B.1 · pp. 18–19

B.2 FORMAL ANALYSIS

The formal analysis gives conditional guarantees for PACG’s credit transformation, while distinguishing fixed-update properties from empirical claims about training behavior.

B.2 FORMAL ANALYSIS

  • These propositions are conditional fixed-update properties, not convergence guarantees; per-token scaling does not establish a hard trust region, unchanged future clipping, or training stability.The same bound argument extends to sequence-mean GRPO with corresponding weights; unit gates recover the parent objective.Source: §B.2 · pp. 19–20
  • Under stated distributional and conditional-independence assumptions, the gate retains a no-greater relative share of positive credit for unsupported than supported claims.The strict inequality requires α > 0 and gmin < 1, plus the stated strictness condition.Source: §B.2 · p. 19
  • The assumptions concern the evolving rollout policy and are not established by fixed-Base evaluation; observed mean orderings prove neither stochastic dominance nor the proposition’s assumptions on online tokens.The authors describe the proposition as a possible operating regime, not an experimentally established explanation of PACG’s gains.Source: §B.2 · pp. 19–20
  • At a fixed update, PACG bounds the reduction in the token-mean surrogate by (1 − gmin)A(1 + ϵhigh) for positive credit A.The result holds with sampled tokens, masks, parent advantages, and gates fixed.Source: §B.2 · p. 20
  • For positive-credit tokens, a fixed gate preserves the parent surrogate’s active clipping branches and scales the token gradient by the gate.This does not guarantee preservation of the summed-gradient direction or eventual optimizer step.Source: §B.2 · p. 20
  • Residual calibration cancels a shared additive shift across response tokens, but does not remove heterogeneous token-specific corruption effects or certify claim-localized evidence removal.This cancellation motivates the residual-calibration ablation.Source: §B.2 · p. 20
  • The mechanism reference uses 2,000 questions and four fixed Base responses per question, retaining incorrect answers and responses without eligible direct claims.Question selection is blind to correctness, span presence, evidence labels, and mechanism scores.Source: §C.1 · pp. 20–21
  • Checkpoint scoring teacher-forces identical responses and aligned spans under original and hash-fixed intervened images, with question-clustered bootstrap intervals.Responses without eligible spans contribute no claim-level observation; 8,000 is the reference-bank size, not every subgroup’s valid-observation count.Source: §C.2 · p. 21

C.4 BENCHMARK COVERAGE

The section covers benchmark scope, claim routing and audits, and mechanism measurements that distinguish image-specific persistence from online positive-credit exposure.

C.4 BENCHMARK COVERAGE

  • The nine accuracy benchmarks cover visual mathematics, geometry, counting, and broader multimodal reasoning; EFS and persistence are measured on CrossBench4, while HallusionBench uses separate LLM judging.HallusionBench results do not enter the nine-benchmark average.Source: §C.4 · p. 21

D.1 HUMAN AUDIT

  • Human agreement is highest for direct-claim identification and lower for evidence-status classification; PACG routes with the former and reserves support labels for offline analysis.Table 7 reports a 500-question CrossBench4 audit and distinguishes consensus agreement from annotator agreement.Source: §D.1 · p. 21, Table 7 · p. 22

D.2 ROUTER SELECTION

  • Qwen3.6-35B-A3B is selected as production router with F1 0.84 and 165 versus 425 seconds per batch, a 2.6× latency advantage over Qwen3.8-27B.Qwen3.8-27B has the highest F1, at 0.86.Source: §D.2 · p. 22, Table 8 · p. 22

D.3 QUALITATIVE EXAMPLES OF DIRECT VISUAL CLAIM ROUTING

  • Qualitative examples show that supported and contradicted direct visual claims are both routed even when the response reaches a correct final answer.Derived reasoning and question paraphrases are excluded from routing.Source: §D.3 · pp. 22–23

D.4 BASE CLAIM ANNOTATION CENSUS

  • 27.81% of 8,521 answer-correct Base trajectories contain at least one annotated unsupported or contradicted direct visual claim.This is trajectory-level prevalence, not the claim-level unsupported rate.Source: §D.4 · p. 24

D.5 FREE-GENERATION AVOIDANCE AUDIT

  • PACG lowers unsupported-claim rate by 2.68 points with DAPO and 0.83 points with VPPO, while mean response lengths increase from 516.82 to 528.46 and 681.46 to 697.47 tokens, respectively.The lower unsupported-claim rates accompany continued visual-claim generation rather than shorter responses.Source: §D.5 · p. 24, Table 9 · p. 25

E.1 IMAGE-VERSUS-TEXT DIFFERENCE-IN-DIFFERENCES

  • Image-versus-text persistence comparisons pair measurements by sample; lower values indicate a more negative image-specific persistence change relative to the text control.The accuracy column uses free generation and is separate from teacher-forced scoring of fixed Base responses.Source: §E.1 · p. 25

E.3 ONLINE POSITIVE-CREDIT EXPOSURE

  • DAPO and VPPO have broadly comparable positive routed-token rates but substantially greater effective positive update mass than GRPO; the comparison is descriptive, not causal.DAPO has a lower positive-token rate than GRPO yet a higher update mass.Source: §E.3 · p. 27, Table 13 · p. 27

E.4 CHECKPOINT TRAINING DYNAMICS AND SENSITIVITY–RETRACTABILITY PHASE TRAJECTORY

Checkpoint results track a divergence between visual sensitivity and unsupported-claim retractability, followed by corruption, optimizer-composition, and component-mechanism analyses.

E.4 CHECKPOINT TRAINING DYNAMICS AND SENSITIVITY–RETRACTABILITY PHASE TRAJECTORY

  • DAPO EFS rises from 0.2189 to 0.2856 while unsupported persistence shifts from −0.1369 to −0.1227; DAPO+PACG reaches EFS 0.3117 and unsupported persistence −0.1441.PACG prevents the unsupported-persistence drift observed during DAPO training; the series uses a fixed 500-question subset.Source: §E.4 · p. 27, Figure 6 · p. 27

E.5 ROBUSTNESS TO COUNTERFACTUAL CORRUPTION

  • PACG lowers unsupported persistence under all three tested image corruptions, with matched effects from −0.013 to −0.018.The illustration describes image-level controls, not verified claim-localized evidence removals.Source: §E.5 · p. 28, Table 14 · p. 28, Figure 7 · p. 28

F.1 COMPATIBILITY WITH PLAIN GRPO

  • GRPO+PACG raises the reported nine-benchmark average from 56.8 to 57.3, with displayed gains on MMK12, We-Math, MMMU-Pro, and Clever-Count.DynaMath changes from 65.1 to 65.0.Source: §F.1 · p. 28, Table 15 · p. 29

F.2 ADDITIONAL COMPONENT MECHANISM METRICS

  • The additional component-mechanism table reports direct EFS and overall corrected persistence; its caption marks a degenerate EFS value associated with training collapse, not improved visual grounding.The table complements the nine-benchmark accuracy and unsupported-persistence breakdown.Source: Table 16 · p. 29, §F.2 · p. 29

F.3 VARIATION ACROSS TRAINING SEEDS

This block reports seed variation, a persistence-margin sweep, and online diagnostics of PACG’s gate, reward, selector overlap, and optimization behavior.

F.3 VARIATION ACROSS TRAINING SEEDS

  • Both nine-benchmark average gains exceed reported seed-level standard deviations and are significant under two-sided Welch’s t-tests (p < 0.05).Table 17 reports Qwen2.5-VL-7B means and sample standard deviations over three training seeds.Source: §F.3 · p. 29, Table 17 · p. 29

F.4 SENSITIVITY TO THE PERSISTENCE MARGIN

  • At m = 0.05, MMK12 accuracy is 81.2%, EFS is 0.31, and positive-credit retention is 81%; the sweep identifies this as the joint operating point.The zero-margin setting retains 95% of positive credit at 81.0% accuracy, while m = 0.10 retains 65% and has EFS 0.28 and accuracy 81.0%.Source: §F.4 · p. 29, Table 18 · p. 30

G ONLINE TRAINING DIAGNOSTICS

  • Online diagnostics use each run’s own trajectories; optimization statistics, routing coverage, and selector overlap provide complementary views of gate behavior on the evolving response distribution.One representative seed is shown for each method, and sparse GRPO+PACG logs are not interpolated.Source: G ONLINE TRAINING DIAGNOSTICS · p. 30, §G.4 · p. 32, Figure 12 · p. 32

G.1 GATE DYNAMICS

  • Across the three PACG compositions, the mean span gate decreases gradually, around half of positive direct-claim tokens are downweighted, and fewer reach the gate floor.The logged token-multiplier mean is not weighted by advantage magnitude.Source: §G.1 · p. 30, Figure 8 · p. 30

G.2 ONLINE ACCURACY REWARD

  • PACG preserves GRPO’s online accuracy-reward trajectory and raises the later trajectory for DAPO and VPPO.Figure 9 compares each parent optimizer with its PACG composition through step 200.Source: §G.2 · p. 30, Figure 9 · p. 31

G.3 VPPO VISUAL-SELECTOR SCORES AND GATE OVERLAP

  • VPPO+PACG generally scores selected tokens higher and unselected tokens lower, while visual-token selection and PACG attenuation have intersecting but distinct scopes.These policy-dependent populations differ from the fixed direct claims used to measure EFS and retractability.Source: §G.3 · p. 31, Figure 10 · p. 31, Figure 11 · p. 31

G.4 OPTIMIZATION AND ROUTING COVERAGE

  • Figure 13 separates mean corrected delta, its 90th percentile, and margin-violation rate from routed-rollout rate, direct-token fraction, and intervention-valid rate.The 90th percentile describes the span distribution, not a confidence interval; spans include supported and unsupported claims.Source: Figure 13 · p. 32, §G.4 · p. 32

H TRAINING EFFICIENCY

The appendix reports PACG’s training-time overhead and documents the annotation prompts, audit procedure, and standardized templates used in the reported runs.

H TRAINING EFFICIENCY

  • 730.4 seconds per step versus 464.3 seconds for DAPO, a 1.57× increase (57.3% overhead); PACG takes 40.6 versus 25.8 hours over 200 steps.The reported PACG timing includes 159.4 seconds per step waiting for annotation; PACG adds no inference-time computation.Source: Table 19 · p. 33, H TRAINING EFFICIENCY · p. 33

I PROMPTS AND TEMPLATES

I.1 MULTIMODAL SPAN ANNOTATION PROMPT

  • Direct VEF is a functional label for claims that read information from the image, independent of whether the image supports or contradicts them.The annotation separately records evidence status and excludes derived reasoning, question restatements, and deductions from direct VEF.Source: §I.1 · p. 34

I.2 FREE-GENERATION AVOIDANCE-AUDIT PROMPT

  • The free-generation audit annotates both correct and incorrect responses while keeping answer scores separate from the annotation request.It uses Qwen3.8-max with the original image, question, and response, and hides the source checkpoint with an opaque identifier.Source: §I.2 · pp. 35–36

I.3 TRAINING AND EVALUATION PROMPT TEMPLATE

  • All reported training and evaluation runs use one standardized reasoning template that supplies the image and question and supports final-answer parsing.The template requests reasoning inside <think> tags and the final answer inside \boxed{}.Source: §I.3 · p. 36

I.4 CROSSBENCH4 BASE ANNOTATION REQUEST

  • CrossBench4 Base annotation supplies images, question text, and generated traces without asking the annotator to grade the answer or provide the gold answer.Source: §I.4 · p. 36

J LIMITATIONS AND BROADER IMPACT

The section bounds the mechanism analysis and describes selected examples where PACG responses correct visual or reasoning errors made by parent optimizers.

J LIMITATIONS AND BROADER IMPACT

  • Mechanism comparisons use fixed Base responses on four benchmarks, so they do not represent each policy’s full generation distribution.Image corruption measures response to the chosen intervention, not factual correctness; routing and evidence annotation remain imperfect, and new domains may need calibration. The tested margin supports m = 0.05 only among the settings examined, while routing adds service-dependent training latency; PACG is credit assignment, not factuality verification. The authors frame sensitivity and retractability as complementary dimensions.Source: J LIMITATIONS AND BROADER IMPACT · p. 37

K.1 COUNTING SMALL, ADJACENT AIRPLANES

  • DAPO omits one of two small central airplanes and reports six objects, while the selected PACG response includes both and reports seven.Source: §K.1 · p. 38

K.2 CUBE-FOLDING REASONING

  • The cube-folding example contrasts DAPO’s choice A with PACG’s orientation-consistent choice D, which preserves cyclic face order and rejects the mirrored fold.Source: §K.2 · p. 39

K.3 HISTOGRAM REASONING

  • The histogram example contrasts a 300-boy height-sum estimate with PACG’s density-area calculation, which yields 600 boys.The selected bins are each 2 kg wide; their total area is 0.40, giving 1,500 × 0.40 = 600.Source: §K.3 · p. 40

K.4 EXPERIMENTAL-SETUP REASONING

  • The apparatus example contrasts parent choice A with PACG choice B: A lacks required heating, while B removes an acidic impurity and retains the target gas.Source: §K.4 · p. 41

K.5 COUNTING PARTIALLY OVERLAPPING MOTORCYCLES

  • In the motorcycle-count example, VPPO lists two motorcycles, while PACG identifies the cyan motorcycle and reports seven objects.The image contains three motorcycles, three airplanes, and one bicycle.Source: §K.5 · p. 42

K.6 READING THE ANGLE ASSOCIATED WITH THE TARGET ARC

  • In the arc-length example, VPPO uses 118° after recognizing 130°, whereas the selected PACG response uses 130° and obtains 4.54 inches.Source: §K.6 · p. 43

Related papers

Loading 2609.36572v3…