PaperlayerView exact revision on arXiv

Source-linked AI summary

VICO: Visual Environments Co-Evolving for Vision-Language Model Reasoning

Meng Lu, Ligeng Zhu, Olivia Xiao, Yuchen Zhuang, Zihan Wang, Kuncheng Wu, Bangya Liu, Yu Wang, Charles Fleming, Wenqi Shi, Xuan Wang

arXiv:2610.10782v1cs.CVcs.AI

TL;DR

Fixed visual tasks can become trivial or unsolvable as VLM actors improve, weakening RLVR’s learning signal. VICO jointly trains an actor and an EnvRewriter that edits verifiable visual structures using rollout pass rates to calibrate difficulty. Across nine multimodal benchmarks, it exceeds the strongest self-evolution baseline by +4.3% on out-of-domain tasks and uses up to 160× fewer labeled samples.

  • Problem

    Static RLVR environments can leave many tasks trivial or unsolvable as the actor improves, while useful adaptive edits must preserve valid labels and match actor capability.

    Source: §1 · p. 2

  • Method

    VICO jointly trains an actor and an EnvRewriter that edits verifiable visual structures, using rollout pass rates to calibrate task difficulty.

    Source: §1 · p. 2

  • Results

    Across nine multimodal benchmarks, VICO exceeds the strongest self-evolution baseline by +4.3% on out-of-domain tasks and uses up to 160× fewer labeled samples.

    Source: §6 · p. 10

  • Takeaways & Limitations

    The results suggest that the fixed nature of the environment, rather than static-corpus size, is a bottleneck in visual reasoning post-training.

    Source: §6 · p. 10

    VICO relies on verifiable image-side structures or reliable region annotations, limiting extension to visual domains without them.

    Source: A Limitations, Future Work, and Impact… · p. 14

Abstract

from arXiv · show

Reinforcement learning with verifiable rewards (RLVR) has become a standard recipe for post-training vision-language models (VLMs), but it typically assumes a static training environment. As the actor improves, fixed tasks drift out of its learning frontier: many become trivial, others remain unsolvable; and the learning signal collapses. We argue that VLM post-training should evolve the visual environment alongside the actor, not just the actor itself. We propose VICO, a co-evolutionary framework in which an actor and an Environment-as-Rewriter (EnvRewriter) are trained jointly: the EnvRewriter edits verifiable image-side structures, such as scene graphs, chart tables, or protected region masks, and re-renders them to produce label-valid training samples whose difficulty is calibrated to the actor's current ability through a pass-rate-based reward. This loop continuously realigns task difficulty with actor capability without any additional human annotation. Across nine multimodal benchmarks spanning mathematical reasoning and visually grounded understanding, VICO-8B improves over its base model by up to +5.0% on out-of-domain tasks, surpasses the strongest self-evolution and text-editing co-evolution baselines by +4.3% and +8.4% respectively, and stays comparable to chart-specialized RLVR methods using 16-160 times fewer labeled samples. By shifting from human-labeled supervision to image-editing co-evolution, VICO offers a scalable path beyond static-corpus RLVR for visual reasoning.

1 Introduction

Static visual tasks can become trivial or unsolvable as an actor improves, weakening the learning signal. VICO jointly adapts the visual environment and actor using verifiable edits and rollout feedback.

1 Introduction

  • Fixed tasks drift from the actor’s learning frontier: pass rates approach 1 or 0, GRPO reward variance collapses, and the learning signal weakens.The empirical pass rate already indicates task difficulty, but existing methods use it only for normalization or filtering.Source: §1 · p. 2
  • Useful visual edits must remain verifiable and align task difficulty with actor capability; human annotation meets both constraints but does not scale.Offline augmentation with stronger teachers reduces annotation cost but cannot track the actor’s evolving weaknesses.Source: §1 · p. 2
  • VICO jointly trains an actor and EnvRewriter, using rollout pass rates to reward edits near the actor’s learnable zone.Binary rollout rewards update the actor through GRPO, while a pass-rate-based curriculum reward updates the EnvRewriter.Source: §1 · p. 2
  • +15.5% on average over RLVR baselines across nine multimodal benchmarks, +4.3% over the strongest self-evolution baseline on out-of-domain tasks, and +2.5% over Gemini-2.5-Pro.VICO is also competitive with chart-specialized RLVR methods using 16×–160× fewer labeled samples.Source: §1 · p. 2

2 Related Work

Prior work reduces reliance on human-curated supervision through RLVR and self-evolution, but most approaches keep the visual task pool fixed. VICO instead co-adapts the visual environment with the actor.

2 Related Work

  • RLVR methods and self-evolution approaches reduce reliance on dense human rationales or costly supervision, but commonly assume a fixed pool of visual tasks.The cited work spans symbolic formalization, rule-based rewards, SFT-RL pipelines, efficient visual processing, problem generation, exploration, and world-model reasoning.Source: §2 · pp. 2–3
  • Self-evolving VLMs generate, solve, or refine training data, yet their actors evolve while the visual environment remains fixed.This leaves the training distribution increasingly misaligned with the actor’s current ability.Source: §2 · p. 3
  • Co-evolution methods vary tasks through planning, text perturbation, model-driven generation, or offline retrieval; VICO edits visual substructures and uses actor pass rates to calibrate difficulty.This couples environment updates to the actor’s learning trajectory rather than perturbing text or retrieving from a fixed image pool.Source: §2 · p. 3

3 Methodology

VICO treats visual reasoning training as a data-evolving curriculum: an EnvRewriter generates verifiable, difficulty-aligned samples as the actor learns. Shared rollout feedback updates both policies.

3 Methodology

  • VICO maintains an actor and an EnvRewriter, which produces edited images aligned with the actor’s current capability and targets the learnable zone.Unlike fixed-source RLVR, both policies co-evolve through the task difficulty shaped by the current EnvRewriter.Source: §3 · p. 3

3.1 Data-Evolving Paradigm: From Static RLVR to Adaptive Co-evolution

  • Each round’s training dataset is dynamically generated by the EnvRewriter from the actor’s current pass rate, creating a feedback loop around the actor’s breaking points.VICO contrasts this data-evolving paradigm with standard RLVR over a static dataset.Source: §3.1 · p. 3

3.2 Seed Visual Data and Verifiable Sample Generation

  • VICO edits structured or protected representations rather than generating unconstrained pixels, keeping samples label-valid through programmatic answer recomputation or constrained answer preservation.The seed data include 2,000 CLEVR, 2,000 ChartQA, and 1,000 ZwZ samples; the editing interface varies by available structure.Source: §3.2 · p. 4, Table 1 · p. 4

3.3 Rewards and Co-Evolutionary Optimization

  • The actor receives +1 for a correct, properly formatted answer and −1 otherwise; EnvRewriter updates favor pass rates near α = 0.5 and filter edits too far from that target.Actor updates use GRPO; EnvRewriter updates use Reward-Weighted Regression with a KL penalty to its initial policy and a maximum KL threshold.Source: §3.3 · p. 4

4 Theoretical Analysis of Difficulty-Calibrated VICO

The analysis explains why intermediate-difficulty edits provide strong actor learning signals and why a fixed 0.5 pass-rate target can become costly as the actor matures. A drift penalty shifts the optimal target toward the unedited pass rate.

4 Theoretical Analysis of Difficulty-Calibrated VICO

  • The actor’s gradient signal is strongest for intermediate-difficulty edits because p(1 − p) is maximized at p = 1/2.The analysis considers a binary-reward bandit with success probability p and a REINFORCE-style update under a variance-minimizing baseline.Source: §4 · p. 5
  • Forcing edited samples to pass at 0.5 can require increasingly aggressive edits as the actor’s unedited pass rate rises, moving training away from the natural image manifold.The analysis models this trade-off with a quadratic drift penalty.Source: §4 · p. 5
  • The optimal target is a convex combination of 0.5 and the unedited pass rate, moving toward the latter as learning headroom shrinks or drift cost increases.The analysis predicts diminishing per-round gains, fewer sweet-zone samples, and eventual saturation.Source: §4 · p. 5

5 Experiments

VICO is evaluated across nine multimodal benchmarks, with results covering out-of-domain performance, image-editing comparisons, and actor–environment co-evolution. Its strongest reported variant improves OOD accuracy, while experiments examine how the loop calibrates task difficulty and contributes to gains.

5.1 Experimental Setups

  • The experiments evaluate nine benchmarks, compare three categories of baselines, and train VICO on CLEVR, ChartQA, and ZwZ data.The tested backbones are Qwen3-VL-8B, Qwen2.5-VL-7B, and InternVL3-8B.Source: §5.1 · p. 5

5.2 Main Results

  • VICO-8B (Chart) reaches 64.9% OOD average accuracy, improving over its base model by +5.0%.The OOD average excludes ChartQA, the in-domain benchmark.Source: Table 2 · p. 6, §5.2 · p. 6
  • With the same CLEVR seed source, image editing improves the overall average by +6.1%, versus +2.4% for text edits.On ChartQA, image editing gains +9.4% versus +0.9% for text-only perturbation.Source: Table 4 · p. 7, §5.2 · p. 7
  • VICO surpasses MM-Zero by +4.3% on OOD benchmarks and V-Zero by +8.4% on overlapping benchmarks.VICO-8B also outperforms Gemini-2-Flash and Gemini-2.5-Pro by +5.2% and +2.5% on OOD benchmarks.Source: §5.2 · p. 7
  • Without math-specific training, VICO-8B variants improve over Qwen3-VL-8B by +5.5%, +5.4%, and +5.3% on five math benchmarks.VICO-Qwen2.5-VL-7B (Chart) and VICO-InternVL3-8B (Chart) improve over their base models by +4.7% and +7.1% on average.Source: §5.2 · p. 7
  • ChartQA accuracy rises from 76.9% to 89.1% by Round 4, with the largest gains on ratio/percentage (+29.8%) and value lookup (+20.1%).Figure 3 also reports smoothly increasing GRPO reward and actor responses growing by +49%.Source: Figure 3 · p. 7, §5.2 · p. 8

5.3 Co-Evolution Dynamics, Capability Emergence, and Ablations

  • The EnvRewriter makes easy ChartQA harder and hard ZwZ easier, bringing pass rates into [0.35, 0.65] within the first few rounds.Edit intensity increases from 0.22 to 0.51 while instruction length remains nearly unchanged (973 → 981 tokens).Source: §5.3 · p. 8, Figure 4 · p. 9
  • VICO-8B (Chart) reaches 89.12% on ChartQA, exceeding the strongest static method by +7.21%; co-evolution also beats Direct GRPO by +3.7% on average.Actor SFT alone gives +2.5%, while a vanilla EnvRewriter gives +1.7%.Source: Table 5 · p. 7, §5.3 · pp. 8–9, Table 6 · p. 8

6 Conclusion

VICO reframes visual reasoning post-training as actor–environment co-evolution: an EnvRewriter edits verifiable visual substructures toward the actor’s learning frontier. Across nine multimodal benchmarks, it outperforms strong baselines and generalizes to out-of-domain tasks while using fewer labeled samples.

6 Conclusion

  • VICO jointly trains an actor and an EnvRewriter that uses rollout pass rates to edit verifiable visual substructures toward the actor’s learning frontier.This produces difficulty-calibrated training samples without additional human annotation.Source: §6 · pp. 9–10
  • +4.3% on out-of-domain tasks over the strongest self-evolution baseline, while using up to 160× fewer labeled samples.Across nine multimodal benchmarks, VICO also generalizes to out-of-distribution tasks and remains stable across multiple VLM backbones.Source: §6 · p. 10
  • VICO’s results suggest that the fixed nature of the environment, rather than static-corpus size, is a bottleneck in visual reasoning post-training.The authors present difficulty-aligned visual editing as a scalable path where high-quality human annotation is costly.Source: §6 · p. 10

A Limitations, Future Work, and Impact Statement

VICO’s loop uses shared actor rollouts to train both the actor and EnvRewriter, while retaining label-valid edits. Its current scope depends on verifiable visual structures and answers, and co-evolution costs more compute than one-shot static augmentation.

A Limitations, Future Work, and Impact Statement

  • VICO relies on editable verifiable structures, such as scene graphs, chart tables, or protected region masks; domains lacking these or reliable region annotations remain an extension challenge.The framework’s current evaluation is limited to visual reasoning tasks with verifiable answers, excluding open-ended generation, dialogue, and real-time interactive environments.Source: A Limitations, Future Work, and Impact… · p. 14
  • Each round adds compute because multiple actor rollouts estimate pass rates and both the actor and EnvRewriter are updated, costing more than one-shot static augmentation.The authors note that this enables the training distribution to adapt to the actor’s evolving capabilities.Source: A Limitations, Future Work, and Impact… · p. 14
  • VICO’s proposed extensions include broader visual domains, adaptive difficulty targets, and more general-purpose image editors.Future tasks include richer visual grounding, multi-image comparison, diagram reasoning, and tool-assisted multimodal problem solving, which require editing higher-level task structure while preserving verifiable supervision.Source: A Limitations, Future Work, and Impact… · p. 14
  • Automatic image editing can introduce artifacts, biases, or label drift, so the authors say careful validation remains necessary before high-stakes use.They describe deterministic relabeling, answer-preserving constraints, and filtering as mitigations.Source: A Limitations, Future Work, and Impact… · p. 14

B.1 VICO: Algorithm Details

  • Each round generates edited tasks, evaluates actor rollouts, updates the actor with task rewards, and updates the EnvRewriter with difficulty-calibration rewards.The same rollout pass rate supplies the EnvRewriter’s difficulty signal, while the actor uses verifiable rollout rewards.Source: §B.1 · pp. 14–15
  • The actor is updated with GRPO, while the EnvRewriter uses reward-weighted regression on edits within a pass-rate band around the target.The EnvRewriter update weights retained edits by their difficulty-calibration reward and applies a KL penalty to limit drift.Source: §B.1 · p. 15
  • Edited samples remain label-valid through programmatic recomputation or answer-preserving constraints, and the loop needs no new human-labeled QA.Valid edited triples and weighted edit transitions accumulate in persistent actor and EnvRewriter pools across rounds.Source: §B.1 · p. 15

B.2 Actor GRPO Training Details

The appendix specifies rule-based actor training, two-stage EnvRewriter training, pass-rate-based difficulty rewards and filtering, a shared cross-domain recipe, and distributed compute implementation.

B.2 Actor GRPO Training Details

  • Actor rollouts use exact-match answer verification plus a format check, with temperature 0.7, top-p 1.0, and a 3,824-token response limit.ChartQA multi-round runs train each round on about 1,500–2,500 generated images plus 2,000 original samples.Source: §B.2 · pp. 15–16

B.3 EnvRewriter SFT and RWR Details

  • The EnvRewriter is format-aware SFT-initialized on verified edit demonstrations, then updated during co-evolution with reward-weighted regression over valid edit transitions.GPT-5 proposes structured edit commands, but labels and rewards come from domain-specific recomputation or answer-preservation checks and actor rollouts.Source: §B.3 · p. 16

B.4 Reward Functions and Difficulty Filtering

  • Actor reward is +1 for a format-correct, answer-correct response and −1 otherwise; EnvRewriter updates favor pass rates near α = 0.5.The difficulty reward uses β = 4.0, and the filter uses kmin = 0.15 to exclude poorly calibrated edits from EnvRewriter updates.Source: §B.4 · p. 16

B.5 Hyperparameters

  • The core hyperparameters are shared across visual domains, and retained RWR transitions are duplicated ⌊5 · w(p̂)⌋ times.The duplication count depends on the actor pass rate for the sample produced by that edit turn.Source: Table 8 · p. 17, Table 9 · p. 17

B.6 Compute and Hardware

  • Experiments use distributed training on up to 8×A100-80GB GPUs with a VERL-based implementation for GRPO and parallel rollout generation.The actor is optimized with reinforcement learning for multi-step visual reasoning.Source: §B.6 · p. 17, §B.5 · p. 17

C Data Sources and Verifiable Visual Editing

This section describes how VICO produces label-valid visual variants by recomputing answers or constraining edits to preserve them, filtering invalid edits and using actor pass rates to assess difficulty. It also explains how difficulty targets adapt to the actor’s remaining capacity and how sweet-zone hits change as the actor matures.

C.1 Seed Data Sources

C.3 CLEVR: Scene-Graph Editing

  • CLEVR edits graph attributes, categories, object counts, or relations, re-renders the scene, and executes the original question program to compute the new answer exactly.Edits that make a question ambiguous or non-executable are discarded.Source: §C.3 · pp. 18–19

C.4 ChartQA: Table-Level Editing and Re-rendering

  • ChartQA assigns unique answer-cell matches to Tier 1, where edited-table lookup updates answers; Tier 2 keeps values fixed and varies answer-preserving visual factors, with edited tables re-rendered as charts.Tier 2 includes cases with no unique cell match, often derived answers or duplicate values.Source: §C.4 · p. 19

C.5 ZwZ: Answer-Preserving Natural-Image Editing

  • For ZwZ, answer-relevant target objects are protected while only non-target regions are edited, keeping the original question-answer pair fixed; edits that alter evidence or introduce ambiguity are rejected.ZwZ uses answer-relevant region annotations to identify target objects.Source: §C.5 · p. 20

C.6 Filtering Invalid or Ambiguous Edits

  • Edits must be executable, preserve a valid answer, and provide useful difficulty for the Actor; invalid, ambiguous, unexecutable, or out-of-band edits are removed from EnvRewriter updates.Difficulty is assessed using the Actor’s empirical pass rate after rollouts.Source: §C.6 · p. 20

C.8 Robustness to Tool Failures and Difficulty Alignment as the Bottleneck

  • After filtering invalid edits, corrupting 20% of R1 training data reduced performance by less than one point.Invalid, label-breaking, or out-of-band edits contribute no gradient to the EnvRewriter update.Source: §C.8 · p. 21

D.3.1 What Happens without Answer Preservation?

  • Direct image generation changed a street-lamp relation answer from “left of” to “right of” and created ambiguity, while VICO removed distractors and preserved the answer-relevant objects and relation.Source: §D.3.1 · p. 23, §D.3.2 · p. 23

E.2 Distribution Drift Cost

  • Under standard assumptions, the Actor’s learning signal peaks at pass rate 0.5, but edits that force this target can increase distribution drift and threaten transfer to natural images.Source: §E.1 · p. 24, §E.2 · p. 25

E.3 Optimal Difficulty Target

  • The optimal pass-rate target interpolates between 0.5 and the actor’s current pass rate, weighted by drift cost relative to remaining capacity.The interpolation weight is ρ_t; greater drift cost relative to remaining capacity shifts the target toward the current pass rate.Source: §E.3 · p. 25

E.4 Two Limiting Regimes

  • With ample capacity, the target approaches 0.5 and calls for aggressive calibration; near saturation, it shifts toward the original pass rate, reducing edits as further editing can degrade performance.The saturation regime arises when edits needed to reach 0.5 have drift costs exceeding their learning benefit; the actor is then near its capacity ceiling.Source: §E.4 · p. 25

E.6 Empirical Saturation Evidence

VICO improves ChartQA accuracy through R4 before entering a drift-dominated saturation regime at R5. As the actor matures, useful pass-rate targets shift, sweet-zone edits become rarer, and per-round gains diminish.

E.6 Empirical Saturation Evidence

  • 89.12% ChartQA accuracy is reached at R4, followed by a slight retreat to 88.94% at R5 as co-evolution saturates.The table caption reports that gains shrink through R4 and turn slightly negative at R5.Source: §E.6 · p. 26, Table 13 · p. 26
  • The optimal pass-rate target shifts away from 0.5 as the actor strengthens; the observed rate moves from 0.502 at R3 to 0.375 at R5.Edit strength improves across rounds without lengthening EnvRewriter instructions.Source: §E.6 · p. 26, Figure 10 · p. 27
  • Edits landing in the [0.35, 0.65] trainable band fall from 53.4% at R1 to 11.1% at R5, weakening the available gradient signal.Edits continue at the same rate, but fewer land in the trainable band.Source: §E.6 · p. 26
  • Round-over-round gains decline from +8.34 to +2.26, +0.97, +0.65, and −0.18, while value_lookup and ratio_pct regress first at R5.The analysis identifies these categories as having gained most through R4 and therefore having less remaining headroom.Source: §E.6 · p. 26

E.7 Pass-Rate Distribution Shift

  • Pass rates cluster near α = 0.5 early, then shift toward the extremes, explaining fewer sweet-zone hits and diminishing returns.The fixed [0.35, 0.65] band is no longer aligned with the moving optimal target by R5; future work could adapt α or widen the band.Source: §E.7 · p. 27, §E.6 · p. 27

F.2 Answer Normalization and Scoring

  • Scoring normalizes trivial answer-format differences and uses ChartQA’s relaxed numerical protocol; rollout answers must be reliably parseable.The EnvRewriter prompt requests executable chart edits toward the learnable zone, while evaluation exposes only the test image and question.Source: §F.2 · p. 27, §F.3 · p. 27, §F.4 · p. 27

F.1 Evaluation Benchmarks and Metrics

The evaluation covers mathematical reasoning and vision-centric understanding, with comparisons against RLVR, self-evolution, and co-evolution baselines. VICO’s improvements extend across tested VLM backbones, and co-evolution requires no additional human annotation.

F.1 Evaluation Benchmarks and Metrics

  • Evaluation uses six mathematical-reasoning benchmarks and three vision-centric benchmarks, with the main-table average equally weighting the two benchmark-group averages.The mathematical set includes MathVista, MathVision, MathVerse, WeMath, LogicVista, and ChartQA; the vision-centric set includes MMMU, MMVP, and BLINK.Source: §F.1 · p. 27

F.6 Baseline Implementation Details

  • Baselines span RLVR-trained reasoning, self-evolution, and co-evolution without visual-environment editing; in-house comparisons use the same actor backbone and evaluation protocol.Published results are used when available under comparable settings.Source: §F.6 · p. 28

G.1 Full OOD Average Calculation

  • For the Chart-domain variant, the out-of-distribution average excludes ChartQA because it is the in-domain visual source.CLEVR and ZwZ variants retain the same benchmark list for consistency.Source: §G.1 · p. 28

G.2 Cross-Backbone Results

  • VICO improves ChartQA accuracy across both tested VLM backbones, indicating that the co-evolution loop is not tied to one model family.Per-round ChartQA accuracy rises for both models.Source: §G.2 · p. 28, Figure 11 · p. 29

G.3 Cost and Annotation Efficiency

  • Co-evolution needs no additional human-written QA pairs: edited representations yield training pairs programmatically, while seed images need no new labels.The reported Qwen3VL-8B actor-training run through R4 costs approximately 305 A100 GPU hours, with trajectory generation adding approximately 40 hours.Source: §G.3 · p. 29

G.4 Comparison with Chart-Specialized Methods

VICO reaches competitive ChartQA accuracy using seed visual inputs and no additional labeled QA pairs. The comparison emphasizes reduced labeling requirements relative to chart-specialized approaches.

G.4 Comparison with Chart-Specialized Methods

  • 89.12% ChartQA accuracy is reached by VICO-Qwen3VL-8B using seed inputs and no additional labeled QA pairs, while competing with methods using tens or hundreds of thousands.The comparison uses ChartQA’s official relaxed accuracy protocol with 5% numerical tolerance.Source: §G.4 · p. 29, Table 15 · p. 30
  • VICO avoids additional labeled QA supervision and uses substantially fewer seed inputs than chart-specialized SFT/RL approaches.The table caption states that chart-specialized baselines require tens or hundreds of thousands of labeled pairs.Source: §G.4 · p. 29, Table 15 · p. 30
  • The authors argue that VICO can scale to new domains without proportional annotation effort, especially where labeled QA is scarce.Source: §G.4 · p. 30

H.1 EnvRewriter SFT as a Co-evolution Catalyst

The comparisons favor combining EnvRewriter supervised initialization with iterative co-evolution, while also testing offline augmentation, editing modality, robustness, seed stability, initial difficulty, and co-adaptation. Together, they show stronger visual editing and actor-relative difficulty calibration, with co-adaptation eventually reaching a capacity ceiling.

H.1 EnvRewriter SFT as a Co-evolution Catalyst

  • 89.1% on ChartQA (12.2 percentage points above base) follows EnvRewriter SFT plus co-evolution, exceeding direct Actor SFT (79.4%) and vanilla-EnvRewriter co-evolution (78.6%).The initialization demonstrations supervise both command format and editing strategy; the comparisons do not isolate their separate contributions or GPT-5 supervision.Source: §H.1 · p. 30

H.2 Static Offline Augmentation Details

  • Static offline augmentation improves over the base model, but remains below iterative visual co-evolution, indicating an advantage for the dynamic feedback loop over one-shot data generation.Source: §H.2 · p. 30

H.3 Text-Only Editing vs. Visual Editing

  • Visual editing produces larger gains than text-only perturbation under the same CLEVR source, especially on chart reasoning.Text-only perturbation changes questions or answer format while leaving the visual scene fixed.Source: §H.3 · p. 31

H.4 Robustness to Imperfect Image Editing

  • 66.8% to 66.0% is the average-performance change after corrupting 20% of R1 data, and the noisy model remains +2.4% above the base model.The corruption combines blank images and unedited images paired with edited labels.Source: §H.4 · p. 31

H.5 Stability Across Random Seeds

  • 0.4 points on ChartQA and 1.1 points on MathVista are the final-accuracy standard deviations across three first-round seeds, supporting single-seed reporting in the main text.The analysis focuses on Round 1 because later rounds use EnvRewriter-generated samples rather than the initial seed distribution.Source: §H.5 · p. 31, Table 16 · p. 31

H.6 Robustness to Initial Data Difficulty

  • Easy initial seeds can leave little room for improvement, while hard seeds can stall fixed-distribution methods; VICO instead pushes both toward a pass rate near 0.5.The reported convergence from both directions suggests gains are not tied to a specially curated starting difficulty.Source: §H.6 · p. 31

I.1 Bidirectional Difficulty Calibration

  • On ChartQA, the EnvRewriter lowers a strong Actor’s pass rate; on ZwZ, it raises a weak Actor’s, with both moving toward the [0.35, 0.65] sweet zone.Source: Table 17 · p. 32, §I.1 · p. 32

I.2 Editing Intensity and Response Length

  • Editing intensity rises from 0.22 to 0.51 while EnvRewriter instructions stay near 973–981 characters; Actor responses grow from 811 to 1207 characters (+49%).Per-round gains decline from +8.34% to −0.18%, and sweet-zone hit rate falls from 53.4% to 11.1% as the EnvRewriter struggles to keep pace with the Actor’s capacity ceiling.Source: §I.2 · p. 32, Table 18 · p. 32

I.3 Sample-Level Flip Analysis

On ChartQA, VICO’s Round 4 gains are characterized by more wrong-to-correct than correct-to-wrong sample flips, with newly solved samples often unlocking early and remaining stable. The accompanying analyses describe reasoning-heavy question types and stepwise chart-reading behavior in corrected cases.

I.3 Sample-Level Flip Analysis

  • Wrong-to-correct flips dominate correct-to-wrong regressions on the ChartQA test set, indicating gains are not merely error redistribution.Source: §I.3 · p. 32, Table 19 · p. 33
  • Newly solved samples often unlock early and remain correct, with R1 unlocks covering the largest fraction of cases.Source: §I.3 · p. 33

I.4 Question-Type Breakdown

  • VICO’s larger gains concentrate on reasoning-heavy ChartQA types such as ratio/percentage, value lookup, and difference, which require multi-step reading and computation.Source: §I.4 · p. 33

I.5 Chain-of-Thought Case Studies

  • In 20 randomly sampled base-wrong to R4-correct cases, the trained model more often decomposes chart reasoning into region identification, axis reading, and computation.The base model tended toward direct, single-step answers that missed key constraints.Source: §I.5 · p. 33

J Preliminary Experiments on Difficulty-Aware Visual Editing

This section presents preliminary experiments on difficulty-aware visual editing, including whether online edits calibrate task difficulty and how valid actor-conditioned edits are motivated. It also covers VICO asset documentation and release, core VLM use, offline augmentation baselines, and reproducibility details, limitations, and human-subject status.

J Preliminary Experiments on Difficulty-Aware Visual Editing

  • Early online Rewriter-GRPO edits often remained too easy or too hard, and weak reward differentiation motivated verified edit transitions that separate validity from actor-measured difficulty.The preliminary reward and optimization procedure differs from the final method, which uses domain-specific validity rules and RWR updates.Source: J Preliminary Experiments on… · pp. 33–34

K Case Study: Co-Evolution Matters More Than Image-Generation Strength

  • VICO’s case study argues that noise or unconstrained generation can impair reasoning difficulty or label validity, while strong generators still need a dynamic actor-conditioned policy to select valid edits.Validity rules recompute CLEVR answers, constrain ChartQA answer-bearing values, and protect answer-relevant regions in ZwZ.Source: K Case Study: Co-Evolution Matters… · p. 35

NeurIPS Paper Checklist

  • The checklist reports reproducibility details across models, datasets, benchmarks, rewards, rollouts, and prompts, but notes that results lack error bars, confidence intervals, or significance tests.The authors plan to include repeated-run statistics in future versions.Source: NeurIPS Paper Checklist · pp. 37, 39
  • The checklist identifies reduced reliance on human annotation as a potential benefit, while warning that edits may introduce artifacts, label drift, or bias and recommending verification and filtering.Source: NeurIPS Paper Checklist · p. 40
  • VICO’s pipeline, EnvRewriter prompts, editing procedures, generated samples, and training trajectories are documented and will be released with accompanying documentation.The documentation includes editing protocols, reward functions, filtering rules, hyperparameters, implementation details, and prompt templates.Source: NeurIPS Paper Checklist · p. 41
  • The paper reports no crowdsourcing or human-subject research, making IRB approval or equivalent review inapplicable.Source: NeurIPS Paper Checklist · p. 42
  • The Actor and EnvRewriter use VLM backbones and train jointly; GPT-5 and Gemini (Nano Banana) serve as offline augmentation baselines.The paper reports their model backbones, prompts, reward functions, and training settings.Source: NeurIPS Paper Checklist · p. 42

Related papers

Loading 2610.10782v1…