PaperlayerView exact revision on arXiv

Source-linked AI summary

Scalable Minimal-Change Learning for Controllable Image Editing

Shuo Chen, Fengming Huang, Yu Yao, Mingming Gong, Tongliang Liu

arXiv:2610.06021v1cs.CVcs.AI

TL;DR

Instruction-based editors can change image content beyond an instruction’s scope. The paper optimizes edit outcomes with reinforcement learning and an agentic VLM audit of missed and unintended changes. On FLUX.1 Kontext-dev, average EditScore rises from 5.21 to 5.88 across four benchmarks, while masked off-target pixel change falls by 8.4%.

  • Problem

    Instruction-based editors can unintentionally alter image content outside the attributes specified by an edit, despite the goal of preserving everything else.

    Source: §1 · p. 1

  • Method

    ARRO uses outcome-level RL and an agentic VLM to score missed and unintended edits, with a group-level rubric aligning candidates to canonical issues.

    Source: §1 · pp. 2–3

  • Results

    Average EditScore increases from 5.21 to 5.88 across four benchmarks, and masked off-target pixel change is 8.4% lower.

    Source: §5 · p. 10

  • Takeaways & Limitations

    The results support supervising intervention effects with structured rewards and direct outcome optimization.

    Source: §1 · p. 3, §5 · p. 10

    Reward quality is bounded by VLM perception, especially for subtle differences, ambiguous or compositional instructions, and candidate-specific issue assignment.

    Source: §5 · p. 10

Abstract

from arXiv · show

Image editing should change only the attributes specified by an instruction while preserving everything else, yet current methods often make unintended changes. We treat this minimal-change principle as an optimization objective for instruction-based editing. Latent L1 regularization is a poor proxy for output locality in modern nonlinear generators and often requires supervision unavailable at scale. We instead optimize edit outcomes with reinforcement learning. An agentic vision-language reward model audits each source image, instruction, and edited image for two failure types: unimplemented requested changes and unintended changes. A group-level rubric merges and verifies these issues to provide consistent rewards across candidate edits without per-instruction human annotations. On FLUX.1 Kontext-dev, ARRO raises average EditScore from 5.21 to 5.88 across MinEval, MagicBrush, AnyBench, and Emu-Edit. On 600 evaluation examples, it reduces off-target pixel change by 8.4% relative to the base editor. Reward and SFT controls, blinded human evaluations, and transfer to OmniGen2 provide complementary evidence. Code: https://github.com/Showwwwwwwww/ARRO

1 Introduction

Instruction-based editors can make broad unintended changes, while minimal change has not been explicitly optimized in modern image editing. ARRO instead rewards edited outcomes through structured audits, improving benchmark EditScore and reducing measured off-target change.

1 Introduction

  • Instruction-based editors can alter hair, skin, backgrounds, or lighting beyond the requested edit, undermining their use as controllable tools.The paper identifies minimal change—altering only intended attributes while leaving others invariant—as an under-explored optimization objective.Source: §1 · pp. 1–2
  • Latent L1 regularization may leave dense edits, sparse latent changes may still spread through nonlinear decoders, and required attribute labels or counterfactual pairs are costly or unavailable.These are the paper’s three obstacles to adapting latent-space minimal-change methods to modern editors.Source: §1 · p. 2
  • ARRO reformulates minimal change as reinforcement learning on edit outcomes, using source–instruction pairs and rewards computed directly on edited images.This avoids optimizing an L1 proxy in latent space and addresses decoder nonlinearity at the output level.Source: §1 · p. 2
  • An agentic VLM audits source image, instruction, and edit for unimplemented and unintended changes, then a shared deduplicated rubric stabilizes candidate rewards.The rubric is formed from group audit lists and used in a second-pass evaluation of each candidate.Source: §1 · p. 3
  • Average EditScore rises from 5.21 to 5.88 across four benchmarks, while masked off-target pixel change falls by 8.4% on 600 examples.Reward and SFT controls, blinded human comparisons, and transfer to OmniGen2 provide complementary evidence.Source: §1 · p. 3
  • The authors introduce MinEval, with automatically generated prompts and source images spanning atomic changes across six edit types and disjoint training and held-out evaluation splits.The dataset is designed to support research into controllable image editing.Source: §1 · p. 3

2 Related Work

Related work addresses image-editing locality through post-training rewards, latent constraints, learned representations, and safety methods. ARRO instead derives instruction-specific, decomposed rewards from candidate edits through a zero-shot VLM audit.

Instruction-based Image Editing

  • Prior post-training methods use preference optimization or online RL with learned reward models, while the authors argue human-preference rewards can conflate aesthetics with adherence.Their zero-shot agentic VLM reward is presented as avoiding expensive learned reward-model training.Source: §2 · p. 3

Flow Matching and Sampling Efficiency

  • Flow-matching work addresses transport, diversity, or sampling cost, whereas ARRO targets the scope of changes made by instruction-conditioned editing.Source: §2 · pp. 3–4

The Minimal Change Principle

  • Latent-space approaches operationalize minimal change through structured representations or training-free constraints, but ARRO treats latent locality as a brittle proxy and audits output changes directly.The paper frames its approach as output-space visual auditing rather than constraining latent representations.Source: §2 · p. 4

Task-specific representations and metrics

  • EditCLIP, NearID, and StyleID learn task-specific representations or metrics, whereas ARRO constructs instruction-specific decomposed rewards without per-instruction human labels.The authors do not claim these representation-learning tasks are interchangeable with online editing rewards.Source: §2 · p. 4

Safety and Asset Protection

  • Related safety and asset-protection work studies misuse or unauthorized modification, while this paper focuses on preserving untargeted content during authorized edits.Source: §2 · p. 4

Agentic VLM Rewards

  • Where standard VLM rewards provide scalar scores and recent work uses structured reasoning or checklists, ARRO audits unimplemented versus unintended changes for granular locality rewards.The paper positions this design at the intersection of agentic VLMs and reinforcement learning.Source: §2 · p. 4

3 Method

ARRO frames minimal change as output-space reinforcement learning: edits should realize the instruction while preserving other aspects, with rewards tracking missing and out-of-scope changes.

3.1 Preliminaries

  • FLUX.1 Kontext is fine-tuned with GDPO, which normalizes separate reward channels within each candidate group before combining their advantages.The flow-matching loss is not used during fine-tuning.Source: §3.1 · p. 5

3.2 Agentic Reward Model via Chain-of-Thought Auditing

  • For each source image, instruction, and candidate edit, the VLM audits what was not implemented and what changed outside the instruction’s scope.It returns descriptive lists rather than a continuous numerical score.Source: §3.2 · pp. 5–6

3.3 Rubric-Refined Reward

  • An LLM consolidates group-wide audit lists into canonical rubrics, which the VLM uses to verify each candidate and produce comparable counts for both failure modes.Shared rubrics address duplicate, fragmented, or inconsistently granular raw list entries; fewer verified issues yield higher rewards.Source: §3.3 · pp. 6–7

3.4 Learning Objective

  • The two channels are normalized separately within groups, combined by weighted sum, and batch-renormalized to produce the advantage used in the GDPO update.Separate normalization prevents a channel from dominating solely because of greater raw variance; zero-variance groups on both channels are skipped at the gradient step.Source: §3.4 · p. 7

4 Experiments

Across four benchmarks, ARRO improves average EditScore and reduces off-target pixel change; experiments also examine edit-type generalization, design ablations, human judgments, and transfer to OmniGen2.

4.1 Setup

  • The evaluation spans held-out MinEval, MagicBrush, AnyBench, and Emu-Edit, using EditScore, HPSv2, ImageReward, and masked OffTarget-L1.The benchmarks cover six held-out atomic edit types, human-curated edits, 18 edit types, and out-of-distribution robustness, respectively.Source: §4.1 · p. 8

4.2 Implementation Details

  • FLUX.1 Kontext-dev (12B) is fine-tuned for 230 steps with MixGRPO; reward generation for 16 candidates takes 16.06 seconds under the reported serving configuration.The reward-generation timing is not end-to-end training throughput.Source: §4.2 · p. 8

4.3 Main Results

  • 5.88 vs. 5.80 average EditScore: ARRO leads the four-benchmark average, though BAGEL leads on MinEval and MagicBrush and ARRO leads on AnyBench and Emu-Edit.The comparison uses an unweighted average across benchmarks that differ in size by about 6×.Source: §4.3 · p. 8, Table 1 · p. 9
  • 8.4% relative reduction: ARRO lowers OffTarget-L1 from 0.20366 (Base) to 0.18659 on a 600-example masked evaluation.This measures pixel change outside the designated target mask, not whether pixels inside the mask were correctly edited.Source: §4.3 · p. 8
  • 16/18 categories: ARRO beats the un-tuned base on AnyBench under both EditScore and HPSv2 despite training on six atomic edit types.The reported largest EditScore gains include movement (+1.58), implicit_chg. (+1.30), and resize (+1.02).Source: §4.3 · pp. 8–9, Table 2 · p. 9

4.4 Ablations

  • Ablations identify distinct shortcuts: merged scalar rewards lose about 1.5 EditScore, efficacy-only over-edits, and locality-only reward-hacks to high-frequency noise.The dual-channel design addresses both over-editing and empty outputs; blinded checks confirmed correct merging for 560/565 issues, and ARRO transfer improved EditScore on three unseen OmniGen2 benchmarks.Source: §4.4 · p. 10

5 Conclusion

ARRO optimizes minimal change through outcome-level reinforcement learning, improving editing scores and reducing off-target pixel change. Its reward quality and evaluation scope have limitations, and greater controllability may facilitate deceptive manipulation.

5 Conclusion

  • EditScore rises from 5.21 to 5.88 across four benchmarks, while masked off-target pixel change falls by 8.4% on 600 evaluation examples.Reward and SFT controls, blinded human evaluations, and transfer to OmniGen2 provide complementary evidence.Source: §5 · p. 10
  • Reward quality is bounded by VLM perception, and OffTarget-L1 depends on segmentation and measures pixel change rather than full semantic correctness.The expert study validates the evaluated configuration, not all possible auditors.Source: §5 · p. 10
  • More controllable editing could facilitate deceptive image manipulation, so deployment requires provenance and misuse safeguards beyond the studied reward mechanism.Transfer to diffusion backbones and multi-seed variability are not established or quantified.Source: §5 · p. 11

A Related Work (Detailed)

Prior editing work treats locality as emergent, relies on preference rewards or latent constraints, or targets narrow domains. The appendix describes a group-normalized, weighted multi-channel advantage for flow-based policy optimization.

A Related Work (Detailed)

  • Pretrained editors treat locality as an emergent property rather than optimizing it directly, while benchmarks report background leakage, identity shifts, and unintended attribute changes.Post-training rewards based on human preference mix visual quality, instruction adherence, aesthetics, and plausibility rather than specifically penalizing out-of-scope edits.Source: A Related Work (Detailed) · p. 15
  • The reward uses an off-the-shelf VLM to audit edits and provide two minimal-change components without preference data or reward-model training.The cited reward models require tens to hundreds of thousands of annotated preference pairs and their own training pipelines.Source: A Related Work (Detailed) · pp. 15–16
  • Causal-representation methods structure latent spaces around causal graphs and use weak labels or counterfactual supervision to target individual factors.The paper argues that latent constraints are brittle proxies because nonlinear decoding can disrupt output locality.Source: A Related Work (Detailed) · p. 16
  • Prior image-level counterfactual and StyleGAN editing methods achieve locality on narrow domains with attribute labels or counterfactual pairs, but face obstacles when scaled to free-form instructions.This work instead measures minimal change on generated images and optimizes it end-to-end via an agentic reward.Source: A Related Work (Detailed) · p. 16

B Optimization Details

  • GDPO normalizes each reward channel within a candidate group, combines channel advantages with positive weights, then renormalizes across the batch to stabilize scale.The resulting advantage drives a per-step PPO-clip update along an SDE discretization of rectified flow.Source: B Optimization Details · pp. 16–17
  • MixGRPO samples candidates through an SDE discretization of rectified flow over denoising steps, and the policy update uses PPO-clip.The batch-normalized advantage and clip range are distinguished from the stability offset and noise variable.Source: B Optimization Details · p. 17
  • The two reward channels are unimplemented and unintended change counts, whose group-level advantages are combined to drive the flow-based update.The appendix instantiates the GDPO pipeline with two channels.Source: B Optimization Details · p. 17

C.1 Additional Validation

Additional validation compares ARRO rewards with alternative reward and selection strategies, tests off-target pixel change and evaluator behavior, and clarifies the scope of these measurements.

Off-target pixel modification

  • OffTarget-L1 measures mean absolute pixel difference outside a SAM 2 target-object mask on 600 randomly selected evaluation examples.The mask covers the region expected to change; the metric depends on mask quality and image alignment and does not measure semantic correctness.Source: §C.1 · p. 17

Reward construction and offline selection

  • Without aggregation, four-benchmark average ES falls from 5.880 to 4.961; HPSv2 falls from 0.2220 to 0.2101, and ImageReward from −0.5100 to −0.7672.The variant retains verification but does not consolidate semantically overlapping issues.Source: §C.1 · p. 18
  • On the separate 600-example MinEval evaluation in Table 7, full ARRO-GDPO achieves EditScore (ES) 5.6690, whereas the four-benchmark experiment reports MinEval ES 5.73.Top-10 SFT selects the highest-reward 10% of MinEval candidates under ARRO or EditScore reward; EditScore-Top1-SFT instead uses the highest-EditScore candidate in each source–instruction group.Source: §C.1 · pp. 17–18, Table 7 · p. 18

EditScore rewards under fixed GDPO

  • On MagicBrush, AnyBench, and Emu-Edit, average ES is 3.848 for EditScore(PF, Cons)-GDPO and 5.927 for ARRO.These three-benchmark means exclude MinEval and are distinct from the four-benchmark averages.Source: §C.1 · p. 18

Controlled test of intended versus unintended change

  • In the CLEVR diagnostic, verified ARRO counts remain zero as requested recolorings increase from 5 to 50, while EditScore Consistency varies despite zero ground-truth unintended changes.Each condition has five samples; the test concerns intended edits, not all natural images.Source: Table 9 · p. 18, §C.1 · p. 18

Evaluator scale and reward-generation cost

  • For groups of 16 candidates, reward generation takes 16.06 seconds and processes 61,950 tokens on the stated hardware and server configuration.The pipeline uses 2N + 1 inference passes; the reported latency is not end-to-end training throughput.Source: §C.1 · p. 19
  • A separate evaluator-scale diagnostic supports a cost–accuracy trade-off only for a controlled CLEVR count task, not full editing-model training.It uses 20 outputs across five scenes and four ground-truth off-target-modification counts.Source: §C.1 · p. 19

Motivation and scope of MinEval

  • MinEval uses atomic objectives with multiple valid outputs and grouped candidates assessed separately for intended and unintended changes, supporting RL comparisons and candidate filtering.Disentangled-representation-learning uses remain prospective rather than experimentally established.Source: §C.1 · p. 19

C.2 Human Evaluation

The section reports blinded expert checks of reward issues and pairwise judgments of model outputs, with randomized presentation and a bounded example base.

Expert validation of issue judgments

  • Two independent reviewers examined 960 candidate images from 60 MinEval groups and checked 565 final aggregated issues for support, merging, and missing evident failures.Groups were balanced across six edit types, and reviewers did not see model identities, reward values, or reward-generation procedures.Source: Table 11 · p. 19, §C.2 · p. 19

Blinded pairwise preference

  • Ten independent students made 600 judgments per comparison across 60 examples, whose identities and left/right presentation were hidden or randomized.The 600 judgments share 60 underlying examples and are not 600 independent image pairs.Source: §C.2 · pp. 19–20
  • The pairwise prompt asked for overall editing quality rather than a separate locality-only preference.Evaluators could choose the better edit or a tie.Source: §C.2 · p. 20

C.3 Transfer to OmniGen2

The appendix reports ARRO transfer to OmniGen2 on unseen benchmarks and breaks down EditScore’s component scores to contextualize the overall metric.

C.3 Transfer to OmniGen2

  • Adapting ARRO to OmniGen2 improves results on three unseen benchmarks, demonstrating transfer to a second rectified-flow editor but not validating diffusion backbones.The adaptation uses the same Audit–Aggregation–Verification reward construction and rejection sampling.Source: Table 12 · p. 20, §C.3 · p. 20

C.4 EditScore Sub-Scores

  • ARRO’s four-benchmark prompt-following average rises from 6.60 to 7.35, while consistency changes from 8.36 to 8.31; Overall combines prompt following, consistency, and perception quality.The passage attributes the Overall improvement to better instruction execution rather than larger, more disruptive edits.Source: §C.4 · p. 20

C.5 Per-Benchmark HPSv2 and ImageReward

The appendix supplements EditScore results with per-benchmark HPSv2 and ImageReward comparisons and per-edit-type breakdowns on AnyBench.

C.5 Per-Benchmark HPSv2 and ImageReward

  • ARRO is highest on HPSv2 on every benchmark, tied with InstructPix2Pix on MinEval, and highest on ImageReward on every benchmark.The accompanying tables report per-benchmark HPSv2 and ImageReward values alongside the EditScore Overall results.Source: §C.5 · p. 20, Table 14 · p. 21

C.6 Per-Edit-Type Breakdown on AnyBench

  • The AnyBench tables break out HPSv2 and ImageReward by edit type across 50 prompts per type, using the same methods and best/second-best markings as the EditScore table.These breakdowns complement the main-paper EditScore per-edit-type table.Source: §C.6 · p. 21

D Training Settings

All FLUX.1 Kontext-dev runs use a shared training configuration, while ablations change only their specified reward components.

D Training Settings

E Training Data Construction

This section describes how training data is constructed from validated instructions and base-editor candidates, with prompts auditing edit failures and consolidating findings into consistent rubrics. It also explains how visual verification applies separate rubrics to unimplemented requested changes and unintended changes.

E Training Data Construction

  • The training-data pipeline synthesizes atomic instructions, filters them through a structured VLM audit, then executes them with the untuned base editor and rejects degenerate candidate-score distributions.Source: E Training Data Construction · p. 23

E.1 Atomic Edit Types

  • The pipeline constrains instructions to six atomic edit types with fixed patterns and allowed verbs, rejecting verb/type mismatches before VLM review.The types are add, remove, move, replace, appearance_change, and action_change.Source: §E.1 · p. 23

E.2 Instruction Generation and Validation

  • Instruction generation uses validation of drafts or fresh self-audited proposals, with up to five validation rounds before parallel generation at five temperatures; failed instructions are discarded.Acceptance requires all checklist checks to pass and a non-empty instruction after a pipeline-level verb/type recheck.Source: §E.2 · pp. 23–24

E.3 Per-edit-type guidance injected into Pval and Pgen

  • Across add, remove, move, replace, appearance, and action edits, type-specific guidance makes requests verifiable and guards against side effects such as displacement, residual objects, or unintended attribute changes.The guidance also requires explicit targets and states, including source and destination for moves and a visually verifiable, species-appropriate action.Source: §E.3 · pp. 25–26

E.4 Edit-Level Rejection Sampling

  • N=16 base-editor candidates are sampled per instruction, and filtering retains about 10K triplets after removing groups with degenerate reward variation.The default filter drops groups where most candidates fully implement the instruction or where unintended-change counts are constant; the retained triplets form the RL training set.Source: §E.4 · pp. 26–27

F Prompt Templates

  • All three model-invocation templates use fixed JSON schemas with input placeholders, which are parsed to obtain the required outputs.Source: F Prompt Templates · p. 27

F.1 Audit prompt Paudit

  • The audit prompt compares source and edited images against the instruction and lists unimplemented requests separately from visible, out-of-scope changes.It asks for visually verifiable issues rather than numerical scores or speculation.Source: §F.1 · pp. 27–28

F.2 Aggregation prompt Pagg

  • The aggregation prompt filters requested changes from unintended issues and consolidates each failure mode into a separate, deduplicated checklist of atomic, visually checkable items.Source: §F.2 · p. 28
  • Aggregation produces separately identified rubrics of neutral, candidate-checkable unimplemented and unintended issues.The two issue types are not merged.Source: §F.2 · p. 28

F.3 Verification prompt Pverify

  • For a candidate group of N edits, reward generation requires 2N + 1 calls: N audits, one aggregation, and N verifications.Verification is invoked once per candidate.Source: §F.3 · p. 29
  • Verification parses the instruction and compares source and edited images, then marks rubric issues true only when visibly present, without adding or rewording issues.It marks unimplemented aspects true when absent, partial, or incorrect, and unintended changes true when visible; when uncertain, it marks false.Source: §F.3 · p. 29

Related papers

Loading 2610.06021v1…