PaperlayerView exact revision on arXiv

Source-linked AI summary

ACEdit: Stable Multi-Turn Image Editing via Asymmetric Conditioning

Sara Dorfman, Nir Goren, Gal Barak, Daniel Cohen-Or, Or Patashnik

arXiv:2610.22328v1cs.CVcs.AIcs.GR

TL;DR

Multi-turn image editing can accumulate distribution shifts that lead to artifacts and loss of detail. ACEdit trains an editor on its own successive outputs while a teacher remains anchored to the original clean image and cumulative instructions. The authors report reduced drift and improved editing capability, while noting that very long sequences and difficult instructions remain challenging.

  • Problem

    Repeated editing shifts the input distribution away from the editor’s training distribution, leading to artifacts, detail loss, and unintended appearance changes.

    Source: §1 · p. 1

  • Method

    The student edits its own prior outputs and receives score-distillation guidance from a teacher conditioned on the original image and compound instructions.

    Source: §1 · p. 2

  • Results

    The method reports significant qualitative and quantitative improvements in preserving editing capabilities while mitigating accumulated distribution drift.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors suggest that self-forcing with clean-reference teacher supervision may also be useful for other generative tasks that repeatedly operate on their own outputs.

    Source: §6 · p. 11

    Particularly difficult instructions may still affect previously edited content, and extending robustness to much longer sequences remains future work.

    Source: §6 · p. 11

Abstract

from arXiv · show

Image editing models produce impressive results for individual edits, but their quality often deteriorates over multiple editing turns. Each editing turn introduces small distribution shifts that accumulate over successive edits, creating a gap between the input distributions encountered during the editor's original training and multi-turn inference. We introduce a training scheme that makes multi-turn image editing stable under this accumulating distribution shift. A key is to force the editor to operate on its own outputs from previous editing turns during training, bringing the principle of Self-Forcing to image editing. The main challenge is that self-generated editing trajectories deviate from ground-truth trajectories, making direct supervision unreliable. We address this challenge through a distillation objective that leverages a teacher image editor to provide a reliable training signal when the model operates on its own previous outputs. We introduce an asymmetric conditioning scheme: while the student performs edits sequentially on its own outputs, the teacher remains anchored to the original clean image and receives a compound editing instruction. Experiments show substantially reduced degradation over long multi-turn sequences while maintaining editing quality and instruction adherence.

1. Introduction

Multi-turn editing accumulates distribution shifts that can cause artifacts, detail loss, and unintended appearance changes. ACEdit trains on self-generated editing trajectories and uses a clean-conditioned teacher to guide them, reporting improved preservation of editing capabilities while mitigating drift.

1. Introduction

  • Accumulating distribution shifts create artifacts, loss of fine details, and unintended appearance changes across successive edits.Stability across turns matters in addition to the quality of individual edits.Source: §1 · p. 2
  • Training on ground-truth sequences leaves a training–inference gap, while self-generated sequences create supervision challenges.Prior self-generated-sequence methods can have inconsistent targets, restricted training sequences, or objectives that do not explicitly address distribution shifts.Source: §1 · p. 2
  • The method trains the editor on its own previous outputs so it learns from the shifts encountered during multi-turn inference.Self-generated trajectories deviate from ground-truth trajectories, making direct ground-truth supervision unreliable.Source: §1 · p. 2
  • Score distillation uses a teacher–student framework to train on self-generated trajectories without ground-truth targets for intermediate outputs.The teacher receives the original image and compound instructions, while the student edits sequentially on its own outputs.Source: §1 · p. 2
  • The method reports qualitative and quantitative improvements in preserving editing capabilities while mitigating accumulated distribution drift.The authors also report efficient training with no inference-time overhead, no assumptions about shift type or restrictions on editing sequences, and ablation evidence for asymmetric conditioning.Source: §1 · p. 2

2. Related Work

Prior work spans single-step editing, inference-time controls, sequential training, and reward-based approaches, but iterative editing can still propagate errors and degrade identity. ACEdit follows evolving semantic edits and aligns student outputs with a clean-conditioned teacher distribution.

2. Related Work

  • Single-step editing capabilities do not prevent error propagation, identity degradation, or semantic drift over extended iterative sequences.Prior text-guided editing work includes GAN latent steering, diffusion and flow controls, and instruction-tuned architectures.Source: §2 · p. 2
  • Multi-turn approaches include planning and memory, training-free consistency controls, sequential editing models, and trajectory-level reinforcement learning.MT-OPSD distills clean-conditioned behavior onto states generated through identity-edit rollouts, whereas this work follows evolving semantic edits and preservation requirements.Source: §2 · p. 2
  • Figure 2 depicts a student editing its own rollout while a clean-image-conditioned teacher and student-distribution auxiliary editor provide the distillation comparison.Both teacher and auxiliary editor receive the original image and compound instruction; their prediction difference yields the distillation gradient.Source: Figure 2 · p. 3

3. Preliminaries: Score Distillation

Score distillation uses diffusion-model noise predictions to guide image generation without paired targets, and later variants adapt its correction signal for training generative models. ACEdit adopts a teacher–student formulation and extends score distillation to flow-matching models.

3. Preliminaries: Score Distillation

  • Score Distillation Sampling uses a pretrained text-to-image diffusion model’s noise prediction to guide images from a differentiable renderer without paired target images.The method samples a timestep and Gaussian noise, using the prediction as a score-based update signal.Source: §3 · p. 3
  • Variational Score Distillation replaces sampled noise in the prediction difference with an auxiliary model fine-tuned on renderer-produced images.This addresses high variance associated with subtracting sampled noise, a factor linked to blurry or oversaturated SDS results.Source: §3 · p. 3
  • Noise-Free Score Distillation interprets conditional noise prediction as condition alignment, correction toward the model’s image distribution, and denoising.Under this interpretation, VSD approximately cancels denoising, leaving condition-alignment and domain-correction directions.Source: §3 · p. 3
  • Distribution Matching Distillation extends score distillation to train a student generative model guided by a pretrained diffusion teacher.ACEdit adopts this teacher–student formulation; score distillation can also extend to flow-matching models by converting predicted velocities into score estimates.Source: §3 · p. 3

4. Method

The method trains an editor on its own multi-turn outputs, using score distillation because those rollouts can diverge from ground-truth sequences. Asymmetric conditioning keeps teacher supervision anchored to the original image and accumulated instructions.

4. Method

  • Ground-truth images cannot reliably supervise later rollout edits because valid outputs can differ from the ground-truth sequence, making subsequent inputs diverge.The method therefore uses score distillation from a pretrained flow-matching teacher rather than requiring ground-truth targets for intermediate outputs.Source: §4 · p. 4

4.2. Training

  • The student and teacher use asymmetric conditioning: the student edits its own prior output, while the teacher evaluates against the original image and all instructions so far.The teacher’s clean-source conditioning avoids relying on potentially corrupted intermediate images.Source: §4 · p. 4, §4.2 · p. 4

4.1. Self-Generated Editing Rollouts

  • Rollouts apply each of E instructions sequentially to the student’s previous output, so after the first turn the editor trains on its own generated images.Training uses E = 10 turns.Source: §4.1 · p. 4

4.2. Training

  • The student’s distillation update uses the difference between pretrained-teacher and auxiliary-editor predictions on the same noisy clean-output estimate under the original-image and accumulated-instruction conditioning.The weight w_e(τ) accounts for converting velocities to clean-output estimates and normalizes the update.Source: §4.2 · p. 5
  • Training alternates student and auxiliary-editor updates, warms up the auxiliary editor for five iterations, then updates the student once per five auxiliary updates.A new E-turn rollout is generated for each update, and losses are averaged across turns.Source: §4.2 · p. 5

5. Experiments

The experiments evaluate multi-turn stability on ten-edit sequences, finding that the method preserves scene content and editing quality while reducing degradation. Ablations show that teacher conditioning and rollout length matter, while adding the source image to student conditioning can undo prior edits.

5.1. Experimental Setup

  • Training uses 96 synthetic images with ten local edits each, and evaluation uses 252 ten-edit chains on real photographs in EdiVal-10.The student is FLUX.2-klein-9B fine-tuned with a rank-16 LoRA; its rollouts use four sampling steps.Source: §5.1 · p. 5

5.2. Qualitative Evaluation

  • After ten successive object removals, competing methods visibly damage the synthetic garage scene, while our method preserves its surrounding walls, floor, and workbench.Training-free approaches can retain traces of earlier content; MT-OPSD instead blurs the image and washes out its colors.Source: §5.2 · p. 7, Figure 4 · p. 7

5.3. Quantitative Evaluation

  • After ten edits, our method achieves the highest KRIS quality, a KRIS consistency score of 3.85, the lowest background LPIPS distance (0.254), and instruction-following comparable to FreqEdit (3.38 vs. 3.39).The smallest quality margin over another method is over Emu Edit (0.30±0.08); the LPIPS distance is lower than the base model’s 0.351.Source: Table 1 · p. 8, §5.3 · pp. 8–9
  • A single edit lowers HPSv3 least for our method, by 0.18, versus 0.41 for the base model (0.23 ± 0.02).The comparison applies each method once to 606 GEdit-Bench instructions.Source: §5.3 · pp. 9–10

5.4. Ablation Study

  • Conditioning the teacher and auxiliary editor on the clean source and compound instruction yields a KRIS consistency score of 3.85, versus 2.81 and 2.28 for alternatives.The alternatives provide the previous student output to both models, or replace the source and compound instruction with the previous output and current instruction.Source: §5.4 · p. 10, Table 2 · p. 10
  • Adding the original image to student conditioning raises consistency from 3.85 to 4.08 but lowers instruction following from 3.38 to 2.80, and can undo earlier edits.The recolored stool and ball return to their original colors in the illustrated sequence.Source: Figure 5 · p. 9, §5.4 · p. 10
  • Training on five-edit rollouts improves ten-turn consistency over the base model (3.29 vs. 3.03), but recovers only about a third of the full method’s gain (3.85).The five-edit variant is significantly worse than the full method on every preservation measure.Source: §5.4 · p. 10, Table 2 · p. 10

6. Conclusions, Limitations and Future Work

The method stabilizes recursive editing by training the student on its own trajectories while the teacher stays anchored to the original image and accumulated instructions. The authors focus on edits where unaffected content must persist, and note that difficult instructions and much longer sequences remain challenging.

6. Conclusions, Limitations and Future Work

  • The teacher stays anchored to the original image and accumulated instructions while the student learns from its own generated editing trajectories.This provides supervision for a long, imperfect student sequence through a shorter clean alternative path, without exposing the teacher to accumulated student errors.Source: §6 · p. 10
  • Repeatedly replacing the subject and background can preserve visual quality over long sequences, whereas edits requiring content to persist across turns are more susceptible to accumulated errors.The paper therefore focuses on sequences where unaffected content must remain consistent.Source: §6 · pp. 10–11
  • Both the base model and the proposed method follow “Make the hiker smile” but remove sunglasses added in an earlier edit.The method reduces accumulated drift but inherits the base model’s editing capabilities and limitations.Source: Figure 6 · p. 11
  • Difficult instructions can still affect previously edited content, and extending robustness to much longer sequences remains future work.The authors describe stability and instruction following as related but distinct objectives.Source: §6 · p. 11
  • The authors suggest that Self-Forcing with asymmetric teacher supervision may help other recursive generative tasks where models repeatedly use their own outputs.Score distillation connects the student and clean-reference teacher paths without requiring paired target outputs, which the authors note is useful when many results are valid.Source: §6 · p. 11

A. Training Pseudocode

The pseudocode summarizes self-generated editing rollouts and student training with a frozen teacher. It defines the rollout and sampling controls, latent-space operations, and teacher guidance used in the procedure.

A. Training Pseudocode

  • The training procedure uses rollout length E, sampling-step count K, warm-up duration W, student-update interval R, guidance scale γ, and iteration count T.The pseudocode also defines sg as stop-gradient and the loss norm as mean squared error over latent elements.Source: A. Training Pseudocode · p. 15
  • The teacher uses classifier-free guidance with scale γ, while the auxiliary editor vψ is not guided.The teacher’s compound instruction h_e represents accumulated edit intent through turn e.Source: A. Training Pseudocode · p. 15
  • The training algorithm initializes vψ from the teacher vϕ and freezes vϕ.The algorithm headings identify separate self-generated rollout and training procedures.Source: A. Training Pseudocode · p. 15
  • The rollout algorithm repeatedly samples edits from the student and passes edited images between turns as latents.The excerpt identifies Sample<s as the sampling steps before the exit step and Sample>s as completing sampling from the exit-step state and velocity.Source: A. Training Pseudocode · p. 15

B. Evaluation Metrics

The evaluation measures final-turn performance over EdiVal-10 chains, covering instruction following, image quality, and preservation. It combines judged scores with learned or feature-based image distances and reports chain-level comparisons.

B. Evaluation Metrics

  • All metrics are computed at turn 10 of each EdiVal-10 chain and averaged over chains, with paired comparisons and standard errors.The three judged scores use KRIS-Bench prompts unchanged, with Gemma-4-31B as judge instead of GPT-4o.Source: B. Evaluation Metrics · p. 15
  • EdiVal-10 source images come from GEdit-Bench photographs with a short side of at least 768 pixels, limiting center-crop and resize upsampling to 1.33×.The sources are resized to 1024 × 1024.Source: B. Evaluation Metrics · p. 15
  • Quality is judged on coherence, natural appearance, artifacts, and texture fidelity; ΔHPSv3 measures change from the first edited image to the final image.Measuring from turn 1 separates quality lost over the chain from the effect of a single edit.Source: B. Evaluation Metrics · p. 15
  • Preservation is judged against the source for content instructions do not ask to change, and locally measured with anchor-object DINOv2 distance and background LPIPS.Anchors are included only when at least 30% of their box remains after masking; background LPIPS is read together with instruction following because added objects occupy that region.Source: B. Evaluation Metrics · pp. 15–16
  • Instruction following is judged from 1 to 5 by checking whether the ten requested changes appear correctly in the final image.The source-image negative control scores 1 for all 20 chains.Source: B. Evaluation Metrics · p. 16

C. Long Sequences with Repeated Content Replacement

A 40-edit example shows that long sequences need not progressively degrade when the subject and background are repeatedly replaced. The authors distinguish sequence length from the demand to preserve content across many edits.

C. Long Sequences with Repeated Content Replacement

  • The unmodified FLUX.2 Klein 9B model remained visually coherent through a 40-edit sequence without obvious progressive quality loss when the subject and background were replaced every two turns.The animal’s appearance and room layout varied across cycles, so returning to the same semantic categories did not restore the original image.Source: C. Long Sequences with Repeated… · p. 16

D. Results on LME-Bench

On LME-Bench, our model completes all ten turns in more sessions and collapses in fewer than MT-OPSD, though it is close to its base model. Results depend on the judging protocol.

D. Results on LME-Bench

  • The 512-pixel evaluation uses Gemma-4-31B; absolute results depend on the judge and are not comparable with the numbers reported in.SR requires every turn through k to succeed; CR counts sessions with two consecutive destroyed turns by k.Source: D. Results on LME-Bench · p. 16
  • At turn 10, SR is 0.53 vs. 0.49 for our model and its base model, and most of the gap over MT-OPSD comes from the backbone and sampler.At 1024 pixels, the two are tied in every column.Source: D. Results on LME-Bench · p. 16
  • 53 vs. 10 sessions complete all ten turns for our model and MT-OPSD, respectively; collapse counts are 2 vs. 16.These are results among 100 sessions.Source: Table 3 · p. 16, D. Results on LME-Bench · p. 16

E. Single Edits

On single edits, our model loses less HPSv3 than its base model and other methods. A separate 40-edit replacement sequence shows that sequence length alone need not cause progressive visual degradation.

E. Single Edits

  • Across 606 English instructions on 604 photographs and eleven edit types, each method is applied once to assess single-edit quality.The photographs retain their aspect ratio; most methods generate at the photograph’s resolution, capped at about one megapixel.Source: E. Single Edits · p. 16
  • After 40 repeated content-replacement edits, visual quality remains coherent at late turns, while subject appearance and scene details change across cycles.The sequence replaces the subject and background every two turns, illustrating that sequence length alone need not cause progressive degradation.Source: Figure 7 · p. 17, E. Single Edits · p. 17
  • 0.18 vs. 0.41: the HPSv3 decrease is smaller for our model than its base model on single edits (difference 0.23 ± 0.02).Every other method loses significantly more than our model; MT-OPSD loses 0.75.Source: Table 4 · p. 17, E. Single Edits · p. 17

F. Successive Background Changes

Across successive background and semantic edits, our model better preserves scenes and prior content while reducing accumulated drift. The examples contrast this with artifacts left by training-free methods and the original model.

F. Successive Background Changes

  • Across three successive background changes, our model preserves the teddy bear, alarm clock, and toy train and produces a clean gradient backdrop.Other methods retain earlier backgrounds, alter object colors, print a prior pattern into the gradient, or blur and wash out the image.Source: F. Successive Background Changes · pp. 17–18, Figure 8 · p. 18
  • In 20-step sequences with four no-op instructions after each displayed edit, fine-tuning yields more stable results than the original FLUX.2 Klein.The fine-tuned model better preserves the scene and prior edits while following requested changes.Source: Figure 9 · p. 19
  • Over 15 consecutive semantic edits, our method reduces accumulated drift in color and scene details while retaining object additions, removals, and pose or expression changes.Each edit’s result becomes the next edit’s input.Source: Figure 10 · p. 20

Related papers

Loading 2610.22328v1…