PaperlayerView exact revision on arXiv

Source-linked AI summary

MT Forcing: Towards Stable and Consistent Multi-Turn Image Editing with Autoregressive Diffusion

Houyuan Chen, Fanzhou Wang, Xiangyu Fan, Zesong Qiu, Yukun Wei, Songlin Yang, Zhitao Yang, Lvmin Zhang, Pei Yang, Hao Zhao, Dahua Lin, Lei Yang, Anyi Rao

arXiv:2610.25404v2cs.CV

TL;DR

Multi-turn image editing can drift in visual quality and content as edits accumulate, limiting the preservation of details and prior changes. MT Forcing trains autoregressive unified multimodal models with single-turn distribution matching and teacher-forcing guidance. Experiments report improved quality, consistency, and instruction following across turns.

  • Problem

    Multi-turn editors can accumulate visual artifacts and alter preserved regions or prior edits, while training on reference histories differs from inference on recursively generated histories.

    Source: §1 · p. 2

  • Method

    MT Forcing combines student self-rollouts matched to a single-turn teacher applying cumulative edits from the source image with teacher-forced supervision on reference trajectories during warmup and distillation.

    Source: §1 · p. 2

  • Results

    MT Forcing outperforms both base editors on all four metrics from Turn 4 onward; at Turn 8 on SenseNova-U1, IF improves by 7.45.

    Source: §4.2 · p. 7

  • Takeaways & Limitations

    Across the reported experiments, MT Forcing yields more stable visual quality, greater content consistency, and better instruction following across successive turns.

    Source: §1 · p. 2

Abstract

from arXiv · show

Multi-turn image editing aims to modify an image according to a sequence of instructions, enabling more effective interactive visual content creation for users. However, existing methods often exhibit noticeable drift after only a few editing turns, manifesting in two aspects: 1) unstable visual quality, with accumulating artifacts and degraded details; and 2) inconsistent image content, with unintended changes to preserved regions and prior edits. To address these challenges, we propose MT Forcing, a novel training paradigm for multi-turn image editing with unified multimodal models (UMMs) based on autoregressive diffusion. Specifically, we propose Single-Turn Distribution Matching Distillation (Single-Turn DMD), which supervises student self-rollouts with a teacher that applies cumulative edits to the source image in a single turn, providing visual-quality guidance without recursive error accumulation. Additionally, we introduce Teacher-Forcing Guidance, which supervises history-dependent transitions on reference trajectories during warmup and as a regularizer during distillation, encouraging cross-turn content consistency beyond what per-turn distribution matching alone can ensure. Extensive experiments demonstrate that our approach significantly improves the multi-turn image editing capabilities of UMMs, yielding more stable visual quality and stronger cross-turn consistency.

1 Introduction

Multi-turn image editing supports successive creative refinement, but existing systems can drift in visual quality and content because inference uses imperfect histories unlike reference histories used in training. MT Forcing addresses these problems with complementary distillation and teacher-forcing objectives, and experiments report improved quality, consistency, and instruction following.

1 Introduction

  • Multi-turn editing enables users to refine visual content as their creative intentions evolve, but existing methods can accumulate artifacts and alter regions or prior edits that should remain unchanged.The paper attributes this drift to exposure bias between reference histories during training and recursively generated histories at inference.Source: §1 · p. 2
  • Single-Turn DMD aligns student self-rollouts with a teacher that applies cumulative editing intent to the source image in one turn.The teacher supplies a quality reference without cross-turn error accumulation in its supervision.Source: §1 · p. 2
  • Teacher-Forcing Guidance uses consistent reference trajectories during warmup and as a lower-weight regularizer during distillation to preserve history-dependent behavior.Per-turn distribution matching alone does not fix visual details established in prior student outputs.Source: §1 · p. 2
  • MT Forcing improves visual quality, content consistency, and instruction following across successive turns in quantitative and qualitative experiments.It also improves geometric alignment under Pixel, which measures alignment between successive editing outputs.Source: §1 · p. 2

2 Related Work

Related work covers multi-turn editing, diffusion distillation, and autoregressive diffusion. The paper positions MT Forcing against methods that address editing drift and notes challenges in transferring autoregressive diffusion methods to multi-turn editing.

Multi-Turn Image Editing

  • Multi-turn editing methods address consistency, frequency preservation, identity, trajectory rewards, session intent, or degraded-input distillation, yet reference-history training can still mismatch self-generated inference histories [28].The section also describes shared-context UMM methods and recursive single-turn editing approaches [28] [40] [112].Source: §2 · pp. 2–3

Diffusion Distillation

  • Diffusion distillation spans trajectory regression, consistency methods, and distribution matching; MT Forcing uses DMD2 to transfer single-turn editing quality to outputs under imperfect multi-turn histories [55] [94] [95].DMD matches distributions using the difference between target and student-distribution scores [95].Source: §2 · p. 3

Autoregressive Diffusion

  • Autoregressive diffusion work includes self-rollouts and methods for long-horizon stability, context, and cache management, but bidirectional teachers for complete editing trajectories are unavailable.The section identifies that teacher limitation as a challenge to directly transferring these methods to multi-turn image editing.Source: §2 · p. 3

3 Method

MT Forcing combines distribution matching on student rollouts with supervision on reference trajectories to address drift in multi-turn image editing. Single-Turn DMD supplies visual-quality guidance, while Teacher-Forcing Guidance anchors history-dependent transitions for cross-turn consistency.

3.2 Single-Turn Distribution Matching Distillation

  • Single-Turn DMD matches student rollout marginals to single-turn teacher distributions because multi-turn editing lacks a bidirectional UMM teacher for jointly generated trajectories.The student uses its history and current instruction; the teacher instead receives the source image and bundled prompt, a weaker conditioning signal.Source: §3.2 · pp. 4–5
  • The single-turn teacher produces clear outputs at Turns 1, 5, and 8, providing per-turn distributional supervision while maintaining visual quality across editing stages.Bundled prompts express accumulated editing intent as one edit applied directly to the source image.Source: §3.2 · p. 4, Figure 3 · p. 5

3.3 Teacher-Forcing Guidance

  • Matching per-turn marginals does not determine responses to individual histories: a student can independently sample teacher outputs yet change content across turns with zero ideal DMD loss.The changing teapot pattern illustrates this ambiguity.Source: §3.3 · p. 5, Figure 4 · p. 5
  • Teacher-Forcing Guidance supervises outputs using reference histories and current instructions, complementing marginal matching by training history–output dependencies.Flow-matching supervision is used in a warmup before distillation and as regularization during distillation; the warmup checkpoint initializes the student, teacher, and fake model.Source: §3.3 · pp. 5–6

4 Experiments

MT Forcing is evaluated on two unified multimodal models using an eight-turn benchmark, where it improves multi-turn editing results. Ablations show complementary benefits from its two components and a trade-off between reference supervision and visual detail.

4.1 Experimental Setup

  • Evaluation compares SenseNova-U1 and SenseNova-U1.5 on 100 eight-instruction sessions, measuring instruction following, content consistency, drift robustness, and geometric alignment.IF, CC, and DRS use higher-is-better scores; Pixel measures matched-point displacement, where lower is better.Source: §4.1 · p. 6

4.2 Main Results

  • At Turns 1, 4, and 8, comparisons show fewer artifacts while preserving accumulated edits after MT Forcing.By Turn 8, examples show cleaner textures and coherent scene structure than the baselines.Source: Figure 5 · p. 7, §4.2 · p. 7
  • From Turn 4 onward, MT Forcing outperforms both base editors on all four metrics; SenseNova-U1.5’s eight-turn mean Pixel falls from 6.34 to 0.74.That is an 88.4% reduction in matched-point displacement.Source: §4.2 · p. 7, Table 1 · p. 7

4.3 Ablation Studies

  • Either component alone improves all four metrics over the pretrained baseline, but the full method performs best, reaching 9.66 IF, 8.40 CC, 4.30 DRS, and 0.74 Pixel.The variants isolate Single-Turn DMD, Teacher-Forcing Guidance, and their combination.Source: §4.3 · p. 8, Table 2 · p. 8, Figure 6 · p. 8
  • Higher λTF weights of 1 and 0.5 visibly blur fur detail, while 0.25 and 0.1 restore finer strands and sharper coat boundaries.The lower weights offer a favorable balance between content consistency and visual clarity.Source: §4.3 · p. 8, Figure 7 · p. 9

5 Conclusion

MT Forcing combines self-generated histories with reference supervision for autoregressive UMMs. Experiments on SenseNova-U1 and SenseNova-U1.5 show improved editing outcomes, with stronger benefits at later turns.

5 Conclusion

  • MT Forcing combines learning on self-generated histories with reference supervision for stable and consistent multi-turn editing with autoregressive UMMs.Single-Turn DMD uses a teacher applying cumulative editing intent directly to the source image, while Teacher-Forcing Guidance supplies reference-conditioned supervision during warmup and distillation.Source: §5 · p. 9
  • Experiments on SenseNova-U1 and SenseNova-U1.5 show improved instruction following, content consistency, and drift robustness on the eight-turn step-wise MTEdit-Bench sub-bench.Benefits become increasingly pronounced at later turns.Source: §5 · p. 9
  • Single-Turn DMD avoids recursive error accumulation in its supervision, while Teacher-Forcing Guidance encourages history–output dependencies needed for content preservation.The methods combine distribution matching on student rollouts with reference-conditioned supervision.Source: §5 · p. 9

Related papers

Loading 2610.25404v2…