PaperlayerView exact revision on arXiv

Source-linked AI summary

MTEdit: Packed Trajectories Beat Rollouts for Multi-Turn Image Editing

Houyuan Chen, Fanzhou Wang, Xiangyu Fan, Songlin Yang, Zhitao Yang, Lvmin Zhang, Pei Yang, Hao Zhao, Lei Yang, Anyi Rao

arXiv:2610.23751v1cs.CV

TL;DR

Sequential rollout can accumulate errors and visual-quality drift, while existing multi-turn datasets may lack long trajectories or cover only one editing scenario. MTEdit uses packed trajectory canvases to build a large dataset spanning both scenarios and introduces a benchmark for multi-turn editing. On that benchmark, Qwen-Image-MTEdit achieves the best performance among evaluated open-source models.

  • Problem

    Sequential rollout can accumulate errors and visual-quality drift, and existing datasets are often limited in trajectory length or scenario coverage.

    Source: §1 · p. 2

  • Method

    MTEdit builds multi-turn training data through packed trajectory canvas generation and introduces a benchmark covering step-wise and context-aware editing.

    Source: §1 · p. 2

  • Results

    Qwen-Image-MTEdit achieves the best performance among evaluated open-source models on MTEdit-Bench.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Qwen-Image-MTEdit demonstrates strong performance on long-chain step-wise editing and competitive context-aware editing capabilities.

    Source: §5 · p. 9

    The PTC pipeline relies on closed-source image generation and editing models.

    Source: §A.7 · p. 21

Abstract

from arXiv · show

Multi-turn image editing aims to modify an image according to a sequence of instructions, making it better aligned with human-AI collaborative creation than single-turn editing. However, constructing high-quality multi-turn training data remains challenging, as conventional sequential rollout accumulates errors and introduces visual-quality drift across editing turns. To address this challenge, we introduce MTEdit, a multi-turn image editing dataset built through a pipeline centered on packed trajectory canvas (PTC) generation. By jointly generating multiple editing states within a unified visual context, this pipeline reduces visual-quality drift across turns. Covering both step-wise and context-aware editing scenarios, MTEdit comprises 500K multi-turn image editing trajectories, with up to 8 turns and an average of 5.38 turns per trajectory, making it one of the most extensive datasets for multi-turn image editing. We further introduce MTEdit-Bench, a benchmark designed to evaluate both scenarios by assessing models' ability to follow editing instructions, maintain content consistency, resist visual-quality drift, and effectively use historical context across turns. Using MTEdit, we fine-tune Qwen-Image-Edit-Causal to obtain Qwen-Image-MTEdit. On MTEdit-Bench, we evaluate a broad range of closed-source and open-source image editing models, including Qwen-Image-MTEdit. Experimental results show that Qwen-Image-MTEdit achieves the best performance among the evaluated open-source models, demonstrating the effectiveness of MTEdit in improving models' multi-turn image editing capabilities.

1 Introduction

MTEdit targets limitations in multi-turn training data and evaluation with a packed-trajectory dataset and benchmark covering step-wise and context-aware editing. Fine-tuning on MTEdit produces Qwen-Image-MTEdit, which performs best among the evaluated open-source models on MTEdit-Bench.

1 Introduction

  • Multi-turn editing includes step-wise edits based on the previous output and current instruction, as well as context-aware edits that use prior instructions and generated images.The setting is described as more aligned with iterative real-world creative workflows than single-turn editing.Source: §1 · p. 2
  • Sequential rollout can accumulate errors and artifacts as each generated image becomes the next edit’s input, while existing datasets often have short trajectories or cover only one editing scenario.High-quality training data is needed for both scenarios.Source: §1 · p. 2
  • Its packed trajectory canvas pipeline jointly generates editing states in a unified visual context to reduce visual-quality drift across turns.The pipeline plans an editing sequence, generates and filters canvases, derives turn-level instructions, and aligns edit pairs using SIFT-based matching and image warping.Source: §1 · p. 2
  • MTEdit contains 500K trajectories and 2.69M edit pairs, covering both editing scenarios with 4–8 turns per trajectory and 5.38 turns on average.It supports 23 common editing categories and offers more trajectories and longer sequences than the compared datasets.Source: §1 · p. 2
  • MTEdit-Bench evaluates step-wise and context-aware editing for instruction following, content consistency, resistance to visual-quality drift, and use of historical context.Existing benchmarks often use sequences of around three turns and may overlook visual-quality drift.Source: §1 · p. 2
  • Qwen-Image-MTEdit achieves the best performance among evaluated open-source models on MTEdit-Bench.The model is obtained by fine-tuning Qwen-Image-Edit-Causal on MTEdit.Source: §1 · p. 2

2 Related Work

Prior image-editing work established instruction-based editing and expanded its training datasets, but much of that work centers on single-turn transformations. Existing multi-turn datasets address different settings and have limitations in scale, trajectory length, or editing-intent coverage.

Instruction-based Image Editing

  • Instruction-based image editing changes an image according to a natural-language command while preserving content irrelevant to the instruction.The related work notes continued advances in both closed-source and open-source editing models.Source: §2 · p. 2

Datasets for Single-turn Image Editing

  • Most editing datasets supervise single-turn transformations, from human-annotated pairs to synthetic, hybrid, and frontier-model-distilled collections.This motivates datasets capturing coherent sequences across successive user interactions.Source: §2 · p. 3

Datasets for Multi-turn Image Editing

  • Prior multi-turn datasets variously lack scale, long trajectories, systematic context-dependent coverage, common step-wise data, or diverse user editing intents.The cited approaches include human annotations, video-derived sequences, and rollout pipelines that may accumulate errors and visual drift.Source: §2 · p. 3

3 Technical Details

The technical details define step-wise and context-aware editing, then describe MTEdit’s packed-trajectory construction, dataset scale, benchmark, and fine-tuned model. The benchmark evaluates instruction following, consistency, drift robustness, and use of historical context.

3.1 Task Definition

  • Step-wise editing conditions each turn on the previous image and current prompt, while context-aware editing also uses history through persistent constraints, textual references, or short instructions.Source: §3.1 · p. 3

3.2 Data Construction Pipeline

  • PTC generation jointly creates multiple turn states in one shared context to reduce drift, using a VLM-derived prompt and a closed-source image model.The default canvas is a 4K square with four approximately 2K panels.Source: §3.2 · p. 4
  • PTCs are filtered, converted into turn-level prompts and target panels, then aligned within each edit pair using SIFT feature matching and image warping.The pipeline can extend beyond four turns by using the last generated image as the next source image.Source: §3.2 · pp. 4–5

3.3 Dataset Statistics

  • 500K trajectories and 2.69M edit pairs span up to 8 turns, average 5.38 turns, and both step-wise and context-aware editing.MTEdit organizes edits into 23 leaf categories; its data comprise 56% step-wise and 44% context-aware trajectories.Source: §3.3 · p. 5

3.4 MTEdit-Bench

  • MTEdit-Bench covers four tasks, with 100 eight-turn step-wise samples and 50 four-turn samples for each context-aware task.The step-wise subset evaluates long-chain editing and drift; context-aware subsets test use of prior prompts and images.Source: §3.4 · p. 5
  • The benchmark scores instruction following, content consistency, and drift robustness; context-aware judgments receive the full trajectory, while failures zero subsequent IF and CC scores.DRS averages VisualQuality-R1 and RALI image-quality scores and is not zeroed after failure.Source: §3.4 · pp. 5–6

3.5 Qwen-Image-MTEdit

  • Qwen-Image-MTEdit fine-tunes Qwen-Image-Edit-Causal with teacher forcing to train multiple turns jointly and 3D interleaved RoPE to encode temporal order.Source: §3.5 · p. 6

4 Experiments

On the step-wise benchmark, Qwen-Image-MTEdit improves over its base model across all eight turns and leads the compared open-source methods. A matched-budget ablation favors PTC-generated training data over sequential rollout on instruction following, content consistency, and resistance to visual drift.

4.1 Experimental Setup

  • The benchmark reports turn-wise IF, CC, and DRS; methods using historical image-text context are tested in both scenarios, while previous-image-only methods are tested step-wise.Each turn's score is averaged over test samples evaluated at that turn.Source: §4.1 · p. 7
  • Figure 5 compares the source image with Turn-8 outputs from PTC- and sequential-rollout-trained models; enlarged crops show fewer artifacts and cleaner details with PTC.Table 3 presents step-wise ablation results by training-data construction method, with higher values indicating better performance for every metric.Source: Figure 5 · p. 8, Table 3 · p. 8

4.3 Ablation Studies

  • At Turn 8, PTC-trained data yields a DRS of 4.22 versus 3.96 for sequential-rollout training, alongside higher IF from Turn 2 and higher CC at every turn.PTC-trained models also maintain DRS scores between 4.17 and 4.26 across all eight turns; Fig. 5 shows cleaner fur, whiskers, and bell details than rollout-trained outputs.Source: Table 3 · p. 8, §4.3 · p. 9

4.2 Experimental Results

  • Qwen-Image-MTEdit reaches a DRS of 4.33 at Turn 4 and the highest DRS through Turn 8, while improving IF, CC, and DRS over its base model at every turn.It trails the strongest closed-source models in instruction following and content preservation but achieves state-of-the-art performance among the compared open-source methods.Source: §4.2 · p. 8
  • At Turn 8, qualitative comparisons show Qwen-Image-MTEdit better preserving visual quality and avoiding obvious artifacts, while some methods lose source-image consistency and most drift as editing progresses.Fig. 4 compares models at Turns 4 and 8 of the eight-turn step-wise task; examples include sky artifacts and outputs that diverge from the source beyond intended edits.Source: §4.2 · p. 8

4.3 Ablation Studies

  • The controlled ablation matches source images, edit prompts, training steps, and hyperparameters when comparing PTC generation with sequential rollout.Both datasets contain 10,000 eight-turn step-wise trajectories generated using Nano Banana 2, and both models are fine-tuned from Qwen-Image-Edit-Causal for 5,000 steps.Source: §4.3 · pp. 8–9

5 Conclusion

MTEdit provides training data for step-wise and context-aware editing, constructed with PTC generation to reduce visual-quality drift. Its fine-tuned model performs strongly on long-chain step-wise editing and competitively on context-aware editing.

5 Conclusion

  • MTEdit combines PTC-generated training data for step-wise and context-aware editing with a benchmark of instruction following, content consistency, visual stability, and historical-context use.Fine-tuning Qwen-Image-Edit-Causal on MTEdit produces Qwen-Image-MTEdit, which shows strong long-chain step-wise and competitive context-aware editing performance.Source: §5 · p. 9

A Appendix

Figure 6 depicts block causal attention and 3D interleaved RoPE for Qwen-Image-MTEdit. Image and text tokens are temporally interleaved.

A Appendix

  • Figure 6 shows image and text tokens interleaved in temporal order as [Image] - [Text] - [Image] - ....The figure concerns block causal attention and 3D interleaved RoPE in Qwen-Image-MTEdit.Source: Figure 6 · p. 14

A.1 Qwen-Image-MTEdit Details

Qwen-Image-MTEdit adapts Qwen-Image-Edit-Causal to multi-turn editing with block causal attention and 3D interleaved RoPE. The model is trained with AdamW for 20,000 steps on 32 H100 GPUs.

A.1 Qwen-Image-MTEdit Details

  • Block causal attention lets each editing turn attend to prior image states and instructions while blocking future turns, avoiding training leakage and enabling parallel turn-level objectives and KV-Cache reuse.The model is obtained by supervised fine-tuning Qwen-Image-Edit-Causal on MTEdit.Source: §A.1 · p. 14
  • 3D interleaved RoPE arranges each turn's image state and instruction in temporal order, helping align instructions with editing states and support longer sequences.The sequence follows the pattern [Image] - [Text] - [Image] - ....Source: §A.1 · p. 14
  • Training uses AdamW for 20,000 steps on 32 H100 GPUs, with a fixed learning rate of 5×10−5, weight decay of 10−2, and per-GPU batch size 1.Training uses bfloat16 mixed precision and (β1, β2) = (0.9, 0.999).Source: §A.1 · p. 14

A.2 Additional Discussion on the Data Construction Pipeline

The section compares PTC and rollout-style visual quality across turns, with Table 4 identifying DRS as the quantitative measure.

A.2 Additional Discussion on the Data Construction Pipeline

  • By Turn 4, rollout-style examples show visual degradation, while PTC results remain relatively stable across Turns 1, 4, and 8.The comparison includes Seedream 5.0 Lite, GPT Image 2, and Nano Banana 2.Source: Figure 8 · p. 16
  • Table 4 reports Drift Robustness Score (DRS) at each editing turn, with higher values indicating better visual quality.Source: Table 4 · p. 16

A.2.1 Comparison of Visual Quality Drift

An eight-turn comparison on 5,000 cases finds rollout-style pipelines lose visual quality as editing progresses, while PTC scores remain stable.

A.2.1 Comparison of Visual Quality Drift

  • At Turn 8, DRS is 3.51 for Seedream 5.0 Lite, 3.83 for GPT Image 2, and 3.70 for Nano Banana 2; PTC ranges from 4.27 to 4.30 across turns.The comparison uses eight-turn trajectories for the same 5,000 cases and evaluates image quality at each turn.Source: §A.2.1 · p. 16
  • All methods achieve similar DRS values at Turn 1, but rollout-style pipeline scores decline substantially as editing progresses.Source: §A.2.1 · p. 16
  • The analysis reports that PTC maintains stable scores across all eight turns, while rollout-style degradation is evident by Turn 4 and worsens by Turn 8.Source: §A.2.1 · p. 16

A.2.2 PTC Layout Choice

Preliminary experiments indicate that denser PTC layouts make generation more difficult and reduce per-panel quality and cross-turn consistency; the paper uses a 2×2 layout.

A.2.2 PTC Layout Choice

  • A denser 3×3 PTC layout increases the difficulty of maintaining visual consistency and editing semantics across panels, degrading per-panel quality and cross-turn consistency.Source: §A.2.2 · pp. 16–17
  • The paper therefore adopts a 2×2 layout as its default PTC configuration.Source: §A.2.2 · p. 17

A.2.3 Cost Efficiency of PTC-based Data Construction

PTC-based construction reduces image-editing-model calls and instruction-generation calls compared with the described rollout and panel-splitting alternatives.

A.2.3 Cost Efficiency of PTC-based Data Construction

  • One PTC generation pass can produce target images for multiple editing turns, reducing calls to expensive image-editing models relative to rollout-style construction.Source: §A.2.3 · p. 17
  • The PTC-to-multi-turn conversion uses one call to generate instructions, rather than separate Gemini 3.1 Pro calls for each turn after splitting panels.The passage says this substantially reduces data-construction cost.Source: §A.2.3 · p. 17

A.2.4 Examples of PTC Generation

Figure 7 illustrates PTC generation through the source image, PTC prompt, and generated PTC; the generated canvas has four panels in a 2×2 layout, one for each editing turn.

A.2.4 Examples of PTC Generation

  • Figure 7 presents the source image, PTC prompt, and generated PTC as examples of the generation process.Source: §A.2.4 · p. 17
  • The generated PTC contains four panels arranged in a 2×2 layout.Source: §A.2.4 · p. 17
  • Each panel in the generated PTC corresponds to a different editing turn.Source: §A.2.4 · p. 17

A.3 Additional Experimental Results

This section details prompt-generation, reconstruction, and visible-evidence evaluation procedures for persistent-constraint, textual-reference, and short-instruction editing across single-edit and context-aware multi-turn scenarios. It also reports benchmark results, alignment between rankings and human judgments, and limitations in PTC generation.

A.3.1 Persistent-constraint Results

  • Qwen-Image-MTEdit has the highest average IF∗ and CC∗ among compared open-source models across three sub-benches, with stable DRS; on textual-reference editing it also leads both metrics at Turn 4.The textual-reference results additionally report that it outperforms all compared open-source methods on both metrics at Turn 4.Source: §A.3.1 · p. 18, §A.3.2 · p. 19, §A.3.3 · p. 19
  • Qualitative examples show persistent-constraint forgetting, edits applied to wrong objects or scene drift, and short-instruction failures; Qwen-Image-MTEdit better preserves constraints, referenced objects, flowers, and surroundings.Instruction-following scores still decline in later persistent-constraint turns, showing that maintaining the constraint remains challenging.Source: §A.3.1 · p. 18, §A.3.2 · p. 18, Figure 9 · p. 18

A.3.4 Additional Qualitative Examples

  • Additional examples span all four editing settings, including eight-turn step-wise trajectories, shared-constraint and textual-reference edits, and progressive attribute adjustments under short instructions.MTEdit examples also illustrate the diversity of step-wise, persistent-constraint, textual-reference, and short-instruction trajectories.Source: §A.3.4 · pp. 19–20, §A.5 · p. 20, Figure 12 · p. 21

A.4 Alignment Between Benchmark Evaluation and Human Judgments

  • DRS aligns most with human judgments on step-wise editing (ρ=0.92, τ_b=0.83), while IF aligns most on textual-reference editing (ρ=0.92, τ_b=0.82).The evaluation compares model rankings from human judgments and benchmark scores using Spearman’s ρ and Kendall’s τ_b across three aspects and four sub-benches.Source: §A.4 · p. 20, Table 8 · p. 20

A.6 Data Distribution Statement

  • MTEdit and MTEdit-Bench use third-party source images, whose rights remain with their respective holders; releasing generated edits remains subject to those rights and model-provider terms.The authors state that licenses on their original contributions do not override third-party rights or restrictions.Source: §A.6 · p. 21

A.7 Limitations

  • PTC generation depends on closed-source editing models, requires extra filtering for cross-panel interference, and may drift visually when extended substantially beyond four-turn trajectories.The passage gives 24 turns as an example of a substantially longer sequence that may still experience visual-quality drift.Source: §A.7 · p. 21, Figure 13 · p. 21

A.8.1 Seed Image Curation

  • Seed-image curation uses Gemini 3.1 Pro to score aesthetic quality, while step-wise PTC prompts specify four cumulative edits in a 2×2 canvas and recovered prompts describe each new edit.The step-wise instructions aim to preserve identity, composition, lighting, and unedited regions across panels.Source: §A.8.1 · p. 31, §A.8.2 · pp. 31–32

A.8.3 Persistent-Constraint Editing

  • Persistent-constraint PTC prompts require four cumulative edits to maintain one shared constraint established in the first turn, while preserving prior edits and the unchanged scene.The shared constraint may be a color, material, style, or other characteristic naturally compatible with the input image.Source: §A.8.3 · pp. 32–33
  • Turn 1 must state a persistent rule applying to subsequent edits and introduce its own edit; Turns 2–4 prompts describe only each new edit, while results remain consistent with the rule.The task also requires cumulative results in each panel, a fixed panel order, and edits and rules grounded in clearly visible changes.Source: §A.8.3 · pp. 33–34

A.8.4 Textual-Reference Editing

  • Textual-reference PTCs use four cumulative edits around one stable, unambiguous target, while the complete PTC prompt repeats its full name and describes each panel independently.The reconstructed user prompts instead identify the target in Turn 1 and use resolvable pronouns or shortened references in Turns 2–4.Source: §A.8.4 · pp. 34–35

A.8.5 Short-Instruction Editing

  • Short-instruction trajectories use four progressive edits with one clear, verifiable request per turn, while PTC prompts fully describe each cumulative result and preserve the unchanged scene.Reconstructed prompts make Turn 1 complete and later turns very short, without repeating completed edits or using vague instructions.Source: §A.8.5 · pp. 36–37

A.8.6 Instruction Following Evaluation (IF)

  • The instruction-following evaluation procedure calls for checking requested visually verifiable details and determining the scope of changes authorized by the instruction.The supplied passage also lists object identity, count, color, material, texture, text, position, style, and spatial qualifiers as details to assess.Source: §A.8.6 · p. 38
  • Instruction-following scores combine completion of the requested edit with preservation of content outside its scope, using visible evidence and a 0–10 scale.For text edits, the requested literal text must be visually correct; general aesthetics and image quality are excluded unless they prevent verification or visibly alter content that should remain unchanged.Source: §A.8.6 · p. 38

A.8.7 Content-Consistency Evaluation (CC)

  • Content-consistency scores measure whether untargeted content—including subjects, layout, framing, background, and untouched attributes—remains visually stable on a 0–10 scale.Requested changes are allowed and must not be scored as unintended drift; the evaluator should judge only preservation, not instruction following or general image quality.Source: §A.8.7 · p. 39

A.8.8 Context-Aware Instruction Following Evaluation (IF∗)

  • Context-aware instruction following evaluates four turns separately, judging each current edit and preservation against its before-edit reference while using prior context to interpret the instruction.The evaluator returns one 0–10 score per turn; historical requirements clarify the current instruction but are not scored separately.Source: §A.8.8 · pp. 39–40

A.8.9 Context-Aware Content-Consistency Evaluation (CC∗)

  • Context-aware consistency compares each turn with the initial image, using prompts through that turn to identify authorized changes and scoring preservation rather than edit completion or aesthetics.The procedure returns a separate 0–10 consistency score for each of the four turns.Source: §A.8.9 · p. 41

Related papers

Loading 2610.23751v1…