PaperlayerView exact revision on arXiv

Source-linked AI summary

Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network

Yulong Chen, Ziqian Zhang, Haoyu Zhang, Ao He, Yaxing Wang, Senmao Li, Kai Wang

arXiv:2609.20633v3cs.CV

TL;DR

Text-guided editing must make requested changes while preserving unrelated source content, but spatial controls and fixed causal decoding complicate that balance. RefineEdit routes editable bits through a pretrained GRN's evolving binary refinement and stabilizes their selection; across nine PIE-Bench categories it ranks best on background-preservation and CLIP scores among evaluated methods.

  • Problem

    Text-guided editing must change prompt-specified content while preserving unrelated source content, a balance complicated by spatial-control inaccuracies and causal editors' fixed decoding order.

    Source: §1 · p. 2

  • Method

    RefineEdit is a training-free GRN-based editor that uses signed probability differences to route selected bits to editing updates and anchors the rest to the evolving source.

    Source: §1 · p. 3

  • Results

    Across nine PIE-Bench categories, RefineEdit achieves the best background-preservation scores in PSNR, LPIPS, MSE, and SSIM, and the highest whole-image and edited-region CLIP scores among compared methods.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The results support using global refinement to couple edit localization with content generation as editing evidence evolves with the visual state.

    Source: §1 · p. 3, §6 · p. 10

Abstract

from arXiv · show

Text-guided image editing must introduce the requested changes while preserving unrelated source content. In training-free editing, diffusion editors often use spatial controls whose inaccuracies can leave edits incomplete or alter unrelated regions. Causal autoregressive editors face a further constraint: their fixed decoding order limits revision of earlier decisions. As the first to explore training-free image editing with Generative Refinement Networks (GRN), we observe that its refinement process is inherently suitable for editing and offers a promising way to address these limitations. Motivated by this observation, we introduce RefineEdit, a training-free prompt-to-prompt image editing framework built on the GRN. Our key idea is to couple edit localization with content generation through the global refinement of binary image codes, allowing editing evidence to be revised as the image evolves. More specifically, RefineEdit combines bit routing with two stabilization mechanisms: adaptive spatial freezing and finite bit locking. Bit routing starts from an intermediate source state and uses signed probability differences between the two branches to identify editable positions and bits. It directs selected bits toward editing refinement while anchoring the rest to the evolving source trajectory. Adaptive spatial freezing limits unnecessary expansion of the editing region, while finite bit locking maintains recent bit activations to support continued editing. The overall framework requires no additional training, external masks, or attention control. Across nine editing categories of PIE-Bench, RefineEdit achieves the best background-preservation scores in PSNR, LPIPS, MSE, and SSIM, together with the highest whole-image and edited-region CLIP scores among the evaluated methods. Code is available at https://github.com/mura1n/RefineEdit.

1 INTRODUCTION

Text-guided editing must make prompt-specified changes while preserving unrelated source content, but localizing edits and revising them during generation remain challenging. RefineEdit uses GRN refinement to coordinate edit localization with content generation, and its reported PIE-Bench results show strong preservation and prompt alignment.

1 INTRODUCTION

  • Training-free editors use spatial controls that can leave requested changes incomplete or alter unrelated content, while causal autoregressive decisions are difficult to revise.The central challenge is identifying where the source conflicts with the editing prompt while preserving other content.Source: §1 · p. 2
  • GRN repeatedly refines the full visual token map, and aligned binary coordinates let its editing branch reuse intermediate source structure while updating appearance.As the visual state changes, new probability differences provide editing evidence, coupling localization and content generation.Source: §1 · pp. 2–3
  • RefineEdit starts an editing branch from an intermediate source state and routes selected bits using signed probability differences, while unselected bits follow the evolving source.It is presented as the first training-free prompt-to-prompt editing framework built on GRN and requires no external masks or attention control.Source: §1 · p. 3
  • Across nine PIE-Bench editing categories, RefineEdit achieves the best background-preservation scores in PSNR, LPIPS, MSE, and SSIM, and the highest whole-image and edited-region CLIP scores among compared methods.The reported comparison covers object replacement, addition and removal, content and pose modification, color and material modification, background modification, and style transfer.Source: §1 · p. 3
  • Adaptive spatial freezing limits unnecessary edit-region expansion, while finite bit locking preserves recent editing permission to support continued refinement.Ablations are reported to show complementary roles for the two mechanisms in stabilizing localization and supporting bitwise editing.Source: §1 · pp. 3–4

2 RELATED WORK

The related work contrasts generative approaches whose refinement or decoding order shapes whether earlier image decisions can be revised. It also positions RefineEdit among training-free editors that control edits through spatial, distributional, or trajectory-based mechanisms.

2.1 TEXT-TO-IMAGE GENERATION

2.2 TRAINING-FREE IMAGE EDITING

  • Training-free editors use diffusion controls, flow-based semantic transport, or discrete token-distribution differences; RefineEdit routes prompt-sensitive bits along aligned trajectories.The passage notes trade-offs in continuous spatial control, implicit editable support in flow methods, and fixed causal order in discrete autoregressive editing.Source: §2.2 · p. 4

3 PRELIMINARIES

The preliminaries describe images as binary codes and GRN refinement as repeated parallel prediction combined with a resampled selection mask. RefineEdit branches from an intermediate source state and routes editing updates while preserving other source coordinates.

3 PRELIMINARIES

  • A frozen Hierarchical Binary Quantization tokenizer encodes an image as a binary array indexed by spatial position and bit channel.Each coordinate identifies one bit at one spatial position.Source: §3 · p. 4
  • GRN begins from a random binary code and predicts all binary coordinates in parallel from the current state and text condition.Its frozen Transformer outputs two logits per coordinate, which are converted into a distribution for sampling each bit.Source: §3 · p. 4
  • At the switch step, RefineEdit's editing branch inherits the source state, then bit routing selects editing updates or copies the evolving source state.Both branches retain GRN's global prediction and random refinement.Source: Figure 3 · p. 5
  • At each refinement step, a resampled Bernoulli selection mask determines the expected proportion of sampled predictions retained in the next state.As the retained prediction proportion increases, refinement moves from random bits toward a complete image; all coordinates are predicted again at the next step.Source: §3 · pp. 4–5

4 METHOD

RefineEdit is a training-free GRN framework that combines bit routing with spatial and temporal stabilization to balance source preservation and continued editing refinement.

4.1 REFINEMENT-GUIDED BIT ROUTING

  • At the switch step, the editing branch inherits intermediate source structure; signed probability differences identify editable positions and bits, routing selected updates from the editing proposal and anchoring the rest to the source state.Source: §4.1 · pp. 5–6

4.2 MASK STABILIZATION

  • Adaptive spatial freezing uses each sample’s initial editing response to retain the switch-step spatial mask or continue updating it, while bitwise selection and content refinement continue.The default freezing threshold is τfrz = 2τspa.Source: §4.2 · p. 6
  • Finite bit locking preserves editing permission for bits selected within the latest K steps, but not their values; permission expires after K consecutive unselected steps.Together with bit routing and adaptive spatial freezing, it balances source preservation with continued refinement of edited content.Source: §4.2 · pp. 6–7

5 EXPERIMENT

Across PIE-Bench, RefineEdit leads key preservation and CLIP measures; the experiments also examine runtime and tradeoffs in branching, thresholds, and mask stabilization.

5.1 EXPERIMENTAL SETUPS

  • The evaluation uses nine PIE-Bench editing categories, shared source images and evaluation masks, and nine training-free baselines.RefineEdit uses a frozen pretrained GRN without attention control; outputs are resized to 512×512 for comparison, and EditEval v2 is evaluated without retuning. Metrics cover structure, unedited-content preservation, and prompt alignment; category-specific settings vary ts, τspa, and τpow, with K = 4 and τfrz = 2τspa.Source: §5.1 · pp. 7–8

5.2 EXPERIMENTAL RESULTS

  • Qualitative examples show localized edits—including object replacement, addition, removal, gaze, and color changes—with limited disruption to unrelated content.Examples include retaining a crouching pose and roof layout when replacing Spider-Man with a robot, and preserving a mountain scene when adding a goat.Source: §5.2 · p. 8
  • RefineEdit leads all four background-preservation metrics and both CLIP metrics; versus FlowEdit, PSNR improves 5.33 dB and edited-region CLIP exceeds LEDits++ by 0.15.FlowEdit remains stronger in Structure Distance; gains in MSE and CLIP are described as modest.Source: Table 1 · p. 7, §5.2 · p. 8

5.3 ABLATIONS AND ANALYSIS

  • At 1024 × 1024, RefineEdit takes 26.77 seconds per edit, comparable to FlowEdit and about three times faster than PnP and PnP-DirectInv.Runtime is averaged over ten runs on one NVIDIA A100 GPU, excluding I/O; it is lower than RF-Inversion.Source: §5.3 · p. 8, Table 2 · p. 8
  • Branching too early can spread edits into the background, while branching too late can preserve source structure but leave the intended edit incomplete.The authors identify ts = 18 as a suitable balance; their qualitative comparison does not distinguish whether late branching is harder because the visual state is established or because fewer refinement steps remain.Source: §5.3 · p. 8, Figure 4 · p. 9
  • Increasing τspa reduces the selected region, while increasing τpow reduces active bits and weakens edits; stronger bitwise editing can also expand the spatial mask.The examples illustrate a balance between editing coverage and background preservation.Source: §5.3 · p. 8, Figure 5 · p. 9
  • Without adaptive spatial freezing, the mask spreads into the roof and background; without finite bit locking, the fingers remain incompletely edited.Together, the mechanisms balance spatial control with continued refinement of edited content.Source: §5.3 · p. 9, Figure 6 · p. 9

6 CONCLUSION

RefineEdit uses evolving bit-level evidence and stabilization mechanisms to jointly refine edit localization and content. Results across nine PIE-Bench categories support this approach, while controlled comparisons show preservation–editing trade-offs as parameters change.

6 CONCLUSION

  • Across nine PIE-Bench categories, RefineEdit shows strong preservation of unedited content and alignment with editing prompts, supporting refinement as a process where localization and generation evolve together.It uses aligned bit probability differences, adaptive spatial freezing, and finite bit locking; the conclusion suggests exploring the insight beyond GRN.Source: §6 · p. 10

A.1 FIXED GRN SETTINGS

  • Inference keeps the pretrained GRN and tokenizer fixed, and editing branches from an intermediate source state without adding refinement steps or restarting the schedule.The code uses 64 × 64 spatial positions with 256 bits per position; both branches finish at step 49.Source: §A.1 · p. 14

A.2 RefineEdit HYPERPARAMETERS

  • Category-specific switch steps and spatial and bitwise thresholds are used; K = 4 and τfrz = 2τspa are fixed across categories.Recently selected bits remain eligible for editing, while the spatial mask freezes if its mean excess response reaches the freezing threshold.Source: §A.2 · p. 15, Table 4 · p. 15

A.3 CONTROLLED HYPERPARAMETER COMPARISONS

  • Later switching improves background preservation but lowers both CLIP scores in the object-replacement comparison.The endpoint stays fixed, so switch timing changes both the inherited source state and the number of editing iterations; neither factor is isolated as the sole cause.Source: §A.3 · p. 16
  • On the color-modification subset, increasing τspa improves background preservation but lowers edited-region CLIP.The comparison uses the full AdaSF rule, so τfrz scales with τspa.Source: §A.3 · p. 16
  • Increasing τpow improves all reported background metrics but decreases whole-image and edited-region CLIP in the color-modification comparison.A higher threshold restricts which bits can be edited at selected positions.Source: §A.3 · p. 16

A.4 MECHANISM ABLATIONS

Ablations show complementary effects from adaptive spatial freezing and finite bit locking. In a separate user study, RefineEdit received a higher proportion of preference votes than each of three baselines.

A.4 MECHANISM ABLATIONS

  • The ablation disables one mechanism at a time on the object-replacement subset, leaving the other mechanism and all other settings unchanged.Without FBL, only the current step’s candidate bits remain active, equivalent to K = 1; without AdaSF, the spatial mask updates every step.Source: §A.4 · p. 16
  • Removing FBL improves background preservation but lowers both CLIP scores, while removing AdaSF improves CLIP scores at the cost of background preservation and structural consistency.The authors characterize the mechanisms as complementary: AdaSF restricts unwanted spatial expansion, and FBL sustains editing activity across refinement steps.Source: §A.4 · p. 16, Table 6 · p. 17

B USER STUDY

  • In a blinded comparison with 20 participants and 400 votes, RefineEdit received a higher proportion of preference votes than LEDits++, FlowEdit, and ChordEdit.The 20 result sets were randomly sampled from a pool spanning all editing categories.Source: Figure 7 · p. 17, B USER STUDY · p. 17

C.1 DATASET

The evaluation set pairs source and editing prompts across nine PIE-Bench categories. RefineEdit edits generated source trajectories, while baselines receive the same source images and evaluation-only masks.

C.1 DATASET

  • The PIE-Bench-derived evaluation set contains 560 source–editing prompt pairs across nine editing categories.Each editing prompt specifies the desired change to its source prompt.Source: §C.1 · p. 18
  • For each source prompt, GRN generates a 1024 × 1024 source image; RefineEdit edits its generation trajectory while baselines receive that same image.Using a shared source makes preservation comparisons reference the same image across methods, not a ground-truth edited target.Source: §C.1 · p. 18
  • Grounded-SAM supplies source-aligned reference edit masks for evaluation, but RefineEdit does not receive those masks at inference.Images are resized to 512 × 512 before evaluation, and binary masks use nearest-neighbor resizing.Source: §C.1 · p. 18

C.2 EVALUATION METRICS

The evaluation covers structural consistency, preservation outside the edit mask, and alignment with the editing prompt. It uses masked image comparisons, perceptual and structural metrics, CLIP scores, and per-image averaging.

C.2 EVALUATION METRICS

  • Preservation is evaluated outside the binary reference edit mask, where M = 1 marks editable regions; for background modifications, the preserved content may be the foreground subject.The evaluation also includes structural consistency and editing-prompt alignment.Source: §C.2 · p. 18
  • Structure Distance measures changes in image-token relationships using a pretrained DINO ViT-B/8; lower values indicate stronger structural consistency.It is the mean squared difference between source and edited self-similarity matrices, computed on full images.Source: §C.2 · p. 20
  • MSE measures pixel differences in masked preservation images, and PSNR expresses pixel reconstruction fidelity; lower MSE and higher PSNR indicate better preservation.MSE uses the full RGB image size as its denominator, and PSNR is averaged per image rather than computed from pooled MSE.Source: §C.2 · pp. 20–21
  • LPIPS measures perceptual differences using pretrained visual features, while SSIM compares local luminance, contrast, and structure.Lower LPIPS and higher SSIM indicate greater perceptual similarity and better preservation, respectively.Source: §C.2 · p. 20
  • CLIP ViT-L/14 scores editing-prompt alignment for both the full image and the edited region, with higher scores indicating stronger target alignment.For edited-region CLIP, pixels outside the reference edit mask are set to black; the image is not cropped to the mask’s bounding box.Source: §C.2 · p. 21
  • Metrics are computed per image and averaged over finite values with equal weight per valid image; undefined scores and infinite PSNR values are excluded.Valid counts can differ across metrics and methods.Source: §C.2 · p. 21

C.3 COMPARED METHODS

The comparison covers nine training-free editors built on diffusion or rectified-flow models, grouping baselines by generative backbone and contrasting spatial-control methods with transport-based approaches.

C.3 COMPARED METHODS

  • The evaluation compares nine training-free editing methods using pretrained diffusion or rectified-flow backbones, including SD 1.4, SD 1.5, SD-Turbo, and FLUX.1-dev.The methods are grouped by the generative backbones used in the evaluation.Source: §C.3 · p. 21
  • Diffusion baselines control cross-attention, mutual self-attention, semantic guidance, injected source features, or implicit masks to edit while retaining source content or layout.The compared methods include Prompt-to-Prompt [Hertz et al., 2023], MasaCtrl [Cao et al., 2023], Pix2Pix-Zero [Parmar et al., 2023], PnP [Tumanyan et al., 2023], PnP-DIRINV [Ju et al., 2024], and LEDits++ [Brack et al., 2024].Source: §C.3 · pp. 21–22
  • FlowEdit and ChordEdit construct prompt-conditioned transport trajectories, while RF-Inversion controls inversion dynamics to support editing with source-content retention.FlowEdit [Kulikov et al., 2025] avoids inversion and test-time optimization; ChordEdit [Lu et al., 2026] uses dynamic optimal transport and is evaluated with SD-Turbo; RF-Inversion [Rout et al., 2025] uses a linear quadratic regulator.Source: §C.3 · p. 22

C.4 CROSS-DATASET EVALUATION ON EDITEVAL V2

The appendix tests whether PIE-Bench settings transfer to EditEval v2 without retuning and specifies the RefineEdit inference procedure. RefineEdit leads on three preservation metrics while ranking second on both CLIP measures.

C.4 CROSS-DATASET EVALUATION ON EDITEVAL V2

  • EditEval v2 evaluation uses 150 source/edit prompt pairs across seven categories, applying corresponding PIE-Bench settings without retuning.The source images are generated by GRN rather than taken from the benchmark photographs.Source: §C.4 · p. 22

D ALGORITHM OF RefineEdit

  • RefineEdit samples binary coordinates independently, reuses a shared random code and refinement schedule, and runs separate source and editing branches from the switch step.The algorithm takes a frozen GRN, HBQ decoder, source and editing prompts, prediction schedule, thresholds, and bit-lock duration as inputs.Source: D ALGORITHM OF RefineEdit · pp. 22, 24
  • At the final prediction step, the algorithm decodes routed sampled predictions directly without constructing another input state or mixing with the shared code.The source output is decoded from its final sampled prediction.Source: D ALGORITHM OF RefineEdit · pp. 22, 24

C.4 CROSS-DATASET EVALUATION ON EDITEVAL V2

  • RefineEdit achieves the best PSNR, LPIPS, and SSIM and the second-best whole-image and edited-region CLIP scores among evaluated methods on EditEval v2.The reported balance is maintained using PIE-Bench settings without retuning.Source: §C.4 · p. 22, Table 8 · p. 23

D ALGORITHM OF RefineEdit

  • Spatial thresholds select editable locations, freezing can reuse the initial spatial mask, and a finite window retains recent bit selections.The algorithm forms bit selections from the spatial mask and probability-difference threshold, then constructs the next branch inputs before the final step.Source: D ALGORITHM OF RefineEdit · p. 24

E ADDITIONAL QUALITATIVE RESULTS

Figures 10–18 provide qualitative comparisons across nine editing categories, illustrating requested changes alongside the goal of preserving content unrelated to the prompt.

E ADDITIONAL QUALITATIVE RESULTS

  • Object replacement, addition, and removal compare changing scene objects while retaining other content and, for removal, coherently filling the former object region.The examples include replacing fishes with sharks, adding balloons, and replacing a shepherd dog with a poodle.Source: Figure 10 · p. 25, Figure 11 · p. 26, Figure 12 · p. 27
  • Content, pose, color, and material modification compare changes to an object's attributes or structure while retaining unrelated scene characteristics.The examples include changing a bear's posture or color and changing a woman into a golden sculpture.Source: Figure 13 · p. 28, Figure 14 · p. 29, Figure 15 · p. 30
  • Background modification preserves the foreground subject, whereas style transfer allows image-wide changes while retaining recognizable scene content and composition.The qualitative figures cover all nine evaluation categories and complement the paper's quantitative comparisons.Source: E ADDITIONAL QUALITATIVE RESULTS · p. 25, Figure 17 · p. 32, Figure 18 · p. 33

Related papers

Loading 2609.20633v3…