PaperlayerView exact revision on arXiv

Source-linked AI summary

The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignment

Ziheng Ouyang, Yiren Song, Yaoli Liu, Shihao Zhu, Qibin Hou, Ming-Ming Cheng, Mike Zheng Shou

arXiv:2511.20614v1cs.CV

TL;DR

Customized image generation still struggles with consistent fine-grained details, and existing approaches may not accurately localize regions for correction. ImageCritic uses reference-guided post-editing with curated degraded examples, attention alignment, and a detail encoder; experiments report improved detail consistency across open- and closed-source models.

  • Problem

    Customized generation methods often produce inconsistent or blurry fine-grained details, and existing models can struggle to attend to and localize regions for precise correction.

    Source: §1 · p. 1, §4.2 · p. 4

  • Method

    ImageCritic combines reference-guided post-editing with reference–degraded–target data, attention alignment loss, a detail encoder, and an agent chain for local correction.

    Source: §1 · pp. 1–2, §4 · p. 4

  • Results

    ImageCritic effectively mitigates generated-image inconsistencies and improves detail consistency across open- and closed-source models.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The agent workflow supports automated evaluation, discrepancy localization, reference retrieval, and refinement, including multi-round interaction for complex scenarios.

    Source: §1 · p. 2

    The correction task is scoped to a customized generated image paired with a reference image.

    Source: §3 · p. 3

Abstract

from arXiv · show

Previous works have explored various customized generation tasks given a reference image, but they still face limitations in generating consistent fine-grained details. In this paper, our aim is to solve the inconsistency problem of generated images by applying a reference-guided post-editing approach and present our ImageCritic. We first construct a dataset of reference-degraded-target triplets obtained via VLM-based selection and explicit degradation, which effectively simulates the common inaccuracies or inconsistencies observed in existing generation models. Furthermore, building on a thorough examination of the model's attention mechanisms and intrinsic representations, we accordingly devise an attention alignment loss and a detail encoder to precisely rectify inconsistencies. ImageCritic can be integrated into an agent framework to automatically detect inconsistencies and correct them with multi-round and local editing in complex scenarios. Extensive experiments demonstrate that ImageCritic can effectively resolve detail-related issues in various customized generation scenarios, providing significant improvements over existing methods.

1 Introduction

ImageCritic targets fine-grained inconsistencies in customized image generation through reference-guided post-editing. The introduction presents a dataset, attention-aware correction components, and an agent workflow, with improved detail alignment in the illustrated results.

1 Introduction

  • Customized generation methods often produce blurry or inconsistent fine-grained details, while reference-based super-resolution can leave errors uncorrected or worsen local regions.The introduction attributes detail problems to VAE encoding and decoding discrepancies and lost shallow-layer information in decoder-only structures.Source: §1 · p. 1
  • The authors construct reference–target pairs using VLM-based selection and simulate detail inconsistencies through explicit local degradation.They introduce attention alignment loss with region masks to decouple attention and localize regions for correction.Source: §1 · p. 1
  • ImageCritic improves local text and logo consistency while preserving spatial alignment, with corrected details aligning to original objects across closed- and open-source outputs.Figure 1 contrasts the approach with editing and super-resolution methods and shows corrections on results from both model types.Source: Figure 1 · p. 2
  • A detail encoder supports reference-image understanding, while an agent workflow automates inconsistency evaluation, localization, reference retrieval, and refinement.The workflow supports multi-round human–agent interaction and automated refinement in complex scenarios.Source: §1 · p. 2

2 Related Work

Prior work controls image generation through structural or semantic conditions, and uses multimodal attention and agent workflows for generation and editing. Fine-grained detail inconsistency remains a limitation.

2 Related Work

  • Conditional generation distinguishes pixel-aligned structural control from semantic or object-specific reference control, with IP-Adapter [66] and Textual inversion [12] injecting subject information.Condition-aligned methods control pose or shape, whereas condition-referenced methods aim to preserve reference-subject consistency.Source: §2 · p. 2
  • For subject consistency, DreamO [35] uses a router to focus attention on target subjects, XVerse [3] uses text-stream modulation to transform reference images into token-specific offsets, and MOSAIC [44] adds fine-grained regional constraints via semantic point correspondences.Source: §2 · p. 3
  • LLM agent frameworks invoke tools for complex tasks, while generate-then-edit systems optimize outputs and LayerCraft [71] unifies layout planning with object integration.GenArtist [56] is cited as an example of the generate-then-edit paradigm.Source: §2 · p. 3

3 Data Curation

The data curation pipeline builds reference–degraded–target triplets for repairing customized generated images. VLM-based filtering selects consistent pairs, and controlled degradation simulates common detail errors.

3 Data Curation

  • The task takes a customized generated image and a reference image as input, aiming to improve fine-grained consistency.The authors identify construction of consistent reference–target pairs and simulation of existing generation issues as the two data challenges.Source: §3 · p. 3
  • The pipeline filters high-quality generated images with Qwen-VL, annotates and segments objects, then uses Qwen to recheck masks before forming reference–target pairs.Flux-Fill corrupts selected masked regions, and Qwen screens out degraded samples with severe visual errors.Source: §3 · p. 3
  • The curation produces 10k triplets whose degraded patches reproduce common artifacts such as logo misalignment and text-rendering errors.The filtered targets remain consistent with the original input, providing realistic supervision signals for training.Source: §3 · pp. 3–4

4 Methodology

ImageCritic combines reference-guided image correction with attention alignment and a detail encoder, then uses an agent chain to identify and refine patches. Its pipeline feeds reference and input images with a prompt into the DiT for denoising.

4 Methodology

  • Reference and input images, together with a prompt, feed the detail encoder; its text tokens join VAE-encoded image tokens before DiT denoising.Training adds an Attention Alignment Loss, while inference uses an agent chain to identify patches and organize the prompt.Source: §4 · p. 4

4.1 Preliminaries

  • The base DiT uses multimodal attention over concatenated noisy-image and text tokens, with Flux Kontext adding an image-control token for fine-grained consistency correction.The paper adopts Flux Kontext as its base model to use its image-control capability.Source: §4.1 · p. 4

4.2 Consistency Critic

  • Attention Alignment Loss separates branch roles: input tokens guide object-level details, while reference tokens attend to background, lighting, and other global aspects.The loss aligns attention to target regions; it is applied only in double-stream blocks, where both branches respond strongly to noise regions.Source: §4.2 · pp. 4–5
  • The Detail Encoder couples trigger-word representations with image features, helping the model associate IMG1 and IMG2 with their respective reference and input images.It fuses T5 trigger states with CLIP image representations and projects the result back to the original dimension.Source: §4.2 · pp. 5–6
  • Visual examples show ImageCritic improving consistency in outputs from open- and closed-source customized-generation models.The method is also illustrated preserving style and illumination consistency across varying viewpoints, object categories, and stylizations.Source: Figure 6 · p. 6, Figure 7 · p. 7

4.3 Agent Chain

  • The agent chain assigns consistency assessment, discrepancy localization, reference-patch selection, prompt summarization, and image correction to specialized agents.A Qwen-Agent coordinator orchestrates the workflow, and users can interactively adjust patches for reprocessing.Source: §4.3 · pp. 6–7

5 Experiments

The experiments evaluate ImageCritic through qualitative and quantitative comparisons, component ablations, and agent localization against human annotations.

5.1 Implementation details.

  • Training uses Flux.1-Kontext-dev with LoRA for 20,000 steps, then retrains after enhancing a subset of low-consistency targets with ImageCritic.The reported setup uses learning rate 1 × 10−4, rank 128, and two GPUs with batch size four each.Source: §5.1 · p. 7

5.3 Quantitative comparisons.

  • ImageCritic substantially improves consistency against reference images on DreamBench++ and CriticBench.The paper reports results in Tables 1 and 2 but supplies no metric values here.Source: Table 1 · p. 7, Table 2 · p. 7, §5.3 · p. 8

5.2 Qualitative comparisons.

  • Across varied viewpoints, object categories, and stylization, corrections improve detail consistency while retaining image background, illumination, and style.Qualitative comparisons also report preservation of lighting and texture.Source: §5.2 · p. 7

5.4 Ablation analysis

  • Dataset fine-tuning improves consistency; adding either Attention Alignment Loss or Detail Encoder brings further gains, and using both yields a larger improvement.The reported ablation also finds attention disentanglement and better localization of target regions after adding the respective modules.Source: §5.4 · p. 8, Table 3 · p. 8

5.5 Discussion

  • The agent achieves 75.3% mean IoU and 88.4% mean mAP@50 against human-annotated correction regions.The evaluation compares agent-predicted and human-defined boxes on the proposed benchmark.Source: §5.5 · p. 8, Table 4 · p. 8

6 Conclusions

ImageCritic is a post-editing correction framework for customized image generation, paired with a reference-degraded-target dataset. Experiments report improved detail consistency across open- and closed-source models.

6 Conclusions

  • ImageCritic is a post-editing correction framework designed to improve consistency in customized image generation.The paper also constructs a reference-degraded-target triplet dataset for the task.Source: §6 · p. 8
  • The reference-degraded-target triplet dataset is tailored to support more accurate correction.Source: §6 · p. 8
  • Experiments report that ImageCritic mitigates generated-image inconsistencies and significantly improves detail consistency across open- and closed-source models.Source: §6 · p. 8

Supplementary Material

The supplementary material provides additional dataset, comparison, and application details.

Supplementary Material

  • Supplementary Section A gives further details on dataset construction, while Section B compares additional editing methods on restoration.Source: Supplementary Material · p. 12
  • Section C presents representative dataset cases and compares them with examples from other datasets; Section D shows multilingual corrections across models.Source: Supplementary Material · p. 12
  • Section E provides an agent-chain example and the prompts used by its components.Source: Supplementary Material · p. 12

A Data Curation details

The data-curation pipeline proceeds sequentially through generation, filtering, annotation, degradation, and final image composition.

Step 1: Reference-target pairs generation

Step 2: Generation quality filtering

Step 3: Semantic tags generation

Step 4: Image Grounding

Step 5: Consistency Verification

Step 6: Image degradation

  • For inpainting, the pipeline samples a random rectangular region covering 20%–70% of a verified object mask and applies Flux-Fill prompts for words, Chinese characters, logos, or none.Source: Step 6: Image degradation · p. 12

Step 7: Degradation quality filtering

Step 8: Final Image Composition

  • Final samples combine degraded content inside the full-object mask with generated content outside it, keeping degradation within the object and preserving the background.Source: Step 8: Final Image Composition · p. 12

Step 9: Iterative Data Enhancement Strategy

ImageCritic is used to improve Chinese-character correction data through iterative regional restoration, while comparison experiments report better consistency correction with background fidelity than existing methods. The data-enhancement procedure crops grounded object regions and uses corrected images as new targets.

Step 9: Iterative Data Enhancement Strategy

  • Grounding masks identify object regions that are cropped with the original input image for consistency correction, and corrected images become new dataset targets.This iterative enhancement is intended to improve overall data quality.Source: Step 9: Iterative Data Enhancement… · p. 13

B Image Correction Comparison

  • Existing methods fail to correct consistently while maintaining background fidelity, whereas ImageCritic substantially improves consistency while preserving the original background.The comparison evaluates Nano-banana, GPT-Image 1, Omnigen2, and Qwen-image on consistency correction.Source: B Image Correction Comparison · p. 13

C Dataset Comparison

The visualized reference–target pairs maintain global consistency and rich local details across varied settings, while comparisons identify local-consistency problems in other datasets.

C Dataset Comparison

  • The paired data maintain global consistency and rich local details across diverse languages, scenes, generation sources, and viewpoints.The section presents visualized reference–target pairs for comparison with other datasets.Source: C Dataset Comparison · p. 13
  • Subjects-200k [50] and UNO-1M [60] do not strictly preserve local consistency and often show region-level blurring or mismatches.Source: C Dataset Comparison · p. 13

D Additional Visual Results

Additional visual results report consistency restoration across languages and scenarios, with OCR results supporting text matching to references. The accompanying agent workflow iteratively detects discrepancies, finds reference regions, and applies corrections with user feedback.

D Additional Visual Results

  • OCR results show corrected text matching the reference across multilingual and varied contextual settings.The Figure 13 caption reports that all recognized text matches and that correction preserves overall structural or contextual integrity.Source: D Additional Visual Results · p. 13, Figure 13 · p. 19

E Agent Chain Details

  • The multi-agent workflow iteratively compares images, crops discrepant regions, and corrects them until the user is satisfied.Users can accept proposed patches or provide a new target region or description to guide further correction.Source: E Agent Chain Details · pp. 13–14, Figure 14 · p. 20

OCR &

  • The OCR & workflow assigns detection, reference retrieval, and correction to three agents, with TagGrounder locating user-tagged products and user review steering each step.The detector returns a target-region box, the finder returns a matching reference crop, and ImageCritic returns the corrected image and patch information.Source: OCR & · p. 23
Loading 2511.20614v1…