Source-linked AI summary
WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editing
Hui Zhang, Juntao Liu, Zongkai Liu, Liqiang Niu, Fandong Meng, Zuxuan Wu, Yu-Gang Jiang
TL;DR
Text-centric image editing lacks specialized training, large-scale multilingual data, and standardized benchmarks, limiting precise edits and character fidelity. WeEdit addresses these gaps with glyph-guided supervised fine-tuning, multi-objective reinforcement learning, an HTML-based data pipeline, and bilingual and multilingual benchmarks, outperforming previous open-source models across diverse operations.
Problem
Text-centric editing remains difficult because existing models struggle with complex instructions and blurry or hallucinated characters, while specialized data and standardized evaluation are lacking.
Method
WeEdit combines glyph-guided supervised fine-tuning, multi-objective reinforcement learning, scalable HTML-based data construction, and standardized multilingual evaluation.
Results
WeEdit establishes a new state of the art among open-source models, achieving 7.47/8.19/9.01 in IA/TC/BP on the Bilingual benchmark and 6.70/7.10/8.49 overall across 15 languages.
Takeaways & Limitations
The results support glyph guidance, multi-objective rewards, scalable multilingual data, and standardized benchmarks as a unified approach to text-centric image editing.
Takeaways & Limitations
Extremely dense or tiny text in non-ultra-high-resolution images can still produce minor blurring or structural artifacts.
Abstract
from arXiv · showhide
Instruction-based image editing aims to modify specific content within existing images according to user-provided instructions while preserving non-target regions. Beyond traditional object- and style-centric manipulation, text-centric image editing focuses on modifying, translating, or rearranging textual elements embedded within images. However, existing leading models often struggle to execute complex text editing precisely, frequently producing blurry or hallucinated characters. We attribute these failures primarily to the lack of specialized training paradigms tailored for text-centric editing, as well as the absence of large-scale datasets and standardized benchmarks necessary for a closed-loop training and evaluation system. To address these limitations, we present WeEdit, a systematic solution encompassing a scalable data construction pipeline, two benchmarks, and a tailored two-stage training strategy. Specifically, we propose a novel HTML-based automatic editing pipeline, which generates 330K training pairs covering diverse editing operations and 15 languages, accompanied by standardized bilingual and multilingual benchmarks for comprehensive evaluation. On the algorithmic side, we employ glyph-guided supervised fine-tuning to inject explicit spatial and content priors, followed by a multi-objective reinforcement learning stage to align generation with instruction adherence, text clarity, and background preservation. Extensive experiments demonstrate that WeEdit outperforms previous open-source models by a clear margin across diverse editing operations.
1 Introduction
Text-centric image editing targets embedded text while preserving non-target regions, but current models struggle with complex instructions, blurry or hallucinated characters, and non-Latin scripts. WeEdit addresses these gaps with glyph-guided training, scalable multilingual data construction, and standardized evaluation.
- Text-centric editing modifies, translates, or rearranges embedded text for applications including infographic updates, poster modification, interface localization, and document editing.
- Current models often produce blurry or hallucinated characters, with further degradation on Arabic, Thai, and Hindi scripts.
- WeEdit uses glyph-guided supervised fine-tuning to provide explicit spatial control and improve glyph fidelity.
- Its reinforcement learning stage jointly balances instruction adherence, text readability, and preservation of non-edited regions.
- WeEdit also introduces an HTML-based pipeline for diverse editing pairs and a multilingual benchmark spanning bilingual data and 15 languages.
2 Related Work
Related work establishes strong general image editing and text-rendering techniques, but text-centric editing remains insufficiently covered in task-specific data, benchmarks, and post-training methods. WeEdit combines glyph priors with automated multilingual data construction and dedicated evaluation.
- General image editing models modify targeted regions while preserving non-targeted content, but text-centric editing is a distinct and underexplored dimension.
- Prior text-rendering methods use linguistic features, character boxes, or glyph images, while few address text-centric image editing specifically.
- WeEdit uses VLM planning to generate glyph images as explicit spatial priors for text-centric editing.
- Existing editing datasets and benchmarks are limited in operation diversity, language coverage, and evaluation granularity.
- The paper presents glyph-guided SFT followed by tailored RL rewards to improve editing accuracy.
3 Method
WeEdit uses a two-stage text-centric editing framework: glyph-guided supervised fine-tuning supplies explicit text layout and content priors, while reinforcement learning optimizes human-centered editing qualities. Its pipeline combines VLM planning, glyph-conditioned diffusion, candidate scoring, and multi-objective policy optimization.
- Method overview: The framework first injects explicit spatial and content priors through glyph-guided supervised fine-tuning, then optimizes instruction adherence, text clarity, and background preservation with RL.
- Glyph-guided SFT: The glyph-guided SFT stage conditions MM-DiT on the source image, instruction, and a rendered glyph image.
- Task formulation: The editing task generates a target image aligned with the instruction while preserving non-target source regions.
- Glyph-guided SFT: A VLM detects target text regions, predicts target content and placement, and renders glyphs encoding character content, position, and scale.
- Reinforcement learning: Pixel-level losses cannot capture legibility, instruction fidelity, and background coherence, motivating an RL stage with a composite reward.
- Reinforcement learning: RL samples multiple candidate images and scores them through reward-guided positive and negative policy branches.
- Reinforcement learning: Each candidate receives logit-weighted continuous scores from VLM-based reward models rather than sparse single-integer rewards.
4 Dataset and Benchmark
WeEdit provides complementary structured and unstructured data-construction pipelines plus a benchmark spanning diverse text-editing operations, languages, and evaluation dimensions. Its resulting dataset contains 330K samples, while the benchmark evaluates editing across eight task categories and three metrics.
- Structured Data: The structured pipeline converts images to HTML, extracts and edits text content, backfills modified HTML, and renders source–target pairs with Playwright.Because the HTML differs only in designated text entries, non-target regions are pixel-perfect identical after deterministic rendering.
- Multilingual Data: The HTML-based workflow supports multilingual construction across 15 languages, including Arabic, Thai, Hindi, Chinese, Japanese, Korean, and Vietnamese.Texts can first be translated in HTML before the same pair-construction process is applied.
- Unstructured Data: The unstructured pipeline handles complex layouts by proposing edits with a VLM, executing them with an editing model, and retaining only candidates that pass verification.Verification checks instruction adherence, text legibility, and background preservation, with failed candidates retried using feedback.
- Dataset Statistics: 330K samples span 7 editing operation types, 15 languages, 1–13+ editing regions, and text lengths from under 20 to over 1,000 characters.The training set includes approximately 160K unstructured and 170K structured text pairs.
- Benchmark Tasks: The benchmark covers eight task categories, adding Reasoning to direct manipulation and style-editing operations.Reasoning requires deducing target text from knowledge or logical context before performing the edit.
- Benchmark Evaluation: The bilingual benchmark contains 2,000 test cases, while the multilingual version contains 2,000 cases across 15 languages and evaluates IA, TC, and BP.IA measures instruction adherence, TC text legibility and spelling, and BP visual integrity of preserved regions.
5 Experiment
Experiments evaluate WeEdit on bilingual and multilingual benchmarks, across diverse editing operations and complementary quality dimensions. WeEdit achieves strong open-source results, generalizes across 15 languages, and benefits from glyph guidance, supervised fine-tuning, and reinforcement learning.
- 5.1 Experimental Setup: The evaluation uses bilingual and multilingual benchmarks, each with 2,000 test cases, covering 8 editing operations and instruction adherence, text clarity, and background preservation.The comparison includes 15 previous state-of-the-art baselines, spanning proprietary and open-source systems.
- 5.2 Main Results: Existing models remain weak on text-centric editing: the best open-source baseline reaches only 4.16 overall IA, while all open-source systems score below 1.8 IA on Rearrange and Translate.Proprietary systems also degrade on these challenging operations.
- 5.2 Main Results: WeEdit-RL achieves 7.47/8.19/9.01 in IA/TC/BP on the Bilingual benchmark, improving over the base model by +3.98/+2.35/+2.21 and surpassing prior open-source results.It is surpassed only by Gemini-3-Pro-Image and performs strongly across the other editing operations.
- 5.2 Main Results: On the Multilingual benchmark, WeEdit achieves 6.70/7.10/8.49 overall across 15 languages, improving over the base model by +3.07/+1.28/+1.82.The benchmark includes complex non-Latin scripts such as Arabic, Thai, and Hindi.
- 5.2 Main Results: Qualitative and user-study comparisons report clearer text and stronger instruction execution than major open-source baselines, with performance comparable to Gemini-3-Pro-Image in the user study.The qualitative comparison spans all 8 editing operations, while the user study evaluates instruction adherence and text clarity.
- 5.3 Ablation Study: Ablations show that glyph guidance requires task-specific training, while combining SFT with glyph guidance and RL produces the strongest overall pipeline.Separate reward models and reference-image grounding improve the independent assessment and optimization of editing quality.
6 Conclusion
WeEdit combines glyph-guided supervised fine-tuning, multi-objective reinforcement learning, scalable HTML-based data construction, and bilingual and multilingual benchmarks for text-centric image editing. Experiments show that it surpasses existing open-source models and most proprietary counterparts.
- 6 Conclusion: WeEdit addresses model capability, data scarcity, and evaluation standardization through a unified text-centric image-editing framework.Its components include training methods, scalable data construction, and bilingual and multilingual evaluation across 15 languages.
- 6 Conclusion: Glyph-guided supervised fine-tuning uses rendered glyph images as explicit spatial priors for precise text placement and character-level fidelity.The method is followed by multi-objective reinforcement learning with separate reward models and reference-image grounding.
- 6 Conclusion: The HTML-based pipeline automatically synthesizes diverse editing pairs and extends naturally to multilingual settings, while the benchmark covers diverse operations across 15 languages.These components target the lack of high-quality training data and standardized evaluation.
- 6 Conclusion: Extensive experiments show that WeEdit surpasses all existing open-source models and most proprietary counterparts.
7 Appendix
The appendix details WeEdit’s dataset composition, evaluation reliability, qualitative comparisons, and remaining limitations across text-editing scenarios.
- Dataset Statistics: 330K training samples span seven editing operations and 15 languages, with translation and replacement comprising over 60% of the dataset.The dataset includes approximately 160K unstructured and 170K structured text pairs.
- Dataset Statistics: Over 100K samples edit two or more regions, while nearly 30K involve 13 or more regions, covering localized and coordinated modifications.Edited text lengths range from frequent short edits to a tail exceeding 1,000 characters.
- Evaluation Reliability: 243 cross-judge cell scores align closely, and GPT-5.4 produces the same Overall-IA ranking as Gemini-3-Pro across nine models.WeEdit remains the strongest open-source model and trails only the proprietary Gemini-3-Image in this comparison.
- Qualitative Results: Qualitative comparisons assess inaccurate instruction execution, unclear text rendering, and unintended edits to non-target regions across eight operation types.The comparisons include add, replace, delete, rearrange, translate, style change, combined, and reasoning-based editing scenarios.
- Qualitative Results: WeEdit consistently renders sharper text and preserves backgrounds better than many baselines, while most baselines struggle with complex spatial or semantic edits.The reported comparisons cover non-Latin scripts, delete and style-change tasks, rearrangement, combined editing, and reasoning-based editing.
- Limitations: Dense or tiny text can still blur or develop structural artifacts, and extremely complex domain-specific reasoning can produce factual hallucinations or logical errors.These limitations are identified as directions for future work in higher-resolution rendering and more robust reasoning modules.