Source-linked AI summary
GlyphAnchor: Enhancing Visual Text Rendering via Position-Anchored Glyph Priors
Qiang Xiang, Shuang Sun, Binglei Li, Yibo Chen, Xu Tang, Yao Hu, Junping Zhang
TL;DR
Accurate rendering of long, complex, densely arranged, and rare-character text remains difficult for image generation and editing models. GlyphAnchor adds lightweight glyph patches anchored through native positional encoding, with staged training and text-aware post-training; across challenging benchmarks, it consistently improves text fidelity and visual quality while preserving image quality.
Problem
Visual text rendering remains unreliable for long, complex, densely arranged text and rare characters in both text-to-image and image-editing models.
Method
GlyphAnchor provides compact glyph patch conditions anchored to target-image locations through native positional encoding, trained with staged supervised finetuning and text-aware NFT post-training.
Results
GlyphAnchor consistently improves text fidelity and overall visual quality across multiple backbones and benchmarks covering challenging text-rendering scenarios.
Takeaways & Limitations
InfoTextBench enables direct evaluation of text-rich visual rendering across image generation and editing, while GlyphAnchor offers a lightweight, adaptable glyph-prior enhancement.
Takeaways & Limitations
The paper notes misuse risks from high-fidelity text generation, including forged or misleading posters, advertisements, or documents.
Abstract
from arXiv · showhide
Rendering accurate text remains difficult for image generation and editing models, especially when the target contains long, complex, and densely arranged text or rare characters. Existing approaches either improve native text rendering through stronger backbones and data-centric training without explicit glyph priors, or incorporate glyph priors through specialized designs that remain insufficiently accurate and robust under challenging scenarios. We introduce GlyphAnchor, a novel text-rendering enhancement method for both text-to-image and image-editing diffusion transformer models. GlyphAnchor enhances the backbone with lightweight glyph patch conditions whose positions are anchored to the target image through the model's native positional encoding. We train this capability with staged supervised finetuning and further refine it with text-aware post-training to improve robustness. We also introduce InfoTextBench, a benchmark for evaluating text-rich visual text rendering in both generation and editing settings. Experiments across multiple backbones and benchmarks, including long, complex, and densely arranged text and rare character scenarios, show that GlyphAnchor consistently improves text fidelity while preserving overall image quality.
1 Introduction
GlyphAnchor addresses unreliable rendering of long, complex, densely arranged, and rare-character text by combining lightweight position-anchored glyph patches with dedicated training and evaluation. Across challenging benchmarks, it consistently improves text accuracy and visual quality over corresponding backbones.
- Motivation: Visual text rendering remains unreliable for long, complex, densely arranged text and rare characters in both generation and editing.These errors can compromise usability in posters, notes, infographics, and other text-rich applications.
- Method: GlyphAnchor anchors compact glyph patch images to intended target-image locations through the model’s native positional encoding.The design supplies explicit character-shape information and spatial cues without changing the attention pattern.
- Evaluation: InfoTextBench evaluates word-level accuracy, phrase hit rate, aesthetics, and prompt following for image editing and text-to-image settings.The benchmark targets text-rich visual rendering scenarios directly.
- Results: GlyphAnchor consistently improves text accuracy, phrase hit rate, and overall visual quality across multiple challenging benchmarks and corresponding backbones.The reported scenarios include long, complex, densely arranged text and rare characters.
- Training: GlyphAnchor uses staged supervised finetuning, text-aware rewards for NFT post-training, and train-time glyph augmentation.The training recipe is designed specifically for position-anchored glyph priors.
2 Related Work
Prior work improves visual text rendering through explicit glyph priors, stronger foundation models and data, layout control, post-hoc correction, or reinforcement learning. GlyphAnchor is positioned as complementary, focusing on glyph-prior integration for modern diffusion-transformer backbones.
- Glyph priors: Glyph-prior methods inject rendered glyph images, masks, positions, text attributes, or glyph-aware encoders into generation and editing models.These approaches target character accuracy, controllability, structure-style disentanglement, or font control.
- Inference-time and post-hoc methods: Inference-time glyph injection and post-hoc correction provide additional routes for improving visual text rendering after or during generation.FreeText and GlyphBanana inject glyph information at inference, while other methods address correction and controllability.
- Foundation models and data: Foundation-model and data-centric methods improve native text rendering through stronger backbones, curated data, or character-aware training without dedicated glyph modules.Examples include Qwen-Image, Z-Image, LongCat-Image, ERNIE-Image, and X-Omni.
- Layout-guided control: Layout-guided approaches predict or localize text regions and use masks, attention binding, or separate denoising to improve text placement and rendering.TextDiffuser, DCText, and TextCrafter represent distinct layout-control strategies.
- Positioning: GlyphAnchor complements these directions by focusing on how to leverage glyph priors in modern diffusion-transformer backbones.Its emphasis is glyph-prior integration rather than replacing layout-guided or correction-based methods.
3 Method
GlyphAnchor adds lightweight, position-anchored glyph patch conditions to diffusion-transformer backbones for text-to-image and image editing. Its design combines compact glyph references, layout-based positional anchoring, staged training, and text-aware post-training to improve text rendering while preserving visual quality.
- GlyphAnchor design: GlyphAnchor supplies explicit character-shape guidance through compact glyph patches while preserving the backbone’s standard attention pattern.The method is designed as a lightweight conditioning mechanism compatible with standard attention acceleration kernels.
- Design rationale: Full-canvas glyph references improve text accuracy but can weaken prompt following, decorative detail, background texture, and stylistic fidelity.They also incur substantial token overhead because their spatial size matches the target image.
- Design rationale: For a 1536×1536 target, glyph patches containing 1000 characters occupy about 31% of a full image frame for Chinese and 15% for English.Compact patches reduce token overhead, but explicit spatial grounding remains necessary for reliable placement.
- GlyphAnchor design: GlyphAnchor pairs each glyph patch with a normalized target bounding box and maps that box to position IDs on the target latent grid.The glyph tokens occupy a virtual glyph plane whose coordinates are anchored to the intended layout and are processed jointly with standard visual tokens.
- Training: Staged supervised finetuning shifts training from ground-truth image crops toward rendered glyph patches, bridging stronger training supervision with inference-time conditions.Ground-truth crops provide stronger supervision, whereas rendered patches better match inference but are harder to learn from.
- Training: NFT post-training adds rewards for text correctness, layout consistency, typography style, and visual fusion with the surrounding image.The layout reward jointly considers recognized text and spatial consistency, while style and fusion rewards are judged by an MLLM.
4 Experiments
GlyphAnchor is evaluated across text-to-image and image-editing backbones on general, rare-character, and text-rich rendering benchmarks. It consistently improves text accuracy and visual quality, with ablations showing that positional anchoring and text-aware post-training are important components.
- Experimental Setup: GlyphAnchor is evaluated on LongTextBench, OneIGBench-text, CVTG-2K, ChineseWord, and InfoTextBench across image-editing and text-to-image settings.The evaluation covers long, complex, densely arranged text and rare characters.
- General Text-Rendering Benchmarks: GlyphAnchor consistently improves long-text rendering across all three backbones and the three general text-rendering benchmarks.The gains are particularly pronounced for the two editing models, while Z-Image also benefits on LongTextBench and CVTG-2K.
- Rare Character Rendering: GlyphAnchor improves rendering of both common and rare Chinese characters across all evaluated backbones.The results support using explicit glyph evidence when prompts alone cannot reliably specify target glyph structure.
- Text-Rich Generation and Editing: GlyphAnchor improves word-level accuracy, phrase hit rate, and overall visual quality for both editing and text-to-image models on InfoTextBench.InfoTextBench tests rendering many text items while preserving layout and visual coherence.
- Qualitative Results: Qualitative comparisons show GlyphAnchor handling long titles, dense small text, and structured layouts better than several baselines.The reported baseline errors include corrupted characters, missing or duplicated words, illegible small text, and incorrect layouts.
- Ablation Study: The ablation shows progressively larger gains from glyph patches without anchoring, positional anchoring, and text-aware NFT post-training.Positional anchoring particularly improves phrase hit rate, while NFT produces the best results on LongTextBench and InfoTextBench.
5 Conclusion
GlyphAnchor enhances visual text rendering in text-to-image and image-editing models with lightweight, position-anchored glyph priors. Combined with staged training and text-aware post-training, it improves text fidelity while preserving image quality, while InfoTextBench evaluates text-rich rendering.
- GlyphAnchor anchors compact glyph patch images to target layouts through the model’s native positional encoding.This supplies character-shape information and spatial cues while keeping conditioning lightweight and adaptable across diffusion-transformer backbones.
- Staged supervised finetuning and text-aware NFT post-training complete GlyphAnchor’s training recipe.
- GlyphAnchor consistently improves corresponding backbones on long, complex, densely arranged text and rare-character scenarios.
- InfoTextBench evaluates text-rich visual text rendering across generation and editing settings.
- GlyphAnchor improves text fidelity while preserving image quality.
- High-fidelity visual text generation may increase misuse risks involving forged or misleading posters, advertisements, or documents.The authors encourage responsible use and further study of safeguards.
A.1 Qualitative ablation on positional anchoring
The positional-anchoring ablation compares glyph patch supervision with and without layout-based position IDs. Positional anchoring progressively improves text accuracy by aligning glyph priors with their target regions.
- Glyph patch SFT without positional anchoring improves over the backbone but often produces incomplete text or corrupted characters.Without anchoring, glyph patches remain only weakly associated with target regions.
- Positional anchoring further improves text accuracy by assigning glyph tokens position IDs according to their layout anchors.The comparison holds training strategy, data, glyph patches, layout inputs, and inference pipeline constant.
- Figure 5 qualitatively compares glyph patch SFT with and without positional anchoring on FireRed-Image-Edit-1.1.
A.2 Qualitative ablation on staged SFT
The staged-SFT ablation compares ground-truth crops, rendered glyph patches, and staged supervision under the same position-anchored setup. Staged SFT best balances text accuracy, style preservation, and visual consistency at inference.
- GT crop SFT introduces white-background copy-paste artifacts because training uses real text crops while inference uses rendered glyph patches.This train–test mismatch encourages transfer of conditioning-patch appearance.
- Rendered glyph SFT alleviates the train–test mismatch and improves visual consistency but remains less faithful to the prompt-specified style.
- Staged SFT best preserves target image style while maintaining comparable text accuracy, yielding the most consistent overall results.
- Figure 6 compares three SFT variants using FireRed-Image-Edit-1.1 with the same position-anchored glyph-prior strategy and inference pipeline.
A.3 Prompt Templates
The appendix specifies MLLM prompts for NFT rewards and InfoTextBench evaluation. The prompts separate style consistency, visual fusion, aesthetics, and prompt following into distinct evaluation roles.
- A.3.1 Prompt templates for NFT rewards.: The style-consistency reward evaluates typography and visual-style agreement between corresponding text regions in target and generated images.It explicitly excludes text-content correctness and natural text fusion from this score.
- A.3.1 Prompt templates for NFT rewards.: The visual-fusion reward evaluates how naturally generated text integrates with its background and surrounding visual context.It considers lighting, perspective, occlusion, material attachment, edges, and overall visual style.
- A.3.1 Prompt templates for NFT rewards.: Visual-fusion scoring uses different standards for real-world scenes and graphic-design images.Real-world images emphasize physical integration, while graphic designs emphasize coordination with the design system, layout, balance, and completeness.
- A.3.1 Prompt templates for NFT rewards.: The visual-fusion prompt assigns an integer score from 0 to 5.
- A.3.2 Prompt templates for InfoTextBench evaluation.: InfoTextBench’s overall visual-quality score averages aesthetics and prompt following.Aesthetics covers visual and design quality, while prompt following measures consistency with the input prompt.
- A.3.2 Prompt templates for InfoTextBench evaluation.: The InfoTextBench evaluation prompt requests an aesthetic score and a prompt-following score in a structured output.
- A.3.2 Prompt templates for InfoTextBench evaluation.: The aesthetics score ranges from 0 to 10 and evaluates composition, color harmony, visual hierarchy, detail, and stylistic coherence.