PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Scene-Text Refinement via Pixel-Space Grouped Image Generation

Yaoli Liu, Guo-Hua Wang, Yiyang Chen, Qing-Guo Chen, Zhao Xu, Weihua Luo, Yao-Xiang Ding

arXiv:2610.18284v1cs.CV

TL;DR

Small-text errors in generated images limit readability, while existing approaches face challenges preserving image content and efficiency during refinement. PixText uses grouped pixel-space generation to edit target regions, and achieves higher text-refinement scores than the strongest baseline, with a faster four-step variant.

  • Problem

    Small-text and complex-character errors limit readability, while VAE reconstruction can lose fine strokes in text refinement.

    Source: §1 · p. 1, §3.1 · p. 3

  • Method

    PixText conditions on the original image, generates target regions as grouped pixel-space items, and pastes them back while preserving unedited pixels.

    Source: §1 · p. 2, §3.2 · p. 5

  • Results

    83.06% Pass and 94.66% exact target-character accuracy, versus 36.77% and 52.31% for FLUX-Text; the four-step variant costs less than one GPU-second per image.

    Source: §4.2 · p. 8, §5 · p. 10

  • Takeaways & Limitations

    Targeted pixel-space generation improves text readability while preserving image appearance.

    Source: §5 · p. 10

Abstract

from arXiv · show

Generated images often have coherent layouts and visual content, yet local errors in small text and complex characters still limit their readability and practical use. Scene-text refinement requires accuracy, preservation, and efficiency. Existing methods based on variational autoencoders (VAEs) can lose fine strokes during reconstruction, while full-image editing can shift background colors. In this work, we introduce PixText, a model that formulates local text refinement as grouped image generation in pixel space. Conditioned on the original image, PixText generates only the target regions, preserves unedited content, and uses MeanFlow for four-step generation. We further introduce Text Refinement Bench to evaluate text correction and image preservation in generated images. On this benchmark, PixText achieves 83.06% glyph pass and 94.66% exact target-character OCR accuracy, versus 36.77% and 52.31% for the strongest baseline. Its four-step MeanFlow variant retains an 81.84% pass rate with less than one GPU-second of amortized generation cost per image.

1 INTRODUCTION

Small-text errors can undermine readability even when generated images have coherent layouts, and existing refinement approaches make it difficult to balance accuracy, preservation, and efficiency. PixText uses grouped pixel-space generation, while Text Refinement Bench evaluates localized correction and image preservation.

1 INTRODUCTION

  • Small-text and complex-character errors occupy little image area but directly limit readability and practical usability.Refinement must balance faithful text correction, unchanged unedited content, and low computational cost.Source: §1 · pp. 1–2
  • PixText conditions on the original image, generates only target regions in pixel space, and pastes those pixels back to preserve unedited content.The design also avoids VAE reconstruction bottlenecks and reduces background color drift associated with full-image generation.Source: §1 · p. 2
  • Uniform training-timestep sampling mitigates patch-boundary artifacts, while a revised x-prediction sampling update reduces speckle from incomplete denoising.The authors report consistent improvement from uniform sampling in PixText and MiniT2I.Source: §1 · p. 2
  • PixText outperforms existing baselines on text refinement, and its four-step MeanFlow variant retains refinement performance at lower computational cost.Text Refinement Bench evaluates correction accuracy, background preservation, and inference efficiency.Source: §1 · p. 2
  • Text Refinement Bench addresses the lack of dedicated evaluation for localized text errors in generated images with character-level annotations and a unified protocol.It evaluates correction accuracy, background preservation, and inference efficiency.Source: §1 · p. 2

2 RELATED WORK

Prior scene-text work advanced text generation and editing, while pixel-space methods address latent reconstruction limits. PixText builds on these directions to refine multiple local text regions in pixel space while retaining original-image context.

2.1 SCENE-TEXT GENERATION AND EDITING

  • Scene-text methods progressed from content-and-style replacement to diffusion conditioning and multilingual, open-vocabulary, and instruction-driven editing.Related work also includes text correction in generated images; PixText focuses on jointly refining a few erroneous characters while preserving unedited content.Source: §2.1 · p. 2

2.2 PIXEL-SPACE GENERATION

  • Pixel-space generation avoids latent compression, and PixText applies it to local refinement conditioned on the original image.MeanFlow supports few-step generation; Figure 1 depicts grouped item and glyph inputs with predicted item pixels composed into the original image.Source: §2.2 · pp. 2–3, Figure 1 · p. 3

3 METHOD

PixText refines text in pixel space by generating only target regions while retaining unedited image content. The method combines grouped image generation with training and sampling changes, then distills a teacher for four-step MeanFlow generation.

3.1 PIXEL-SPACE GENERATION FOR QUALITY AND EFFICIENCY

  • Direct pixel-space flow matching avoids VAE encoding and decoding, and the JiT-L/16 output head takes less than 1% of evaluated autoencoder decoding time per call.The output heads run at every denoising step, motivating the lightweight JiT design.Source: §3.1 · pp. 3–4

3.2 TEXT EDITING AND REFINEMENT AS GROUPED IMAGE GENERATION

  • Grouped image generation uses the source image for context and glyphs for text conditions, predicts only target-item pixels, and pastes them into the original while retaining pixels outside the edits.The source text is locally blurred before conditioning; item crops can use continuous original-image coordinates rather than align to the main-image patch grid.Source: §3.2 · pp. 4–6
  • Across 116,038 text regions, continuous crop coordinates reduce item tokens by 13.7% relative to grid-aligned crops.Only crop dimensions need expansion to multiples of the patch size.Source: §3.2 · p. 6

3.3 TRAINING AND SAMPLING

  • Uniform timestep sampling weakens patch-boundary artifacts, while a clean-prediction-based final update reduces residual background speckle.The uniform-sampling improvement is reported for PixText and MiniT2I; at the final step, the revised update returns the clean-image prediction.Source: §3.3 · p. 6

3.4 EFFICIENT GENERATION WITH MEANFLOW

  • Four-step MeanFlow reduces average GPU generation cost per image to about one fifth of the 25-step teacher’s cost, with a 1.21 percentage-point decrease in Pass.The student starts from Gaussian noise at t0 = 0 and reaches t4 = 1.Source: §3.4 · p. 7

4 EXPERIMENTS

Experiments evaluate PixText on Text Refinement Bench and AnyText Bench, comparing refinement, reconstruction, and qualitative outputs. PixText leads the refinement comparison, while its four-step MeanFlow variant retains an advantage at lower generation cost.

4.1 EXPERIMENTAL SETUP

  • Text Refinement Bench uses 1,034 generated images and 2,638 repair targets with human-verified character errors and correction targets, and evaluates correction and image preservation.CJK-CLIP distinguishes structurally incorrect but visually similar glyphs that standard OCR may recognize as intended characters.Source: §4.1 · p. 8

4.2 QUANTITATIVE COMPARISONS

  • PixText achieves the highest Chinese text accuracy and ranks second to FLUX-Text on English on AnyText Bench.For AnyText ZH, Table 2 uses each baseline’s higher-Acc run for all metrics.Source: §4.2 · p. 8, Table 2 · p. 9
  • 83.06% Pass and 94.66% exact target-character accuracy for PixText exceed FLUX-Text’s 36.77% and 52.31%, respectively, on Text Refinement Bench.PixText also attains the lowest FID and LPIPS; Table 2 compares eight text-specialized methods with PixText and its four-step MeanFlow variant.Source: §4.2 · p. 8, Table 2 · p. 9
  • The four-step MeanFlow variant retains a refinement advantage over the baselines while reducing generation cost relative to the base PixText model, making localized correction practical in generation workflows.Source: §4.2 · p. 9

4.3 QUALITATIVE COMPARISONS

  • Across six matched-scale cases, PixText and four-step MeanFlow produce legible text while preserving local appearance; some baselines distort strokes, style, or nearby content.The examples span lettering styles and backgrounds.Source: §4.3 · p. 9, Figure 6 · p. 9

5 CONCLUSION

PixText uses localized pixel-space grouped generation to refine text while preserving fine strokes and surrounding content. It improves correction on Text Refinement Bench, and its four-step MeanFlow variant costs less than one amortized GPU-second per image.

5 CONCLUSION

  • PixText refines text through localized pixel-space grouped generation.The approach preserves fine strokes and surrounding content.Source: §5 · p. 10
  • PixText substantially improves correction on Text Refinement Bench.The conclusion also reports that the four-step MeanFlow variant costs less than one amortized GPU-second per image.Source: §5 · p. 10
  • Targeted pixel-space generation improves text readability while preserving image appearance.Source: §5 · p. 10

A VARIABLE-LENGTH BATCH TRAINING WITH FLEXATTENTION

Variable-length image groups are concatenated without padding and processed with attention masks that maintain within-group interactions while isolating samples. Glyph-conditioned classifier-free guidance combines predictions with and without glyph conditions, with enabled steps requiring separate predictions.

A VARIABLE-LENGTH BATCH TRAINING WITH FLEXATTENTION

  • Variable-length groups concatenate main-image, item-image, and glyph tokens without padding, retaining group membership and spatial positions.This allows images with different resolutions and numbers of edit regions to be trained together.Source: A VARIABLE-LENGTH BATCH TRAINING WITH… · p. 16
  • The attention mask isolates different groups while allowing main images and items to interact bidirectionally and items to attend to one another.Each glyph interacts across images only with its paired item; FlexAttention skips fully masked blocks without storing a dense attention matrix.Source: A VARIABLE-LENGTH BATCH TRAINING WITH… · p. 16

B GLYPH-CONDITIONED CLASSIFIER-FREE GUIDANCE

  • During training, glyph conditions are dropped for an entire image group with probability 0.1, while scene context and region geometry remain available.At inference, the conditional and glyph-unconditioned predictions share the noisy state, timestep, main image, and region positions.Source: B GLYPH-CONDITIONED CLASSIFIER-FREE… · p. 16
  • MeanFlow uses the same group-level glyph dropout for teacher and student, and four steps means four interval updates, with two predictions required when CFG is enabled.Source: B GLYPH-CONDITIONED CLASSIFIER-FREE… · p. 16

C PRECISION AND EFFICIENCY OF MEANFLOW DDE

The section tests mixed-precision computation for MeanFlow’s finite-difference derivative estimation. Using higher precision for endpoint predictions and subtraction reduces derivative error at a modest added cost relative to BF16 and is much faster than strict FP32.

C PRECISION AND EFFICIENCY OF MEANFLOW DDE

  • DDE estimates derivatives by subtracting nearby predictions, so the estimate is sensitive to endpoint-evaluation precision.The mixed-precision scheme uses FP32 endpoint predictions and subtraction while retaining BF16 for trainable center passes.Source: C PRECISION AND EFFICIENCY OF MEANFLOW… · p. 16
  • 18.92% median relative derivative error, versus 192.64% for BF16, with 28% more computation time than BF16 and approximately 34× faster runtime than strict FP32.Table 3 reports medians over three samples, with seven timing measurements per sample.Source: Table 3 · p. 17, C PRECISION AND EFFICIENCY OF MEANFLOW… · p. 17
  • The comparison uses three training samples at (t, s) = (0.4, 0.8) with finite-difference step size h = 0.005 and times derivative estimation, the center pass, loss computation, and backpropagation.Timing runs use dedicated NVIDIA H800 GPUs, two warm-up runs, and seven measurements per configuration.Source: C PRECISION AND EFFICIENCY OF MEANFLOW… · p. 17

D TEXT REFINEMENT BENCH AND EVALUATION SETTINGS

Text Refinement Bench evaluates local character repair in generated images, jointly considering target-glyph agreement, recognition, and image preservation. Its evaluation settings include reviewed repair targets, shared preprocessing for compared methods, and batch-one generation timing.

Evaluation settings

  • Text Refinement Bench covers 1,034 images and 2,638 repair targets, and complements text-generation and editing benchmarks with localized repair evaluation.Its targets come from actual generation errors, and its protocol jointly measures target-glyph agreement, character identification, and image preservation.Source: D TEXT REFINEMENT BENCH AND EVALUATION… · p. 17
  • Pass measures target-glyph agreement, while R@1 measures whether the target ranks first by similarity over the character vocabulary.Pass uses character-specific CJK-CLIP thresholds; R@1 can exceed Pass when a malformed glyph still ranks closest to its intended character.Source: D TEXT REFINEMENT BENCH AND EVALUATION… · p. 17
  • OCR Acc measures exact target-character transcription, but high-confidence OCR can miss malformed strokes that fail the CJK-CLIP criterion.Figure 7 shows four such glyphs recognized as their targets with 99.67%–99.97% confidence, consistent with human annotations.Source: D TEXT REFINEMENT BENCH AND EVALUATION… · p. 18, Figure 7 · p. 18
  • LPIPS uses AlexNet on full images resized to 256 × 256, and GPU-s/img divides summed GPU generation time by the number of images.Parameter counts include the generation network, VAE, and text encoders, including control branches and unmerged adapters.Source: D TEXT REFINEMENT BENCH AND EVALUATION… · pp. 18–19
  • AnyText Chinese evaluations use reviewed character boxes and shared recognition preprocessing, while Chinese-capable baselines report the input setting with higher sentence accuracy.AnyText improves from 65.87% to 66.54% Acc with character input; other three baselines achieve higher Acc with line inputs, and the selected run supplies all four main-table metrics.Source: D TEXT REFINEMENT BENCH AND EVALUATION… · p. 18, Table 4 · p. 19
  • 1.062 s per request is PixText + MeanFlow’s batch-one time with batched CFG branches, versus 3.126 GPU-s/img for the fastest compared baseline.Without batching the CFG branches, PixText + MeanFlow averages 1.152 s per request on one NVIDIA H800.Source: D TEXT REFINEMENT BENCH AND EVALUATION… · p. 19

E COMPACT CROPS AND TOKEN SAVINGS ENABLED BY CCPE

CCPE permits item crops to start independently of the main-image patch grid while keeping their dimensions patch-size multiples, reducing extra border area without losing target text.

Compact crops and token savings enabled by CCPE

F DATA PREPARATION AND SAMPLING DETAILS

The supplementary experiments examine timestep sampling, grid-artifact measurement, and the final sampling update, alongside additional qualitative comparisons. They also describe training-data preparation and the evaluation protocols used for these analyses.

F DATA PREPARATION AND SAMPLING DETAILS

  • Training glyph conditions are constructed by using CRAFT to detect text regions and PaddleOCR to recognize their contents (Cui et al., 2025).The timestep-scheduling ablation compares logit-normal and uniform sampling from the same earlier-model checkpoint, with evaluation after 5k and 10k continuation updates.Source: F DATA PREPARATION AND SAMPLING DETAILS · p. 20
  • 1.2419 to 1.1511 is MiniT2I’s boundary-gradient score change without inference shifting after uniform fine-tuning; all five inference schedules have paired intervals below zero.The paired difference is −0.0908 with a 95% interval of [−0.1194, −0.0619].Source: F DATA PREPARATION AND SAMPLING DETAILS · p. 20
  • −0.0573 at 5k updates and −0.0445 at 10k are PixText’s uniform-minus-logit-normal differences, with both 95% intervals below zero.Intervals are [−0.0733, −0.0394] at 5k and [−0.0703, −0.0184] at 10k updates.Source: F DATA PREPARATION AND SAMPLING DETAILS · p. 20
  • The grid-artifact probe compares adjacent RGB differences across 16-pixel patch boundaries with differences across other adjacent pixel pairs.It uses fixed Text Refinement Bench samples, target text, and noise seeds; 297 qualifying crops from 126 images are measured at original resolution before resizing or compositing.Source: F DATA PREPARATION AND SAMPLING DETAILS · p. 20
  • The final Heun update returns the guided clean prediction directly, avoiding the baseline update’s residual 60% contribution from the current noisy state.Figures 9 and 10 provide additional endpoint-update comparisons; Figure 9 shows scene context and close-ups with high-frequency maps, while the maps in Figure 10 follow the same layout.Source: F DATA PREPARATION AND SAMPLING DETAILS · p. 20, Figure 9 · p. 21, Figure 10 · p. 21

G ADDITIONAL QUALITATIVE COMPARISONS

H GENERATION BEYOND THE MAIN TRAINING SCRIPTS

Although PixText is trained primarily on Chinese and English, glyph conditions can specify characters beyond those scripts. Qualitative examples show target generation while retaining adjacent text and background.

H GENERATION BEYOND THE MAIN TRAINING SCRIPTS

  • PixText follows supplied glyphs for characters beyond its main training scripts while retaining adjacent text and background in the shown examples.Figure 16 pairs glyph conditions with output crops at shared crop coordinates and display scale; pale-blue frames mark model outputs.Source: H GENERATION BEYOND THE MAIN TRAINING… · p. 28, Figure 16 · p. 28

I COMPARISON WITH QWEN-IMAGE-3.0-PRO

PixText and PixText + MeanFlow achieve substantially higher text accuracy and better image preservation than Qwen-Image-3.0-Pro on Text Refinement Bench. The comparison uses the same 1,034 images, with API refusals counted as failures for text-accuracy metrics.

I COMPARISON WITH QWEN-IMAGE-3.0-PRO

  • The API receives each full image and all repair targets in one bounding-box text prompt, returns 1,027 edited images, and refuses the remaining seven inputs.The 11 repair targets in refused cases count as text-accuracy failures.Source: I COMPARISON WITH QWEN-IMAGE-3.0-PRO · p. 29
  • Both PixText variants achieve substantially higher text accuracy and better image preservation than Qwen-Image-3.0-Pro.The comparison uses the same 1,034-image Text Refinement Bench as the main results.Source: I COMPARISON WITH QWEN-IMAGE-3.0-PRO · p. 29
  • The comparison indicates that even an advanced image-editing model can remain unreliable for precise, localized text repair.Source: I COMPARISON WITH QWEN-IMAGE-3.0-PRO · p. 29
  • Table 5 evaluates 2,638 repair targets, counting targets in refused cases as failures for Pass, R@1, and OCR Acc; image metrics use returned images.The table compares the methods on the same 1,034 images.Source: Table 5 · p. 29

Related papers

Loading 2610.18284v1…