PaperlayerView exact revision on arXiv

Source-linked AI summary

Advancing WordArt-Oriented Scene Text Recognition: Datasets and Methods

Xingsong Ye, Yongkun Du, Jiaxin Zhang, Haojie Zhang, Chong Sun, Chen Li, Jing Lyu, Zhineng Chen

arXiv:2606.24484v1cs.CV

TL;DR

WordArt recognition is challenging because stylized fonts, textures, layouts, and visual distractions exceed the coverage of existing STR data and fixed-template methods. The paper introduces WATER-S, WATER-R, and WATERec, achieving state-of-the-art performance and 90.40% accuracy on WordArt-Bench.

  • Problem

    WordArt recognition remains challenging because existing STR data are small and insufficiently diverse, while current methods target regular text and fixed-template inputs.

  • Method

    The paper builds complementary rendered and generative synthetic data with WATER-S, reorganizes real data as WATER-R, and proposes WATERec for arbitrary-shaped inputs and complex layouts.

  • Results

    WATERec achieves state-of-the-art performance on WordArt-Bench and challenging Union14M subsets, reaching 90.40% accuracy on WordArt-Bench with combined synthetic-data training.

  • Takeaways & Limitations

    Dedicated data and model design are crucial for handling highly stylized, layout-rich WordArt scenarios.

  • Takeaways & Limitations

    Remaining errors largely arise from semantic ambiguities, with some predictions considered reasonable despite differing from the labels.

Abstract

from arXiv · show

WordArt (artistic text) features highly customized fonts, textures, and layouts, making WordArt-oriented scene TExt Recognition (WATER) substantially more challenging than general Scene Text Recognition (STR). Existing STR datasets and methods, typically built around regular scene text and fixed-template inputs, struggle to scale to WATER. Thus, we aim to advance this task from both data and model perspectives. On the data side, we construct a 2M synthetic dataset, WATER-S, with the scale improved by hundreds of times compared to existing artistic text data. WATER-S consists of two complementary subsets. One rendered by an upgraded rendering pipeline (SynthWordArt), which provides highly accurate and controllable synthetic WordArt data. The other is generated by combining Qwen3-VL for prompt mining and Z-Image for image synthesis, which improves the coverage of realistic and diverse data. On the model side, we propose WATERec. It adopts an visual encoder supporting arbitrary-shaped inputs and an autoregressive decoder to model complex layouts, structurally breaking the bottleneck of fixed-template STR on WordArt. Experiments show that this architecture outperforms prior STR methods, achieving state-of-the-art performance on irregular texts such as WordArt. Together with WATER-R, carefully reorganized from existing real STR data, our strong baseline with the new synthetic data and model design reaches 90.40% accuracy on WordArt-Bench, surpassing both general-purpose and OCR-specialized vision-language models by a large margin. Code and data are available at https://github.com/YesianRohn/WATER.

1 Introduction

The paper frames WordArt-oriented scene text recognition as a harder variant of STR because artistic styling and irregular layouts obscure linguistic content. It addresses this through scalable synthetic data, an arbitrary-shape recognizer, and rigorously deduplicated evaluation data.

  • Motivation: WordArt combines customized fonts, textures, layouts, patterns, graphics, and semantic elements, creating visual distractions that make WATER harder than general STR.The text functions simultaneously as a linguistic carrier and a visual symbol.
  • Motivation: Data scarcity is the primary bottleneck: WordArt provides only 4,805 training images, while annotations are expensive, inconsistent, and insufficiently diverse.Annotators must inspect complex textures and ambiguous character shapes, motivating large-scale synthesis.
  • Data: Existing scene-text synthesis methods target regular text, so the paper explores tool-based rendering and generative-model-based synthesis for stylistically varied WordArt with accurate labels.The synthesis paradigms target fonts, textures, shadows, realistic effects, and broad stylistic variation.
  • Model: WATERec supports arbitrary-shaped inputs and complex layouts, addressing distortion from fixed-shape resizing and the inflexibility of manually designed multi-shape templates.WordArt often contains highly irregular text with unfixed aspect ratios.
  • Evaluation: WATER-R is reconstructed from Union14M-L, WordArt-Train, and WAS-R with strict hashing deduplication to prevent label leakage between training and evaluation.The reorganized real-data resource supports fair and comprehensive evaluation on artistic text.
  • Results: WATERec achieves new state-of-the-art performance on WordArt-Bench and challenging Union14M-Benchmark subsets, while WATER-T or WATER-Z augmentation consistently improves results.The reported gains occur when either synthetic subset is added, and the combined WATER-R, WATER-S, and WATERec baseline achieves new SOTA results.

2 Related Work

Related STR work spans datasets that progressively address irregular and artistic text, alongside model architectures differentiated by decoding strategy. CTC methods remain effective for regular horizontal text, while newer approaches target irregular inputs and combine arbitrary-shaped visual encoding with autoregressive decoding.

  • Model architectures: STR architectures are commonly categorized by decoding strategy into CTC-based, parallel-decoding, and autoregressive models.The supplied passage identifies these as the main architectural divisions from a general STR perspective.
  • Model architectures: CTC methods convert 2D images into 1D character sequences through CTC alignment and work well for regular, horizontally aligned text.SVTRv2 improves irregular-text robustness through multi-scale resizing and CTC feature rearrangement.
  • Model architectures: The baseline combines arbitrary-shaped visual input support with autoregressive decoding, drawing inspiration from vision-language models.The supplied passage states that this design is used as the baseline.
  • Datasets: STR datasets progressed from regular text benchmarks to irregular-text datasets covering blur, perspective distortion, and curved text.Early benchmarks include IIIT5K, ICDAR13, and SVT; later datasets include ICDAR15, SVTP, and CUTE80.
  • Datasets: WordArt introduced a dedicated artistic-text dataset and competition, while WAS-R expanded coverage with more real artistic text.WordArt was derived from TextSeg.

3 Synthetic Dataset: WATER-S

WATER-S addresses scarce real-world artistic text data through two complementary synthetic suites targeting controllability and diversity. WATER-T uses SynthWordArt for accurate, controllable rendering, while WATER-Z uses mined prompts and generative synthesis for greater stylistic realism.

  • Motivation: WATER-S uses two synthetic data paths to overcome the scarcity, cost, and training bottleneck of large-scale real artistic text data.The suites explicitly target controllability and diversity.
  • WATER-T: SynthWordArt provides controllable artistic-text rendering while better reproducing diverse layouts and visual distractions found in real design scenarios.It is tailored to artistic text and builds on SynthText and SynthTIGER.
  • WATER-T: 11,250 artistic fonts expand font coverage, while SynthWordArt adds curved, vertical, multi-oriented, perspective, and stretched layouts.The rendering pipeline samples text, fonts, layouts, and backgrounds to generate diverse compositions.
  • WATER-Z: WATER-Z mines prompts from 31,335 WordArt and WAS-R training samples, filtering and deduplicating them into 273,488 high-quality prompts.The prompt resource uses Qwen3-VL-8B and remains independent of any particular generation model.
  • WATER-Z: Using Z-Image-Turbo at 256×256 resolution, WATER-Z randomly replaces prompt placeholders with target strings to synthesize 1M images.The configuration balances visual fidelity and generation efficiency.
  • Complementarity: WATER-T ensures accurate, exactly controllable text and fonts, whereas WATER-Z offers complex materials and globally coherent styles but less predictable character shapes and legibility.Their differing strengths make the two generation mechanisms complementary.

4 Strong Baseline: WATERec

WATERec is a generic baseline for artistic text recognition that avoids fixed-shape templates by supporting arbitrary aspect ratios, layouts, and visual styles. Its arbitrary-shape visual encoder with RoPE and autoregressive decoder is designed to model complex WordArt inputs and reading orders.

  • Design motivation: WATERec unifies arbitrary aspect ratios, unconstrained layouts, and diverse visual styles without relying on fixed-shape templates.It is designed as a simple yet effective baseline for artistic text recognition.
  • Arbitrary-shape encoding: The visual encoder is a 6-layer Transformer with RoPE attention that processes variable-length token sequences from arbitrary-shaped inputs.RoPE applies position-dependent rotations to query and key vectors, implicitly encoding relative positional information for variable image sizes and layouts.
  • Autoregressive decoding: A 2-layer cross-attention autoregressive Transformer decoder attends to encoder outputs and predicts characters sequentially under standard cross-entropy loss.This design improves robustness to non-standard reading orders.
  • Design motivation: Traditional STR methods often assume fixed-template inputs such as 32 × 128, limiting their effectiveness on artistic text with extreme aspect-ratio variation.Artistic text ranges from very long horizontal titles to nearly square or vertical text.
  • Arbitrary-shape encoding: WATERec rescales arbitrary-resolution images while preserving aspect ratio, producing 64–256 visual tokens with patch size p = 4.The token range balances fair comparison with existing methods against the need to avoid representing very small images.

5 Experiments

Experiments show that WATERec and WATER-S improve artistic-text recognition across benchmarks and architectures, while ablations validate arbitrary-shape modeling, positional encoding, and flexible token ranges. Qualitative analyses confirm robustness to diverse distortions but reveal remaining semantic ambiguities and occasional errors.

  • Evaluation Setup: A-Bench contains 1,511 WordArt test images, while C-Bench comprises six common STR benchmarks and U-Bench evaluates Union14M generalization.These benchmarks assess artistic-text recognition alongside general STR performance.
  • Comparison with OCR Systems: 81.54% is the highest A-Bench accuracy among evaluated general VLMs, OCR-specialized VLMs, and OCR tools, achieved by HunyuanOCR.Most compared systems fall in the 70%–80% range.
  • Synthetic Data Scaling: +1.85% is the A-Bench improvement from adding 2M WATER-S samples to WATER-R, with further gains on C-Bench and U-Bench.WATER-S combines tool-rendered and model-based generation and shows robust performance across scales.
  • Ablation Studies: Flexible token ranges, suitable positional encoding, and arbitrary-shape modeling improve recognition by adapting sequence length and geometric representation to artistic text.A minimum token limit of 64 and larger maximum limits can further improve accuracy, although very short sequences may be suboptimal for some small images.
  • Qualitative Analysis: WATERec handles ambiguous characters, distorted fonts, multiple orientations, curved text, and perspective distortions, but remaining errors often stem from semantic ambiguity.Some predictions are considered reasonable despite differing from the ground truth.

6 Conclusion · Appendix

The paper revisits artistic text recognition through dedicated data and model design for stylized, layout-rich WordArt. It introduces complementary synthetic and real datasets, while the appendix visualizes the artistic font tags used in WATER-T.

  • 6 Conclusion: Dedicated design is crucial for highly stylized, layout-rich WordArt scenarios.The conclusion frames this requirement from both data and model perspectives.
  • 6 Conclusion: WATER-S is a large-scale synthetic suite for artistic text recognition.It combines two complementary subsets produced through different data-generation approaches.
  • 6 Conclusion: One WATER-S subset is rendered with the SynthWordArt engine.This subset is part of the synthetic suite described in the conclusion.
  • 6 Conclusion: The other WATER-S subset is generated from generative models.The conclusion identifies this as the second component of the synthetic suite.
  • 6 Conclusion: WATER-R is a carefully deduplicated real training set.It complements the synthetic WATER-S data in the paper’s dataset design.
  • 6 Conclusion: WATER-S and WATER-R complement each other in font controllability and layout diversity.The supplied conclusion begins describing their complementary properties before the passage ends.
  • Appendix: The appendix includes a word cloud of artistic font tags used in WATER-T.The figure is labeled as Fig. 1.

A More Data Details … Caption Mining Prompt

WATER-S combines curated artistic fonts, real-dataset text labels, and Qwen3-VL-based caption mining to build diverse, reusable artistic-text generation resources. The mining pipeline produces 31,335 image–caption pairs and 273,488 deduplicated prompt templates.

  • A.1 Resources Used and Generated in WATER-S: WATER-S collects artistic fonts from open-source platforms, design-asset websites, and public code repositories, retaining styles explicitly tagged as artistic while requiring redistribution- or derivative-work-compatible licensing.Examples of retained style tags include art, display, handwriting, cartoon, and playful.
  • Artistic Fonts: 598,615 unique text entries from existing large-scale real datasets are reused for synthetic rendering, with most labels ranging from 1 to 25 characters.This strategy addresses the distribution gap caused by sampling labels from dictionaries or news corpora.
  • Real Text Corpus: Qwen3-VL-8B captions existing artistic text images to support prompt-template extraction.The captioning procedure is introduced as the first step of the caption-mining pipeline.
  • Caption Mining Prompt: The caption extraction prompt asks for a descriptive image-generation template that replaces the original text with <Text> for later substitution.It is designed for artistic text areas cropped from real photographs and targets use with large generative models.
  • Caption Mining Prompt: 31,335 image–caption pairs are produced, with one caption per image.The captions describe visual style and scene attributes while retaining a replaceable text placeholder.
  • Caption Mining Prompt: 273,488 distinct prompt templates result after few-shot mining with Qwen3-VL-8B, merging newly mined prompts with original captions and deduplicating them.Few-shot mining uses three reference prompts to imitate their style, tone, and structural pattern.

A.2 Filtering of WATER-Z … A.5 Multilingual Support

The appendix examines filtering, generator limitations, synthesis efficiency, and multilingual support for WATER-S. It cautions that correctness filtering can remove difficult but valid WordArt samples, while reporting distinct generation costs and easy language extension.

  • A.2 Filtering of WATER-Z: WATER-Z may render text incorrectly, but OCR-style correctness filtering can mistakenly remove difficult samples that are actually correct.The authors compare filtering with their WATERec / R recognition baseline rather than assuming filtering is uniformly beneficial.
  • A.2 Filtering of WATER-Z: The appendix evaluates how recognition-baseline filtering affects WATER-Z training quality.The comparison is summarized in Table 1, whose caption identifies correctness filtering as the experimental factor.
  • A.3 Extending the Image Generator: Diminishing returns at larger synthetic-data scales are partly attributed to a single generative model’s distributional upper bound and noise characteristics.The pipeline further mitigates this noise by incorporating the label-accurate WATER-T subset.
  • A.3 Extending the Image Generator: The prompt library is generator-agnostic, enabling future variability gains through ensembles of pretrained generative models.The current experiments already demonstrate the effectiveness of Z-Image-based synthesis.
  • A.4 Runtime Efficiency of WATER-S: 23.24 samples per second is the reported WATER-T throughput when 16 worker processes generate 1 million samples in 43,036 seconds.WATER-T generation uses an Intel Xeon Platinum 8255C CPU with 96 cores and 375 GB RAM.
  • A.4 Runtime Efficiency of WATER-S: 730 images per hour per GPU is the reported WATER-Z production rate when 8 NVIDIA V100 GPUs generate 1 million samples in 10,272 minutes.The WATER-Z configuration uses batch size 8.
  • A.5 Multilingual Support: The Chinese-language validation directly tests the language-agnostic claim on artistic text.The validation uses a small-scale set of 101 BCTR-Test WordArt samples, as stated in Table 2.

Chinese WordArt validation. … SPE Implementation

The paper validates WATER-S on Chinese WordArt and characterizes its complementary synthetic subsets, while detailing learned and sinusoidal 2D positional embeddings for arbitrary-shaped visual tokens.

  • Chinese WordArt validation.: 92.08% accuracy is achieved by WATERec with synthesized Chinese WATER-S, improving over 87.13% using only BCTR-Train and exceeding Qwen3-VL-8B’s 82.77%.The evaluation uses 101 Chinese WordArt samples from BCTR-Test.
  • A.6 Visualization of WATER-S: WATER-S combines WATER-T’s controllable, layout-diverse synthesis with WATER-Z’s more designer-like styles but less precise control.The two subsets are presented as complementary for WordArt synthesis.
  • B.1 Position Embedding: Visual tokens are formed by partitioning each image into non-overlapping patches, flattening them into a sequence, and recording each patch’s 2D grid coordinate.The patch embeddings are denoted x_n ∈ R^d and coordinates p_n=(p_n^x,p_n^y).
  • B More Model Details: The model details describe positional encoding as an additive injection into token representations for 2D patch coordinates.This point summarizes the shared implementation pattern across the position-embedding variants.
  • APE Implementation: APE uses separate learnable lookup tables for height and width positions, summing the indexed vectors into a d-dimensional positional embedding.The tables are E^x,E^y ∈ R^{L×d}, where L is the maximum token length.
  • APE Implementation: The positional embedding is added directly to each patch embedding, producing x′_n=x_n+e_n.This additive composition is specified after constructing the positional vector from the two axis-specific tables.
  • SPE Implementation: SPE constructs deterministic 2D sine-cosine features by allocating half the channels to each axis and concatenating the horizontal and vertical embeddings.It assumes d is divisible by 4 and uses frequencies ω_t=10000^{-t/(d/4)}.
  • SPE Implementation: For SPE-based attention, query and key vectors are converted to complex form by pairing consecutive channels, then allocated across horizontal and vertical axes.The formulation uses q_n,k_n ∈ R^d and complex representations in C^{d/2}.

RoPE Implementation · B.2 Computational Cost of WATERec

The RoPE implementation encodes two-dimensional patch positions as complex rotations applied element-wise to queries and keys, supporting relative relationships and variable-length sequences. WATERec uses no extra parameters over vanilla ViT, runs slower than SVTRv2 but faster than MAERec, while NAR methods remain more efficient.

  • RoPE Implementation: RoPE uses a base frequency θ=100 and assigns separate frequency schedules to the x and y spatial coordinates.The implementation defines coordinate-specific frequencies with θ as the base frequency.
  • RoPE Implementation: Each patch position is mapped to a complex rotation vector in C^{d/2} using its x and y coordinates.The rotation vector interleaves rotations derived from the x- and y-coordinate frequencies.
  • RoPE Implementation: The rotary embedding is applied to queries and keys through element-wise complex multiplication with the position-specific rotation.Both query and key vectors are rotated using the same position-dependent vector.
  • RoPE Implementation: The rotated query and key vectors are converted back from complex form to the real domain.The conversion produces real-domain query and key representations for subsequent model computation.
  • RoPE Implementation: This positional design captures relative relations without explicit learnable positional vectors and supports variable-length sequences across scales and resolutions.The passage attributes strong generalization across different scales and resolutions to this design.
  • B.2 Computational Cost of WATERec: NAR methods such as PD and CTC achieve higher inference efficiency than the compared Transformer-based approaches.The comparison uses runtime under the same setting, measured in frames per second.
  • B.2 Computational Cost of WATERec: WATERec runs slightly slower than SVTRv2 but faster than MAERec, while introducing no extra parameters relative to the vanilla ViT baseline.The runtime and parameter comparison was conducted with the OpenOCR framework on an NVIDIA V100 GPU.

B.3 Analysis of the C-Bench Trade-off · C More Evaluation Details

On regular C-Bench, fixed-resolution MAERec slightly outperforms WATERec because regular scene text has stable, near-horizontal aspect ratios suited to fixed-size resizing, unlike WordArt’s variable shapes, layouts, and extreme aspect ratios.

  • B.3 Analysis of the C-Bench Trade-off: On regular C-Bench, fixed-resolution MAERec slightly outperforms WATERec because regular text’s stable, near-horizontal aspect ratios suit fixed-size ViT resizing.The passage contrasts this distribution with WordArt’s highly variable shapes, layouts, and extreme aspect ratios, for which aspect-ratio preservation is more important.

C.1 Evaluation of VLMs · C.2 Fine-tuning VLMs with WATER-S

The evaluation uses text-only prompting tailored to general and OCR-specialized VLMs, while fine-tuning Qwen3-VL-8B with WATER-S tests whether adaptation narrows WordArt recognition gaps. The fine-tuning setup yields clear gains, though the supplied passage does not provide the complete resulting metric.

  • C.1 Evaluation of VLMs: General VLMs are evaluated with a fixed prompt requesting all readable text and forbidding explanations or descriptions.The evaluation directly targets text recognition in images.
  • C.1 Evaluation of VLMs: OCR-specialized VLMs use their official text-only prompts rather than the general VLM prompt.The supplied official example is “Extract the text from this image.”
  • C.1 Evaluation of VLMs: PP-OCRv5 evaluation calls only PP-OCRv5_server_rec, with all other parameters kept at official defaults.No additional modifications are applied.
  • C.2 Fine-tuning VLMs with WATER-S: Qwen3-VL-8B is fine-tuned with LoRA for 20k steps using ms-swift and WATER-S data.The experiment examines whether adaptation helps a strong general VLM close the WordArt gap.
  • C.2 Fine-tuning VLMs with WATER-S: Supervised fine-tuning brings clear gains on A-Bench, as reported in Table 4 comparing an expert baseline with SFT.The supplied passage begins the improvement range at 72.01% but is truncated before the complete result.
  • C.2 Fine-tuning VLMs with WATER-S: The final WATERec / RS model is illustrated with additional bad-case examples showing labels, predictions, and confidence values.The figure formats information below each image as label | prediction | confidence.
Loading 2606.24484v1…