Source-linked AI summary
Confidence-Aware Ensemble and Long-Word Refinement for Artistic Text Recognition
Lucas A. Dias, Henrique A. Schulz, Rafaela de Miranda, Guilherme L. Peres, Pedro L. Bittencourt, Rayson Laroca
TL;DR
Artistic Text Recognition remains difficult when designed words combine stylization, unusual layouts, and severe distortions. The paper evaluates fine-tuned recognizers on WordArt-V1.5, combines them with confidence-aware fusion and long-word refinement, and reaches 89.90% WRA on Test B. Its error analysis also finds that many remaining mistakes involve labeling, ambiguity, or illegibility.
Problem
Artistic Text Recognition is challenging because artistic word images combine decorative forms, unusual layouts, object-like characters, clutter, and severe distortions.
Method
The paper fine-tunes SVTRv2, PARSeq, and MAERec, selects among disagreements using confidence, and refines long words with sequence alignment and lexicon-guided correction.
Results
89.90% WRA on WordArt-V1.5 Test B, improving the best individual fine-tuned model by 1.77 percentage points.
Takeaways & Limitations
The compact ensemble recovers errors beyond a single strong recognizer, while targeted refinement improves its long-word subset by 2.72 percentage points.
Takeaways & Limitations
48.8% of the 303 remaining Test B errors were associated with labeling issues, visual ambiguity, or illegible samples.
Abstract
from arXiv · showhide
Artistic Text Recognition (ATR) remains challenging because word images often combine decorative fonts, curved layouts, object-like characters, clutter, and severe distortions. This paper studies WordArt-V1.5 as a standardized benchmark for this setting and evaluates recent scene and artistic text recognizers under a common protocol. We propose a confidence-aware ensemble that combines SVTRv2, PARSeq, and MAERec after fine-tuning on the official training split. The ensemble selects predictions using the minimum confidence over disagreement positions, emphasizing characters that separate competing hypotheses. For long words, where a single character error can invalidate the whole prediction, we add a targeted refinement stage based on Needleman-Wunsch alignment and lexicon-guided correction. On the WordArt-V1.5 Test B split, the proposed system reaches 89.90% Word Recognition Accuracy, improving the best individual fine-tuned model by 1.77 percentage points. The long-word refinement produces a modest global gain, but improves the targeted long-word subset by 2.72 percentage points. Finally, an error analysis of all remaining mistakes shows that 48.8% are associated with labeling issues, visual ambiguity, or illegible samples, highlighting the value of diagnostic reporting for future ATR benchmarks and models. Our source code is available at https://github.com/lucas-azdias/Artistic-Text-Recognition/.
I. INTRODUCTION
Artistic Text Recognition targets visually designed words whose decorative forms, layouts, and distortions create challenges beyond conventional scene text. This paper evaluates recent recognizers on WordArt-V1.5 under a fixed protocol and proposes compact ensemble and long-word refinement strategies.
- Motivation: ATR reads words shaped by visual design, including non-standard fonts, color variation, object-like characters, and severe deformations.These properties introduce both visual ambiguity and language-level uncertainty.
- Benchmark: WordArt-V1.5 provides standardized partitions for controlled comparisons among general-purpose recognizers, ATR-oriented models, and ensembles.The dataset contains 6,000 training images and a 6,000-image test set split evenly into Test A and Test B.
- Approach: The paper fine-tunes MAERec, SVTRv2, and PARSeq under a common protocol, then combines them with disagreement-aware confidence selection.The long-word module addresses word-level sensitivity to isolated character errors.
- Contributions: The contributions include a standardized WRA benchmark, confidence-aware ensemble with targeted long-word refinement, and analysis of 303 remaining Test B mistakes.The authors provide source code to support reproducibility.
II. RELATED WORK AND WORDART-V1.5 PROTOCOL
The paper situates ATR among evolving scene-text recognizers and uses complementary recent architectures as benchmark candidates. WordArt-V1.5 extends this setting toward artistic text with altered shapes, spacing, and boundaries.
- Related Work: Scene text recognition has progressed from convolutional and recurrent models toward transformer, masked-reconstruction, and refined CTC-based approaches.ASTER addressed irregular layouts through rectification and attention, while ViTSTR used a compact Vision Transformer without recurrent decoding or explicit rectification.
- Related Work: Recent ATR-relevant models include CornerTransformer, PARSeq, MAERec, and SVTRv2, which use complementary contour, decoding, representation, and CTC strategies.Their complementary architectures make them suitable candidates for benchmarking and ensemble construction.
B. WordArt-V1.5 Protocol
WordArt-V1.5 uses a fixed benchmark protocol and full-word accuracy metric for evaluating artistic text recognition. Its metric makes longer labels especially vulnerable to isolated character errors.
- Metric: Word Recognition Accuracy (WRA) ignores case and symbols and requires the normalized full prediction to match the ground truth.It is a word-level metric rather than a character-level average.
- Metric: Any incorrect, missing, or inserted character invalidates the full word prediction, making longer labels more exposed to errors.This motivates analyzing label-length statistics.
- Evaluation Protocol: The paper fine-tunes public checkpoints on Train, uses Test A for validation and model selection, and reserves Test B for final reporting.The public leaderboard provides context rather than evidence of an official competition entry.
C. Public Baselines on WordArt-V1.5
Public WordArt-V1.5 systems commonly combine complementary models and objectives. The paper instead constructs a compact, reproducible baseline using public checkpoints, fixed fine-tuning, and transparent fusion.
- Public Baselines: Existing systems use supervised, semi-supervised, self-supervised, ensemble, distillation, contrastive-adaptation, and mixed-model strategies.Examples include ViettelAI-OCR, Let Me See, and iPad_OCR.
- Public Baselines: The proposed baseline prioritizes compactness and reproducibility rather than reproducing a larger leaderboard-oriented pipeline.It uses public checkpoints, a fixed fine-tuning protocol, and a transparent fusion rule.
A. Candidate Models and Fine-Tuning
The paper selects three complementary recognizers, fine-tunes them under a fixed protocol, and uses Test A to guide model and checkpoint selection before evaluating the ensemble.
- Candidate selection: SVTRv2, PARSeq, and MAERec were selected from five candidates for complementary performance and architectural diversity.Their Test A WRA values before fine-tuning were 85.80%, 83.97%, and 83.40%, respectively.
- Candidate selection: 90.43% was the theoretical Test A oracle upper bound if a correct prediction could always be chosen whenever at least one model was right.Only 287 of 677 errors were shared by all three selected models, motivating fusion.
- Fine-tuning protocol: Each recognizer was fine-tuned on the official training split, with Test A reserved for validation, hyperparameter selection, and checkpoint selection.The models used AdamW and were trained for at most 50 epochs.
- Fine-tuning protocol: After fine-tuning, SVTRv2 reached 86.23% WRA on Test A, while PARSeq and MAERec both reached 84.53%.Online geometric and photometric augmentations were used to reduce sensitivity to distortions and appearance variations.
B. Disagreement-Aware Confidence Rule
The ensemble identifies positions where recognizers disagree and selects the model with the strongest minimum confidence at those positions, with normalization for calibration differences.
- Disagreement detection: The rule identifies sequence positions at which at least two recognizers disagree.Each recognizer produces a predicted sequence and character-level confidence scores.
- Confidence scoring: A model’s score is its minimum confidence over disagreement positions, or over the whole sequence when all predictions agree.This focuses scoring on characters that can change the final word.
- Prediction selection: The ensemble outputs the sequence predicted by the model with the largest score.The selected prediction therefore comes from the recognizer judged most reliable under the disagreement-focused score.
- Confidence scoring: Scores are normalized by each model’s mean character confidence to reduce calibration differences across recognizers.The rule penalizes uncertainty at positions where recognizers disagree rather than relying on global average confidence.
C. Long-Word Refinement
The long-word module targets labels with at least nine characters, aligning model outputs before applying confidence-triggered lexicon correction to address isolated character errors.
- Motivation and scope: Words with at least nine characters contain 749 samples in the full dataset and 184 samples in Test B.Because word accuracy requires every character to be correct, longer labels accumulate more opportunities for word-level errors.
- Alignment and voting: Predictions from SVTRv2, PARSeq, and MAERec are aligned with Needleman-Wunsch before character-level voting.Alignment handles insertions and deletions that shift positions across model outputs.
- Lexicon correction: Lexicon correction is considered when an aligned word contains a character with confidence below 60%.The correction searches the wlist_match3 English word list using bigram-based Levenshtein similarity and requires at least 70% similarity for acceptance.
- Motivation and scope: The refinement is restricted to long words with low-confidence evidence rather than applied to every output.This limits potential harm for brand names, invented words, and proper nouns absent from the lexicon.
IV. RESULTS AND ANALYSIS
On the controlled WordArt-V1.5 Test B evaluation, the compact ensemble improves over the best individual fine-tuned model, while qualitative examples show both successful selection and persistent ambiguity.
- Overall Recognition Accuracy: 89.90% WRA was achieved by the proposed ensemble on Test B, outperforming the best individual fine-tuned model by 1.77 percentage points.SVTRv2 remained the strongest individual model, while PARSeq and MAERec supplied alternative hypotheses.
- Overall Recognition Accuracy: The comparison uses the same Test B split and WRA metric for all entries.The system is presented as a transparent compact baseline rather than a replacement for larger leaderboard-oriented pipelines.
- Qualitative analysis: The ensemble selects a correct hypothesis in some cases where another recognizer produces a visually similar wrong word.These examples illustrate the practical effect of confidence-aware fusion.
- Qualitative analysis: In other cases, all recognizers converge on the same wrong word or assign high confidence to an incorrect interpretation.The remaining errors include stylized shapes with multiple plausible readings.
B. Effect of Long-Word Refinement
Long-word refinement provides a modest overall gain because it is applied to a small targeted subset, where alignment and dictionary correction improve difficult cases.
- Overall effect: 0.23 percentage points: complete refinement raises Test B WRA from 89.67% to 89.90% across all 3,000 test samples.The gain is modest because the module is intentionally targeted.
- Targeted effect: 2.72 percentage points: refinement improves accuracy within the 184-sample long-word subset.The subset targets words with at least nine characters, where a single shifted character can invalidate word-level recognition.
- Threshold selection: The nine-character threshold yields 5.70% coverage and 87.73% WRA on Test A, balancing refinement coverage against accuracy.Thresholds of 10 and 11 characters reduce coverage to 2.53% and 1.13% without consistent accuracy gains.
- Mechanism: Needleman-Wunsch alignment helps with insertion or deletion disagreements, while dictionary correction addresses localized low-confidence characters.The combined strategy improves intended cases without degrading remaining predictions.
C. Qualitative and Error Analysis
Manual analysis of 303 Test B errors separates model failures from labeling issues, ambiguity, and illegibility, clarifying what recognition accuracy alone cannot explain.
- Error categories: 51.2% of the 303 remaining Test B errors are model errors, commonly involving cursive characters, heavy stylization, missing strokes, or decorative components.These cases represent recognition failures associated with difficult visual forms.
- Error categories: 48.8% of errors are associated with labeling issues, ambiguity, or illegibility rather than purely model failure.The breakdown is 33.0% labeling errors, 10.9% ambiguity, and 5.0% illegibility.
- Interpretation limits: Labeling errors can penalize correct predictions when visual content differs from the official label.Ambiguous samples admit another visually defensible reading, while illegible samples lack sufficient evidence for reliable word-level decisions.
- Benchmark implications: WordArt-V1.5 is useful as a diagnostic benchmark as well as for ranking methods.Reporting error categories helps prevent attributing every remaining error to model weaknesses and supports interpretation of small global gains.
V. CONCLUSIONS
The paper concludes that a confidence-aware ensemble with targeted long-word refinement improves ATR on WordArt-V1.5, while error analysis exposes limits of recognition accuracy alone.
- Conclusions: 89.90% WRA: the ensemble improves the best individual fine-tuned model by 1.77 percentage points on Test B.The system combines SVTRv2, PARSeq, and MAERec with disagreement-aware confidence selection and long-word refinement.
- Conclusions: 2.72 percentage points: long-word refinement improves accuracy on its targeted subset despite a modest global improvement.The refinement addresses isolated character errors under word-level evaluation.
- Limitations: 48.8% of the 303 remaining Test B errors involve labeling issues, visual ambiguity, or illegibility.These limitations are not captured by recognition accuracy alone.
- Future work: Future work will explore learned fusion, generative training-data expansion, VLM-based filtering, and complementary long-word and uncertainty-aware metrics.These directions target styles, long words, rare characters, challenging distortions, readability, and label consistency.