PaperlayerView exact revision on arXiv

Source-linked AI summary

Systematic Literature Review of Machine Learning Models and Applications for Text Recognition

Nuzhat Khan, Ab Al-Hadi Ab Rahman, Shahriyar Masud Rizvi, Ibrahim Yousef Alshareef, Muhammad Nadzir Marsono, Muhammad Paend Bakht, Mohd Shahrizal Rusli, Shahidatul Sadiah

arXiv:2608.26500v1cs.CVcs.LG

TL;DR

OCR systems must recognize heterogeneous text across varying scripts, styles, and document conditions. This PRISMA-based review analyzes 97 studies to trace model evolution, applications, and persistent challenges, finding a shift toward deep learning and broader real-world use.

  • Problem

    OCR models face difficulty with font, style, layout, script, handwriting, and document-quality variation in heterogeneous text.

  • Method

    The review uses a PRISMA-based analysis of 97 research articles to examine OCR model evolution, application domains, dataset complexity, and accuracy improvements.

  • Results

    OCR research shifted from traditional techniques toward CNNs, LSTMs, GANs, and Transformers, while expanding from document digitization to healthcare, banking, scene text, multilingual, assistive, and immersive applications.

  • Takeaways & Limitations

    Real-time OCR can benefit from efficient transformer variants, quantization, knowledge distillation, pruning, and script-specific models such as BiLSTM for Devanagari.

  • Takeaways & Limitations

    OCR remains constrained by complex scripts, visually similar characters, and dataset limitations including insufficient size, scope, balance, and language coverage.

Abstract

from arXiv · show

Optical Character Recognition (OCR) for text recognition using machine vision has significantly improved, particularly when handling heterogeneous textual data. Traditional OCR models struggle with script variations, writing styles, and degraded documents. Advancements in technology are leading to new AI models with improved architecture for handling multiple languages and complex data formats. Despite this progress, a comprehensive evaluation of OCR advancements remains limited. Based on the established preferred reporting items for systematic reviews and meta-analysis (PRISMA) guidelines, this literature review presents an extensive assessment of OCR research to trace the evolution of AI models over the past decade. It explores the transition in AI models, application domains, data types, linguistic coverage, and challenges. Through a detailed analysis of 97 selected studies published during January 2015 - January 2025, key OCR models are identified, and their performance, strengths, and limitations are analyzed. The findings highlight how OCR technologies have evolved to address structured and unstructured text, scene text recognition, and multilingual processing. Unresolved challenges include limited resources for underrepresented languages, high variability in handwritten text, visual similarity among characters, and constraints in real-time OCR applications. To address these issues, several promising approaches are proposed. Key suggestions include self-supervised learning, multimodal AI, automated machine learning (AutoML), AI-assisted postprocessing, tiny machine learning (TinyML), and the creation of joint corpora for script matching. The future recommendations aim to enhance OCR accuracy and tackle the challenges identified for real-time industrial applications. This study will guide future research and establish a foundation for OCR field.

I. INTRODUCTION

OCR has evolved from rule-based recognition of structured printed text toward deep learning and Transformer-based systems that address varied scripts, handwriting, and contextual interpretation.

  • Traditional OCR used handcrafted features, template matching, and optical feature detection, but struggled with font, style, and layout variations.
  • Deep learning enabled OCR models to learn hierarchical features automatically from visual inputs.
  • CNNs became foundational for printed OCR by extracting spatial features and classifying characters.
  • LSTMs addressed sequential dependencies in handwritten and cursive text, improving recognition of connected characters.
  • Transformer models including TrOCR, ViT, and CharFormer use self-attention for contextual text interpretation and have achieved state-of-the-art results.
  • The review examines state-of-the-art models, multilingual recognition trends, OCR challenges, and gaps in research.

II. REVIEW METHODOLOGY

The review uses a PRISMA-based, systematic process to identify and analyze OCR studies from multiple academic databases, with explicit scope and selection criteria.

  • The review addresses a gap in structured comparisons of core OCR models, their strengths, limitations, and language or script performance.
  • Its scope covers OCR model development and transition from January 2015 to January 2025 under the PICOC framework.
  • Searches covered seven academic databases using predefined inclusion and exclusion criteria to select relevant studies.
  • 45,959 records were retrieved, reduced to 497 for title screening, 310 for abstract review, and 182 for full-text evaluation.
  • The search process refined keyword combinations and applied language, date, and article-type filters between 15–26 January 2025.

E. RESEARCH QUESTIONS AND MOTIVATION

The review is organized around model performance, script coverage, and system challenges, linking these questions to a comprehensive assessment of OCR research.

  • The review defines three research questions covering dominant AI models, supported writing scripts, and challenges with future research directions.
  • RQ1 examines OCR models by accuracy, efficiency, and multilingual performance.
  • RQ2 evaluates which writing scripts are well supported and identifies coverage gaps.
  • RQ3 investigates significant OCR challenges and research directions that may address them.
  • The combined questions form a framework for analyzing OCR models, inclusivity gaps, and obstacles to future advancement.
  • The review categorizes OCR applications broadly into printed and handwritten text across different languages and scripts.

A. AI MODELS IN OCR (RQ1)

OCR research has progressed from conventional and recurrent models toward hybrid and Transformer-based architectures. The review tracks model prominence and architectural evolution across the past decade.

  • Model landscape: The review manually analyzed 97 studies and grouped OCR models into conventional machine learning, deep learning, Transformer, and hybrid categories.Figures 6 and 7 present temporal trends and broader architectural groupings.
  • Deep learning models: CNNs remained widely used, while LSTMs gained prominence after 2017 for sequential and handwritten text recognition.CNNs supported image-based recognition, whereas LSTMs addressed sequential dependencies in connected characters.
  • Transformer models: Since 2020, Transformers have grown substantially and largely replaced LSTMs in OCR applications.Their dynamic feature selection supports sequential-data recognition, including handwritten text.
  • Auxiliary models: GANs increasingly support OCR by generating synthetic data, enhancing low-quality images, and refining recognition outputs rather than performing OCR classification directly.They function primarily as auxiliary techniques for other OCR models.
  • Hybrid models: Hybrid OCR models combine approaches to improve recognition when limited training data, distorted text, or interpretation challenges make a single method inadequate.Neural Fuzzy Systems are highlighted for processing noisy or highly variable text through fuzzy logic and learning models.

3) DECLINING TRADITIONAL MACHINE LEARNING MODELS

Traditional machine learning models declined as OCR datasets and visual variability became more complex. Deep learning, hybrid methods, GANs, and Transformers expanded recognition capabilities, while resource constraints remained relevant.

  • Traditional models: Before 2019, KNN, SVM, Naïve Bayes, Logistic regression, and statistical models were widely used when labeled data and computational resources were limited.These models were commonly applied to character classification.
  • Traditional models: Traditional models declined because manual feature engineering and limited capacity for fonts, handwriting styles, layouts, and high-dimensional data hindered complex OCR.Their earlier effectiveness was strongest on simple datasets.
  • Hybrid models: Hybrid architectures such as CNN-LSTM, CNN-RNN, and CNN-YOLO combine feature extraction with sequence learning but increase training time and computational costs.The review reports higher accuracy than standalone models while noting reduced feasibility for large-scale use.
  • Emerging and practical models: GANs supported low-resource or degraded-text OCR through synthetic-data generation, while lightweight models remained important for resource-constrained applications.These roles complemented the dominant CNN, LSTM, and Transformer architectures.
  • Transformer models: Transformers gained prominence for contextual understanding and multilingual capabilities, achieving 99.89% character-recognition accuracy in one reported result despite high computational requirements.Attention mechanisms support character and word relationship modeling and direct image-to-text conversion.

C. PERFORMANCE OF OCR MODELS ON HETEROGENEOUS DATASETS

OCR performance varies across heterogeneous datasets because studies differ in data, preprocessing, training, architecture, and decoding. Transformers perform strongly on multilingual and complex scripts, but their computational demands and data requirements constrain deployment.

  • Evaluation: Accuracy was the primary evaluation metric in 72% of studies, while reported values were extracted across datasets without averaging or statistical aggregation.The review cautions that differing training, testing, preprocessing, and splitting practices prevent direct identification of the best model.
  • Model performance: Transformer-based models outperform CNNs and LSTMs for multilingual and complex-script OCR through self-attention and long-range contextual modeling.They demonstrate state-of-the-art performance for languages with complex diacritics.
  • Computational trade-offs: TrOCR uses pretrained Vision Transformers for image encoding and autoregressive decoding, with base and large versions containing 334 million and 558 million parameters.These models require substantially more memory and longer inference than traditional CNN-based models.
  • Heterogeneous inputs: Real-scene OCR must handle low contrast, noise, skewed perspectives, motion blur, occlusion, partial visibility, and multilingual text.Models are challenged when training data does not reflect this variability.
  • Data coverage: Real-scene datasets often lack associated lexicons and remain limited in scale and script diversity relative to unconstrained environments.Domain-specific augmentation, advanced feature learning, and semi-supervised techniques are described as ways to bridge controlled and real-world inputs.
  • Generalization: Deep learning reduces dependence on manual feature engineering but relies on large, well-labeled datasets and may overfit when data are insufficient.Augmentation can simulate rotations, scaling, contrast shifts, and synthetic noise.

D. TEXT SCRIPTS IN OCR RESEARCH (RQ2)

OCR research increasingly addresses multilingual script recognition, but script-specific structural and typographical differences require tailored approaches rather than one universal model.

  • D. TEXT SCRIPTS IN OCR RESEARCH (RQ2): English OCR is comparatively mature because extensive training data, simple character sets, consistent spacing, and limited diacritics support high accuracy.English is also the most researched OCR language in the reviewed literature.
  • D. TEXT SCRIPTS IN OCR RESEARCH (RQ2): Arabic OCR requires specialized models for right-to-left writing, connected positional letter forms, diacritics, and similar-looking characters.The script’s letters can take four shapes depending on word position, increasing recognition complexity.
  • D. TEXT SCRIPTS IN OCR RESEARCH (RQ2): Multilingual OCR remains challenging because writing systems differ in structure, linguistic properties, and typography.The review identifies script variation as a central barrier to accurate recognition across languages.
  • D. TEXT SCRIPTS IN OCR RESEARCH (RQ2): The review examines prominent scripts and their supported languages to assess multilingual recognition and identify gaps in OCR research.The study also analyzes temporal trends in language-specific OCR models.
  • 1) LATIN SCRIPT: Latin script is relatively compatible with OCR because its consistently shaped, individually spaced characters are easy to segment and recognize.Its standardized digital representation further supports processing.

2) ARABIC SCRIPT

OCR limitations arise across data, model, and resource dimensions, with script complexity, noisy inputs, computational demands, and weak generalization constraining reliable deployment.

  • 3) CHINESE SCRIPT: Chinese OCR must handle massive character inventories, visually similar strokes, variable orientations, and substantial computational complexity.Reported responses include hierarchical encoding, transformer architectures, multi-scale features, synthetic augmentation, and orientation-aware processing.
  • E. KEY CHALLENGES IN ACCURATE OCR SYSTEMS: Frequently reported obstacles include noise, low contrast, resolution limits, character-shape variation, diacritics, and dataset scarcity.Less-explored issues include distortion, overlapping characters, skew, shading, imbalance, positioning errors, fragmented text, overfitting, and punctuation recognition.
  • E. KEY CHALLENGES IN ACCURATE OCR SYSTEMS: OCR challenges are grouped into data, model, and resource constraints, although these categories are interconnected and multilayered.Data issues concern quality, quantity, and availability; model issues include accuracy, speed, complexity, adaptability, and generalization.
  • E. KEY CHALLENGES IN ACCURATE OCR SYSTEMS: Complex scripts, diacritics, conjunct or connected characters, variable fonts, and overlapping text reduce recognition accuracy.Lighting changes and background complexity further hinder text recognition in natural scenes.
  • E. KEY CHALLENGES IN ACCURATE OCR SYSTEMS: Complex models may improve recognition while increasing computational cost and latency, limiting real-time OCR performance.Insufficient training data also raises risks of overfitting and poor generalization to new scripts and fonts.
  • E. KEY CHALLENGES IN ACCURATE OCR SYSTEMS: Hardware, memory, energy, dataset availability, and optimization constraints restrict real-time and embedded OCR deployment.Deep-learning complexity contributes to high latency in resource-constrained environments.

1) DATA-RELATED CHALLENGES

OCR models depend on extensive, high-quality labeled data, but available datasets are often limited, imbalanced, or poorly representative of real-world text. These data deficiencies hinder generalization across complex scripts and degraded conditions.

  • OCR models primarily rely on extensive, high-quality labeled training data.
  • Limited dataset size, scope, balance, and language coverage restrict adequate model generalization.
  • Complex scripts with strokes, diacritics, connected characters, and glyph structures require specialized training data.
  • Noisy, low-resolution, and degraded text conditions are insufficiently represented in many datasets.

2) MODEL-SPECIFIC LIMITATIONS AND COMPUTATIONAL CONSTRAINTS

OCR models face recognition and deployment limitations involving complex handwriting, similar characters, difficult layouts, data imbalance, and script variation. Computational constraints further complicate the balance among accuracy, speed, efficiency, and real-time operation.

  • Model-specific limitations: Overlapping, connected, handwritten, and cursive scripts remain difficult for OCR models to recognize.
  • Model-specific limitations: Visually similar characters such as ’O’ and ’0’ or ’2’ and ’z’ produce high misclassification rates.
  • Model-specific limitations: Complex layouts, handwritten scripts, and multi-column documents expose weaknesses in segmentation and text extraction.
  • Model-specific limitations: Small or unbalanced datasets can cause overfitting and inefficiency, while architectures often lack adaptation to script, font, and language variation.
  • Computational constraints: Real-time OCR is impractical on resource-limited embedded systems because of high power consumption and processing demands.
  • Computational constraints: Lightweight optimized models remain limited for mobile and edge environments, making accuracy, speed, and computational efficiency difficult to balance.

F. OCR ENHANCEMENT STRATEGIES (RQ3)

The review organizes OCR enhancement around data quality, model capabilities, and post-recognition refinement. It recommends adaptive and multimodal methods, specialized strategies for scene text, and automated optimization to address difficult inputs and deployment needs.

  • Enhancement approaches: Transformer architectures, self-supervised learning, adaptive preprocessing, multimodal learning, augmentation, and GAN-based denoising target handwritten, distorted, and low-resolution text.
  • Specialized applications: License plate and scene text recognition can use domain adaptation, improved segmentation, and multi-frame fusion for lighting variation and motion blur.
  • Enhancement approaches: OCR enhancement techniques span data-centric preprocessing, model-oriented optimization, and post-recognition refinement.
  • Data-centric strategies: Preprocessing addresses lighting, resolution, and geometric distortions through input-quality improvement and data curation.
  • Data-centric strategies: Future preprocessing pipelines could adapt dynamically to image characteristics using super-resolution, adaptive denoising, and confidence-based self-learning.

2) MODEL-ORIENTED STRATEGIES

Model-oriented OCR strategies select and optimize architectures for accuracy and computational efficiency, while post-processing corrects residual recognition errors. The review also points toward integrated, adaptive, and multimodal systems for broader robustness.

  • MODEL-ORIENTED STRATEGIES: Model-oriented strategies select OCR architectures for accuracy and computational efficiency, then tune hyperparameters and feature extraction.
  • MODEL-ORIENTED STRATEGIES: AutoML could automate architecture selection and hyperparameter optimization to identify efficient OCR configurations.
  • Post-recognition refinement: Post-recognition refinement addresses residual misclassifications and contextual inconsistencies in complex text environments.
  • Integrated systems: OCR has progressed from traditional image-processing pipelines toward integrated systems combining multiple machine-intelligence capabilities.
  • Future directions: Future systems should jointly process text, layout, and visual features while supporting multilingual, multiscript, zero-shot, and on-device adaptation.

1) ADVANCED LEARNING APPROACHES FOR L0W-RESOURCE LANGUAGES

The review identifies data scarcity, computational demands, and degraded inputs as central OCR challenges, highlighting learning, architecture, and image-restoration strategies for addressing them.

  • Advanced learning approaches for low-resource languages: Self-supervised and few-shot learning can improve low-resource OCR by extracting patterns from unlabeled text with fewer annotations.These approaches are presented as alternatives for insufficient training data and reduced manual labeling.
  • Optimized architectures for real-time applications: Efficient transformer variants, quantization, distillation, and pruning can reduce OCR model size and inference time for mobile and embedded deployment.The review identifies transformer computational cost as a key real-time limitation.
  • Image enhancement for degraded documents: GAN-based enhancement can reconstruct degraded text, mitigate background noise, and improve clarity in noisy or low-resolution images.SRGAN and DESRGAN are cited as document-enhancement approaches for challenging real-world environments.
  • Image enhancement for degraded documents: GANs can restore missing or corrupted visual content by generating plausible character structures at local and document-wide levels.The review contrasts this learned reconstruction with filters that only modify existing pixels, including applications to damaged license plates.
  • Review-level challenges and solutions: The review organizes OCR research challenges into data limitations, algorithmic constraints, and computational demands, recommending transfer learning, multimodal AI, and advanced pretrained models.These recommendations are intended to support faster, reliable, and multilingual OCR systems.
Loading 2608.26500v1…