PaperlayerView exact revision on arXiv

Source-linked AI summary

Handwritten Optical Character Recognition (OCR): A Comprehensive Systematic Literature Review (SLR)

Jamshed Memon, Maira Sami, Rizwan Ahmed Khan

arXiv:2001.00139v1cs.CVcs.LG

TL;DR

Handwritten OCR research is dispersed across languages, techniques, and datasets, motivating a systematic synthesis of the field. The paper applies a predefined search and selection protocol to studies published from 2000–2018, retaining 142 articles. It reports script-dependent technique performance, growing CNN use, and gaps involving language coverage and real-world data.

  • Problem

    Evidence on handwritten OCR spans multiple languages and techniques, while many systems and datasets remain limited in language coverage and real-life writing variation.

  • Method

    The paper conducts an SLR using a predefined protocol, database and keyword searches, forward and backward reference searching, quality assessment, and data synthesis.

  • Results

    142 primary studies were selected, and the review found that technique performance varies by script while CNN use is increasing in handwritten and printed character recognition.

  • Takeaways & Limitations

    The review provides a multilingual account of handwritten OCR research and identifies research directions involving weaker language coverage, realistic datasets, and recognition in daily-life scenes.

  • Takeaways & Limitations

    Most studies address one language or a language subset, and public datasets often omit real-world writing styles, distorted strokes, variable thickness, and illumination.

Abstract

from arXiv · show

Given the ubiquity of handwritten documents in human transactions, Optical Character Recognition (OCR) of documents have invaluable practical worth. Optical character recognition is a science that enables to translate various types of documents or images into analyzable, editable and searchable data. During last decade, researchers have used artificial intelligence / machine learning tools to automatically analyze handwritten and printed documents in order to convert them into electronic format. The objective of this review paper is to summarize research that has been conducted on character recognition of handwritten documents and to provide research directions. In this Systematic Literature Review (SLR) we collected, synthesized and analyzed research articles on the topic of handwritten OCR (and closely related topics) which were published between year 2000 to 2018. We followed widely used electronic databases by following pre-defined review protocol. Articles were searched using keywords, forward reference searching and backward reference searching in order to search all the articles related to the topic. After carefully following study selection process 142 articles were selected for this SLR. This review article serves the purpose of presenting state of the art results and techniques on OCR and also provide research directions by highlighting research gaps.

1 Introduction

OCR converts document text into machine-encoded, searchable data, while handwritten OCR research addresses offline and online inputs using increasingly advanced machine-learning approaches. This review synthesizes multilingual handwritten OCR research and identifies weak areas for future investigation.

  • OCR converts handwritten and typewritten documents into machine-encoded formats, supporting digitization, retrieval, and preservation of historical, legal, and educational records.
  • Handwritten OCR is categorized into offline systems using scanned images and online systems using pen-movement data.Online systems capture velocity, projection angle, position, and locus information, making them more complex than offline systems.
  • OCR evolved from early reading machines to software deployed in education, census processing, stamped-character recognition, historical-document binarization, and accessibility applications.
  • Recent research combines machine-learning and image-processing techniques, with deep learning increasingly applied to handwritten-document digitization.
  • The SLR presents OCR literature across languages and highlights weak areas requiring further investigation.

2 Review methods

The review uses a predefined SLR protocol to search, screen, assess, and synthesize handwritten OCR studies. Automatic and manual searching produced a final set of 142 studies after duplicate removal, eligibility screening, and quality assessment.

  • Review objectives: The review summarizes machine-learning techniques and databases for handwritten character recognition across different languages.
  • Review objectives: It identifies research weaknesses and new research areas within handwritten OCR.
  • Review protocol: The SLR protocol specifies the review background, search strategy, data extraction, research questions, and quality-assessment criteria.The protocol is intended to improve consistency and reduce researcher bias relative to traditional reviews.
  • Eligibility and search: The review included journal, conference, symposium, and workshop studies on six languages published from January 2000 through December 2018.
  • Eligibility and search: 954 records were found through keyword searching, while screening removed keyword-matched but irrelevant, duplicate, unavailable, or research-question-inconsistent articles.
  • Search strategy: Automatic database searching was supplemented by forward and backward reference searching, snowballing, and spreadsheet-based data extraction.The databases included IEEE Xplore, ISI Web of Knowledge, Scopus–Elsevier, and Springer.
  • Study selection: 512 duplicate studies were removed from 954 records, 230 of the remaining 442 were excluded, and manual searching added 43 studies to a pre-final set of 166.
  • Study selection: 24 studies failed final quality assessment, leaving 142 primary studies for the SLR.The quality assessment used five questions scored at two marks each, with a selection threshold of at least 5 out of 10.

3 Statistical results from selected studies

The review analyzes 142 selected handwritten OCR studies by publication source, citations, publication year, and investigated language. Publications increased after 2010, with English and Arabic studies most represented.

  • Overview: 142 selected studies were statistically analyzed by publication source, citation count, temporal distribution, language, and research methodology.The extracted statistics cover the review’s selected primary studies.
  • Publication sources: 87 studies appeared in research journals (61%), 47 in conference articles (33%), 5 in workshops, and 3 in symposiums.Journal and conference publications constituted the majority of included studies.
  • Research citations: Approximately 95% of selected studies had at least one citation; 33 had more than 100 citations, while 46 had 1–15 citations.Citation counts were obtained from Google Scholar.
  • Temporal view: 59 publications appeared during the last eight years of the review period, with publication counts rising steadily in 2017 and 2018.The review reports additional increases in 2002, 2007, and 2009.
  • Investigated languages: English accounted for 45 studies, Arabic for 40, Indian scripts for 26, Chinese for 17, Urdu for 14, and Persian for 11.Some selected articles addressed multiple languages.
  • Language over time: Figure 7 provides a year-by-year language view of handwritten OCR research from 2000 to 2018.Some articles covered more than one language.

4 Research questions

The review’s research questions determine its search queries and keywords. They are intended to help researchers understand the state of optical character recognition research.

  • Research questions: The research questions determine the search queries and keywords used to explore research publications.They are presented in Table 4 alongside their motivation.
  • Motivation: The questions are designed to support both experienced researchers and those entering optical character recognition research.The stated purpose is to clarify where the field stands.

5 Classification methods of handwritten OCR

Handwritten OCR classification trains algorithms on known datasets to map alphabets and digits to predefined classes. The review discusses neural-network approaches, including MLPs, RNNs, and CNNs, across languages and studies.

  • Classification: Classification learns a model from input data and maps alphabets or digits to predefined categories or classes.The reviewed classification techniques span studies published from 2000 through 2018.
  • Artificial neural networks: Artificial Neural Networks use interconnected processing units called neurons to model input data and map it to predefined labels.In supervised learning, node weights are adjusted to reduce squared error on labeled training samples.
  • Deep architectures: Deep architectures such as Recurrent Neural Networks and Convolutional Neural Networks renewed interest in neural-network methods.The passage links this development to architectures with many layers.
  • MLP: Multi Layer Perceptrons are feed-forward neural networks whose weights can be estimated using error back-propagation.The review describes early handwritten OCR applications to Urdu, Farsi, and Bangla numerals.
  • CNNs: Convolutional Neural Networks have been widely used for classification and recognition across nearly all languages reviewed.The review reports great success for CNNs in character-recognition tasks.

5.2 Kernel methods

Kernel methods support handwritten OCR classification by mapping feature vectors into higher-dimensional spaces and finding separating hyperplanes. The review highlights SVM, KFDA, and KPCA as relevant kernel-based approaches.

  • Kernel models: Kernel-based models including SVM, KFDA, and KPCA have practical relevance for classification, text categorization, and related prediction problems.These models are presented as powerful kernel-based learning approaches.
  • Support Vector Machines: An SVM kernel maps feature vectors into a higher-dimensional space to find a hyperplane that separates classes with as much margin as possible.The classifier is defined for labeled training examples and a new test example.
  • SVM formulation: In the SVM decision function, K(., .) is the kernel function, b is the hyperplane threshold, and α_i are Lagrange multipliers describing the separating hyperplane.These symbols specify components of the classification function.
  • Applications: Before deep learning became popular, SVM was considered a robust technique for handwritten digit recognition and several image and text classification tasks.KFDA and KPCA were also used in offline handwritten character-recognition systems.

5.3 Statistical methods

The review describes statistical classifiers used in handwritten OCR, distinguishing parametric models with fixed parameters from flexible non-parametric methods. kNN is highlighted as a widely used classifier that assigns classes from nearby training examples.

  • Parametric classifiers use a fixed number of parameters and can learn quickly from small training sets.Examples include Logistic Regression, Linear Discriminant Analysis, and Hidden Markov Models.
  • Non-parametric classifiers are more flexible but generally increase in complexity with training-set size.kNN and Decision Trees are given as examples.
  • kNN is a widely used, easy-to-train non-parametric method for handwritten OCR classification.The review notes its use across different datasets and reports relatively good experimental performance.
  • kNN estimates an unknown sample’s class from the nearest training samples and their target-function values.The model calculates the probability that sample q belongs to class y using its nearest-neighbor set.
  • kNN represents K as the nearest-neighbor set, k_y as a neighbor’s class, and d(k, q) as Euclidean distance.
  • In majority-vote kNN, an object receives the class most prevalent among its k nearest neighbors; with k = 1, it takes the single neighbor’s class.
  • Earlier OCR studies also used HMMs, which model data as a Markov process with hidden states.In a Markov process, the probability of one state depends only on the previous state.

5.4 Template matching techniques

Template matching classifies characters by comparing image regions or features with predefined templates using similarity or distance measures. The review distinguishes rigid matching from deformable matching, which accommodates writer-specific shape variation.

  • Template matching compares image regions or features with predefined templates to classify characters.A sliding-window process determines similarity between the template and the image.
  • Common template-matching measures include Euclidean distance, city block distance, cross correlation, and normalized correlation.
  • Template matching techniques are grouped into rigid and deformable shape-matching families.The review presents their taxonomy in Figure 9.
  • Deformable template matching compares writer-shaped deformations with known database images to accommodate character variation between writers.It is further divided into parametric and other subclasses in the review.
  • Rigid template matching ignores shape deformations and commonly extracts or matches image features, including with Hough transforms.The review cites Arabic and Chinese OCR applications.
  • Correlation-based matching first calculates image similarity, then extracts and compares features from specific regions.

5.5 Structural pattern recognition

Structural pattern recognition classifies characters through relationships among structural primitives such as edges, contours, and connected components. Graphs, grammars, strings, and trees provide representations for comparing or classifying these structures.

  • Structural pattern recognition classifies objects through relationships among primitives such as edges, contours, and connected-component geometry.Chain Code Histogram is one image primitive used in OCR.
  • Structural models are divided by context into graphical methods and grammar-based methods.
  • Graphs represent OCR structures as nodes and edges, whose arrangement defines characters, digits, or alphabets.For OCR, edges can represent writing-stroke arcs connecting nodes.
  • Graph-based recognition compares structural arrangements using measures such as similarity flooding, SimRank, graph similarity scoring, and vertex similarity.
  • Grammar-based analysis finds similarities among graph primitives using formal grammar restrictions such as unrestricted, context-free, context-sensitive, and regular grammar.
  • Grammar-based OCR models use strings or trees to represent structures and classify characters through generated strings or hierarchical primitive relations.

6 Datasets

The review emphasizes standardized datasets as essential for meaningful OCR evaluation and surveys widely used resources across English, Arabic, Indian, Chinese, Urdu, and Persian or Farsi languages. These datasets vary in scripts, sample types, writers, and train-test organization.

  • Standardized datasets support meaningful comparison of OCR algorithms, while sufficient training and testing data are fundamental to quality research.
  • English datasets: CEDAR was developed in 2002 as an early large handwritten-character database, with images scanned at 300 dpi.
  • English datasets: MNIST contains 60,000 training and 10,000 test images of handwritten digits, normalized as 28 x 28 grayscale images.The normalization reduces preprocessing and formatting requirements.
  • Urdu datasets: UCOM contains 53,248 Urdu characters and 62,000 nasta’liq words from 100 writers, and supports character recognition and writer identification.Its evaluation uses 50 text-line training images and 20 test images, with a reported error rate of 0.004 -0.006%.
  • Arabic datasets: IFN/ENIT contains 26,459 handwritten images of Tunisian town and village names, totaling 212,211 characters from 411 writers.It is widely used for Arabic handwriting-recognition research.
  • Farsi datasets: CENPARMI Farsi resources range from 18,000 numeral samples to an extended dataset of 432,357 images spanning dates, words, letters, digits, symbols, and documents.The 18,000-sample version is divided into 11,000 training, 2,000 verification, and 5,000 testing samples.
  • Chinese datasets: HCL2000 contains 3,755 frequently used Chinese characters written by 1,000 subjects, with separate character and writer-information subsets.
  • English datasets: IAM contains English sentences from 400 writers, comprising 1,066 forms and a vocabulary of 82,227 words; it was also used for writer identification.Experiments identified writers 98% of the time.

7 Languages

The review focuses primarily on six languages, with English contributing the most studies. It also reports language-specific recognition results and identifies endangered languages as an area needing further OCR research.

  • Language coverage: Six languages dominate the reviewed OCR literature: English, Arabic, Indian scripts, Chinese, Urdu, and Persian.The authors identify OCR systems for other languages as future work.
  • Research gap: At least 43% of the world’s languages are endangered, motivating OCR research that could preserve documents from languages receiving limited attention.The review explicitly identifies building OCR systems for languages beyond the six commonly studied ones as future work.
  • English: English handwritten OCR has been extensively studied and has supported a wide range of commercial applications.The review lists both handcrafted feature methods and deep neural networks among techniques used for English OCR.
  • Farsi / Persian script: 99.5% recognition was reported for Farsi characters using an RBF-kernel support vector machine.Another Farsi system using Fast Zernike wavelet moments and ANN improved average recognition speed eightfold.
  • Farsi / Persian script: 99.40%, 99.16%, and 99.73% were reported for handwritten Bangla and Farsi numerals on three public datasets, respectively.The reported datasets were ISI Bangla numerals, CENPARMI Farsi numerals, and IFHCDB Farsi numerals.
  • Urdu language: Urdu studies reported 94% ligature separation, 98.61% isolated-digit recognition, and up to 99.33% accuracy with LSTM.A texture-based multilingual script-identification method reached 88.6% for three scripts and 97.5% for two scripts.
  • Arabic script: Arabic font recognition using diacritic segmentation and validation achieved an average recognition rate of 98.73% for ten popular fonts.The method was described as having minimum computation cost and being faster than other approaches.

8 Research trends

From 2015 to 2018, handwritten OCR research shifted toward deep learning, especially CNNs, while traditional techniques continued to be combined with neural models. The reviewed studies span multiple scripts and a broad range of feature and classification methods.

  • Recent shift: Recent OCR research has moved toward deep learning with little emphasis on handcrafted features.The review analyzes techniques used mainly in publications from 2015–2018.
  • Recent shift: CNNs are used extensively for optical-character recognition across the reviewed scripts.The review also reports continued use of SVM, HMM, and SIFT alongside CNNs.
  • Evidence base: The trend table organizes studies by script, employed technique, publication year, and reference number.It provides a holistic view of approaches used for widely studied languages.
  • Script coverage: The trend evidence covers English, Arabic, Chinese, Indian scripts, and other entries represented in the review’s script categories.The Indian category includes scripts such as Devanagari, Bangla, Hindi, Gurmukhi, and Kannada.
  • Methods: The reviewed techniques include CNN, LSTM, HMM, CRNN, recurrent neural networks, SVM, random forests, and MLP.The table also includes handcrafted descriptors and representations such as HOG, wavelets, texture, entropy, and word graphs.

9 Conclusion and future work

The review finds that handwritten OCR performance depends on language and dataset characteristics, while current studies remain narrow in language coverage and real-world variability. It therefore points toward broader-language datasets, robust text-in-the-wild systems, and practical commercialization.

  • Conclusion: Machine learning and deep learning have enabled individual researchers to develop more accurate handwritten-manuscript recognition algorithms.
  • Conclusion: Multilayer perceptron classifiers perform better on Devanagari and Bangla numerals than on several other languages, reflecting script and dataset differences.
  • Conclusion: Most studies address one language or a language subset, while public datasets underrepresent real-world writing styles, distorted strokes, character thickness, and illumination.
  • Conclusion: Convolutional Neural Networks are increasingly used for handwritten and machine-printed character recognition because their architectures suit image-based recognition tasks.
  • Future work: Future work should extend OCR research to regional and endangered languages to support preservation of vulnerable communities’ cultural heritage.
  • Future work: Text-in-the-wild research requires systems and challenging datasets that handle background clutter, variable illumination, camera angles, and multilingual daily-life text.
  • Future work: Commercializing OCR research could help produce low-cost real-world systems that convert valuable information into searchable digital data.
Loading 2001.00139v1…