PaperlayerView exact revision on arXiv

Source-linked AI summary

Open Vocabulary Word Recognition From Transcribed Bangla Texts

Faias Satter, Sk. Md. Masudul Ahsan

arXiv:2610.01134v1cs.CVcs.LG

TL;DR

Handwritten Bangla word recognition remains challenging, and the paper targets open-vocabulary recognition without character segmentation. It applies two object-detection models and an ensemble to a custom dataset; the ensemble recognizes 96.12% of test words to some extent.

Abstract

from arXiv · show

An optical character recognition (OCR) can scan a paper and extract text using technology, making people's jobs easier. While various OCR systems are available in the software industry, finding a reliable equivalent solution for Bangla takes much work. When it comes to handwritten texts, the situation is much more unusual. Recognizing words from word images is the most critical stage in any OCR process. It is the second stage after segmenting words from text pictures. If this stage fails, the overall performance of the OCR will be poor, regardless of how well the other phases perform. This study aims to recognize words using deep learning in a handwritten Bangla word image. Three object detection models, SSD with MobileNetV2, Faster R-CNN with InceptionResNetV2, and an ensemble model of these two, have been used to train and test handwritten word images. A modified Non-Maximum Suppression has been introduced to enhance the effectiveness of the models' results. A customized dataset of 9841 handwritten Bangla word images has been compiled, featuring diverse handwriting styles from various individuals. All three models' performances have been checked against the test dataset, and the ensemble model has been the most impressive, with an F1-score of 92.61%. Also, at the word level, the ensemble model correctly recognizes 96.12% of the words to some extent. The system can be further improved by introducing a post-processing phase to correct errors generated by the system.

I. Introduction

The paper targets open-vocabulary recognition of handwritten Bangla words, where script complexity and cursive writing complicate character recognition and segmentation. It proposes character-detection models and a bespoke word-image dataset for this task.

I. Introduction

  • The study focuses on recognizing handwritten Bangla words in images, addressing a stated lack of good OCR for Bangla.The introduction describes word recognition as OCR’s second stage, following word segmentation.Source: §I · p. 1
  • The system compares SSD with MobileNetV2, Faster R-CNN with InceptionResNetV2, and an ensemble selecting the highest-confidence prediction.The ensemble considers the results of both models.Source: §I · p. 1
  • Similar-looking characters, the connecting Matra, variant character forms, conjuncts, and differences in handwriting make Bangla word recognition challenging.The paper notes that characters do not have separate spaces within a handwritten word as they do in English.Source: §I · p. 1
  • The proposed system detects characters and rearranges them into words, aiming to recognize open-vocabulary words without first segmenting characters from word images.The authors note that segmentation-based approaches can be challenging for cursive Bangla handwriting.Source: §I · pp. 1–2
  • The authors created a handwritten Bangla word-image dataset that includes many non-words, or character combinations absent from the dictionary.The paper frames this dataset as support for its open-vocabulary recognition goal.Source: §I · p. 2

II. Related Works

Prior work spans handwritten Bangla character recognition, license-plate character detection, and word-level recognition. The cited studies use CNN-based methods, object detection, and end-to-end or holistic approaches.

II. Related Works

  • Bangla character-recognition studies used DCNNs, supervised layerwise training, and MobileNetV1; reported accuracies include 89.93% for 80 classes and 96.46% for 231 classes.The cited studies also report difficulty distinguishing similar-form characters and reduced error from 19% to 9.67%.Source: §II · p. 2
  • Bangla license-plate systems detected and reordered characters, reporting 91.67% precision for character detection and over 92.7% character-identification accuracy.The cited methods used Faster R-CNN with SSD, and a YOLO-based network with a ResNet-20 character recognizer.Source: §II · p. 2
  • For Bangla word recognition, cited methods include CRNN and Gated-CNN, CNN-RNN, holistic H-WordNet, and character spotting; Hasan et al. report 39.67% correctly recognized word images.H-WordNet reported 96.17% recognition accuracy on a Bangla word database.Source: §II · p. 2

III. Proposed Methodology

The proposed workflow is organized into steps and illustrated with a flowchart. Handwritten Bangla word images enter the process as binary images.

III. Proposed Methodology

  • The working procedure is organized into steps, with its flowchart presented as Fig. 1.The supplied passage does not describe the flowchart’s individual stages.Source: A. Working Procedure · p. 2, Figure 1 · p. 2
  • The methodology concerns a working procedure for recognizing handwritten Bangla word images.The cited passage identifies the workflow’s figure but does not specify its internal operations.Source: A. Working Procedure · p. 2
  • Handwritten Bangla word images are input as binary images, with black character pixels against a white background.Fig. 2 is captioned as an example of a binary image and character labels.Source: 1) Binary Word Images · p. 3, Figure 2 · p. 3

1) Binary Word Images:

The method uses labeled character boxes, a custom word-image dataset, and two parallel object-detection models. The dataset is split between training and evaluation-related use.

1) Binary Word Images:

  • Characters, vowels, and diacritics are labeled after bounding boxes are drawn, across 88 classes including numbers, punctuation, and 17 commonly used conjuncts.Fig. 2b illustrates character labeling.Source: 2) Labeling Each Character · p. 3

2) Labeling Each Character:

  • The dataset contains 9,841 labeled word images; nearly 80% train the models, while the remaining 20% support result manipulation and performance evaluation.The passage describes the vocabulary as open.Source: 3) Object Detection Models · p. 3

3) Object Detection Models:

  • SSD with MobileNetV2 and Faster R-CNN with InceptionResNetV2 are trained in parallel using pretrained models from the TensorFlow detection model zoo.Each model is fine-tuned using empirically determined parameter values.Source: 4) Train · p. 3

4) Train:

The paper trains two object-detection models to localize and recognize characters, then combines their outputs by selecting the highest-confidence result for each character. The resulting ensemble can draw different character predictions from the two models.

5) Result Acquisition From Models:

  • The ensemble merges SSD and Faster R-CNN outputs, selecting the highest-confidence result for each character.The models produce bounding boxes, character classes, and confidence scores.Source: 5) Result Acquisition From Models · p. 3

4) Train:

  • SSD uses MobileNetV2 as its feature extractor, while Faster R-CNN uses InceptionResNetV2.The paper also trains these two object-detection models independently.Source: 5) Result Acquisition From Models · p. 3

5) Result Acquisition From Models:

  • For the example word '১২', the ensemble selects '১' from SSD and '২' from Faster R-CNN, yielding '১২'.SSD predicted '১হ'; Faster R-CNN predicted '১২'.Source: 5) Result Acquisition From Models · p. 3

6) Applying Threshold on Class Confidence Scores:

The pipeline first removes low-confidence predictions, then uses overlap-aware suppression to select among competing classes for a character. It lowers the overlap threshold for specified diacritic cases to preserve overlapping characters.

6) Applying Threshold on Class Confidence Scores:

7) Non-Maximum Suppression(NMS):

  • When overlapping classes have OV below 2, the highest-confidence class is selected; in the example, 'ষ্ট' wins at 89%.The OV values for 'ষ' and 'ট' are 1.33 and 1.08, respectively, so both are eliminated in favor of 'ষ্ট'.Source: 7) Non-Maximum Suppression(NMS) · pp. 3–4
  • For specified overlapping diacritics and consonants or conjuncts, the OV threshold is reduced from 2 to 0.5 to preserve both.The passage gives 'ি◌' and 'ঘ' as an example: lowering the threshold helps retain the diacritic that otherwise would be eliminated.Source: 8) Preserving Necessary Overlap(PNO) · p. 4
  • The modified threshold is also used for '◌ঁ' overlapping characters with extended parts over the matra, while 'ে◌' and 'ৈ◌' are excluded from this step.Those diacritics can resemble consonants such as 'ট,' 'ঢ,' and 'চ' across handwriting styles, leading to multiple class detections.Source: 8) Preserving Necessary Overlap(PNO) · p. 4

8) Preserving Necessary Overlap(PNO):

After detection, the system reorders and processes characters to form the word text, accounting for Bangla characters and diacritics whose written and typed orders differ.

8) Preserving Necessary Overlap(PNO):

  • Detected characters are rearranged by coordinate value, with certain diacritics and '◌র্' moved before their corresponding character in writing but typed after it.The listed marks include 'ে◌', 'ি◌', 'ৈ◌', and 'ে◌ৗ'.Source: 9) Text Words · p. 4
  • When 'অ' and '◌া' occur together, they are substituted with 'আ'.'আ' was not one of the identified classes.Source: 9) Text Words · p. 4

9) Text Words:

  • The rearrangement also accounts for an exception cited as [17].The passage identifies it as an exception handled while reordering characters.Source: 9) Text Words · p. 4

IV. Results

The evaluation uses a customized collection of handwritten Bangla word images and character-level metrics, with macroaveraging across character classes.

A. Datasets

  • The customized dataset contains 9,841 handwritten Bangla word images, 70% collected from different writers and 30% from CMATERdb datasets.Images were segmented into individual words, binarized, manually fine-tuned, and labeled by character class.Source: A. Datasets · p. 4
  • The split uses 7,701 images for training and 2,140 for testing, with at least 50 instances of each class in the test set.The training set has 18,526 character, diacritic, number, and punctuation instances; the test set has 5,451.Source: A. Datasets · p. 4

B. Evaluation Metrics

  • Character-level recognition is measured with recall, precision, and F1-score, summarized across character classes by macroaveraging.True positives are correctly identified and recognized classes; false negatives are present characters not detected, and false positives are correctly identified but incorrectly classified characters.Source: B. Evaluation Metrics · p. 4

C. Experimental Results on Character Level

The ensemble performs best among the three proposed models on character recognition, with a reported recall of 96.83%. The section also compares the proposed method with prior character-recognition approaches.

C. Experimental Results on Character Level

D. Experimental Results on Word Level

The word-level evaluation uses Levenshtein Distance on 2,140 test words, with accuracy reflecting the proportion of correctly predicted classes within a word. The ensemble recognizes 96.12% of test words to some extent, though word-level scoring can obscure character-level performance.

D. Experimental Results on Word Level

  • Word-level performance was evaluated with Levenshtein Distance on 2,140 test words, where 100% means an exact match and 75% means three of four annotated classes were predicted correctly.Tables IV and V present cumulative accuracy and results when NMS and PNO are omitted.Source: D. Experimental Results on Word Level · p. 5
  • Single-character test words can score 0% when their character is missed, while one wrong character in a short word can yield 50% accuracy.The authors caution that these cases can make word-level performance appear different from character-level results.Source: D. Experimental Results on Word Level · p. 5
  • 96.12% of test words were recognized to some extent by the ensemble, compared with 75% word-recognition accuracy for Hossain et al. on poor handwriting.Hossain et al. tested only 100 word images and excluded conjuncts; their system relied on Matra Row alignment, unlike the proposed method.Source: D. Experimental Results on Word Level · p. 5

V. Conclusion

The paper presents an open-vocabulary handwritten Bangla word-recognition system using deep learning and a bespoke dataset. It reports comparisons across character and word levels and identifies limited conjunct and punctuation coverage, as well as post-processing, as areas for further work.

V. Conclusion

  • The open-vocabulary handwritten Bangla word-recognition system uses a bespoke dataset and deep-learning models, but includes only 17 conjunct characters, with almost 200 more potentially includable.The authors note that post-processing can be introduced to correct errors in the text.Source: §V · p. 5
  • Word-level accuracy and results with NMS and PNO omitted are presented in Tables IV and V, respectively.The conclusion also describes the recognized words as a basis for developing post-processing correction.Source: Table IV · p. 5, Table V · p. 6, §V · p. 6
  • The ensemble takes outputs from two parallelly trained object-detection models, while NMS and PNO are reported to help improve results.The models were trained on varied handwriting and are not limited to a closed vocabulary.Source: §V · p. 6
  • The system’s performance was measured at both character and word levels, and its three outputs were compared.The conclusion notes that recognized words could be used to develop post-processing word correction.Source: §V · p. 6

Related papers

Loading 2610.01134v1…