PaperlayerView exact revision on arXiv

Source-linked AI summary

HMM-based Indic Handwritten Word Recognition using Zone Segmentation

Partha Pratim Roy, Ayan Kumar Bhunia, Ayan Das, Prasenjit Dey, Umapada Pal

arXiv:1708.00227v1cs.CV

TL;DR

Indic handwritten-word recognition is difficult because modifiers, compounds, touching, and overlaps complicate character segmentation, while prior Indic word-recognition work is limited. The paper segments words into three zones and recognizes their components using zone-wise methods. It reports perfect recognition for the illustrated words despite differing skew and slant angles, while noting specific segmentation errors and difficult cursive cases.

  • Problem

    Noise, touching, and overlapping make character segmentation difficult in Indic scripts, and the paper notes that only a few works address Indic handwritten-word recognition.

    Source: §1 · p. 2

  • Method

    The framework segments words into upper, middle, and lower zones, recognizes zone components, and combines them into full words.

    Source: §4.3.3 · p. 16, §5.3 · p. 26

  • Results

    The authors report that the illustrated words with different skew and slant angles are recognized perfectly by the proposed system.

    Source: §5.3 · p. 26

  • Takeaways & Limitations

    Zone-wise modeling reduces the character classes needed for Indic recognition by representing complex combinations as simpler units.

    Source: §1 · p. 3

    Segmentation can fail when the Matra is omitted, lower-zone modifiers are misidentified, or cursive overlaps make words difficult to interpret.

    Source: §5.4 · p. 28

Abstract

from arXiv · show

This paper presents a novel approach towards Indic handwritten word recognition using zone-wise information. Because of complex nature due to compound characters, modifiers, overlapping and touching, etc., character segmentation and recognition is a tedious job in Indic scripts (e.g. Devanagari, Bangla, Gurumukhi, and other similar scripts). To avoid character segmentation in such scripts, HMM-based sequence modeling has been used earlier in holistic way. This paper proposes an efficient word recognition framework by segmenting the handwritten word images horizontally into three zones (upper, middle and lower) and recognize the corresponding zones. The main aim of this zone segmentation approach is to reduce the number of distinct component classes compared to the total number of classes in Indic scripts. As a result, use of this zone segmentation approach enhances the recognition performance of the system. The components in middle zone where characters are mostly touching are recognized using HMM. After the recognition of middle zone, HMM based Viterbi forced alignment is applied to mark the left and right boundaries of the characters. Next, the residue components, if any, in upper and lower zones in their respective boundary are combined to achieve the final word level recognition. Water reservoir feature has been integrated in this framework to improve the zone segmentation and character alignment defects while segmentation. A novel sliding window-based feature, called Pyramid Histogram of Oriented Gradient (PHOG) is proposed for middle zone recognition. An exhaustive experiment is performed on two Indic scripts namely, Bangla and Devanagari for the performance evaluation. From the experiment, it has been noted that proposed zone-wise recognition improves accuracy with respect to the traditional way of Indic word recognition.

1. Introduction

Indic handwritten-word recognition remains comparatively underexplored, and character-level segmentation is difficult when noise, touching, and overlapping produce disjoint components. The paper proposes zone-wise HMM recognition to reduce Indic character classes and extends the framework to Bangla and Devanagari.

1. Introduction

  • Character segmentation can fail amid noise and touching, while overlapping and touching characters particularly hinder segmentation in Bangla writing.Preliminary segmentation can also produce disjoint components.Source: §1 · p. 2
  • Holistic word-level HMMs are limited to trained words and require substantial training data per word, so they cannot recognize untrained words.Sub-word character HMMs can instead be concatenated to recognize arbitrary word strings, but Indic scripts have many character classes due to modifiers and compounds.Source: §1 · p. 3
  • Zone-wise modeling reduces Bangla character classes from 1,960 to 287, an 85.36% reduction, by modeling upper, middle, and lower components separately.The paper contrasts XY+YZ classes without zone segmentation with X+Y+Z using it.Source: §1 · p. 3
  • The paper extends an earlier zone-wise approach with water-reservoir integration, PHOG features for middle-zone recognition, and evaluation on Bangla and Devanagari.The authors state that their framework is generalized and tested for both scripts.Source: §1 · pp. 3–4

2. Properties of Bangla and Devanagari Scripts

Bangla and Devanagari words combine base characters with modifiers and compound forms, while their shared horizontal headline and baseline help define three recognition zones. These combinations and handwriting distortions complicate character segmentation and recognition.

2. Properties of Bangla and Devanagari Scripts

  • Modifiers and compound characters expand the possible forms: vowel modifiers may appear left, right, both sides, or below a consonant.Characters typically hang from the Matra, and neighboring characters’ horizontal lines generally touch to form a long line.Source: §2 · p. 4
  • A word divides into upper, middle, and lower zones: above the Matra, between the Matra and baseline, and below the baseline.Fig.3 depicts the three zones for a Bangla word image.Source: §2 · p. 5, Figure 3 · p. 5
  • Zone segmentation represents complex character combinations with simpler units, reducing the distinct character classes needed for recognition.The paper illustrates this by decomposing a consonant-plus-vowel shape into component units.Source: §2 · p. 5, Figure 4 · p. 5
  • Noise, touching, overlaps, omitted or irregular Matras, and slant or skew can make segmentation and recognition difficult.The authors state that their framework is designed to address these challenges.Source: §2 · p. 6, Figure 5 · p. 6

3.1. Skew and Slant Correction

The approach estimates skew from bottom water-reservoir depth points and corrects slant separately using vertical-projection analysis.

3.1. Skew and Slant Correction

  • Bottom reservoirs shorter than 3×Sw are filtered out, where Sw is estimated as the mode of foreground run lengths.Depth points from the remaining reservoirs are used for skew estimation.Source: §3.1 · p. 7
  • A linear regression through valid-reservoir depth points estimates skew; the image is rotated in the opposite direction by the fitted angle.The depth points are local minima along the reservoir contours.Source: §3.1 · p. 7
  • Slant is corrected separately using vertical-projection peaks and troughs after shear transforms across angles from −45° to +45°.The angle with clear peaks and troughs is selected as the slant angle.Source: §3.1 · p. 7, Figure 6 · p. 8

4.1. Zone Segmentation

Zone segmentation estimates the Matra row and separates upper, middle, and lower components for recognition. The section also describes several sliding-window feature descriptors for middle-zone recognition.

4.1. Zone Segmentation

  • The Matra row is estimated from three candidates: the horizontal-projection peak, reservoir depth points, and a sharp projection decline above the Matra; the authors report efficient detection in most word images.The reservoir-based candidate minimizes squared distances to the depth points.Source: §4.1 · p. 9
  • For upper-zone segmentation, a Matra-centered window of height 4×Sw guides skeleton-point analysis; broken Matra pixels are joined with Bresenham’s algorithm.The curvilinear Matra lies within this window in more than 98% of words in the experiment dataset.Source: §4.1 · p. 10
  • Because projection baselines can fail on irregular words or dense lower modifiers, the method uses skeleton tracing and SVM confidence to separate lower-zone modifiers.A component is treated as a modifier when SVM confidence exceeds 0.6 or the top two scores differ substantially.Source: §4.1 · pp. 11–12
  • Touching middle-zone components are recognized by HMM, using sliding-window PHOG that concatenates gradient-orientation histograms across pyramid levels into a 168-dimensional descriptor.The PHOG implementation uses three resolution levels and eight orientation bins.Source: §4.1 · p. 12

4.2.2. LGH Feature:

  • LGH extracts 128-dimensional sliding-window descriptors by concatenating eight-bin gradient histograms from 16 cells.The image is Gaussian-smoothed before feature extraction.Source: §4.2.2 · pp. 12–13

4.2.3. GABOR Feature:

  • Gabor features use filter responses in four orientations, divide each frame into 12 rows, and concatenate the results into 48 dimensions.The orientations are 0°, 45°, 90°, and 135°.Source: §4.2.3 · p. 13

4.2.4. G-PHOG Feature:

  • The G-PHOG experiment combines Gabor and PHOG features, and the authors report improved Gabor efficiency with this combination.Source: §4.2.4 · p. 13

4.2.5. Marti-Bunke Feature:

  • The profile descriptor uses nine per-column features: three global measures and six local measures of foreground profiles and transitions.The local measures include profile positions, pixel transitions, foreground fraction, and profile gradients.Source: §4.2.5 · p. 13

4.3. Recognition

The recognition pipeline applies HMMs to middle-zone sequences and SVMs to upper- and lower-zone components. It aligns and associates zone results, then selects a lexicon match, with water reservoirs used to refine segmentation.

4.3.1. Middle Zone Recognition using Hidden Markov Model

  • Left-to-right continuous-density HMMs process sliding-window feature sequences, with separate Gaussian-mixture output models for each state.The HMM is trained on middle-zone transcriptions and feature sequences; recognition uses Viterbi decoding.Source: §4.3.1 · p. 14

4.3.2Support Vector Machine (SVM) and Modifier Recognition

  • Upper- and lower-zone components are classified with SVMs using 168-dimensional PHOG features; the experiment selected an RBF kernel and grid-searched its parameters.Components are resized to 150×150 before feature extraction.Source: 4.3.2Support Vector Machine (SVM) and… · p. 15

4.3.3. Combination of Zone-wise Recognition Results

  • Viterbi forced alignment estimates middle-zone character boundaries, which are extended into the upper and lower zones to associate modifiers with characters.The association allows modifiers from adjacent positions, while each character is used only once.Source: §4.3.3 · pp. 15, 16, 18
  • Associated candidate words are compared with the lexicon by Levenshtein distance, and the minimum-distance lexicon word is selected as the final result.The distance accounts for substitutions, insertions, and deletions.Source: §4.3.3 · pp. 16, 18
  • When forced alignment yields improper boundaries, the method shifts segmentation lines toward nearby deepest points of sufficiently tall bottom reservoirs.The paper illustrates this refinement as correcting some Viterbi segmentation errors.Source: §4.3.3 · p. 18, Figure 15 · p. 19

5. Experimental Results and its Analysis

Experiments on Bangla and Devanagari evaluate zone-wise recognition, its component stages, and its behavior under segmentation errors, larger lexicons, and synthetic noise. Full-word recognition with zone segmentation substantially outperformed the within-dataset approach without it.

5. Experimental Results and its Analysis

  • Experiments used 17,091 Bangla and 16,128 Devanagari word images, with 3,856 and 3,589 test samples and lexicons of 1,547 and 1,957 words, respectively.The datasets were collected from 60 Bangla document images and Devanagari documents; words with four characters were most common in both.Source: §5 · p. 19, Table I · p. 19, Figure 16 · p. 20

5.1. Middle zone recognition

  • With 32 mixtures, PHOG reached 92.98% (Bangla) and 94.59% (Devanagari) validation accuracy at top 5; eight states performed best, and accuracy rose with word length.Zone segmentation reduced the character-component classes from 124 to 42 for Bangla and from 116 to 40 for Devanagari.Source: §5.1 · pp. 20, 23, Figure 17 · p. 21, Figure 18 · p. 21

5.2. Upper and Lower Zone Recognition Results

  • Modifier evaluation used 1,723 upper and 1,437 lower Bangla training samples, 1,656 upper and 1,351 lower Devanagari samples, and 500 test modifiers per zone.The paper reports SVM-based modifier classification and presents its results in Table VI.Source: §5.2 · p. 26, Table VI · p. 26, Figure 23 · p. 26

5.3. Full Word Recognition Result Combining Zone-wise Results

  • 83.39% and 92.89% (Bangla), and 84.24% and 94.51% (Devanagari), were the full-word top-1 and top-5 accuracies, respectively.The zone-wise results combine middle-zone recognition with upper- and lower-zone modifiers, followed by lexicon search using Levenshtein distance.Source: §5.3 · p. 26, Figure 24 · p. 26, Table VII · p. 27

5.4. Zone Segmentation Analysis

  • Type-2 segmentation errors had little effect; Type-3 cases could recover through N-best hypotheses and modifiers, while Devanagari segmentation was better with straighter Matra.Reported error sources include omitted Matra affecting headline-row estimates, lower modifiers mistaken for letter parts, and severely cursive overlaps; the proposed method also improves middle-zone segmentation over [21].Source: §5.4 · pp. 27–29, Figure 29 · p. 29
  • 83.39% (Bangla) and 84.24% (Devanagari) full-word accuracy compared with 43.21% and 42.65%, respectively, without zone segmentation.The comparison uses PHOG features for the no-zone approach and parameters tuned on the dataset.Source: §5.4 · p. 28, Figure 27 · p. 29
  • As lexicon size increased from 1K to 5K, recognition performance fell; at 5K, accuracy was 70.47% (Bangla) and 71.30% (Devanagari).The lexicons include words collected from sources such as newspapers and webpages.Source: §5.4 · p. 29, Table X · p. 30

5.7. Experiment with Noisy Images

  • With 20% Gaussian noise, performance dropped approximately 9% in both datasets; preprocessing used Gaussian smoothing and contour joining by Roy et al..The experiment also tested synthetic noise levels of 10% and 30%.Source: §5.7 · p. 30, Figure 31 · p. 30, Figure 32 · p. 31

6. Conclusion

The paper proposes HMM-based zone-wise recognition for Bangla and Devanagari handwritten words, combining middle-zone recognition with upper- and lower-zone modifier classification. Experiments support the approach’s recognition capability and suggest that zone-wise recognition may offer insights for similar scripts.

6. Conclusion

  • The framework segments words into upper, middle, and lower zones, using HMM for the middle zone, SVM for modifiers, and combining zone-wise results into full words.The authors report that zone segmentation reduces the number of possible character-component combinations.Source: §6 · p. 31
  • Experiments on Bangla and Devanagari showed stronger recognition capability with zone-based segmentation, which reduces the number of possible character combinations.Source: §6 · p. 31
  • The authors suggest zone-wise recognition may offer insights for other similar scripts, including Assamese and Gurumukhi.Source: §6 · p. 31
Loading 1708.00227v1…