PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Handwritten Character Recognition

Ranjit Raut, Aarav Subedi, Ashim Shrestha

arXiv:2609.10572v1cs.CVeess.IV

TL;DR

Non-Latin handwritten character recognition remains understudied, while dominant methods treat characters as generic images and rely on model scale to learn stroke structure. GraphemeNet introduces a unified architecture whose scaffold and decoder choices encode script geometry and spatial reasoning requirements. Across fourteen benchmarks and eight writing systems, it reports strong generalization, a 99.75% DHCD result with 3.9M parameters, and support for structural-prior efficiency, while Arabic remains below stronger published baselines.

  • Problem

    Non-Latin HCR is understudied, and dominant generic-image methods rely on model scale to learn script-specific stroke structure.

  • Method

    GraphemeNet uses two orthogonal axes: persistent scaffold injection for script geometry and decoder topology for spatial relational reasoning, with shared linear capsule routing.

  • Results

    99.75% on DHCD with 3.9M parameters, alongside strong results across fourteen additional benchmarks and eight writing systems.

  • Takeaways & Limitations

    Explicitly encoding script-structural priors can improve accuracy while reducing parameter requirements across the evaluated writing systems.

  • Takeaways & Limitations

    GraphemeNet-Arabic achieves 98.10%, below published CNN-14 and CNN-SVM results of 99.36% and 99.71%; closing this gap remains future work.

Abstract

from arXiv · show

Non-Latin handwritten character recognition (HCR) remains understudied. Dominant methods consider it as generic image classification, which uses model scale to implicitly learn stroke structure. Structural-prior efficiency---the principle that explicitly encoding script-geometric regularities as architectural inductive biases can be both more accurate and require fewer parameters. We introduce GraphemeNet, a unified multi-script architecture, governed by two orthogonal binary axes. Axis 1 operationalises stroke-level geometric regularity via Persistent Scaffold Injection (PSI): a script-specific asymmetric convolution injects a stroke scaffold as a weighted residual at every encoder stage, continuously anchoring learned features to script geometry---distinct from skip connections, auxiliary losses, or attention reweighting. Axis 2 selects between global average pooling with gated fusion and cross-scale attention with a Stroke Topology Module (STM), depending on whether glyph discrimination requires spatial relational reasoning. A Linear Capsule Routing (LCR) with $O(n)$ routing is shared universally. On fourteen benchmarks across eight writing systems, the architecture generalises with only scaffold and decoder topology varying per script, consistently challenging, outperforming published baselines, and establishing structural-prior efficiency as a broadly applicable principle for multi-script HCR.

1 Introduction

Non-Latin HCR remains structurally inefficient because dominant models learn script geometry implicitly through scale. GraphemeNet addresses this with persistent scaffold injection and decoder choices tied to script geometry and spatial reasoning.

  • Motivation: Existing HCR models reach high accuracy by scaling parameters rather than explicitly representing stroke structure, leaving open whether that burden is necessary.ResNet-85 reaches 99.72% on DHCD with approximately 39M parameters, while MallaNet reaches 99.71% with 17M parameters.
  • Motivation: Writing systems encode discriminative structure in script-specific stroke geometry, including top bars, diacritics, loops, and cursive tails.These regularities are consistent across writers and can therefore serve as geometric priors.
  • Motivation: Prior structural methods inject signals only at isolated stages, so learned texture features can dilute the stroke details needed for character discrimination.The paper contrasts preprocessing, single-stage attention, and stem-only asymmetric convolutions with persistent anchoring.
  • Approach: GraphemeNet uses two orthogonal axes: Persistent Scaffold Injection for consistent stroke geometry and decoder topology for spatial relational reasoning requirements.PSI extracts a script-specific scaffold, downsamples and projects it, then adds it as a weighted residual with α = 0.1 at every encoder stage.
  • Approach: The axis assignments identify distinct script configurations: PSI with GAP for Indic and Semitic scripts, PSI with attention for Japanese, and attention without PSI for Thai and English.The supplied passage states that no script is best served by the no-PSI plus GAP combination.

2 Related Work

Prior HCR work either scales generic visual models or specializes models by script, while adjacent fields show that domain-specific inductive biases can improve efficiency. GraphemeNet combines structural priors with linear capsule routing in a shared multi-script architecture.

  • Multi-script recognition: Multi-script HCR commonly uses independent script-specific models or shared extractors with script-specific heads, rather than one architecture designed for multi-script generalization.Comparative and multitask systems address multiple scripts or script identity but do not establish a unified architecture with structural configuration choices.
  • Generic visual modeling: Published HCR results repeatedly compensate for absent structural priors with larger capacity, including strong but scale-driven results across Devanagari, Bengali, Kuzushiji, and Arabic.Reported examples include ViT at 98.40% on Bengali, 97.16% on Kuzushiji-49, and 99.71% for an Arabic CNN-SVM hybrid.
  • Inductive bias: Adjacent domains support encoding task-relevant structure directly, motivating structural-prior efficiency as a way to match or exceed larger unstructured models with fewer parameters.Examples include equivariant networks, anatomical constraints, and spectral-spatial priors.
  • Capsule routing: GraphemeNet’s Linear Capsule Routing reduces routing complexity from O(n^2) to O(n) while retaining class-discriminative spatial selectivity.LCR uses a single bilinear projection and operates on a multi-scale fused feature vector shared across decoder configurations.
  • Capsule routing: Unlike MallaNet’s branch-specific homogeneous filter capsules, LCR is shared across all GraphemeNet decoder configurations and processes multi-scale fused features.This distinguishes the routing mechanism by both input representation and architectural scope.

3 Datasets

The evaluation covers fourteen handwritten-character benchmarks spanning Indic, Semitic, Japanese, Thai, English, and other writing datasets. The datasets vary substantially in class count, resolution, split size, imbalance, and storage format.

  • Non-English benchmarks: The benchmark suite includes Devanagari DHCD, Bengali CMATERdb, Kannada-MNIST, Arabic AHCD, Persian HODA, and Japanese Kuzushiji-49.These datasets range from 28 to 49 classes and include both character and digit recognition tasks.
  • Evaluation scope: Table 1 defines the evaluation scope as fourteen benchmarks.The supplied caption identifies the table as the complete benchmark inventory.
  • Non-English benchmarks: Thai Burapha-TH contains 68 character classes at 64×64 resolution, with 63,327 training and 13,600 test images.The passage also notes a publicly available Kaggle version has a train/test class-count mismatch.
  • English benchmarks: Seven English benchmarks use 28×28 grayscale inputs, including EMNIST variants, MNIST, and PG-HWLD.The evaluation notes label remapping, augmentation restrictions, class weighting, and dataset-specific split sizes.
  • English benchmarks: EMNIST/ByClass is identified as the most challenging split, with severe imbalance and Macro F1 as its primary metric.Its class weights range from 0.29 to 5.90, and it shares split sizes with ByMerge.

4 Architecture

GraphemeNet is a shared backbone configured by two binary decisions: whether to inject a script-specific scaffold and whether to use global pooling or relational cross-scale decoding. The architecture adapts stem paths and decoder topology while retaining common encoder and classification components.

  • Unified design: GraphemeNet’s unified architecture varies only two binary axes while keeping the dense residual encoder, capsule module, and classification head shared.Axis 1 selects PSI or a richer multi-path stem; Axis 2 selects gated multi-scale pooling or cross-scale attention with STM.
  • Unified design: The script-to-configuration assignment is determined by script-geometric properties rather than tuned independently for each script.Table 2 summarizes these assignments across the four configuration cells.
  • Axis 2: Decoder topology: Cross-scale relational decoding is reported as critical for cursive scripts, distinguishing Option B from multi-scale global average pooling.Option B uses cross-scale attention, STM, and 3-way gated fusion, whereas Option A uses multi-scale GAP with 2-way gated fusion.
  • Axis 1: Stem Design and Persistent Scaffold Injection: Scripts using PSI begin with a dual-path stem combining texture and script-specific asymmetric-convolution scaffold branches.The branches are concatenated, SE-reweighted, and projected through a 1×1 convolution; Table 3 identifies the script-specific scaffold kernel.
  • Axis 1: Stem Design and Persistent Scaffold Injection: PSI retains the scaffold signal and injects it into every encoder stage after spatial downsampling and channel matching, using weight α = 0.1.Sensitivity analysis found weights at least 0.2 degraded accuracy by 0.1–0.3%, while weights at most 0.05 added no accuracy gain.
  • Axis 1: Stem Design and Persistent Scaffold Injection: Thai and English omit PSI and use a triple-path stem with texture, horizontal, and vertical branches, widening encoder channels to 96/192/384.This design addresses Thai loop complexity and English horizontal-vertical stroke structure without a scalar scaffold bypass.

4.3 Axis 2: Decoder Topology

GraphemeNet chooses between pooled gated fusion and topology-aware cross-scale decoding according to whether glyph discrimination depends on spatial stroke relationships. Option B preserves inter-stroke geometry and adds STM features before three-way fusion.

  • Option A — Multi-scale GAP with 2-way gated fusion: Option A globally average-pools each encoder output, concatenates the resulting multi-scale vector, and uses two-way gated fusion.This decoder is preferred when PSI has already anchored spatial information in the encoder features.
  • Option B — Cross-scale attention with STM and 3-way gated fusion: Option B applies cross-scale attention across encoder scales and STM on enc3 to preserve relational and topological stroke information before three-way fusion.The STM captures loop closures, intersection counts, and stroke endpoints.
  • Option B — Cross-scale attention with STM and 3-way gated fusion: 6.38 percentage points separate Option B from Option A on Kuzushiji-49, with 97.81% versus 91.35%, respectively.This comparison supports topology-aware decoding for cursive scripts with spatially complex glyphs.

4.4 Linear Capsule Routing (LCR)

LCR supplies a shared capsule-space class signal alongside the dense head, while gated fusion combines these streams and, for Option B, the STM stream. The four configurations share the backbone, LCR, and classification head while varying scaffold encoding and decoder topology.

  • Linear Capsule Routing (LCR): LCR provides a complementary capsule-space class signal using O(n) agreement scoring instead of quadratic dynamic routing.The shared LCR computes per-class agreement scores from the global feature and learned class filters.
  • Dense head: The dense head projects the multi-scale feature vector into K-dimensional classification logits before fusion.The head uses H = 256 hidden units in all experiments.
  • Gated fusion: Option A uses a learned two-way gate to blend dense-head logits with LCR scores, with gate preference varying across script regularity and handwriting variability.More geometrically regular scripts tend toward LCR-dominant weighting, whereas EMNIST/ByClass tends toward dense-head dominance.
  • Gated fusion: Option B adds STM as a third gated stream, allowing sample-wise weighting of topology, capsule agreement, and direct projection.The three-way gate combines the STM, LCR, and dense-head outputs.
  • Four configurations: Architecture 1 uses PSI with multi-scale GAP and LCR for Indic and Semitic scripts, reaching 99.75% on Devanagari with 3.9 M parameters.Architecture 2 is the no-PSI plus GAP empty cell, and no script achieves its best result under it.

4.7 Side-by-Side Specification Table

The specification tables distinguish variant-specific architectural parameters, PSI sensitivity, decoder comparisons, and controlled ablations while holding common training settings fixed. They document why the selected configuration varies by script and where evidence remains incomplete.

  • Specification scope: Parameters not listed in the variant specification, including optimizer and learning-rate schedule, are identical across variants.Thai alone uses the 96/192/384 channel-depth override among the no-PSI cases described.
  • PSI sensitivity: α = 0.1 is selected for PSI because lower weights provide no accuracy benefit, whereas weights ≥0.2 degrade accuracy by 0.1–0.3%.The operating point balances structural anchoring with learned feature diversity.
  • Decoder comparisons: Option B is compared with Option A across all PSI-active scripts, while Thai and English receive a separate no-PSI comparison.Positive and negative deltas indicate whether cross-scale attention or GAP fusion is sufficient.
  • Ablation study: Removing the scaffold lowers Devanagari accuracy by 0.15%, from 99.75% to 99.60%, with other components unchanged.The controlled variant removes the 1×5 scaffold path and all three scaffold residual injections.
  • Ablation study: The STM variant reaches 99.72% test accuracy with 4,237,405 parameters, below the base model despite added capacity and longer training.The result suggests no complementary STM benefit on DHCD, while the LCR and gated-decoder ablations remain future work.

5 Results

Across fourteen benchmarks, GraphemeNet reports strong recognition results across Indic, Semitic, Japanese, Thai, and EMNIST datasets. The reported script-specific models pair accuracy with compact parameter counts where those measurements are provided.

  • Devanagari: 99.75% accuracy and macro-F1 are achieved on Devanagari, exceeding MallaNet by 0.04% with 4.4× fewer parameters and ResNet-85 by 0.03% with 10× fewer.The model uses 4.4× fewer parameters than MallaNet and 10× fewer than ResNet-85.
  • Bengali: 98.90% accuracy and macro-F1 are achieved on Bengali with 3.88M trainable parameters.This is reported as the smallest model in the GraphemeNet family.
  • Kannada: 98.29% accuracy and macro-F1 are achieved on Kannada with 5.34M parameters.Validation accuracy reached 99.67% by epoch 10 before convergence.
  • Persian: 99.72% accuracy and macro-F1 are achieved on Persian with 5.34M parameters.The reported dataset contains 54,000 training and 26,000 test examples.
  • Japanese: 97.81% test accuracy is achieved on Japanese Kuzushiji-49, using a 3×3 scaffold for diagonal cursive geometry.Random rotation is enabled, while horizontal flipping is disabled because mirrored glyphs are invalid.
  • Thai: 96.93% accuracy and macro-F1 are achieved on Thai with 11.8M trainable parameters.The result is reported on the Burapha-TH dataset.

6 Discussion

GraphemeNet supports structural-prior efficiency across diverse handwritten-character benchmarks by sharing most of its architecture while varying scaffold and decoder topology by script. Results include substantial parameter savings on Devanagari, strong generalisation to independently collected handwriting, and a remaining performance gap on Arabic.

  • Structural-prior efficiency: 99.75% accuracy on Devanagari is achieved with 3.9M parameters, versus 39M for ResNet-85 and 17M for MallaNet.This result supports structural-prior efficiency through comparable accuracy with substantially fewer parameters.
  • Unified multi-script design: One architecture generalises across eight writing systems with only scaffold and multi-scale feature topology varying by script, while LCR and the gated decoder remain shared.Indic scripts use 1×5 or 1×3 scaffolds, Arabic, Persian, and Kuzushiji use 3×3 scaffolds, and Thai and English share the dual-axis stem.
  • Script-specific configuration: Thai removes encoder scaffold injection because its embedded circular loops are handled by a richer triple-path stem and an unconstrained end-to-end encoder.The design avoids a scalar scaffold bypass that would blur loop-body geometry across encoder stages.
  • Cross-dataset generalisation: GraphemeNet surpasses the strongest reported PG-HWLD baseline by 12.58 percentage points, demonstrating better generalisation to independently collected handwriting.The baseline comparison is against VGG-5, which reported 84.27% accuracy.
  • Remaining limitations: Arabic reaches 98.10%, below CNN-14 at 99.36% and a CNN-SVM hybrid at 99.71%, leaving this gap as a stated objective for future work.The paper characterises GraphemeNet-Arabic as competitive single-model performance within the unified architecture framework despite the lower score.
  • Evaluation considerations: On unbalanced EMNIST/ByClass, Macro F1 is 76.20% versus 84.45% accuracy, showing that accuracy can overstate recognition capability under class imbalance.EMNIST/Balanced reaches 90.23%, while ByClass contains 62 case-sensitive classes and class weights ranging from 0.29 to 5.90.

7 Conclusion

GraphemeNet combines explicit script-structural priors with a compact architecture, achieving strong multi-script recognition while reducing parameter requirements. Results and ablations support structural-prior efficiency across eight writing systems, while broader script coverage remains a future direction.

  • 7 Conclusion: Across fourteen additional benchmarks, GraphemeNet generalises across scripts, reaching 99.74% on EMNIST/Digits, 99.72% on Persian, and 84.45% on EMNIST/ByClass.Reported results span fourteen additional benchmarks across multiple writing systems.
  • 7 Conclusion: Removing Persistent Scaffold Injection lowers DHCD accuracy by 0.15 percentage points while reducing parameters by only 18K, indicating an informational rather than parametric benefit.The augmented STM variant reaches 99.72%, so added decoder complexity does not improve the base result on DHCD.
  • 7 Conclusion: Explicit architectural encoding of script-structural priors simultaneously improves accuracy and reduces parameter requirements across all eight evaluated writing systems.The Thai variant further uses a richer multi-path stem when loop geometry is too complex for a scalar scaffold bypass.
  • 7 Conclusion: Future work should extend evaluation to additional Indic, Latin, and other writing systems.The conclusion explicitly calls for broader script coverage.

A Dataset Sample Images

The dataset figures present handwritten samples from eight writing systems and several Latin benchmark variants. Together, they illustrate differing class counts, image sizes, script geometries, and writer or style variation.

  • A Dataset Sample Images: The samples span Devanagari, Bengali, Kannada, Arabic, Persian, Japanese Kuzushiji, Thai, PG-HWLD, EMNIST, and MNIST datasets.The figures cover multiple scripts plus independently collected and standard benchmark distributions.
  • A Dataset Sample Images: The figures include intra-class writer variation in DHCD and a case-sensitive, heavily unbalanced EMNIST ByClass split with class weights from 0.29 to 5.90.The captions identify writer variation for Devanagari and imbalance for EMNIST ByClass.
  • A Dataset Sample Images: The samples expose script-specific geometry, including shirorekha bars, rounded loops, diacritics, flowing harai tails, and embedded Thai loops.These structural properties are described in the Devanagari, Kannada, Arabic, Kuzushiji, and Thai captions.
  • A Dataset Sample Images: PG-HWLD is independently collected and distinct from the EMNIST distribution, while MNIST contains ten digit classes.Both dataset distinctions are stated in the figure captions.
Loading 2609.10572v1…