PaperlayerView exact revision on arXiv

Source-linked AI summary

UniLipi: A Unified Multi-Script OCR for Historical Indic Manuscripts

Tathagata Ghosh, Sai Madhusudan Gunda, Simran Singh Sandral, Ravi Kiran Sarvadevabhatla

arXiv:2608.28195v1cs.CV

TL;DR

Handwritten Indic manuscript OCR is difficult to scale because existing systems are often script-specific and historical data are ultra low-resource. UniLipi jointly trains one model across 13 Indic scripts using manuscript-realistic synthetic data and multitask outputs, and the reported results show strong cross-script performance with transfer beyond historical Indic manuscripts.

  • Problem

    Handwritten Indic manuscript OCR remains fragmented and difficult to deploy at scale because existing approaches are typically script-specific while historical data are ultra low-resource.

  • Method

    UniLipi jointly trains a unified OCR model across 13 Indic scripts using realistic synthetic manuscript lines, shared representations, and transcription, script-identification, and glyph-count objectives.

  • Results

    UniLipi achieves 6.9% CER across 13 scripts, outperforming Tesseract, Kraken, VLT, and HTR-VT, while also showing promising transfer beyond historical Indic manuscripts.

  • Takeaways & Limitations

    UniLipi reduces reliance on script-specific OCR pipelines and functions as a foundational pretrained model for contemporary Indic handwriting and several non-Indic scripts.

Abstract

from arXiv · show

Optical character recognition (OCR) for handwritten Indic manuscripts is essential for large-scale digitization and computational access to manuscript heritage. However, existing approaches are typically developed for one script at a time and require substantial script-specific customization. This limits scalability and practical deployment across diverse collections. We present UniLipi, a unified multi-script OCR model for handwritten Indic manuscripts trained jointly across 13 Indic scripts within a single framework. UniLipi directly handles realistic manuscript conditions, including extreme variation in line geometry, large variation in line length, and partial interruptions caused by non-textual manuscript entities such as holes, stains, or pictorial illustrations. To operate effectively under ultra low-resource conditions, the model leverages script-aware synthetic manuscript data generation, substantially reducing reliance on large volumes of real annotated data. Beyond historical manuscripts, we show that UniLipi serves as an effective foundational pretrained model. Specifically, its learned representations enable good OCR performance for contemporary Indic handwriting and extend to several non-Indic scripts, including Tibetan, Italian, Latin, and Chinese scripts. In addition to transcription, UniLipi predicts script identity and per-line native character counts, supporting practical manuscript cataloging workflows.

1 Introduction

Handwritten Indic manuscript OCR remains fragmented and difficult to scale because collections are low-resource, visually diverse, and typically handled with script-specific systems. UniLipi addresses this gap with a jointly trained multi-script model that uses realistic synthetic data and supports transcription, script identification, and glyph counting.

  • Research gap: Indic manuscript OCR is difficult to deploy at scale because data are scarce, annotations are limited, and writing conditions vary substantially across scripts and collections.Challenges include dense ligatures, diacritics, similar glyph inventories, diverse materials, and degradation patterns.
  • Research gap: Script-specific OCR requires separate training, tuning, evaluation, and maintenance pipelines, limiting scalability and cross-collection generalization.The resulting workflows are described as brittle even when manuscripts share linguistic or structural properties.
  • Motivation: Shared language and orthographic structure across Indic scripts motivate treating handwritten recognition as a unified learning problem rather than isolated script-specific tasks.Many manuscripts use different scripts to encode the same underlying language, prominently Sanskrit.
  • UniLipi: UniLipi jointly trains across multiple Indic scripts and collections while handling curved lines, variable lengths, and interruptions from holes, stains, or illustrations.The model combines limited real annotations with script-aware synthetic manuscript lines to learn shared and script-sensitive representations.
  • Broader applicability: UniLipi extends beyond historical Indic manuscripts to contemporary Indic handwriting and several non-Indic scripts as a foundational pretrained model.The introduction reports transfer to non-Indic scripts without enumerating them in the supplied passage.
  • Additional outputs: The system predicts transcription, script identity, and per-line glyph counts, adding cataloging-relevant outputs beyond OCR text.These contributions are stated alongside the unified model and synthetic-data approach.

2 Related Work

Prior handwritten text recognition spans sequence models, synthetic-data methods, and multilingual architectures, but historical Indic manuscripts add low-resource and domain-specific challenges. UniLipi’s related-work context motivates a shared recognition framework that accommodates long, irregular lines and diverse manuscript backgrounds.

  • Historical OCR/HTR: Modern handwritten text recognition uses CNN-RNN, LSTM, Transformer, autoregressive, and CTC-based sequence models.CTC formulations remain competitive for long and irregular text lines, especially in low-resource settings.
  • Historical OCR/HTR: Many HTR evaluations use homogeneous corpora or short word crops, while rectification methods are often tested under limited curvature and cleaner conditions.These constraints motivate models that explicitly address background variability and long-line geometry.
  • Low-resource Indic OCR: Transfer learning, synthetic pretraining, and cross-lingual representation sharing address data scarcity, but modern-corpus pretraining may not cover historical manuscript domain gaps.Historical Indic manuscripts combine degradation, variable backgrounds, contiguous irregular lines, dense ligatures, and diacritics.
  • Synthetic data: Manuscript OCR synthesis includes reference-based generation, text-to-handwriting models, deformation-based augmentation, and newer generative approaches.The supplied passage frames these methods as responses to annotation scarcity.
  • Multi-script recognition: Multilingual recognition has evolved from script-identification routing toward shared architectures that jointly perform script identification and recognition.This trajectory supports unified modeling across scripts within a shared decoding space rather than cascaded routing.
  • UniLipi architecture: UniLipi’s architecture combines CNN visual features, Transformer encoding, CTC transcription, script identification, glyph counting, and joint OCR, script, and count losses.The architecture overview describes task tokens and task-specific heads within one model.

3 Methodology

UniLipi jointly recognizes manuscript text, script identity, and glyph count within a unified multi-task architecture. Its auxiliary objectives provide script-aware and length-aware supervision alongside CTC-based transcription.

  • Model outputs: UniLipi predicts a Roman WX text sequence, a script identity label, and a visually distinct glyph count from each manuscript line image.The glyph count measures graphical units rather than Roman WX characters because Indic conjuncts and diacritics distort character counts.
  • Architecture: The model uses a hybrid CNN-Transformer with a ResNet visual backbone, vertical pooling, linear projection, and task-specific output heads.Input lines are resized to height H = 68 and padded to maximum width Wmax = 3200 before feature extraction.
  • Efficient sequence processing: Windowed self-attention restricts each token’s attention to neighboring tokens, reducing computational complexity for long image-token sequences.The Transformer encoder has 6 layers and 8 heads.
  • Training objective: CTC trains text recognition, cross-entropy trains script identification, and MAE trains glyph-count prediction in the joint objective.The coefficients λscript and λcount are fixed empirically across experiments.
  • Representation learning: Joint training encourages representations that capture textual content, script identity, and structural length information across scripts.The paper links this representation learning strategy to improved transcription accuracy across diverse manuscripts.

4 Real Manuscripts Data

UniLipi’s real-manuscript collection is designed for pan-Indic diversity across materials, regions, and writing systems. Text is converted losslessly into a shared Roman WX target for script-agnostic supervision.

  • Collection coverage: The collection covers 13 scripts from five regional traditions, including North/Central, Western, Eastern, Southern, and Himalayan/Buddhist groups.The scripts include both shirorekha and non-shirorekha systems, alongside rounded, angular, and cursive glyph forms.
  • Collection coverage: The data spans palm-leaf folios, handmade paper, and religious archives drawn from public and private collections.This material diversity supports coverage of varied manuscript formats and institutional contexts.
  • Unified supervision: All manuscript text is mapped to a unified Roman WX representation using a deterministic, lossless conversion.The mapping preserves every orthographic unit while enabling one shared vocabulary across scripts.

5 Synthetic Manuscripts Data

The synthetic-data pipeline uses real manuscript lines as appearance and geometric references while rendering new script-aware text into valid writing regions. Each generated line retains realistic backgrounds and supervision for transcription, script, and glyph count.

  • Motivation: Synthetic lines are generated from reference manuscript images because real annotated lines are extremely limited across all 13 scripts.The references provide appearance and geometric priors for large-scale training.
  • Reference-conditioned rendering: Text is sampled in Roman WX, transliterated into the reference script, and rendered using the reference line’s writing mask, estimated size, ink color, and script-specific fonts.Glyphs are placed sequentially within valid regions, with exceeding glyphs truncated.
  • Training samples: The pipeline derives appearance, geometry, and structural priors from real manuscript references to approximate authentic manuscript conditions.The figure summarizes the sequence from text sampling and rendering through compositing and supervision generation.
  • Background preparation: Text-guided inpainting removes the original writing to create a clean background that preserves manuscript texture and aging patterns.The resulting background is combined with the newly rendered text.
  • Training samples: The final synthetic sample includes a truncated Roman WX transcription target, script identity label, and glyph count.Manuscript-specific augmentations further reduce the distribution gap between synthetic and real manuscripts.

6 Experimental Setup

Experiments pretrain on large synthetic corpora and fine-tune on limited real manuscript data across 13 scripts. Evaluation reports script-wise transcription, monolingual comparisons, glyph-count error, and overall baseline performance.

  • Data and splits: Approximately 2.5M synthetic line images are generated per script, totaling 32.5M samples across 13 scripts.Real manuscript training sets typically contain 500 to 3000 line images per script.
  • Data and splits: UniLipi is fine-tuned on 90% of each real-manuscript dataset and evaluated on the remaining 10%.A held-out synthetic validation set is used for early stopping during pretraining.
  • Evaluation: Table 1 reports CER for OCR, Mono CER for script-specific models, and Count-MAE for glyph-count prediction across geographically grouped scripts.The evaluation covers all 13 scripts and distinguishes joint training from monolingual training.
  • Baselines: Table 2 compares overall CER and reports the lowest- and highest-CER scripts to summarize robustness across UniLipi-13.The comparison includes classical OCR engines and recent HTR models.
  • Baselines: Baselines include LSTM-based and Transformer-based handwritten text-recognition systems trained under the same conditions as UniLipi.Overall scores are weighted averages based on dataset size.

7 Results

UniLipi achieves strong multi-script OCR and auxiliary prediction performance across diverse Indic manuscripts, while transferring to contemporary, non-Indic, and unseen scripts.

  • Per-script performance: UniLipi attains low CER on Newar, Sharada, Kannada, Gurmukhi, and Malayalam, while remaining competitive on Grantha and Telugu.CER values are 5.1, 5.2, 5.5, 5.5, 5.7, 12.4, and 12.3, respectively.
  • Comparison with baselines: 6.9% CER overall, outperforming LSTM and Transformer HTR baselines across 13 Indic scripts.UniLipi scores 6.9% CER versus 32.4% for Tesseract, 28.5% for Kraken, 19.6% for VLT, 14.8% for HTR-VT, 12.9% for HTR-ConvText, and 14.7% for TrOCR.
  • Per-script performance: Modi and Siddham show the largest performance drops, with CER values of 18.6 and 18.1, respectively.These difficulties are associated with complex scripts containing dense ligatures and lower training line counts.
  • Auxiliary predictions: 1.1 Count-MAE for glyph count prediction provides auxiliary structural information and can flag unusual line-length predictions for analysis.Count errors are lower for Newar and Sharada and higher for ligature-dense Modi.
  • Auxiliary predictions: Macro-F1 close to 0.99 for script classification provides reliable script metadata.The reported classification performance indicates consistent script identification in the experiments.
  • Transferability: Fine-tuning transfers UniLipi to IAM, RIMES, LAM, Tibetan, Chinese, and contemporary Indian handwriting benchmarks.Reported CER values include 6.9, 7.2, 8.1, 1.3, 0.4, and 9.5, respectively; increasing pretraining script diversity also lowers CER on unseen scripts.

8 Ablation Study

The ablation study shows that Roman WX decoding provides the strongest combination of transcription and glyph-count performance compared with IAST and script-specific vocabularies.

  • Output Representation: 6.9 CER and 1.1 Count-MAE make Roman WX the best tested output representation.IAST produces 11.0 CER and 1.7 Count-MAE, while script-specific vocabularies produce 14.1 CER and 1.9 Count-MAE.

9 Conclusion

UniLipi combines unified multi-script modeling with manuscript-realistic synthetic pretraining, Roman WX decoding, and multitask supervision. Experiments show competitive accuracy across 13 scripts, improved robustness, and promising transfer beyond Indic scripts.

  • Framework: UniLipi is presented as a unified multi-script OCR framework for Indic scripts.The conclusion identifies it as the first such framework in the paper.
  • Design: Manuscript-realistic synthetic pretraining, Roman WX decoding, and multitask supervision reduce the need for script-specific OCR pipelines.The multitask supervision includes script labels and character counts.
  • Findings: Experiments show competitive per-script accuracy across 13 scripts, improved robustness for diverse manuscript lines, and promising cross-domain transfer beyond Indic scripts.These findings summarize the paper’s reported evaluation outcomes.
Loading 2608.28195v1…