Source-linked AI summary
What Can Low Resource Languages Learn From Each Other?
Achyuth P, Kahaan Shah, Chetan Arora
TL;DR
Low-resource scripts remain underserved by VLMs, especially in OCR, where digitizing and preserving cultural and administrative heritage is critical. The paper proposes PSMC, which merges language-specific experts and jointly refines them across scripts; across 10 Indian scripts, it improves WRR by an average of 2% over individual specialists.
Problem
VLM support remains concentrated in high-resource languages, while low-resource OCR scripts lack the datasets needed for traditional scaling.
Method
PSMC derives language-specific experts from a pretrained base, fuses their task-arithmetic weight offsets into one multilingual backbone, and jointly co-trains the languages.
Results
2% average improvement in Word Recognition Rate (WRR) over individual specialist models was achieved across 10 Indian scripts covering more than 20 languages.
Takeaways & Limitations
Joint training in a merged latent space enables constructive cross-script transfer by pooling shared script patterns while preserving script-specific features.
Takeaways & Limitations
The Gujarati evaluation set contained mislabeled Hindi samples and samples with no visible text, which were omitted from reported results.
Abstract
from arXiv · showhide
Despite the rapid advancement of Vision-Language Models (VLMs), their linguistic reach remains largely confined to high-resource languages, leaving the majority of the world's 7,000+ living languages on the wrong side of a growing digital divide. This disparity is especially pronounced in Optical Character Recognition (OCR), where low-resource scripts lack the massive datasets required for traditional scaling laws. We investigate OCR adaptation in extreme data-scarce regimes (<10K real and <250K synthetic images), demonstrating that conventional fine-tuning strategies often reach a performance ceiling. Our key finding reveals a structural inefficiency in language-specific adaptation: while higher layers of specialized models diverge to capture unique script nuances, the lower layers learn redundant, highly similar features. Motivated by this observation, we propose PSMC (Pre-train, Specialize, Merge, and Co-train), a data-efficient framework that capitalizes on a cross-script "transfer effect". Our approach first derives language-specific experts from a high-resource base model, then employs task arithmetic to fuse these experts into a unified, high-performance multilingual back- bone. Extensive evaluation across 10 Indian scripts (supporting 20+ languages) shows that PSMC achieves a ~2% average improvement in Word Recognition Rate (WRR) over individual specialist models without increasing parameter count. Our results indicate that joint training in the merged latent space facilitates a constructive knowledge transfer that benefits all constituent scripts, providing a scalable pathway for inclusive VLM development. Source code and datasets will be released post publication.
1 Introduction
Low-resource OCR faces severe data constraints and performance ceilings, while specialist models redundantly relearn low-level visual features. PSMC merges language-specific experts and co-trains them to build a unified multilingual OCR model that improves recognition across Indian scripts.
- Motivation: VLM support remains concentrated in high-resource languages, leaving most of the world’s 7,000+ living languages underserved.OCR is especially important for digitizing and preserving low-resource communities’ cultural and administrative heritage.
- Motivation: Fewer than 10,000 real training images per language can make standard fine-tuning overfit, forget general visual features, and hit a performance ceiling.The low-resource setting limits conventional scaling-based adaptation for long-tail scripts.
- Motivation: Specialist models learn highly similar lower-layer visual features but diverge in deeper layers to capture script-specific characters.This hierarchical redundancy motivates sharing early representations while preserving specialized higher-level features.
- Approach: PSMC derives language-specific experts from a common pretrained base, merges their task-vector offsets, and jointly co-trains the resulting multilingual backbone.The four stages are Pre-train, Specialize, Merge, and Co-train.
- Results: Across 10 Indian scripts supporting more than 20 languages, the unified model consistently outperforms individual specialist models.The evaluation is designed to test whether joint refinement in the merged space improves multilingual OCR under data constraints.
- Results: PSMC improves Word Recognition Rate by an average of 2% through a constructive cross-script transfer effect.The reported benefit comes without increasing parameter count, according to the paper context.
2 Related Work
Prior OCR work includes large-scale VLMs, specialized low-resource transfer learning, multilingual Indic systems, and substantial word-level datasets. Existing Indic specialists achieve high-fidelity recognition but are generally optimized in isolation, motivating unified multilingual modeling.
- Multimodal OCR and Transfer Learning: Large industrial VLMs achieve strong page-level understanding through massive data scaling, whereas traditional OCR emphasizes high-fidelity word- and line-level recognition.The related work contrasts holistic page understanding with granular recognition needed for archival digitization.
- Multimodal OCR and Transfer Learning: Prior transfer-learning studies adapted pretrained OCR backbones into specialized low-resource models, while earlier multilingual Indic training did not surpass Bangla and Hindi benchmarks.These results establish both the value and limitations of existing adaptation and multilingual approaches.
- Indic Languages OCR Datasets: IndicSTR-Roadside, Mozhi, and IndicSTR12 provide substantial word-level scene, printed, and synthetic resources across multiple Indic languages.The cited datasets vary in scale and domain, including 17,500 scene-text samples, 80,000 printed-text samples, and large synthetic corpora.
- Indic Language OCR Models: State-of-the-art Indic OCR models are typically language-specific specialists trained for individual scripts rather than unified multilingual representations.These specialists provide high-fidelity recognition for their target scripts but are optimized in isolation.
- Model Merging and Weight Editing: Model merging and weight editing aim to synthesize a unified multitask model from multiple single-task specialists with computational efficiency.This line of work provides the conceptual basis for combining separate OCR experts.
3 Datasets and Backbone Architecture
The study focuses on word-level OCR across 10 Indic scripts supporting more than 20 languages, using real and synthetic data with a Parseq-based multilingual backbone. The constrained setup emphasizes data-efficient recognition under limited resources.
- Datasets: Experiments span 10 Indic scripts supporting 20+ languages, with word-level recognition chosen for its data efficiency.A single digitized page or scene image can yield hundreds of word-level samples.
- Datasets: The constrained training regime generates 240K synthetic samples per language, split equally between scene and printed modalities.The pipeline varies typography, chromaticity, background textures, and kerning, while also using preexisting synthetic data.
- Backbone Architecture: A shared multilingual charset of 987 characters spans all 10 target scripts.This charset supports a unified representation across scripts.
4 Methodology: The PSMC Framework
PSMC adapts a pretrained English OCR model into language-specific experts, merges their task vectors, and jointly refines the resulting multilingual model. The framework exploits shared early-layer visual features while preserving script-specific specialization.
- 4 Methodology: The PSMC Framework: PSMC first derives language-specific experts from a common pretrained anchor, then merges and co-trains them as one multilingual model.The stages are Pre-train, Specialize, Merge, and Co-train.
- 4.1 Transfer Learning from a Pre-trained Anchor: CKA analysis finds high similarity in early encoder layers across specialists, while deeper layers diverge for script-specific character features.This redundancy indicates that separate experts relearn common stroke and curve features.
- 4.3 Task Arithmetic and Joint Refinement: Task Arithmetic combines language-specific weight offsets around the English anchor to create an intermediate merged model.The merged model retains a shared structural foundation while aggregating expertise from multiple language vectors.
- 4.3 Task Arithmetic and Joint Refinement: PSMC evaluation uses WRR and NED scores on the Mozhi and IndicSTR-Roadside datasets.These evaluations compare multilingual model performance across established Indic OCR benchmarks.
- 4.3 Task Arithmetic and Joint Refinement: Joint refinement resolves functional interference among script-specific updates after merging the experts.The merged initialization is reported to converge faster and reach higher final accuracy than a simple pretrained initialization.
5 Experiments and Results
Experiments evaluate OCR metrics, representation similarity, low-resource adaptation strategies, and qualitative errors. PSMC combines task-arithmetic merging with co-training to improve recognition and approach full-data performance.
- Evaluation Metrics: Word Recognition Rate (WRR) measures the percentage of word-level predictions that exactly match the ground truth.Normalized Edit Distance additionally accounts for relative sequence length using Levenshtein distance.
- Representation Similarity Analysis: CKA analysis finds that early layers across specialists remain highly similar while deeper layers diverge for script-specific nuances.This indicates that independently trained experts relearn common visual primitives.
- Low-Resource Regimes: 1.24% average WRR improvement on Mozhi and 0.51% on IndicSTR-Roadside follows from low-resource adaptation compared with individual fine-tuning.These results compare individual fine-tuning with the reported low-resource alternatives.
- Low-Resource Regimes: Task-arithmetic initialization followed by co-training achieves the highest performance among the evaluated adaptation strategies.The joint model is initialized from the merged weight space θ_Merged.
- Results: PSMC often stays within 0.4% of the full-data upper bound and converges faster than naive co-training.Qualitative results also show improved resolution of subtle character ambiguities.
6 Conclusion
The paper addresses data scarcity in low-resource Indic OCR by merging language-specific experts and refining them jointly. Across 10 Indian scripts, PSMC improves WRR over individual fine-tuning while approaching skyline performance.
- Conclusion: Lower-level visual primitives remain largely script-agnostic, whereas higher layers capture script-specific nuances.This hierarchical redundancy motivates merging language-specific experts.
- Conclusion: PSMC uses task arithmetic to unify language-specific experts into one backbone, followed by joint refinement.The framework targets cross-script transfer under extreme data scarcity.
- Conclusion: ∼2% average WRR improvement across 10 Indian scripts brings PSMC near skyline models trained on substantially larger datasets.The conclusion presents this as evidence that cross-script transfer can compensate for extreme data scarcity.