Source-linked AI summary
Towards a Joint Khmer Text Recognition and Word Segmentation
Marry Kong, Rina Buoy, Sovisal Chenda, Nguonly Taing, Masakazu Iwamura, Koichi Kise
TL;DR
Khmer document processing needs word segmentation after recognition because the script has no visible word delimiters, making sequential pipelines error-prone and slow. This paper proposes KTRWS, a single CTC-based model that jointly recognizes text and outputs optional word boundaries. Across document, scene, and handwritten benchmarks, it recognizes Khmer characters and locates word boundaries, removing the separate segmentation step; scene and handwritten performance still has room to improve.
Problem
Khmer lacks visible word delimiters, so recognition-then-segmentation pipelines require separate sequential models that are error-prone and latency-intensive for large-scale processing.
Method
KTRWS jointly performs Khmer text recognition and word segmentation in one model using a CTC decoder and optional boundary output controlled by b.
Results
Across document, scene, and handwritten benchmarks, KTRWS recognizes Khmer characters and locates word boundaries, eliminating the extra segmentation step; its b = 0 variant achieved CERs of 1.75% on KHOB and 2.77% on KhmerST.
Takeaways & Limitations
Joint parallel decoding removes separate word segmentation and lowers combined recognition–segmentation latency, while providing visual grounding of predicted words.
Takeaways & Limitations
Training relies on a synthetic word-boundary dataset, and performance has room to improve for scene and handwritten modalities.
Abstract
from arXiv · showhide
Text recognition, or extracting electronic text from document images, has been indispensable for knowledge retrieval tasks, such as retrieval-augmented generation (RAG). For Khmer, extracted text is subject to an extra word segmentation step, as Khmer does not use any visible word delimiters to denote word boundaries. Thus, a recognition-then-segmentation pipeline for Khmer requires two separate sequential models; this is not only error-prone but also adds significant latency for large-scale document processing. This paper proposes a novel joint Khmer text recognition and word segmentation framework in a unified model. The proposed model, using a connectionist-temporal-classification (CTC) decoder for fast, parallel decoding, can be instructed to recognize Khmer text with ($b=1$) and without ($b=0$) word segmentation. Experimental results on different benchmark datasets of different document modalities (document, scene, and handwritten images) show that the proposed model can not only recognize characters in document images but also locate word boundaries, removing the need for an extra word segmentation step in a conventional sequential pipeline.
1 Introduction
Khmer document understanding requires recognition and word segmentation because Khmer lacks visible word delimiters. The paper addresses the error and latency of sequential processing by unifying both tasks in one controllable model.
- Khmer text recognition extracts electronic text from document images for knowledge retrieval tasks such as RAG.
- Because Khmer lacks visible word delimiters, extracted text often requires a separate word segmentation step for downstream search and spell-checking.
- Sequentially applying separate recognition and segmentation models is error-prone and adds significant latency for large document batches.
- KTRWS performs Khmer text recognition and word segmentation jointly in a single model using a parallel CTC decoder.
- The model can be instructed to output Khmer text with word segmentation (b = 1) or without it (b = 0).
2 Related Work
Khmer recognition and segmentation have developed through separate lines of work shaped by a complex script, limited datasets, and the absence of visible word delimiters. The paper motivates a unified approach for downstream applications such as knowledge retrieval.
- Khmer Text Recognition: Khmer script combines multi-layered character stacking, no visible word delimiters, and a large inventory including consonants, vowels, and diacritics.
- Khmer Text Recognition: Khmer text recognition remains relatively under-explored because public training and benchmark datasets are limited.
- Khmer Text Recognition: Recent Khmer recognition methods use Transformers and Khmer character clusters, reducing character error rates across scene, document, and handwritten benchmarks.
- Khmer Word Segmentation: Khmer word segmentation supports downstream NLP tasks, but the field lacks a standard word definition and standard corpus.
- Khmer Word Segmentation: Prior segmentation approaches range from dictionary-based BiMM and CRF models to neural BiLSTM and joint segmentation–POS tagging.
- Unifying recognition and segmentation targets their complementary roles in applications where document images are recognized and words are segmented for retrieval.
3 The Proposed Method
The KTRWS framework combines a boundary-conditioned projector, visual encoder, modality-aware feature selector, and CTC text decoder to recognize Khmer text with or without word boundaries. It uses height-averaged visual-temporal features and character-cluster tokenization for parallel decoding.
- Modality-Aware Feature Selector: The modality-aware feature selector combines boundary projections with visual-temporal features and adapts them across scene, handwritten, and document modalities.It uses GPOOL, ROUTER, and modality-specific adapters before producing the decoder input feature map.
- Word Boundary Projector: KTRWS uses a binary boundary flag to condition whether the model outputs word boundaries during inference.The projector embeds b and produces projection vectors that are fused with encoder features.
- Visual Encoder: The visual encoder applies a CNN, Transformer encoder, and height averaging to convert RGB images into one-dimensional visual-temporal features.The feature dimension is d = 512.
- Text Decoder: The model uses Khmer character-cluster tokenization rather than character-level tokenization for CTC output.The resulting one-dimensional feature map U_1D is the decoder input.
- Text Decoder: The CTC decoder independently predicts tokens over an augmented vocabulary containing character clusters, a blank symbol, and a word boundary marker.Greedy decoding collapses repeated tokens and blanks, while the word boundary marker is invisible when rendered.
4 Datasets & Experimental Setup
The experiments combine public Khmer datasets spanning document, scene, and handwritten modalities with a newly generated dataset containing word-boundary annotations. Training proceeds from general document representation learning to modality adaptation, followed by evaluation across modality-specific benchmarks.
- Datasets: The dataset collection covers real and synthetic Khmer document, scene, and handwritten text images for training, adaptation, and evaluation.Table 1 uses D, S, and H to denote document, scene, and handwritten modalities, respectively.
- Datasets: The authors synthetically generate 60,000 text-line images with word-boundary annotations because existing datasets lack such annotations.A Khmer Transformer-based segmenter supplies invisible boundary markers before text lines are rendered as images.
- Experimental Setup & Training Strategy: The visual encoder uses six ResNet blocks followed by a three-layer Transformer with eight attention heads and embedding dimension 512.The configuration also uses dropout 0.1 and a feed-forward dimension of 2048.
- Datasets: The evaluation group includes KhmerST, KHOB, the KH evaluation set, GKST, and KHT across scene, document, and handwritten modalities.The datasets include both real and synthetic sources used for modality-specific evaluation.
- Experimental Setup & Training Strategy: Training has a general phase on large-scale document datasets followed by a modality-adapting phase.The general phase lasts five epochs with batch size 32 and uses a cyclic learning rate from 10^-5 to 10^-4.
5 Results & Discussion
The KTRWS model jointly recognizes Khmer text and word boundaries, achieving competitive recognition while reducing combined recognition-segmentation latency. Results also show that segmentation is more difficult for scene and handwritten images, whereas projected boundaries provide visual grounding with some downsampling-related positional error.
- Recognition Performance: KTRWS achieves competitive character recognition against state-of-the-art methods, with KTRWS (b = 0) improving CER over UKTR on four of five datasets.KTRWS (b = 0) reports CERs of 1.75%, 2.77%, 5.78%, 4.51%, and 8.91% on KHOB, KhmerST, KH, GKST, and KHT, respectively, versus UKTR’s 2.46%, 3.02%, 5.89%, 4.41%, and 9.52%.
- Latency: KTRWS (b = 1) achieves significantly lower combined recognition and segmentation latency than sequential recognition-then-segmentation approaches on KHOB.The reduction is attributed to parallel joint decoding, which removes the separate word-segmentation step.
- Segmentation Accuracy: Word segmentation accuracy declines from document to scene and handwritten modalities because shorter or non-textual inputs provide less contextual information.KHOB contains long text lines with complete sentences, while the other datasets mainly contain single words, short phrases, or numbers.
- Qualitative Assessment: Recognition errors are infrequent in qualitative examples, but missing word boundaries occur more often in scene and handwritten images.The main exception is a historically degraded image, which shows a recognition error as well.
- Visual Grounding: The model can visually ground predicted words by mapping CTC-predicted word-boundary tokens to positions on the input image.Projected boundary locations are reasonably accurate except for highly curved text, with slight offsets caused by fourfold width downsampling.
- Alternative Projector Design: The default word-boundary projector outperforms simple gated feature fusion on character recognition and word segmentation accuracy across evaluation datasets.The paper attributes this advantage to fusing the boundary instruction through feature-wise linear modulation.
6 Limitations & Future Work
The study identifies limitations in data coverage and word-boundary supervision. Recognition and segmentation perform strongly on document images but leave room for improvement in scene and handwritten modalities.
- Data limitations and future work: Recognition and segmentation performance is high for document images but has room for improvement on scene and handwritten images.Future work will construct a diverse real dataset with word boundary annotations.
- Data limitations and future work: The training relies on a new synthetic dataset with word boundary annotations.The authors identify constructing a diverse real dataset with such annotations as a future direction.
- Data limitations and future work: Word boundaries are handled implicitly by minimizing the LCTC loss rather than through explicit word-boundary supervision.Future work will incorporate explicit word-boundary supervision.
7 Conclusion
The paper presents a single-model framework for joint Khmer text recognition and word segmentation. Using fast parallel CTC decoding, it supports recognition with or without segmentation and locates word boundaries across document modalities.
- Conclusion: KTRWS jointly recognizes Khmer text and segments words within a single model using a CTC decoder for fast, parallel decoding.The model can be instructed to recognize text with (b = 1) or without (b = 0) word segmentation.
- Conclusion: Experiments on document, scene, and handwritten images show that KTRWS recognizes characters and locates word boundaries.The framework removes the need for an extra word segmentation step in a conventional sequential pipeline.