PaperlayerView exact revision on arXiv

Source-linked AI summary

Developing an OCR model for Extracting Information from Invoices with Korean Language

Xiem HoangVan, Phu TranQuang, Minh DinhBao, Tien VuHuu

arXiv:2609.35796v1cs.CL

TL;DR

Manual invoice extraction is time-consuming at scale, and existing OCR methods were not mainly designed for Korean-language invoices. The paper combines image preprocessing with deep-learning OCR and reports about 87% F1-Score with negligible processing time.

  • Problem

    Manual invoice extraction is inefficient at high transaction volumes, while existing OCR methods were not mainly designed for Korean-language invoices.

    Source: §I · p. 1

  • Method

    The system preprocesses Korean invoice images, detects and recognizes text, and extracts purchase time, purchased items, and total invoice amount.

    Source: §III · p. 2

  • Results

    The proposed OCR model achieved about 87% in F1-Score while requiring negligible processing time.

    Source: §V · p. 6

  • Takeaways & Limitations

    The authors conclude that the system is suitable for practical use by businesses or individuals.

    Source: §V · p. 6

    Background removal is not always successful and alignment may be suboptimal, while occlusion and poor lighting challenge invoice object detection.

    Source: A. Test methodology · p. 4

Abstract

from arXiv · show

Invoices are commercial documents that contain various pieces of information, including the purchased items, time, and total money. Making the extraction of important information crucial. The stored information serves different purposes. Korean language is the native language of about 80 million people, playing an important role in not only South and North Korea but also in many other countries such as Vietnam, Philippine where a large number of Korean companies are located. In this context, to automatically extract proper information from the invoices with Korean language, we propose an efficient Optical Character Recognition (OCR) model in which a deep learning model is combined with some image preprocessing techniques. The proposed OCR model is assessed in a rich set of collected invoices showing that 87% F1-score can be achieved with negligible time processing.

I. INTRODUCTION

Invoice extraction matters for commercial transactions, but manual transcription becomes inefficient and time-consuming at scale. The paper proposes Korean-invoice OCR combining deep learning and image processing, including text detection, Korean-character recognition, and key-information extraction.

I. INTRODUCTION

  • Manual invoice transcription is inefficient and time-consuming when transaction volumes are large.Invoices record purchased products, transaction time, and total expenditure.Source: §I · p. 1
  • Existing OCR methods for Korean invoices were not mainly designed for that language and task, motivating combined deep learning and image processing.The paper proposes a text detector, an SVTR architecture for Korean-character features, and a complete invoice-information extraction system.Source: §I · p. 1

A. Text Detection

  • CRAFT detects curved, long, and deformed text using region scores and relationship scores to group characters into text regions.Its VGG16-based network requires significant processing and storage resources, making operation costly and challenging.Source: A. Text Detection · pp. 1–2

B. Text Recognization

The section reviews sequence-based character recognition and two information-extraction alternatives. It describes their mechanisms alongside limitations involving sequential computation, document layout, and positional features.

B. Text Recognization

  • CRNN extracts an image feature sequence with a CNN and uses an RNN or LSTM to predict characters frame by frame.The review contrasts this with conventional methods that locate and recognize characters individually and cannot exploit character-sequence semantics.Source: B. Text Recognization · p. 2
  • CRNN-CTC addresses prediction errors such as word repetition, but CNN-RNN models can train slowly and perform worse on long sentences.Their sequential computations lengthen training, and their dependence on long-distance information is a stated drawback.Source: B. Text Recognization · p. 2

C. Key Information Extraction

  • Template-based extraction applies predefined rules and keyword matching to stable document formats, but requires document-specific rules and substantial manual effort.Source: C. Key Information Extraction · p. 2
  • NLP-based classification or NER can adapt to new data, but text-layout dependence limits tabular use and it cannot exploit positional features of text cells.Source: C. Key Information Extraction · p. 2

D. Korean language

The paper situates its task in the Korean writing system and outlines a pipeline that preprocesses invoice images before detecting and recognizing text and extracting key fields.

D. Korean language

  • Modern Korean is written in Hangul in South Korea and Chosŏn'gŭl in North Korea; the script has 40 characters, including 21 vowels and 19 consonants.Source: D. Korean language · p. 2

III. PROPOSED METHOD

  • The pipeline first removes invoice-image backgrounds and adjusts perspective to address noise, poor image quality, and unfavorable camera angles.Source: §III · p. 2
  • After preprocessing, the system detects and recognizes text, then extracts purchase time, purchased items, and total invoice amount.Source: §III · p. 2

A. Dataset creation and overal work flow

The workflow builds a Korean-invoice dataset from collected images, filters and preprocesses them, and refines automatically generated labels manually before model training.

A. Dataset creation and overal work flow

1. Preprocessing

The preprocessing stage crops invoices and adjusts their perspective to a frontal view. Text detection then treats text as segmentation and uses a learned dynamic threshold.

1. Preprocessing

  • Canny edge detection locates invoice contours, whose corner coordinates are used to crop the invoice and adjust its perspective to a frontal view.The authors describe this conventional image-processing approach as versatile across similar scenarios, though not necessarily as accurate as task-specific machine-learning models.Source: §1 · p. 3, Figure 4 · p. 3

2. Text detection

  • The proposed Segmentation-based method treats text detection as object segmentation using the Real-time Scene Text Detection with Differentiable Binarization model.The authors also fine-tune a pre-trained module on their invoice dataset.Source: §2 · p. 3
  • DB learns a dynamic threshold during training instead of relying on a fixed threshold, which requires time-consuming tuning.The threshold map is derived from the segmentation model's feature map and indicates whether pixels belong to the text object's surrounding edge, supporting longer-sentence learning and more precise coverage.Source: §2 · p. 3

3. Text Recognition

Text recognition uses SVTR to map detected text regions to recognized text. Its multi-scale blocks mix character-level and broader visual information, with a Combining Block at the network's end.

3. Text Recognition

  • SVTR takes text-region positions from detection as input and generates recognized text, beginning with Patch Embedding, which divides each input image into a sequence.The authors describe SVTR as effective at capturing semantic relationships between characters while optimizing training time and accuracy.Source: §3 · pp. 3–4
  • Three layers of Mix Blocks combine global mixing of text and non-text elements with local mixing of strokes within a character to capture features at multiple scales.The layers are designed to prevent information redundancy.Source: §3 · p. 4
  • Merge Blocks perform multi-scale feature extraction and reduce redundancy, while a final Combining Block addresses potential feature degradation from convolutions on very small feature matrices.Source: §3 · p. 4

4. Key Information Extraction

Key information extraction uses PICK to combine invoice text, detected regions, and image information. Its graph-learning, encoder, and decoder modules represent relationships in documents with diverse structures.

4. Key Information Extraction

  • PICK takes invoice images, detected text regions, and recognized text to learn both structural features and textual content.Source: §4 · p. 4
  • PICK combines graph learning, an encoder, and a decoder to extract features from text and images and represent their relationships.The graph-learning module adjusts the model to capture relationships between extracted regions.Source: §4 · p. 4
  • The paper states that PICK achieves superior performance compared with other extraction models by encompassing features from a wide range of information.Source: §4 · p. 4
  • The authors describe PICK as suited to extracting key information from documents with complex structures and diverse layouts.They attribute this capability to representing relationships and features from both text and images.Source: §4 · p. 4

IV. PERFORNAMCE EVALUATION

The evaluation reports that the fine-tuned text detector surpasses PPOCR, while accuracy declines at higher IoU thresholds. Processed images still show background-removal and alignment issues, alongside reported speed–accuracy balance.

A. Test methodology

  • Visual inspection of processed invoices shows that background removal can remain incomplete and alignment can remain suboptimal.The authors use visual observation to assess processed-image credibility and identify possible errors.Source: A. Test methodology · p. 4, Figure 6 · p. 4
  • The authors characterize the system's overall performance as balancing speed and accuracy while providing reliable results in implementation.Source: A. Test methodology · p. 5

B. Text Detection assessment

  • The fine-tuned model surpasses PPOCR; raising IoU to 0.75 brings a 14% accuracy decrease, while IoU 0.8 brings an approximate 10% decrease.Table 1 reports IoU prediction measurements for the proposed model with PPOCR pre-training.Source: Table 1 · p. 5, B. Text Detection assessment · p. 5
  • The authors report an average accuracy increase of 6% over initial pre-training, while citing limited data, equipment, and time as constraints on training duration.They also identify accurate localization of small invoice text as a challenge.Source: B. Text Detection assessment · p. 5
  • Text-region detection is reported as stable in most images, including blurry or low-light cases, but exceptions include distorted, skewed, or poorly delineated images.Source: B. Text Detection assessment · p. 5

C. Text Recognization assessment

The text-recognition assessment uses a pre-trained model on Korean text images and reports character accuracy, while noting limitations on low-quality or distorted inputs.

C. Text Recognization assessment

D. Key Information Extraction assessment

The information-extraction assessment reports an overall F1-Score of around 84%, with performance varying by field. It describes the training inputs and links uneven scores to data representation and text-recognition accuracy.

D. Key Information Extraction assessment

E. Overall system

The system combines invoice-image preprocessing, text detection and recognition, and information extraction. It reports an average extraction time of 5–10 seconds and an 87% F1-Score compared with a relevant PPOCR method.

E. Overall system

  • Extraction averages 5–10 seconds per invoice; model initialization may take notable time, with the extraction model alone occupying 1.1GB.The specific extraction duration varies with invoice length and complexity.Source: E. Overall system · p. 6

V. CONCLUSION

  • 87% F1-Score was reported compared with a relevant PPOCR method, with negligible processing time.Source: §V · p. 6
  • The system preprocesses Korean invoice images with perspective transformation before text detection, recognition, and information extraction.Source: §V · p. 6
  • The authors describe the system as suitable for practical applications by businesses and individuals.Source: §V · p. 6
Loading 2609.35796v1…