PaperlayerView exact revision on arXiv

Source-linked AI summary

EMBLEM: Enhancing Multi-script Table Detection through Masking

Dhruv Kudale, Udhay Brahmi, Ganesh Ramakrishnan

arXiv:2609.08330v1cs.LG

TL;DR

Multilingual table detection is limited by script diversity and scarce labeled data. The paper introduces MANDALA and EMBLEM, which uses masking to emphasize script-agnostic layout cues. EMBLEM consistently improves multi-script detection, including a 20.8% absolute F1-score gain over a strong baseline, without degrading Latin-script performance.

  • Problem

    Existing table detection models perform well mainly on English and Chinese documents, while multilingual, multi-script settings lack robust labeled data and generalization.

  • Method

    The paper introduces MANDALA and EMBLEM, which fine-tunes pretrained detectors on masked images that suppress script-specific details while preserving structural layout cues.

  • Results

    20.8% absolute F1-score gain is achieved by EMBLEM-DocL over the strongest DocL-PTN baseline, while EMBLEM consistently outperforms PTN and FTN baselines across benchmarks.

  • Takeaways & Limitations

    EMBLEM provides consistent cross-script gains across YOLO, TATR, and DocL without multilingual training data or degraded Latin-script performance.

  • Takeaways & Limitations

    EMBLEM does not explicitly address table structure recognition or nested tables, and fixed-threshold binarization may limit performance on low-quality inputs.

Abstract

from arXiv · show

Table detection is a core task in document analysis, supporting downstream applications such as information retrieval, document reconstruction, and visual question answering. While existing deep learning models perform well on English and Chinese documents, they struggle with multilingual, multi-script documents due to script diversity and the limited availability of labeled data. To address this challenge, we introduce MANDALA (Multi-script Annotated Documents for Table Detection), a manually curated dataset of 2,323 table-containing pages spanning 18 languages and 15 scripts across diverse domains. We also propose EMBLEM, a masking-based paradigm for Multi-script Table Detection (MTD). EMBLEM generates masked images that conceal script- and font-specific details, enabling models pre-trained on abundant English documents to focus on script-agnostic page layout. Experiments across three table detection architectures show that EMBLEM consistently outperforms strong baselines on MANDALA while remaining competitive on five standard English-dominant benchmarks. Using only English masked images for fine-tuning, with no multi-script training data, EMBLEM achieves an absolute F1-score gain of 20.8% on MANDALA. We release MANDALA along with the accompanying code and models at https://github.com/IITB-LEAP-OCR/EMBLEM.git.

1 Introduction

Table detection struggles to generalize from English- and Chinese-focused data to multilingual, multi-script documents. EMBLEM addresses this gap alongside MANDALA, a broad multi-script benchmark.

  • Multi-script table detection remains difficult because models trained on monolingual data can fail when only the document script changes.Such failures can produce missed or fragmented table predictions.
  • 2,323 pages across 15 scripts and 18 languages define MANDALA, a manually annotated benchmark spanning diverse document domains.
  • EMBLEM is a masking-based fine-tuning paradigm designed to learn script-agnostic structural cues for multi-script table detection.
  • EMBLEM-based models use masked English or Chinese training documents while retaining competitive performance on standard English-dominated benchmarks.

2 Related Work

Prior table detection methods range from handcrafted computer-vision features to deep object detectors and vision-language models, but cross-script generalization remains underexplored. EMBLEM builds on script-independent table structure and masking to target this gap.

  • TD Approaches: Earlier computer-vision and machine-learning methods relied on heuristics or handcrafted features, whereas modern systems largely formulate table detection as object detection.
  • TD Approaches: Deep table detectors include CNN-based, YOLO, SSD, and transformer architectures, while vision-language models face cross-script challenges linked to language-specific pretraining.
  • Towards Script-Agnostic Generalization: Existing data-efficient approaches remain largely English-centric and generally overlook script variation despite the cost of annotating tables across scripts.
  • Towards Script-Agnostic Generalization: Table structure provides potentially script-independent cues through ruled lines, grids, and whitespace, motivating masking-based cross-script table detection.

3 MANDALA Dataset

MANDALA is a manually curated benchmark for realistic multi-script table detection, varying both script and document domain. Its annotations cover diverse table appearances under explicit inclusion rules.

  • Dataset Scope: MANDALA is designed primarily as a test and analysis benchmark rather than a large-scale training corpus, with explicit variation in script and domain.
  • Dataset Scope: MANDALA contains 2,323 document images spanning 18 languages, 15 scripts, and seven document domains.
  • Dataset Scope: The benchmark separates script from language, including multiple languages that share scripts, to support targeted evaluation of script-level generalization.
  • Annotation: All document images were manually annotated under strict guidelines, with tables treated as a single class and independently reviewed before inclusion.
  • Annotation: Annotations include bordered, partially bordered, unbordered, rotated, skewed, and large tables, while nested tables are not annotated separately.
  • Annotation: Captions are included only when entirely inside table boundaries, and partially or completely bordered lists with explicit columns may be annotated as tables.

4 Experiments

EMBLEM fine-tunes table detectors on masked images that suppress script-dependent details while preserving structural layout cues. Its masking procedure uses image binarization, contour detection, and selective pixel-level masking.

  • Masking using EMBLEM: EMBLEM suppresses script-specific visual cues while preserving whitespace, ruled lines, alignment, and table boundaries for structural learning.
  • Masking using EMBLEM: The method fine-tunes detectors pretrained on English documents using masked images, aiming to generalize across scripts without multi-script annotations.
  • Masking using EMBLEM: Unlike OCR-based masking, EMBLEM uses pixel-level computer-vision masking to avoid over-masking regions and distorting layout cues.
  • Masking using EMBLEM: The procedure converts images to grayscale, applies Otsu thresholding, extracts contours, and masks sufficiently small bounding boxes with filled rectangles.

12 end

EMBLEM is fine-tuned on masked document images generated from predominantly English-labeled datasets, with no exposure to multi-script images during fine-tuning. The experiments use YOLO, TATR, and DocL to assess model-agnostic applicability.

  • Data and Models: The masked images are split into 2,000 training and 183 validation samples after masking.
  • Data and Models: EMBLEM fine-tunes YOLO, TATR, and DocLayout models on masked images to test applicability across detection architectures.These architectures were selected for their strong table detection performance on English documents.

5 Results

Across the evaluated benchmarks, EMBLEM consistently outperforms pretrained and fine-tuned baselines, including multimodal and traditional architectures, while retaining competitive English-dominant performance. Ablation results identify fine-tuning on masked images with inference on original images as the strongest configuration.

  • Performance Comparison: EMBLEM consistently outperforms pretrained and unmasked fine-tuned baselines across all reported benchmarks.The comparison uses confidence threshold 0.75 and Non-Maximum Suppression IoU threshold 0.1.
  • Performance Comparison: EMBLEM outperforms multimodal generative models and traditional object detectors on cTDaR and MANDALA while retaining fast, scalable inference.The reported comparison attributes the efficiency advantage to EMBLEM’s object detection backbones rather than autoregressive decoding.
  • Performance Comparison: 20.8% absolute F1-score gain is achieved by EMBLEM-DocL over the strongest DocL-PTN baseline.
  • Performance Comparison: EMBLEM ranks second on IIIT-AR-13K and ties the best result on TNCR, while matching or surpassing strong methods on English benchmarks.EMBLEM-TATR achieves the highest F1-score on ICDAR 2013.
  • Ablation: Fine-tuning on masked images followed by inference on original images is the strongest of the four evaluated fine-tuning–inference configurations.The ablation uses DocL on cTDaR and MANDALA at IoU 0.5.

6 Conclusion

The paper presents EMBLEM as a model-agnostic masking paradigm and MANDALA as a diverse benchmark for cross-script table detection. Together, they support script-agnostic table detection across architectures and provide a basis for broader multi-script document understanding.

  • Conclusion: EMBLEM is a model-agnostic masking-based fine-tuning paradigm that encourages reliance on script-independent structural cues.
  • Conclusion: Fine-tuning on masked images followed by inference on original images is the most effective evaluated configuration.
  • Conclusion: EMBLEM produces cross-script gains across pretraining datasets without requiring multilingual non-Latin training data or degrading Latin-script performance.
  • Conclusion: MANDALA spans 18 languages and 15 scripts, while EMBLEM’s effectiveness is validated across YOLO, TATR, and DocL.

7 Limitations and Future Work

The study is limited to table detection and does not explicitly address table structure recognition or nested tables. Its fixed-threshold binarization strategy may also limit performance on low-quality or visually degraded inputs.

  • Limitations: EMBLEM does not explicitly address table structure recognition or nested tables in complex documents.The paper identifies both as remaining challenges and proposes extending the framework to structure recognition in future work.
  • Limitations: Fixed thresholds and binarization may limit EMBLEM on low-quality or visually degraded inputs.Future work proposes adaptive masking with dynamic thresholds, blurred masks, and richer representations such as grayscale.
Loading 2609.08330v1…