PaperlayerView exact revision on arXiv

Source-linked AI summary

Tables Decoded: DELTA for Structure, TARQA for Understanding

Jahanvi Rajput, Dhruv Kudale, Saikiran Kasturi, Utkarsh Verma, Ganesh Ramakrishnan

arXiv:2609.17458v1cs.CVcs.LG

TL;DR

Table understanding needs representations that preserve complex cell relationships while remaining practical for LLM processing, and image-based VLM approaches can be inflexible and weaker on non-Latin scripts. The paper proposes DELTA for reconstruction and TARQA for OTSL-based question answering, reporting results across English and Hindi benchmarks.

  • Problem

    Table understanding needs representations that preserve cell relationships without HTML and LaTeX’s verbosity, while VLM pipelines can be inflexible and show a performance gap on non-Latin scripts.

    Source: §1 · p. 1

  • Method

    DELTA separates physical and logical structure recognition from OCR and produces compact OTSL tables; TARQA is an LLM fine-tuned to answer questions over OTSL sequences.

    Source: §1 · p. 2

  • Results

    DELTA reports high TEDS-S and TARQA strong WTQ and FinTabNetQA results; OTSL inputs exceed the strongest HTML baseline by 9.3 p.p. in ANLS, and the pipeline performs strongly on TORQUE.

    Source: §5 · p. 7, §6 · p. 8

  • Takeaways & Limitations

    The framework provides a flexible, extensible approach to high-fidelity multilingual table reconstruction and TabVQA within the evaluated scope.

    Source: §6 · p. 8

    Experiments cover English and Hindi benchmarks, and OCR errors can lower TEDS scores and affect downstream TabVQA performance.

    Source: §7 · p. 8

Abstract

from arXiv · show

Table understanding is a core task in document intelligence, encompassing two key subtasks: table reconstruction and table visual question answering (TabVQA). While recent approaches predominantly rely on vision- language models (VLMs) operating on table images, we propose a more scalable and effective alternative based on structured textual representations. These representations are easier to process, align more naturally with LLMs, and eliminate the need for language-specific visual encoders, making them particularly suitable for multilingual documents. We present DELTA, which separates physical structure recognition, logical structure recognition, and OCR to extract both layout and content accurately. DELTA outputs tables in Optimised Table Structure Language (OTSL), a compact and unified format that encodes cell arrangements and textual content. On table structure recognition (TSR), DELTA achieves TEDS- Structure scores comparable with state-of-the-art methods across FinTabNet, PubTabNet, and PubTables-1M. We further establish its robustness on non-English tables through our curated Hindi benchmark, TORQUE. Building on this, we introduce TARQA, an LLM fine-tuned on OTSL sequences. Our approach yields gains of 9.3 p.p. on WTQ (TabQA) and 9.2 p.p. on FinTabNetQA (TabVQA), respectively. On TORQUE, our method ranks second among all VLMs and DELTA + LLM variants. We release our code, models, and benchmark at: https://github.com/Tihiitborg/Tables-Decoded

1. Introduction

Table understanding must preserve complex cell relationships while supporting efficient language-model processing across languages. The paper proposes a doubly decoupled reconstruction pipeline and OTSL-based question answering, alongside a Hindi benchmark.

1. Introduction

  • Plain text loses table structure, while HTML and LaTeX can be verbose; Markdown cannot faithfully represent complex structures such as merged cells.These trade-offs motivate a compact representation that preserves both structure and content for LLM processing.Source: §1 · pp. 1–2
  • DELTA separates physical and logical table structure recognition from OCR, then represents structure and content compactly in OTSL.The paper also introduces a lossless HTML-to-OTSL conversion and TARQA, an LLM fine-tuned on OTSL for table question answering.Source: §1 · p. 2
  • The paper introduces TORQUE, a Hindi benchmark for evaluating table reconstruction and TabVQA, and reports effective performance on both tasks.The authors present the benchmark as evidence of the framework’s multilingual capability.Source: §1 · p. 2

2. Related Literature

  • Table understanding approaches include image-processing VLMs and LLM pipelines that serialize tables, with the latter relying on the quality and format of that representation.The paper builds on decoupled pipelines that use TSR and OCR before applying an LLM to textual questions.Source: §2 · p. 3

3. Our Methodology

The methodology combines DELTA, which reconstructs tables from images into OTSL, with TARQA, which answers questions using those sequences. DELTA separates structure recognition from OCR and physical from logical structure recognition.

3.1. DELTA for Structure

  • DELTA processes table images through structure recognition and OCR to produce HTML, then converts the result losslessly into OTSL.The methodology has two components: DELTA generates OTSL, and TARQA performs TabVQA on the extracted sequences.Source: §3.1 · p. 3, §3 · p. 3

3.1.1. TSR for Cell Demarcation

  • SPRINT predicts logical cell arrangements, while TATR identifies physical rows and columns; their outputs combine into HTML with cell bounding boxes and span attributes.The resulting representation captures features such as merged cells, row spans, and column spans.Source: §3.1.1 · p. 3

3.1.2. OCR for Content Extraction

  • EasyOCR extracts text from cell crops defined by TSR bounding boxes and inserts it into the HTML, keeping OCR modular and extendable to multilingual tables.The authors choose EasyOCR for its TSR integration, GPU compatibility, and processing speed.Source: §3.1.2 · p. 3

3.1.3. HTML to OTSL conversion

  • The HTML-to-OTSL conversion preserves table structure and content while reducing the input length compared with verbose HTML.The algorithm accounts for row and column spans and uses distinct tokens for filled, empty, and merged cells.Source: §3.1.3 · p. 4, §3.2 · p. 5

3.2. TARQA for Question Answering

  • TARQA reframes table question answering as text-to-text reasoning over OTSL, avoiding the need for table images as downstream reasoning inputs.It can pair with DELTA’s OTSL output to form a TabVQA pipeline.Source: §3.2 · p. 4

4. Experiments

The experiments evaluate table reconstruction and question answering across English datasets and the Hindi TORQUE benchmark. They also compare table representations for TARQA fine-tuning.

4. Experiments

  • The experiments introduce English and non-English table datasets and compare OTSL with other representations for TabQA and TabVQA.DELTA is evaluated without training because it combines off-the-shelf blocks.Source: §4 · p. 5, §4.1 · p. 5

4.1.1. English Datasets

  • The evaluation uses FinTabNet, PubTables, and PubTabNet for reconstruction, WTQ for TabQA fine-tuning, and FinTabNetQA for the integrated TabVQA pipeline.Reconstruction results use test splits for FinTabNet and PubTables and the PubTabNet validation split for comparison with prior work.Source: §4.1.1 · p. 5

4.1.2. TORQUEDataset

  • TORQUE evaluates Hindi table reconstruction and TabVQA using 210 tables and 422 manually verified question-answer pairs.The tables include 109 scanned and 101 digital-born examples, with simple and complex structures.Source: §4.1.2 · p. 5

4.2. Generating Table Representations

  • For WTQ, HTML ground truth is converted to OTSL with the proposed algorithm or flattened row-wise to plain text before LLM fine-tuning.The selected representation is used to fine-tune the corresponding model for TabQA.Source: §4.2 · p. 5

4.3. Finetuning TARQA Variants

  • TARQA fine-tunes Meta-LLaMA-3-8B-Instruct to answer questions from OTSL tables using a structured prompt with a table, question, and ground-truth answer.Inputs are tokenized with a 4096-token maximum length, and inference uses greedy decoding.Source: §4.3 · pp. 5–6

4.4. Ablation Study for OTSL Format

  • Around 14 p.p. separates TARQA-OTSL from TARQA-HTML on both ANLS and Exact Match (EM) metrics.The WTQ test-set comparison evaluates fine-tuning on different table representations.Source: §4.4 · p. 6, Table 3 · p. 6

5. Results and Discussions

DELTA shows strong structural alignment across table-reconstruction benchmarks, while OCR limits its overall content fidelity. OTSL-based TARQA improves WTQ TabQA results, and the integrated system is evaluated on FinTabNetQA and the Hindi TORQUE benchmark.

On Table Reconstruction

  • DELTA achieves consistently strong TEDS-S scores across reconstruction benchmarks, while its TEDS results trail methods with PDF-parsed text fidelity and MTL-TabNet.The authors identify OCR as the main bottleneck, while describing the construction itself as reliable.Source: §5 · pp. 6–7, Table 4 · p. 7

On TabQA and TabVQA

  • 9.3 p.p. is the reported ANLS gain of OTSL-based inputs over the strongest HTML baseline on WTQ.The comparison evaluates TARQA-OTSL and TARQA-HTML using DELTA’s corresponding reconstructed outputs.Source: §5 · p. 7, Table 5 · p. 7
  • DELTA + TARQA achieves higher relieved accuracy than all other baselines on FinTabNetQA TabVQA.The evaluation includes skyline results using ground-truth HTML and OTSL inputs to TARQA.Source: Table 6 · p. 7, §5 · p. 7

On Multilingual TabVQA

  • On TORQUE, the approach exceeds all VLMs by more than 10 p.p. except Qwen-2.5 VL Instruct, and no Hindi-capable LLM surpasses it.The pipeline is zero-shot on TORQUE, with no components trained on Hindi.Source: Table 7 · p. 8, §5 · p. 8

6. Conclusion

The paper presents a modular table-understanding pipeline built around DELTA and compact OTSL representations. It reports strong results on standard benchmarks and the Hindi TORQUE benchmark.

  • DELTA separates physical and logical structure recognition from OCR, producing compact OTSL tables; TARQA fine-tunes an LLM on OTSL for TabVQA.The paper also introduces a lossless HTML-to-OTSL conversion.Source: §6 · p. 8
  • DELTA achieves high TEDS-S scores and surpasses recent VLM-based approaches on table reconstruction.The conclusion also describes strong results on WTQ and FinTabNetQA for TARQA.Source: §6 · p. 8
  • On TORQUE, the paper reports that its zero-shot approach significantly outperforms other models, including LLMs with inherent Hindi understanding.The conclusion presents DELTA and DELTA + TARQA results on this curated Hindi benchmark.Source: §6 · p. 8

7. Limitations and Future Work

The paper identifies OCR quality as a constraint on reconstruction and downstream TabVQA, and limits its experiments to English and Hindi. Future work targets broader language coverage and more complex queries.

  • OCR errors lower overall TEDS and can affect downstream TabVQA, while better OCR is expected to move TEDS closer to 100%.The authors say DELTA + TARQA could then more consistently approach skyline performance.Source: §7 · p. 8
  • Experiments cover English and Hindi benchmarks; extension to other languages is prospective.The authors point to the modular design and OCR support as enabling that extension.Source: §7 · p. 8
  • Future work will address reasoning for complex queries, including abstractive QA.The stated aim is to broaden the approach's scope and impact.Source: §7 · p. 8

Supplementary Material

The supplement provides terminology references and motivates DELTA's finer separation of table structure recognition stages. This decomposition aims to improve flexibility, debugging, and modularity.

9. Symbols and abbreviations

  • Table 8 gives terminology used in the proposed approaches, and Table 9 lists abbreviations used throughout the paper.Source: §9 · p. 12

10. Motivation for Doubly Decoupled Approach

  • DELTA separates physical structure from logical structure recognition, then combines their outputs before OCR.The physical stage handles image-coordinate-dependent cell boundaries, while the logical stage predicts cell arrangements and relationships; the authors say this supports flexibility, debugging, and independently strengthening components.Source: §10 · p. 12

11. DELTA in detail

DELTA combines separate logical- and physical-structure predictions into spatially grounded tables, with OCR supplying cell text. Its OCR comparisons identify content recognition as an important constraint on reconstruction quality.

11. DELTA in detail

  • DELTA independently predicts physical and logical structure, then combines them before OCR to reconstruct complete HTML tables.The framework uses TATR for physical layout and SPRINT for logical relations; outputs include bounding boxes and row and column spans.Source: §11 · p. 12

11.1. SPRINT for Logical Structure

  • SPRINT generates compact OTSL or HTML sequences for logical table structure using a Global Context Attention encoder and transformer decoder.It focuses on layout rather than text semantics, which the authors describe as supporting language independence and robustness to multilingual and noisy documents.Source: §11.1 · p. 12

11.2. TATR for Physical Structure

  • TATR predicts table rows and columns for DELTA's physical structure stage, using a model pretrained on FinTabNet, PubTabNet, and PubTables-1M.The pipeline aligns those detections with SPRINT's output.Source: §11.2 · pp. 12–13

11.3. TATR and SPRINT for Complete TSR

  • DELTA maps SPRINT's logical cells to TATR's bounding boxes and writes HTML with explicit spans and spatial grounding.The described conversion uses OTSL entries to determine cells and row boundaries, and preserves rowspan and colspan attributes for merged cells.Source: §11.3 · pp. 13–14

11.4. OCR Ablations

  • DELTA's TSR exceeds 95% TEDS-Structure, but OCR is the primary bottleneck for overall TEDS; replacing predicted cell content with ground truth yields TEDS above 90% in most cases.The authors attribute OCR errors to noise, complex layouts, slanted text, and special symbols.Source: §11.4 · p. 14
  • 12.45 p.p.: EasyOCR increases weighted-average TEDS across FinTabNet, PubTabNet, FinTabNetQA, and TORQUE versus Tesseract, also improving TabVQA accuracy.On PubTables-1M, EasyOCR achieves TEDS 54.8, while Tesseract is described as impractically slow for experiments.Source: §11.4 · pp. 14–15

12. Evaluation Metrics

The evaluation uses structure- and content-sensitive metrics for table reconstruction and answer-correctness metrics for TabQA and TabVQA.

12. Evaluation Metrics

  • TEDS-S measures structural fidelity, while TEDS accounts for both table structure and content alignment across reconstruction datasets.The evaluated reconstruction datasets are PubTabNet, PubTables-1M, FinTabNet, and TORQUE.Source: §12 · p. 15
  • The evaluation also compares OCR modules using TEDS scores and reports per-image Tesseract and EasyOCR latency on the FinTabNet test set.The supplied captions do not provide the comparison values.Source: Table 11 · p. 15, Table 12 · p. 15
  • EM measures exact correctness, ANLS tolerates minor variations, and Relieved Accuracy reflects robustness to different semantic answer formats.These metrics are reported for WTQ, FinTabNetQA, and TORQUE.Source: §12 · p. 15

13. OTSL Ablation

The OTSL ablation examines its compactness relative to HTML and reports qualitative benefits for LLM-based table understanding, alongside an OCR-related boundary.

13. OTSL Ablation

  • OTSL preserves structural fidelity while providing a significantly more compact representation than HTML in examples of large and structurally complex tables.The analysis presents one large-table scenario and another with a more complex structure.Source: §13 · p. 15
  • The analysis reports that some tables misprocessed in HTML were correctly interpreted in OTSL, and that shorter representations allow larger tables within context limits.The passage links reduced context length with more efficient and accurate table understanding.Source: §13 · p. 15
  • Mathematical-expression handling depends on the OCR module applied after structure recognition, rather than an inherent limitation of OTSL.Source: §13 · p. 15

14. Qualitative Results

Qualitative examples show DELTA reconstruction successes and failures on FinTabNet and TORQUE, alongside TARQA answer successes and errors on WTQ and FinTabNetQA.

14. Qualitative Results

  • The qualitative-results section covers reconstruction, TabQA, and TabVQA examples; an accompanying table is described as an overview of evaluation metrics.Source: §14 · p. 15, Table 13 · p. 16

14.1. Table Reconstruction

  • TEDS-S reaches 100 on two clear FinTabNet examples, while OCR errors, low resolution, borderless layout, and bullet points accompany reconstruction failures.The successful examples have TEDS values of 80.98 and 88.11; failure examples have TEDS values of 19.35 and 19.95.Source: §14.1 · p. 17, Figure 7 · p. 18, Figure 8 · p. 19
  • TORQUE reconstruction examples range from TEDS-S 97.22 and TEDS 96.06 to TEDS-S 44.44 and TEDS 14.60, with slanted text and low resolution hindering the latter.Source: §14.1 · p. 17, Figure 9 · p. 20

14.2. Table Question Answering

14.3. Table Visual Question Answering

  • DELTA+TARQA answers numerical and textual questions accurately on difficult borderless FinTabNetQA tables, but complex question types and poor image quality yield errors.The authors identify improved OCR and reasoning as future directions.Source: §14.3 · p. 17, Figure 12 · p. 23, Figure 13 · p. 24

Related papers

Loading 2609.17458v1…