PaperlayerView exact revision on arXiv

Source-linked AI summary

EidosDoc: Implicit Structure Encoding for Cost-Effective Semi-Structured Document QA

Teng Lin, Yuyu Luo, Nan Tang

arXiv:2609.27784v1cs.IRcs.AI

TL;DR

Semi-structured document QA requires connecting evidence across text, tables, charts, and complex layouts, while existing approaches face trade-offs in accuracy, cost, and latency. EidosDoc combines implicit structure encoding, hybrid retrieval, and dynamic evidence expansion. Across four benchmarks, it outperforms prior state of the art while reducing per-query cost by over 50× and latency by nearly 4×.

  • Problem

    Prior methods face a trade-off: tree-based approaches incur high retrieval cost and latency, while explicit structure construction can be brittle and vulnerable to OCR or hierarchy errors.

    Source: C. Limitations of Prior Work · pp. 4–5

  • Method

    EidosDoc encodes text, layout, and hierarchy in dense vectors, then uses hybrid retrieval and dynamic expansion to gather spatially and structurally related evidence.

    Source: §I · pp. 2–3

  • Results

    Across four benchmarks, EidosDoc consistently outperforms prior state of the art, reducing per-query cost by over 50× and latency by nearly 4×.

    Source: §I · p. 3

  • Takeaways & Limitations

    The reported results establish an accuracy-cost-latency trade-off for semi-structured document question answering.

    Source: §I · p. 3

    The paper’s limitation discussion notes that explicit tree construction depends on accurate OCR and hierarchy detection, whose errors can propagate or cause evidence to be missed.

    Source: C. Limitations of Prior Work · p. 5

Abstract

from arXiv · show

Semi-structured documents are ubiquitous in scientific reports, financial statements, and technical manuals. Question answering over such documents requires simultaneous understanding of text, tables, charts, and complex hierarchical layouts. Existing methods either rely on repeatedly calling large language models for structure parsing and retrieval, leading to high cost and large latency, or they flatten the document and lose layout and hierarchy information, sacrificing answer accuracy. To address this, we propose EidosDoc, a novel system that achieves state-of-the-art accuracy with minimal computational expense. Our approach introduces three core innovations. (1) An Implicit Structure Encoder trained via contrastive learning and a structure consistency loss. This module jointly embeds hierarchical relationships, spatial positions, and textual content into a dense vector space, capturing document structure holistically without the need for manually defined and error-prone constructions. (2) A Hybrid Retrieval Pipeline that leverages BM25, layout fingerprints, and a lightweight cross-encoder to perform high-precision retrieval entirely without invoking an LLM, drastically reducing cost and latency. (3) A Dynamic Evidence Expansion mechanism that adaptively retrieves spatially adjacent and structurally related evidence, overcoming the evidence omission common in fixed-path retrieval methods. We evaluate EidosDoc on four benchmarks, and comprehensive evaluations show that EidosDoc achieves a new state-of-the-art accuracy on the four benchmarks. Crucially, it does so with a 50 times reduction in cost and 4 times lower latency compared to the previous state-of-the-art Method. These results demonstrate that EidosDoc establishes a new optimal trade-off among accuracy, cost, and speed, offering a practical and scalable path for accurate semi-structured document analysis.

I. INTRODUCTION

Semi-structured document QA must connect text, layout, and evidence scattered across regions, while prior approaches trade accuracy against cost and latency. EidosDoc combines implicit structure encoding, hybrid retrieval, and dynamic evidence expansion to address these challenges.

I. INTRODUCTION

  • Semi-structured QA can require combining numerical table evidence with chart patterns and synthesizing evidence across document regions.The example asks for a model comparison using a table’s Spearman correlation values and plotted predictions.Source: §I · pp. 1–2
  • Prior methods either discard layout, produce brittle or flattened hierarchies, miss cross-element relations, or incur high retrieval cost and latency.The introduction describes these limitations across extraction, multimodal, RAG, and tree-based approaches.Source: §I · pp. 1–2
  • EidosDoc’s implicit structure encoder jointly embeds text, layout, and hierarchical relationships without explicit tree construction.It uses contrastive learning and a structure consistency loss.Source: §I · pp. 2–3
  • Its fully local hybrid retrieval combines BM25, layout fingerprints, dense vectors, and a lightweight cross-encoder without LLM calls during retrieval.The pipeline is designed to reduce cost and latency relative to LLM-based retrieval.Source: §I · pp. 2–3
  • Dynamic evidence expansion retrieves spatially and structurally related evidence atoms, addressing omissions from fixed-path tree retrieval.The method aims to recover cross-region evidence that a fixed tree path could prune.Source: §I · pp. 2–3
  • Over 50× lower per-query cost and nearly 4× lower latency accompany consistent outperformance of prior state of the art across four benchmarks.The introduction also reports leading performance across all question types.Source: §I · p. 3

II. PRELIMINARIES

  • The preliminaries section establishes concepts, notation, and problem definitions, then characterizes limitations of prior work.Source: §II · p. 3

A. Document Representation

The document representation treats content as granular atoms linked by spatial adjacency, while the problem formulation defines QA as answering questions from relevant document evidence. The formulation covers text, non-text, hierarchy, and location-dependent questions.

A. Document Representation

  • Each document is a visual mix of textual and non-textual elements; its granular document atoms are self-contained and represented as (type, content, bbox, page, meta) tuples.Table cells remain separate atoms rather than being aggregated; their relationships can be captured through spatial adjacency.Source: A. Document Representation · p. 3
  • A spatial adjacency graph links atoms whose bounding boxes overlap or lie within a small distance, capturing local layout without imposing a hierarchical tree.The graph is used during retrieval to expand evidence.Source: A. Document Representation · p. 3

B. Problem Formulation

  • Semi-structured document QA maps a document and natural-language question to an answer supported by a relevant set of atoms, which may require reasoning across them.The relevant atoms are defined as a minimal sufficient set, and answer quality depends on retrieval recall and precision.Source: B. Problem Formulation · p. 3
  • For the protein-fluorescence question, only EidosDoc reports the correct numerical improvement of 0.46 and faithfully describes effectiveness.The figure compares document-processing methods answering that question.Source: Figure 2 · p. 4
  • The formulation spans text-only, hybrid, hierarchy-aware, and location-aware questions; multi-hop questions can combine these types, but no particular decomposition method is assumed.The system focuses on core retrieval and reasoning challenges and can be combined with existing query-decomposition techniques.Source: B. Problem Formulation · p. 4

C. Limitations of Prior Work

Prior approaches to semi-structured document QA trade off structural fidelity, retrieval quality, cost, latency, and fine-grained detail. Their limitations include flattening or brittle explicit structures and constraints of end-to-end multimodal processing.

a) Retrieval-Augmented Methods

  • TextRAG [24] flattens documents and loses layout and hierarchy, while page-level M3DocRAG [22] and SV-RAG preserve visual cues but retrieve coarsely and noisily.RAG methods also depend on retrieval quality, may require brittle explicit structure extraction, and incur high cost and latency when LLMs are used during retrieval.Source: C. Limitations of Prior Work · p. 4

b) Explicit structure methods

  • Explicit-structure systems such as ZenDB, DocAgent, MoDora, and TWIX [33] can achieve state-of-the-art accuracy, but incur cost and latency and risk brittle structures that prune relevant evidence.Their retrieval decisions may also depend on LLMs, introducing unpredictable behavior and high variance.Source: C. Limitations of Prior Work · pp. 4–5

c) End-to-end multimodal LLMs

  • End-to-end multimodal systems including GPT-5V [19], DocOwl2, and LayoutLMv3 face limited context windows, high image-encoding cost, lossy fine detail, and difficulty with non-adjacent-page reasoning.The passage reports their accuracy as far below tree-based methods and their cost as even higher.Source: C. Limitations of Prior Work · p. 5

III. SYSTEM OVERVIEW

EidosDoc separates one-time document preparation from local retrieval and evidence assembly, then uses an LLM only to generate the answer. Its implicit structure encoder embeds content and layout together while learning semantic, hierarchical, and spatial relationships.

III. SYSTEM OVERVIEW

  • EidosDoc preprocesses each document once, retrieves and expands evidence with local components, then sends condensed context to an LLM for answer generation.Preprocessing takes under two seconds for a typical ten-page document; retrieval and expansion take less than 80 milliseconds on a single CPU core.Source: §III · p. 5

A. Dual-Encoder Architecture

  • The dual encoder fuses separate text and layout embeddings into one atom vector, preserving semantic and spatial information without constructing an explicit document tree.The text tower encodes atom text and structural tokens, while the layout tower encodes bounding boxes, page number, atom type, and relative area.Source: §IV · p. 5, A. Dual-Encoder Architecture · p. 6

B. Training Objectives

  • Contrastive learning brings related atoms together, while structure consistency and layout proximity losses encode title-child and spatial relationships.Positive pairs include atoms in the same logical block and a title with its following paragraph; the layout loss regularizes embeddings of spatially adjacent atoms.Source: B. Training Objectives · p. 6

C. Inference and Use

The encoder’s stored atom embeddings support dense retrieval and parent-title matching, while question embeddings omit layout features.

C. Inference and Use

  • Each document atom’s embedding is computed once and stored in a FAISS IVF-PQ index for approximate nearest-neighbor search.The encoder runs locally and adds negligible pipeline cost.Source: C. Inference and Use · p. 6
  • At query time, the same text tower encodes the question without layout features, and cosine similarity retrieves the highest-scoring atoms.Source: C. Inference and Use · p. 6
  • The stored embeddings also support parent-embedding matching during dynamic evidence expansion.Source: C. Inference and Use · p. 6

D. Spatial Adjacency Graph Construction

EidosDoc routes questions through lightweight retrieval strategies and expands evidence using spatial relationships. Its spatial adjacency graph links nearby or overlapping atoms on the same page.

D. Spatial Adjacency Graph Construction

  • The spatial graph connects same-page atoms whose boxes overlap or whose closest points fall within a normalized distance threshold of 0.05.The graph averages 2–4 neighbors per atom, including relationships such as a chart and its caption.Source: D. Spatial Adjacency Graph Construction · pp. 6–7

V. HYBRID RETRIEVAL AND DYNAMIC EVIDENCE EXPANSION

  • Retrieval combines question-type classification, hybrid retrieval, and dynamic evidence expansion without invoking a large language model.Source: §V · p. 7

A. Question Type Classifier

  • A BERT-tiny classifier predicts one of four question types with 96% held-out accuracy and runs in under 1 ms on CPU.Its predictions select among text-only, hybrid, hierarchy-aware, and location-aware retrieval strategies.Source: A. Question Type Classifier · p. 7

B. Hybrid Retrieval Strategies

  • Strategies combine sparse or layout-based coarse retrieval with dense reranking, tailoring candidates to text, hybrid, hierarchical, or location-aware questions.Location-aware queries map spatial phrases to grid coordinates and can be padded with text-only candidates when fewer than 20 atoms are retrieved.Source: B. Hybrid Retrieval Strategies · p. 7

C. Cross-Encoder Reranking

The cross-encoder jointly scores question-atom pairs after retrieval, and dynamic expansion adds spatial and structural context to core evidence.

C. Cross-Encoder Reranking

  • The cross-encoder jointly processes each question and candidate atom, using teacher-LLM distillation to train relevance scores without extensive human annotations.Its distilled scores correlate 0.92 with GPT-4 on held-out validation and run over 1000× faster, at zero local inference cost.Source: C. Cross-Encoder Reranking · pp. 7–8
  • 94% recall@3 on the validation set compares with 82% for the dual-encoder alone, with larger gains for hybrid and hierarchy-aware questions.Source: C. Cross-Encoder Reranking · p. 8

D. Dynamic Evidence Expansion

  • For each core atom, spatial expansion adds graph neighbors, while structural expansion adds its parent title when the atom is not itself a title.Expansion supplies nearby cells, captions, or explanatory text and links values or paragraphs to their section heading; the final evidence set is capped at 18 atoms and restored to reading order.Source: D. Dynamic Evidence Expansion · p. 8

VI. ANSWER GENERATION

EidosDoc sends a short, high-signal evidence context to a single LLM call for answer generation. Its prompt and fallback modes are designed to handle insufficient evidence, while model choice allows users to trade off cost, latency, and accuracy.

VI. ANSWER GENERATION

  • The generator receives concatenated evidence atoms as a short, high-signal context in one LLM call.The evidence text preserves original reading order by page number and vertical position; answers are prompted to be concise, with “Not enough information” as a fallback.Source: §VI · p. 8, A. Prompt Design · p. 8

A. Prompt Design

  • The minimal prompt uses no few-shot examples or extra formatting instructions and asks the model to answer concisely or say “Not enough information.”Evidence is inserted as plain text in its original reading order.Source: A. Prompt Design · p. 8

B. Model Choice

  • EidosDoc supports cloud and local LLMs with context lengths of at least 1024 tokens, letting users trade off cost, latency, and privacy.Cloud models cost $0.0003–$0.001 per query; local models have negligible electricity-only cost and may be slower and slightly less accurate, while a 7B model can suffice for many practical purposes.Source: B. Model Choice · pp. 8–9

C. Handling Insufficient Evidence

  • An empty expanded evidence set bypasses the LLM; if the top-three reranker scores average below θ=0.4, strict mode abstains while relaxed mode lets the LLM judge sufficiency.The threshold is validation-set based and can be adjusted by the user for their accuracy-cost trade-off.Source: C. Handling Insufficient Evidence · p. 9
  • 94%: GPT-4o in relaxed mode refuses truly insufficient cases, versus 98% in strict mode; relaxed mode offers a higher answer rate with 2% vs. 0% error.Users can choose the mode that best fits their application.Source: C. Handling Insufficient Evidence · p. 9

VII. EXPERIMENTS

Across four benchmarks, EidosDoc outperforms prior baselines, with the largest reported MMDA comparison against MoDora. The experiments evaluate its retrieval and structural components alongside varied document and question settings.

A. Experimental Setup

  • The four benchmarks are MMDA, MP-DocVQA, DUDE, and M3DocVQA, with MMDA used as the primary analysis platform for its balanced question types.MMDA includes text-only, hybrid, hierarchy-aware, and location-aware questions.Source: A. Experimental Setup · p. 9
  • The study compares 11 representative baselines, including RAG, content- and structure-extraction, multimodal LLM, and tree-based methods.Evaluation reports AIC-Acc, ANLS, Recall@k, and MRR.Source: A. Experimental Setup · p. 9

B. Overall Performance

  • 76.21% AIC-Acc on MMDA exceeds MoDora’s 73.33% by 2.88 points; EidosDoc also leads on DUDE (+1.42), M3DocVQA (+1.84), and MP-DocVQA (+1.51).The M3DocVQA gain is noted for a benchmark requiring multi-page, multi-document reasoning.Source: B. Overall Performance · p. 9, Table I · p. 10
  • Removing the implicit structure encoder causes the largest ablation drop, 17.38 points, while the encoder learns dense hierarchical and layout representations instead of explicit trees.The encoder uses contrastive learning and structure consistency loss; the reported ablation attributes robustness to OCR noise and layout variability to learned soft relationships.Source: B. Overall Performance · pp. 9–10
  • 78.4% Recall@5 from the local cross-encoder exceeds MoDora’s LLM-based retrieval at 71.2%; layout fingerprints add 4.07 points in ablation.The retrieval pipeline combines BM25, layout fingerprints, dense vectors, and a distilled cross-encoder reranker.Source: B. Overall Performance · p. 10
  • Dynamic evidence expansion adds 3.36 points overall by retrieving spatial neighbors and parent titles, with larger gains on hybrid and multi-hop questions.Expansion is query-adaptive rather than restricted to a predetermined tree path.Source: B. Overall Performance · p. 10
  • The three innovations together are reported to deliver state-of-the-art accuracy with lower cost and latency than prior methods.The paper characterizes this as breaking the traditional accuracy-cost trade-off.Source: B. Overall Performance · p. 10

C. Performance by Question Type

On MMDA, EidosDoc leads MoDora across all four question categories, with gains tied to distinct retrieval and structure capabilities. Retrieval analysis also reports higher Recall@5 for EidosDoc than for MoDora.

C. Performance by Question Type

  • On MMDA, EidosDoc achieves the highest AIC-Acc across all four question categories, exceeding MoDora by 3.20 points on hybrid, 2.89 on hierarchy-aware, and 5.66 on location-aware questions.Source: C. Performance by Question Type · p. 10
  • On hybrid subcategories, EidosDoc scores 73.5% on table reasoning versus MoDora’s 68.2%, and 68.9% on chart questions versus 63.1%.The analysis attributes the alignment to the cross-encoder attending to question spatial references and atom positional metadata.Source: C. Performance by Question Type · p. 10

D. Retrieval Quality Analysis

  • 78.4% Recall@5 for EidosDoc exceeds MoDora’s 71.2%; the cross-encoder reranker yields the largest retrieval jump, +10.6 points in Recall@5.Retrieval quality was measured on 500 sampled MMDA questions; Fig. 4 presents the comparison.Source: D. Retrieval Quality Analysis · p. 10, Figure 4 · p. 11
  • Among 50 retrieval failures, OCR corruption accounted for 28%, ambiguous spatial references 22%, and three-or-more-atom synthesis 18%.Other reported causes were chart-to-text description errors (16%) and others (16%).Source: D. Retrieval Quality Analysis · p. 10

E. Ablation Study

Ablations identify the structural encoder and spatial signals as important contributors, while cost, latency, robustness, and generator sensitivity analyses probe system behavior under varied conditions.

E. Ablation Study

  • Removing the implicit structure encoder drops accuracy by 17.38 points; the layout tower contributes 7.74 points and the layout fingerprint 4.07 points.Removing both layout signals yields 65.23% accuracy, below the sum of their individual drops, indicating complementary effects.Source: E. Ablation Study · p. 11, Table II · p. 11

F. Cost and Latency Analysis

  • About $0.0005 per query and 1.1 seconds latency with GPT-4o compare with MoDora’s $0.025 and 4.3 seconds; retrieval takes 80 ms for 10 pages and 150 ms for 50.The paper reports near-linear offline preprocessing of 0.2 seconds per page for atom extraction plus 0.05 seconds per page for embedding.Source: F. Cost and Latency Analysis · p. 11

G. Robustness Analysis

  • At a simulated 20% character error rate involving random substitutions, deletions, and insertions, EidosDoc AIC-Acc drops from 76.21% to 66.8% (9.4% reduction), while MoDora drops from 73.3% to 58.1% (15.2%).The paper attributes EidosDoc’s robustness to embeddings being less sensitive to individual character errors than the exact keyword matching used in MoDora’s tree construction.Source: G. Robustness Analysis · pp. 11–12
  • From 10 to 50 pages, EidosDoc retrieval latency rises from 80 ms to 410 ms, while MoDora end-to-end latency rises from 4.3 to 19.2 seconds.The paper identifies MoDora’s LLM context window as its primary bottleneck in this setting.Source: G. Robustness Analysis · p. 12
  • On legal-document domain-shift testing, EidosDoc maintains 74.8%, while MoDora drops from 73.3% to 65.9%.The paper relates EidosDoc’s generalization to shared layout and hierarchy patterns across domains.Source: G. Robustness Analysis · p. 12

H. Generator Sensitivity Analysis

  • The GPT-4o and Qwen2.5-7B performance gap is 3.4 points, and the paper describes a 7B local model as viable for cost-sensitive applications.The paper attributes the smaller gap relative to long-context scenarios to EidosDoc’s focused, noise-free retrieval context.Source: H. Generator Sensitivity Analysis · p. 12, Table III · p. 12

I. Error Analysis

EidosDoc reports state-of-the-art accuracy on four benchmarks alongside lower cost and latency. Its error analysis identifies non-retrieval errors, while a fluorescence case illustrates multi-component evidence retrieval and reasoning.

I. Error Analysis

  • Among 100 randomly sampled incorrect answers, retrieval failure was the largest error source, but 58% were reasoning, OCR, or expansion errors.These error types indicate improvement opportunities beyond retrieval.Source: §I · p. 12

J. Case Study: Multi-Component Reasoning

  • The fluorescence case combined table values 0.22 and 0.68 with figure and text evidence to report a 0.46 gain while noting residual misclassifications.The case attributes this result to header-linked table cells, cross-encoder ranking, and dynamic evidence expansion.Source: J. Case Study: Multi-Component Reasoning · p. 12

VIII. CONCLUSION

  • EidosDoc achieved new state-of-the-art accuracy on four public benchmarks, with 50× lower per-query cost and 4× lower latency than existing methods.The conclusion also reports robustness under OCR degradation, long documents, and domain shifts.Source: §VIII · p. 12
Loading 2609.27784v1…