PaperlayerView exact revision on arXiv

Source-linked AI summary

A Table Is Worth 64 Tokens: Pixel-level Compression for Multi-Table Document Question Answering

Iñigo Alonso, Mirella Lapata

arXiv:2608.26949v1cs.AI

TL;DR

The paper studies whether pixel-level table compression can reduce the cost of QA over long documents without harming table understanding. It evaluates multiple VLMs and benchmarks, then uses compressed tables for relevance identification and native-resolution tables for reasoning. On long documents, this two-step approach uses 41% fewer tokens while gaining 7 accuracy points over single-step native-resolution QA.

  • Problem

    Prior optical context-compression evidence focuses on plain text, leaving unclear whether pixel-level table compression improves efficiency without reducing performance.

  • Method

    A training-free two-step method identifies relevant tables from a compressed context, then supplies only those tables at native resolution for reasoning.

  • Results

    41% fewer total tokens and 7 accuracy points over single-step QA with native-resolution tables are achieved on long documents.

  • Takeaways & Limitations

    Compression is useful for table identification even when it is too aggressive for reliable cell-level reasoning, enabling selective restoration at native resolution.

  • Takeaways & Limitations

    The experiments cover English-language financial documents and two datasets, use clean uniformly rendered tables with known boundaries, and measure decoder tokens rather than full computational cost.

Abstract

from arXiv · show

Answering questions over real-world documents requires processing long inputs that interleave text with tables. Optical context compression, which represents context as images, promises to reduce token cost, but its effect on table understanding remains unclear. We study pixel-level table compression for question answering over documents with multiple tables, evaluating five VLMs across two benchmarks and five visual-token budgets. Representing tables as images at native resolution matches text in both performance and efficiency, but downscaling them makes models compensate the loss in readability with longer, less effective reasoning traces that cancel the expected savings. Highly downscaled tables, however, preserve enough signal to identify whether they are relevant to a question. We exploit this asymmetry with a training-free, two-step method: the model first identifies the tables needed to answer a question from a pixel-compressed context, and then reasons over those at native resolution. On long documents, our method saves 41% of total tokens and gains 7 accuracy points over single-step QA with native resolution tables. It also uses 15% fewer tokens than the most efficient single-step compressed configuration, with no accuracy loss.

1 Introduction

The paper asks whether pixel-level compression can make table-heavy document QA more efficient without sacrificing performance. It finds that compression preserves coarse relevance signals but harms detailed reading, motivating a two-stage method that identifies tables cheaply before reasoning at native resolution.

  • Motivation: Long, heterogeneous documents make efficient representation necessary for document QA at scale.Inputs mix free-form text with tables, and processing costs rise as contexts become longer.
  • Study scope: Pixel-level table compression is studied across 5 VLMs, two datasets, and five visual-token budgets.The evaluation covers multi-table document QA and compares visual representations with HTML serialization.
  • Findings: Native-resolution table images match HTML accuracy while using fewer tokens, whereas aggressive compression makes tables illegible and reduces accuracy.Downscaling removes cell-level information needed for numerical reasoning.
  • Findings: Lower resolutions induce longer reasoning traces that offset input-token savings, yet heavily downscaled tables retain enough coarse signal to identify relevant tables.This creates an asymmetry between fine-grained reading and question-conditioned table identification.
  • Method: The two-stage method first identifies relevant tables in compressed context, then reasons over only those tables at native resolution.Stage 1 performs identification instead of answering; Stage 2 supplies the selected tables for answer generation.
  • Results: 41% fewer total tokens and 7 accuracy points over single-step QA with native-resolution tables are achieved on long documents.The method also matches the accuracy of the most efficient single-step compressed configuration using 15% fewer tokens.
  • Comparison: The method matches or exceeds retrieval at lower token cost and continues saving tokens when applied on top of retrieved context.A dedicated retriever identifies evidence tables more accurately, but that advantage does not produce a better accuracy-efficiency trade-off.

2 Related Work

The work builds on long-context efficiency, optical context compression, and table-representation research. It targets a gap in prior benchmarks and methods by studying training-free pixel compression for structured tables embedded across real documents.

  • Efficient Long-context Inference: Long-context approaches reduce processed context through prompt compression, memory, agentic reading, retrieval, or evidence-identification stages.The paper adopts evidence identification followed by reasoning as a related strategy.
  • Optical Context Compression: Optical context compression represents rendered language as pixels that visual encoders may encode with fewer tokens than raw text.Prior work extends this idea from OCR-style reconstruction to downstream tasks with trained and off-the-shelf VLMs.
  • Optical Context Compression: The paper studies training-free pixel compression for information-dense tables across fixed- and variable-resolution visual encoders.This contrasts with visual-token pruning approaches often tied to particular architectures.
  • Table Representation and Reasoning: Table research includes linearization, visual-textual representation comparisons, and dynamic routing between modalities.Linearization can impair structural reasoning over long tables, motivating alternative representations.

3 Experimental Framework

The experimental framework evaluates table representations in realistic multi-table document QA using matched text and image content, multiple benchmarks, models, and visual-token budgets. Efficiency is measured with decoder-token counts while preserving the study’s dataset and rendering constraints.

  • Benchmarks: The study uses MultiHiertt and FinLongDocQA, representing short documents with typically 3 tables and 4 pages and long documents with typically 80 tables and 100 pages.Both benchmarks interleave multiple tables with surrounding text.
  • Evaluation subsets: MultiHiertt is evaluated on its development set, while FinLongDocQA uses 400 randomly sampled table-category questions from context-compatible documents.The MultiHiertt split has private test labels, and FinLongDocQA samples documents fitting within 128k tokenizer tokens.
  • Input Settings: The text and hybrid settings differ only in table representation: HTML serialization versus images rendered from the same HTML.This preserves content and structure so modality comparisons isolate representation effects.
  • Evaluation caveat: The evaluation restricts FinLongDocQA to its table category because some evidence annotations were incorrect at dataset selection.The dataset creators later corrected the annotations used for reported results.
  • Models: The evaluation covers five open-weight VLMs with fixed- or variable-length visual tokenization and differing context windows and reasoning modes.Gemma 4 uses fixed visual-token configurations, while Qwen 3 VL and Qwen 3.5 use token counts that scale with image area.
  • Visual Token Budgets: Five nominal budgets are tested per table image: B ∈{64, 128, 256, 512, 1,024}.Images are downscaled to fit the budget when necessary; Gemma maps these to its discrete configurations.
  • Efficiency metric: Efficiency is measured as total language-decoder tokens, including textual input, visual, and generated tokens.This captures reasoning-token overhead but excludes visual-encoder computation and direct latency.

4 Does Pixel-Level Compression Affect Table Comprehension?

Pixel-level table compression preserves table understanding only at sufficiently large visual-token budgets. Stronger compression degrades transcription and QA, while also inducing longer reasoning traces that offset input-token savings.

  • 4.1 Table Transcription: Table transcription evaluates recovery of table structure and cell values using TEDS across HTML→HTML, LATEX→HTML, and IMG→HTML settings.HTML→HTML controls copying, LATEX→HTML isolates format conversion, and IMG→HTML tests visual transcription across five budgets.
  • 4.1 Table Transcription: Transcription from images degrades progressively as the visual-token budget decreases, indicating loss of fine-grained structure and cell-value information.The HTML→HTML control is nearly perfect, while LATEX→HTML already incurs format-conversion error.
  • 4.2 Question Answering: Larger visual-token budgets let table images match or sometimes exceed HTML QA accuracy while using fewer input tokens.Below the sufficient-resolution point, accuracy generally falls as compression tightens.
  • 4.2 Question Answering: Lower visual-token budgets produce longer reasoning traces without improving performance, driven by repeated cell-value misreadings and uncertainty over illegible entries.On FinLongDocQA, input-token savings can still outweigh added generation because documents contain many more tables.
  • 4.2 Question Answering: Aggressive compression removes detail needed for direct numerical reasoning, creating a limitation that motivates identifying relevant tables before restoring them at native resolution.Moderate compression can save input tokens without sacrificing accuracy, but aggressive compression offsets savings through reduced readability and extra generation.

5 Our Two-Stage Method: Identify first, Reason Later

The two-stage method uses compressed tables to identify relevant evidence, then restores only those tables at native resolution for reasoning. This separation improves the token-accuracy trade-off and remains useful alongside external retrieval.

  • 5.1 Stage 1: Relevant-Table Identification: The first stage receives document text and compressed tables, then requests the tables needed to answer the question.The second stage appends requested tables at native resolution for final reasoning.
  • 5.1 Stage 1: Relevant-Table Identification: The method is training-free, prompt-level, and model-agnostic, with identification performed without test-time reasoning to minimize overhead.Preliminary experiments found no significant improvement from enabling reasoning during identification.
  • 5.1 Stage 1: Relevant-Table Identification: Table identification remains nearly as accurate at 64 tokens as at higher resolutions or in HTML, unlike transcription and direct QA.Aggressive compression preserves coarse cues about a table’s topic and relevance while removing fine-grained readability.
  • 5.1 Stage 1: Relevant-Table Identification: At 64 visual tokens, compressed images improve identification F1 over TABLEID by 16 points for Gemma 4 26B-A4B and 10 points for Qwen 3.5-9B.This indicates that compressed renderings provide evidence beyond table position and surrounding text.
  • 5.2 Stage 2: Full-Resolution Reasoning: On FinLongDocQA, the method gains 7.1 accuracy points over native-resolution single-step QA while using 40.6% fewer tokens on average.At 64 tokens, it exceeds native-resolution QA by 3.0 points for Gemma 4 26B-A4B and 11.2 points for Qwen 3.5-9B.
  • 5.3 Relationship to External Retrieval: External retrieval identifies evidence tables more accurately, but it does not deliver a better end-to-end trade-off than compressed identification.On FinLongDocQA, the two-stage method exceeds retrieval by 16 accuracy points at the same cost.

6 Conclusions

Pixel compression reduces the cost of multi-table document QA, but aggressive downscaling harms cell-level reasoning while preserving coarse table identification. A two-stage method exploits this asymmetry by locating relevant tables cheaply before native-resolution reasoning.

  • Moderate compression lets table images match HTML serialization while using fewer tokens, whereas aggressive compression loses cell-level information and prolongs reasoning.The resulting longer reasoning traces erode the expected efficiency gains.
  • Compression affects table reading and relevance identification differently: unreadable tables can still provide enough coarse signal to locate relevant evidence.
  • The training-free two-stage method identifies relevant tables in compressed context, then restores only those tables for native-resolution reasoning.
  • Pixel compression is most effective as an inexpensive evidence-location step rather than as the representation used for direct reasoning.

Limitations

The evaluation has methodological and measurement boundaries: it uses clean, known table structures in a narrow document setting and token counts as an incomplete efficiency proxy.

  • The experiments cover English-language financial documents and two datasets, with MultiHiertt results reported on the development set.
  • Tables are rendered from clean structured representations, so real-world fonts, layouts, scanning artifacts, and recognition errors are not evaluated.
  • The method assumes known table boundaries; raw documents may require an additional table-detection or extraction stage.
  • The effective compression level depends on table dimensions, information density, rendering style, and the model’s visual encoder.
  • The reported token budgets may not transfer directly to other domains or document formats, although the evaluation procedure can identify suitable budgets elsewhere.
  • Decoder-token counts omit visual-encoder computation, extra-pass latency and overhead, and hardware-dependent costs.Wall-clock latency, memory use, and FLOPs would be needed for the full computational trade-off.

E Full Results

The full-results appendix reports accuracy and total-token comparisons for direct QA and the two-stage method in full-document and retrieved-context settings.

  • The appendix provides complete results for Sections 4.2, 5.2, and 5.3 across all models and both datasets.
  • On FinLongDocQA, evaluation uses only the largest and most recent model of each family because Gemma 4 E4B’s context window cannot accommodate these documents with reasoning traces.
  • In full-context evaluation, direct QA answers in one step, while the two-stage method identifies needed tables before native-resolution reasoning.
  • In retrieved-context evaluation, BGE-M3 assembles contexts using k=5 on MultiHiertt and k=100 on FinLongDocQA.

F Table Identification: All Models

Table identification remains robust across visual-token budgets, but coarse page-level evidence annotations can obscure whether the model selected the correct table within a page.

  • Identification varies little across visual-token budgets for all models, with gains over TABLEID clearest for Qwen 3.5 and Gemma 4 models.The gain is marginal for Qwen 3 VL.
  • On FinLongDocQA, low-budget identification F1 is almost identical to TABLEID, but page-level annotations provide only an upper bound on the true evidence set.
  • At 64 tokens, removing table contents reduces downstream accuracy to 37.0 for Qwen 3.5 and 43.2 for Gemma 4 26B-A4B, despite indistinguishable F1.
  • The annotation granularity can reward locating the right page even when distinguishing which table matters remains necessary for answering.

G At What Resolution to Return the Requested Table

The method uses independent budgets for compressing all tables and returning the requested table. Returning the evidence at native resolution preserves accuracy and avoids compression-induced reasoning overhead.

  • The first stage compresses every table, while the second returns the requested table at a chosen resolution.The main text fixes the return budget at 1,024 visual tokens, which is uncompressed on these datasets.
  • HTML and 1,024 visual tokens perform equally, while accuracy degrades from 512 tokens downward.The degradation is slight initially and becomes clearer at lower budgets.
  • Lower-budget returned tables reintroduce longer reasoning traces, removing the efficiency benefit of compression.The second stage therefore returns the requested table at 1,024 tokens.

H Retriever Comparison

The paper compares BGE-M3 and ColQwen2 according to the modality of the retrieval pool. ColQwen2 is stronger for table-only retrieval, while BGE-M3 is required when paragraphs and tables are mixed.

  • BGE-M3 indexes linearised text, whereas ColQwen2 indexes rendered images.Only BGE-M3 can operate over a pool mixing paragraphs and tables.
  • ColQwen2 outperforms BGE-M3 when retrieving tables from a table-only pool.This comparison uses BGE-M3 over textual serialization and ColQwen2 over rendered table images.
  • BGE-M3 is used for mixed-modality pools, while ColQwen2 is used for table retrieval and as the identification reference.The choice follows each retriever’s supported input modality and table-only retrieval performance.
  • FinLongDocQA uses page-level evidence annotations, so every table on an evidence page counts as relevant.This inflates the evidence set; its candidate pool contains around 80 tables per document versus three for MultiHiertt.

I Retriever for Identification

The paper evaluates model-based table identification against dedicated retrieval and direct QA across two datasets, models, and visual-token budgets. The comparisons track accuracy, token cost, retrieval quality, and the number of retrieved units.

  • The retrieval condition supplies ColQwen2-selected, uncompressed tables while providing the full document text.ColQwen2 therefore assumes the role otherwise played by the method’s first identification stage.
  • Figures 10 and 11 compare accuracy against total tokens for direct QA, two-step QA, and BGE-M3 retrieval references.The comparisons cover MultiHiertt and FinLongDocQA across four listed VLMs; Figure 11 uses increasing retrieval k as reference.
  • Relevant-table identification is evaluated with F1 across token budgets and table representations.The figures decompose F1 into achieved score and shortfall on requested tables, with HTML and TABLEID as references.
  • FinLongDocQA identification scores are upper bounds because every table on an evidence page is treated as relevant.Its retrieval reference uses ColQwen2@4, the dataset’s best-F1 operating point.
  • Retrieval performance is reported as average recall and F1 against the number of retrieved units.The comparison covers BGE-M3 and ColQwen2 in table-only pools, while mixed pools use only the text retriever.
Loading 2608.26949v1…