PaperlayerView exact revision on arXiv

Source-linked AI summary

PARTAB: Partition-Aware Reasoning with Structured Evidence for Scalable Table Understanding

Md Mahadi Hasan Nahid, Davood Rafiei

arXiv:2608.24082v1cs.CLcs.AIcs.IR

TL;DR

Large language models struggle to localize evidence as tables become larger and more complex, while full-table and single-view reasoning can obscure necessary row–column relationships. PARTAB constructs semantically coherent, row-linked evidence regions and hierarchically selects column groups and row partitions before answer generation. It improves over full-table prompting and recent methods on key benchmarks while remaining competitive on numerical reasoning.

  • Problem

    LLM table reasoning degrades on large and complex tables because irrelevant context makes the required localized evidence harder to identify.

  • Method

    PARTAB builds a query-conditioned structured evidence state of semantically organized, row-linked regions and selects groups and parts before composing evidence for reasoning.

  • Results

    PARTAB achieves 79.31 EM on WikiTableQuestions, 90.48 accuracy on TabFact, and remains competitive on TableBench with 70.33 EM for numerical reasoning and 82.71 accuracy for fact checking.

  • Takeaways & Limitations

    Structured, partition-aware evidence construction improves evidence localization and provides larger benefits on complex tables for scalable table understanding.

  • Takeaways & Limitations

    PARTAB does not explicitly enforce global completeness for aggregation tasks, so it can miss evidence when full-table coverage is required.

Abstract

from arXiv · show

Large Language Models (LLMs) have shown strong capabilities in table reasoning, but their effectiveness degrades as tables grow in size and complexity due to irrelevant context and difficulty localizing the evidence required for reasoning. Existing approaches typically reason over either the full table or a single reduced view, which can still obscure important row-column relationships. We introducePARTAB (Partition-Aware Reasoning overTables), a framework that constructs a structured evidence interface between the LLM and the table. PARTAB represents query-relevant evidence as semantically coherent, row-linked table regions and performs hierarchical selection over column groups and row-level partitions before composing the selected evidence for answer generation. We evaluate PARTAB on multiple table reasoning benchmarks, covering question answering, fact verification, and numerical reasoning. PARTAB consistently improves over full-table prompting and several recent table reasoning methods, achieving strong performance on WikiTableQuestions and TabFact while remaining competitive on numerical reasoning. Additional analyses show that semantic partitioning and targeted evidence selection improve evidence localization, substantially reduce the reasoning context, and provide larger benefits on complex tables. These results demonstrate the value of structured, partition aware evidence construction for scalable table reasoning.

1 Introduction

Table reasoning becomes harder as tables grow because irrelevant content obscures localized row–column evidence, while full-table and single-view approaches each have limitations. PARTAB addresses this by organizing query-relevant evidence into linked partitions and selecting them hierarchically for reasoning.

  • LLM effectiveness degrades as tables grow in size and complexity because irrelevant content makes evidence localization difficult.
  • Existing table reasoning includes executable program generation and direct reasoning over serialized tables, with different strengths and limitations under noise and ambiguity.
  • Full-table prompting exposes models to irrelevant context, whereas single-view pruning can remove evidence needed for complete reasoning.
  • The paper reframes table reasoning as operating over semantically coherent subsets rather than a single global view.
  • PARTAB constructs a query-conditioned evidence state of independently addressable, row-linked regions and composes selected regions during reasoning.
  • Across benchmarks, PARTAB improves performance and evidence localization, reduces context, and provides larger benefits on complex tables.

2 Methodology

PARTAB uses a modular pipeline that converts a question and table into structured, row-linked parts, selects relevant column groups and row chunks, and executes reasoning over the selected evidence.

  • PARTAB processes a question and table through question analysis, partition building, group and part selection, and answer execution.
  • Question Analyzer: Question analysis identifies question type, required operations, and expected answer type to guide partitioning and selection.
  • Partition Builder: The partition builder groups columns semantically and divides each group into fixed-size row chunks linked by row identifiers.The default chunk size is c = 5, and each part stores its group name, row range, column set, and data content.
  • Group and Part Selector: Group selection routes the question to necessary column groups before part selection chooses specific row chunks within those groups.
  • Group and Part Selector: PARTAB supports basic, TF–IDF similarity-based, and LLM-based part selection strategies, with selection intended to reduce noise while preserving necessary evidence.
  • Answer Executor: The answer executor serializes selected parts into compact context and instructs the model to use only those parts, linking information across them with row_id.

3 Evaluation and Experimental Setup

PARTAB is evaluated on question answering, fact verification, and numerical reasoning benchmarks using a modular multi-model setup and zero-shot answer generation. The evaluation includes WikiTableQuestions, TabFact, and TableBench, with standard task metrics reported.

  • Datasets: The evaluation covers WikiTableQuestions, TabFact, and TableBench across compositional question answering, fact verification, and numerical reasoning.
  • Datasets: WikiTableQuestions uses Exact Match accuracy, while TabFact uses classification accuracy for entailment or refutation decisions.
  • Implementation: PARTAB uses GPT-5-nano for partition construction and GPT-4o-mini for answer execution, with additional evaluation using Gemini-2.5-Flash-Lite and DeepSeek-v4-Flash.
  • Implementation: Answer-generation experiments use zero-shot prompting without few-shot demonstrations, chain-of-thought, or program-of-thought prompting.
  • Implementation: The default configuration uses row chunk size c = 5 and top-k = 6 for TF-IDF similarity retrieval.

4 Results and Discussion

PARTAB improves table reasoning across benchmarks by combining semantic partitioning with targeted evidence selection. Its benefits are especially pronounced on large and complex tables, while remaining dependent on effective partition selection and executor reasoning.

  • Downstream Results: PARTAB achieves 79.31 EM on WikiTableQuestions and 90.48 Acc on TabFact, while remaining competitive on TableBench with 70.33 EM for numerical reasoning and 82.71 Acc for fact checking.
  • Partition Construction Performance: 96% of generated semantic column groups are fully coherent in manual evaluation over 100 tables.
  • Partition Selection Performance: LLM-based part selection performs best across benchmarks, indicating that semantic understanding helps identify minimal yet sufficient partitions.
  • Distribution of Semantic Column Groups: Most tables contain three or four semantic groups: 77.6% for WikiTQ, 78.9% for TabFact, and 79.1% for TableBench.
  • Frontier Models on Large and Complex Tables: PARTAB improves over full-table prompting for every backbone–dataset combination, with average gains of +18.96, +9.03, and +12.65 points for GPT-4o-mini, GPT-5-mini, and DeepSeek-v4-Pro.
  • Frontier Models on Large and Complex Tables: On TabFact-Hard, PARTAB gains 25.53, 21.12, and 34.04 points for GPT-4o-mini, GPT-5-mini, and DeepSeek-v4-Pro, respectively.
  • Evidence Reduction: The selector retains only 2.44–3.88 parts from 9.84–17.56 candidates per table, reducing the candidate partition set by 77.9% on WikiTQ and 75.2% on TabFact and TableBench.
  • Ablation Analysis: Ablations show that removing the Question Analyzer reduces exact match by 7.10 points, random grouping reduces performance by 10.47 points, and row chunking without semantic groups reduces it by 22.12 points.

5 Related Work

Prior table-reasoning approaches trade off structured computation, flexibility under noisy text, scalability, and evidence completeness. PARTAB instead organizes query-relevant information into semantically coherent, row-linked regions and selects evidence hierarchically.

  • Direct serialized-table prompting handles ambiguity and noise flexibly, but performance degrades as tables grow and row–column relationships become harder to capture.
  • Text-to-SQL systems support structured and aggregation-heavy queries but struggle when tables contain noisy textual descriptions.
  • Decomposition and retrieval methods reduce reasoning complexity, yet many retain single-view reasoning or fail to explicitly structure reasoning within tables.
  • PARTAB constructs a structured evidence interface from semantically coherent, row-linked regions and combines semantic grouping with hierarchical partition selection and cross-region evidence composition.

6 Conclusion

PARTAB provides a partition-aware interface between LLMs and tables by composing selected, semantically coherent, row-linked regions rather than relying on a full table or single reduced view. Across benchmarks, it improves evidence localization and reduces context, with larger benefits on complex tables.

  • PARTAB organizes query-relevant evidence into semantically coherent, row-linked regions for composition during reasoning.
  • Across multiple table reasoning benchmarks, PARTAB shows strong performance, improved evidence localization, and substantial context reduction.
  • PARTAB yields larger benefits on complex tables, supporting structured partition-aware evidence construction as a scalable reasoning interface.

Limitations

PARTAB is scoped to inference-time reasoning when query evidence is localized to subsets of rows and columns. Its limitations include dependence on LLM components, incomplete global coverage for some aggregation tasks, heuristic partitioning, and added latency.

  • PARTAB targets inference-time reasoning over tables whose query evidence is localized to subsets of rows and columns.
  • LLM-based analysis, grouping, and selection make PARTAB sensitive to prompt design and model variability, with errors potentially propagating across stages.
  • PARTAB does not explicitly enforce global completeness for aggregation tasks, while heuristic partitioning may not generalize optimally across diverse table structures.
  • Although its stages use reduced context, PARTAB’s multi-stage pipeline can introduce additional latency compared with single-pass prompting.
  • Future work includes symbolic execution for aggregation, improved learned or hybrid retrieval, and extensions to multi-table reasoning and temporal queries.

Ethics Statement

PARTAB does not introduce or collect sensitive user data. The paper nevertheless frames it as a research framework requiring validation and human oversight for consequential applications.

  • PARTAB does not introduce or collect sensitive user data.
  • Incorrect answers may arise when evidence is omitted or downstream reasoning fails, and such errors could matter in high-stakes domains without verification.
  • PARTAB requires multiple LLM inference calls, adding computational cost relative to single-pass prompting despite lower token usage than several multi-stage baselines.
  • The authors recommend appropriate validation and human oversight for consequential applications.

A Additional Results

Additional experiments show that PARTAB’s gains persist under matched inference budgets and that its selection procedure usually identifies sufficient evidence with limited fallback use.

  • Budget-Matched Full-Table Control: 79.31 exact match versus 64.61 for five-call full-table prompting, an improvement of 14.70 points.The budget-matched control indicates that repeated full-table prompting does not reproduce PARTAB’s localized evidence state.
  • Selection Stability and Fallback Frequency: Fallback selection was activated for 2.7% of WikiTQ, 1.9% of TabFact, and 0.24% of TableBench examples.Most cases arose from malformed or unparsable structured outputs, and the fallback did not drive reported performance.
  • Evidence Coverage: 87% evidence recall was achieved on 100 manually inspected WikiTQ examples.Remaining failures primarily involved aggregation-heavy questions or required rows in unselected partitions.
  • Error Analysis: Evidence-localization failures are distinct from downstream answer-execution or computation errors.An incorrect answer can occur even when partition selection supplied the correct evidence.

B Additional Analysis

PARTAB reduces reasoning context while retaining strong accuracy, with LLM-based selection outperforming more aggressive similarity-based reduction and benefiting especially from structural partitioning.

  • Accuracy–Efficiency Trade-off: Five LLM calls and approximately 2.3k tokens per query yielded the highest TabFact accuracy among compared methods.Chain-of-Table uses up to 25 calls and approximately 13.2k tokens, while TableMaster uses 11 calls and approximately 3.3k tokens.
  • Accuracy–Efficiency Trade-off: Approximately 1.6k tokens per query made the similarity-based variant the most token-efficient configuration.Its lower accuracy indicates that context reduction is useful only when selected evidence is semantically adequate.
  • Partition Reduction Efficiency: 3.88 average selected parts and 79.31 EM characterized LLM-based selection on WikiTableQuestions.This configuration achieved the best trade-off between partition reduction and accuracy.
  • Dataset Structural Statistics: WikiTQ includes tables with up to 517 rows, while TableBench reaches 176 rows and TabFact 47 rows; all have maximum column counts of 20–21.These structural differences motivate partitioning along both rows and columns.

C Discussions

PARTAB is most useful when evidence is localized within long, wide, or noisy tables, while completeness-sensitive aggregation and selection failures remain important boundaries.

  • When Partition-Aware Reasoning Helps: The largest improvements occur on long or wide tables where required evidence occupies only a small portion of the complete table.Full-table prompting exposes the model to irrelevant values, increasing attention dilution and complicating evidence localization.
  • When Partition-Aware Reasoning Helps: Partitioning is particularly effective for lookup, comparison, and fact verification questions with localized attributes and rows.Large gains on TabFact-Hard across all three model backbones support this pattern.
  • Noisy Tables and Symbolic Reasoning: Semantic grouping and LLM-based selection can identify relevant attributes when cells contain noisy or free-form text and surface forms differ.This complements exact symbolic filtering rather than replacing symbolic execution.
  • Relationship to Full-Table and Single-View Reasoning: PARTAB preserves complementary row-linked table parts, aligning evidence across semantic groups through a universal row_id key.Unlike a single reduced view, this structure can retain multiple related regions for answer execution.
  • Remaining Limitations: Localized selection is less suitable for exhaustive aggregation, where counting, global extrema, or large-scale aggregation require broad row coverage.An adaptive global–local controller is proposed to route completeness-sensitive questions toward full-table or symbolic execution.

D Extended Literature Review

The extended review situates PARTAB among symbolic, decomposition, retrieval, and hybrid table-reasoning methods, while illustrative examples and prompts show its modular reasoning workflow.

  • Text-to-SQL and Symbolic Table Reasoning: Symbolic methods offer reliable aggregation and numerical execution but depend on precise predicates that may fail on noisy textual cells.Direct LLM reasoning is semantically flexible but becomes less reliable as tables and irrelevant context grow.
  • Decomposition and Context Reduction: Decomposition and retrieval methods improve scalability by reducing context, but selected subsets can omit evidence required for exhaustive global computation.This trade-off motivates coordinating localized semantic reasoning with global or symbolic execution.
  • Hybrid Neural–Symbolic Table Reasoning: Hybrid neural–symbolic systems use LLMs for interpretation and deterministic engines for numerical correctness and interpretability.PARTAB addresses evidence localization as a complementary bottleneck rather than replacing symbolic execution.
  • Illustrative Examples: Illustrative examples cover numerical computation, comparison, and entity lookup, showing selected columns, partitions, and rows supporting task-specific answers.The examples include elevation differences, biodiversity comparison, and extracting country information from an athlete-name field.
  • Prompt Templates: The modular prompt pipeline analyzes questions, groups columns, selects semantic groups and row-linked parts, and executes answers from selected regions.Figures 9–13 document the question analyzer, grouping, selection, part-selection, and answer-execution prompts.
Loading 2608.24082v1…