PaperlayerView exact revision on arXiv

Source-linked AI summary

TabScope: Question-Adaptive Scope Selection for Table Question Answering

Yuxiang Wang, Junhao Gan, Jianzhong Qi

arXiv:2609.03395v1cs.CLcs.AI

TL;DR

TableQA accuracy degrades with larger tables, and existing benchmarks provide limited evidence supervision and long-table coverage. TABSCOPE predicts whether each question needs localized or full-table reasoning, using operation-aware decomposition when localization is selected, while introducing silver sub-tables and SLQA. Experiments on WikiTQ and SLQA show that adaptive scope selection improves answer accuracy across moderate- and large-scale tables, while decomposition quality also improves.

  • Problem

    As tables grow, LLMs struggle to identify supporting rows and columns, while existing benchmarks lack explicit evidence annotations and broad long-table coverage.

  • Method

    TABSCOPE predicts the appropriate table scope and applies operation-aware decomposition for localized reasoning while retaining the full table when broader context is needed.

  • Results

    TABSCOPE improves answer accuracy across WikiTQ and SLQA, and operation-aware decomposition improves over the strongest baseline by 6.1 and 4.5 points on WikiTQ and 9.2 and 4.7 points on SLQA with GPT-5-mini and LLaMA-3.3-70B, respectively.

  • Takeaways & Limitations

    Effective long-table TableQA depends not only on how tables are processed, but also on deciding how much table context to retain for each question.

  • Takeaways & Limitations

    The evaluation covers English TableQA benchmarks and two representative LLMs, while the scope policy uses a predefined question-type taxonomy.

Abstract

from arXiv · show

Large Language Models (LLMs) have shown strong performance on table question answering, yet their accuracy often degrades as table size increases. We find that this degradation is not uniform across question types. Localization-sensitive questions are particularly affected by irrelevant table content, while questions requiring broader evidence may still benefit from full-table reasoning. Based on this observation, we propose a question-adaptive framework that dynamically selects between localized and full-table reasoning. The framework constructs question-specific sub-tables through operation-aware table decomposition and uses the predicted question type to determine the appropriate reasoning mode. We further introduce silver reference sub-tables for evaluating evidence selection and construct SLQA, a benchmark based on real-world long tables. Experiments on WikiTQ and SLQA show that localization is particularly effective for lookup and local reasoning questions, while adaptive selection between localized and full-table reasoning achieves the best overall performance. These results highlight that long-table QA requires deciding not only how to localize, but also when to localize. Our code and datasets will be made available upon publication of the paper.

1 Introduction

TableQA accuracy becomes more sensitive to irrelevant content as tables grow, but the value of localization depends on the question. TABSCOPE therefore treats scope as a question-adaptive decision, combining localized reasoning with full-table reasoning when broader context is needed.

  • Motivation: LLMs struggle to identify the rows and columns needed for answers as tables grow longer, reducing TableQA accuracy.The problem is attributed to noisy inputs and the difficulty of isolating the small supporting evidence set.
  • Motivation: Localization helps when answers depend on compact regions, whereas broad comparisons, complete aggregations, or information spread across many rows may require the full table.Decomposition can remove necessary context or introduce retrieval errors for questions requiring broader evidence.
  • Approach: TABSCOPE first decides whether a question is better served by the full table or a compact sub-table.Its localized path uses operation-aware decomposition and refinement; otherwise, the model reasons directly over the full table.
  • Contributions: The framework constructs silver reference sub-tables to evaluate decomposition quality and introduces SLQA for long-table question answering.These contributions address missing evidence annotations and limited long-table coverage in existing benchmarks.
  • Contributions: The paper explicitly analyzes when to localize and selects between localized and full-table reasoning rather than applying fixed decomposition.This reframes TableQA scope selection as part of question-dependent reasoning.

2 Related Work

Prior TableQA work localizes evidence through semantic retrieval or operation-guided decomposition, but benchmark limitations hinder direct evidence evaluation and long-table assessment. TABSCOPE is situated within these efforts through its refined sub-table construction and long-table benchmark.

  • Evidence localization and retrieval: Evidence-localization methods select smaller sub-tables or retrieve relevant schema and cell information before reasoning.Examples include LLM-based table and question decomposition, hybrid symbolic–textual extraction, and retrieval-augmented table reasoning.
  • Program-guided table decomposition: Program-guided methods obtain intermediate tables through explicit operations, iterative transformations, critique, refinement, or executable tools.Chain-of-Table, Table-Critic, ReAcTable, and TabSQLify represent this direction.
  • Evidence supervision and long-table evaluation: Existing benchmarks often provide final answers without annotations for supporting rows and columns, limiting direct evaluation of intermediate sub-tables.This limitation is stated for WikiTableQuestions and related benchmarks.
  • Evidence supervision and long-table evaluation: Many commonly used benchmarks contain relatively small tables, making them less suited to evaluating long-table reasoning.The cited discussion notes that such tables are typically under 4K table tokens.
  • Evidence supervision and long-table evaluation: SLQA addresses long-table evaluation by using real-world long tables, while silver reference sub-tables support decomposition-quality evaluation.The benchmark and supervision resources target the two identified gaps.

3 Methodology

TABSCOPE selects localized or full-table reasoning per question, then decomposes localized inputs into operation-aware sub-tables and refines them before answering.

  • Question-Adaptive Scope Selection: TABSCOPE predicts question type and applies a fixed policy to choose localized or full-table reasoning, accounting for table-size regime.At inference, the LLM predicts the question type while the policy determines the final scope.
  • Operation-Aware Table Decomposition: When localization is selected, an operation-aware decomposer retrieves question-relevant rows and columns through retrieval, aggregation, and refinement.It targets the reasoning operation rather than relying only on lexical overlap.
  • Evidence Aggregation: TABSCOPE aggregates K = 4 retrieval outputs, merging identical normalized row or column selections and weighting groups by frequency or candidate reliability.When generation confidence is unavailable, weighting reduces to frequency-based voting.
  • Candidate Scoring: Candidate sub-tables combine progressively unioned row and column groups, then maximize support while penalizing retained-table size.The compactness coefficient α ∈ [0.1, 0.3] controls the penalty on large sub-tables.
  • Evaluation Resources: The evaluation resources include WikiTQ, WTQ-SUBTAB with silver sub-tables for 4,344 WikiTQ test questions, and SLQA tables exceeding 4,096 serialized tokens.These resources support evaluation of scope selection, decomposition quality, and long-table reasoning.
  • Answer Generation: After scope selection, the answer model uses the refined sub-table for localized reasoning or the original table for full-table reasoning.The refinement verifier can add missing rows, columns, or both before answer generation.

4 Experiments

Experiments evaluate TABSCOPE on standard and long-table benchmarks, evidence-selection quality, and the effects of operation-aware decomposition and adaptive scope selection. Across datasets and models, adaptive reasoning achieves the strongest answer accuracy, while localization benefits depend on question type and table size.

  • Experimental Setup: TABSCOPE is evaluated on WikiTQ, SLQA, and WTQ-SUBTAB, using answer accuracy and row-, column-, and cell-level evidence metrics.SLQA targets tables exceeding 4,096 tokens, while WTQ-SUBTAB provides silver sub-tables for decomposition evaluation.
  • Operation-aware decomposition: 6.1 and 4.5 points are the WikiTQ gains of decomposition-only TABSCOPE over the strongest baseline with GPT-5-mini and LLaMA-3.3-70B, respectively.On SLQA, the corresponding gains are 9.2 and 4.7 points.
  • Evidence selection: 2.70, 4.32, and 1.02 points are TABSCOPE’s gains over DATER in cell, row, and column F1, respectively, on WTQ-SUBTAB.Under exact match, the corresponding gains are 4.40, 9.24, and 1.20 points; TABSCOPE is best across all six metrics.
  • Localization on long tables: 67.1% is decomposition-only accuracy on SLQA, nearly matching full-table CoT at 67.3% while removing substantial table content.SLQA contains only tables exceeding 4,096 tokens.
  • Question-type analysis: Localization helps lookup, ranking, extrema, nearby-record reasoning, and filtered-count differences, whereas full-table reasoning remains stronger for broad counting and multi-entity comparison.Frequency counting favors full-table reasoning on WikiTQ but shifts toward localization on SLQA, showing that question type and table size interact.
  • Question-adaptive scope selection: 0.9 and 4.3 points are the WikiTQ gains from evidence scoping over always applying decomposition with GPT-5-mini and LLaMA-3.3-70B, respectively.The corresponding SLQA gains are 1.1 and 0.9 points; the complete framework achieves the highest accuracy across both datasets and answer models.
  • Question-adaptive scope selection: 1.2 and 2.4 points are TABSCOPE’s WikiTQ gains over the strongest full-table baseline with GPT-5-mini and LLaMA-3.3-70B, respectively.On SLQA, the gains are 1.3 and 0.3 points.
  • Ablation analysis: 3.1 and 2.0 points are the WikiTQ and SLQA accuracy decreases after removing operation-aware retrieval.This is the largest ablation effect among the decomposition components; removing scope selection decreases accuracy by 4.3 points on WikiTQ and 0.9 points on SLQA.

5 Conclusion

The conclusion presents TABSCOPE as a question-dependent scope decision that combines adaptive selection with operation-aware decomposition. Results indicate that it improves TableQA accuracy while producing sub-tables closer to the evidence needed for answering.

  • 5 Conclusion: TABSCOPE selects table scope per question instead of treating decomposition as a fixed preprocessing step.It combines question-adaptive scope selection with operation-aware decomposition.
  • 5 Conclusion: The framework reduces distracting content when localization helps and preserves the full table when broader coverage is required.Experiments on WikiTQ and SLQA report improved answer accuracy across moderate- and large-scale tables.
  • 5 Conclusion: WTQ-SUBTAB results show that TABSCOPE’s decomposed tables more closely match the rows, columns, and cells needed for answering.The conclusion frames effective TableQA as deciding not only how to process a table, but also how much of it to retain.

6 Limitations

The evaluation is limited to English TableQA benchmarks and two representative LLMs. Broader testing is identified as future work, alongside more flexible scope-selection routing.

  • Evaluation covers English TableQA benchmarks and two representative LLMs.WikiTQ and SLQA differ in table scale, but the study does not assess additional domains, languages, or model families.
  • Generalization across additional domains, languages, and model families remains unassessed.
  • The current scope-selection policy relies on a predefined question-type taxonomy.Future work could explore more flexible routing strategies.

A Question Type-wise Scope Selection Analysis

The analysis derives a question-type-based policy for choosing localized or full-table reasoning. Type-wise comparisons motivate the policy, while refinement checks and expands selected evidence when needed.

  • Scope preferences by question type: Localized reasoning performs better for lookup, order/superlative, local reasoning, and count-difference questions.These question types typically use evidence concentrated within a restricted table region.
  • Scope preferences by question type: Full-table reasoning is stronger for general counting, frequency counting, and comparison questions.Questions requiring broad comparison sets, complete aggregation domains, or information distributed across many rows may need the full table.
  • Large-table analysis: The overall type-wise preference remains stable on large tables, except that localization becomes more effective for frequency counting.The large-table comparison reports only question types with sufficient examples.
  • Scope-selection policy: The policy maps predicted question type and table-size regime to the final reasoning scope rather than directly predicting localization.Default preferences are used generally, with large-table preferences substituted when sufficient validation examples exist.
  • Evidence selection and refinement: An operation-aware decomposer selects rows and columns, then one refinement round retains or expands the sub-table before final reasoning.The verifier may add missing rows, columns, or both when the current selection lacks sufficient information.

C Detailed Experimental Settings

The experiments use two LLM families with separate serving setups and fixed decoding budgets. Retrieval sampling is stochastic, whereas other components use deterministic decoding.

  • Open-source models run locally with swift.llm and HuggingFace weights, while GPT-5-mini uses the OpenAI Chat Completions API.
  • QA decoding allows 2048 new tokens; retrieval uses temperature 0.5, while other components use temperature 0.

D.1 Construction of SLQA

SLQA is constructed from real-world long tables with self-adaptive question generation and manual quality control. It retains both executable and flexible non-executable reasoning cases.

  • Source tables: SLQA retains Spider tables whose serialized representations exceed 4,096 tokens and generates new direct TableQA questions.Original Spider questions are excluded because they target NL-to-SQL evaluation and executable logical forms.
  • Question generation: Self-adaptive generation lets the model identify an answer from an evidence scope before formulating the question.This replaces forcing a predetermined answer, a procedure whose manual acceptance rate was below 50%.
  • Reasoning coverage: The benchmark includes SQL-executable operations and non-SQL-executable questions over semi-structured table content.This covers filtering, comparison, counting, aggregation, and more flexible reasoning.
  • Quality control: Every generated pair undergoes manual review for answerability, table grounding, and correctness.More than 70% are accepted without modification, while fewer than 30% require manual correction; unreliably repairable pairs are discarded.
  • Evidence-scope variants: Generation uses cell-, row-, column-, and sub-table-based evidence scopes to create diverse question–answer pairs.The selected evidence defines the information from which each question and answer is generated.

D.2 Selection of the Silver Construction Strategy

The study compares three strategies for constructing silver sub-tables from WikiTableQuestions examples, using manually annotated rows and columns as reference. Direct generation with verifier-guided refinement is selected because it improves evidence-selection quality across all reported metrics.

  • The evaluation samples 150 WikiTableQuestions QA pairs and manually annotates the gold rows and columns required for each question.These annotations provide the reference for comparing silver sub-table construction strategies.
  • Schema-based construction retrieves table regions from question-related schema elements and values, whereas direct generation predicts the target sub-table from the question, answer, and original table.A refined direct-generation variant adds verification to repair missing, invalid, or redundant selections.
  • Verifier-guided refinement raises cell F1 from 71.58% to 74.67% and column F1 from 84.55% to 90.15%.The passage reports increases across all six metrics relative to direct generation without refinement.
  • The authors use direct generation with refinement to construct WTQ-SUBTAB.

D.3 Construction of WTQ-SUBTAB

WTQ-SUBTAB is constructed to evaluate intermediate evidence decomposition for TableQA. Its procedure predicts reasoning type, evidence scope, rows, and columns, then verifies and repairs the selected sub-table before retaining it for offline evaluation.

  • WTQ-SUBTAB enables direct evaluation of intermediate sub-table decomposition, which standard question–table–answer datasets do not annotate.The construction addresses the absence of explicit supporting-row and supporting-column annotations.
  • For each WikiTQ instance, an LLM predicts an evidence specification containing reasoning type, evidence scope, selected rows, and selected columns.Reasoning types include lookup, filtering, comparison, superlative or ordinal selection, neighborhood reasoning, counting, aggregation, and Boolean reasoning.
  • The evidence policy determines whether to retain answer-bearing evidence, a full comparison domain, all aggregation instances, or neighboring context.The silver sub-table is extracted from the original table using the selected rows and columns, grounding evidence in the source table.
  • A verifier checks whether selected rows and columns are valid, sufficient, compact, and consistent with the inferred reasoning type and evidence policy.
  • Verifier feedback can add missing rows or columns, remove redundant evidence, or revise the reasoning type and evidence policy; only verified constructions are retained.
  • WTQ-SUBTAB evaluates row selection, column selection, and cell-level evidence coverage, while reference answers and silver sub-tables remain limited to offline construction and evaluation.They are not provided to TABSCOPE during inference.
Loading 2609.03395v1…