PaperlayerView exact revision on arXiv

Source-linked AI summary

TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoning

Jiaru Zou, Soumya Roy, Vinay Kumar Verma, Ziyi Wang, David Wipf, Pan Lu, Sumit Negi, James Zou, Jingrui He

arXiv:2510.06217v1cs.AIcs.CLcs.LG

TL;DR

Step-level PRM supervision for tabular reasoning remains underexplored, despite PRMs’ use in test-time scaling. TATTOO combines table-grounded, tool-based verification with a two-stage training approach; across five benchmarks, it improves downstream policy models and outperforms strong PRM baselines. Its results underscore the importance of table-grounded reward supervision for structured reasoning.

  • Problem

    A step-level PRM for supervising LRMs’ tabular reasoning remains notably absent, while existing PRMs struggle with table retrieval and schema interaction.

    Source: §1 · pp. 1–2

  • Method

    TATTOO provides table-grounded rewards for table operations and inner-reasoning rewards for text steps, integrating tools and training through supervised fine-tuning followed by reinforcement learning with reward shaping.

    Source: §1 · p. 2

  • Results

    Across five benchmarks, 8B TATTOO improves downstream policy models by 30.9% and outperforms Qwen-2.5-Math-PRM-72B and GenPRM-32B with up to 9x parameter efficiency.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results underscore the importance of table-grounded reward supervision and show generalization across multiple test-time scaling strategies.

    Source: §7 · p. 10

    Automated verification may amplify errors in tools or training data rather than correct them.

    Source: G LIMITATIONS AND BROADER IMPACTS · p. 24

Abstract

from arXiv · show

Process Reward Models (PRMs) have recently emerged as a powerful framework for enhancing the reasoning capabilities of large reasoning models (LRMs), particularly in the context of test-time scaling (TTS). However, their potential for supervising LRMs on tabular reasoning domains remains underexplored. Through detailed empirical analyses, we identify that existing PRMs, though widely adopted for supervising text-only reasoning steps, struggle with table-specific operations such as sub-table retrieval and schema interaction, leading to critical performance bottlenecks. To address this limitation, we propose TaTToo, a novel table-grounded PRM framework that (i) reasons explicitly over tabular reasoning steps and (ii) integrates tool-based verification to provide precise reward supervision. Concretely, we first design a scalable data curation pipeline that constructs over 60k high-quality step-level annotations by integrating table verification rationales with tool-based executions. Building on the collected data, we train TaTToo with a dual-stage paradigm: cold-start supervised fine-tuning to capture tool-use reasoning patterns, followed by reinforcement learning with tool-grounded reward shaping to align our model with table-based verification. We provide a comprehensive evaluation of the policy improvement induced by our newly designed PRM. Across 5 challenging tabular reasoning benchmarks covering numerical reasoning, fact-checking, and data analysis, TaTToo improves downstream policy LRMs by 30.9% at inference, surpasses strong PRM baselines such as Qwen-2.5-Math-PRM-72B with only 8B parameters, and demonstrates strong generalizability across diverse TTS strategies.

1 INTRODUCTION

Existing PRMs lack reliable step-level supervision for tabular reasoning, particularly for table retrieval and schema interaction. TATTOO combines table-grounded verification with tool use and improves downstream policy performance across five benchmarks.

1 INTRODUCTION

  • Although PRMs provide step-level supervision for test-time scaling, a corresponding verifier for tabular reasoning remains notably absent.The paper frames the central question as how to provide reliable step-level supervision to advanced LRMs in tabular reasoning.Source: §1 · p. 1
  • Existing PRMs struggle to verify table retrieval and schema interaction, and errors in their table lookups or operations add supervision noise during test-time scaling.Schema interaction failures include missing attention collapse when models overlook long-range table dependencies.Source: §1 · p. 2
  • TATTOO provides table-grounded rewards for tabular operations, inner-reasoning rewards for text-based steps, and tool-integrated verification.Its construction uses over 60k supervision instances integrating expert verification rationales with tool executions, followed by supervised fine-tuning and reinforcement learning with reward shaping.Source: §1 · p. 2
  • Across five benchmarks, 8B TATTOO improves downstream policy models by 30.9% and outperforms Qwen-2.5-Math-PRM-72B and GenPRM-32B with up to 9x parameter efficiency.The dual-stage training paradigm yields a 10.2% improvement over standard PRM training, and TATTOO generalizes across Beam Search and DVTS.Source: §1 · p. 2

2 PRELIMINARY

The paper formulates table reasoning as a trajectory of inner-thinking steps and tool-integrated table operations. A process reward model scores each step, aggregates the scores, and supplies rewards to test-time compute strategies.

2 PRELIMINARY

  • A table is represented by headers defining schema-level semantics and rows of entries aligned with those headers.The reasoning trajectory includes intermediate reasoning steps and a final answer, with intermediate steps comprising both inner-thinking traces and programs operating directly on the table.Source: §2 · p. 2
  • A process reward model assigns each reasoning step a reward conditioned on the table, query, and preceding trajectory, then aggregates step rewards into a trajectory reward.The aggregation function can use MEAN or SUM.Source: §2 · p. 2
  • Test-time compute strategies use PRM rewards to guide resampling, refinement, and candidate selection among policy-model responses.Examples include Best-of-N and Beam Search.Source: §2 · p. 2

3 WHY TABLE REASONING REQUIRES VERIFIERS BEYOND CURRENT PRMS?

In pilot evaluations, general-domain PRMs help at small test-time sampling sizes but soon reach a performance bottleneck. Error analyses trace key weaknesses to table retrieval and schema interaction, while a table prefix improves supervision but brings its own requirements.

3 WHY TABLE REASONING REQUIRES VERIFIERS BEYOND CURRENT PRMS?

  • For N ≥ 8, accuracy across three TableBench tasks converges to a bottleneck; on fact-checking, Qwen2.5-Math-PRM-72B reaches 79.19%, 79.82%, and 79.84% at N = 8, 16, and 32.At small N, step-level verifiers generally improve on single-shot generation, with PRMs providing the largest gains.Source: §3 · p. 3
  • Among 500 incorrect PRM-selected responses, errors cluster in Table Retrieval (47.7%) and Schema Interaction (34.3%).The analysis groups errors into four reasoning-step categories and contrasts table-specific operations with inner reasoning.Source: §3 · p. 3
  • Existing PRMs give nearly identical rewards to real and randomly replaced sub-tables, indicating they fail to distinguish retrieval correctness.Table 1 presents representative errors, descriptions, and PRM judgments.Source: §3 · p. 4, Table 1 · p. 4
  • Schema interaction steps attend mainly to nearby steps, often misinterpreting or discarding earlier retrieved content; PRMs miss these failures because their evaluations are localized.The attention comparison is between schema interaction at step 8 and table retrieval at step 0.Source: §3 · p. 4
  • Prepending retrieved table contents to schema interaction steps improves PRM supervision and downstream performance, but PRMs must identify those steps and the prefixes must be correct.The prefix provides direct access to retrieval context rather than requiring long-range dependencies.Source: §3 · p. 5
  • The analyses motivate a verifier that supervises both table-grounded operations and models’ inner reasoning.The proposed TATTOO is designed specifically for step-level supervision in tabular reasoning.Source: §3 · p. 5

4 BUILDING A TABLE-GROUNDED STEP VERIFIER

TATTOO separates supervision for table operations from inner reasoning and incorporates external tools into verification. Its curation pipeline generates trajectories, assigns step-level rewards, and synthesizes tool-use rationales.

4 BUILDING A TABLE-GROUNDED STEP VERIFIER

  • TATTOO provides step-level supervision for both table operations and model inner-thinking steps.The framework combines large-scale data curation with dual-stage training for tool-use verification.Source: §4 · p. 5

4.1 TABLE-AWARE AND TOOL-INTEGRATED SUPERVISION

  • Table-aware rewards distinguish table-grounded operation accuracy from inner-reasoning correctness, alongside a trajectory-level reward.External table-oriented tools support more reliable step supervision.Source: §4.1 · p. 5

4.2 TATTOO DATA CURATION PIPLINE

  • Curation begins with expert-LRM trajectories from table-based questions, filtered through human and expert-LLM verification.The resulting high-quality trajectory pool is used for subsequent labeling.Source: §4.2 · p. 5
  • For table steps, curation assigns rewards based on retrieval relevance or operation correctness, using retrieved sub-tables as prefixes for schema interaction.These labels complement the separate inner-reasoning rewards described in the following curation step.Source: §4.2 · p. 5

TATTOO DATA CURATION

TATTOO’s pipeline produces tool-augmented verification data and trains the PRM with supervised fine-tuning followed by tool-grounded reinforcement learning. The paper’s analysis states that decomposed rewards can contribute additively to policy improvement when each component aligns with policy advantage.

TATTOO DATA CURATION

  • Curation labels table operations and inner-thinking separately, adds tool calls, execution results, and feedback, and yields over 60k training instances.Table tools cover computation and lookup; inner-thinking steps receive rewards based on reasoning quality.Source: TATTOO DATA CURATION · p. 6

4.3 TOOL-GROUNDED DUAL-STAGE TRAINING

  • Training first uses supervised fine-tuning for tool-integrated verification patterns, then RL with dense per-step, tool-grounded rewards.The RL reward combines label matching, confidence calibration, and whether the rationale correctly incorporates tool outputs.Source: §4.3 · pp. 6–7

4.4 INFERENCE-TIME POLICY IMPROVEMENT – AN INTUITIVE VIEW

  • The analysis states that decomposed reward components can contribute additively to policy improvement when each component aligns with policy advantage.The components separately target table-involved operations and inner reasoning.Source: §4.4 · p. 7

5 EMPIRICAL EVALUATIONS

Across five tabular reasoning tasks, TATTOO improves Best-of-N verification performance and generalizes to Beam Search and DVTS. Analyses find gains from RL after SFT and identify tool-grounded reward shaping as a key component.

5 EMPIRICAL EVALUATIONS

  • The evaluation uses DeepSeek-R1-Distill-Qwen-14B across five tabular reasoning tasks and tests Best-of-N, Beam Search, and DVTS.TATTOO is trained from Qwen-3-8B using 60k curated instances.Source: §5 · pp. 7–8

5.1 MAIN RESULTS

  • TATTOO consistently outperforms strong PRM baselines on five Best-of-N tasks with 8B parameters, and continues improving as N grows.On TB-NR it rises from 74.2% at N=8 to 78.1% at N=32, while Qwen2.5-Math-PRM-72B moves from 74.9% to 75.3% between N=16 and N=32; on TB-DA, TATTOO reaches 34.3% at N=32.Source: §5.1 · p. 8, Table 2 · p. 8
  • Across Beam Search and DVTS, TATTOO's average accuracy improves steadily as N increases, while other PRM baselines often plateau.In Beam Search, TATTOO improves from 45.0% to 54.8%, GenPRM saturates around 51%, and Skywork-PRM remains below 46%.Source: §5.1 · pp. 8–9, Figure 5 · p. 8

5.2 IN-DEPTH ANALYSES ON TATTOO

  • RL after SFT raises average accuracy across three tasks from 72.3% to 78.5%, with the paper reporting a total gain of 10.2%.The authors describe SFT as a solid initialization and RL as further improving reasoning and tool-use effectiveness during verification.Source: §5.2 · p. 9, Table 3 · p. 9
  • Removing tool-grounding causes the largest reward-shaping drop, including a 4.0% decrease on TB-DA at N=32; excluding confidence calibration reduces average performance by 1.6%.The original rule-based GRPO group-relative reward yields only marginal improvement over SFT.Source: §5.2 · p. 9, Table 3 · p. 9

6 RELATED WORKS

Prior tabular reasoning research has developed encoder, prompting, fine-tuning, and RL approaches. Robust, verifiable reward supervision for complex table-specific reasoning trajectories remains comparatively unexplored.

6 RELATED WORKS

  • Tabular reasoning methods progressed from table encoders such as TAPAS and TaBERT to LLM prompting, supervised fine-tuning, and RL-based reasoning approaches.The cited RL methods include the Table-R1 series and Reasoning-Table.Source: §6 · pp. 9–10
  • Prior tabular work spans table QA, fact verification, text-to-SQL, and multi-hop reasoning, with newer benchmarks extending toward multi-table tasks.The passage cites WTQ and TabFact as early evaluation paradigms and discusses MultiTableQA, MT-RAIG, and MMQA as more recent work.Source: §6 · pp. 9–10
  • How to provide robust, verifiable reward supervision for lengthy table-specific reasoning trajectories remains largely unexplored.This gap motivates the paper's tool-use and thinking PRM for test-time scaling on tabular reasoning tasks.Source: §6 · p. 10

7 CONCLUSION

The paper concludes that TATTOO provides tool-augmented, table-grounded reward supervision and reports strong benchmark performance and generalization across TTS strategies. Its related-work discussion situates this contribution alongside PRM and tool-integration research.

7 CONCLUSION

  • TATTOO combines expert rationales, table prefixes, tool-augmented verification, and SFT followed by RL with reward shaping for tabular reasoning.The paper reports comparable performance across five table benchmarks, strong-PRM comparisons at up to 9× parameter efficiency, and generalization across multiple TTS strategies.Source: §7 · p. 10

A ADDITIONAL RELATED WORK

  • Table QA research has moved from Wikipedia-derived QA pairs and structured logical-expression benchmarks toward multi-hop reasoning resources.The passage cites WTQ and TabFact [Pasupat and Liang, 2015a; Chen et al., 2019], WikiSQL and Spider [Zhong et al., 2017], and newer resources including MMQA.Source: A ADDITIONAL RELATED WORK · p. 17
  • Prior PRM research provides step-level feedback across domains and uses PRMs for training-time optimization and inference-time scaling, including beam search and Best-of-N [Lightman et al., 2024; Uesato et al., 2022; Zhang et al., 2024a].The related-work discussion distinguishes discriminative classification from generative rationale-and-judgment verification [Zhong et al., 2025].Source: A ADDITIONAL RELATED WORK · p. 17
  • Although prior work integrates tools into LLMs, tool executions within reward modeling for reliable verification remain underexplored.The passage frames TATTOO as a tool-augmented PRM using external executions to supervise reasoning trajectories.Source: A ADDITIONAL RELATED WORK · p. 17

B DETAILED ERROR ANALYSIS

The error analysis finds that table retrieval and schema interaction account for most categorized failures, while purely logical errors are less common. Errors include both basic selection and arithmetic mistakes and broader structural issues.

B DETAILED ERROR ANALYSIS

  • Table retrieval accounts for 47.7% and schema interaction 34.3% of errors, while inner-thinking errors account for 12.0%.The analysis categorizes 500 incorrect LRM responses into 13 predefined error types.Source: Figure 7 · p. 18, B DETAILED ERROR ANALYSIS · p. 18
  • The most frequent individual errors are unit mismatch (15.0%), row mis-selection (14.8%), and miscalculation (11.2%).Other common categories include column mis-selection (10.4%), partial aggregation (9.6%), and missing or incomplete context (7.6%).Source: B DETAILED ERROR ANALYSIS · p. 18
  • Retrieval errors include row or column mis-selection, unit mismatch, and partial aggregation; schema-interaction errors include miscalculation, grouping mistakes, double counting, and semantic misinterpretation.The remaining listed category covers context omission or improper output formatting.Source: B DETAILED ERROR ANALYSIS · p. 18
  • Most weaknesses lie in table-related operations rather than general logical reasoning, and PRMs must verify that retrieved sub-tables align with the query.Source: B DETAILED ERROR ANALYSIS · p. 18

C TATTOO DATA CURATION PIPLINE

TATTOO’s data pipeline builds tool-grounded step-verification examples, while the accompanying proof and inference setup describe the policy-improvement analysis and test-time search strategies.

C TATTOO DATA CURATION PIPLINE

  • Table retrieval and schema-interaction steps receive table-grounded correctness rewards, while other reasoning steps receive reasoning rewards.For schema interaction, the preceding retrieved sub-table is attached as context; calculations and lookups are replaced with tool calls and execution results.Source: C TATTOO DATA CURATION PIPLINE · p. 19
  • Roughly 60k training examples combine verification rationales with code snippets and real-time interpreter feedback.Trajectory responses are filtered through human and expert-LLM review before step-level labeling and tool-use synthesis.Source: C TATTOO DATA CURATION PIPLINE · p. 19

D PROOF OF THEOREM 4.1

  • The proof combines the performance-difference lemma with a natural-policy-gradient update to derive a variance–alignment lower bound.Under the reward design’s mutual exclusivity and per-state centering, the covariance term is eliminated.Source: D PROOF OF THEOREM 4.1 · pp. 20–21

E.1 POLICY MODEL CONFIGURATIONS

  • Inference evaluation uses Best-of-N, Beam Search, and Diverse Verifier Tree Search as test-time scaling strategies.The passages also describe majority voting and LLM-as-a-judge as response-selection or evaluation methods.Source: §E.1 · p. 22

E.2 DOWNSTREAM DATASETS

The downstream evaluation covers tabular benchmarks with varied reasoning demands, and TATTOO is trained in supervised fine-tuning followed by reinforcement learning.

E.2 DOWNSTREAM DATASETS

  • TableBench contains 886 annotated samples across fact-checking, numerical reasoning, and data analysis, with questions averaging 6.3 reasoning steps.It has 3,681 tables and emphasizes numerical reasoning, with over 65% of cells containing numerical values.Source: §E.2 · p. 22
  • WikiTableQuestions tests compositional reasoning and generalization to unseen schemas, whereas MMQA evaluates multi-table, multi-hop question answering.WikiTableQuestions separates training and test tables; MMQA provides 5,000 samples over interlinked tables with SQL queries, gold answers, and key relations.Source: §E.2 · p. 22

E.3 TATTOO TRAINING DETAILS

  • TATTOO is trained from Qwen-3-8B on the curated 60k dataset, using supervised fine-tuning followed by policy-optimization reinforcement learning.The trained TATTOO-8B serves as a verifier guiding the downstream LRM under different test-time scaling strategies.Source: §E.3 · p. 23

F.1 ABLATION STUDY ON TATTOO

On TB-NR, TATTOO scales with larger response pools; ablations, a training-stage case study, and limitations further characterize its behavior and scope.

F.1 ABLATION STUDY ON TATTOO

  • TableBench accuracy improves as λcal rises, peaking at 0.8–1.0, while λtool performs strongest around 1.0–1.3.The ablations vary each coefficient separately during RL and evaluate with N=32.Source: §F.1 · p. 23

F.2 PERFORMANCE GAIN OF TATTOO WITH INCREASING NUMBER OF RESPONSES

  • 78.3% at N=32 gives TATTOO the highest TB-NR Best-of-N performance, while its gap over GenPRM-32B grows from under 1% at N=2 to over 6% at N=32.The baselines described saturate or plateau as N increases, whereas TATTOO continues improving.Source: Figure 8 · p. 23, §F.2 · p. 23

G LIMITATIONS AND BROADER IMPACTS

  • RL adds training overhead that may hinder reproducibility and low-resource access, and the current framework is limited to text–table reasoning.The paper also flags the risk that errors in automated verification tools or training data may be amplified.Source: G LIMITATIONS AND BROADER IMPACTS · p. 24

(i) Before RL Training

  • Step 3 is incorrect because its final calculation result, 22,460, does not equal 16,920, reflecting an incorrect summary of overall carrying capacity; it receives a -1(incorrect) step reward.Source: (i) Before RL Training · p. 25

(ii) After RL Training

  • After RL training, Python computes a total of 22,460, matching the step’s result and earning a correct reward.The example uses a code-based summation over the table’s capacity column.Source: (ii) After RL Training · p. 25

F.3 CASE STUDY

  • RL training raises the tool-integration ratio by 26.3% across 500 sampled trajectories and shows more reliable calculation verification in the case study.The SFT-stage example makes a numerical error using inner reasoning; the RL-stage example uses Python code to verify the calculation.Source: §F.3 · p. 24, Figure 9 · p. 25
Loading 2510.06217v1…