PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Stepwise Model Routing: A Cost-Efficient Table Reasoning Perspective

Shenghao Ye, Yuxiang Wang, Yu Guo, Dong Jin, Shuangwu Chen, Jian Yang

arXiv:2605.29319v1cs.CL

TL;DR

Large reasoning models make table reasoning costly, while stepwise routing for tables remains underexplored. EcoTab routes steps using separate table-token and text-token uncertainty estimates mapped to next-step failure risks. Across table reasoning benchmarks, it outperforms strong baselines and achieves a better accuracy–efficiency balance.

  • Problem

    Large reasoning models incur substantial inference cost on table reasoning, and the effectiveness of stepwise routing methods on structured table tasks remains underexplored.

    Source: §1 · p. 1

  • Method

    EcoTab separately estimates table-token and text-token uncertainties, maps them to small-model next-step failure risks, and combines the risks to route each step.

    Source: §1 · p. 2

  • Results

    Across five table reasoning benchmarks and both collaboration settings, EcoTab consistently outperforms all baselines and achieves the best accuracy–efficiency trade-off.

    Source: §5.2 · pp. 6–7

  • Takeaways & Limitations

    Separating table-token and text-token signals is more effective for table reasoning than directly applying free-form text routing methods.

    Source: §5.2 · p. 7

    EcoTab chooses the model for the next reasoning step but does not control when reasoning stops, so unnecessary long traces may remain.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large Reasoning Models (LRMs) achieve strong performance on table reasoning tasks but incur substantial inference cost due to long reasoning traces. Stepwise model routing mitigates this issue by dynamically assigning reasoning steps to smaller or larger models. However, stepwise model routing for table reasoning remains underexplored. Through empirical analysis, we find that reasoning steps involving tables contain two types of tokens with distinct uncertainty distributions: table tokens grounded in table structure, such as cell values and headers, and text tokens representing surrounding natural-language reasoning. The uncertainty of both token types is correlated with the risk that the model makes an error in the next reasoning step. However, existing methods fail to model them separately, leading to suboptimal routing decisions. To address this, we propose EcoTab, a table-aware stepwise routing framework for efficient table reasoning. At each reasoning step, EcoTab separately estimates the uncertainties of table tokens and text tokens, maps them to next-step failure risks for the small model, and combines the two risks for routing. Experiments on multiple table reasoning benchmarks show that EcoTab consistently outperforms strong baselines and achieves a better balance between accuracy and efficiency.

1 Introduction

Table reasoning is costly for large reasoning models, while stepwise routing for tables remains underexplored. EcoTab separately estimates table-token and text-token uncertainty to route reasoning steps, and the paper reports a better accuracy–efficiency balance than strong baselines.

1 Introduction

  • Large reasoning models use long reasoning chains that bring high computational overhead and complicate deployment in latency-sensitive, resource-constrained settings.Table reasoning is challenging because tables have complex structures and implicit relations across rows and columns.Source: §1 · p. 1
  • Stepwise routing assigns simpler steps to smaller, cheaper models and more complex steps to larger models, but its effectiveness on structured table reasoning remains underexplored.Prior routing methods perform well on free-form text reasoning tasks.Source: §1 · p. 1
  • Table reasoning steps contain table tokens grounded in table structure and surrounding text tokens, whose uncertainties have distinct distributions and correlate with the small model’s next-step failure risk.Existing methods do not jointly model the two token types, and the paper identifies this as a source of poor routing performance on table reasoning.Source: §1 · pp. 1–2
  • EcoTab separately estimates table-token and text-token uncertainty, maps each to a small-model next-step failure risk, then combines the risks to route the next step.The routing decision selects a small or large model.Source: §1 · p. 2
  • EcoTab consistently outperforms strong baselines on multiple table reasoning benchmarks and achieves a better balance between accuracy and efficiency.The paper presents this as its central experimental result.Source: §1 · p. 2

2 Preliminary

The preliminary setup represents LRM reasoning as a sequence of steps and describes stepwise routing as choosing a model for each next step. In this work, routing uses the previously generated step as its information source, avoiding extra token generation.

2 Preliminary

  • An LRM generates a sequence of reasoning steps for a table and natural-language query, with each step containing tokens.The trace is segmented into steps using a newline delimiter.Source: §2 · p. 2
  • Stepwise routing dynamically selects the small or large reasoning model to generate the next step from the current reasoning prefix.The routing function uses the previous step as routing information in this work, avoiding additional token generation.Source: §2 · p. 2

3 Motivation

Existing text-oriented routing methods use accuracy and FLOPs to assess efficiency, but transfer less effectively to table reasoning. The analysis links this gap to table-specific routing errors and finds that both table-token and text-token uncertainty inform next-step failure risk.

3 Motivation

  • Existing routing methods reach near-LRM performance on free-form text benchmarks at about 60% of full FLOPs, but need nearly 80% on table benchmarks for similar performance.The evaluation uses WikiTQ and TableBench for table reasoning, and MATH500 and AIME24 for free-form text reasoning.Source: §3 · p. 3, Figure 3 · p. 3
  • Average entropy separates correct from incorrect free-text steps by 0.14, but table-specific steps by only 0.06, while table-token and text-token entropy distributions differ.The figure contrasts the entropy gap by step type and the token-type distributions within table-specific steps.Source: §3 · p. 3, Figure 3 · p. 3
  • Errors occur in mixed high–low entropy groups as well as the high–high group, indicating that either token type’s high uncertainty can be associated with failure risk.The four groups are based on whether table-token and text-token entropy is above or below the 70th-percentile threshold.Source: Figure 4 · p. 3, §3 · p. 4
  • Table-token and text-token uncertainty distributions differ, and both are informative of the small model’s next-step failure risk.This motivates modeling both types rather than relying on a single step-level entropy score.Source: §3 · pp. 3–4

4 EcoTab

EcoTab separates table and text tokens, estimates their uncertainties, maps them to failure risks, and combines those risks into a score for choosing the model for the next reasoning step.

4 EcoTab

  • EcoTab uses a Table Trie to separate table and text tokens, estimates each group’s uncertainty, maps both to failure risks, and combines them into a next-step model-selection score.Source: §4 · p. 4

4.1 Table Trie Construction

  • A word-level Table Trie stores normalized headers and cell values; longest-prefix matching marks spans and maps them to token positions for a boolean mask.Source: §4.1 · p. 4

4.2 Step-Level Uncertainty Estimation

  • EcoTab computes Shannon-entropy uncertainty per token, then averages it separately over table-token and text-token sets for each reasoning step.Source: §4.2 · pp. 4–5

4.3 Routing Score via Noisy-OR

  • Two validation-fitted sigmoid mappings convert token-group uncertainties to next-step SRM failure risks, which Noisy-OR combines into a thresholded routing score.Source: §4.3 · p. 5

4.4 Overall Pipeline

  • The SRM generates the first step; EcoTab may regenerate it with the LRM above threshold, then routes each next step until completion or the iteration limit.Source: §4.4 · p. 5

5 Experiments

The experiments evaluate EcoTab across five table-reasoning benchmarks and same-family and cross-family model settings. They report its overall routing performance, test component contributions and robustness, and examine threshold behavior and routing latency.

5.1 Experimental Setup

  • The evaluation uses Qwen3-1.7B as the SRM, two 14B LRMs, five table-reasoning benchmarks, and accuracy–FLOPs trade-off measures.The reported operating points are accuracy at 60% of LRM-only FLOPs and FLOPs needed to reach 98% of LRM-only accuracy; A/F is also reported.Source: §5.1 · pp. 5–6

5.2 Main Results

  • EcoTab achieves the best overall accuracy, FLOPs, and Accuracy-per-FLOPs across five benchmarks in both same-family and cross-family settings.It also achieves the highest accuracy on all five benchmarks with either evaluated LRM while maintaining the lowest overall inference cost, without extra token generation during routing.Source: §5.2 · pp. 6–7, Table 1 · p. 6

5.3 Ablation Study

  • Removing token separation causes the largest performance drop and increases cost; using either token type alone or linear weighting also hurts accuracy and efficiency.The full EcoTab performs best under both evaluated settings on TableBench.Source: Table 2 · p. 6, §5.3 · p. 7

5.4 Robustness Analysis

  • WikiTQ-fitted risk mapping transfers to TableBench: OOD beats Random and STEER on both reported metrics, while ID performs slightly better.The reported metrics are accuracy at 60% of LRM-only FLOPs and FLOPs required to reach 98% of LRM-only accuracy.Source: §5.4 · p. 7, Figure 6 · p. 7

5.5 Further Analysis

  • Higher routing thresholds reduce FLOPs, and EcoTab stays above STEER and Random on WikiTQ and TableBench; its TableBench routing latency is 48.6 seconds.EcoTab’s latency is comparable to STEER at 52.1 seconds and far below RSD, SpecCoT, and SpecReason.Source: §5.4 · p. 7, §5.5 · pp. 7–8, Figure 7 · p. 8

6 Related Work

Prior table-reasoning work spans specialized encoders, LLM prompting and fine-tuning, and reinforcement learning. Efficient reasoning research includes several routing levels, but existing step-level methods do not account for tables’ structured nature.

Table Reasoning

  • Table-reasoning approaches progressed from Transformer-based encoders such as TAPAS and TaBERT to LLM prompting, supervised fine-tuning, and reinforcement learning for reasoning trajectories.The cited works include TAPAS [Herzig et al., 2020], TaBERT [Yin et al., 2020], and the Table-R1 series [Yang et al., 2025b; Wu et al., 2025b; Jin et al., 2025].Source: §6 · p. 8

Efficient Reasoning

  • Efficient reasoning methods offload work at query, step, or token level; step-level approaches include reward guidance, multi-path selection, post-hoc verification, and uncertainty-based routing.Examples include RSD, SpecCoT, SpecReason, entropy-based routing, and confidence-based routing.Source: §6 · p. 8
  • Existing step-level routing methods are mainly designed for free-form text and do not account for the structured nature of table reasoning.Table reasoning requires natural-language understanding and structured reasoning over rows, columns, and cell values.Source: §6 · p. 8

7 Conclusion

The paper studies stepwise model routing for table reasoning and introduces EcoTab, which distinguishes table and text tokens when estimating step difficulty. Across multiple benchmarks, it reports a better accuracy–cost trade-off than existing stepwise routing methods.

7 Conclusion

  • The paper studies efficient stepwise model routing for table reasoning with LRMs.Source: §7 · p. 8
  • EcoTab distinguishes table tokens from text tokens when estimating step difficulty and uses probabilistic fusion to guide model selection.Source: §7 · p. 8
  • Across multiple table-reasoning benchmarks, EcoTab consistently achieves a better trade-off between reasoning accuracy and computational cost than existing stepwise routing methods.Source: §7 · p. 9

Limitations

EcoTab shares a limitation with existing routing methods: it cannot explicitly control reasoning output length.

Limitations

  • EcoTab does not explicitly control the output length of the reasoning process.Source: Limitations · p. 9
  • Routing determines which model generates the next step, but does not determine when reasoning stops.Source: Limitations · p. 9
  • Unnecessarily long reasoning traces may still occur even when routing is accurate.Source: Limitations · p. 9

Ethics Statement

The appendix documents the ethics stance, model configurations, benchmark coverage, and evaluation protocol used for EcoTab. The evaluation uses held-out calibration data and reports accuracy–FLOPs trade-offs.

Ethics Statement

  • The authors caution that intermediate reasoning and final answers may be incorrect, advising verification of critical outputs; they also report following the licenses and usage policies of their open-source components.The named components include Qwen3, DeepSeek-R1-Distill, PyTorch, and Hugging Face.Source: Ethics Statement · p. 9

A.1 Model Configurations

  • Qwen3-1.7B serves as the SRM, paired with either Qwen3-14B or DeepSeek-R1-Distill-Qwen-14B as the LRM.The pairings test same-family and cross-family collaboration; all methods use temperature 0.7, maximum generation length 16,384, and top-p 0.95.Source: §A.1 · p. 11

A.2 Dataset Details

  • The five-benchmark evaluation spans complex table QA, fact verification, hierarchical tables, and financial reasoning that integrates tables with unstructured text.The supplied dataset descriptions include TableBench, WikiTQ, TabFact, HiTab, and FinQA.Source: §A.2 · pp. 11–12

A.3 Implementation Details

  • Thresholds are swept in 0.05 increments; main results report accuracy at 60% of LRM-only FLOPs and FLOPs to reach 98% of LRM-only accuracy.FLOPs use the 2N-per-generated-token approximation, and results are averaged over three independent runs.Source: §A.3 · p. 12

A.4 Four Error Types for Failed Steps

The appendix categorizes failed reasoning steps and details how EcoTab identifies table-related tokens through trie matching. The matching procedure normalizes text, favors the longest matching span, and maps matches back to generated-token positions.

A.4 Four Error Types for Failed Steps

  • Table retrieval errors account for 47.7% of failed steps, while table operation errors account for 34.3%.Retrieval failures include mis-selection, unit mismatch, and partial aggregation; operation failures include miscalculation, grouping mistakes, double counting, and semantic misinterpretation.Source: §A.4 · pp. 12–13
  • Inner-thinking errors account for 12.0% of failed steps, while other failures include context omission, incomplete responses, and improper formatting.The appendix also presents representative retrieval, operation, and inner-thinking cases, showing how an early error can propagate to the final answer.Source: §A.4 · p. 13, Table 3 · p. 14

B Table Trie Construction Details

  • EcoTab builds a word-level trie from normalized table headers and cell values, then uses its matches to construct a table-token mask.Matched spans are mapped back to token positions in the original reasoning step.Source: B Table Trie Construction Details · p. 13

B.1 Normalization Rules

  • Normalization lowercases text, cleans whitespace and punctuation, and standardizes common number formats to reduce surface-form mismatches.For example, “1,200” and “1200” are treated as the same value, while semantically meaningful decimals and percentages are preserved.Source: §B.1 · p. 13
  • Both headers and cell values enter the trie as complete word-level paths, while matching normalization is mapped back to original token positions.This keeps the final boolean mask defined over the generated tokens.Source: §B.1 · p. 13

B.2 Trie Matching Algorithm

  • The matcher scans left to right, marks spans using the longest valid trie match, and advances one token when no match is found.The resulting mask partitions a step into table-token and text-token sets; the authors report the surface-form matching sufficient in practice.Source: §B.2 · p. 13

C.1 Construction of Fitting Data

EcoTab’s offline risk mappings are fitted with step-level supervision derived from validation trajectories. The appendix describes the suffix-replacement procedure and discusses its supervision quality, inference overhead, and cross-domain transfer.

C.1 Construction of Fitting Data

  • Suffix replacement identifies the shortest SRM-regenerated suffix that reliably flips an LRM-correct trajectory to an incorrect answer, locating a critical routing boundary.For each tested suffix length, generation is repeated five times; a stable flip requires at least four incorrect outcomes.Source: §C.1 · p. 14
  • The boundary step is labeled positive, earlier steps negative, and later steps discarded; samples without a stable flip are excluded from fitting.This yields step-level supervision aligned with next-step routing, and both uncertainty mappings share the retained steps and binary labels.Source: §C.1 · p. 14

C.2 Fitting Failure-risk Mappings

  • EcoTab fits independent sigmoid mappings for table-token and text-token uncertainty, then combines their mapped risks with Noisy-OR for threshold-based next-step routing.The mappings are fitted offline using retained validation samples and binary cross-entropy.Source: §C.2 · pp. 14–15

C.3 Discussion

  • Suffix replacement provides cleaner supervision without human annotation, and offline fitting adds no token-generation overhead during inference.The WikiTQ-fitted mapping also outperforms Random and STEER on TableBench for both reported metrics, with only a small gap from the in-domain setting.Source: §C.3 · p. 15

D Case Study

The case studies compare EcoTab with SRM-only and STEER on table retrieval and table operation examples. The reported examples show EcoTab routing critical steps to the LRM and reaching correct final predictions.

D Case Study

  • Across retrieval cases, SRM-only and STEER fail while EcoTab succeeds; the operation cases show the same qualitative pattern.The appendix attributes the correct predictions to EcoTab more accurately identifying the critical reasoning step and routing it to the LRM.Source: D Case Study · p. 15, Figure 10 · p. 15, Figure 11 · p. 16
  • In the retrieval example, EcoTab counts six Electrodyne wins and answers 6.The question asks how many times Electrodyne won; the worked reasoning identifies six GTO entries for Electrodyne.Source: D Case Study · p. 15
  • In the date-order example, the reasoning concludes that the dates are not in consecutive order.The worked answer is “no” after listing the dates chronologically.Source: D Case Study · p. 16

Related papers

Loading 2605.29319v1…