PaperlayerView exact revision on arXiv

Source-linked AI summary

RECAST: Learning to Compute the Right Context through Adaptive Evidence Routing

Yilun Hao, Krishna Sayana, Isabella Ye, James S Ren, Sukhdeep Sodhi, Craig Boutilier, Chuchu Fan

arXiv:2610.10507v1cs.AI

TL;DR

Some tasks require deriving evidence from heterogeneous sources rather than retrieving a single relevant item, a need that retrieval-centric approaches address only partly. RECAST uses a learned RouterLM to adaptively select retrieval and computation operations, including synthesized operations, before passing sufficient evidence to an answer model. It achieves 75.6% average success across six in-domain benchmarks and 79.3% across three held-out benchmarks, outperforming the strongest baselines by 15.9% and 15.0%, respectively.

  • Problem

    Retrieval-centric approaches offer limited support when task-relevant evidence must be derived through computation across heterogeneous source items.

    Source: §1 · p. 1

  • Method

    RECAST trains a RouterLM to iteratively formulate retrieval or computation operations, including synthesized operations, assess evidence, and pass sufficient evidence to a frozen AnswerLM.

    Source: §1 · p. 2

  • Results

    75.6% average success across six in-domain benchmarks and 79.3% across three held-out benchmarks, exceeding the strongest baselines by 15.9% and 15.0%, respectively.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    RECAST generalizes across held-out tasks and heterogeneous source representations, and retains strong performance with smaller code-generation models.

    Source: §5 · p. 11

    Training-data construction adds computational overhead, while iterative inference and occasional code synthesis may increase latency and computational cost relative to single-pass retrieval.

    Source: §5 · p. 11

Abstract

from arXiv · show

Large language models are increasingly applied to tasks grounded in long, heterogeneous information sources. Conventional Retrieval-Augmented Generation (RAG) relies on fixed similarity-based retrieval, while agentic variants adapt queries and tool use but remain largely retrieval-centric. However, in many tasks, the evidence required for a solution is not explicitly present in any single source item. Instead, it must be derived through filtering, aggregation, or computation across multiple source items. In this work, we introduce RECAST (Routing Evidence through Computation, Access, and Synthesized Tools), a learned framework that formulates evidence construction as a sequential decision process over heterogeneous retrieval and computation operations, allowing evidence to be actively derived rather than merely retrieved. A lightweight RouterLM iteratively selects and formulates primitive operations or specifies customized operations for a frozen CompilerLM to translate into executable code. Once it judges the evidence sufficient, RouterLM passes the accepted evidence to a frozen AnswerLM to produce the final solution. We train RouterLM with supervised fine-tuning (SFT) followed by group relative policy optimization (GRPO). Across six heterogeneous benchmark families, RECAST achieves a mean success rate of 75.6%, outperforming the strongest large-model baseline by 15.9%. Moreover, training enables the Qwen3.5-9B RouterLM to outperform a training-free Gemini 3.5 Flash RouterLM by 5.0%. On three held-out benchmarks, RECAST improves over the strongest baseline by 15.0% on average, demonstrating strong zero-shot generalization across tasks and heterogeneous source representations.

1. Introduction

RECAST addresses tasks where useful evidence must be derived from heterogeneous sources, rather than simply retrieved. It adaptively combines retrieval and computation, achieving strong results across in-domain and held-out benchmarks.

1. Introduction

  • Conventional and agentic retrieval methods primarily locate or refine source content, but some tasks require evidence derived through filtering, aggregation, or computation.For example, identifying the three-month period with the largest operating-margin increase requires calculating margins and comparing changes across periods.Source: §1 · pp. 1–2
  • RECAST treats evidence construction as a learned, multi-round process in which RouterLM selects or formulates retrieval and computation operations and decides when evidence is sufficient.Customized operations are translated into executable code by a frozen CompilerLM; accepted evidence is passed to a frozen AnswerLM.Source: §1 · p. 2, Figure 1 · p. 2
  • 75.6% average success across six in-domain benchmarks and 79.3% across three held-out benchmarks, exceeding the strongest baselines by 15.9% and 15.0%, respectively.The held-out comparison is 79.3% versus 64.3% for the strongest baseline.Source: §1 · pp. 2–3
  • The framework broadens evidence construction by combining retrieval and computation for heterogeneous sources, and improves RouterLM's policy through SFT followed by GRPO.The trained Qwen3.5-9B RouterLM also outperforms a training-free Gemini 3.5 Flash RouterLM.Source: §1 · pp. 2–3

2. Related Works

Prior work adapts retrieval and adds tools or executable programs, but generally selects existing source content, targets specific source types, or generates answers directly. RECAST instead adaptively constructs task-relevant evidence across heterogeneous sources.

2. Related Works

  • Adaptive retrieval methods refine search across multiple rounds, but still primarily select or refine information already present in the source.RECAST extends adaptive context construction to relational computation and synthesized transformations that derive evidence.Source: §2 · p. 3
  • Structured-source, tool-use, and program-synthesis methods add computational capabilities, but often target particular source types or generate task answers directly.RECAST iteratively evaluates primitive or synthesized operations to construct evidence across heterogeneous sources before final solution generation.Source: §2 · p. 3

3. RECAST

RECAST frames evidence construction as an adaptive process over heterogeneous sources, combining retrieval, computation, and task-specific synthesized operations. A learned RouterLM iterates through evidence operations and training stages to decide when the evidence is ready for final inference.

3.1. Problem Formulation and Challenges

  • The evidence-construction process maps a task and heterogeneous source collection to compact, query-specific evidence that may be retrieved or derived across source items.Derivations include filtering, aggregation, arithmetic, and linking.Source: §3.1 · p. 4
  • Different tasks require lexical, semantic, relational, or customized operations, and each operation must be formulated with task-specific inputs and evidence-return requirements.Multiple rounds may be needed to handle incomplete evidence or runtime failures, with stopping dependent on whether retained evidence supports final inference.Source: §3.1 · p. 4

3.2. RECAST Overview

  • RouterLM alone is trained to select, formulate, and assess operations, while frozen CompilerLM translates customized specifications into code and frozen AnswerLM generates the final solution.A rule-based preprocessor converts sources into records and a compact profile; RouterLM then loops through operation formulation, execution, feedback, and evidence assessment until accepting the evidence.Source: §3.2 · pp. 4–5

3.3.1. CALL_PRIMITIVE

  • CALL_PRIMITIVE applies lexical, semantic, or relational operations, returning retrieved content or computed results with source identifiers and execution feedback.Successful execution does not itself establish that the evidence is sufficient for the task.Source: §3.3.1 · p. 5

3.3.2. SYNTHESIZE

  • When available primitives cannot express the needed transformation, RouterLM specifies a synthesized operation and CompilerLM translates it into executable Python.The specification preserves task constraints and identifies the transformation and evidence to return; the program can process the full source or use earlier evidence.Source: §3.3.2 · pp. 5–6, Figure 2 · p. 6

3.3.3. ACCEPT_CONTEXT

  • After each evidence-producing action, RouterLM uses results and execution feedback to decide whether to continue or accept evidence for AnswerLM.Accept_Context ends evidence construction and passes the constructed evidence to AnswerLM for solution generation.Source: §3.2 · p. 5, §3.3.3 · p. 6

3.4. RECAST Training

  • Training updates RouterLM in two stages: SFT establishes valid multi-round behavior, then GRPO improves routing using downstream outcomes from groups containing both successful and unsuccessful trajectories.The reward combines judged correctness, token-level answer F1, and structural validity, with weights of 0.90, 0.08, and 0.02.Source: §3.4 · pp. 6–7

4. Experimental Results

RECAST is evaluated across heterogeneous in-domain benchmarks, ablations, held-out tasks, and alternative compilers. It achieves the strongest reported overall success rates, while the ablations and transfer evaluations probe the contributions and scope of its evidence-routing approach.

4.1. Experimental Setup

  • The in-domain evaluation covers six source representations and compares RECAST with direct answering, fixed retrieval, code generation, IRCoT, and Interact-RAG.Each benchmark family has 100 test questions; methods use the same questions, source contexts, AnswerLM, and judge, with in-domain results averaged across three runs.Source: §4.1 · pp. 7–8

4.2. RECAST Performance

  • 75.6% average success rate, a 15.9% improvement over Gemini-based Interact-RAG (59.7%), on six in-domain benchmark families.Table 1 reports success rates across the six benchmark families.Source: Table 1 · p. 8, §4.2 · p. 8

4.3. Ablation Studies

  • 75.6% with SFT+GRPO exceeds 68.9% with answer-correctness-only reward and 73.0% without mixed-outcome filtering.The filtered GRPO pool is smaller, and its higher performance indicates improved sample efficiency.Source: Table 2 · p. 9, §4.3 · p. 10
  • 75.6% with both training stages exceeds 72.6% with GRPO alone and 67.3% with SFT alone, from 64.1% training-free.SFT establishes valid operation selection and formulation; GRPO optimizes routing trajectories using downstream outcomes.Source: Table 2 · p. 9, §4.3 · p. 10
  • 64.1% training-free average with both operation types exceeds primitive-only RECAST (56.8%) and synthesis-only RECAST (59.9%).Neither operation type is uniformly superior across benchmark families; effectiveness depends on task evidence requirements.Source: Table 2 · p. 9, §4.3 · pp. 9–10

4.4.1. Generalization to unseen benchmarks

  • 79.3% average success rate on three held-out benchmarks exceeds the strongest baseline (64.3%) by 15.0%.Improvements span Wikipedia passage collections, financial text-and-table reports, and semi-structured Wikipedia tables.Source: §4.4.1 · p. 10, Table 3 · p. 11

4.4.2. Transfer across CompilerLMs

  • With Gemini 3.5 Flash Lite as CompilerLM, RECAST reaches 73.2% in-domain and 79.3% held-out; with Qwen3.5-9B, it reaches 67.2% and 73.3%.The learned routing policy transfers to a lighter compiler and across model families and scales.Source: §4.4.2 · p. 10, Table 4 · p. 11

5. Conclusion

RECAST adaptively derives task-relevant evidence through retrieval and computation, with a trained RouterLM directing operations and assessing evidence sufficiency. The paper reports performance across in-domain and held-out benchmarks, while noting added data-collection and inference costs from its iterative process.

5. Conclusion

  • RECAST broadens context construction from retrieving source content to adaptively deriving task-relevant evidence through retrieval and computation.A RouterLM formulates primitive operations or requests customized code, assesses outcomes, and determines when evidence is sufficient.Source: §5 · p. 11

AI use statement

  • The authors used generative AI for writing support, literature discovery, and research execution, and report reviewing and testing AI-assisted code.They state that authors verified references and take responsibility for the paper's content.Source: AI use statement · p. 11

Ethics statement

  • Experiments use public benchmarks and collect no new human-subject data; code execution is isolated and relational queries are read-only.For private or sensitive data, the paper recommends sandboxing, access controls, and privacy safeguards.Source: Ethics statement · pp. 11–12

Reproducibility statement

  • The paper describes its framework, training, dataset construction, hyperparameters, reward definition, implementations, prompts, and three-run standard deviations for reproducibility.The authors state that code, data, and the trained RouterLM checkpoint will be released under an open-source license upon acceptance.Source: Reproducibility statement · p. 12

A.1. Benchmark Descriptions

The benchmarks span heterogeneous source formats and evidence tasks, from dataframe and financial reasoning to multi-hop question answering and personalized language modeling. Their examples illustrate how questions can require retrieval, linking, aggregation, or numerical reasoning.

A.1. Benchmark Descriptions

  • DataBench covers scalar or list answers from real-world dataframes, while FinQA requires numerical reasoning over financial-report prose and tables.DataBench questions analyze provided tables; FinQA may require identifying information in either source and performing arithmetic.Source: §A.1 · p. 17
  • HiTab tests reasoning over tables with hierarchical headers, while MultiHiertt requires numerical reasoning across multiple financial tables or text and tables.The tasks include lookup, comparison, aggregation, and numerical reasoning that depends on interpreting table structure.Source: §A.1 · p. 17
  • HotpotQA and 2WikiMultiHopQA use Wikipedia passages for multi-hop questions that require reasoning across documents.2WikiMultiHopQA questions may use compositional, comparison, or inference relations.Source: §A.1 · p. 17
  • LaMP pairs a current input with a user’s historical profile items for personalized language modeling across six included tasks.The family covers citation identification, tagging, rating, and text generation tasks; LaMP-6 is excluded because it uses the restricted Avocado email corpus.Source: §A.1 · p. 17
  • TAT-QA combines financial-report prose and tables for span-based, counting, and arithmetic questions, while WikiTableQuestions tests compositional questions over semi-structured tables.WikiTableQuestions may require selecting, comparing, or aggregating values from multiple entries.Source: §A.1 · p. 17

A.2. Benchmark Examples

  • The benchmark examples span text passages, flat and hierarchical tables, financial reports, dataframes, and user profiles, with evidence tasks ranging from retrieval and multi-hop linking to aggregation and personalized generation.The examples use shortened, representative source snippets because complete sources may contain many records.Source: §A.2 · p. 18

B.1. RECAST Training Details

RECAST training uses execution-tested trajectories to create SFT and GRPO datasets, then optimizes the RouterLM with correctness-dominant trajectory rewards.

B.1. RECAST Training Details

  • 11,563 candidate trajectories were collected from 4,660 questions using an adaptive process that added attempts only after earlier attempts failed.Collection used a pretrained Qwen3.5-9B RouterLM, BGE-M3 for semantic retrieval, and Gemini 3.5 Flash as CompilerLM, AnswerLM, and judge.Source: §B.1 · p. 21
  • 3,064 questions had expert trajectories; after reserving 306 for validation, 2,758 questions formed the SFT set, yielding 3,368 trajectories after capped upsampling.Selected trajectories were correct, executable, structurally valid, and ranked by answer agreement, token F1, and execution and token cost.Source: §B.1 · p. 21
  • GRPO uses 1,200 questions with both valid expert and failed attempts, retaining mixed-outcome groups to compare successful and unsuccessful routing trajectories.After validation and benchmark-family balancing, its training set contains 608 prompt rows from 567 unique questions.Source: §B.1 · p. 21
  • SFT supervises RouterLM decisions across multi-round trajectories, and GRPO continues fine-tuning from the selected SFT checkpoint.SFT uses LoRA for two epochs; GRPO samples four routing trajectories per prompt and trains for three epochs.Source: §B.1 · pp. 21–22
  • The trajectory reward makes final-answer correctness dominant: every correct trajectory scores above every incorrect one, while token-level F1 and structural validity provide bounded secondary signals to distinguish trajectories sharing a correctness outcome.With the accuracy-only reward R(tau)=C(tau), all trajectories with the same correctness outcome receive identical advantages, so GRPO cannot distinguish a near-miss answer from a malformed or failed trajectory.Source: §B.1 · p. 22

B.2. RECAST Main-Result Performance with Standard Deviations

Across repeated evaluations, in-domain success-rate variability is low, while learned evidence-operation use varies by benchmark.

B.2. RECAST Main-Result Performance with Standard Deviations

  • 0.3% to 0.8% is the overall standard-deviation range for success rates across three repeated evaluations.The evaluations use the same 600-question test set; the corresponding mean results appear in Table 1.Source: Table 6 · p. 23, §B.2 · p. 23

B.3. Evidence-Construction Behavior

  • 27.3% of questions invoke synthesis overall, with usage varying by benchmark and routing length ranging from 2.16 rounds on DataBench to 4.22 on FinQA.DataBench predominantly uses synthesized computation, HotpotQA uses primitives in this run, and FinQA and MultiHiertt combine primitive use with synthesis on some questions.Source: Table 7 · p. 24, §B.3 · p. 24

B.4. Robustness to Evaluation Choices

RECAST’s in-domain advantage persists under alternative judges and a lexical-overlap metric. Judge agreement is high, although alternative judges assign equal or lower success rates than Gemini.

B.4. Robustness to Evaluation Choices

  • 74.8% under GPT-4.1 and 72.8% under DeepSeek-V4-Flash, with advantages of 15.8% and 15.3% over the strongest baseline, respectively.RECAST remains the strongest method overall under both alternative evaluators.Source: Table 8 · p. 25, §B.4 · p. 25, Table 9 · p. 26
  • 57.3% average token-level F1 for trained RECAST compares with 45.3% for the strongest baseline.The authors report F1 as supplementary because lexical overlap can penalize semantically equivalent answers.Source: §B.4 · p. 26
  • 95.2% to 98.7% is the Gemini–GPT-4.1 raw-agreement range, and 95.3% to 98.8% is the Gemini–DeepSeek range across methods.For trained RECAST, agreement is 96.7% with GPT-4.1 and 95.3% with DeepSeek; Cohen’s κ ranges are 0.90–0.96 and 0.88–0.98, respectively.Source: §B.4 · p. 25, Table 10 · p. 27, Table 11 · p. 27

C.1. Baselines

C.1 defines direct, retrieval, code-generation, and iterative retrieval baselines; C.2 describes RECAST variants that vary training, evidence operations, and compiler models. Table 12 reports in-domain token-level F1 across six benchmark families.

C.1. Baselines

  • Direct answers without source access, while Fixed Retrieval passes the three highest-ranked chunks from one BGE-M3 call to AnswerLM.Fixed Retrieval does not reformulate queries or retrieve again.Source: §C.1 · p. 27
  • Direct Code generates and executes one program; IRCoT and Interact-RAG instead iteratively retrieve evidence before shared AnswerLM generation.IRCoT alternates retrieval with sentence-level reasoning; Interact-RAG refines retrieval requests through planning, reasoning, and execution.Source: §C.1 · pp. 27–28
  • Table 12 reports in-domain token-level F1 across six benchmark families, with bold marking the best result in each column.Source: Table 12 · p. 28

C.2. RECAST Variants

  • Training-free RECAST lets Qwen3.5-9B or Gemini 3.5 Flash route among primitives, synthesized code, and evidence assessment over multiple rounds.The remaining framework components are held identical to those used by the trained model.Source: §C.2 · p. 28
  • Primitive Only removes synthesis, while Synthesis Only removes primitive calls, isolating the two evidence-operation families.Source: §C.2 · p. 29
  • Training and data variants compare SFT alone, GRPO without SFT initialization, binary correctness rewards, and unfiltered GRPO questions.The unfiltered variant uses all 1,200 GRPO candidate questions; Accuracy Only retains SFT initialization and mixed-outcome data.Source: §C.2 · p. 29
  • Compiler variants replace Gemini 3.5 Flash with Gemini 3.5 Flash Lite or Qwen3.5-9B only when the router selects synthesis.Source: §C.2 · p. 29

D. Cost Analysis

Trained RECAST achieves higher success than Gemini-based Interact-RAG while using fewer tokens, and it also improves on training-free RECAST with a Gemini RouterLM while reducing token usage.

  • Token totals include input and output across routing, code generation, and final answering, but exclude the evaluation judge.The table separates Qwen3.5-9B and Gemini 3.5 Flash tokens.Source: D. Cost Analysis · p. 30, Table 13 · p. 30
  • 75.6% versus 59.7% success; trained RECAST uses 27.9% fewer total tokens and 79.5% fewer Gemini tokens than Gemini-based Interact-RAG.Table 13 reports average task success and mean inference token usage per question.Source: Table 13 · p. 30, D. Cost Analysis · p. 30
  • Against training-free RECAST with a Gemini RouterLM, success rises from 70.6% to 75.6% while total token usage falls from 31.8k to 19.4k.Source: D. Cost Analysis · p. 30

E. Failure Case Analysis

The failure analysis examines unsuccessful source-grounded trajectories, excluding Direct from its method-specific discussion, and the prompts specify the roles of the router, compiler, answer model, and correctness judge.

E.1. Baselines

  • Fixed Retrieval cannot revise its single similarity search when initial results reveal a missing cross-source connection or need for more context.Source: §E.1 · p. 31
  • IRCoT trajectories can repeat earlier reasoning while retrieval adds little; Interact-RAG trajectories show early stopping or repeated refinement with limited progress.Neither Interact-RAG pattern alone establishes the cause of an error.Source: §E.1 · p. 31
  • Direct Code may print records without computing an answer, produce invalid or failing programs, and cannot revise a failed program using execution feedback.The inspected programs particularly include Gemini 3.5 Flash cases that inspect rather than compute and Qwen3.5-9B cases truncated after extended comments.Source: §E.1 · p. 31

E.2. RECAST

  • RECAST failures include relevant evidence that misses a required relation or constraint, and incorrect interpretation of the requested computation despite successful execution.The paper identifies more precise evidence requests and sufficiency assessment as promising directions for policy improvement.Source: §E.2 · p. 31

F.1. RouterLM

  • RouterLM selects one action per round, using primitives or requesting synthesized code, then accepts context when it judges the evidence sufficient.Its prompt defines evidence-operation requirements and interfaces for lexical, semantic, relational, and code-based operations.Source: §F.1 · p. 32

F.2. CompilerLM

  • CompilerLM translates the router’s synthesis specification into a temporary evidence program without receiving the original task, and must implement the specification literally.Its inputs include source profile, samples, accumulated evidence, and recent operation outcomes.Source: §F.2 · pp. 33–34

F.3. AnswerLM

  • AnswerLM uses the original task and accepted evidence to answer, returning “I don’t know” when that evidence is insufficient or contradictory.Source: §F.3 · p. 34

F.4. Correctness Judge

  • The shared correctness judge compares candidate and reference answers, accepting semantic equivalents but rejecting omitted requirements or material false claims.The judge does not participate in evidence construction.Source: §F.4 · p. 35
Loading 2610.10507v1…