PaperlayerView exact revision on arXiv

Source-linked AI summary

FORGE: Form-Optimal Routing of Grounded Evidence for Frozen LLM Agents

Xi Xiao, Yunbei Zhang, Chen Liu, Lin Zhao, Jialin Chen, Tianchen Zhao, Xiang Xu, Youngeun Kim, Tianyang Wang, Min Xu

arXiv:2609.34358v1cs.LGcs.CL

TL;DR

Frozen, closed-weight models cannot be adapted through host fine-tuning, while evidence choice and reasoning depth both affect answer quality and token cost. FORGE routes jointly over these choices using a cost-aware target and trains a lightweight router around the frozen host. Across five benchmarks, it improves macro F1 at lower cached token cost on both main hosts.

  • Problem

    Closed-weight frozen models make host-side fine-tuning infeasible, while fixed evidence and reasoning choices do not adapt to query-level variation.

    Source: §1 · p. 2

  • Method

    FORGE trains a lightweight router to select evidence form and thinking depth jointly per query under an entropy-regularized, cost-aware utility.

    Source: §1 · pp. 2–3

  • Results

    +2.4 macro F1 on Qwen3-8B and +2.9 on Mistral-7B versus Always-Raw, at 45% and 42% lower cached token cost, respectively.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Results across host families support joint evidence-form and thinking control as a practical adaptation mechanism with quality, token, and latency tradeoffs.

    Source: §5 · p. 10

    Evaluation is limited to knowledge-intensive QA and fact verification; reasoning-only benchmarks requiring a different action alphabet are not evaluated.

    Source: §F.1 · p. 33

Abstract

from arXiv · show

In agentic AI systems, frozen foundation models are increasingly deployed as closed-weight API endpoints, making downstream adaptation possible only through the inputs and inference procedures surrounding the model. As a result, for each input query, two coupled decisions largely determine both answer quality and token cost: what evidence to provide and how much reasoning budget to allocate. Fixed defaults along these axes are often suboptimal, misallocating support form or reasoning depth on roughly 80% of queries in our analysis. To address this challenge, we propose FORGE, a unified framework for adapting frozen models through per-query routing over a joint action space that spans both support form and thinking depth. Under an entropy-regularized, cost-aware utility objective, we derive a closed-form Boltzmann routing target and instantiate the policy as a lightweight 269K-parameter factorized router. The routing policy is trained around the frozen host, without any weight access, through a three-stage pipeline: offline arm enumeration, supervised Kullback-Leibler (KL) distillation from the Boltzmann target, and Group Relative Policy Optimization (GRPO) refinement with host feedback. Across 5 knowledge-intensive benchmarks and 8 frozen backbones ranging from 7B to 671B parameters, FORGE improves accuracy at 42-45% lower token cost on both main hosts, transfers zero-shot across hosts at lower token cost, and composes with intrinsic thinking budgets where available.

1 INTRODUCTION

The introduction asks how frozen, closed-weight models can be adapted when host fine-tuning is infeasible, and argues that evidence form and reasoning depth need per-query, joint decisions. FORGE combines these decisions under a cost-aware objective and reports improved accuracy with lower token cost across its main hosts.

1 INTRODUCTION

  • Fixed support and reasoning choices can misallocate resources: on Qwen3-8B HotpotQA, 7.3% of queries favor no retrieval, while 12% are harmed by long reasoning.The same probe found identical F1 for 79.4% under compressed summaries versus full top-k passages and for 60% under NOTHINK versus THINK-HIGH.Source: §1 · p. 2
  • The paper identifies three gaps: no joint per-query policy, no unified training use of offline labels and host feedback, and no closed-form target tied to cost-aware decisions.These gaps arise in adapting frozen hosts whose private weights and internal states are inaccessible.Source: §1 · p. 2
  • FORGE jointly selects evidence form and thinking depth per query using one utility that trades answer F1 against normalized input and output token costs.Its factorized router has 269K parameters, selects an action in one forward pass, and includes standard RAG, Adaptive-RAG, and tiered-memory routing as special cases.Source: §1 · pp. 2–3
  • FORGE trains around the frozen host through warm-start arm enumeration, KL distillation to a Boltzmann target, and GRPO refinement using host feedback.The target follows from entropy-regularized cost-aware utility maximization; the paper says this characterizes the target and initialization, not optimizer convergence.Source: §1 · p. 3
  • Across five knowledge-intensive benchmarks, FORGE Pareto-dominates Always-Raw on every tested Qwen3-8B and Mistral-7B cell, adding +2.4 and +2.9 macro F1 at 45% and 42% lower cached token cost, respectively.The evaluation spans eight frozen backbones from 7B to 671B parameters across three deployment regimes.Source: §1 · p. 3

2 PRELIMINARIES AND RELATED WORKS

This section positions FORGE as joint routing over evidence and thinking choices, then defines a per-query quality–cost objective and its entropy-regularized target. It specifies the action alphabet and qualifies the role of query-dependent utility maximizers.

2.1 RELATED WORKS

  • FORGE combines support-form selection with host-exposed thinking settings under one measured quality–cost utility, extending approaches that select only the host, retrieval, or reasoning effort.The related-work discussion includes partial support-form selection among the choices it builds on.Source: §2.1 · p. 3

2.2 ROUTING OBJECTIVE AND BOLTZMANN SOLUTION

  • The per-query utility is answer F1 minus weighted normalized input and output token costs, with the weights setting the quality–cost operating point.The entropy-regularized objective has a unique Boltzmann solution; its temperature controls concentration on utility-maximizing actions, and it supplies Stage 1 soft labels.Source: §2.2 · p. 4

2.3 THE ADAPTIVE-THINKING ACTION ALPHABET

  • The action alphabet pairs Direct, Summary, or Raw support with available thinking settings, yielding six actions on non-thinking hosts and twelve on thinking-capable hosts.Summary uses query-aware sentence-level compression of BM25 retrieval, while Raw uses full top-k BM25 passages; the section treats oracle rows as descriptive references, not utility or F1 upper bounds.Source: §2.3 · p. 4

3 METHOD

FORGE routes jointly over evidence form and reasoning depth before the frozen host answers, using their interdependence to evaluate candidate actions by F1 and token cost. Its router has a Full variant with host-probe features and a Lite variant without probes, and training progresses from offline evaluation to distillation and host-feedback refinement.

3 METHOD

  • Candidate actions’ F1 and input/output token costs define the utility and soft targets used to route jointly over evidence form and reasoning depth.Evidence quality changes the value of extra reasoning, while the reasoning setting changes what support is worth providing.Source: §3 · p. 4

3.1 ROUTER AND DEPLOYMENT FEATURES

  • The factorized router uses a support-form head and a support-conditioned reasoning head; Full uses four host calls for fresh-query features, while Lite needs only the final-answer call.Both variants use the same single-step action set and frozen retrieval pipeline; fresh-query cost includes all host calls.Source: §3.1 · pp. 4–5

3.2 TRAINING

  • Training evaluates warm-start actions on labeled queries, distills the Boltzmann target on those actions, then refines over the full action set with sampled host rewards and a KL anchor.Stage 2 uses 5,000 × 32 = 160K query groups with eight sampled actions each, totaling 1.28M host completions.Source: §3.2 · p. 5, Figure 3 · p. 5
  • Stage 2 uses PPO-clipped policy feedback, a KL anchor to Stage 1, and an entropy bonus; zero-variance groups receive zero advantage.The same-action-set ablation isolates refinement from action-alphabet expansion.Source: §3.2 · p. 5

4 EXPERIMENTS

Across five knowledge-intensive benchmarks, experiments compare FORGE with fixed and adaptive baselines, evaluate online costs and grounding, and test thinking-budget control and zero-shot transfer. The results show quality–cost gains alongside task- and deployment-specific trade-offs.

4.1 SETUP AND EVALUATION PROTOCOLS

  • Evaluation covers five QA and fact-verification benchmarks, six-arm policies on non-thinking hosts, and 12-arm policies on thinking-capable hosts, with cached and fresh-online costs reported.Fresh Online accounting includes host calls and input/output tokens; Full’s four feature calls are parallelized.Source: §4.1 · p. 5, Figure 4 · p. 6

4.2 ANSWER QUALITY AND THE COST OF A NEW QUERY

  • 59.5/50.1 macro F1 on Qwen/Mistral exceeds Always-Raw by 2.4/2.9 points with 45%/42% fewer cached answer tokens.Lite improves Qwen/Mistral F1 by 1.6/1.9 while saving 44%/40% of online tokens with roughly 5.5 ms added p50 latency.Source: Table 1 · p. 6, §4.2 · p. 6, Table 2 · p. 7

4.3 RELIABILITY, STRONG BASELINES, AND STAGE-2 VALUE

  • At matched actions and features, Stage 2 adds 1.8/1.7 macro F1 and saves 7.1%/8.1% cached tokens; feature ablations find the largest HotpotQA gain from embeddings and +2.3 F1 from the host probe.The 2,500-update point delivers about 70% of the final Stage 2 gain.Source: Table 3 · p. 7, §4.3 · p. 7, Table 4

4.4 GROUNDING, PROXY REWARDS, AND OPERATING POINTS

  • On HotpotQA, Full lowers gold evidence recall from 91.2 to 64.8 while source support rises from 61.9 to 65.3; frozen-judge training reaches 58.6 F1, within 0.9 F1 of gold-reward training.Cost weights were tuned on Qwen HotpotQA development data and held fixed; a development-set fallback selects FORGE for HotpotQA, 2Wiki, and MuSiQue, and Summary for PopQA and FEVER.Source: Table 5 · p. 8, §4.4 · p. 8, Table 6 · p. 8

4.5 THINKING CONTROLS AND TRANSFER

  • On thinking-capable hosts, the 12-arm policy adds 1.4–1.9 macro F1 over Raw+Think-High at roughly half its cached cost; the full policy reaches DKL = 0.02 from the Boltzmann target.Stage 1 reaches DKL = 0.49 and the policy without the KL anchor reaches 0.64; Direct+Think-High scores 38.5/47.3/48.4 F1 versus 63.4/69.6/69.5 for Raw+Think-High.Source: §4.5 · p. 8, Table 7 · p. 9, Figure 5 · p. 9

4.6 CROSS-HOST TRANSFER

  • A Qwen-trained Full router matches or exceeds Raw in 11/15 task–host cells, with macro F1 gains of +2.7/+0.0/+2.5 and 35.5%/38.7%/35.1% fewer cached tokens.The transfer evaluation uses no target-host Stage 2; cost excludes fresh probes.Source: Table 8 · p. 9, §4.6 · p. 9

5 CONCLUSION

FORGE jointly controls evidence form and reasoning depth around a frozen host, using a factorized router trained with a quality–cost target and host feedback. Across five tasks, it improves cached macro F1 over Raw on Qwen and Mistral while reducing tokens, with Lite and Full offering different latency and probe trade-offs.

5 CONCLUSION

  • FORGE jointly controls evidence form and reasoning depth around a frozen host.Its factorized router distills an entropy-regularized quality–cost target and refines decisions through sampled host feedback.Source: §5 · p. 10
  • Across five tasks, Full improves nine-run cached macro F1 over Raw by 2.4/2.9 points on Qwen/Mistral; matched controls attribute 1.8/1.7 points to Stage 2.Lite improves local-host F1 and token cost with one online call, while Full uses four parallel probes for further accuracy.Source: §5 · p. 10
  • FEVER favors Summary, while DeepSeek transfer primarily saves tokens; results indicate quality, token, and latency trade-offs across host families and deployment settings.Source: §5 · p. 10

Technical Appendices

The related work organizes adaptive inference around reasoning depth, model and retrieval routing, and support form; FORGE focuses on choosing support form and available thinking settings around a fixed host. The appendices specify the frozen-host setup and how retrieval and action choices are constructed.

A RELATED WORK

  • Prior work spans intrinsic thinking control and extrinsic model, retrieval, and support-form choices; FORGE studies joint support-form and thinking-setting decisions around a fixed host.The paper identifies single-step, pre-answer routing as its scope.Source: A RELATED WORK · p. 17, §A.1 · p. 17

A.1 INTRINSIC ADAPTIVE THINKING

  • Adaptive-thinking methods change how a reasoning-trained or user-controllable host executes, while FORGE routes among exposed thinking options without changing host weights.The evaluation covers single-step decisions.Source: §A.1 · p. 17

A.2 EXTRINSIC ADAPTIVITY: MODEL AND RETRIEVAL ROUTING

  • Model routers select among backbones, while adaptive-retrieval methods vary whether or how much to retrieve; these approaches generally leave the form of delivered support fixed.Prior model-routing work includes cost–quality routing, while retrieval methods include per-query choices such as no, single-step, or iterative retrieval.Source: §A.2 · p. 17

A.3 SUPPORT-FORM ROUTING WITH A FIXED HOST

  • Prior support-form work includes summary-to-raw memory routing and free-form advisor hints; FORGE adds a Direct option and a joint support-and-thinking choice.The paper distinguishes its contribution from inventing support-form routing itself.Source: §A.3 · p. 18

B IMPLEMENTATION DETAILS

  • The study evaluates eight frozen backbones across five benchmarks and 40 host–benchmark cells, with local hosts on HBM accelerators and API hosts on managed endpoints.The reported training setup uses one 8-GPU node for Stage 2 per Pareto operating point.Source: B IMPLEMENTATION DETAILS · p. 18

B.2 FROZEN HOST CONFIGURATION

  • The host pool is grouped by deployment mode and thinking capability; API hosts have no weight access, and cross-host API tables use cached-feature evaluation.Fresh Online API latency is measured only for Lite, not Full.Source: §B.2 · pp. 18–19

B.3 RETRIEVAL AND ACTION CONSTRUCTION

  • All actions share a fixed retrieval backend, isolating routing from retriever tuning; Direct, Summary, and Raw vary support, while thinking settings vary by host capability.Summary is deterministic BM25 sentence selection, and thinking-capable hosts expose Think-Low and Think-High budgets of 1,024 and 4,096 output tokens.Source: §B.3 · p. 20

B.4 ROUTER ARCHITECTURE AND THREE-STAGE TRAINING

The appendices detail FORGE’s factorized router, its staged training, and the feature distinction between Full and Lite. Full uses host-derived features and pre-routing probes; Lite is retrained without those features.

B.4 ROUTER ARCHITECTURE AND THREE-STAGE TRAINING

  • Training proceeds from warm-start arm enumeration to KL distillation against a Boltzmann target, then GRPO refinement with host feedback on the expanded action alphabet.The default warm start uses three actions, and the finite-capacity Stage-1 router only approximates the closed-form target.Source: §B.4 · p. 20
  • Full FORGE uses a factorized MLP, while Lite is retrained on 778 host-independent input dimensions; both select one joint action before the frozen host answers.The deployment schematic distinguishes Full’s four pre-routing host generations from Lite’s probe-free path.Source: §B.4 · p. 20, Figure S1 · p. 21
  • Stage 2 uses a group-relative clipped surrogate aligned with measured utility, not an exact gradient step toward or convergence guarantee for the Boltzmann target.The KL anchor and entropy bonus keep updates near the warm start and discourage premature collapse on the larger action alphabet.Source: §B.4 · p. 21

B.5 FEATURE EXTRACTION

  • Full’s 789 features include 11 host-dependent dimensions and require four pre-routing calls on a fresh query; Lite uses 778 host-independent features and no host probes.The host-independent feature set comprises embeddings, BM25 statistics, query–passage similarity, and query-structure features.Source: §B.5 · p. 21, Figure S1 · p. 21, Table S6 · p. 22
  • The BGE+BM25 baseline uses 773 host-independent dimensions and excludes query-structure features, host probes, and GRPO.It matches Stage-0 data, MLP capacity, optimizer, and development-set search budget.Source: §B.5 · p. 21

B.6 DATASET CONFIGURATION

The appendices distinguish canonical point estimates from a nine-run uncertainty analysis and document the assets and licenses used. The reported tables summarize split sizes and feature-and-training variants.

B.6 DATASET CONFIGURATION

  • The uncertainty analysis uses three independent train/dev/test splits and three router-training seeds per split on both main local hosts, distinct from canonical-split point estimates.Paired-query bootstrap intervals on the canonical test split are also distinct from nine-run intervals.Source: §B.6 · p. 22
  • Table S7 reports canonical cached F1 point estimates for matched feature and training variants, with BGE+BM25 as a reference and Lite and Full adding different feature sets.These point estimates should not be treated as the nine-run means.Source: Table S7 · p. 23

B.7 ASSET LICENSES AND ATTRIBUTION

  • The work uses existing datasets, frozen language models, and software under their original licenses, and does not redistribute datasets or model weights.Table S9 summarizes the assets and licenses.Source: §B.7 · p. 23, Table S9 · p. 23

B.8 COMPUTE SCALE

The appendices report the cost and measured effects of Stage 2, fresh-query inference overhead, and reproduction settings. They distinguish cached selected-answer accounting from fresh-query costs and note that API Full latency was not measured.

B.8 COMPUTE SCALE

  • 1.8 and 1.7 macro-F1 points are added by Stage 2 on Qwen3-8B and Mistral-7B, while cached selected-answer tokens fall by 7.1% and 8.1%, respectively.The matched comparison uses a six-action, nine-run setup; Stage 2 takes 8–12 hours on one 8-GPU node per operating point.Source: §B.8 · pp. 23–24
  • At 0, 2,500, and 5,000 updates, Qwen3-8B reaches 57.6, 58.9, and 59.4 macro-F1 with 0.255, 0.244, and 0.237k cached tokens per query.Mistral-7B reaches 48.3, 49.5, and 50.0 macro-F1 with 0.272, 0.259, and 0.250k tokens; the trajectory requires the one-time training expenditure.Source: §B.8 · p. 24

B.9 INFERENCE LATENCY

  • Full FORGE raises Qwen3-8B p50 latency from 122.5 to 271.4 ms relative to Always-Raw, while Lite has p50 near Always-Raw.Fresh Full queries require feature-acquisition host calls; Lite needs only the routed-answer call.Source: §B.9 · p. 24
  • The reported comparison is a token-saving trade-off with slightly lower F1 and higher latency, not a three-axis Pareto improvement; its latency is about 2.1 s.Fresh Online Full FORGE latency was not measured on API hosts.Source: §B.9 · p. 25

B.10 REPRODUCIBILITY

  • Reproduction of the reported pipeline requires model outputs, enumeration tables, and Stage-1 and Stage-2 checkpoints; Stage 2 also requires the host endpoints used for rollout.Given pre-enumerated arm outputs, Stage 1 completes on one CPU in under two minutes.Source: §B.10 · p. 25
  • The study fixes cost weights and Boltzmann temperature using Qwen3-8B HotpotQA development data, then holds them fixed across the original comparisons and target-host transfer.The appendix states that these values specify a study operating point, not a universal price for user value.Source: §B.10 · p. 25

C.1 STAGE-2 TRAINING AND KL-ANCHOR ABLATIONS

The controlled six-action comparison estimates the Stage-2 refinement gain separately from action-alphabet expansion. Table S12 compares refinement methods and KL-anchor settings, while the broader framework treats support-form routing as an extrinsic axis that can compose with host reasoning budgets.

C.1 STAGE-2 TRAINING AND KL-ANCHOR ABLATIONS

  • +1.8 and +1.7 macro-F1 on Qwen3-8B and Mistral-7B, respectively, are the matched six-action Stage-2 gains.The comparison uses 1.28M model completions per operating point; the original three-arm-to-six-arm differences also include action-alphabet expansion.Source: §C.1 · p. 25
  • GRPO and Dr. GRPO differ by at most 0.2 F1 in the displayed point estimates; both exceed DPO and RLOO, without establishing a statistical tie.The comparisons apply to this implementation and tested action alphabets.Source: §C.1 · p. 25
  • Support-form routing is an extrinsic axis, orthogonal to host-exposed reasoning budgets with which the framework composes where available.Model and retrieval routing are the other two extrinsic axes identified in Figure S2.Source: Figure S2 · p. 26
  • Table S12 compares six-arm refinement methods from a shared Stage-1 checkpoint and varies the GRPO KL anchor around its default adaptive β=0.05 setting.Its three-arm Stage-1 row is a shared pipeline reference, not a matched six-arm control.Source: Table S12 · p. 26

C.2 FACTORIZED VERSUS FLAT POLICY HEAD

The supplementary analyses compare factorized and flat policy heads, test sensitivity to cost weights and training size, and distinguish descriptive allocation accounting from a matched estimate of Stage 2. The findings are specific to the tested settings and do not establish a general scaling law or causal decomposition of allocation terms.

C.2 FACTORIZED VERSUS FLAT POLICY HEAD

  • The factorized head differs from the flat head by 0.6 F1 at K=6 and 1.6 F1 at K=12, with parameter counts of 269K versus 273K.The tested differences are consistent with a benefit from conditioning thinking depth on support form, but do not establish a general scaling law in K.Source: §C.2 · p. 26, Table S13 · p. 27
  • Across the displayed cost-weight grid, F1 spans 1.2 points for non-thinking Qwen3-8B and 2.5 points for Qwen3-8B-Thinking.Higher λin shifts policy toward cheaper input arms, while higher λout suppresses thinking-budget arms; the (0.1, 0.2) default was selected on held-out development data and held fixed for main results.Source: §C.2 · p. 26, Table S14 · p. 27
  • On the HotpotQA training-size sweep, Stage 1 empirically saturates around Nwarm=500, and Stage 1+2 exceeds Stage 1 by +2.1 F1 at the default Nwarm=1,500.The displayed Stage 1-to-Stage 1+2 differences range from +1.7 to +3.8 F1; no finite-sample theorem predicts the saturation threshold.Source: §C.2 · p. 27, Table S15 · p. 27
  • The single-benchmark sweep does not isolate GRPO from action-alphabet changes, and Stage 2 requires rollout calls but no additional warm-start data.The three-arm-to-six-arm comparison changes both action availability and optimization.Source: §C.2 · p. 27, §C.3 · p. 27

C.3 PER-ARM ALLOCATION CHANGES FROM THREE TO SIX ACTIONS

  • +1.8 [1.3, 2.3] and +1.7 [1.2, 2.2] macro-F1 on Qwen3-8B and Mistral-7B, respectively, are the matched six-action Stage-2 increments.These paired 95% confidence intervals come from the canonical test split; the original three-arm-to-six-arm differences also change action availability and are not GRPO-only effects.Source: §C.3 · p. 27
  • The largest descriptive allocation term is +2.9 for CoT-Prompt, an option unavailable to the three-arm warm-start policy.Direct and Summary shifts contribute +1.3 combined and Raw contributes −0.7; these terms do not partition GRPO’s effect from adding CoT-Prompt.Source: §C.3 · p. 27, Table S16 · p. 28

C.4 ROUTER BEHAVIOR IN BGE EMBEDDING SPACE

FORGE’s routing is structured across query regions in BGE embedding space and closely matches a per-instance utility oracle. The accompanying analyses describe transfer, exact-match performance, action-alphabet and baseline comparisons, and theoretical properties and limits of the routing objective.

C.4 ROUTER BEHAVIOR IN BGE EMBEDDING SPACE

  • Five query-type regions emerge in the BGE projection, with most clusters dominated by one arm and mixed actions concentrated near boundaries.The visualization is consistent with query-dependent routing but cannot rule out memorization or explain why an individual action succeeds.Source: §C.4 · p. 28, Figure S3 · p. 28
  • 87% of queries match the six-action utility oracle, with the remaining disagreements concentrated near cluster boundaries where actions are near-tied.The UMAP plot does not explain individual answer outcomes.Source: §C.4 · pp. 28–29, Figure S4 · p. 29

C.6 EXACT MATCH (EM) SCORES

  • FORGE leads the non-oracle policies in macro EM on both hosts, with the relative ordering largely preserved from F1.EM/F1 ratios range from 0.65 on MuSiQue to 0.96 on FEVER; the enumerated Oracle row is descriptive, not an EM upper bound.Source: §C.6 · pp. 29, 31, Table S18 · p. 31

C.5 CROSS-HOST TRANSFER: PER-TASK BREAKDOWN

  • Table 8 reports macro F1 for zero-shot transfer to three API hosts without target-host weight updates; Table S17 adds per-task F1 and cached cost.The supplied passages do not give the table’s numeric results.Source: §C.5 · p. 29, Table S17 · p. 30

C.7 NUMERICAL TABLE FOR THE ACTION ALPHABET SWEEP

  • Adding CoT-Prompt from K=3 to K=6 raises F1 while lowering cost, adding Think-Low from K=6 to K=9 gives +1.0 and +1.1 F1, and unlocking Think-High from K=9 to K=12 gives the largest absolute improvement.The sweep is reported for Qwen3-8B-Thinking on HotpotQA and MuSiQue.Source: §C.7 · p. 31, Table S19 · p. 31

C.8 HEURISTIC ROUTING BASELINES

  • Self-Routing raises Mistral-7B macro F1 from 24.2 to 28.2, while cached costs omit its initial confidence call and the cascade’s earlier calls.The omitted calls must be included in Fresh Online cost or latency comparisons.Source: §C.8 · pp. 31–32, Table S20 · p. 32

D.1 VALUE OF PER-QUERY ROUTING

  • Expected utility from per-query routing is at least that of the best fixed action, and is strictly greater when every action is suboptimal on a positive-probability set of queries.The result is stated for integrable utilities and a finite action set.Source: §D.1 · p. 32

D.2 A FIXED-REFERENCE REGULARIZED OBJECTIVE

  • For a fixed-reference entropy-and-KL objective, the idealized unrestricted policy has a unique optimum, but this does not guarantee convergence or regret bounds for implemented Stage 2.The proposition assumes known utilities and fixed coefficients; Stage 2 instead uses sampled, group-standardized rewards, a clipped surrogate, a finite router, and an adaptive KL coefficient.Source: §D.2 · p. 32, D PROPERTIES OF THE ROUTING OBJECTIVE… · p. 32

F.1 LIMITATIONS

The paper’s scope and deployment implications are bounded by its evaluated tasks, host and feature-acquisition settings, and measured outcomes. It identifies extensions and risks that remain to be tested, while disclosing AI assistance in manuscript revision.

F.1 LIMITATIONS

  • Evaluation covers knowledge-intensive QA and fact verification; reasoning-only benchmarks are not evaluated and would require a different action alphabet.Stage 0 enumeration scales linearly with the warm-start alphabet, while Stage 2 incurs host-call rollout costs; joint retrieval and routing optimization is also out of scope.Source: §F.1 · p. 33

F.2 FUTURE WORK

  • Proposed extensions include target-side calibration, joint support-form and retrieval or model routing, sequential escalation, and testing dense or hybrid retrieval backends.Their training cost and empirical value remain to be tested.Source: §F.2 · p. 34

F.3 BROADER IMPACTS

  • Fresh Online Qwen3-8B results trade lower host-token use for higher median latency with Full FORGE, while FORGE-Lite avoids pre-routing calls.Token counts alone do not establish energy savings or lower end-to-end cost for every provider; grounding requirements and effects on bias, hallucination, and misuse were not evaluated.Source: §F.3 · p. 34

G USE OF LARGE LANGUAGE MODELS

  • An AI assistant helped revise the manuscript’s prose, organization, and internal consistency; the authors are responsible for verifying its technical claims, references, and results.Source: G USE OF LARGE LANGUAGE MODELS · p. 34

Related papers

Loading 2609.34358v1…