Source-linked AI summary
SlimSearcher: Training Efficiency-Aware Web Agents via Adaptive Reward Gating
Zequn Xie, Junjie Wang, Dan Yang, Jie Feng, Yue Shen, Jian Wang, Jinjie Gu
TL;DR
Web agents often achieve research-task success through redundant tool use and reasoning, making long-horizon search computationally expensive. SlimSearcher jointly filters SFT trajectories for Pareto-efficient behavior and applies adaptive, correctness-gated reward shaping during RL. Across long-horizon benchmarks, it reduces average tool-call rounds by 17%–58% while maintaining or improving accuracy.
Problem
Existing web agents use brute-force strategies involving blind tool dependency and performative reasoning, leading to wasteful tool calls and excessive computational costs.
Method
SlimSearcher combines efficiency-aware SFT data filtration with Adaptive Reward Gating anchored to the Minimal Necessary Path and cascaded with a strict correctness gate.
Results
17%–58% fewer average tool-call rounds are achieved across GAIA, BrowseComp, and XBench-DeepSearch while task accuracy improves.
Takeaways & Limitations
SlimSearcher integrates efficiency optimization into SFT and RL to move web agents toward the accuracy-computational-cost Pareto frontier.
Takeaways & Limitations
The framework is optimized for text-based reasoning, and its RL effectiveness partly depends on SFT initialization quality.
Abstract
from arXiv · showhide
Deep research agents have demonstrated remarkable capabilities in complex information-seeking tasks, yet this power comes at a steep computational cost. Driven by accuracy-focused training paradigms, current models adopt brute-force strategies characterized by blind tool dependency and performative reasoning-generating long, redundant trajectories that are far from necessary for resolving these tasks, leading to wasteful tool calls and excessive token consumption. To overcome this efficiency trap, we propose SlimSearcher, a principled framework that pushes the Pareto frontier between accuracy and computational cost across both Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). In the SFT stage, SlimSearcher employs Pareto-efficient filtration to distill trajectories that are both successful and economical, guiding the model toward inherently efficiency-aware search behaviors. During RL, we introduce Adaptive Reward Gating, a dynamic reward-shaping mechanism that evaluates relative tool and token efficiency within a sampled cohort. By cascading these adaptive efficiency metrics with a strict correctness gate, our approach effectively avoids the brevity bias associated with absolute penalties and mitigates reward hacking. Extensive experiments on long-horizon benchmarks, including GAIA, BrowseComp, and XBenchDeepSearch, demonstrate that SlimSearcher reduces average tool-call rounds by 17%-58% while maintaining or improving accuracy.
1 Introduction
Existing web agents often pursue correctness through redundant tool use and reasoning, creating an efficiency trap. SlimSearcher addresses this by jointly filtering SFT data for efficiency and correctness and shaping RL rewards adaptively, with experiments showing lower tool-call rounds and improved accuracy.
- The Efficiency Trap: Several hundred reasoning rounds may be required for one question because success-focused training incentivizes indiscriminate tool usage.The resulting tool calls increase computational, infrastructure, time, and operational costs.
- The Efficiency Trap: Blind tool dependency wastes API calls on simple queries, while performative reasoning adds redundant loops and dead-end branches to complex searches.These failure modes affect initiation and execution efficiency, respectively.
- Training Causes: Accuracy-only rejection sampling treats any correct trajectory as positive, allowing redundant patterns to enter SFT data and encouraging bloated behavior.Existing RL strategies focused primarily on success can also produce efficiency collapse through expanded search rounds and context lengths.
- SlimSearcher: SlimSearcher integrates efficiency optimization across SFT and RL to move agents toward the Minimal Necessary Path while balancing accuracy and computational cost.Its SFT stage uses joint efficiency evaluation rather than correctness alone to construct the training dataset.
- SlimSearcher: Adaptive Reward Gating uses dynamic, task-adaptive reward shaping to encourage minimal trajectory complexity without compromising task accuracy.The mechanism is designed to avoid the brevity bias of fixed penalties.
- Results: 17%–58% fewer average tool-call rounds are reported across GAIA, BrowseComp, and XBench-DeepSearch while task accuracy improves.This cross-benchmark result identifies an efficiency-accuracy Pareto improvement.
2 Related Work
Prior web-agent work spans prompting and training-centric approaches, but many methods prioritize success rates and leave inference efficiency under-addressed. SlimSearcher unifies efficiency optimization across SFT and RL, targeting redundant external actions as well as reasoning length.
- Prompting and Engineering Frameworks: Prompting and engineering frameworks elicit agentic behavior from frozen LLMs but can incur high token costs through verbose context management.Examples include SelfDC, AutoGen, and GPT-Researcher.
- Training-Centric Agent Evolution: Training-centric methods internalize search and reasoning through SFT or RL, yet most prioritize success rates as the sole metric.This emphasis can incentivize brute-force strategies that exhaust computational resources.
- Efficiency Gaps: Prompt engineering and RL penalties can reduce internal reasoning length, but web-agent efficiency also requires pruning redundant external actions such as cyclic searches.Reducing text tokens alone does not address unnecessary tool behavior.
- Efficiency-Aware Training: WebLeaper addresses efficient search behavior through high-density task synthesis, whereas SlimSearcher unifies efficiency optimization across the entire training pipeline.The comparison distinguishes data synthesis from end-to-end training-pipeline optimization.
- SlimSearcher: SlimSearcher combines a Minimal Necessary Path anchor with a strict correctness gate to preserve accuracy while reducing computational pleonasm.The design targets efficiency optimization without treating correctness as a secondary objective.
3 Methodology
SlimSearcher unifies SFT and RL through Multi-Stage Gating to optimize correctness and efficiency jointly. It filters successful trajectories for Pareto-efficient demonstrations and uses adaptive, cohort-relative rewards to suppress redundant tool use and token generation.
- SlimSearcher unifies SFT and RL under a Multi-Stage Gating mechanism that optimizes accuracy and computational efficiency together.The mechanism acts as a Pareto Filter during SFT and a Cascading Reward Function during RL.
- Multi-Stage Gating: The correctness gate sets the final reward to zero for trajectories that fail to recover the ground truth, preventing brevity from overriding accuracy.This strict gate is intended to prevent reward hacking through short but hallucinatory answers.
- Multi-Stage Gating: For correct trajectories, Adaptive Reward Gating anchors tool efficiency to the lowest-cost successful path in the sampled candidate set.Tool costs are computed from weighted external tool invocations, with uniform weights used in the experiments.
- Multi-Stage Gating: The relative tool-efficiency score is transformed exponentially into a bounded multiplier, giving trajectories that diverge from the empirical optimum a sharper penalty.The resulting multiplier lies in rtool ∈(0, 1], while Sopt = 0 denotes the candidate-set optimum.
- Multi-Stage Gating: Adaptive Efficiency Anchoring constrains internal token length relative to the most concise successful candidate, penalizing verbose trajectories non-linearly.The mechanism targets performative reasoning that can persist even when external tool use is reduced.
- Efficiency-Aware SFT: Pareto-Efficient Filtration first removes incorrect trajectories, then selects the Minimal Necessary Path using joint tool-call and token efficiency.The resulting Dsft dataset is intended to teach parsimonious, cost-aware search behavior from initialization.
- Policy Optimization: During RL, SlimSearcher samples trajectory groups, computes cascading rewards, standardizes group-relative advantages, and updates the policy with a clipped surrogate objective.The trajectory-level advantage is applied to every generated token in the trajectory.
4 Experiments
SlimSearcher is evaluated across four long-horizon web-agent benchmarks using accuracy, tool-call rounds, and token consumption, with results showing improved efficiency while maintaining or improving accuracy. Detailed analyses attribute these gains to Pareto-filtered training, adaptive reward mechanisms, and correctness constraints.
- Experimental Settings: Experiments cover XBench-DeepSearch, BrowseComp, GAIA, and HLE, using accuracy, tool-call rounds, and model-generated token efficiency as evaluation dimensions.The implementation uses repeated evaluations and reports average Pass@1 and efficiency metrics.
- Main Results: SlimSearcher improves efficiency while maintaining accuracy across different backbones and benchmarks.Table 1 evaluates task accuracy, interaction rounds, and total token consumption.
- Main Results: 48.4% fewer tool-call rounds on GAIA accompanies token savings and accuracy increasing from 0.682 to 0.709 for Tongyi-DeepResearch.Rounds decrease from 20.56 to 10.61, while token usage decreases by 33.4%.
- Main Results: On Browsecomp, tool-call rounds decrease from 63.70 to 47.63 and accuracy increases from 0.410 to 0.447.Token usage also decreases from 12014 to 11093.
- Main Results: For Qwen3, RL reduces HLE tool-call rounds from 27.86 to 19.51 and increases accuracy from 0.259 to 0.278 after SFT increases token usage to 7299.Prompt Control does not consistently improve efficiency, while the combined training approach prunes redundant calls without sacrificing accuracy.
- Detailed Performance Analysis: SlimSearcher shifts the Pareto frontier toward higher accuracy with a more compact action space across representative benchmarks.On GAIA, it resolves most queries within the 0–20 round bucket and approaches 70% cumulative accuracy, while reducing blind tool dependency.
- Ablation Study: Reward-guided rejection sampling on GAIA raises accuracy from 0.641 to 0.665 while reducing rounds from 25.90 to 24.46 and tokens from 7478 to 7299.Removing the correctness gate instead causes accuracy to fall to 0.136 with near-zero tool exploration at 0.07 rounds.
- Ablation Study: Removing Adaptive Efficiency Anchoring increases HLE tool-call rounds from 19.51 to 31.05 while yielding only a marginal accuracy gain.Without the dynamic anchor, the model defaults to brute-force exploration and loses computational discipline.
5 Conclusion
SlimSearcher addresses web-agent inefficiency with a Multi-Stage Gating framework that unifies supervised fine-tuning and reinforcement learning. Across long-horizon benchmarks, it reduces tool-call rounds by 17%–58% without compromising task accuracy and aims to produce accurate, computationally disciplined agents.
- Conclusion: SlimSearcher unifies supervised fine-tuning and reinforcement learning through strict correctness and adaptive reward gates.The framework treats efficiency as a central training objective rather than a secondary constraint.
- Conclusion: 17%–58% fewer average tool-call rounds are reported across long-horizon benchmarks without compromising task accuracy.The conclusion presents this as the paper’s principal experimental outcome.
- Conclusion: SlimSearcher transforms open-source models into accurate and computationally disciplined agents.The paper frames this as a scalable approach for developing efficient web agents.
Limitations
SlimSearcher’s current limitations concern multimodal extension, dependence on SFT initialization, and uniform treatment of tool costs. These constraints define important directions for future research.
- Scope: The framework is optimized for text-based reasoning and does not yet handle multimodal redundancy or the computational cost of visual processing.Extending Adaptive Efficiency Anchoring to visual environments is identified as a critical next step.
- Initialization: RL effectiveness depends partly on SFT initialization, especially because optimization lacks empirical anchors when the base model finds no Minimal Necessary Path.This limitation is most relevant in extremely niche domains.
- Operational costs: The implementation uses uniform tool-call weighting, overlooking differences in tool latency and financial cost.Future work is directed toward fine-grained, cost-sensitive tool weighting for real-world deployments.
A Implementation Details
The implementation uses curated SFT and RL data, distributed training infrastructure, and bounded rollout settings. It also specifies evaluation-related components within the broader training setup.
- Training data: The training environment uses curated information-seeking datasets for both SFT and RL, assembled through rejection sampling and answer-consistency verification.The source collections span Asearcher, TaskCraft, WebWalker, Voyager, WebShaper, RedSearcher, WebDancer, and MegaScience.
- Training data: The distillation corpus retains queries with 0 < PR(q) ≤1 after four executions per query, targeting challenging yet solvable tasks.This filtering excludes trivial queries that do not require deep research.
- SFT configuration: 13,863 training trajectories were used for SFT, with a two-phase learning-rate schedule and multi-level parallelism.The schedule combines 5% warmup with 95% cosine decay, while training uses micro-batch size 1 and global batch size 16.
- RL configuration: 1,510 high-quality QA pairs were used for RL with GRPO on the RLLM framework, while asynchronous vLLM rollouts sampled 8 responses per prompt.The RL corpus draws from Voyager, WebShaper, REDSearcher, WebDancer, and MegaScience.
- RL configuration: RL training ran on 64 H800 GPUs across 8 nodes, allowing each agent up to 100 interaction steps and a 7,200-second trajectory timeout.Training proceeded for up to 10 epochs with checkpoints saved every 10 steps.
A.5 Datasets
The evaluation covers four web-agent benchmarks, with text-only subsets specified for GAIA and HLE. The selected subsets contain 103 GAIA queries and 500 HLE questions.
- Benchmark selection: Experiments evaluate SlimSearcher on XBench-DeepSearch, BrowseComp, GAIA, and HLE.These are described as widely adopted web-agent benchmarks.
- Benchmark subsets: GAIA evaluation uses 103 text-only queries from its development set.The evaluation uses a subset rather than the full benchmark.
- Benchmark subsets: HLE evaluation uses 500 text-only questions following the protocol of previous studies.The subset and protocol are specified for comparability with prior work.
A.6 Evaluation Details
Evaluation uses benchmark-specific judge models and prompts aligned with prior protocols, with detailed prompts planned for public release. The agent is initialized by a system prompt defining its role, tools, and interaction schema.
- Judging: GAIA and xBench use Qwen2.5-72B-Instruct as the judge model, while xBench-DeepSearch additionally uses Gemini-2.5-Flash and BrowseComp uses GPT-4o-2024-11-20.The supplied passage lists benchmark-specific judge assignments.
- Evaluation protocol: Evaluation prompts remain consistent with the respective original papers to support alignment and reproducibility.Detailed prompts are intended for public release on the project’s GitHub repository upon acceptance.
- Agent initialization: The system prompt initializes SlimSearcher as a “Deep Research Assistant” and specifies web search, page visiting, Python execution, and file parsing.It also defines tool parameters and enforces a strict interaction schema.
C Prompt Variations and Analysis of the PromptControl Baseline
PromptControl tests whether explicit instructions can reduce inefficient web-agent behavior, while the case study shows the prompt and comparative reasoning traces used for analysis.
- C Prompt Variations and Analysis of the PromptControl Baseline: PromptControl tests whether explicit instructions can mitigate the efficiency trap and performative reasoning.The experiments evaluate whether prompting alone can address inefficient long-horizon behavior.
- C.1 Prompt Variations Tested: Three prompt variations range from soft efficiency guidance to strict prohibitions on redundant calls and concise reasoning.The tested directives target search queries, tool calls, prior knowledge, and reasoning length.
- C.1 Prompt Variations Tested: Variation 3 slightly reduced tool usage without immediately collapsing accuracy and became the PromptControl baseline.Despite this optimized prompt, severe efficiency degradation remained on long-horizon tasks such as HLE.
- D Case Study: The case study evaluates SlimSearcher against MiroThinker using representative reasoning traces from GAIA and XBench.The traces are presented in Figures 6, 8, and 7.
- D Case Study: The system prompt instructs the agent to conduct thorough multi-source investigations and enclose the final answer within answer tags.It supports broad open-domain and specialized academic queries and emphasizes credible, diverse sources.
- D Case Study: The prompt exposes search, webpage-visit, and Python-interpreter functions through XML tool signatures.Search and visit accept structured query or URL arguments, while Python executes code in a sandboxed environment.
- D Case Study: Tool calls use a JSON object containing the function name and arguments inside tool_call XML tags.The Python tool additionally requires an empty arguments object and code enclosed in code tags, with output printed.
- D Case Study: On GAIA, SlimSearcher reaches the same answer as MiroThinker using 22 tools across 4 phases rather than 13× more tools across 4 redundant loops.The XBench figures compare SlimSearcher’s strategic aggregation with MiroThinker’s exhaustive verification loop.