PaperlayerView exact revision on arXiv

Source-linked AI summary

HyWe: Unifying Search and Operation for Real-World Hybrid Web Tasks

Zijian Cai, Haowei Liu, Kailai Chen, Lin Peng, Cong Wan, Ziqi Zhang, Chunfeng Yuan, Bing Li, Weiming Hu, Haiyang Xu

arXiv:2610.16319v1cs.AI

TL;DR

Real-world web tasks often require both finding information and interacting with pages, although search- and operation-based agents are developed separately. HYWE unifies these capabilities through switchable modes that share evidence, and HYWE-35B outperforms evaluated search- and operation-based agents on hybrid tasks, reaching 47.0% HYWEBENCH success.

  • Problem

    Real-world hybrid web tasks require both information gathering and page interaction, while search- and operation-based agents are developed and evaluated separately.

    Source: §1 · p. 1

  • Method

    HYWE sustains sub-goal progress in search and operation modes, switching between them as needed and carrying evidence across modes.

    Source: §3.1 · p. 4

  • Results

    Across hybrid and operation-based tasks, HYWE-35B outperforms evaluated agents, achieving 47.0% success on HYWEBENCH.

    Source: §1 · p. 2, §4.2 · p. 7, §5 · p. 10

  • Takeaways & Limitations

    Mode switching improves hybrid task completion over simply exposing search and operation tools together under one prompt.

    Source: §4.4 · p. 8

    The ablations assess the framework as a whole, and the capability probe depends on its search tools and budget.

    Source: §5 · p. 10

Abstract

from arXiv · show

Real-world hybrid web tasks need both information gathering and page interaction, yet search-based and operation-based agents are developed and evaluated separately. To bridge this gap, we introduce HyWe, a framework that unifies both capabilities within one model and trajectory. Its two modes use dedicated prompts and tool sets to sustain sub-goal execution; explicit switching selects the needed capability, and shared history lets each mode build on the other's findings. To provide demonstrations of these behaviors, we develop HyWeCollector, a fully automatic pipeline that uses task prototypes, four switching patterns, and multi-agent synthesis to produce verified hybrid trajectories. We train HyWe-35B on these demonstrations to unify efficient information gathering and browser interaction in a single 35B web agent. To evaluate hybrid task completion, we introduce HyWeBench: 100 human-written tasks on websites distinct from training, with human-written sub-goal checklists and an outcome-based evaluation pipeline. Hybrid training enables HyWe-35B to achieve 47.0% success on this benchmark, improving on the untrained base model by 19.0 percentage points and outperforming existing search-based and operation-based agents. Even before training, the same base model solves 28.0% of tasks with HyWe, versus 11.0% with all tools under one prompt. Beyond hybrid tasks, HyWe-35B achieves 59.0% and 88.3% success on Online-Mind2Web and WebVoyager, outperforming existing open-source baselines and cutting action cost per successful task by 48.3% and 51.9% compared with Fara-7B.

1 INTRODUCTION

The introduction frames real-world web tasks as requiring both information search and browser interaction, then presents HYWE, HYWECOLLECTOR, and HYWEBENCH as a unified approach to completing and evaluating them. On HYWEBENCH, HYWE-35B achieves 47.0% success after training, outperforming search- and operation-based agents.

1 INTRODUCTION

  • Search agents gather information faster but lack page interaction, while operation agents can interact with pages yet face lengthy navigation; existing benchmarks largely separate these capabilities.In the sampled tasks, search recovered about 40% of facts obtained through browser interaction, and 29% versus 6% were solved within four steps by search and operation agents, respectively.Source: §1 · pp. 1–2
  • HYWE-35B achieves 59.0% and 88.3% success on Online-Mind2Web and WebVoyager, with 48.3% and 51.9% fewer actions per successful task than Fara-7B.It also reaches 64.0% accuracy on xbench-DeepSearch-2510.Source: §1 · p. 2
  • HYWE unifies search and browser operation through sub-goal execution, mode-specific prompts and tools, explicit SWITCH actions, and shared history; against all tools under one prompt, it raises information/operation sub-goal completion from 36.9%/15.8% to 74.8%/43.7%.Each mode gathers evidence or completes interactions for sub-goals, while shared history carries earlier findings forward to guide later work.Source: §1 · pp. 2–3
  • HYWECOLLECTOR automatically creates verified hybrid trajectories using reusable task prototypes, four switching patterns, and multi-agent synthesis of task generation, execution, phase control, and review.The switching patterns capture capability limits, efficiency, and dependencies between sub-goals.Source: §1 · pp. 2–3
  • HYWE-35B reaches 47.0% success on HYWEBENCH, versus 28.0% before training, beating XYZ-Aquila-mini (22.0%) and Fara-7B (13.0%) while matching Kimi-K3 (46.0%) and Qwen3.8-Max (44.0%).Its success rate is achieved with 1/80 and about 1/69 of Kimi-K3’s and Qwen3.8-Max’s total parameters, respectively.Source: §1 · p. 2
  • HYWEBENCH contains 100 human-written tasks on websites distinct from training, with info/op checklists and outcome-based evaluation from execution evidence regardless of mode.Each task includes at least one operation sub-goal.Source: §1 · p. 2

2 RELATED WORK

Related work treats search-based information gathering and browser operation largely as separate capabilities, while HyWe combines them in one trajectory. HyWeBench evaluates both information gathered and actions completed on real websites.

Search-based agents and benchmarks

  • HYWE combines search tools and page actions in one trajectory, letting retrieved requirements guide product configuration and interaction results provide further information.Shared history lets each mode build on the other’s findings; a single search can replace several site visits when required facts are searchable.Source: §2 · p. 3
  • HYWEBENCH uses real-website tasks with human-written info/op checklists, requiring at least one op goal and evaluating both obtained facts and completed actions.Its evaluator checks text tool observations and screenshots for each sub-goal, distinguishing completed work from unsupported final-answer claims and supporting different tool sets.Source: §2 · p. 3
  • HYWECollector demonstrates mode selection and information handoff; its four switching patterns are grounded in capability probes.It provides demonstrations of when to choose each mode and how to pass information between them.Source: §2 · p. 3
  • Search-o1 (Li et al., 2025a) and WebThinker (Li et al., 2025b) integrate search into multi-step reasoning; Search-R1 (Jin et al., 2025) and DeepResearcher (Zheng et al., 2025) train search use through reinforcement learning.WebExplorer (Liu et al., 2025) and Tongyi DeepResearch (Tongyi DeepResearch Team, 2025) scale training with synthetic information-seeking data; these methods focus on finding and reasoning over web information.Source: §2 · p. 3
  • GAIA (Mialon et al., 2023) evaluates tool-assisted reasoning, BrowseComp (Wei et al., 2025) hard-to-find facts, SealQA (Pham et al., 2025) noisy results, and DeepSearchQA (Gupta et al., 2026a) complete answer sets.These benchmarks mainly grade final answers against references, so they do not establish whether an agent performed page actions required by hybrid tasks.Source: §2 · p. 3

Operation-based agents and benchmarks

3 METHOD

HYWE unifies search and browser interaction through mode-specific execution, explicit switching, and shared evidence history. HYWECOLLECTOR creates and verifies hybrid demonstrations, which are converted into training samples for supervised fine-tuning.

3.1 HYWE FRAMEWORK

  • HYWE uses mode-specific prompts and tools, an explicit SWITCH action, and shared history so each mode can advance sub-goals using evidence gathered in earlier segments.Search guidance emphasizes verifying facts and unchecked constraints; operation guidance emphasizes checking page changes and recovering from failed actions.Source: §3.1 · pp. 4–5

3.2 HYWECOLLECTOR TRAJECTORY SYNTHESIS PIPELINE

  • HYWECOLLECTOR probes real websites, abstracts mode-specific task prototypes, and designs four switching patterns for capability changes or dependent sub-goals.It abstracts 36 search-based and 48 operation-based prototypes from facts, with 40% assigned to the search fact pool and 31% to the operation fact pool.Source: §3.2 · p. 5
  • A multi-agent pipeline synthesizes, executes, controls, and independently reviews trajectories, retaining 3,735 verified demonstrations from 25,662 tasks run across 1,110 websites.The synthesizer binds typed prototypes to sites and entities; an executor acts without seeing the private plan, a controller checks phase conditions, and a reviewer refines leakage while preserving facts and decisions.Source: §3.2 · pp. 5–6

3.3 TRAINING

  • Training converts 3,735 hybrid trajectories into 8,055 segment-based samples and supplements them with 7,000 adapted search trajectories from XYZ Agentic Team (2026).The model is fine-tuned with next-token cross-entropy while the vision encoder and projector remain frozen; no additional operation-based corpus is used.Source: §3.3 · p. 6

4 EXPERIMENTS

HYWE-35B combines strong hybrid-task completion with gains on operation-based and deep-search benchmarks. Ablations show that explicit modes and training improve over naive tool merging and the base model.

4.1 SETUP

  • Experiments assess operation-based success and efficiency, hybrid task completion, and deep search, comparing open-source and frontier agents alongside the Qwen3.6-35B-A3B base model.Operation benchmarks are Online-Mind2Web and WebVoyager; HYWEBENCH covers hybrid completion, while HLE, xbench-DeepSearch-2510, and BrowseComp-EN assess deep search.Source: §4.1 · pp. 6–7

4.2 SUCCESS AND EFFICIENCY ON OPERATION-BASED BENCHMARKS

  • HYWE-35B leads evaluated open-source agents with 59.0% success on Online-Mind2Web and 88.3% on WebVoyager.Before training, the base model uses 30.9 and 20.8 Steps/SR, versus at least 61.5 and 34.9 for every listed operation-based agent; training raises success by 7.3 and 11.5 points and yields 31.8 and 16.8 Steps/SR. Training raises switch rates from 37.3%/20.8% to 93.7%/87.9%; 79%/95% of trained trajectories switch at most twice, usually O →S or O →S →O.Source: Table 1 · p. 7, §4.2 · p. 7

4.3 HYBRID CAPABILITY

  • 47.0% HYWE-35B success on HYWEBENCH exceeds XYZ-Aquila-mini (22.0%) and Fara-7B (13.0%), and is comparable to Kimi-K3 (46.0%) and Qwen3.8-Max (44.0%).HYWE-35B also leads single-mode agents on information (82.2%) and operation (68.8%) sub-goal completion; training raises task and operation completion by 19.0 and 25.1 points, respectively. HYWEBENCH contains 100 human-written tasks on websites distinct from training, with human-written checklists; an LLM judge evaluates outcomes from text and screenshots.Source: §4.3 · pp. 7–8, Table 2 · p. 8

4.4 FRAMEWORK AND MODE ABLATIONS

  • On HYWEBENCH, HYWE raises same-base-model success from 11.0% with naive tool merging to 28.0% with explicit mode switching.Information and operation completion rise from 36.9%/15.8% to 74.8%/43.7%; search completes more information sub-goals, while operation mode completes more operation sub-goals. Training on HYWECOLLECTOR trajectories raises success to 47.0%; in a reward-chart case study, the trained model switches from search to browser interaction and completes all seven sub-goals, unlike the base model.Source: §4.4 · pp. 8–9, Table 3 · p. 9

4.5 RETENTION OF DEEP SEARCH CAPABILITIES

  • HYWE-35B improves over the base model from 30.5% to 47.8% on HLE, 49.0% to 64.0% on xbench-DeepSearch-2510, and 59.2% to 74.6% on BrowseComp-EN.Its 64.0% on xbench-DeepSearch-2510 is the highest among reported open-source baselines; on BrowseComp-EN, its 74.6% is close to Agents-A1 (75.5%) and above FORT-Searcher (72.2%). The agent rarely calls SWITCH, so these evaluations primarily measure deep research in search mode.Source: Table 4 · p. 9, §4.5 · p. 9

5 CONCLUSION · A CASE STUDIES · A.1 HYWEBENCH

HyWe unifies search and browser interaction, and the case study illustrates how switching between them supports completion of a hybrid web task. The conclusion also reports the framework’s contributions, limitations, and research practices.

5 CONCLUSION

  • HyWe unifies search and operation by sustaining sub-goal progress within each mode and carrying evidence across switches; framework ablations improve completion over exposing all tools together.Hybrid training further strengthens joint use. HYWE-35B outperforms evaluated search- and operation-based agents on hybrid tasks, achieves higher success and efficiency on operation-based tasks, and preserves deep search capabilities. The ablations assess the framework as a whole, the capability probe depends on its search tools and budget, and isolating individual design contributions and improving robustness to changing websites remain open directions.Source: §5 · p. 10

AI USE STATEMENT

  • AI tools assisted with manuscript preparation, polishing, literature search, LaTeX formatting, and code debugging; their use in data synthesis and evaluation is described in Sections 3.2 and Appendix C.The authors take responsibility for the final manuscript and research artifacts’ accuracy, originality, and integrity.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • The pipeline uses ordinary browsing on public websites without creating accounts, making purchases, submitting personal information, or durably altering site state; released data contains no personal information.The fact pool contains public information, and operation-mode training data comes from audited synthetic hybrid trajectories on public sites rather than authenticated or private services; the release targets research reproducibility despite web agents’ dual-use potential.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper specifies its inference harness, data pipeline, training, and benchmark evaluation, and says it will release the code, pipelines, task blueprints, training data, checkpoints, tasks, and checklists.All reported results use full benchmark sets; live website changes mean identical seeds do not guarantee identical observations.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 HYWEBENCH

  • HYWE-35B completes all 7 of 7 sub-goals in 26 steps, versus 4 of 7 for search-only, 5 of 7 for operation-only, and 2 of 7 for naive merging.The case-study figure compares these agents on the same seven-sub-goal task.Source: Figure 3 · p. 13, §A.1 · p. 14
  • The case study compares search-only, operation-only, and naively merged tools using the same Qwen3.6-35B-A3B base model against HYWE-35B on a seven-sub-goal Pirates reward-chart task.The agents differ in their tools and prompt; the figure shows their steps on a shared scale.Source: Figure 3 · p. 13, §A.1 · pp. 13–14
  • Under HyWe, the base model gathers the same information but never switches, completing 3 of 7 sub-goals in 25 steps.HYWE-35B switches after seven search steps, then completes the chart in 17 browser actions; the case study suggests the gain comes from learning when to switch, not the tool interface alone.Source: §A.1 · p. 14

A.2 OPERATION BENCHMARKS · B PILOT STUDY: CAPABILITY AND EFFICIENCY

The operation benchmarks show HyWe-35B using search to locate targets and browser interaction when pages require operation. The pilot study finds complementary strengths in search and operation agents, motivating search-first execution with an operation fallback.

A.2 OPERATION BENCHMARKS

  • HYWE-35B answered the BBC News Market Data question correctly in 8 steps by switching to search, while the page’s discontinued section was unreachable by navigation.Search found the BBC discontinuation notice and a Press Gazette report identifying Morningstar as the feed’s former licensee.Source: §A.2 · p. 14, Figure 4 · p. 15
  • HYWE-35B completed the state-comparison chart task in 14 steps by searching for the page, then switching to the browser to select Michigan.Search found the page, but reading it returned only the default Illinois–United States comparison because page reading could not operate the state selector.Source: §A.2 · p. 14, Figure 5 · p. 16

B PILOT STUDY: CAPABILITY AND EFFICIENCY

  • For information lookup, the search-based agent solved 26/52 tasks (50.0%) within four steps versus 6/52 (11.5%) for the operation-based agent; final success rates were 82.7% and 67.3%, respectively.These task labels describe requirements rather than exclusive suitability for either mode.Source: B PILOT STUDY: CAPABILITY AND EFFICIENCY · p. 17, Figure 6 · p. 17
  • For interaction-oriented tasks, the operation-based agent solved 24/43 (55.8%), compared with 15/43 (34.9%) for the search-based agent.The interaction-oriented tasks included conditional queries, configured quotes, and form submissions.Source: B PILOT STUDY: CAPABILITY AND EFFICIENCY · p. 17, Figure 6 · p. 17
  • The pilot motivates preferring search when it suffices (A1) and switching to operation when search fails to obtain required information (A2).When reachability is uncertain, A2 allows a bounded search attempt before the switch; learned switching is evaluated in Section 4.3.Source: B PILOT STUDY: CAPABILITY AND EFFICIENCY · p. 17
  • On 100 InSTA tasks, the search-based agent solved 60 tasks and the operation-based agent 62, with only 44 solved by both.The search-based agent alone solved 16 tasks, while the operation-based agent alone solved 18.Source: Figure 6 · p. 17, B PILOT STUDY: CAPABILITY AND EFFICIENCY · p. 17

B.1 COST AND FAILURE ANALYSIS … C.3 EXECUTION FRAMEWORKS ON HYWEBENCH

The appendix analyzes paired costs and failure patterns in a filtered 100-task pilot, then specifies HyWeBench task annotation, evaluation, framework controls, and execution setups. Together, these passages describe the scope and protocols behind the reported analyses without providing the table’s task-group outcome values.

B.1 COST AND FAILURE ANALYSIS

  • Operation-based costs had median ratios of 1.9 for steps, 9.0 for tokens, and 6.3 for time relative to search-based costs.The paired comparisons were significant for steps (p = 0.0089), tokens (p = 5.15 × 10−9), and time (p = 1.63 × 10−9); time uses 39 pairs, while steps and tokens use all 44 joint successes.Source: §B.1 · p. 17
  • Among 16 tasks solved only by search, operation-based failures involved extraction in 7, looping or giving up in 5, widget failures in 2, and page-load failures in 2.Fourteen failures concerned page interpretation or interaction rather than loading; failed operation runs took a median 25 steps versus 10 for successful search runs, and search bypassed on-site navigation in 12 tasks.Source: §B.1 · p. 18
  • Among 18 tasks solved only by operation, search failures included inaccessible dynamic state in 7 and answers sourced only from other sites in 5; operation runs were interaction-dependent in 16/18 such tasks.Interaction dependence was also annotated for 26/44 joint successes (two-sided Fisher exact p = 0.0347); because pilot modes were locked, these outcomes do not measure whether a model would switch.Source: §B.1 · p. 18

B.2 TASK SELECTION AND ANNOTATION

  • The pilot is a filtered, curated sample of 100 tasks from 100 domains, with task-demand labels assigned retrospectively using task wording and reference steps.Selection retained 81 tasks from one pool and sampled 19 from another after excluding cases primarily attributed to anti-bot blocking; five uncertain tasks remain in aggregates but are excluded from stratified panels, and labels are AI-assisted rather than independently blinded or human-validated.Source: §B.2 · p. 18

C.1 HYWEBENCH TASKS AND EVALUATION

  • HyWeBench uses human-written checklists fixed across systems, runs each system once per task with a 100-step budget, and requires every checklist item for task success.The 556 checklist items comprise 309 information and 247 operation sub-goals; Kimi-K3 judges execution records, while completion rates aggregate achieved items by type.Source: §C.1 · p. 19

C.3 EXECUTION FRAMEWORKS ON HYWEBENCH

  • External agents use their released or official execution frameworks where available, while trajectories are converted to a common record format for the HyWeBench judge.The implementations include Playwright for UI-TARS-1.5-7B and the official browser environment for Fara-7B; HYWE evaluates the base model, HYWE-35B, frontier models, and internal mode ablations.Source: §C.1 · p. 19, §C.3 · p. 19

C.2 BASE-MODEL AND FRAMEWORK CONTROLS

  • The framework controls compare naive merging with full HYWE and evaluate search-only and operation-only variants using the same base model on all 100 tasks.Full HYWE adds mode-specific instructions, tool isolation, explicit switching, and shared history; the compared base checkpoint is Qwen3.6-35B-A3B, with approximately 35B total and 3B active parameters.Source: §C.2 · p. 19

C.4 OPERATION-BASED EVALUATION PROTOCOL · C.5 SEARCH-BASED EVALUATION PROTOCOL · C.6 SHARED SETTINGS AND BENCHMARK COVERAGE

The protocols adapt operation-based judges to assess evidence from both modes and define separate search-answer grading, while shared execution settings and benchmark coverage support controlled evaluation. They also specify how action efficiency and access blocks are measured.

C.4 OPERATION-BASED EVALUATION PROTOCOL

  • Agents start in operation-based mode and are evaluated on all 300 Online-Mind2Web and 643 WebVoyager tasks, with Steps/SR counting actions from failed runs and switch calls.Steps/SR divides the mean action count over all retained task runs by their Kimi-K3 success rate; Switch is the fraction of tasks with an executed mode switch.Source: §C.4 · p. 19
  • Fara-7B’s Steps/SR is 61.5 on Online-Mind2Web and 34.9 on WebVoyager, versus the base model’s 30.9 and 20.8, when failed runs are counted.On tasks it solves, Fara-7B uses 12.2 and 10.0 actions on Online-Mind2Web and WebVoyager, respectively.Source: §C.4 · p. 20
  • For hybrid trajectories, WebJudge rates text observations against task key points on the screenshot-matching 1–5 relevance scale (threshold 3), while WebVoyager judges the last 15 observations in a merged chronological text-and-screenshot timeline.Both adapted prompts also ask judges to report site-blocking separately from success; this flag does not enter the success label.Source: §C.4 · p. 20
  • Both adapted judges separately flag CAPTCHA, bot detection, or region blocks, but this access-block flag does not affect the success label.The adaptations otherwise retain the official prompts verbatim.Source: §C.4 · p. 20

C.5 SEARCH-BASED EVALUATION PROTOCOL

  • Search-based agents start in search mode and are evaluated on HLE, xbench-DeepSearch-2510, and BrowseComp-EN, with full sets of 2,500, 100, and 1,266 tasks.Published baselines retain reported settings; HLE text-only scores are marked t, and scores are reported for xbench-DeepSearch-2510 only when that version is explicitly identified.Source: §C.5 · p. 20
  • Search-based answers are graded against references, accepting exact matches directly and deciding other answers by majority vote from five Kimi-K3 judge votes at temperature 0.6.QUEST and REDSearcher use reported context-management results on BrowseComp-EN.Source: §C.5 · p. 20

C.6 SHARED SETTINGS AND BENCHMARK COVERAGE

  • The base model and HYWE-35B share tools, prompts, sampling parameters, reasoning, and context compression, with 100-step operation and 300-step search attempt budgets.HYWEBENCH comparisons use a 100-step budget for all systems; SWITCH remains available unless an ablation disables it.Source: §C.6 · p. 20
  • Table 6 catalogs benchmark sizes and scoring protocols; Online-Mind2Web and WebVoyager assess operation, HYWEBENCH assesses hybrid completion, and three named benchmarks assess deep-search retention.Controlled comparisons evaluate every system on the same tasks.Source: §C.6 · p. 20, Table 6 · p. 20

D SWITCHING PATTERNS AND EXAMPLE TASKS · E TRAJECTORY REVIEW AND REFINEMENT · F PIPELINE ALGORITHMS

Sections D–F define four search/operation switching patterns, illustrate them with naturalized tasks, and describe how HyWeCollector synthesizes, reviews, and refines trajectories through coordinated execution.

D SWITCHING PATTERNS AND EXAMPLE TASKS

  • Medium and hard A3/A4 tasks chain phases across sites through one or two search-to-operation bridges, producing two to five switches.Difficulty is determined by the number of bound sites and cross-site bridge rounds.Source: D SWITCHING PATTERNS AND EXAMPLE TASKS · p. 21
  • A1 starts on a finance site but switches immediately to search because all requested stock metrics are available on indexed pages.The task compares three stocks on current price, market cap, P/E ratio, 52-week high and low, and average volume.Source: D SWITCHING PATTERNS AND EXAMPLE TASKS · p. 21
  • A2 searches for active Uniswap proposals but switches to the governance dashboard after search fails to surface live vote tallies.The task asks for proposals with deadlines on or after August 15, including status, For and Against counts, deadline, and quorum.Source: D SWITCHING PATTERNS AND EXAMPLE TASKS · p. 21
  • A3 uses search to identify Vienne’s nearest major TGV station and fare policy, then operation phases to map the drive and check transit routing and fare.Each operation phase consumes an output from search.Source: D SWITCHING PATTERNS AND EXAMPLE TASKS · p. 21
  • A4 uses operation to find Apple’s senior ML engineer listing, search to bridge its role family, and operation to find the matching Airbnb opening.The Apple listing must include PyTorch and TensorFlow as required skills.Source: D SWITCHING PATTERNS AND EXAMPLE TASKS · p. 21

E TRAJECTORY REVIEW AND REFINEMENT

  • Reviewers and refiners verify and improve executor trajectories, and every judge quotation is mechanically checked against the trajectory; unlocatable quotations void the judgment.Table 8 lists the checks, inputs, and outputs.Source: E TRAJECTORY REVIEW AND REFINEMENT · p. 21
  • HyWeCollector probes sites, filters and labels facts, composes A1–A4 blueprints, naturalizes validated contracts, then audits execution and removes leakage or invalid trajectories.Unsuccessful tasks are discarded; leakage is rewritten and re-verified, while trajectories failing site fidelity, switch rationale, or answer coverage are rejected or repaired by merging earlier-phase fields.Source: E TRAJECTORY REVIEW AND REFINEMENT · p. 23

F PIPELINE ALGORITHMS

  • The controller approves switches before new segments open, carries verified bindings forward, and prompts the sanitizer when phase budgets expire or progress stalls.A stall means no progress for three actions, or eight actions since the last check; detected human-verification gates terminate execution.Source: F PIPELINE ALGORITHMS · p. 24

G TRAINING DETAILS · H BASELINE DETAILS

Training data are rendered as multi-turn SFT samples using inference-matched processing, with long examples preserved through splitting. Baseline reporting distinguishes published scores from same-task HyWeBench evaluations and documents evaluation scope and controls in the appendix.

G TRAINING DETAILS

  • Both data sources become multi-turn SFT samples through the same code paths used at inference.System prompts, history blocks, context compression, and screenshot placement follow Section 3.1.Source: G TRAINING DETAILS · p. 25
  • Samples exceeding the context budget after folding are split into two or three parts by greedy bisection rather than dropped.This preserves long-horizon supervision.Source: G TRAINING DETAILS · p. 25
  • Table 9 presents the training configuration, but the supplied passage does not specify its settings.The table caption identifies its scope only.Source: Table 9 · p. 25, G TRAINING DETAILS · p. 25

H BASELINE DETAILS

  • Baseline descriptions appear in Tables 1, 2, 3, and 4, while published benchmark scores serve as external references.These are distinguished from the paper’s same-task HyWeBench comparisons.Source: H BASELINE DETAILS · p. 25
  • HyWeBench evaluations compare agents on the same tasks and completion checklists.Source: H BASELINE DETAILS · p. 25
  • Appendix C describes evaluation protocols and benchmark-coverage differences, while Appendix C.2 gives base-model and framework controls.Source: H BASELINE DETAILS · p. 25

H.1 OPEN-SOURCE OPERATION-BASED AGENTS · H.2 OPEN-SOURCE SEARCH-BASED AGENTS · H.3 FRONTIER LLM SYSTEMS

The appendix describes open-source operation- and search-based agents included in the comparisons, alongside frontier-system references and frontier models evaluated in HYWE on HYWEBENCH.

H.1 OPEN-SOURCE OPERATION-BASED AGENTS

  • Fara-7B and UI-TARS-1.5-7B represent visual operation agents that predict browser actions from screenshots, with synthetic-interaction training and GUI reasoning, respectively.Fara-7B uses the released model; UI-TARS tests whether a general GUI agent can solve web tasks through browser interaction.Source: §H.1 · p. 25
  • GUI-Owl-1.5 and MolmoWeb-8B add cross-environment interaction and rendered-page visual grounding to the operation-based baseline set.GUI-Owl-1.5-8B-Thinking is used on Online-Mind2Web and WebVoyager, while the 32B variant is included on WebVoyager and HYWEBENCH; no 32B Online-Mind2Web result is published.Source: §H.1 · pp. 25–26

H.2 OPEN-SOURCE SEARCH-BASED AGENTS

  • Tongyi DeepResearch and OpenSeeker-v1 provide search-agent baselines trained on long search-and-reading trajectories and synthetic multi-step-search trajectories, respectively.Tongyi is a 30B-class mixture-of-experts model with roughly 3B active parameters; OpenSeeker-v1 uses about 11.7K synthetic trajectories and supervised fine-tuning.Source: §H.2 · p. 26
  • REDSearcher and FORT-Searcher construct difficult search tasks by controlling evidence connections and removing easy routes, using reinforcement learning and supervised fine-tuning, respectively.REDSearcher also trains for evidence grounding, planning, and tool use with simulated rollouts; FORT-Searcher trains on refined trajectories using supervised fine-tuning alone.Source: §H.2 · p. 26
  • MiroThinker-1.7-mini and QUEST-35B target sustained deep research through planning and accumulated-context reasoning or synthetic tasks with rubric-based rewards.QUEST combines mid-training, supervised fine-tuning, and reinforcement learning for evidence gathering and report writing.Source: §H.2 · p. 26
  • Agents-A1 and XYZ-Aquila-mini are evaluated both on deep search and, in search-based configurations, on HYWEBENCH; Agents-A1 matches HYWE-35B’s total parameter scale.XYZ-Aquila-mini shares HYWE-35B’s backbone family, and its pipeline uses connected web evidence, supervised trajectories, and verified evaluation feedback.Source: §H.2 · pp. 26–27

H.3 FRONTIER LLM SYSTEMS

  • Frontier references cover operation-based systems paired with browser interfaces and deep-search models whose published scores reflect tool-assisted information gathering and reasoning.The cited tables retain differences in evaluation sets through annotations and Appendix C descriptions.Source: §H.3 · p. 27
  • Kimi-K3, Qwen3.8-Max, and Qwen3.7-Plus are evaluated zero-shot within HYWE on HYWEBENCH with access to both search-based and operation-based modes.These are the authors’ evaluations, separate from the published frontier reference rows.Source: §H.3 · p. 27

I FRAMEWORK IMPLEMENTATION DETAILS … I.3 CONTEXT COMPRESSION

The implementation defines mode-specific and shared tool interfaces, shared runtime parameters, and a context-compression procedure that controls how long interaction histories are retained.

I.1 TOOL FUNCTIONS AND SIGNATURES

  • The tool interface groups functions into search-based, operation-based, and shared tools; search APIs return text, while browser actions return a screenshot and status.The reason for a switch appears in the preceding reasoning.Source: §I.1 · p. 27, Table 10 · p. 28
  • Browser coordinates use a 0–1000 logical grid over a 1280 × 720 viewport and are converted to pixels at runtime; this convention is learned from data, not specified in the prompt.Table 10 groups tool functions and arguments by mode and marks required arguments in bold.Source: §I.1 · p. 27, Table 10 · p. 28
  • The search and scholar tools use Serper, while the visit tool fetches page content through Jina and summarizes it with deepseek-flash-0731 according to the extraction goal.Source: §I.1 · p. 27

I.2 RUNTIME PARAMETERS

  • Table 11 lists runtime parameters shared by the base and fine-tuned rows of Table 1.The supplied passage identifies the shared parameter set but does not provide its values.Source: §I.2 · p. 27, Table 11 · p. 28

I.3 CONTEXT COMPRESSION

  • Context compression triggers when rendered context exceeds 131,072 tokens, then folds steps until it is at most 65,536 tokens; the trigger reserves room for generation.The token count includes text, image tokens, and chat-template overhead; the effective high watermark also depends on the serving window and maximum output length.Source: §I.3 · p. 27
  • Folding truncates an observation to 600 characters and drops its reasoning and screenshot, proceeding oldest-first while preserving the newest step and tool calls.Evidence-cited steps are exempt, and screenshots remain only for unfolded steps; search-mode observations never contain images. Folding to the lower watermark avoids repeatedly rewriting the prefix and invalidating the key–value cache.Source: §I.3 · p. 27

J DATA COLLECTION IMPLEMENTATION DETAILS · J.1 NATURALIZATION VALIDATORS · K SYSTEM AND EVALUATION PROMPTS

The appendices specify HyWeCollector’s language models, task-naturalization checks, two mode-specific agent prompts, and a reference-answer judging prompt. Together, these details constrain task fidelity, tool use, safe recovery, output format, and answer matching.

J DATA COLLECTION IMPLEMENTATION DETAILS

J.1 NATURALIZATION VALIDATORS

  • Naturalized tasks are resampled after any validation failure, with checks covering blueprint fidelity, site scope, switching, answer schema, dependencies, and downstream-value leakage.The checks also require non-trivial questions, complete component goals and switch triggers, and explicit target sites for medium and hard A3/A4 tasks.Source: §J.1 · p. 29

K.1 MODE-SPECIFIC SYSTEM PROMPTS

  • Both modes share an output format requiring one tool call or one final answer per turn, with reasoning separated into tagged think and answer or tool-call blocks.The answer-shape instruction allows operation mode to report the reached end state, while search mode requests a concise entity, number, or list.Source: §K.1 · pp. 30–31
  • Search mode uses search, scholar, visit, code_sandbox, and switch; it treats snippets as leads, verifies decisive facts via visit, and redirects stalled searches toward alternatives or unverified constraints.The prompt also prohibits near-duplicate queries and retrying failed URLs, and advises working constraints in parallel.Source: §K.1 · p. 30
  • Operation mode acts on live pages with browser controls and grounds actions in screenshots, while switching back to search when interaction cannot reach needed information.The prompt requires stopping at human-verification gates and ending with a brief account of the block and prior progress; it also directs agents to verify state changes rather than assume success.Source: §K.1 · pp. 30–31

K.2 SEARCH-BASED ANSWER JUDGING

  • The reference-answer judge extracts the agent’s committed final answer, accepts semantic or straightforward formatting equivalence, and rejects conflicting, missing, materially incorrect, or unsupported answers.It returns JSON with the extracted answer, brief reasoning, and a predicted label.Source: §K.2 · pp. 31–32

L FULL TASK PROTOTYPE VOCABULARIES

HyWe’s typed search and operation task-prototype vocabularies define reusable components for composing hybrid tasks. They cover interactions with typed outputs and information needs ranging from focused verification and comparison to multi-source synthesis and cross-site workflows that connect one site’s results to another’s operations.

L Full Task Prototype Vocabularies

  • A search–operation pair is admissible when the search prototype’s output type belongs to the operation prototype’s accepted input types.Operation prototypes declare typed interfaces; the passage lists output types for both search and operation prototypes.Source: L FULL TASK PROTOTYPE VOCABULARIES · p. 33
  • The operation vocabulary contains 48 prototypes, and the search vocabulary contains 36 prototypes.The tables list the full vocabularies; the search prototypes also serve as producers, consumers, or cross-site bridges.Source: Table 12 · p. 33, Table 13 · p. 37
  • Search prototypes cover information needs such as finding thresholds, resolving classifications, and checking official policies or eligibility rules.These needs can identify thresholds for interactive tools or use a surfaced entity or result to determine applicable rules.Source: L FULL TASK PROTOTYPE VOCABULARIES · p. 38

Related papers

Loading 2610.16319v1…