SynthAgent: Adapting Web Agents with Synthetic Supervision

1 Introduction

• New websites often lack task demonstrations, while human-expert or manually scripted data collection is labor-intensive and difficult to scale.This leaves agents facing unfamiliar states or tasks when deployed in new environments.Source: §1 · pp. 1–2
• Existing synthesis methods face a grounding trade-off: weakly grounded tasks can hallucinate, while continuous task changes during execution can misalign earlier trajectory segments.The paper contrasts OS-Genesis’s single-step task proposals with Explorer’s continuous refinement.Source: §1 · p. 2
• SynthAgent explores web elements by category, refines tasks when observations conflict, then uses global context to refine trajectories before fine-tuning open-source agents.Task edits mitigate hallucinations but can introduce trajectory noise, which subsequent refinement addresses.Source: §1 · p. 2
• SynthAgent improves synthetic-data quality and downstream agent adaptation across benchmarks, highlighting task diversity, feasibility, and trajectory alignment.The paper presents these findings as evidence for jointly improving all three aspects of synthetic supervision.Source: §1 · p. 2

2.1 Web Agent

• Web-agent research combines reasoning and interaction, but many approaches rely on costly human annotation; web-agent reinforcement learning can also be impractical for complex sites due to online-training costs.ReAct interleaves reasoning and actions, while other work explores tools, planning, and observation-action alignment.Source: §2.1 · p. 2

2.2 Data Synthesis

• General synthetic-data methods bootstrap instruction-tuning or reasoning data, while some agent pipelines use test-set tasks and raise leakage concerns.Recent web-agent work has begun synthesizing both tasks and trajectories from scratch.Source: §2.2 · p. 3

2.3 Synthetic data for Web Agents

• Web-agent synthesis methods have limits including repetitive tasks, outdated offline tutorials, weak grounding, or hallucinated simulated environments; SynthAgent combines categorized exploration with task and trajectory refinement.Explorer and OS-Genesis are identified as the closest works, with the former producing noisy trajectories and the latter suffering hallucinations and inefficient coverage.Source: §2.3 · p. 3

3 Method

• The method synthesizes task–trajectory pairs in a previously unseen website environment, then adapts an open-source web agent through supervised fine-tuning.A website is treated as a partially observable environment, with observations and actions forming the trajectory.Source: §3 · p. 3

3.1 Task Synthesis

• Categorized exploration groups interactive elements by functional category, samples up to two unvisited elements per category, and proposes multi-step tasks grounded in resulting page transitions.The per-category budget limits dominance by dense page regions, while a URL pool tracks discovered pages for deeper exploration.Source: §3.1 · p. 4

3.2 Task Refinement

• Task refinement activates when task-observation conflicts indicate an invalid goal, missing essential details, or stalled progress, then revises the task while preserving its category and intent.Refinement concretizes missing details, aligns tasks with observations, and may simplify blocked tasks.Source: §3.2 · p. 5

3.3 Trajectory Refinement

• Because task edits can misalign earlier trajectory segments, SynthAgent post-processes the full trajectory against the final task to remove noise without re-interacting with the environment.Edits can remove or reorder steps, discard noisy trajectories, or keep aligned ones; uncertain swaps are rejected to avoid unintended state changes.Source: §3.3 · p. 5

3.4 Agent Fine-tuning

• Agent fine-tuning uses supervised learning to predict each next action from the task and historical observations and actions, with a context window of three.The context window is empirically set to three considering training cost and inference latency.Source: §3.4 · pp. 5–6

4.1 Experimental Setup

• Evaluation covers five WebArena websites and 136 live Online-Mind2Web websites, comparing SynthAgent with Self-Instruct, OS-Genesis, and Explorer.The study fine-tunes Qwen2.5-VL-7B-Instruct and UI-TARS-1.5-7B; synthesis baselines use GPT-4.1.Source: §4.1 · p. 6

4.2 Main Results

• Fine-tuning Qwen3 with SynthAgent data improves overall success on both benchmarks, despite smaller relative gains from its stronger starting performance.The authors interpret this as support for the pipeline's model-agnostic nature; Table 3 and Table 4 report Qwen3 results.Source: §4.2 · pp. 6–7, Table 3 · p. 7, Table 4 · p. 7
• SynthAgent gains +10.2 average absolute points over base models and +5.1 over OS-Genesis on WebArena.It consistently outperforms the synthetic-data baselines and narrows the gap to test-task SFT.Source: §4.2 · p. 6, Table 1 · p. 7
• Without synthesis for or retraining on targeted websites, SynthAgent outperforms baselines across three judges on Online-Mind2Web and is competitive with GPT-4.1.The benchmark spans 136 live websites, and absolute success rates remain modest.Source: Table 2 · p. 7, §4.2 · p. 7

5.1 Synthetic Data Quality

• SynthAgent achieves the highest task diversity, with synthesized tasks resembling human-written tasks and categorized exploration providing consistent functional coverage across websites.OS-Genesis scores 83 and Explorer 54; t-SNE proximity represents task similarity, and points farther apart indicate greater diversity.Source: Figure 3 · p. 8, §5.1 · p. 8
• SynthAgent completes 96.5% of trajectories after refinement, while Explorer exceeds the step budget on 68.3% and incurs 60% more OpenAI API cost.Explorer's continuous task changes are described as causing trajectories to exceed the step budget.Source: Table 6 · p. 8, §5.1 · pp. 8–9
• Trajectory quality reaches 82.6 (83.4 average across judges); reorder is 4.1% of edit operations, and reordered trajectories score 42% vs. 27% in win rates and 68.9 vs. 62.1 in quality.The method uses explicit independence checks before reordering.Source: §5.1 · p. 8, Table 8 · p. 9, Table 9 · p. 9

5.2 Ablation Study

• Removing categorized exploration, task refinement, or trajectory refinement drops performance by 3.54%, 4.87%, or 3.99%, respectively.Removing task refinement produces the largest drop; the authors describe task and trajectory refinement as synergistic because trajectory refinement mitigates noise from task edits.Source: §5.2 · p. 9

5.3 Data Scaling

• Average success rate rises from approximately 10.6 with 4% data to 20.8 with 100% data, although Map peaks at 40% while CMS and Gitlab keep improving.The tested range is 20 to 500 tasks per website, and the authors report that data quality remains consistent as synthesis scales.Source: Figure 4 · p. 9, §5.3 · p. 9

• SynthAgent targets adaptation to new environments where environment-specific tasks and demonstrations are scarce and synthetic tasks or trajectories can be low quality.The paper identifies task hallucinations and noisy collected trajectories as the quality issues it addresses.Source: §6 · p. 9
• Dual refinement of tasks and trajectories, together with categorized exploration, is the framework's approach to improving synthetic data quality and task synthesis.Categorized exploration is described as systematically covering web elements and interactions.Source: §6 · p. 9
• Extensive experiments and analysis demonstrate SynthAgent's effectiveness in adapting web agents to new environments.Source: §6 · p. 9
• The authors identify large-scale, diverse tasks as crucial for the emerging field of agentic reinforcement learning.Source: §6 · p. 9

• Static snapshots and simplified environments do not support the interactive exploration SynthAgent relies on for data synthesis.Live websites bring challenges including frequent change, CAPTCHA solving, security mechanisms, and copyright and privacy constraints on releasing synthesized data.Source: Limitations · p. 10
• Task and trajectory synthesis relies solely on GPT-4.1 without hyperparameter tuning, and does not optimize execution parameters or explore alternative prompting strategies and more advanced LLMs.Source: Limitations · p. 10
• Fine-tuning uses standard SFT on synthetic data; the paper does not explore DPO, online reinforcement learning, or richer supervision signals.The authors note these approaches may yield better performance or larger gains, but typically require verification mechanisms or reward models beyond the paper's scope.Source: Limitations · p. 10

• The research is conducted entirely in controlled, self-hosted environments.Source: Ethics Statement · p. 10
• The authors warn that live-site misuse could enable unauthorized scraping or account-related security issues.Source: Ethics Statement · p. 10
• The authors urge compliance with laws, website terms, and ethical guidelines, and call for safeguards and sensitive-information detection.Source: Ethics Statement · p. 10

Use of AI Assistants

• AI assistants were used mainly to correct grammar and improve sentence fluency; authors reviewed the generated content.Source: Use of AI Assistants · p. 10

A.1 Environment

• The action space distinguishes “none” for a completed task from “stop” for exceeding the step budget or encountering an unrecoverable error.Source: §A.1 · p. 13
• WebArena evaluation uses 226 tasks, one per template, excluding cross-website tasks; the full 812-task set is used for training with test-set tasks.Source: §A.1 · p. 13
• Online-Mind2Web covers 300 tasks across 136 live websites, with evaluation using an LLM judge and key-point, screenshot, and outcome judgments.Source: §A.1 · pp. 13–14
• Figure 5 depicts task refinement when execution progression reveals infeasibility, guided by the observed failure and current environment state.Source: Figure 5 · p. 14
• Tables 12 and 13 report trajectory-quality ratings by GPT-4.1 and Qwen3-30B, respectively.Table 11 lists available atomic low-level browser interactions and their descriptions for WebArena.Source: Table 11 · p. 13, Table 12 · p. 14, Table 13 · p. 14

A.2 Baseline Implementation

• Self-Instruct generated tasks from seed examples using standard prompting, while OS-Genesis used random exploration and single-step observations.OS-Genesis’s generation code was adapted to WebArena.Source: §A.2 · p. 14
• Explorer was adapted for WebArena, including authentication handling for login-gated pages unsupported by the original implementation.The authors describe the adaptation as a faithful implementation with minimal changes.Source: §A.2 · p. 14
• Explorer averaged 8.6 refinements per trajectory, and 68.3% of its trajectories exceeded the step budget.Trajectory inspection attributed this behavior to Explorer’s algorithmic design.Source: §A.2 · p. 14

A.3 Trajectory Quality Evaluation

• SynthAgent consistently produced higher-quality task-trajectory data than baselines across GPT-5.1, GPT-4.1, and Qwen3-30B judges.Absolute scores varied, but the relative method ranking remained stable across judges.Source: §A.3 · pp. 14–15

A.4 Task Generation

• Task generation categorizes visible web elements into functional groups, then uses low-level actions to generate high-level tasks.The structured categorization is intended to support contextually grounded actions across functional groups.Source: §A.4 · p. 15

A.5 Trajectory Generation

• Trajectory generation executes tasks step by step, refines tasks when observations reveal inconsistencies or stalls, then removes unachievable trajectories or reorders steps.The final refinement uses the task objective to assess the collected trajectory.Source: §A.5 · p. 15, Figure 6 · p. 15

A.6 Agent Framework

• The browser agent interleaves chain-of-thought reasoning with environment actions using URL, accessibility-tree, candidate-element, and screenshot observations.Execution ends on an explicit termination, a self-assessed unachievable task, or exhaustion of the 30-step budget.Source: §A.6 · pp. 15–16

A.7 Tasks Analysis

• Task quality is evaluated by an LLM judge for how broadly and non-redundantly generated tasks cover user intentions for a website.The evaluation uses the prompt identified as Table 19.Source: §A.7 · p. 16

Categorized Category Sets

• Categorized exploration identified task-relevant functional groups across environments, including navigation, search, account management, and content interaction.The examples span shopping, CMS, Reddit, GitLab, and Maps.Source: B Case Study · p. 16

Task Refinement

• When the page failed to reach the intended subcategory, task refinement changed the goal to finding the lowest-priced product in the accessible Health & Household category.The revised task aligned the high-level goal with the observed page context.Source: B Case Study · p. 16

Trajectory Refinement

• The refined trajectory kept 9 essential steps after removing 19 repeated attempts on an unresponsive sort option and consolidating redundant scrolling.The example task was to find the cheapest available product in Electronics; the refined sequence successfully completed it.Source: B Case Study · pp. 16–17