PaperlayerView exact revision on arXiv

Source-linked AI summary

WebGuard: Building a Generalizable Guardrail for Web Agents

Boyuan Zheng, Zeyi Liao, Scott Salisbury, Zeyuan Liu, Michael Lin, Qinyuan Zheng, Zifan Wang, Xiang Deng, Dawn Song, Huan Sun, Yu Su

arXiv:2507.14293v1cs.AIcs.CLcs.CV

TL;DR

Web-agent safety lacks large-scale datasets for evaluating guardrails that assess action risk before execution. WebGuard provides real-world action annotations and evaluates guardrail models; fine-tuning substantially improves Qwen2.5-VL-7B, but the paper says reliability remains short of what deployment requires.

  • Problem

    High-quality, large-scale datasets for developing and evaluating web-agent guardrails that assess action risk before execution are lacking.

    Source: §1 · p. 2

  • Method

    WebGuard annotates web-agent actions with three-tier risk labels and evaluates prompted and fine-tuned models on predicting action risk from webpage state and a proposed action.

    Source: §1 · p. 2, §3.1 · p. 5

  • Results

    Fine-tuned Qwen2.5-VL-7B raises accuracy from 38% to 80% and high-risk recall from 20% to over 80%.

    Source: §1 · p. 3

  • Takeaways & Limitations

    WebGuard offers a basis for evaluating guardrail generalization across actions, websites, domains, and low-resource settings.

    Source: §6 · p. 9

    The models remain below dependable deployment reliability, especially on unseen domains, and have not been extensively tested in complex real-world scenarios.

    Source: G Limitations · p. 20

Abstract

from arXiv · show

The rapid development of autonomous web agents powered by Large Language Models (LLMs), while greatly elevating efficiency, exposes the frontier risk of taking unintended or harmful actions. This situation underscores an urgent need for effective safety measures, akin to access controls for human users. To address this critical challenge, we introduce WebGuard, the first comprehensive dataset designed to support the assessment of web agent action risks and facilitate the development of guardrails for real-world online environments. In doing so, WebGuard specifically focuses on predicting the outcome of state-changing actions and contains 4,939 human-annotated actions from 193 websites across 22 diverse domains, including often-overlooked long-tail websites. These actions are categorized using a novel three-tier risk schema: SAFE, LOW, and HIGH. The dataset includes designated training and test splits to support evaluation under diverse generalization settings. Our initial evaluations reveal a concerning deficiency: even frontier LLMs achieve less than 60% accuracy in predicting action outcomes and less than 60% recall in lagging HIGH-risk actions, highlighting the risks of deploying current-generation agents without dedicated safeguards. We therefore investigate fine-tuning specialized guardrail models using WebGuard. We conduct comprehensive evaluations across multiple generalization settings and find that a fine-tuned Qwen2.5VL-7B model yields a substantial improvement in performance, boosting accuracy from 37% to 80% and HIGH-risk action recall from 20% to 76%. Despite these improvements, the performance still falls short of the reliability required for high-stakes deployment, where guardrails must approach near-perfect accuracy and recall.

1 Introduction

WebGuard addresses the lack of large-scale datasets for assessing web-agent action risks and developing guardrails, while evaluating current and fine-tuned models. Frontier LLMs struggle to detect high-risk actions, and fine-tuning improves performance but does not yet reach reliable-deployment levels.

1 Introduction

  • Web-agent safety research has not kept pace with inference-time search, autonomous exploration, and reinforcement learning, which increase website interactions and associated risks.Model errors such as hallucinated actions or grounding failures can also produce unintended interactions.Source: §1 · pp. 1–2
  • WebGuard provides 4,939 action-level risk annotations from 193 websites across 22 domains, including 15 long-tail websites.Its three-tier schema labels actions SAFE, LOW, or HIGH according to their impact on users.Source: §1 · p. 2
  • Frontier models achieve under 60% accuracy and under 60% recall for high-risk actions, often misclassifying consequential behaviors as SAFE or LOW.This exposes limitations in predicting action risk levels.Source: §1 · p. 3
  • Fine-tuned Qwen2.5-VL-7B raises accuracy from 38% to 80% and high-risk recall from 20% to over 80%, but performance remains insufficient for reliable deployment.The paper argues that real-world deployment demands near-perfect accuracy and recall.Source: §1 · p. 3

2 WebGuard Dataset

WebGuard is a real-world dataset of human-annotated web-agent actions, designed to support guardrail development and evaluation. Its risk schema classifies action consequences, and the curation process combines trained annotators, realistic action collection, and human review.

2 WebGuard Dataset

  • WebGuard provides human-annotated action-level risk labels for real-world web actions, addressing prior datasets’ reliance on simulated settings or coarse task-level labels.Its scope centers on the potential consequences of executing a particular action, rather than broader safety dimensions such as toxicity, fairness, or misuse prevention.Source: §2 · p. 3, §2.3 · pp. 4–5

2.1 Risk Schema

  • The three-tier schema assigns action risk according to potential consequences, ranging from trivial and reversible SAFE actions to significant or irreversible HIGH actions.LOW actions have minor, reversible consequences affecting only the individual user.Source: §2.1 · pp. 3–4

2.2 Data Curation Process

  • Curation uses website selection, action collection, and annotation review, with annotators exploring state-changing actions and revising labels when outcomes clarify risk.Three professional reviewers check label validity and snapshot integrity; the finalized data contain 1,108 HIGH-risk, 2,284 LOW-risk, and 1,564 SAFE actions.Source: §2.2 · p. 4

2.3 Comparison with Existing Work

  • Unlike prior datasets that often assess harmful tasks with coarse interaction-level labels, WebGuard annotates specific web actions and spans high-traffic and long-tail sites.The benchmark focuses on significant or irreversible action consequences as one aspect of web-agent safety, complementary to other safety research.Source: §2.3 · pp. 4–5

3 Guardrail Construction

WebGuard frames action-risk assessment as classifying a proposed web action from its current state and a three-level risk schema. It describes prompting and fine-tuning guardrails and their integration with human oversight.

3.1 Problem Formulation

  • The guardrail predicts SAFE, LOW, or HIGH risk from webpage state, proposed action, and a predefined risk schema.The schema supplies textual definitions and examples for each level.Source: §3.1 · p. 5

3.2 Guardrail Design

  • Webpage observations can include HTML, an accessibility tree, a screenshot, or a URL; actions target webpage elements and are grounded differently by modality.Screenshots use bounding boxes, while accessibility-tree actions use node indices with HTML snippets and element metadata.Source: §3.2 · pp. 5–6
  • Prompted guardrails analyze webpage context, predict action outcomes, then assign a schema-based risk label using visual or HTML-based inputs.Prompts include risk-level definitions and examples, with minimal adjustments between modalities.Source: §3.2 · p. 6
  • Fine-tuning trains Qwen2.5 and Qwen2.5-VL models on WebGuard observations, actions, and risk definitions to output the correct label.Because training rationales are unavailable, fine-tuned models output only the risk classification label.Source: §3.2 · p. 6

3.3 Integrating Guardrails with Web Agents

  • During agent inference, users set the unsafe-risk threshold; exceeding it pauses execution and lets users approve, reject, or revise the action.The guardrail evaluates proposed actions concurrently with the agent.Source: §3.3 · p. 6

4 Experiments

The experiments compare guardrails across four generalization settings and evaluate classification accuracy, risk-class recall, and average F1. Fine-tuning improves performance, while results vary by setting, modality, and reasoning approach.

4.1 Experimental Setup

  • The four test splits probe long-tail websites, unseen domains, unseen websites within familiar domains, and familiar environments with held-out actions.TestLong-Tail has 143 actions from 15 websites; TestCross-Domain has 1,669 actions from 39 websites; TestCross-Website has 650 actions; TestCross-Action has 495 actions from 102 websites.Source: §4.1 · p. 7

4.2 Evaluation Metrics

  • Evaluation reports three-class accuracy, HIGH- and LOW-risk recall, and average F1 to capture classification quality, risk sensitivity, and the precision-recall balance.The paper notes that missing HIGH-risk actions may have serious consequences, while misclassifying LOW-risk actions can cause unnecessary alerts.Source: §4.2 · p. 7

4.3 Results

  • 80.4% average accuracy was reached by fine-tuned Qwen2.5-VL-7B, more than double its zero-shot baseline, with HIGH-risk recall gains over 60 points across all splits.Improvements were more modest on Cross-Domain than on Cross-Website and Cross-Action.Source: §4.3 · p. 7, Table 2 · p. 8
  • Text-only models outperform multimodal counterparts zero-shot, but WebGuard-VL-7B achieves the best accuracy and HIGH- and LOW-risk recall after fine-tuning.The authors suggest structured A11y-tree and HTML inputs may help zero-shot reasoning, while task tuning improves multimodal performance.Source: §4.3 · p. 8, Table 2 · p. 8
  • Reasoning models achieve higher zero-shot accuracy than non-reasoning counterparts across splits, but both show lower HIGH-risk recall than their counterparts.The authors describe this as a potential calibration issue, with reasoning models adopting more conservative risk estimates.Source: §4.3 · p. 8, Table 2 · p. 8
  • Fine-tuned guardrails improve accuracy and both risk recalls across settings, and WebGuard-VL-3B exceeds o3 and Claude-3.7-Sonnet with extended thinking on those measures.The error analysis also finds that models can label intermediate steps in a state-changing process HIGH and overgeneralize from button semantics without grounded knowledge of webpage dynamics.Source: §4.3 · pp. 8–9

5 Related Work

Related work discusses risks from misuse, adversarial observations, and agent unreliability, then reviews agent guardrails and gaps in web-specific risk assessment.

Risks of Agents

  • Web agents may cause harm through malicious use, adversarial observations, or inherent unreliability, and this work studies risky behavior in benign conditions rather than intentional attacks.The authors identify adversarial conditions as a direction for future investigation.Source: §5 · p. 9

Agents Guardrails

  • Prior agent guardrails include internal-thought monitoring and specialized scaffolding, while web-focused approaches and realistic, consequential risk evaluation remain limited.The authors contrast WebGuard’s real-world dataset and generalization settings with approaches focused on synthetic safety specifications or retrieved rules.Source: §5 · p. 9

6 Conclusion

The conclusion presents WebGuard as a large-scale, action-level benchmark for generalizable web-agent guardrails. Fine-tuning improves results, but performance remains insufficient for fully reliable deployment.

6 Conclusion

  • WebGuard provides fine-grained risk annotations for 4,939 actions across 193 websites and 22 domains, including long-tail websites, supporting evaluation across multiple generalization settings.The authors contrast its real-world action-level annotations with prior simulated or task-level evaluations.Source: §6 · p. 9
  • Supervised fine-tuning improves results across evaluation splits, and smaller fine-tuned models can outperform much larger general-purpose models.The conclusion names Qwen2.5-VL-3B and GPT-4o and Claude 3.7 Sonnet as examples.Source: §6 · p. 9
  • The authors say performance remains insufficient for fully reliable deployment and present the work as a step toward robust, generalizable guardrails.They plan to open-source the dataset, annotation tools, and trained models.Source: §6 · pp. 9–10

Table of Content:

The appendices define the website domains used in selection, describe how annotations are collected and reviewed, and provide prompts for assessing action risk from webpage context.

A Website Selection Domain Definitions

  • Information, services, shopping, entertainment, and travel cover knowledge, interactive utilities, purchases, leisure, and travel-planning websites, respectively.The domain descriptions distinguish content consumption, interactive tools, transactions, entertainment, and travel services.Source: A Website Selection Domain Definitions · p. 15
  • Social and work domains cover user-interaction platforms and professional tools or services.Examples include messaging or posting platforms and productivity or job-related tools.Source: A Website Selection Domain Definitions · p. 15

C Guardrail Prompts

  • One prompt presents a screenshot with the target element highlighted and the proposed action; another supplies the webpage Accessibility Tree and action details.Both prompt variants also specify the element description, tagHead, and page URL.Source: C Guardrail Prompts · pp. 16–17
  • The guardrail prompts judge each proposed action for unintended or irreversible effects using SAFE, LOW, and HIGH consequence descriptions.The prompt asks the model to assess the action on its own, not the overall task or possible future steps.Source: C Guardrail Prompts · p. 17

D Error Analysis: Misclassified Intermediate Action

An example shows a model overestimating the risk of an intermediate action in a high-stakes workflow: it labels a reversible passport-application checkbox LOW when its ground-truth label is SAFE.

D Error Analysis: Misclassified Intermediate Action

  • The model predicts LOW for a passport-application acknowledgment checkbox, although the ground-truth label is SAFE.The checkbox confirms acknowledgment of privacy and disclaimer terms.Source: Figure 5 · p. 18
  • The checkbox is a reversible, standard procedural step with no immediate legal, financial, or ethical consequences.Its placement within a high-stakes workflow does not make this specific step consequential.Source: Figure 5 · p. 18
  • The example illustrates the model’s tendency to overestimate risk for intermediate steps within a broader state-changing operation.The error arises despite the action’s lack of immediate consequences as described in the example.Source: Figure 5 · p. 18

E Error Analysis: Overgeneralization Action Effect

An example shows the model overgeneralizing from a “Checkout” label: it predicts HIGH for a reversible action that only advances the booking process.

E Error Analysis: Overgeneralization Action Effect

  • The model predicts HIGH for clicking “Checkout,” although the action only advances the booking process and does not commit a transaction.The example’s ground-truth label is not stated in the supplied passage.Source: Figure 6 · p. 19
  • The action is trivially reversible, and the user can cancel or navigate back without an irreversible or financial consequence.The button’s label makes the action appear to finalize a transaction, but the described state transition does not.Source: Figure 6 · p. 19
  • The example attributes the error to reliance on surface cues rather than understanding the website’s actual state transition and consequences.“Checkout” is the surface-level cue in this case.Source: Figure 6 · p. 19

F Compute

Training and inference used a single four-GPU H100 node, while API-based inference used Azure for OpenAI models and AWS for Claude models.

F Compute

  • Training and inference experiments ran on a single node equipped with four H100 GPUs, each with 80 GB.The passage specifies a single GPU node with 4× H100 GPUs.Source: F Compute · p. 20
  • API-based inference used Microsoft Azure for OpenAI models.Source: F Compute · p. 20
  • API-based inference used AWS for Claude models.Source: F Compute · p. 20

G Limitations

The guardrails remain below the reliability needed for dependable real-world deployment, particularly on websites from unseen domains. They have not undergone extensive testing in complex real-world scenarios.

G Limitations

  • Both accuracy and recall for unsafe actions remain below the threshold required for dependable real-world deployment.The authors say safety-critical applications require near-perfect precision and recall because even a single misclassification can have serious consequences.Source: G Limitations · p. 20
  • Performance is especially suboptimal on websites from unseen domains.The passage identifies unseen-domain websites as a particular weakness.Source: G Limitations · p. 20
  • The guardrails have not undergone extensive testing in complex, real-world scenarios beyond this initial investigation.The authors characterize the work as an early exploration.Source: G Limitations · p. 20

Related papers

Loading 2507.14293v1…