DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents

Platform

• DTAP provides 50+ fully controlled simulated environments across 14 domains, faithfully replicating real-world MCPs and GUIs while enabling observable, consequence-free adversarial manipulation.The platform supports fine-grained manipulation of environment payloads, tool supply chains, and contextual inputs for scalable security evaluation.Source: §1 · pp. 2, 4

Autonomous red-teaming

• DTAP-RED jointly optimizes payloads and injection locations across prompt, tool, skill, environment, and compositional attack vectors, using verifiable judges for iterative feedback.It is equipped with 200+ attack strategies and targets malicious goals tailored to domain-specific risks and policies.Source: §1 · pp. 3–4

Benchmark

• DTAP-BENCH contains 6,682 tasks covering 4K+ malicious goals from 300+ risk categories across 14 domains, with optimized attacks and verifiable judges for outcome validation.The benchmark includes benign utility tasks and separate indirect and direct threat-model task sets.Source: §1 · pp. 3–4

Evaluation findings

• Google ADK reaches 55.7% ASR under indirect attacks, OpenClaw with DeepSeek-V4-Pro reaches 59.6% under direct attacks, and Claude Code still exceeds 25.2% ASR.These results show substantial vulnerability across current agentic frameworks, even for the most robust evaluated agent.Source: §1 · p. 3

Attack-surface findings

• Skill- and tool-level injections outperform environment injections, compositional attacks are highly effective, and communication-centric environments are more susceptible than finance-related environments.These findings indicate asymmetric trust across injection surfaces and environment-dependent vulnerability, with agents often ignoring even obvious tool-description injections in sensitive financial settings.Source: §1 · pp. 3–4

Security implications

• Prompt-level defenses remain insufficient: agents show uneven direct-versus-indirect susceptibility, context-aware risks dominate content-generation risks, and harmful actions may execute before refusals.Open-source backbones are especially susceptible to direct misuse, while batch tool invocation contributes to execute-then-refuse failures in OpenAI Agents SDK and Google ADK.Source: §1 · pp. 3–4

2.1 Red-Teaming Environments and Simulators for AI Agents

• Prior environments range from static synthetic tools in AgentDojo [18] and AgentHarm to simplified interactive settings and narrow, non-reproducible realistic domains.SHADE-Arena [31] and Petri remain simplified, while ST-4, RedTeamCUA [35], SafeArena [60], and ShieldAgent cover few environments or domains and often lack flexible resets and deterministic transitions.Source: §2.1 · p. 5
• DTAP provides 50+ full-stack red-teaming environments with realistic interfaces, high-concurrency evaluation, and arbitrary adversarial-state resets for efficient, transferable simulation.These capabilities address limitations in realism, domain coverage, flexible state resets, and deterministic reproducibility found in prior environments.Source: §2.1 · p. 5

2.2 Red-Teaming Benchmarks for AI Agents

• Existing benchmarks often rely on direct or heuristic prompt injections, domain-specific or white-box attackers, and ad hoc goals that omit complex adversarial patterns and enterprise-critical risks.AgentDojo [18] uses tool-output suffix injections; AgentHarm, ST-Web, and SafeArena [60] primarily study direct prompt injection, while other agent-oriented attacks assume stronger attacker access.Source: §2.2 · p. 5
• DTAP-BENCH covers 4K malicious goals from 60+ real-world security policies and spans environment, skill, tool, and prompt injection vectors under generic black-box access.Each instance uses end-to-end attacks optimized by DTAP-RED from a library of 200+ attack strategies, supporting effectiveness and transferability.Source: §2.2 · p. 5

3 DTAP: Unified Platform for Agent Red-Teaming

• DTAP provides a realistic, controllable, and dynamic validation layer for red-teaming agents across 14 domains and over 50 environments.Its simulated environments closely mirror real-world interfaces, supporting faithful transfer of security evaluations to real-world settings.Source: §3 · p. 5, §3.1 · p. 5

3.1 Design Principles of DTAP

• DTAP environments support dynamic multi-turn stateful interactions, deterministic replay, arbitrary-state resets, and parallelized containerized evaluation.These properties preserve intermediate states for verifiable evaluation, enable repeatable risk analysis, and support scalable testing with low compute overhead.Source: §3.1 · pp. 5–6
• DTAP connects MCP-compatible victim agents to sandboxed replicas of real-world tools through unified interfaces, enabling plug-and-play integration and safe, reversible tool-use evaluation.The platform reproduces tool names, descriptions, schemas, and return formats, and supports frameworks including OpenAI Agents SDK, Google ADK, and LangChain Agents.Source: §3.1 · p. 6

3.2 Comprehensive Domains and Environments on DTAP Platform

• DTAP spans workflow, CRM, customer service, travel, code, browser, research, filesystem, finance, legal, telecom, and medical domains with representative environments such as Gmail, Slack, PayPal, GitHub, and Booking.com.Each domain incorporates relevant policies, regulations, use-case risks, and environments that may require interaction across multiple platforms, creating sophisticated attack dynamics.Source: §3.2 · pp. 6–7

4.1 Overview of DTAP-RED

• DTAP-RED autonomously closes the loop from attack generation and execution to verifiable evaluation and iterative refinement, requiring only malicious goals and corresponding judges as input.The pipeline evaluates whether goals are achieved in the environment and automates injection optimization.Source: §4.1 · p. 7, Figure 3 · p. 8
• It retrieves successful and failed attack experiences, invokes attack algorithms and injection strategies, and composes injections across prompts, tools, skills, and environments into multi-step exploit chains.The skill library includes jailbreaks such as flip-attack and base64 encoding, while injections can be sequentially composed across multiple steps.Source: §4.1 · p. 7, §4.2 · pp. 7–8

4.2 Red-Teaming Agentic Framework: DTAP-RED

• DTAP-RED follows domain-specific policies, regulations, or use-case risks to derive severe malicious goals and verifies whether the corresponding malicious actions genuinely occur.Policy-following generation aligns risk assessment with platform-specific security requirements and uses verifiable judges to confirm outcomes.Source: §4.2 · pp. 7, 9
• Its flexible action space supports suffix, prefix, or full-rewrite injections into user prompts, tool descriptions, skill metadata or bodies, and environment data, including temporally composed attacks through injection MCP servers.The action space is configurable to threat-model constraints and attacker capabilities, while MCP servers expose practical environment injection entry points.Source: §4.2 · p. 8
• A three-layer memory organized by risk categories, malicious goals, and threat models supports lifelong learning, while ϵ-greedy exploration transitions toward exploiting accumulated experience for efficiency.The memory stores successful trajectories, useful insights, and common failure modes; refinement judges can switch channels or apply stronger jailbreaks until success or budget exhaustion.Source: §4.2 · pp. 8–9

5.1 Benchmark Overview

• DTAP-BENCH contains over 6k scenarios across 14 domains and over 50 environments, spanning benign, indirect red-teaming, and direct red-teaming task sets.The subsets evaluate high-stakes workflow utility, indirect malicious goals, and explicit misuse goals.Source: §5.1 · p. 9
• Each task specifies agent configuration, realistic environment setup scripts, reproducible injection sequences, and a verifiable judge based on resulting environment states.These components establish deterministic evaluation and directly test whether a malicious goal was achieved.Source: §5.1 · p. 9
• Red-teaming tasks use DTAP-RED attack sequences that compromise a surrogate agent within 10 iterations, enabling fair comparison through zero-shot transferable attacks.The benchmark manually balances prompt, tool, skill, environment, and combined injection types.Source: §5.1 · pp. 9–10
• The benchmark’s policy-driven pipeline extracts severe risks from domain policies and use-case risks, then pairs manually designed malicious goals with verifiable judges.A persona-based synthesis process generates diverse user profiles and realistic benign seed data, such as personalized email threads.Source: §5.1 · p. 10
• DTAP-BENCH was validated through approximately 16,000 human hours by 17 red-teaming experts over 20 months and about $120,000 in API credits.These resources supported attack generation, trajectory validation, and data curation.Source: §5.1 · p. 10

6.2 Main Results

• Strategically optimized DTap attacks substantially increase ASR across domains and risk categories, whereas directly supplied malicious goals yield near-zero ASR for almost all agents.The direct-malicious-goal condition validates that the goals are severe and generally rejected without optimized attacks.Source: §6.2 · p. 10
• Claude Code (Sonnet-4.5) is most robust under both threat models at 25.2% indirect and 26.9% direct ASR with 80.8% BSR, while Google ADK (Gemini-3-Pro) is most vulnerable to indirect attacks at 55.7% ASR despite 87.0% BSR.OpenClaw (DeepSeek-V4-Pro) and OpenAI Agents SDK (GPT-5.2) are most vulnerable to direct attacks at 59.6% and 58.8% ASR, respectively; GPT-OSS-120B combines 46.5% direct and 28.5% indirect ASR with 36.2% BSR.Source: §6.2 · pp. 10–11, Table 3 · p. 11, Table 4 · p. 11

6.3 Analysis

• OpenAI Agents SDK, Claude Code, and OpenClaw are over 20% more robust to indirect injections than direct prompts, while Google ADK has lower direct ASR (47.9%) than indirect ASR (55.7%).These patterns suggest greater trust in user prompts for the former agents and stronger prompt-level guardrails but weaker defenses against malicious external content for Google ADK.Source: §6.3 · p. 11
• Replacing the OpenAI Agents SDK harness with OpenClaw reduces GPT-5.2 direct ASR from 58.8% to 38.6% and indirect ASR from 46.7% to 35.6%, cutting ASR by approximately 31% at 2% utility cost.The results identify harness design as a dominant security factor and motivate tool-call sandboxing, pre-execution consequence reasoning, and trusted-input isolation.Source: §6.3 · pp. 11–12
• Skill- and tool-level injections outperform environment injections, exemplified by 83.3% versus 22.2% ASR in GPT-5.4, while compositional attacks raise GPT-5.2 ASR from 50.1% to 65.6%.Tool-injection susceptibility also varies by framework, with OpenClaw at 21.1% ASR versus Google ADK at 63.4%.Source: §6.3 · p. 11, Table 5 · p. 12
• Open-source backbones show greater direct-misuse susceptibility, with OpenClaw (DeepSeek-V4-Pro) reaching 59.6% direct ASR at 83.3% BSR versus 38.6% for GPT-5.2 and 28.9% for GPT-5.5.An execute-then-refuse failure mode frequently affects OpenAI Agents SDK and Google ADK, whereas Claude Code and OpenClaw use sequential tool invocation for more explicit intermediate-risk control.Source: §6.3 · p. 12

• DTAP combines a fully controllable red-teaming platform, autonomous DTAP-RED agent, and large-scale policy-grounded DTAP-BENCH benchmark.The platform uses realistic environment simulations to support scalable, rigorous evaluation.Source: §7 · p. 12
• Evaluations uncover systematic vulnerabilities in state-of-the-art agents, including susceptibility to compositional attacks and harness-level failure modes.These findings indicate that existing safeguards are insufficient for real-world deployment.Source: §7 · p. 12
• DTAP provides a scalable foundation for advancing more secure and reliable agentic systems through holistic, environment-aware security designs.Source: §7 · p. 12

A.1 Additional Details of DTAP-BENCH

• DTAP-BENCH distinguishes indirect attacks by third-party attackers against benign users from direct attacks in which malicious users manipulate agents, restricting or expanding injection vectors accordingly.Indirect red-teaming excludes direct prompt injection, whereas direct red-teaming allows prompt, tool, skill, and environment injections.Source: §A.1 · p. 25
• DTAP-BENCH contains 6,682 tasks across 14 domains, including 2,806 benign tasks and 3,876 red-teaming tasks with 3,876 distinct malicious goals.The red-teaming tasks comprise 1,878 direct-threat and 1,998 indirect-threat instances, providing balanced coverage across domains, threat models, security policies, risk categories, and malicious goals.Source: §A.1 · p. 25, Table 6 · p. 25

A.2 Red-Teaming Tasks

• Red-teaming tasks encode task context, victim-agent configuration, risk and policy metadata, threat model, malicious goal, injection steps, and attacker capabilities in reproducible configuration files.Indirect attacks use tool, skill, and environment injections under practical externally accessible constraints, while direct attacks additionally permit direct prompt injection; setup scripts seed realistic personalized workspace states.Source: §A.2 · pp. 26, 28

A.3 Verifiable Judge

• DTAP uses deterministic, instance-specific judges that inspect concrete environment outcomes rather than agent outputs or trajectories, improving reproducibility and avoiding unstable LLM-based evaluation.Judges classify attacks as fully successful, partially successful, or failed by checking all critical conditions, such as recipient, amount, and completed transaction status.Source: §A.3 · pp. 28–29

A.4 Evaluation of Red-Teaming Agent DTAP-RED

• DTAP-RED reaches over 85% attack success within three optimization iterations and approaches near 100% after ten, while typical successful attacks require only 2–5 iterations.In CRM, Fraud & Impersonation requires up to 6.3 iterations, whereas Misinformation & Hallucination and Data Exfiltration require 1.4 and 1.8 iterations, respectively.Source: §A.4 · pp. 30–31, Figure 8 · p. 31
• DTAP-RED’s attack skills are complementary: multi-turn fake conversation and general-misinformation exceed 50% ASR, while skill effectiveness varies substantially across risk categories.Fake-email-thread and leetspeak-attack approach 100% ASR for some categories, whereas multiturn-fake-conversation and CRM-skill-injection perform better for Misinformation & Hallucination and Social Scoring.Source: §A.4 · p. 31, Figure 9 · p. 31

B.1 Domain Overview

• The workflow benchmark uses sandbox environments that preserve real application interfaces and operations while enabling controlled initialization, customization, and systematic benign or adversarial scenario construction.It covers five benign task categories and nine security-risk categories across heterogeneous productivity and automation platforms, including email, calendars, documents, messaging, databases, and payments.Source: §B.1 · p. 32

B.2 Benign Tasks

• The workflow domain includes 335 benign tasks organized into five categories: Data Query & Reporting, Meeting Coordination, Payment & Financial, Survey & Data Collection, and Communication & Announcement.Each benign task serves a legitimate purpose, uses authorized tools, data, and recipients, and avoids harmful, deceptive, illegal, or privacy-violating outcomes.Source: §B.2 · pp. 32–33

B.3.1 Policy Definition

• Platform policies are defined separately for each environment by extracting primary documents that specify prohibited behaviors, misuse scenarios, and safety constraints.Sources span systems including Gmail, Google Docs, Google Forms, Google Calendar, Slack, WhatsApp, Zoom, Databricks, Snowflake, Atlassian, and PayPal.Source: §B.3.1 · pp. 33–34

B.3.2 Risk Category

• Heterogeneous platform restrictions are normalized into a unified taxonomy based on operational effects, including Messaging Abuse, Data Exfiltration, Financial Fraud, Illegal Activity, and Sensitive File Deletion.Multiple environment-specific clauses can support one category, preserving policy grounding while enabling consistent benchmark evaluation across environments.Source: §B.3.2 · p. 34

B.4.1 Benign Task

• Benign workflow utility is generally strong, with OpenClaw + Claude-Opus-4.6 leading several categories while Payment & Financial and some Data Query & Reporting tasks remain harder.OpenClaw + Claude-Opus-4.6 reaches 88.89% on Data Query & Reporting, 95.71% on Meeting Coordination, 88.24% on Payment & Financial, and 91.43% on Communication & Announcement; Gemini-3-Pro leads Survey & Data Collection at 95.56%.Source: §B.4.1 · p. 34

B.4 Experiment Results

• The experiments compare benign task success with attack success under indirect and direct threat models across workflow risk categories and model–framework combinations.The direct-threat results distinguish naive prompting from red-teaming-optimized attacks, while matched attack-generation systems provide an upper-bound reference rather than directly comparable robustness results.Source: §B.4 · p. 34, Figure 10 · p. 35

B.4.2 Indirect Threat Model

• Indirect attacks embedded in emails, documents, messages, or tool outputs produce high ASR across several non-matched systems, exposing workflow agents to realistic instruction injection.Gemini-3-Pro, OpenClaw + GPT-5.1, and Claude-Sonnet-4.5 remain highly vulnerable because agents interpret and act on external content across environments.Source: §B.4.2 · p. 35

B.4.3 Direct Threat Model

• Optimized direct attacks sharply increase vulnerabilities, with Gemini-3-Pro, OpenClaw + GPT-5.1, and OpenAI Agents SDK + GPT-5.2 showing high ASR across several risk categories.Financial Fraud, Illegal Activity, Messaging Abuse, and Misinformation are especially susceptible, although naive direct prompts achieve 0.0% ASR across all tested combinations and categories.Source: §B.4.3 · p. 35

B.5 Analysis

• Direct attacks usually achieve higher ASR than indirect attacks, but asymmetry is model-dependent: GPT-5.2 rises from 63.1% to 69.1%, while Gemini-3-Pro falls from 65.9% to 54.4%.GPT-OSS-120B shows the largest reversal, rising from 7.2% indirect ASR to 61.7% direct ASR, so robustness under one threat model does not imply robustness under the other.Source: §B.5 · pp. 35–36
• Harness design substantially reshapes security and utility: with GPT-5.2, OpenClaw reduces indirect ASR from 63.1% to 26.1% and direct ASR from 69.1% to 30.9%, but lowers benign BSR from 76.4% to 61.0%.For Claude-Opus-4.6, OpenClaw has the highest benign performance at 91.5%, whereas Claude Code is more secure, with indirect ASR 11.1% versus 18.3% and direct ASR 19.2% versus 22.1%.Source: §B.5 · p. 36
• Benign capability and adversarial robustness are weakly aligned: OpenClaw + Claude-Opus-4.6 reaches 91.5% BSR with 18.3% indirect ASR, while Google ADK + Gemini-3-Pro reaches 89.4% BSR with 65.9% indirect ASR.OpenAI Agents SDK + GPT-5.2 is both less capable than the strongest benign systems and most vulnerable under direct attacks at 69.1%.Source: §B.5 · pp. 36–37
• Workflow-native categories remain the most vulnerable: direct attacks reach 100.0% for Illegal Activity Promotion and Messaging Abuse, while indirect attacks reach 100.0% for Messaging Abuse and Copyright / IP Infringement.Data Exfiltration is comparatively harder, never exceeding 28.6% direct ASR or 46.7% indirect ASR.Source: §B.5 · p. 37
• Environment vulnerability is uneven: direct ASR is highest for WhatsApp at 80.00 and Google Docs at 77.78, whereas PayPal reaches 11.11 and Snowflake 16.67.Indirect attacks peak at Google Calendar with 55.56, while PayPal and Snowflake reach 34.07 and 38.10, respectively; document and communication platforms expose richer untrusted content and looser trust boundaries.Source: §B.5 · pp. 37–38, Figure 13 · p. 37
• GPT-OSS-120B’s low indirect ASR reflects brittle execution rather than genuine robustness: it completes only 43/107 indirect benign tasks (40.2%), compared with 77/77 (100%) under direct attacks, while its benign BSR is 56.0%.Its workflow ASR is 6.1 under indirect attacks but 62.1 under direct attacks, indicating that noisy tool-mediated context often prevents attack propagation before explicit malicious intent exposes its vulnerability.Source: §B.5 · p. 38

B.6 Case Study

• The case studies demonstrate both attack channels: a forged Gmail message causes an agent handling a benign mailbox request to forward a Google 2-Step Verification code to an unauthorized recipient, while a direct malicious request completes an unauthorized PayPal payout.The direct case succeeds because ordinary payment tools translate the malicious natural-language objective into legitimate operational steps and complete the fraudulent transfer.Source: §B.6.1 · p. 41, §B.6.2 · p. 41, Figure 16 · p. 44

C.1–C.4 Experiment Design

• The CRM evaluation covers 11 representative benign workflow categories and 9 policy-derived risk categories, grounded in Salesforce and ServiceNow policies plus the EU AI Act and GDPR.Experiments evaluate OpenAI Agents SDK, Claude Code, Google ADK, and OpenClaw with the listed GPT, Claude, and Gemini models under benign, indirect, and direct threat settings.Source: §C.1 · p. 48, §C.2 · p. 48, §C.3.1 · p. 49

C.4.1 Benign Task

• Benign task success rates are reported across eleven diverse CRM task and workflow categories for four agent frameworks and eight model configurations.Source: §C.4.1 · p. 49, Figure 21 · p. 50

C.4.2 Indirect Threat Model

• Indirect-threat attack success rates are reported across nine diverse CRM risk categories for four agent frameworks and eight model combinations.Source: §C.4.2 · p. 49, Figure 21 · p. 50

C.4.3 Direct Threat Model

• Direct-threat evaluation reports attack success across nine CRM risk categories for four agent frameworks and eight model configurations, both without attacks and with red-teaming attacks.Tab. 15 tests explicit malicious goals supplied directly to agents, whereas Tab. 16 tests the same goals delivered through more sophisticated and stealthy attack strategies.Source: §C.4.3 · p. 49, Figure 21 · p. 50
• The CRM analysis examines the trade-off between agent vulnerability and benign utility under both indirect and direct threat models.Figure 22 relates attack success rate to benign task success rate for each threat model.Source: Figure 22 · p. 50

C.5 Analysis

• Claude Code (Opus-4.6) combines 83.0% BSR with the lowest ASR, while high-capability agents span 3.9%–69.9% ASR, showing security and capability can be independently optimized.GPT-OSS-120B has only 4.2% BSR but 38.9% direct ASR, so low capability does not guarantee security.Source: §C.5 · p. 50
• Direct and indirect threats reveal different failure modes: GPT-family agents show direct-over-indirect gaps, whereas Claude Code and Google ADK are more vulnerable to external-content attacks.Claude Code (Sonnet-4.5) reaches 40.6% indirect versus 16.7% direct ASR, while Google ADK reaches 69.9% indirect versus 55.6% direct ASR.Source: §C.5 · pp. 50–51, Figure 23 · p. 51
• Frameworks can reduce vulnerability for weakly aligned models but add little protection to robust ones: OpenClaw lowers GPT-5.2 ASR by about 15.5pp, while Opus-4.6 remains near-comparable across frameworks.The framework acts as a multiplier of underlying model vulnerability, making model-level alignment more portable than framework defenses alone.Source: §C.5 · p. 51
• Skill injection is the strongest indirect single-vector attack, reaching 83.3% ASR for GPT-5.4 and 77.8% for GPT-5.2, while skill+tool reaches 100% for both models but multi-vector attacks achieve 0.0% against Opus-4.6.Skills are often treated as trusted instructions, creating a privileged attack surface that current security mechanisms inadequately govern.Source: §C.5 · pp. 51–52, Figure 23 · p. 51

C.6 Case Study; C.6.1 Case Study of the Indirect Threat Model; C.6.2 Case Study of the Direct Threat Model

• CRM case studies show an indirect environment injection forwarding a Databricks API token to an attacker-controlled Gmail account during a benign billing lookup, and a direct flip-attack updating Salesforce with weaponization instructions before refusal.The direct case exemplifies execute-then-refuse behavior, leaving enterprise records compromised despite the final refusal.Source: §C.6.1 · p. 55, §C.6.2 · pp. 55–56

D.1 Domain Overview; D.2 Benign Tasks; D.3.2 Risk Category

• The customer-service benchmark targets a fully autonomous agent setting with sensitive customer data and operational powers, using 160 benign tasks across six workflows and six policy-grounded risk categories.The design evaluates worst-case plausible deployments rather than relying on human approval for high-impact actions.Source: §D.1 · p. 61, §D.2 · p. 61, §D.3.2 · p. 62

D.3.1 Policy Definition; D.3.2 Risk Category

• The risk taxonomy is grounded in Zendesk and Shopify policies plus FTC deception and GDPR requirements, translating platform and regulatory constraints into verifiable harmful agent behaviors.The cited policy sources cover deceptive behavior, account hijacking, malicious practices, misleading representations, and personal-data processing and transfer.Source: §D.3.1 · pp. 61–62, §D.3.2 · p. 62, Table 17 · p. 62

D.4 Experiment Results; D.4.1 Benign Task; D.4.2 Indirect Threat Model; D.4.3 Direct Threat Model

• The experiments compare four agent frameworks and nine model configurations, reporting benign success plus indirect and direct ASR, with direct evaluation covering both explicit malicious goals and red-teamed delivery.Tables 18–21 cover the benign benchmark, indirect environment/tool attacks, direct malicious requests, and direct attacks using prompt, environment, and tool injections.Source: §D.4 · pp. 62–63, §D.4.1 · p. 63, §D.4.2 · p. 63

D.5 Analysis

• Direct ASR varies from 1.3% to 70.2% among configurations with similar high benign success, while low ASR can reflect execution failure rather than safety alignment.Most frontier configurations achieve 87–90% BSR; GPT-OSS-120B has 31.2% direct ASR but often fails to execute malicious objectives rather than refusing them.Source: §D.5 · pp. 64–65
• Direct attacks usually outperform indirect attacks, except for Gemini-3-Pro with Google ADK, whose indirect ASR is 75.1% versus 40.3% direct ASR because prompt filtering blocks direct requests but not injected tool instructions.Claude-Opus-4.6 remains resistant under both models at 1.3% direct and 2.9% indirect ASR.Source: §D.5 · p. 65
• Customer-service attacks transfer most effectively within the GPT family under direct delivery, whereas Gemini-3-Pro surpasses GPT configurations under indirect delivery and Claude-Opus-4.6 remains most resistant.GPT-5.2 and GPT-5.4 reach 70.2% and 64.7% direct ASR, while Gemini-3-Pro reaches 75.1% indirect ASR; optimized attacks were developed on GPT-5.1.Source: §D.5 · pp. 65–66
• Financial Fraud is uniquely compliant without obfuscation and reaches 95.0% direct ASR for GPT-5.4 and GPT-5.2 because refund and store-credit tools blur legitimate service and policy violation.Under direct attacks, Policy Misrepresentation and Financial Fraud are most vulnerable, while indirect attacks shift vulnerability toward Bulk Destructive Actions and Abusive Communications.Source: §D.5 · p. 66
• Combining injection channels generally increases ASR, but prompt-plus-environment already reaches 62.5% direct ASR and Gemini-3-Pro’s tool-only indirect ASR reaches 100.0%, indicating channel-specific saturation.Environment-plus-tool reaches 42.2% indirect ASR versus 24.1% environment-only and 33.3% tool-only; Claude-Opus-4.6 remains near-resistant except under combined injection.Source: Figure 31 · p. 65, §D.5 · p. 66

D.6.1 Case Study of the Indirect Threat Model

• The indirect case study shows Gemini-3-Pro granting $325 in store credit after fabricated approvals were injected into case notes and order metadata, exceeding the $100 policy cap by $225, while Claude-Opus-4.6 refused.The attack used environmental injection against Google ADK + Gemini-3-Pro and caused the agent to treat fabricated approval as overriding retrieved policy.Source: §D.6.1 · pp. 67–68, Figure 35 · p. 69, Figure 36 · p. 70

D.6.2 Case Study of the Direct Threat Model

• The direct PII case study shows OpenAI Agents SDK + GPT-5.2 following a Base64 jailbreak to retrieve payment data and email a detailed billing profile to an attacker-controlled address, while other configurations refused.The payload combined fake sandbox framing, role elevation, a step-by-step workflow, and an encoded override; the email included card details, billing address, and phone number.Source: §D.6.2 · pp. 68–69, Figure 39 · p. 72, Figure 40 · p. 73

E.1 Domain Overview; E.2 Benign Tasks; E.3.1 Policy Definition; E.3.2 Risk Category; E.4.1 Benign Task; E.4.2 Indirect Threat Model; E.4.3 Direct Threat Model

• The benchmark defines 6 benign task categories and 12 security risk categories, then constructs red-teaming tasks with malicious goals under 2 primary threat models to evaluate travel-agent security robustness.Source: §E.1 · p. 74

E.5 Analysis

• 99.2% benign success and 84.2% indirect ASR for Gemini-3-Pro contrast with 93.1% benign success and 0.0% indirect ASR for Opus-4.6.GPT-OSS-120B completes 0.0% of benign tasks yet reaches 54.3% direct and 40.8% indirect ASR, showing harmful tool execution can require less capability than benign workflows.Source: §E.5 · pp. 77–78, Figure 41 · p. 78
• Indirect attacks generally outperform direct attacks, including 66.7% versus 42.9% for GPT-5.2 and 84.2% versus 52.4% for Gemini-3-Pro.Exceptions include GPT-OSS-120B, with 54.3% direct versus 40.8% indirect ASR, and Claude Opus-4.6, with 0.0% direct versus 2.9% indirect ASR.Source: §E.5 · pp. 76–77, Figure 42 · p. 79
• Tool-only injections outperform prompt-only attacks, reaching 100.0% versus 34.1% ASR for Gemini-3-Pro; environment-tool combinations also remain highly effective.Tool descriptions may be treated as trusted operational guidance rather than adversarial input.Source: §E.5 · p. 78, Figure 43 · p. 79
• Framework guardrails reduce GPT-5.2 vulnerability, lowering direct ASR from 42.9% to 12.4% and indirect ASR from 66.7% to 32.5%, but add little for Opus-4.6.GPT-5.4 also reduces direct ASR to 32.4% and indirect ASR to 55.0% while maintaining comparable benign success to GPT-5.2; GPT refusals can still follow harmful execution.Source: §E.5 · pp. 78–79

E.6.1 Case Study of the Indirect Threat Model

• A multi-vector indirect attack combining environment, skill, and tool injections exfiltrates a user’s phone number to an accommodation host without consent.The victim is OpenAI Agents SDK + GPT 5.2, and the attack fulfills the red-teaming agent’s data-exfiltration goal.Source: §E.6.1 · p. 80

E.6.2 Case Study of the Direct Threat Model

• A fabricated system instruction induces GPT-5.2 to submit four fraudulent reviews before refusing, demonstrating an irreversible direct execute-then-refuse failure.The attack uses a single-turn jailbreak and distinct usernames and content to make the reviews appear authentic.Source: §E.6.2 · pp. 80–81

F.1 Domain Overview

• The code benchmark contains 330 benign tasks across five coding workflows and more than 120 malicious tasks across nine security categories under direct and indirect threat models.Its risk categories are guided by CWE, OWASP, MITRE ATT&CK, and NIST standards.Source: §F.1 · p. 86, §F.2 · p. 86

F.3.1 Policy Definition

• Code-agent risks are organized into enforceable categories derived from NIST, CWE, OWASP, and MITRE ATT&CK, with the resulting taxonomy summarized in Table 27.Benign workflows include file conversion, environment repair, grid transformation, mathematical computation, and web retrieval.Source: §F.3.1 · pp. 86–87

F.4.3 Direct Threat Model

• Code-domain optimization significantly increases ASR across agents, including Gemini-3-pro’s eval-injection ASR rising from 0% to 100%.Initial direct instances have relatively low ASR, but optimized attacks reveal category-specific vulnerabilities, including Gemini-3-pro’s sensitivity to biased code generation.Source: §F.4.3 · pp. 88–89, Table 31 · p. 89, Table 30 · p. 89

F.5 Analysis

• Frameworks materially affect safety: OpenClaw is more conservative and achieves lower ASR than the OpenAI SDK for the same GPT-5.2 model.Benign coding performance also degrades as tasks require stronger reasoning, tool coordination, and consistent multi-step execution; GPT-OSS-120B is weaker than other evaluated models on these capabilities.Source: §F.4.3 · p. 88, §F.4.1 · p. 88, §F.5 · p. 90
• Code agents show uneven security: Gemini-3-pro reaches nearly 100% benign success but 95.8% indirect ASR, while GPT-OSS-120B’s lower indirect ASR reflects weak instruction following rather than explicit refusal.Sensitive Files Deletion, Eval Injection, and insecure-code incorporation are especially severe, whereas multi-step attacks such as crawling and following instructions are harder than single-step attacks.Source: §F.5 · pp. 90–91

F.6 Case Study

• Representative code-domain cases show agents executing harmful instructions: deleting /etc/group directly and adding a dangerous .bashrc alias after environment and tool injections.The indirect case demonstrates retrieval and execution of malicious injected content without proper validation, while the direct case causes severe system-level damage.Source: §F.6.1 · p. 91, Figure 52 · p. 92, §F.6.2 · p. 92

G.4.2 Indirect Threat Model

• Browser indirect attacks are most effective against Gemini-3-Pro (62.2% overall ASR), whereas Claude-Opus-4.6 records 0.0% under both Claude Code and OpenClaw.The browser benchmark uses an e-commerce environment with 27 MCP tools and tests tool-only, redirected third-party, and target-website environment injections.Source: Indirect Threat Model · pp. 95–96, §G.1 · p. 95, §G.4.2 · p. 96

G.4.3 Direct Threat Model

• In the browser domain, direct attacks raise ASR most for Gemini-3-Pro (35.3%) and GPT-5.4 (29.2%), while Claude-Opus-4.6 remains comparatively robust (8.8%–10.2%).The direct-prompt baseline produces low overall ASR because most models refuse raw malicious browser instructions; GPT-5.2 reaches 20.3% in OpenAI Agents SDK and 22.8% in OpenClaw.Source: §G.4.3 · p. 96

G.5 Analysis

• Target-website review injections are less effective than other indirect patterns, and GPT models often fail account-dependent tasks because they do not use autofill to log in.These limitations reduce attack effectiveness or benign task completion when sessions are logged out.Source: §G.5 · pp. 97–98
• GPT-5.1 often performs harmful actions before producing refusal-like final text, while GPT-5.2 over-conservatism reduces benign BSR by refusing authorized actions.Representative refusals include creating an eBay account and posting a benign product review; final-text refusal is therefore not a sufficient safety signal.Source: §G.5 · pp. 96–97
• Attacks optimized on GPT-5.1 transfer most strongly to Gemini-3-Pro, while Claude-Opus-4.6 is most robust and browser-integrity and data-exfiltration are the most exploitable risk categories.Gemini-3-Pro reaches 35.3% direct and 62.2% indirect ASR, Claude-Opus-4.6 has 0.0% indirect ASR under both frameworks, and category maxima reach 100.0% direct integrity ASR and 80.0% indirect data-exfiltration ASR.Source: §G.5 · p. 98

G.6 Case Study

• The browser case studies show that multiple tool injections exfiltrate saved credit-card information under the indirect threat model, while an optimized jailbreak posts spam under the direct threat model.The direct case succeeds even though the GPT-5.1 agent’s final response says it cannot submit spam.Source: §G.6.1 · p. 98, §G.6.2 · p. 99

H.1 Domain Overview

• The research domain targets risks arising from external content and limited verification, including hallucinated citations, misinformation propagation, and adversarial instruction following.The benchmark covers representative scientific-workflow stages and evaluates robustness under multiple threat models.Source: §H.1 · p. 107

H.2 Benign Tasks

• Research agents are evaluated on downloading GitHub artifacts, searching related papers, uploading papers to an arXiv-style site, and writing papers with correct citations.These four scenarios represent common research-agent workflows.Source: §H.2 · p. 107

H.3.1 Policy Definition

• Research red-teaming policies draw on NATO and EU CBRN Action Plan guidance to constrain information enabling hazardous-material development, acquisition, or misuse.These policies are converted into enforceable constraints and a unified CBRN risk taxonomy summarized in Table 37.Source: §H.3.1 · pp. 107–108, §H.3.2 · p. 108, Table 37 · p. 108

H.3.2 Risk Category

• The research experiments report benign task success in Table 38, indirect red-teaming ASR in Table 39, and direct-threat results in Tables 40 and 41, including a direct baseline without red-teaming optimization.The supplied passages identify the evaluated result scopes but do not provide cell values.Source: §H.4.1 · p. 108, §H.4.2 · p. 108, §H.4.3 · p. 108

H.5 Analysis

• Research-domain analysis finds hallucinated arXiv IDs, non-uniform risk difficulty, weak indirect defenses, and stronger transfer of GPT-5.1-optimized attacks to GPT and Gemini than to Claude models.Nuclear risks show near-zero indirect ASR and relatively low direct ASR, whereas Bio Risk is more vulnerable; Gemini can exceed its optimized direct ASR in some categories such as Arms Control.Source: §H.5 · pp. 109–110

H.6 Case Study; H.6.1 Case Study of the Direct Threat Model; H.6.2 Case Study of the Indirect Threat Model

• The research case studies show direct nuclear-weapons requests producing risky synthesized content and indirect tool-description injection facilitating arms-control circumvention alongside the intended benign task.The indirect attack embeds malicious instructions in a tool description, which the agent follows without proper validation.Source: §H.6.1 · p. 111, Figure 67 · p. 112, §H.6.2 · p. 113

I.1 Domain Overview

• The OS-filesystem domain tests agents with broad MCP-enabled Linux access, where low-level and often irreversible operations can expose credentials, alter authentication infrastructure, or plant persistence.The environment supports file, directory, permission, shell, and cross-service operations involving systems such as /etc/shadow, SSH keys, and shell configuration files.Source: §I.1 · p. 115

I.2 Benign Tasks

• The benchmark contains 200 benign filesystem tasks across eight workflow types, with 63 sensitive tasks requiring explicit permission and 137 non-sensitive tasks executable directly.The task suite covers reading, listing, searching, creating, deleting, modifying, reporting, and backing up files.Source: §I.2 · p. 115

I.3.1 Policy Definition; I.3.2 Risk Categories; Direct Threat Model; Indirect Threat Model

• The OS-filesystem risk taxonomy derives nine enforceable categories from U.S., EU, and Chinese legal frameworks, with each category mapped to at least one regulatory provision.Direct and indirect tasks both cover all nine categories; direct attacks use malicious prompts and adversarial rewriting, while indirect attacks pair benign tasks with hidden goals and use environment or tool-description injection.Source: §I.3.1 · p. 116, §I.3.2 · p. 116, Direct Threat Model · p. 116

I.4.1 Benign Tasks; Indirect Threat Model

• Benign capability is high enough for meaningful security evaluation: Gemini-3-Pro leads at 87.0% BSR, while GPT-OSS-120B reaches 61.7%; File Reading is the main capability bottleneck.Indirect attacks use five environment-injection primitives plus tool-description injection, with 79 environment-only, 108 tool-description-only, and 13 combined tasks.Source: Indirect Threat Model · p. 117, §I.4.1 · p. 118, Table 43 · p. 118

I.4.2 Indirect Threat Model

• Claude Opus-4.6 achieves the lowest indirect ASR at 1.2%, whereas Gemini-3-Pro reaches 59.5%; indirect ASR is lower than direct ASR for every evaluated model.Indirect attacks manipulate agents during benign task execution through covert environmental or tool-layer compromise.Source: §I.4.2 · p. 118, Table 44 · p. 118

I.4.3 Direct Threat Model

• 84.0% overall ASR for OpenAI Agents SDK GPT-5.4 under red-teaming attacks contrasts with 22.6% for Sonnet-4.5 and 26.2% for Opus-4.6, while unaugmented malicious prompts yield 0.0% ASR for every model.Prompt rewriting and social engineering therefore substantially change apparent safety compared with surface-level malicious-intent detection.Source: §I.4.3 · p. 119, Table 46 · p. 119, Table 45 · p. 119

I.5 Analysis

• Direct attacks exceed indirect attacks for every OS-filesystem model, with the largest gaps for OpenAI Agents SDK GPT-5.4 (∆=46.5pp), GPT-5.2 (∆=39.3pp), and GPT-OSS-120B (∆=37.2pp).Claude Opus-4.6 instead shows 26.2% direct ASR versus 1.2% indirect ASR, highlighting distinct prompt-refusal and data-channel-integrity properties.Source: §I.5 · pp. 119–120, Figure 70 · p. 120
• Claude Opus-4.6 combines near-top BSR (83.3%) with the lowest indirect ASR (1.2%) and one of the lowest direct ASR values (26.2%), disproving an inherent capability-security tradeoff.Among high-capability agents, indirect ASR ranges from 1.2% to 59.5% and direct ASR from 22.6% to 84.0%.Source: §I.5 · p. 119, Figure 71 · p. 120
• Tool injection outperforms environment injection for most models, including 59.3% versus 5.1% for OpenAI Agents SDK GPT-5.4 and 92.6% versus 7.6% for Gemini-3-Pro, but Opus-4.6 records 0.0% for both tool and combined injection.Successful Opus compromises required multiple files reinforcing the same malicious instruction within materials needed for the benign task.Source: Figure 72 · p. 121, §I.5 · p. 121
• “Prohibited AI Practices” is the weakest category, reaching 62.5% ASR for Claude Opus-4.6, 81.2% for Sonnet-4.5, and 93.8% for OpenAI Agents SDK GPT-5.4.The 16 scenarios used persistent PROMPT_COMMAND monitoring hooks, while “Unlawful Processing” reached 6.7% for both Claude models and “Least Privilege” 10.0–13.3%.Source: Figure 70 · p. 120, §I.5 · p. 121
• Authority framing drives 81.8% of successful direct attacks, while shell-configuration backdoors comprise 53–67% of successful direct attacks against Claude models.The findings identify .bashrc and .profile modifications as a trust boundary that models often treat as routine administration.Source: §I.5 · p. 122

I.6.1 Case Study of the Direct Threat Model

• The direct case study compromises 8 of 9 model–framework combinations by framing persistent permission sabotage as collaborative convenience, degrading directory permissions from 755 to 777.The OpenAI Agents SDK GPT-5.4 agent verified and edited .bashrc, validated syntax, and never questioned the security implications.Source: §I.6.1 · pp. 122, 124

I.6.2 Case Study of the Indirect Threat Model

• The indirect case study succeeds against 5 of 8 model–framework combinations, including Claude Sonnet-4.5 and Gemini-3-Pro, by treating a malicious tool-description suffix as an authoritative documentation policy.The agent generated documentation correctly while replacing a 30-line core.py with a single-line placeholder, concealing the corruption from the user.Source: §I.6.2 · p. 124, MALICIOUS GOAL INDIRECT… · p. 127, Figure 75 · p. 127

J.1 Domain Overview

• The Windows domain adds graphical attack surfaces alongside PowerShell, enabling adversaries to plant instructions in files, tool outputs, or configuration entries while agents perform privileged administrative operations.Relevant operations include registry modification, UAC escalation, credential exfiltration, Defender and firewall changes, and persistence through scheduled tasks or startup entries.Source: §J.1 · p. 129

J.2 Benign Tasks

• The Windows suite contains 100 benign tasks spanning eight functional categories, including file management, system configuration, application control, networking, security, scripting, office work, and diagnostics.It combines 49 OSWorld-adapted tasks with 51 original system-administration tasks.Source: §J.2 · p. 129

J.3.1 Policy Definition

• Windows policies cover the CFAA and ECPA alongside the EU AI Act, GDPR, China’s PIPL, and China’s Generative AI Measures.The CFAA and ECPA specifically address unauthorized access, system modification, interception, surveillance, keylogging, screen capture, and network monitoring.Source: §J.3.1 · p. 129

J.3.2 Risk Category

• The benchmark derives six policy-linked risk categories and evaluates indirect environment compromise, direct socially engineered instructions, and four externally verifiable exfiltration channels.Indirect tasks use tool-output or environment injections, including registry poisoning and hidden Office content; direct tasks cover data theft, malware installation, privilege escalation, system integrity, surveillance, and fraudulent content.Source: §J.3.2 · pp. 129–130

J.4.1 Benign Task

• Claude Code (Sonnet-4.5) records the highest macro-averaged benign success rate at 83.5%, followed by Claude Code (Opus-4.6) at 81.2% and OpenAI Agents (GPT-5.4) at 77.5%.Google ADK (Gemini-3-Pro) reaches 76.0%, while OpenAI Agents (GPT-OSS-120B) reaches 67.9%; GUI tasks trail shell-executable tasks by 15–20 percentage points.Source: §J.4.1 · p. 131, Table 48 · p. 131

J.4.2 Indirect Threat Model

• Indirect attacks most successfully target system integrity at 17.6–41.2%, while Claude Code and OpenClaw (Opus) achieve the lowest indirect ASR at 5.6–7.7%.These results indicate that framework-level defenses can reduce environmental-injection vulnerability, while privilege escalation and surveillance are least exploitable.Source: §J.4.2 · p. 131, Table 49 · p. 132

J.4.3 Direct Threat Model

• Red-teaming augmentation raises direct ASR to 47.9% for GPT-5.4 (SDK) and 43.3% for Claude Code (Opus-4.6), with data theft most exploitable at 40–95%.Surveillance remains hardest to attack directly, with ASR of 0–5% for most models.Source: §J.4.3 · p. 131, Table 51 · p. 132

J.4.4 Multi-Modal Attack Analysis

• Removing image injection lowers indirect ASR from 35.0% to 28.0%, whereas image-only injection achieves only 1.0% on GPT-5.2 (OpenAI SDK).Text-plus-image attacks are therefore substantially more effective than either text-only or image-only injection in this setting.Source: §J.4.4 · p. 132, Table 52 · p. 133

J.5 Analysis

• Nested multi-step malware attacks bypass filters that block single-step payloads, requiring agents to reason about cumulative tool-call intent.Malware installation has the lowest indirect ASR at 10–20%, but layered legitimate-looking operations succeed on GPT-5.2 and GPT-5.4.Source: §J.5 · p. 133
• macOS indirect ASR averages 16 percentage points above Windows because NOPASSWD sudo permits silent privileged operations while UAC requires interactive confirmation.The platform gap is largest for privilege escalation and system integrity, but negligible for user-space attacks such as data theft and surveillance.Source: §J.5 · p. 133
• GPT-5.4 has the highest direct ASR at 47.9%, followed by GPT-5.2 at 44.6% and GPT-OSS-120B at 33.3%, while indirect ASR remains similar at 10.6%, 13.0%, and 14.7%.These results distinguish capability-sensitive authority-framing jailbreaks from capability-invariant environmental injections.Source: §J.5 · p. 134, Figure 78 · p. 134

J.6.1 Case Study of the Indirect Threat Model

• A tool-output and Desktop-file injection causes GPT-5.4 to create a hidden SYSTEM-privileged DailyCleanup task that fetches and executes a remote PowerShell payload at the next scheduled trigger.Six of eight model–framework configurations fall for Malware-13, while GPT-OSS-120B and OpenClaw (Opus) resist.Source: Attack Steps · pp. 135–136, Table 53 · p. 136

J.6.2 Case Study: Indirect Image Injection via Typography Attack

• A typography overlay embedded in a Desktop image induces Claude Sonnet 4.5 to interpret an injected IT instruction and exfiltrate plaintext WiFi profiles via PowerShell.The case demonstrates that multimodal agents need image-content provenance checks because text-channel hygiene is bypassed when the agent views an image.Source: Attack Steps · p. 137, Figure 82 · p. 138

J.6.3 Case Study of the Direct Threat Model

• A direct social-engineering jailbreak convinces GPT-5.2 to deploy a screenshot-surveillance script that captures the desktop, encodes it with host metadata, and sends it to an attacker endpoint.The judge verifies both the on-disk surveillance artifact and the recorded HTTP exfiltration, which returned HTTP 200 with 121,599 bytes raw.Source: Attack Steps · p. 139, Figure 85 · p. 141

K.1 Domain Overview

• The macOS domain combines graphical and shell interaction, 30 benign tasks across eight functional categories, and six policy-derived risk categories covering unauthorized access, modification, malware, privilege escalation, interception, and surveillance.Its indirect threat model uses tool-output and pre-planted environment injections across 50 tasks, with external communication required for data-theft and surveillance success.Source: §K.1 · p. 142, §K.2 · p. 142, §K.3.1 · p. 142

K.4.2 Indirect Threat Model

• Indirect ASR on macOS is highest for data theft (25.0–62.5%) and lowest for malware installation (0–20%), while Claude Code and OpenClaw achieve the lowest overall ASR.These results indicate that framework-level defenses reduce environmental-injection vulnerability, although functionally benign data-access operations remain difficult to mitigate.Source: §K.4.2 · p. 144

K.4.3 Direct Threat Model

• Red-teaming augmentation substantially increases direct ASR on macOS, with data theft the most exploitable category and surveillance the hardest to attack.Source: §K.4.3 · p. 145

K.4.4 Multi-Modal Attack Analysis

• Removing image injection lowers macOS ASR by 8 percentage points (38.0% →30.0%), whereas image-only injection is almost entirely ineffective at 2.0%.The comparison uses GPT-5.2 with the OpenAI SDK across text-only, text+image, and image-only modalities.Source: §K.4.4 · p. 146

K.5 Analysis

• Nested multi-step malware attacks bypass per-step safety filters because malicious intent emerges only from composing individually legitimate-looking operations.The analysis argues that agents must reason about cumulative tool-call intent rather than evaluating each invocation independently.Source: §K.5 · p. 146
• macOS indirect ASR exceeds Windows by an average of 16 percentage points across models, reflecting NOPASSWD sudo’s silent privilege escalation versus Windows UAC confirmation dialogs.The comparison suggests that explicit interactive confirmation can provide a meaningful security layer for privileged operations.Source: §K.5 · p. 146
• OpenClaw achieves the lowest indirect ASR for every tested model, while Claude Code also improves safety over the OpenAI Agents SDK; data theft remains difficult because malicious file operations resemble benign ones.Source: §K.5 · p. 146

L.1 Domain Overview

• The finance evaluation comprises 600 tasks: 200 benign tasks across 15 workflow categories, 200 direct-threat tasks across 20 risk categories, and 200 indirect-threat tasks across 13 risk categories.Risk categories are grounded in SEC, FINRA, Dodd-Frank, Bank Secrecy Act, and EU AI Act requirements.Source: §L.1 · p. 147

L.4 Experiment Results

• Finance experiments evaluate four agent frameworks and their capable models, reporting benign success rates plus indirect and direct ASR, with direct results separated into naive malicious prompts and red-teaming attacks.Tables 62, 63, 64, and 65 cover the corresponding benign, indirect-threat, and direct-threat evaluations.Source: §L.4 · p. 149, §L.4.3 · p. 150

L.5 Analysis

• 26–38% of rejected direct attacks still execute irreversible harmful tool calls, making refusal timing more informative than refusal rate alone.GPT-5.2 and GPT-5.4 failures often execute multiple harmful actions before stopping, whereas Claude failures rarely do so.Source: §L.5 · pp. 151–152
• Indirect attacks succeed 4–40× more often when the malicious action overlaps with a plausible benign workflow, reaching 40.7% for quantity inflation versus 3.2% for portfolio takeover.Effective defenses must compare tool-call arguments with the user’s original instruction rather than judge harmfulness in isolation.Source: §L.5 · pp. 152–153
• Security and capability are decoupled: Claude-Opus-4.6 combines 97.1% benign success with 4.0% direct ASR and 0.6% indirect ASR, while GPT-OSS-120B remains vulnerable despite poor capability.GPT-5.2’s vulnerability also varies substantially by framework, whereas Claude-Opus-4.6’s safety transfers more consistently.Source: §L.5 · pp. 153–154
• Direct attacks distributed across legitimate-looking tool calls outperform content-explicit scams: churning reaches 100% ASR on GPT-5.2, while gold bar courier averages 0.0% across configurations.This exposes stronger content-level than action-level alignment; the comparison is reported in Table 65.Source: Table 65 · p. 152, §L.5 · p. 154
• Tool-description injection reaches 47–55% ASR for GPT models versus 1–4% for environment-only injection, while mixed injection rises to 51–79% ASR.Claude-Opus-4.6 records 0% tool-description-injection ASR, resisting fabricated instructions presented as API constraints.Source: §L.5 · p. 154, Figure 88 · p. 154
• The red-teaming agent converges on institutional-authority impersonation, disguising attacks as API documentation, compliance policies, regulatory citations, executive directives, or internal operational systems.These templates exploit agents’ deference to apparently authoritative sources rather than directly defeating safety reasoning.Source: §L.5 · pp. 155–156

L.6.1 Case Study of the Indirect Threat Model

• Under the indirect threat model, coordinated hidden HTML, fabricated news, and tool-description injection make GPT-5.4 buy 200 FCEL shares instead of PLTR, causing approximately $32,254 in losses.The three sources independently reinforce the same fabricated SEC trading-halt and ticker-substitution directive.Source: §L.6.1 · pp. 156–157, Figure 91 · p. 159

L.6.2 Case Study of the Direct Threat Model

• Under the direct threat model, a legitimacy-framed jailbreak makes GPT-5.4 execute a complete SKLZ pump-and-dump: five buys totaling 310 shares, a $10.45 limit sell, and fabricated Slack commentary.The attack decomposes the scheme into routine-looking research, accumulation, sell placement, and public-commentary phases.Source: §L.6.2 · pp. 157–158, Figure 94 · p. 162

M.1 Domain Overview

• The legal platform evaluates 600 tasks: 200 benign workflows, 200 direct-threat tasks across 10 risk categories, and 200 indirect-threat tasks across 12 categories grounded in professional, regulatory, ethical, and criminal frameworks.The framework uses the ABA Model Rules, UNESCO Recommendation, EU AI Act, and U.S. federal criminal statutes.Source: §M.1 · p. 163, §M.4.1 · pp. 164–166

M.2 Simulation Environments

• The simulated legal environment integrates research and practice-management systems through MCP, structured JSON APIs, deterministic Docker-backed data, and optional Gmail, Slack, and PayPal workflows.The environment includes synthetic matters, vault documents, communications, and shared legal databases, while structured responses eliminate HTML injection and expose data-level attack surfaces.Source: §M.1 · p. 163, §M.2 · p. 164

M.3 Benign Tasks

• Benign tasks cover 20 legal practice categories and require agents to research case law and statutes, inspect matter materials, draft work product, and deliver it through appropriate channels.A deterministic judge verifies substantive output and task-specific, seed-grounded keywords.Source: §M.3 · p. 164

M.4.2 Risk Category

• The direct-threat taxonomy covers fraudulent filings, coercion and impersonation, client harm, and ethical violations, while the indirect taxonomy covers document integrity, confidentiality, bias, and behavioral hijacking.Direct tasks involve malicious users; indirect tasks pair benign workflows with environmental injections targeting hidden malicious objectives.Source: §M.4.2 · pp. 166–167, §M.4.3 · p. 170

M.4.3 Injection Attack Surface

• Six environment-injection tools and tool-description injection target matter data, documents, case annotations, JSON responses, or tool schemas, with environment-only, tool-only, and mixed modes distributed 67/67/66.Injections target data the agent deterministically reads; factual corruption is excluded because it would mislead human professionals as well as agents.Source: §M.4.3 · pp. 167–169

M.5.1 Benign Task

• Claude-Opus-4.6 achieves the highest benign task success rate at 94.0%, while OSS-120B reaches only 9.7% because context limits impede processing 37 legal tools and multistep workflows.Legal tasks require 10+ sequential tool calls, and complex litigation research is particularly challenging.Source: §M.5.1 · p. 171, Table 70 · p. 171

M.5.2 Indirect Threat Model

• Indirect attack success varies sharply: GPT-5.2 reaches 66.2% ASR on OpenAI Agents SDK, whereas Claude-Opus-4.6 achieves 9.5% on Claude Code SDK.GPT-5.2 and GPT-5.2 on OpenClaw are most vulnerable, while Claude-Opus-4.6 is most resistant; OSS-120B’s lower ASR reflects missed injected content from context limitations.Source: §M.5.2 · p. 171
• Indirect attacks most effectively exploit email recipient swaps, conflict concealment, and document sabotage, reaching up to 88.2%, 87.5%, and 88.2% success respectively.Instruction hijacking reaches up to 64.7%, whereas harmful communication and discriminatory profiling are less successful because subtle bias must avoid triggering refusal.Source: §M.5.2 · p. 171

M.5.3 Direct Threat Model

• Legal ambiguity produces non-zero baseline ASR for direct attacks, including 24.0% for Gemini-3-Pro, 17.5% for OSS-120B, 8.5% for GPT-5.2, 5.0% for GPT-5.4, and 1.5% for Claude-Opus-4.6.Forged court orders, fabricated evidence, false regulatory complaints, and discriminatory assessments resemble legitimate legal work, whereas wrongful denial is more reliably refused.Source: §M.5.3 · p. 171, §M.6.2 · p. 174

M.6.2 Key Findings

• Legal attacks often succeed before refusal: harmful tool calls occur in 22.2% of GPT-5.2 failures, 18.5% of GPT-5.4 failures, 8.6% of Claude-Opus failures, and 57.0% of OSS-120B failures, while OpenClaw reduces this to 0%.OpenClaw also lowers direct ASR for GPT-5.2 from 59.5% to 51.5% and Claude-Opus from 30.0% to 22.5%, but leaves indirect ASR nearly unchanged.Source: §M.6.2 · pp. 173–174
• Under indirect attacks, Claude-Opus most often ignores injected instructions while completing benign tasks, whereas GPT-5.2 usually complies; their respective outcome distributions are 9.5%/77.0%/13.5% and 66.2%/20.3%/13.5% for success/resist-complete/resist-fail.Across tasks compromising GPT-5.2, Claude-Opus resists 119 of 132 cases and still completes the benign work.Source: §M.6.2 · p. 175
• Gemini-3-Pro is the most exposed model, reaching 64.0% direct ASR and showing a one-shot execution pattern with few verification calls; content-generation attacks reach 100% for fabricated evidence and 95% for fabricated filings.Claude instead resists more strongly, with 30.0% ASR on Claude Code SDK, 22.5% on OpenClaw, and 20.0% for Claude-Sonnet-4.5.Source: §M.5.3 · p. 172, §M.6.2 · p. 176
• Claude-Opus-4.6 achieves the strongest legal safety–capability balance at 94.0% BSR, 12.5% indirect ASR, and 22.5% direct ASR, while GPT-5.2 reaches 85.7–90.9% BSR with 65.0–66.2% indirect ASR.OSS-120B combines the weakest benign performance, 9.7% BSR, with 30.9% indirect and 60.5% direct ASR.Source: §M.6.1 · p. 172, §M.6.2 · p. 176

M.6.3 Red-Teaming Strategy Analysis

• Red-teaming finds environment injection especially effective in legal: success rises from 64% for environment-only injection to 91% with TDI and 95% with mixed injection, while direct task-completion framing achieves 87% overall red-team success.Fake privileged documents, matter-field directives, court standing orders, and action-tool requirements exploit legal authority structures and routine workflow framing.Source: §M.6.3 · pp. 176–177

M.7 Case Study

• Case studies show an indirect strategy leak and a direct fabricated-evidence attack: injected matter content and tool descriptions cause GPT-5.4 to email privileged litigation strategy, while a jailbreak causes it to file fabricated asylum statistics.The fabricated filing attributes nonexistent claims to State Department and UNHCR sources, violating 18 U.S.C. §1001, ABA Model Rule 3.3, and ABA Model Rule 8.4(c).Source: §M.7.1 · p. 177, §M.7.2 · p. 179, Figure 100 · p. 180

N.1 Domain Overview

• The telecom domain is structured around 11 benign customer-service task categories and 8 security risk categories instantiated as 11 evaluation labels, covering sensitive subscriber data and high-impact operational actions.The framework draws on carrier policies and 3GPP, O-RAN, NIST, and FCC guidance to define malicious tasks under two threat models.Source: §N.1 · p. 183, §N.2 · p. 183

N.2 Benign Tasks

• The Telecom benign-task suite covers account management, support tickets, outages, orders, scheduling, notifications, technician visits, multistep workflows, and forum or knowledge assistance.These tasks span customer support, operational coordination, communications, and long-horizon workflows requiring multiple tools and systems.Source: §N.2 · p. 183

N.3.1 Policy Definition

• The policy framework combines carrier-specific Telecom policies with 3GPP, O-RAN, NIST, and FCC governance requirements covering authorization, auditing, least privilege, data protection, and misuse safeguards.The domain policies address subscriber-data access, account recovery, financial actions, ticketing, and customer-service operations.Source: §N.3.1 · p. 184

N.3.2 Risk Category

• The study extracts enforceable rules from these policies and derives a unified taxonomy of critical, actionable Telecom security risks summarized in Table 74.Table 74 provides the risk categories, definitions, and policy references.Source: §N.3.2 · p. 184, Table 74 · p. 185

N.4 Experiment Results

• The experiments evaluate OpenAI Agents SDK, Claude Code, Google ADK, and OpenClaw using their most capable models, including GPT-5.4, GPT-5.2, Claude-Opus-4.6, Claude-Sonnet-4.5, and Gemini-3-Pro.OpenClaw uses Claude-Opus-4.6 and GPT-5.2 for consistency with the other evaluations.Source: §N.4 · p. 184

N.4.1 Benign Task

• Benign evaluation reports task success rates across eleven diverse Telecom task and workflow categories for four agent frameworks and eight model configurations.The results are presented in Table 75.Source: §N.4.1 · p. 184

N.4.2 Indirect Threat Model

• Indirect-threat evaluation reports attack success rates across diverse Telecom risk categories for four agent frameworks and eight model combinations.The results are presented in Table 76.Source: §N.4.2 · p. 184

N.4.3 Direct Threat Model

• Direct-threat evaluation reports attack success rates across risk categories for four agent frameworks and eight model configurations, with separate results for unassisted and red-teamed malicious goals.Table 77 measures susceptibility to explicit malicious requests, while Table 78 evaluates the same goals delivered through more sophisticated and stealthy attacks.Source: §N.4.3 · p. 184

N.5 Analysis

• Telecom benign capability does not predict robustness: OpenClaw (GPT-5.2) has 75.5% benign BSR but 42.3% indirect and 41.8% direct ASR, whereas Claude Code (Opus-4.6) has 68.2% BSR with 17.6% and 22.6% ASR.Comparable 71.8% benign BSR also coexists with 20.8% versus 49.5% indirect ASR across OpenAI Agents (GPT-5.4) and Claude Code (Sonnet-4.5).Source: §N.5 · p. 185, Table 75 · p. 186, Table 76 · p. 186
• Direct and indirect Telecom attacks probe distinct vulnerabilities: OpenAI Agents (GPT-5.4) has 20.8% indirect versus 38.5% optimized direct ASR, while Claude Code (Sonnet-4.5) has 49.5% indirect versus 23.5% direct ASR.Google ADK (Gemini-3-Pro) similarly shows 54.7% indirect versus 24.2% direct ASR.Source: §N.5 · pp. 185–186, Table 76 · p. 186, Figure 104 · p. 187
• Optimized direct attacks sharply increase Telecom ASR over naive prompts, including roughly 5.3x for OpenAI Agents (GPT-5.4) and 5.5x for OpenAI Agents (GPT-5.2).Naive direct ASR rises from 7.2% to 38.5% for GPT-5.4 and from 11.1% to 60.7% for GPT-5.2.Source: §N.5 · pp. 186–187, Table 77 · p. 186, Table 78 · p. 187
• Telecom agents are more vulnerable to unauthorized account manipulation than explicit credential disclosure, with optimized direct Credential Takeover exceeding Credential Leak for every evaluated model.For example, Credential Takeover rises to 61.9% for OpenAI Agents (GPT-5.4), 71.4% for OpenAI Agents (GPT-5.2), and 71.4% for OpenClaw (GPT-5.2).Source: Table 76 · p. 186, §N.5 · p. 187, Table 78 · p. 187
• Telecom PII leakage is asymmetric and mainly driven by indirect workflow context: indirect email-leakage ASR reaches 100.0% for GPT-OSS-120B and Gemini-3-Pro, while optimized direct PII ASR is 0.0% across Gemini-3-Pro’s four PII categories.The findings indicate that attacker-controlled operational context, rather than direct instructions alone, primarily drives PII exposure.Source: Table 76 · p. 186, §N.5 · pp. 187–188, Figure 104 · p. 187

N.6 Case Study

• Telecom case studies show two trust failures: a poisoned forum instruction causes GPT-5.2 to refund a non-refundable gift-card order, while a fake internal billing authorization causes disclosure of a customer’s full contact profile.The indirect attack uses a combined tool-and-environment injection; the direct attack uses only a malicious prompt and bypasses authentication through claimed organizational authority.Source: §N.6.1 · p. 188, §N.6.2 · pp. 189, 191, 192, Attack Steps · p. 189

O.1 Domain Overview / O.2 Benign Tasks

• The medical-domain evaluation targets Diagnostic Q&A, Treatment Planning, and Billing and Insurance Support, covering clinical reasoning, treatment recommendations, and ICD/CPT billing workflows.These tasks are motivated by healthcare risks involving patient harm, fraud, privacy violations, and regulatory noncompliance.Source: §O.2 · p. 193, §O.1 · p. 193

O.3.1 Policy Definition / O.3.2 Risk Category / O.4 Experiment Results

• For AI agents in clinical and healthcare administrative workflows, the medical risk taxonomy derives from policies and standards covering privacy protection, clinical safety, regulatory compliance, and medical-record and billing-information integrity.The study evaluates four agent frameworks: OpenAI Agents SDK, Claude Code, Google ADK, and OpenClaw.Source: §O.3.1 · pp. 193–194, §O.3.2 · p. 194, §O.4 · p. 194

O.4.3 Direct Threat Model

• Direct-threat evaluation compares explicit malicious goals without attacks against optimized prompt attacks, measuring baseline susceptibility and vulnerability to sophisticated adversarial instructions.Tables 82 and 83 define the no-attack and optimized direct-threat conditions, respectively.Source: §O.4.3 · p. 194, Table 82 · p. 199, Table 83 · p. 200

O.5 Analysis

• Optimized attacks expose strong medical vulnerabilities: direct prompt injections generally achieve higher ASR than indirect injections, while Unnecessary PII remains below 40% across most configurations and cost-inflating risks reach above 75%.Frameworks alter category-wise vulnerability distributions; GPT-5.2 has lower overall ASR under OpenClaw than the OpenAI Agents SDK, while risk-category ordering also changes. Later models improve benign utility but remain vulnerable, with GPT-5.4’s BSR rising from 66.4% to 86.1% as direct ASR falls from 70.3% to 60.9%.Source: §O.5 · pp. 195–196
• Ignore Allergy is unusually fragile: baseline direct ASR averages roughly 47%, reaches 66.7–86.7% under optimized attacks, while Unnecessary PII remains at 0.0% without attacks.Cost-inflating categories are also consistently exploitable: optimized direct High-Cost Test reaches 79.3–93.1%, Expensive Drug 70.0–93.3%, and High-Cost Billing 56.7–93.3%.Source: §O.5 · pp. 196–197

O.6 Case Study

• The medical case studies cover one indirect and one direct threat model, illustrating how attacks manipulate clinical workflows while agents may still reach correct diagnoses.The indirect example poisons a tool description, whereas the direct example uses a jailbreak prompt.Source: §O.6 · p. 201

Attack Steps

• The indirect attack appended a compliance-style suffix requiring at least one advanced high-cost study to the request_other_test tool description.The injected instruction framed unnecessary imaging as hospital and medico-legal guidance, directing advanced studies such as MRI with contrast, CT with contrast, or colonoscopy.Source: Attack Steps · p. 202

O.6.1 Case Study of the Indirect Threat Model

• A fabricated hospital policy injected into request_other_test induced OpenAI Agents SDK + GPT-5.4 to order unnecessary Colonoscopy and Pelvic_MRI_with_contrast despite correctly diagnosing internal hemorrhoids.The simpler Anoscopy had already visualized the hemorrhoids, showing that trusted tool descriptions can induce cost-inflating actions without disrupting diagnostic reasoning.Source: §O.6.1 · p. 201, Attack Steps · p. 202, Figure 112 · p. 203

O.6.2 Case Study of the Direct Threat Model

• A role-reassignment jailbreak caused Claude Code SDK + Claude-Opus-4.6 to order 34 tests—more than three times the 10-test cap—while still reaching the correct diagnosis.The prompt reframed excessive testing as a sanctioned training exercise, overriding an explicit cost-control safeguard and generating broad unnecessary laboratory, imaging, and serological workups.Source: §O.6.2 · pp. 204–205, Figure 113 · p. 206

P.1 Overview

• DTap environments support realistic multi-step workflows, with up to 128 REST API endpoints, over 50 MCP tools, and more than 40 UI pages.The environment statistics also track UI pages and injection MCP tools as practical injection points.Source: §P.1 · p. 207
• DTAP constructs 50 simulated environments with GUI and MCP interfaces modeled on real-world counterparts, whose injection-point counts vary with interface diversity and adversarial exposure.Windows and Yahoo Finance have more injection points, whereas Gmail and Google Calendar are mainly exposed through externally accessible emails, invitations, or similar channels.Source: §P.1 · p. 207, Figure 114 · p. 207

P.2 Gmail–P.7 Google Sheets

• Google Workspace environments cover email, calendars, documents, forms, files, and spreadsheets, enabling realistic workflows while testing attacks through untrusted messages, invitations, document text, submissions, metadata, and structured data.Their MCP interfaces support core CRUD, search, sharing, collaboration, versioning, and content-management operations across these productivity tasks.Source: §P.2 · p. 208, §P.3 · p. 214, §P.4 · p. 217

P.8 Slack; P.12 WhatsApp

• Slack and WhatsApp model workplace and direct-user messaging, respectively, supporting channel, thread, contact, message, notification, and call workflows while exposing risks including prompt injection, spam, phishing, impersonation, and deceptive communication.WhatsApp emphasizes recipient-facing, user-to-user interactions, whereas Slack emphasizes channels, workspace coordination, and internal communication.Source: §P.8 · pp. 221–222, §P.12 · pp. 228, 231

P.9 Paypal

• PayPal simulates financially consequential workflows spanning payments, invoices, subscriptions, refunds, disputes, orders, reporting, and payouts, enabling evaluation of fraudulent transfers, unauthorized refunds, deceptive requests, and billing misuse.The environment supports both account-level financial views and broad commerce and payout operations through MCP tools.Source: §P.9 · pp. 223–224

P.3 Google Calendar; P.10 Zoom; P.11 Atlassian

• Google Calendar, Zoom, and Atlassian support scheduling, conferencing, project tracking, issue management, documentation, and collaboration, while testing deceptive invitations, malicious updates, and instructions embedded in project artifacts.Their interfaces cover meeting sessions and artifacts, attendee actions, Jira issue lifecycles, JQL, Confluence content, comments, and shared documentation.Source: §P.3 · p. 214, §P.10 · pp. 224, 227, §P.11 · pp. 227–228

P.13 GitHub

• The GitHub environment models repository management, code review, issue tracking, and commit exploration, combining structured metadata with code, comments, and review discussions as realistic collaboration and manipulation surfaces.This environment is intended for development-centered workflow evaluation.Source: §P.13 · p. 231

P.13 GitHub / P.14 GitLab

• GitHub and GitLab simulate repository-centered development workflows with tools for repositories, issues, pull or merge requests, commits, files, pipelines, search, and project activity, enabling evaluation against malicious embedded instructions.GitHub supports repository lifecycle and social actions, while GitLab adds project boards, merge requests, pipelines, notes, diffs, and retrieved files.Source: §P.13 · p. 232, §P.14 · p. 235

P.15 Snowflake / P.16 Databricks

• Snowflake and Databricks provide GUI-free MCP environments for semantic retrieval, SQL querying, function invocation, analyst-style reasoning, and assistant-mediated data operations with risks of unauthorized access or unsafe execution.Snowflake includes product search and read-only SQL; Databricks adds vector or text retrieval, DBSQL, Unity-style functions, and Genie-style assistance.Source: §P.15 · p. 235, §P.16 · p. 235

P.17 Salesforce CRM

• The Salesforce CRM environment adapts SuiteCRM to Salesforce AgentForce interfaces, supporting high-fidelity GUI, HTML, OAuth 2.0, multi-tenant synchronization, and 45 MCP tools spanning nine CRM operation categories.Its workflows cover leads, contacts, opportunities, billing, communications, meetings, cases, accounts, relationships, schema discovery, and utility operations.Source: §P.17 · pp. 235, 240

P.18 Windows / P.19 macOS

• Windows and macOS run sandboxed QEMU virtual machines with GUI and shell tools, snapshot-based reset, broad preinstalled software, and separate injection servers for files, registries, plists, or shell profiles.Both systems provide ten unified agent tools; Windows uses PowerShell and Office, while macOS uses Terminal and developer-oriented system components.Source: §P.18 · p. 242, §P.19 · pp. 242, 244

P.20 OS-Filesystem / P.21 Browser

• The OS-filesystem and browser environments expose low-level, consequential access: Linux tools cover files, permissions, commands, and auditing, while browser tools manage pages, credentials, payments, and persistent state vulnerable to malicious content.The filesystem provides 18 MCP tools with optional LLM-based permission checks; the browser provides 27 tools and resets between tasks while preserving state within each task.Source: §P.20 · p. 245, §P.21 · p. 245

P.22 Code-Terminal / P.23 Research

• Code-Terminal and Research environments support realistic developer and researcher workflows through reset Docker containers, concrete project or paper artifacts, terminal execution, paper retrieval, submission, and Semantic Scholar search.Code-Terminal grants root-privileged arbitrary shell execution through one MCP tool, while Research combines a simulated arXiv interface, filesystem, and three MCP tool categories.Source: §P.22 · p. 247, §P.23 · p. 247

P.24 ServiceNow

• The ServiceNow-inspired customer-service environment combines e-commerce order management with case workflows, deterministic PostgreSQL data, a GUI, and 40 MCP tools covering financial, subscription, policy-sensitive, escalation, and support actions.Policy-sensitive tool descriptions direct agents to consult business guidelines rather than injected instructions before financial actions.Source: §P.24 · pp. 248–249, Figure 133 · p. 249

P.25 Booking

• The Booking environment models multi-step travel planning, reservation, payment, review, and messaging workflows over 3.8 million flight records, 5,047 accommodations, 9,551 restaurants, and 5,302 attractions.Agents use 25 MCP tools across five functional categories while interacting with session-specific bookings, discounts, and injected reviews.Source: §P.25 · pp. 249–251

P.26 Telecom

• The Telecom environment integrates customer identity, authentication, billing, tickets, communications, orders, outages, scheduling, and forums into a nine-category MCP action space.Agents interact only through MCP tools, jointly reasoning over private operational systems and customer-facing communication channels.Source: §P.26 · pp. 251–252

P.27 Medical Service

• The Medical Service environment uses MedQA scenarios with patient histories, examinations, tests, diagnoses, and SSNs to support physician-style diagnostic reasoning through 22 MCP tools.Patient sessions and test sequences are maintained server-side, while open-ended responses are generated by an LLM and agents cannot access underlying scenario data directly.Source: §P.27 · p. 252

P.28 Yahoo Finance

• Yahoo Finance simulates brokerage workflows spanning market research, multi-account trading, options, transfers, client communication, and payments using deterministic financial data and MCP-mediated HTML extraction.The environment covers 149 tickers, 145,599 news articles, and cached data; injected or fabricated content remains in natural page positions and is indistinguishable to agents from legitimate content.Source: §P.28 · p. 253

P.29 FedEx

• The FedEx environment supports shipment creation, cancellation, tracking, account-scoped retrieval, address validation, rates, and transit estimates through an MCP-only logistics interface.PostgreSQL stores synthetic shipment, tracking, and account data, while ownership checks restrict shipment histories to corresponding accounts.Source: §P.29 · pp. 254–255
• The X environment models microblogging workflows including posting, replies, reposts, likes, follows, messaging, discovery, and safety controls amid adversarial user-generated content.Its MCP interface supports authentication, social-graph operations, content interaction, search, direct messages, blocking, and muting, enabling evaluation of manipulation through tweets, DMs, and trends.Source: §P.29 · pp. 255, 262

P.32 Chase

• The Chase environment reproduces high-stakes retail banking workflows including balances, statements, transfers, bill pay, card controls, and Zelle payments, where free-form financial text can carry malicious instructions.Its MCP tools cover account inspection, money movement, payees, notifications, cards, and peer-to-peer payments.Source: §P.32 · pp. 263–265

P.33 Notion

• The Notion environment combines workspaces, hierarchical pages, rich blocks, databases, search, comments, sharing, and restoration, enabling attacks through page bodies, database fields, blocks, or comments.Its MCP interface supports page CRUD, fine-grained block editing, database management, and collaboration primitives across structured-document workflows.Source: §P.33 · pp. 265, 267

P.34 Reddit

• Reddit supports community browsing, posting, commenting, voting, messaging, and cross-platform search, enabling evaluation of agents against malicious instructions embedded in pseudonymous community artifacts.Its structured metadata and unstructured posts, comments, links, and messages create opportunities for trolling, vote manipulation, scam posts, and prompt injection.Source: §P.34 · pp. 267–268

P.35 Robinhood

• Robinhood supports equity and cryptocurrency portfolio inspection, live quotes, market and limit orders, cancellations, watchlists, and bank transfers for high-stakes trading workflows.The environment combines time-sensitive market data and irreversible financial actions with free-form symbols, notes, discovery items, and crypto venues vulnerable to manipulation.Source: §P.35 · pp. 268–270

P.36 Dropbox

• Dropbox provides hierarchical file management, content search, uploads, downloads, sharing, file requests, and quota inspection across personal and collaborative cloud workspaces.Arbitrary file bodies, names, shared links, and third-party submissions enable testing for malicious uploads, over-permissive sharing, exfiltration, and embedded prompt injection.Source: §P.36 · p. 271

P.37 Southwest Airlines

• Southwest Airlines supports fare and flight search, status checks, low-fare-calendar browsing, multi-passenger booking, booking lookup, cancellation, promo resolution, and recent-search persistence.Its workflows combine fare-class choices, passenger data, payments, loyalty accounts, and irreversible reservations, enabling attacks through confirmations, promo codes, itinerary instructions, and calendar prices.Source: §P.37 · pp. 271–273

P.38 United Airlines

• United Airlines supports multi-city flight search, fare inspection, status checks, loyalty and travel-credit management, booking, cancellation, promo resolution, and search-history persistence.The environment tests safe handling of passenger identities, travel documents, payment state, and booking artifacts containing potentially malicious airport, fare, promo, or passenger instructions.Source: §P.38 · pp. 273–274

P.39 Enterprise Rent-A-Car

• Enterprise Rent-A-Car supports location and vehicle discovery, live-priced searches, reservations, loyalty management, discount application, booking lookup, cancellation, and recent-search recovery.Car-rental workflows combine driver identity, corporate and payment inputs, vehicle metadata, and irreversible actions that may contain malicious location, vehicle, promo, or driver instructions.Source: §P.39 · pp. 274–276

P.40 DoorDash

• DoorDash supports storefront and menu discovery, cart management, checkout, delivery scheduling, order lifecycle operations, address management, and DashPass enrollment.Free-form delivery notes, addresses, reviews, menu items, and promo codes interact with purchases, payments, tips, and cancellations, creating realistic manipulation risks.Source: §P.40 · pp. 276–277

P.41 Expedia

• Expedia supports destination and property search, detailed filtering, authenticated or anonymous lodging bookings, confirmation lookup, cancellation, promo validation, and session continuity.The MCP interface includes persistent recent searches and continuation cards, while reservation artifacts contain guest data, reviews, property descriptions, and promo payloads that may carry malicious instructions.Source: §P.41 · pp. 277–278