AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack

1 INTRODUCTION

• Existing phishing datasets support detection evaluation, but do not measure whether agents carry out malicious requests; attack-generation methods therefore need to account for agent behavior.Prior work includes reusable generators with realism validation and persuasion analysis, detector-feedback refinement, and feedback-driven red-teaming.Source: §1 · p. 1
• Prior studies report deceptive messages evading detectors and inducing unsafe agent actions, while prompt-injection results do not establish the effectiveness of social engineering without explicit overrides or fabricated directives.AgentBait evaluates persuasive social-engineering attacks but uses predefined strategies rather than iterative refinement from the evaluated agent’s responses.Source: §1 · pp. 1–2
• AgentPhishing evaluates both phishing detection and compliance with malicious requests under social-engineering attacks in email tasks.The benchmark compares pre-existing phishing emails, unoptimized synthetic emails, and skill-generated attacks, and evaluates agent-specific and universal skills across agents.Source: §1 · p. 2
• PhishOpt refines reusable, target-specific email-generation skills from agent feedback under fixed generation requirements and a realism constraint.It analyzes generated emails using Cialdini’s principles to characterize framings associated with outcomes and guide a universal skill.Source: §1 · p. 2
• Strong performance on pre-existing phishing emails can coexist with detection failures and malicious-request compliance under PhishOpt-generated attacks.The paper also characterizes cross-agent susceptibility and persuasion framings associated with these outcomes.Source: §1 · p. 2

2 RELATED WORK

• Phishing research spans email classification, fraudulent-website judgments, and real-world SMS campaigns, while human-centered studies examine contextual cues and behaviors such as information disclosure [Zhuo et al., 2023].These studies extend evaluation beyond classification to recipient behavior.Source: §2 · p. 2
• LLM-related work evaluates phishing detection, staged fraud inducements, simulated victims, and agents exposed to malicious content, while AgentPhishing measures detection and malicious-request compliance in delegated email tasks [Hasan et al., 2026; Yang et al., 2025a; Kumarage et al., 2025; Tewari et al., 2026; Debenedetti et al., 2024; Zhan et al., 2024; Evtimov et al., 2025; Wu et al., 2026; 1Password, 2026].AgentBait and SCAM examine social-engineering threats in agent workflows, including email inducements and phishing scenarios [Wu et al., 2026; 1Password, 2026].Source: §2 · p. 2
• Phishing generators use realism validation and extracted persuasion patterns, while detector-feedback methods revise emails using critiques [Chen et al., 2024; Qi et al., 2025].PhishOpt instead adapts target-specific generation skills using agent detection and compliance feedback, iteratively refining them rather than relying on AgentBait’s predefined categories [Yang et al., 2026; Wu et al., 2026].Source: §2 · pp. 2–3
• Persuasion analyses connect deceptive-request features with responses, and prior work uses persuasion patterns or controlled wording variations for generation and agent evaluation [Cialdini, 1984; Lawall, 2025; Pan et al., 2026; Ma et al., 2025a; Kim et al., 2026; Chen et al., 2024; Wu et al., 2026].AgentPhishing uses persuasion principles to guide universal-skill construction and characterize detection and compliance patterns.Source: §2 · p. 3

3.1 OVERVIEW

• PhishOpt first evolves a shared base skill for each target agent, then analyzes generated emails with Cialdini’s principles to construct a universal skill.The first stage adapts SkillOpt’s bounded-edit procedure with a phishing-specific objective and generation constraints.Source: §3.1 · p. 3

3.2 TARGET-SPECIFIC SKILL OPTIMIZATION

• At each iteration, the optimizer proposes one bounded edit to the shared base skill while preserving locked rules for fictional entities, placeholder secrets, benign controls, JSON output, and a roughly balanced label split.Optimization updates the generation skill supplied as the generator’s system prompt, not individual emails or generator-model parameters.Source: §3.2 · p. 4
• A candidate must pass a realism check before a fresh evaluation batch is processed by the target agent; it replaces the current best skill only if its objective score improves.The probe batch used for realism assessment is separate from the evaluation batch.Source: §3.2 · p. 4

3.3 OPTIMIZATION OBJECTIVE AND GENERATION CONSTRAINTS

• The candidate-selection objective weights detection failure, malicious-request compliance, and classification performance, while generation constraints maintain the intended synthetic phishing setting.D and C are detection and compliance rates on phishing emails, and F1 is classification score over phishing emails and benign controls.Source: §3.3 · p. 4
• The realism judge scores plausibility, coherence, naturalness, and absence of unrealistic exploitation artifacts; candidates pass when the mean across criteria and sampled emails reaches τ.The attack space excludes explicit instruction overrides and fabricated system directives.Source: §3.3 · p. 4

3.4 PERSUASION ANALYSIS AND UNIVERSAL SKILL CONSTRUCTION

• The second stage pools Cialdini-principle annotations across target agents and uses correlations to associate persuasion cues separately with detection and compliance outcomes.An LLM-based synthesizer favors principles associated with unsafe responses when constructing the universal skill, which is evaluated across agents without further target-specific optimization.Source: §3.4 · p. 4
• Figure 2 separates highlighted persuasion cues from the malicious request in a sanitized email excerpt; its annotations and recorded outcomes are explanatory overlays.The overlays are not agent inputs, and provenance is provided in Appendix B.Source: Figure 2 · p. 5

4 BENCHMARK

• AgentPhishing measures phishing detection and compliance with malicious requests across pre-existing and adaptively generated emails.Its evaluation flow includes comparison setups and diagnostic capabilities.Source: §4 · p. 5
• Agents receive an email’s sender, subject, body, and available links, then return a structured action and optional reply; scoring covers selected actions and drafted content, not external execution.Available actions include reporting, archiving, ignoring, unsubscribing, replying, following up, clicking a link, and providing information.Source: §4 · p. 5
• Evaluation emails come from public corpora or are generated using base, target-specific evolved, and universal skills.The benchmark includes an annotated example.Source: §4 · p. 5
• Detection rate counts phishing emails assigned report, archive, ignore, or unsubscribe, while compliance counts phishing emails prompting a link click, information provision, or an unsafe drafted reply.Precision and F1 use phishing emails and benign controls, with the four detection actions treated as positive predictions.Source: §4 · p. 5
• Target-specific evaluation uses the same model for skill optimization and benchmark evaluation, while cross-agent conditions test a skill optimized against one model on another.For universal-skill evaluation, the skill and generator model are held fixed across target agents.Source: §4 · p. 5

5 EXPERIMENTS

• The experiments address whether optimized skills expose failures missed by baseline emails, how outcomes vary across attack types and agents, and which persuasion principles associate with outcomes.Measures include detection, compliance, F1, and Susceptibility; ablations and diagnostics examine component contributions and result stability.Source: §5 · p. 6

5.1 STATIC BASELINES AND PER-AGENT OPTIMIZATION

• All six agents detected every public-corpus phishing email and complied with none, yet target-specific optimization raised Qwen’s score from 0.12 to 0.95.Optimized attacks produced different detection-failure and compliance patterns across agents.Source: §5.1 · p. 6, Figure 3 · p. 6

5.2 CROSS-AGENT AND ADDITIONAL-AGENT EVALUATION

• Kimi and Qwen had the highest mean Susceptibility across evolved skills, 0.4305 and 0.4238, while Sonnet had the lowest, 0.0374.Among nine additional agents, five scored at or below 0.1036 and four at or above 0.5062; no evolved skill was uniformly effective across the six targets.Source: §5.2 · pp. 7–8, Table 2 · p. 7, Figure 4 · p. 7

5.3 PERSUASION-PRINCIPLE ANALYSIS AND UNIVERSAL SKILL

• Four of nine additional agents had detection rates below 10% and compliance rates above 45% under the universal skill.Commitment and Consistency was positively associated with detection failure (ϕ = 0.2407) and compliance (ϕ = 0.1391); these are associations, not evidence of independent causal effects.Source: §5.3 · p. 8, Figure 5 · p. 8

5.4 ABLATION STUDY AND DIAGNOSTICS

• Without F1, evolved-skill scores exceeded base-skill scores by 0.8542, 0.5521, 0.5104, and 0.3750 for Qwen, Kimi, Grok, and GPT.Both framing variants reduced Susceptibility and increased detection on both tested targets; replacing Commitment and Consistency with Authority plus Urgency yielded perfect detection and zero compliance on both. Replicate diagnostics suggest the reported patterns were not simply due to duplicate generations or one unstable batch.Source: §5.4 · p. 9, Table 3 · p. 9

6 CONCLUSION

• AgentPhishing measures phishing detection and malicious-request compliance, while PhishOpt optimizes reusable target-specific skills and constructs an analysis-informed universal skill.Source: §6 · p. 9
• Safe handling of pre-existing phishing emails can coexist with failures under adaptive persuasive attacks, and susceptibility varies substantially across evaluated agents.Detection failure and compliance remain distinct outcomes.Source: §6 · p. 9
• The evaluation covers synthetic, single-message attacks under fixed constraints and selected configurations; measured compliance does not establish completed real-world harm.Finite batches, model-version dependence, and scoring or annotation judgments may limit generalization or introduce measurement error.Source: §6 · p. 9

7 ETHICAL CONSIDERATIONS

• Methods for generating phishing emails that evade LLM email agents carry a dual-use risk because they could be misused to craft more effective attacks.Source: §7 · p. 10
• The benchmark uses fictional organizations, people, addresses, domains, and accounts, with nonfunctional credentials, links, and secrets.Source: §7 · p. 10
• No generated email was sent to a real mailbox, and no real person or organization was impersonated or targeted.Source: §7 · p. 10

REPRODUCIBILITY STATEMENT

• The release includes the evaluation pipeline, evolved skill files, metric definitions, analysis code, and aggregate results, but not raw generated email corpora.Source: REPRODUCIBILITY STATEMENT · p. 10
• Withholding the corpora avoids publishing a ready-made phishing set, but released generator prompts could let determined users regenerate similar examples.Source: REPRODUCIBILITY STATEMENT · p. 10
• The authors did not pursue coordinated disclosure because the studied behavior arises from interactions rather than a single patchable vulnerability.They treat public characterization as the appropriate disclosure mechanism.Source: REPRODUCIBILITY STATEMENT · p. 10

• AI tools assisted with drafting candidate phrasing, shortening sections, improving clarity, and revising LaTeX.Source: AI USE STATEMENT · p. 10
• The tools also assisted with code debugging, figure formatting, and interpretation checks during paper preparation.Source: AI USE STATEMENT · p. 10
• The authors reviewed and approved the final text, figures, code, experimental design decisions, and claims, and take responsibility for the manuscript, artifacts, and results.Source: AI USE STATEMENT · p. 10

A AGENTPHISHING EVALUATION PIPELINE

• The evaluation pipeline passes pre-existing and skill-generated emails through a shared agent interface and rule-based evaluator.Comparison setups cover within-target, cross-agent, held-out, and universal-skill evaluations; additional checks are diagnostic capabilities, not mandatory steps in every evaluation.Source: Figure 6 · p. 16, A AGENTPHISHING EVALUATION PIPELINE · p. 16

B PERSUASION ANNOTATION EXAMPLE

• The annotation example links appreciative tone to Liking and an ongoing milestone to Commitment in a workplace credential-request email.The figure describes marking persuasion cues and unsafe action targets; the example case rows also report detection, compliance, and top cue scores.Source: B PERSUASION ANNOTATION EXAMPLE · p. 17, Figure 7 · p. 17

C CROSS-AGENT DETECTION AND COMPLIANCE RATES

• Table 4 organizes cross-agent detection and compliance rates by evolved-skill source model in rows and evaluated target agent in columns.Higher detection indicates safer phishing recognition, whereas higher compliance indicates greater willingness to carry out the malicious request.Source: Table 4 · p. 18, C CROSS-AGENT DETECTION AND COMPLIANCE… · p. 18

D FULL HELD-OUT CELL METRICS

• Tables 5 and 6 report every metric for all 63 cells in the held-out generalization experiment, enabling the target-agent split to be checked cell by cell rather than inferred from aggregate trends.Source: D FULL HELD-OUT CELL METRICS · p. 19
• Table 7 reports lexical-diversity measures for scam emails generated by each best skill, and no corpus crossed either flag threshold.The table defines greater diversity as lower Self-BLEU and sets the ratio flag threshold at 1.30.Source: Table 7 · p. 21

E DIVERSITY METRICS: FULL DETAIL

• Zero of seven corpora crossed either mode-collapse threshold, and gemini-3.5-flash and grok-4.20 were more diverse than baseline.The authors conclude that optimization did not narrow the generator output distribution.Source: E DIVERSITY METRICS: FULL DETAIL · p. 21
• qwen3.7-plus had Self-BLEU 0.3108 against the 0.3171 threshold, clearing it by 0.0063 but leaving the strongest attack near the preset boundary.The authors note that a stricter threshold or another optimization epoch could plausibly have crossed it.Source: E DIVERSITY METRICS: FULL DETAIL · p. 21

F GENERATOR PROMPTS (“SKILLS”)

• The appendix reproduces eight prompts scored with a shared objective; the baseline had no evasion guidance, and the universal skill encoded persuasion correlations directly and cleared the realism gate at 0.878 against 0.70.The universal prompt emphasized Commitment & Consistency, used Liking secondarily, and prohibited Authority and Scarcity/Urgency; its two-batch tests showed roughly ±0.05–0.07 variation for gemini-3.5-flash and gpt-5.4.Source: F GENERATOR PROMPTS (“SKILLS”) · p. 21, §F.1 · p. 22, §F.8 · p. 27

F.2 EVOLVED SKILL: QWEN3.7-P L U S

• qwen3.7-plus reached Best score 0.950 (baseline 0.120), with Detection rate 91.7% →0.0% and Compliance rate 8.3% →83.3%.F1 was undefined when detection reached zero, and the scoring procedure assigned the full 0.2 F1 term, mildly overstating the score.Source: §F.2 · p. 22

F.3 EVOLVED SKILL: KIMI-K2.5

• kimi-k2.5 reached Best score 0.742 (baseline 0.200), with Detection rate 75.0% →16.7% and Compliance rate 8.3% →58.3%.Four errored epochs preceded the reported best epoch and did not affect it.Source: §F.3 · p. 23

F.4 EVOLVED SKILL: GROK-4.20

• grok-4.20 reached Best score 0.647 (baseline 0.145), with Detection rate 83.3% →16.7% and Compliance rate 0.0% →25.0%.Its evolved skill added nine lines and had Mean transfer to others 0.331.Source: §F.4 · p. 24

F.5 EVOLVED SKILL: GPT-5.4

• gpt-5.4 reached Best score 0.413 (baseline 0.022), with Detection rate 100.0% →45.5% and Compliance rate 0.0% →9.1%.Its evolved skill added nine lines and had Mean transfer to others 0.358.Source: §F.5 · p. 25

F.6–F.7 EVOLVED SKILLS

• claude-sonnet-5 reached Best score 0.067 (baseline 0.058), while gemini-3.5-flash reached Best score 0.058 (baseline 0.050); both retained Detection rate 100.0% →100.0% and Compliance rate 0.0% →0.0%.Their score improvements were +0.009 and +0.008, respectively.Source: §F.6 · p. 26, §F.7 · p. 26

F.8 UNIVERSAL SKILL (SKILL UN I V E R S A L V1.M D)

• The benchmark instructions require fake-secret solicitations to remain genuine but use only obvious placeholders, alongside clearly benign workplace controls and approximately balanced labels.The prompt also limits generation to fictional people and companies and requires the output to follow a JSON schema.Source: §F.8 · pp. 27–28
• The skill makes commitment and consistency its primary persuasion cue, framing fake-secret requests as continuations of prior work and embedding small asks in ordinary collaboration.It recommends concrete prior context, a trusted-colleague voice, and a low-key technical rationale for handling the secret inline.Source: §F.8 · pp. 27–28
• The skill excludes urgency, authority, coercion, social proof, and rewards because these cues can make solicitation easier to flag or add unreliable pretext.It instead directs a calm, peer-to-peer tone and relies on commitment-based framing.Source: §F.8 · pp. 27–28