Source-linked AI summary
Engineered Persuasion: Evaluating Personalized Pretexts in LLM-Generated Spear Phishing
Jerson Francia, Derek Hansen, Benjamin Schooley, Shydra Valynn Murray
TL;DR
The paper examines how cumulative workplace details in LLM-generated phishing pretexts relate to perceived credibility and stated responses. In a disclosed survey, participants compared messages across four personalization levels and explained which details seemed credible or suspicious. More personalized messages were rated as more convincing and had higher stated click intention, but contextual mismatches raised suspicion and the survey does not estimate real-world clicking.
Problem
The paper asks when additional workplace information makes an LLM-generated phishing pretext seem more convincing and when it undermines credibility.
Source: §9 · p. 12Method
The study compares four cumulative workplace-information tiers in a disclosed survey and analyzes participants’ message ratings, stated actions, and explanations.
Source: §1 · p. 2Results
The most personalized messages were rated more convincing and had higher odds of stated click intention than Level 1, while incorrect, vague, or inappropriate details raised suspicion.
Source: §9 · p. 12Takeaways & Limitations
Personalized details should be treated as claims to verify rather than proof of legitimacy, and training should reinforce verification and make suspicious messages easy to report.
Source: §9 · p. 12The disclosed repeated-rating survey cannot estimate real-world click-through rates.
Source: §8 · p. 11
Abstract
from arXiv · showhide
Large language models can insert workplace details into phishing pretexts at low cost, but those details may either support or undermine a message's credibility. We recruited 180 U.S. working adults to evaluate simulated, AI-generated phishing emails in a disclosed survey. The emails used four cumulative levels of information: workplace (Level 1); recipient name and job title; job responsibilities; and coworker/shared-project context (Level 4). Participants rated each message's convincingness from 0 to 100, chose one stated action (open the link, investigate, delete, or report), and explained why their highest- and lowest-rated messages stood out. Across 1,436 valid evaluations, convincingness increased by 2.40 points per personalization level in a sensitivity analysis, while the odds of expressing click intention increased by 28\% per level. Among participants who did not express an intention to click, investigation remained common, reporting declined, and deletion increased. A post-hoc descriptive analysis found higher ratings and click intention for messages from a named person who referenced a supplied coworker than for messages from a department or entity. Qualitative coding showed why added detail could help or hurt: details that matched participants' roles and routines supported credibility, while incorrect, vague, or channel-inappropriate details raised suspicion. Together, the results highlight that personalization is not simply a matter of adding more details: it depends on whether the pretext fits the recipient's work context. We discuss how this distinction can inform workplace cybersecurity training.
1. Introduction
Phishing remains a major threat, and workplace context can make a pretext either more credible or more suspicious. This study compares how cumulative workplace details in AI-generated spear-phishing messages relate to judgments and stated actions in a disclosed survey.
1. Introduction
- Phishing remains a major concern despite improving technical defenses, with reports documenting substantial complaint volume and losses.The APWG recorded 1,003,924 phishing attacks in the first quarter of 2025.Source: §1 · p. 1
- Personal details can strengthen a pretext when they fit the recipient’s work context, but wrong, vague, or procedurally inconsistent details can undermine it.Sender identity and references to known people also contribute different authority and familiarity cues.Source: §1 · p. 1
- LLMs can reduce the time, expertise, and cost of drafting phishing content, but lower generation cost does not ensure that a pretext will fit its recipient.The study asks which workplace details make AI-generated phishing messages seem believable or suspicious.Source: §1 · pp. 1–2
- The study compares four cumulative workplace-information tiers and how they relate to convincingness and stated actions in a disclosed survey of simulated phishing messages.Participants knowingly evaluated the messages; the study was not designed to estimate operational phishing success.Source: §1 · p. 2
- The paper’s contributions include within-participant comparison of four information tiers and participant explanations of how contextual fit shaped credibility.The paper also reports a secondary analysis of sender/pretext configurations.Source: §1 · p. 2
2. Review of Related Literature
Prior work links phishing judgments to message cues, recipient knowledge, workplace norms, and personalization, but often compares broad or binary information conditions. This paper examines cumulative workplace-information depth in LLM-generated pretexts while distinguishing survey judgments from operational field behavior.
2. Review of Related Literature
- Phishing judgments depend on message cues, recipient knowledge and habits, and the setting, including workplace context and organizational controls.People can fall for both crude and polished attacks, so risk is not limited to advanced spear phishing.Source: §2 · p. 2
- Personalization depth is the amount of recipient-specific information available to create a pretext, distinct from persuasion strategies such as authority or urgency.The same request may seem routine or suspicious depending on whether sender, wording, and relationship fit workplace norms.Source: §2 · p. 2
- Perceived legitimacy also depends on sender identity interpreted alongside message content and workplace relationships.An unfamiliar sender may seem plausible when role and request fit, while a familiar name may seem suspicious when the request or channel does not.Source: §2 · p. 2
- Prior studies generally find that personal details can raise phishing susceptibility, but effects vary and many studies use binary or broad information conditions.In SpearSim, creators given more target information deceived more recipients and produced more contextually meaningful narratives.Source: §2 · pp. 2–3
- LLMs can lower the time, cost, and writing skill needed to create phishing pretexts, while their effectiveness depends on organizational context.Prior work describes models supporting drafting, scenario generation, and automated iteration in broader deception workflows.Source: §2 · p. 3
- This study’s controlled survey examines judgments across cumulative workplace-information tiers, whereas a prior field experiment measured clicks after emailing around 7,700 recipients.The authors distinguish their perceived convincingness and stated-action measures from operational field behavior.Source: §2 · p. 3
3. Research Questions
The research questions ask how personalization level relates to self-reported convincingness and stated action, and which cues participants regard as persuasive or suspicious. An exploratory question examines associations between impersonation style and those outcomes.
3. Research Questions
- RQ1 asks how personalization level relates to self-reported convincingness and stated action in AI-generated spear-phishing pretexts.Source: §3 · p. 3
- EQ1 asks what associations appear between impersonation style and the outcomes.Source: §3 · p. 3
- RQ2 asks which personalized pretext cues participants report as persuasive or suspicious.Source: §3 · p. 3
4. Methodology
The methodology recruits working adults, collects workplace context, uses an LLM to generate simulated spear-phishing emails, and evaluates participant ratings, stated actions, and explanations.
4.1. Design Rationale
- Four tiers add workplace, name and title, responsibilities, then coworker and project details; generated messages could vary in scenario, tone, persuasion strategy, and sender.Source: §4.1 · pp. 3–4
4.2. Participants
- The self-selected convenience sample comprised 180 U.S. working adults recruited through Prolific.Source: §4.2 · p. 4
4.3. Ethics and consent
- IRB-approved consent disclosed fictional AI-generated messages, voluntary withdrawal, and restricted storage of participant and workplace identifiers.Source: §4.3 · p. 5
4.4. Survey Procedure
- Participants supplied profile fields for eight GPT-4o messages, rated each, and chose one hypothetical action; four failed outputs left 1,436 evaluations.Source: §4.4 · p. 5
4.5. Impersonation style
- Researchers post-hoc coded sender/pretext styles distinguishing entities, named senders, and coworker references; no message used the supplied coworker as sender.Source: §4.5 · p. 5
4.6. Data Handling and Analysis
- Mixed-effects models included participant random intercepts; categorical levels were primary, with ordinal-trend sensitivity, descriptive sender styles, and exploratory demographic analyses.Source: §4.6 · pp. 5–6
4.7. Qualitative Data Analysis
- Of 720 explanations, 701 entered thematic summaries; iterative calibration reached mean κ = .753 across seven estimable parent codes before one coder coded the remainder.Source: §4.7 · p. 6
5. Quantitative Results
Across 1,436 valid evaluations, greater personalization was associated with higher convincingness and open-link intention. Non-click responses, exploratory sender-style patterns, and subgroup tests provide additional context.
5. Quantitative Results
- Among 1,436 valid evaluations, mean convincingness was 60.2 (SD = 29.6) on a 0–100 scale, and 36.9% selected open-link.Source: §5 · p. 6
5.1. Effect of Personalization Level
- Convincingness increased 2.40 points per personalization level, while open-link odds increased 28% per level in sensitivity analyses.The ordinal trend estimates were 2.40 points (95% CI [1.29, 3.51], p < .001) and OR = 1.28 (95% CI [1.13, 1.46], p < .001), respectively.Source: Figure 2 · p. 6, §5.1 · pp. 6–7
- Among non-click responses, investigation stayed near half (49–52%), reporting fell from 37% at Level 1 to 28% at Level 4, and deletion rose from 14% to 20%.The relative risk of reporting rather than deleting decreased 21.7% per personalization level (RRR = 0.78, 95% CI [0.67, 0.92], p = .002).Source: Figure 3 · p. 7, §5.1 · p. 7
5.2. Impersonation Style
- Exploratorily, coworker-referencing named-person messages had the highest convincingness (65.7) and click intention (47.3%), versus department/entity messages (55.2; 31.0%).These post-hoc differences were not independently randomized; all Style 3 messages occurred at Level 4, so the full categorical style-by-personalization interaction was not estimable.Source: §5.2 · p. 7, Figure 4 · p. 7, Table 3 · p. 7
5.3. Other Factors
- None of 16 adjusted tests for age, gender, prior phishing exposure, or detection confidence remained significant (all adjusted p ≥ .795).This does not establish that the subgroups responded identically.Source: §5.3 · p. 7
6. Qualitative Results
Participants judged personalization by its fit with their work: accurate, specific details could support credibility, while incorrect or vague details raised suspicion. Channel, style, source, and requested-action cues also shaped judgments, and coverage patterns showed that personalization was not uniformly helpful.
6. Qualitative Results
- Accurate role, relationship, and project details often supported credibility, but incorrect or vague details created mismatches with participants’ work.Among 329 valenced Personalization/Relevance mentions, 66.9% were positive and 33.1% negative.Source: §6 · p. 8
- Message style and source cues were mixed: workplace-matched tone and plausible authority could support credibility, while mismatched style or unclear sources raised suspicion.Valenced Message Style mentions were 46.6% positive; Source mentions were 42.1% positive.Source: §6 · p. 8
- Requested actions were often treated as phishing indicators, especially credential checks or sensitive requests, though plausible context could make some actions seem credible.Among valenced Requested Action mentions, 76.4% were negative.Source: §6 · pp. 8–9
- Benefit and urgency cues could cut both ways: rewards were more often described positively, while deadlines could signal either realism or pressure.Among valenced Benefit/Favor/Reward mentions, 75.8% were positive; Urgency/Pressure mentions were 52.4% positive.Source: §6 · p. 9
- Communication medium was mainly a negative cue: 58 of 62 valenced mentions (93.5%) were negative, as participants expected some requests through other channels.Examples included checking Teams rather than clicking a document link and expecting a call for certain matters.Source: §6 · p. 9
6.1. Descriptive Coverage of Qualitative Codes
- Among 701 retained responses, Personalization/Relevance and Message Style each appeared in 325 (46.4%); Requested Action appeared in 161 (23.0%).Responses could receive multiple codes, so percentages do not sum to 100%.Source: §6.1 · p. 9
- Personalization appeared in 61.4% of explanations for convincing messages versus 31.2% for less-convincing messages, but it also appeared in explanations for both.Benefit/Favor/Reward was also more common in convincing explanations, whereas Communication Medium and Requested Action were more common in less-convincing explanations.Source: §6.1 · p. 9, Table 5 · p. 9
- Benefit (+51.6%) and Personalization (+33.7%) skewed positive, while Requested Action (−52.8%) and Communication Medium (−87.1%) skewed negative in the net persuasion index.Personalization’s positive score coexisted with suspicion when details were wrong, vague, or implausible; General was residual-only and not comparable.Source: §6.1 · p. 10, Table 6 · p. 10
7. Discussion
The discussion finds that richer workplace personalization was associated with greater convincingness and click intention, while contextual mismatches could raise suspicion. It emphasizes that stated responses and exploratory sender comparisons should be interpreted within the disclosed survey’s limits.
Personalization and stated actions
- Evidence was strongest at Level 4: convincingness and click intention increased with personalization, while non-click responses often involved investigation and reporting declined relative to deletion.Click intention rose descriptively from 31.9% to 43.0%; the categorical models did not show a clear difference at every intermediate level.Source: §7 · p. 10
Coworker references
- Style 3 messages had higher click intention than department or entity messages (47.3% vs. 31.0%), but all Style 3 messages were Level 4, confounding the descriptive comparison.No valid message directly impersonated the supplied coworker; the sender generally referred to the coworker as supporting context.Source: §7 · p. 10
Individual differences
- Age, gender, phishing exposure, and self-reported confidence showed no reliable main or moderation effects after multiple-comparison correction, but smaller subgroup differences remain possible.All adjusted p-values were ≥.795; the study may have lacked power to detect smaller differences.Source: §7 · p. 11
Contextual fit
- Participants’ explanations linked credibility to contextual fit: role, responsibility, and project details helped when accurate, while mismatches and inappropriate details raised suspicion.Personalization/Relevance and Message Style were the most frequent themes (46.4% each); personalization appeared more often in convincing than less-convincing explanations (61.4% vs. 31.2%).Source: §7 · p. 11
- Personalization was not uniformly persuasive: 33.1% of valenced Personalization/Relevance mentions were negative, often because details were incorrect, vague, or difficult to verify.Benefit and Personalization skewed positive, whereas Communication Medium and Requested Action skewed negative; other cues depended more on context.Source: §7 · p. 11
- Requested actions and communication channels could trigger suspicion even in professional-looking messages, especially when an email link conflicted with normal workplace procedures.Participants described some requests as more appropriate through Teams, phone, or an internal portal.Source: §7 · p. 11
Implications
- Training should reinforce verification and reporting, treating personal details as claims to check rather than proof of legitimacy.The authors recommend known portals or contact methods, easy reporting, and future comparisons of role-tailored with generic training; this study did not test an intervention.Source: §7 · p. 11
8. Limitations and Future Work
The study’s disclosed survey and self-reported responses support comparisons across personalization conditions, but constrain claims about workplace behavior. The authors identify limits in ecological validity, sampling, channel coverage, exploratory sender analysis, and qualitative coding.
Study design
- The repeated, disclosed survey design supports comparing four personalization conditions but cannot estimate real-world click-through rates.It allowed evaluation of highly personalized messages without exposing participants, coworkers, or organizations to deceptive workplace messages.Source: §8 · p. 11
Measures
- Self-reported convincingness and open-link choices do not directly represent behavior, and awareness questions may have prompted closer scrutiny than an everyday inbox encounter.Every personalization condition used the same procedure, but survey cues may have affected overall responses.Source: §8 · pp. 11–12
Ecological validity
- The survey omitted inbox features and legitimate workplace messages, so it cannot show how well participants distinguish phishing from routine communication.Forced-choice responses also excluded ignoring a message and taking sequential or combined actions.Source: §8 · p. 12
Scope
- The self-selected U.S. working-adult sample and email-only design leave generalization to other populations, organizational settings, and communication channels untested.The authors call for research in SMS, workplace messaging, and cross-channel attacks.Source: §8 · p. 12
Sender-style analysis
- The post-hoc sender-style analysis cannot isolate impersonation style from personalization because styles were not balanced across levels.The authors recommend independently controlling sender identity and personalization in a fully crossed design.Source: §8 · p. 12
Qualitative coding
- Double-coding agreement applies only to the 103-response codebook-development subset, not the full qualitative corpus.Future work should independently double-code the full corpus or a larger subset using the finalized codebook.Source: §8 · p. 12
9. Conclusion
In a disclosed survey, participants judged more personalized LLM-generated spear-phishing messages as more convincing and more likely to elicit click intention, particularly at Level 4. The conclusion stresses contextual fit, verification, and reporting while distinguishing these survey findings from real-world clicking or compromise.
Conclusion
- The strongest differences appeared at Level 4, where participants rated the most personalized messages as more convincing and had higher odds of expressing click intention than at Level 1.These are survey responses, not estimates of real-world clicking or compromise.Source: §9 · p. 12
- Personalized details supported credibility when they fit participants’ work context, while incorrect, vague, or inappropriate details raised suspicion.Coworker-referencing messages also received higher ratings and click intention, but all appeared at Level 4, so the comparison needs a balanced experiment.Source: §9 · p. 12
- Among responses without click intention, investigation remained common while reporting became less likely relative to deletion, supporting training on verification and easy reporting.The authors recommend treating personalized details as claims to verify rather than proof of legitimacy.Source: §9 · p. 12
Appendix A. Ethical Considerations
The appendix describes participant protections, data handling, and dual-use safeguards for a disclosed survey using simulated phishing messages.
Appendix A. Ethical Considerations
- The study recruited U.S.-based working adults aged 18 or older under IRB approval and implied consent; participation was voluntary and participants could withdraw without penalty.Participants were compensated $5 and were instructed not to reveal confidential or sensitive information.Source: Appendix A · p. 12
- Participants supplied limited work-context fields, including a coworker’s first name and a shared non-confidential project, which the study treated as third-party personal information.The researchers identified privacy risks to participants and to coworkers or organizations referenced in the data.Source: Appendix A · pp. 12–13
- Workplace fields were transmitted to OpenAI’s API, and default abuse-monitoring logs may retain content for up to 30 days; the team does not claim prompts and outputs were never retained.Generated outputs were inspected informally after collection, without a documented real-time content-screening step.Source: Appendix A · p. 13
- Messages stayed within the survey, were disclosed as fictional and AI-generated, and were not sent to real inboxes, reducing deception-related harm and social confusion risks.The authors state that no poststudy debrief was needed under these conditions.Source: Appendix A · p. 13
- Because findings about persuasive cues could be misused, the authors withheld raw workplace-linked messages, participant data, and operational phishing tools while providing a placeholder-based prompt.They frame the remaining dual-use risk as justified by potential defensive benefits.Source: Appendix A · p. 13
Appendix B. Open Science
The public research artifacts support reproduction of primary and descriptive analyses, but not analyses requiring restricted demographics or raw qualitative text.
Appendix B. Open Science
- The release includes analysis code, deidentified quantitative evaluation data, sanitized qualitative codes and valence labels, and deidentified study materials.It excludes identifiers, raw workplace-linked messages, open-ended responses, demographics, phishing-background variables, and any reidentification key; analyses needing restricted demographics or raw qualitative text cannot be reproduced from public data.Source: Appendix B · p. 13
Appendix C. LLM Prompt
The appendix provides the Level 4 prompt and explains how its versions varied across four cumulative workplace-information levels.
Appendix C. LLM Prompt
- The survey used four API calls per participant, requesting two messages per call, and excluded four fallback evaluations when an output was not returned.The deployed setup used GPT-4o with temperature 0.8, top-p 1, and a 1,000-token maximum; unspecified settings used API defaults.Source: Appendix C · p. 13
- Level 1 retained only workplace; Level 2 added name and job title; Level 3 added responsibilities; and Level 4 also included coworker and project context.The prompt fields specify workplace, name, title, two responsibilities, coworker, and project.Source: Appendix C · pp. 13–14
- The prompt required short email-style messages with a [LINK], plausible workplace references, and no invented titles, departments, or institutions.When a message was unofficial, it instructed the model to use a more casual tone and keep each message under 150 words.Source: Appendix C · p. 14
- The Level 4 prompt required both messages to include all profile details, using the first responsibility in message one and the second in message two.The outputs were to be enumerated as 1) and 2).Source: Appendix C · p. 14
Appendix D. Historical Personalization Examples
The appendix presents deidentified historical examples to illustrate participant-level responses across personalization levels, not controlled message comparisons.
Appendix D. Historical Personalization Examples
- These altered examples are illustrative: GPT-4o generated each message independently, so scenario, sender, tone, and request varied across levels; the participant’s response was nonmonotonic.Eight additional deidentified examples are available in the reproducibility artifact.Source: Appendix D · p. 14
Level 1 (Style 1; rating 100; selected open-link)
- The workplace-context message announced enhancements to Meridian Mobility’s Employee Benefits Portal and invited team members to use a link to explore new features and confirm their current benefit selections.It was signed by Benefits Administration, Meridian Mobility.Source: Level 1 (Style 1; rating 100; selected… · p. 14
Level 2 (Style 2; rating 90; selected open-link)
- At Level 2, the message using the recipient’s name and job title received a rating of 90, and the participant selected open-link.It presented a production-protocol update from a Quality Assurance Team sender.Source: Level 2 (Style 2; rating 90; selected… · p. 14
Level 3 (Style 2; rating 50; did not select open-link)
- The quota-tracking email described an upgraded system, addressed Alex as Production Operations Manager at Meridian Mobility, and invited him to review new features through a link.It said the system would help manage a team of 1,200 production staff and meet its targets more efficiently.Source: Level 3 (Style 2; rating 50; did not… · p. 14
Level 4 (Style 3; rating 70; did not select open-link)
- A workplace project message told Alex that Jordan and Morgan were progressing with automated line loaders, requested input on integrating new systems, and asked for feedback on the implementation plan.It noted Alex’s experience keeping automated assembly equipment running smoothly.Source: Level 4 (Style 3; rating 70; did not… · p. 15
LLM Usage Statement
- GPT-4o generated the simulated messages from participant-provided work-context fields, and the authors retain responsibility for the paper’s text, analyses, and claims.Source: LLM Usage Statement · p. 16