Source-linked AI summary
Enhancing User Resilience Against AI-Augmented Phishing: A Two-Stage Framework for Detection and Personalized Training
Weihao Qu, Gurmeet Singh, Daniel Crawford, Bingjun Li, Jalen Smith
TL;DR
AI has lowered the barrier for phishing attackers and enabled increasingly convincing messages, while security awareness education struggles to keep pace. CyberGLA combines EmailKnight’s multi-level detection with an LLM-based coach that personalizes training from detection results. In a pilot study, CyberGLA training produced higher quiz accuracy than Mimecast, though larger and more diverse longitudinal evaluation is needed.
Problem
AI enables sophisticated, personalized phishing that can evade conventional defenses, while existing security awareness education struggles to match evolving threats.
Method
CyberGLA combines EmailKnight’s multi-level email analysis with an LLM-based security coach that selects personalized training modules from detection outcomes.
Results
Average quiz accuracy improved by 20–25% for CyberGLA participants compared with modest gains for Mimecast participants, and 78% of the CyberGLA group submitted feedback.
Takeaways & Limitations
The pilot suggests that combining detection with adaptive, customized training can improve phishing awareness and engagement compared with a commercial baseline.
Takeaways & Limitations
CyberGLA’s cultural and contextual adaptability remains untested, and its curated phishing phrase lists require frequent updates as attacker language evolves.
Abstract
from arXiv · showhide
The rapid development of artificial intelligence, including agents and deepfake techniques, has accelerated phishing attacks and lowered the threshold for attackers. Modern phishing attacks now blend multiple tactics, including social engineering, URL spoofing, and AI deepfakes enabling adversaries to craft highly convincing messages that exploit human vulnerabilities and bypass traditional detection systems. At the same time, current security awareness education struggles to keep up with the speed, sophistication, and complexity of these evolving threats. To address this challenge, we propose a two-stage anti-phishing framework, CyberGLA, that combines technical defense and user-centered security education. In the Detection stage, we introduce EmailKnight, a spoof detection tool that performs multi-level email analysis. To enhance user awareness, the Training stage incorporates a large language model (LLM)-based security coach that dynamically selects personalized training modules based on the outcomes of the Detection stage. This dual purpose design philosophy enables effective protection against the evolving threats of modern email phishing attacks.
I. INTRODUCTION
AI has intensified phishing by lowering attack barriers and enabling highly sophisticated, personalized messages, while existing defenses and training remain insufficient. CyberGLA addresses these challenges through EmailKnight detection paired with personalized, LLM-driven security training.
- Threat Context: Phishing remains a persistent threat and can enable credential theft, ransomware, and large-scale fraud.APWG reported 1,003,924 phishing incidents in the first quarter of 2024.
- Threat Context: AI lowers the barrier for attackers and enables personalized, grammatically flawless phishing messages that are increasingly difficult to detect.AI-generated messages can combine social engineering, URL spoofing, open-relay exploitation, and other tactics.
- Research Questions: The study asks how AI-powered phishing affects college students and how advanced attacks can be prevented and mitigated.These questions motivate the framework’s combined technical and educational approach.
- Proposed Framework: CyberGLA combines EmailKnight’s multi-level email inspection with an LLM-based coach that delivers adaptive, scenario-specific lessons from detection results.EmailKnight examines headers, metadata, content, and attachments, while training targets users’ identified vulnerabilities.
- Evaluation: A pilot evaluation compared EmailKnight with other detection tools and assessed CyberGLA training across diverse user demographics.The pilot included 51 volunteer undergraduate participants.
- Contributions: The paper contributes EmailKnight and CyberGLA, integrating accurate email detection with personalized, LLM-driven security training.The design targets modern AI-enabled phishing threats.
II. BACKGROUND: DETECTION AND EDUCATION
Phishing defenses generally combine technical detection and user education, but each approach has limitations against evolving AI-powered attacks. CyberGLA responds by integrating detection with adaptive, scenario-specific training.
- Overview: Phishing defenses generally fall into technical detection and user education.
- Detection: Rule-based detection uses metadata and authentication protocols, while machine-learning methods classify emails from extracted features.EmailKnight extends these approaches with human-factor cues, spoofed URLs, and AI-generated content.
- Education: Existing training includes video instruction, interactive simulations, and gamified designs, with interactive approaches improving detection skills more effectively than static instruction.Interactive simulations may lose impact over time.
- Integrated Need: Purely technical defenses may fail against evolving AI-powered attacks, while generic training may struggle to sustain engagement.
- Integrated Need: CyberGLA combines EmailKnight detection with adaptive, scenario-specific education tailored to user vulnerabilities.
III. SYSTEM ARCHITECTURE
CyberGLA uses EmailKnight as a multi-layered detection component that combines protocol, header, message, URL, behavioral, and AI-based analysis. Its outputs support risk explanations and personalized training through an LLM coach.
- Personalized Training Platform: A predefined prompt maps technical anomalies from detection reports to educational modules and guides the LLM coach’s context-aware training summaries.
- Detection Tool: EmailKnight: EmailKnight integrates protocol checks with behavioral and AI-based analysis to identify phishing emails that evade conventional defenses.
- Detection Strategies: The tool evaluates authentication status, open-relay indicators, header anomalies, device origin, Message-ID domains, sender-path mismatches, and displayed versus actual URLs.These checks target spoofing and message-injection signals across multiple email fields.
- Detection Strategies: EmailKnight flags shortened URLs and mismatches between displayed link text and actual destinations.
- Detection Strategies: Human-factor detection uses curated high-risk phrases and a three-term threshold to identify social-engineering cues while limiting premature false positives.
- Detection Strategies: AI-driven attachment analysis scans images, PDFs, videos, and audio for signs of generated or manipulated deepfake content.Verdicts include “Likely Real,” “Possibly Cloned,” and “Likely Fake.”
- Usability and Privacy: Analysis runs locally, assigns color-coded threat scores with explanations, and supports PDF or CSV report export without storing sensitive content externally.The tool connects securely to mail servers through IMAP using app passwords.
2) Usability and Privacy:
Outsourced cybersecurity and limited breach disclosure can leave institutions and individuals with insufficient real-time visibility into security incidents.
- Usability and Privacy: Institutions that outsource cybersecurity may lack real-time forensic visibility.
- Usability and Privacy: Some cyber insurance contracts or regulatory regimes allow non-disclosure of minor breaches below financial or operational thresholds.The passage gives examples including fewer than 500 affected individuals or no material harm.
B. Personalized Training Platform
CyberGLA converts EmailKnight’s detection results into personalized, scenario-based security education. An LLM coach maps reported threats to targeted explanations and training modules.
- CyberGLA uses users’ flagged inbox risks to create adaptive, scenario-based security training.
- EmailKnight categorizes dangerous emails by threats such as lookalike domains, open relays, spoofed links, social engineering, and deepfakes.
- For each flagged message, users receive an AI-generated explanation of risks, red flags, and remediation or reporting steps.
- Repeated exposure to threats such as urgent-payment scams triggers targeted modules instead of generic cybersecurity content.
- A predefined prompt instructs the LLM coach to map technical anomalies from detection reports to educational modules and generate context-aware summaries.
2) Predefined Prompt and LLM Orchestration:
CyberGLA delivers modular training through an LLM orchestration layer that connects detected email anomalies with targeted lessons. Its scenario-based content spans technical, human-factor, URL, authentication, and deepfake threats, with flexible and privacy-conscious deployment.
- Predefined Prompt and LLM Orchestration: The training library contains modular, scenario-driven lessons focused on major phishing techniques.
- Modular, Scenario-Based Lessons: Human-factors lessons address social engineering, urgency, and trust cues exploited by phishing attacks.
- Modular, Scenario-Based Lessons: Lookalike-domain lessons teach users to identify typosquatting and visual deception in URLs and sender addresses.
- Modular, Scenario-Based Lessons: Open-relay lessons explain how SMTP vulnerabilities can bypass authentication and impersonate trusted senders.
- Modular, Scenario-Based Lessons: Spoofed-link lessons cover mismatched anchor text, shortened URLs, and link manipulation.
- Modular, Scenario-Based Lessons: SPF, DKIM, and DMARC lessons describe authentication bypasses and warning signs in email headers.
- Modular, Scenario-Based Lessons: Deepfake lessons use video and audio demonstrations to train users to recognize AI-generated impersonation.
- Modular, Scenario-Based Lessons: Interactive quizzes, annotated email walkthroughs, and multimedia demonstrations personalize practice, while the modular web application supports rapid deployment of new scenarios.
4) Interactive, Multimedia User Experience:
CyberGLA’s experience is designed to evolve with user needs through modular deployment and interactive content. The described interface combines detection and training materials, including multimedia learning elements.
- The platform architecture supports future additions such as gamified progress tracking, badges, certificates, and shareable learning reports.
- Figure 2 presents EmailKnight’s detection result alongside CyberGLA’s customized training recommendation.
- Figure 2 includes an interactive training module focused on human factors in phishing.
IV. EVALUATION
The evaluation tested EmailKnight across five phishing scenarios using automated samples and compared it with established email-security tools. EmailKnight flagged all phishing cases and correctly classified the benign email, while the comparison tools had narrower capabilities.
- The study generated 100 unique phishing emails for each of five scenarios, totaling 500 malicious samples plus benign controls.
- Table I reports mean threat scores and classification accuracy across the 100-sample batches.
- EmailKnight flagged all phishing cases and correctly classified the benign email.
- MXToolbox provided header details but failed to detect human-factor cues, open relays, or AI content, while MailWasher revealed spoofed URLs without flagging phishing or suggesting remediation.
B. Education Effective Evaluation
A controlled pilot study compared CyberGLA with Mimecast and no training, finding stronger quiz performance and positive participant feedback for CyberGLA. The authors characterize the evidence as exploratory and identify limits to generalizability.
- Study Design: 51 participants were divided into CyberGLA, Mimecast, and no-training groups before completing a 25-question phishing quiz.The groups contained 18, 18, and 15 participants, respectively, after 30 minutes of learning.
- Quantitative Results: Average quiz accuracy improved by 20–25% for CyberGLA participants, compared with modest gains for the Mimecast group.CyberGLA outperformed the commercial awareness baseline in the reported quantitative results.
- Qualitative Results: 14 of 18 CyberGLA participants submitted feedback, and all respondents said the system was helpful and recommended it.The reported response rate was 78%.
- Qualitative Results: Participants described the training as fun and requested more detail and additional games in the learning modules.These suggestions point to opportunities for further customization and gamification.
- Interpretation: The pilot suggests improved phishing awareness and engagement relative to a commercial baseline, but larger, more diverse, longitudinal evaluation is needed.The study also notes restricted statistical, technical, contextual, and long-term generalizability.
3) Limitation and Threats:
CyberGLA’s real-world scalability and generalizability remain constrained by untested contextual adaptability, evolving attacker language, and the need for frequent phrase-list updates.
- Scaling Constraints: CyberGLA’s cultural and contextual adaptability remains untested because phishing cues vary across regions and organizations.This limits confidence in deployment across different environments.
- Scaling Constraints: EmailKnight’s curated phishing phrase lists require frequent updates to reflect evolving attacker language.The paper identifies phishing corpus mining and threat-intelligence integration as future priorities.
- Study Setting: The evaluation used simulated phishing experiments with voluntary student participants and anonymous, non-identifiable data under university research guidelines.Participation was voluntary, no incentives were provided, and no institutional login or identifying data were retained.
VI. CONCLUSIONS AND FUTURE WORK
The paper concludes with a two-stage framework combining technical detection and adaptive, customized interactive training, while outlining broader evaluation and deployment directions.
- Conclusion: CyberGLA integrates technical detection with adaptive, customized interactive training in a two-stage anti-phishing framework.Future work includes larger and more diverse evaluations, standardized phishing corpora, longitudinal analysis, and automated phrase-bank updates.
- Future Work: Future work will extend the framework to text phishing, voice-cloning phone scams, video-call deepfakes, and high-risk demographics such as elderly users.The authors also plan to develop additional interactive games to enhance training engagement.