PaperlayerView exact revision on arXiv

Source-linked AI summary

Enhancing User Resilience Against AI-Augmented Phishing: A Two-Stage Framework for Detection and Personalized Training

Weihao Qu, Gurmeet Singh, Daniel Crawford, Bingjun Li, Jalen Smith

arXiv:2608.21547v1cs.CR

TL;DR

AI has lowered the barrier for phishing attackers and enabled increasingly convincing messages, while security awareness education struggles to keep pace. CyberGLA combines EmailKnight’s multi-level detection with an LLM-based coach that personalizes training from detection results. In a pilot study, CyberGLA training produced higher quiz accuracy than Mimecast, though larger and more diverse longitudinal evaluation is needed.

  • Problem

    AI enables sophisticated, personalized phishing that can evade conventional defenses, while existing security awareness education struggles to match evolving threats.

    Source: §I · p. 1

  • Method

    CyberGLA combines EmailKnight’s multi-level email analysis with an LLM-based security coach that selects personalized training modules from detection outcomes.

    Source: §I · p. 2, A. Detection Tool: EmailKnight · p. 3, 1) Integration of Detection and Training · p. 3

  • Results

    Average quiz accuracy improved by 20–25% for CyberGLA participants compared with modest gains for Mimecast participants, and 78% of the CyberGLA group submitted feedback.

    Source: 2) Results · p. 5

  • Takeaways & Limitations

    The pilot suggests that combining detection with adaptive, customized training can improve phishing awareness and engagement compared with a commercial baseline.

    Source: 2) Results · p. 6

    CyberGLA’s cultural and contextual adaptability remains untested, and its curated phishing phrase lists require frequent updates as attacker language evolves.

    Source: C. Self-Critique and Future… · p. 6

Abstract

from arXiv · show

The rapid development of artificial intelligence, including agents and deepfake techniques, has accelerated phishing attacks and lowered the threshold for attackers. Modern phishing attacks now blend multiple tactics, including social engineering, URL spoofing, and AI deepfakes enabling adversaries to craft highly convincing messages that exploit human vulnerabilities and bypass traditional detection systems. At the same time, current security awareness education struggles to keep up with the speed, sophistication, and complexity of these evolving threats. To address this challenge, we propose a two-stage anti-phishing framework, CyberGLA, that combines technical defense and user-centered security education. In the Detection stage, we introduce EmailKnight, a spoof detection tool that performs multi-level email analysis. To enhance user awareness, the Training stage incorporates a large language model (LLM)-based security coach that dynamically selects personalized training modules based on the outcomes of the Detection stage. This dual purpose design philosophy enables effective protection against the evolving threats of modern email phishing attacks.

I. INTRODUCTION

AI has intensified phishing by lowering attack barriers and enabling highly sophisticated, personalized messages, while existing defenses and training remain insufficient. CyberGLA addresses these challenges through EmailKnight detection paired with personalized, LLM-driven security training.

Threat Context

  • Phishing remains a persistent threat and can enable credential theft, ransomware, and large-scale fraud.APWG reported 1,003,924 phishing incidents in the first quarter of 2024.Source: §I · p. 1
  • AI lowers the barrier for attackers and enables personalized, grammatically flawless phishing messages that are increasingly difficult to detect.AI-generated messages can combine social engineering, URL spoofing, open-relay exploitation, and other tactics.Source: §I · p. 1

Research Questions

  • The study asks how AI-powered phishing affects college students and how advanced attacks can be prevented and mitigated.These questions motivate the framework’s combined technical and educational approach.Source: §I · p. 1

Proposed Framework

  • CyberGLA combines EmailKnight’s multi-level email inspection with an LLM-based coach that delivers adaptive, scenario-specific lessons from detection results.EmailKnight examines headers, metadata, content, and attachments, while training targets users’ identified vulnerabilities.Source: §I · p. 2

Evaluation

  • A pilot evaluation compared EmailKnight with other detection tools and assessed CyberGLA training across diverse user demographics.The pilot included 51 volunteer undergraduate participants.Source: §I · p. 2

Contributions

  • The paper contributes EmailKnight and CyberGLA, integrating accurate email detection with personalized, LLM-driven security training.The design targets modern AI-enabled phishing threats.Source: §I · p. 2

II. BACKGROUND: DETECTION AND EDUCATION

Phishing defenses generally combine technical detection and user education, but each approach has limitations against evolving AI-powered attacks. CyberGLA responds by integrating detection with adaptive, scenario-specific training.

Overview

  • Phishing defenses generally fall into technical detection and user education.Source: §II · p. 2

Detection

  • Rule-based detection uses metadata and authentication protocols, while machine-learning methods classify emails from extracted features.EmailKnight extends these approaches with human-factor cues, spoofed URLs, and AI-generated content.Source: §II · p. 2

Education

  • Existing training includes video instruction, interactive simulations, and gamified designs, with interactive approaches improving detection skills more effectively than static instruction.Interactive simulations may lose impact over time.Source: §II · p. 2

Integrated Need

  • Purely technical defenses may fail against evolving AI-powered attacks, while generic training may struggle to sustain engagement.Source: §II · p. 2
  • CyberGLA combines EmailKnight detection with adaptive, scenario-specific education tailored to user vulnerabilities.Source: §II · p. 2

III. SYSTEM ARCHITECTURE

CyberGLA uses EmailKnight as a multi-layered detection component that combines protocol, header, message, URL, behavioral, and AI-based analysis. Its outputs support risk explanations and personalized training through an LLM coach.

Personalized Training Platform

  • A predefined prompt maps technical anomalies from detection reports to educational modules and guides the LLM coach’s context-aware training summaries.Source: §III · p. 3

Detection Tool: EmailKnight

Detection Strategies

  • The tool evaluates authentication status, open-relay indicators, header anomalies, device origin, Message-ID domains, sender-path mismatches, and displayed versus actual URLs.These checks target spoofing and message-injection signals across multiple email fields.Source: §1 · p. 3, §3 · p. 3
  • EmailKnight flags shortened URLs and mismatches between displayed link text and actual destinations.Source: §7 · p. 3
  • Human-factor detection uses curated high-risk phrases and a three-term threshold to identify social-engineering cues while limiting premature false positives.Source: §7 · p. 3
  • AI-driven attachment analysis scans images, PDFs, videos, and audio for signs of generated or manipulated deepfake content.Verdicts include “Likely Real,” “Possibly Cloned,” and “Likely Fake.”Source: §7 · p. 3

Usability and Privacy

  • Analysis runs locally, assigns color-coded threat scores with explanations, and supports PDF or CSV report export without storing sensitive content externally.The tool connects securely to mail servers through IMAP using app passwords.Source: 2) Usability and Privacy · p. 3

2) Usability and Privacy:

Outsourced cybersecurity and limited breach disclosure can leave institutions and individuals with insufficient real-time visibility into security incidents.

Usability and Privacy

  • Institutions that outsource cybersecurity may lack real-time forensic visibility.Source: 2) Usability and Privacy · p. 3
  • Some cyber insurance contracts or regulatory regimes allow non-disclosure of minor breaches below financial or operational thresholds.The passage gives examples including fewer than 500 affected individuals or no material harm.Source: 2) Usability and Privacy · p. 3

B. Personalized Training Platform

CyberGLA converts EmailKnight’s detection results into personalized, scenario-based security education. An LLM coach maps reported threats to targeted explanations and training modules.

2) Predefined Prompt and LLM Orchestration:

CyberGLA delivers modular training through an LLM orchestration layer that connects detected email anomalies with targeted lessons. Its scenario-based content spans technical, human-factor, URL, authentication, and deepfake threats, with flexible and privacy-conscious deployment.

Predefined Prompt and LLM Orchestration

Modular, Scenario-Based Lessons

4) Interactive, Multimedia User Experience:

CyberGLA’s experience is designed to evolve with user needs through modular deployment and interactive content. The described interface combines detection and training materials, including multimedia learning elements.

IV. EVALUATION

The evaluation tested EmailKnight across five phishing scenarios using automated samples and compared it with established email-security tools. EmailKnight flagged all phishing cases and correctly classified the benign email, while the comparison tools had narrower capabilities.

B. Education Effective Evaluation

A controlled pilot study compared CyberGLA with Mimecast and no training, finding stronger quiz performance and positive participant feedback for CyberGLA. The authors characterize the evidence as exploratory and identify limits to generalizability.

Study Design

  • 51 participants were divided into CyberGLA, Mimecast, and no-training groups before completing a 25-question phishing quiz.The groups contained 18, 18, and 15 participants, respectively, after 30 minutes of learning.Source: 1) Setting-up · p. 5

Quantitative Results

  • Average quiz accuracy improved by 20–25% for CyberGLA participants, compared with modest gains for the Mimecast group.CyberGLA outperformed the commercial awareness baseline in the reported quantitative results.Source: 2) Results · p. 5

Qualitative Results

  • 14 of 18 CyberGLA participants submitted feedback, and all respondents said the system was helpful and recommended it.The reported response rate was 78%.Source: 2) Results · p. 5
  • Participants described the training as fun and requested more detail and additional games in the learning modules.These suggestions point to opportunities for further customization and gamification.Source: 2) Results · p. 6

Interpretation

  • The pilot suggests improved phishing awareness and engagement relative to a commercial baseline, but larger, more diverse, longitudinal evaluation is needed.The study also notes restricted statistical, technical, contextual, and long-term generalizability.Source: 2) Results · p. 6, 3) Limitation and Threats · p. 6

3) Limitation and Threats:

CyberGLA’s real-world scalability and generalizability remain constrained by untested contextual adaptability, evolving attacker language, and the need for frequent phrase-list updates.

Scaling Constraints

  • CyberGLA’s cultural and contextual adaptability remains untested because phishing cues vary across regions and organizations.This limits confidence in deployment across different environments.Source: C. Self-Critique and Future… · p. 6
  • EmailKnight’s curated phishing phrase lists require frequent updates to reflect evolving attacker language.The paper identifies phishing corpus mining and threat-intelligence integration as future priorities.Source: C. Self-Critique and Future… · p. 6

Study Setting

  • The evaluation used simulated phishing experiments with voluntary student participants and anonymous, non-identifiable data under university research guidelines.Participation was voluntary, no incentives were provided, and no institutional login or identifying data were retained.Source: §V · p. 6

VI. CONCLUSIONS AND FUTURE WORK

The paper concludes with a two-stage framework combining technical detection and adaptive, customized interactive training, while outlining broader evaluation and deployment directions.

Conclusion

  • CyberGLA integrates technical detection with adaptive, customized interactive training in a two-stage anti-phishing framework.Future work includes larger and more diverse evaluations, standardized phishing corpora, longitudinal analysis, and automated phrase-bank updates.Source: §VI · p. 6

Future Work

  • Future work will extend the framework to text phishing, voice-cloning phone scams, video-call deepfakes, and high-risk demographics such as elderly users.The authors also plan to develop additional interactive games to enhance training engagement.Source: §VI · p. 6
Loading 2608.21547v1…