Source-linked AI summary
HuntGPT: Integrating Machine Learning-Based Anomaly Detection and Explainable AI with Large Language Models (LLMs)
Tarek Ali, Panos Kostakos
TL;DR
False positives and adoption challenges affect ML anomaly detection, and XAI has seen limited acceptance from incident responders. HuntGPT uses a KDD99-trained Random Forest for anomaly detection, SHAP and LIME, and a conversational agent powered by OpenAI’s Language Learning Model API, predominantly known as ChatGPT. HuntGPT achieved 72% to 82.5% success rates on cybersecurity certification exams, and generated responses were generally comprehensible at a basic college education level.
Problem
False positives and adoption challenges affect ML anomaly detection, and XAI has seen limited acceptance from incident responders.
Source: §I · p. 1Method
HuntGPT uses a KDD99-trained Random Forest for anomaly detection, SHAP and LIME, and a conversational agent powered by OpenAI’s Language Learning Model API, predominantly known as ChatGPT.
Source: §I · p. 2Results
HuntGPT achieved 72% to 82.5% success rates on cybersecurity certification exams, and generated responses were generally comprehensible at a basic college education level.
Source: §V · p. 8Takeaways & Limitations
The authors conclude that conversational agents integrated with XAI can provide explainable, actionable intrusion-detection support and promote user understanding and interaction.
Source: §V · p. 8Future work identifies model refinement and real-time detection capabilities as needed for real-time detection and response requirements.
Source: §V · p. 8
Abstract
from arXiv · showhide
Machine learning (ML) is crucial in network anomaly detection for proactive threat hunting, reducing detection and response times significantly. However, challenges in model training, maintenance, and frequent false positives impact its acceptance and reliability. Explainable AI (XAI) attempts to mitigate these issues, allowing cybersecurity teams to assess AI-generated alerts with confidence, but has seen limited acceptance from incident responders. Large Language Models (LLMs) present a solution through discerning patterns in extensive information and adapting to different functional requirements. We present HuntGPT, a specialized intrusion detection dashboard applying a Random Forest classifier using the KDD99 dataset, integrating XAI frameworks like SHAP and Lime for user-friendly and intuitive model interaction, and combined with a GPT-3.5 Turbo, it delivers threats in an understandable format. The paper delves into the system's architecture, components, and technical accuracy, assessed through Certified Information Security Manager (CISM) Practice Exams, evaluating response quality across six metrics. The results demonstrate that conversational agents, supported by LLM and integrated with XAI, provide robust, explainable, and actionable AI solutions in intrusion detection, enhancing user understanding and interactive experience.
I. INTRODUCTION
HuntGPT addresses false positives and adoption challenges in ML-based threat hunting by combining anomaly detection, explainability, and conversational AI. The prototype is evaluated for cybersecurity knowledge and response readability.
I. INTRODUCTION
- Cyber threat hunting draws on human experts who analyze telemetry and indicators of compromise to isolate threats and test hypotheses.Threat hunters can also use external threat intelligence.Source: §I · p. 1
- False positives in real-world operations and potential information overload for responders and model maintainers complicate ML and XAI adoption.XAI is intended to address challenges from ML integration, but rapid advances may slow adoption.Source: §I · p. 1
- The paper positions LLMs and conversational agents as adaptable tools for actionable AI and response suggestions to threat responders.The authors present HuntGPT as a prototype integrating actionable, interpretable, and explainable AI in cybersecurity operations.Source: §I · pp. 1–2
- HuntGPT analyzes network traffic with a Random Forest trained on KDD99, applies SHAP and LIME, and uses an OpenAI conversational agent.The prototype is assessed using CISM practice exams and six metrics for response readability.Source: §I · p. 2
II. BACKGROUND
The background describes network anomaly detection as traffic classification and reviews ML approaches, while noting cost pressures on smaller organizations and real-world adoption concerns.
II. BACKGROUND
- Small and medium-sized enterprises face budget, staffing, and time constraints, while a medium-sized SOC team is estimated to cost $1,635,000.The cited estimate includes personnel and infrastructure.Source: §II · p. 2
1) Network anomaly detection:
- Anomaly detection treats network traffic analysis as a classification task that separates abnormal, potentially malicious instances from normal traffic.The background also notes evolving detection approaches, including deployment in 5G and decentralized architectures.Source: 1) Network anomaly detection · p. 2
- A reviewed NSL-KDD study found Random Forest outperformed four other examined classifiers.The compared methods were Stochastic Gradient Descent, Logistic Regression, Support Vector Machine, and Sequential Model.Source: 1) Network anomaly detection · p. 2
- Prior work includes a two-phase Random Forest and weighted K-means system and federated approaches for anomaly detection and traffic classification.The cited federated studies address data confidentiality or auditability; blockchain integration increased complexity, with a reported performance impact of 5 to 15%.Source: 1) Network anomaly detection · p. 2
- Research has applied federated learning to detect DDoS attacks on a 5G core network, while real-world efficacy remains an adoption barrier for ML models.The cited 5G approach uses collective intelligence across devices to recognize attacks confidentially.Source: 1) Network anomaly detection · p. 2
2) Explainable AI:
This section presents XAI as a way to make ML predictions more understandable and reviews anomaly-explanation methods. It also reports concerns about operational adoption and analyst workflows.
2) Explainable AI:
- XAI explains which input features most affected an ML prediction, supporting transparency and checks for bias in models or training data.The text describes explanations as important for users to understand, trust, and manage AI in critical applications.Source: 2) Explainable AI · p. 3
- GEE combines an unsupervised VAE for network-flow anomaly detection with gradient-based fingerprinting to explain detected anomalies.Its evaluation found the method effective at detecting different anomalies and identifying attack-representative fingerprints.Source: 2) Explainable AI · p. 3
- DeepAID helps analysts interpret unsupervised deep-learning anomaly decisions using a normal reference point and a simpler finite-state-machine distiller.The distiller is proposed to involve analysts in model decision-making.Source: 2) Explainable AI · p. 3
- A pilot deployment found restricted XAI use and no considerable improvement in analysts’ decision accuracy; a review identified weak attention to workflows and user roles.The review notes that these shortcomings could diminish adoption.Source: 2) Explainable AI · p. 3
3) Chatbots for security & ChatGPT:
The section reviews chatbots and GPT applications in cybersecurity, then identifies limited research on LLM integration into cyber-hunting interfaces. It also notes challenges in evaluating cybersecurity accuracy.
3) Chatbots for security & ChatGPT:
- SecBot uses neural networks and NLP to identify cyberattacks, suggest solutions and configurations, and inform cybersecurity investment and risk decisions.It is presented as a conversational agent for cybersecurity planning and management.Source: 3) Chatbots for security & ChatGPT · p. 3
- GPT has been used to generate cybersecurity policies, with customized prompts helping in specific scenarios, and GPT-2C parsed Unix commands with 89% inference accuracy.GPT-2C processed logs from a live Cowrie SSH honeypot with minimal execution latency.Source: 3) Chatbots for security & ChatGPT · p. 3
- The literature describes limited research on integrating LLMs into cyber-hunting interfaces, despite interest in conversational agents for sharing expertise with nonprofessionals.A financial-domain study suggests ChatGPT may help convey complex principles to non-specialists.Source: 3) Chatbots for security & ChatGPT · p. 3
- Prompt engineering is presented as a potential way to address LLM shortcomings, while evaluating LLM performance across domains remains challenging.The cited work reports prompt-related improvements in reliability across generalizability, social biases, calibration, and factuality.Source: 3) Chatbots for security & ChatGPT · p. 3
- The section argues that measuring the accuracy of cybersecurity information is important, following discussion of GPT performance in other domains.The medical-exam study and user-satisfaction finding are cited as examples of broader-domain evaluation.Source: 3) Chatbots for security & ChatGPT · p. 4
III. ARCHITECTURE
The architecture separates analytics, storage, and user interaction into three layers. Its modular design is intended to let components evolve independently.
III. ARCHITECTURE
- Three layers assign packet analysis to the analytics engine, storage to Elasticsearch and Amazon S3, and analyst interaction to a Gradio interface integrated with OpenAI’s Language Model API.The layers are designed to present analytical outcomes and support ongoing analyst discussions.Source: §III · p. 4
- The modular configuration is intended to let each layer be developed, maintained, and enhanced independently, supporting flexibility and scalability.The authors cite component segregation as the reason for this design.Source: §III · p. 4
- Figure 1 presents a high-level diagram of dashboard integration within the three-layer architecture.The text identifies the figure as illustrating the system’s layers.Source: Figure 1 · p. 4, §III · p. 4
A. Component Diagram
The system pairs an anomaly-detection application server with an IDS dashboard. The server handles prediction and explanation workflows, while the dashboard presents information and supports interaction with an AI assistant.
A. Component Diagram
- Figure 2 depicts two top-level components: the Anomaly Detection Application Server and the Intrusion Detection System Dashboard.The component diagram introduces the system’s overall organization.Source: A. Component Diagram · p. 4, Figure 2 · p. 5
1) Anomaly Detection Application server:
- The application server orchestrates anomaly detection through model loading, flow prediction, explanation generation, Elasticsearch storage, and Amazon S3 plot storage.The pre-trained model was trained on KDD99; SHAP and LIME use loaded data to explain predictions.Source: 1) Anomaly Detection Application server · p. 4
2) Intrusion Detection System Dashboard:
- The IDS dashboard combines an OpenAI connector, packet-data retrieval, curated prompts, and an AI assistant that analyzes packet data and supports interactive exchanges with human agents.The dashboard component is intended to present explainability and fairness to end users.Source: 2) Intrusion Detection System Dashboard · pp. 4–5
B. Elasticsearch data schemas
Elasticsearch stores detected anomalies and original packet flows in separate indices. The detected-packets index organizes explanatory information, while the original-packets index supports manual review.
B. Elasticsearch data schemas
- The detected-packets index stores information about network anomalies, including attributes extracted using SHAP and LIME.The schema is intended to structure textual predictions, influential factors, and related data for anomaly explanations.Source: B. Elasticsearch data schemas · p. 5
- The detected-packets schema is designed to expose insights into model behavior through structured predictions and influential-factor information.Table II is described as listing index labels and their corresponding descriptions.Source: B. Elasticsearch data schemas · p. 5
- The original-packets index uses a more complex, dynamic mapping and supports anomaly detection through manual review of the original data.The authors refer readers to the KDD99 dataset description for field coverage.Source: B. Elasticsearch data schemas · p. 5
C. User Experience & Use Cases
HuntGPT combines visual explanations, manual inspection, AI-generated explanations, and conversation to support decisions about network anomalies. Its dashboard also presents detection and model-interpretability tools alongside report and inspection functions.
C. User Experience & Use Cases
- The dashboard combines explanatory visualizations, manual inspection, AI-generated explanations, and interactive conversations to support informed decisions about network anomalies.The system is intended to serve multiple scenarios.Source: C. User Experience & Use Cases · p. 5
- The dashboard supports threat classification for analyst response, model inspection for developers and data scientists, and collaborative analysis and reporting through a user-friendly interface.The stated use cases also include supporting model development and promoting operational cost efficiency.Source: C. User Experience & Use Cases · pp. 5–6
A. Prototype Functionality
- The prototype places model visualizations above original data for manual inspection and an AI assistant for explanations and ongoing conversation.Users can download an incident report containing graphs and data.Source: A. Prototype Functionality · p. 6
B. Response Quality Analysis
The response-quality analysis examines GPT-3.5 Turbo’s cybersecurity knowledge and the quality and difficulty of its answers. Across standardized cybersecurity exams, it reports success rates of 72% to 82.5%, with the lowest result on the cybersecurity fundamentals quiz.
B. Response Quality Analysis
- The response evaluation considers answer quality, appropriateness, and difficulty, with an emphasis on users with minimal cybersecurity experience.The analysis is organized into technical knowledge and response evaluation.Source: B. Response Quality Analysis · p. 6
1) Technical Knowledge in Cybersecurity:
- GPT-3.5 Turbo’s cybersecurity knowledge was assessed using standardized certification exams, including CISM and ISACA practice exams.The authors focused on accuracy and precision of answers and compared the model’s knowledge with that of a certified IT professional.Source: 1) Technical Knowledge in Cybersecurity · p. 6
- 72% to 82.5% success rates across standardized cybersecurity exams, with the lowest rate on the cybersecurity fundamentals quiz.The authors describe the results as promising while noting room to improve fundamental cybersecurity knowledge.Source: 1) Technical Knowledge in Cybersecurity · p. 6
2) Response Evaluation:
The response evaluation examines generated explanations and chatbot conversations using six readability formulas. Most formulas classify both text types at graduate level or equivalent, although the generated text is usually described as comprehensible with basic, non-specialized college education.
2) Response Evaluation:
- The study evaluated 20 generated explanations and 20 chat logs, each chat log containing approximately six questions about a detected anomaly.An example chat log is identified in Table V.Source: 2) Response Evaluation · pp. 6, 8
- Six readability formulas were used, including Flesch-Kincaid Grade Level, Flesch Reading Ease, Dale Chall, Automated Readability Index, Coleman Liau, and Linsear Write.The formulas are described as having distinct fields of application.Source: 2) Response Evaluation · p. 8
- Most readability formulas classified AI explainability text and chatbot logs at graduate level or equivalent.The authors also report that generated text is usually readable with basic, non-specialized college education, with variation by question type and requested detail.Source: 2) Response Evaluation · p. 8
V. CONCLUSION AND FUTURE WORK
HuntGPT combines an LLM-based conversational agent with XAI in an intrusion detection prototype. The authors report cybersecurity exam success rates of 72% to 82.5% and generated responses generally comprehensible at a basic college level, while identifying real-time detection and SIEM command execution as future directions.
V. CONCLUSION AND FUTURE WORK
- 72% to 82.5% success rates on standardized cybersecurity exams, with fundamental cybersecurity concepts identified as an area for improvement.The lower success rate was on the cybersecurity fundamentals practice quiz.Source: §V · p. 8
- Generated responses are generally comprehensible to people with a basic college education and are described as actionable and supportive of user engagement.The conversational agent communicates complex cybersecurity concepts to enhance user comprehension.Source: §V · p. 8
- The study presents its findings as a foundation for developing more sophisticated, explainable, interpretable, actionable, and user-friendly cybersecurity solutions.The authors frame the work as a stepping stone for further research on LLM-driven security tools integrating XAI.Source: §V · p. 8
- Future work includes refining the ML model, adding real-time detection, and embedding chatbot commands that act directly on the SIEM component.The authors connect these directions to real-time detection and response requirements.Source: §V · pp. 8–9