PaperlayerView exact revision on arXiv

Source-linked AI summary

IRCopilot: Automated Incident Response with Large Language Models

Xihuan Lin, Jie Zhang, Gelei Deng, Tianzhe Liu, Tianwei Zhang, Qing Guo, Riqing Chen

arXiv:2505.20945v3cs.CR

TL;DR

LLMs show promise in early cybersecurity tasks, but comprehensive incident response after intrusion remains underdeveloped and lacks a suitable benchmark. The paper builds IRBench and proposes IRCOPILOT, a four-component framework organized across reasoning, action, and reflection. IRCOPILOT outperforms baseline LLMs on benchmark completion rates and is also evaluated on public platforms and real-world attacks.

  • Problem

    LLM cybersecurity work has focused mainly on early threat stages, and comprehensive evaluation of their capabilities across incident response remains lacking.

    Source: §I · p. 1, A. Motivation · p. 4

  • Method

    The paper builds a phased benchmark of real-world incident response tasks and proposes IRCOPILOT, which coordinates four LLM components across reasoning, action, and reflection.

    Source: §I · p. 2

  • Results

    IRCopilot’s sub-task completion rate is 1.5× that of directly applying GPT-4, and it resolves most IR tasks on public platforms.

    Source: §I · p. 2

  • Takeaways & Limitations

    The evaluations demonstrate the feasibility of leveraging LLMs for incident response tasks, including public-platform and real-world scenarios.

    Source: §I · p. 2, §VIII · p. 13

    Complex file-analysis challenges can produce voluminous, obfuscated data that current LLMs struggle to parse and that may require intricate manual analysis.

    Source: E. Case V (Limitation in Reverse) · p. 17

Abstract

from arXiv · show

Incident response plays a pivotal role in mitigating the impact of cyber attacks. In recent years, the intensity and complexity of global cyber threats have grown significantly, making it increasingly challenging for traditional threat detection and incident response methods to operate effectively in complex network environments. While Large Language Models (LLMs) have shown great potential in early threat detection, their capabilities remain limited when it comes to automated incident response after an intrusion. To address this gap, we construct an incremental benchmark based on real-world incident response tasks to thoroughly evaluate the performance of LLMs in this domain. Our analysis reveals several key challenges that hinder the practical application of contemporary LLMs, including context loss, hallucinations, privacy protection concerns, and their limited ability to provide accurate, context-specific recommendations. In response to these challenges, we propose IRCopilot, a novel framework for automated incident response powered by LLMs. IRCopilot mimics the three dynamic phases of a real-world incident response team using four collaborative LLM-based session components. These components are designed with clear divisions of responsibility, reducing issues such as hallucinations and context loss. Our method leverages diverse prompt designs and strategic responsibility segmentation, significantly improving the system's practicality and efficiency. Experimental results demonstrate that IRCopilot outperforms baseline LLMs across key benchmarks, achieving sub-task completion rates of 150%, 138%, 136%, 119%, and 114% for various response tasks. Moreover, IRCopilot exhibits robust performance on public incident response platforms and in real-world attack scenarios, showcasing its strong applicability.

I. INTRODUCTION

Incident response is increasingly difficult to automate across the full lifecycle, while existing LLM work focuses mainly on early threat tasks. The paper introduces IRBench and IRCOPILOT, reporting stronger benchmark performance and validation across public platforms and real attacks.

I. INTRODUCTION

  • Manual, expertise-dependent IR struggles with increasingly complex threats, and prior LLM security studies primarily address early-stage tasks rather than the full IR lifecycle.The exploratory evaluation also identifies strategy formulation, inaccurate guidance, overlooked details, context retention, and privacy as challenges.Source: §I · pp. 1–2
  • IRBench evaluates LLMs across three lifecycle stages using 12 case machines and 130 sub-tasks drawn from XuanJi, ZGSF, and TryHackMe.Sub-tasks map to detailed execution steps in the NIST 3.0 lifecycle guidelines.Source: §I · p. 2
  • IRCopilot coordinates Planner, Generator, Reflector, and Analyst components across reasoning, action, and reflection to plan, generate instructions, and review execution.The Analyst returns execution analysis to the Planner, while the Reflector integrates information and proposes improvements.Source: §I · p. 2
  • IRCOPILOT achieves sub-task completion rates 1.5× those of GPT-4 and resolves most IR tasks on public platforms, with traces identified in five real network attacks.The reported completion-rate ratios versus LLAMA3-70B, DEEPSEEK-V3, GPT-4O, CLAUDE-3.5-SONNET, and GPT-O1 are 1.47×, 1.38×, 1.36×, 1.19×, and 1.14×, respectively.Source: §I · p. 2
  • The authors report that GPT-4O-based IRCOPILOT meets objectives in most tasks and validate its applicability in real-world scenarios across two major operating systems.The system is presented as automated and interactive, with the stated aim of improving IR accuracy and efficiency while limiting system impact.Source: §I · p. 2

II. BACKGROUND & RELATED WORK

The background describes a changing threat landscape and the demands this places on incident response. It reviews LLM applications in cybersecurity and specifies the parties and capabilities assumed in the threat model.

A. Global Cybersecurity Landscape

  • Published CVEs rose from about 7,928 in 2014 to 40,287 in 2024, alongside increasingly diverse attacks and more sophisticated sector-specific patterns.The paper also describes evolving incident types and attacker tactics as increasing IR teams’ workload and error risks.Source: A. Global Cybersecurity Landscape · pp. 2–3

B. Incident Response

  • Incident response spans Detect, Respond, and Recover, but traditional methods and manual workflows struggle with adaptive post-incident analysis and the full spectrum of IR.IR also requires consolidating fragmented information from diverse sources into a unified situational picture.Source: B. Incident Response · p. 3

C. Large Language Models and Their Applications for Cybersecurity

  • Cybersecurity LLM research covers tasks from code analysis to anomaly detection, but automated post-intrusion IR and suitable IR benchmarks remain underdeveloped.The paper reviews planning methods including CoT, ToT, RAG, ReAct, and Reflexion as approaches used to improve complex-task reasoning.Source: C. Large Language Models and Their… · p. 3

III. THREAT MODEL

  • The threat model has a white-box victim with an IDS, a gray-box attacker with partial defensive knowledge, and a gray-box responder with limited access to necessary victim data.The attacker is assumed to leave traces on the compromised system, while the responder must limit disruption to core business operations.Source: §III · p. 4

A. Motivation

On IRBench, evaluated LLMs perform less well on harder tasks, while the benchmark design targets realistic, phased evaluation. The section also describes the benchmark’s construction and the failure patterns found in contemporary-model testing.

A. Motivation

  • Existing cybersecurity benchmarks are domain-specific and do not capture the detailed challenges and interdependencies of real-world incident response processes.The authors argue for realistic tasks, a difficulty gradient, and phase-oriented evaluation.Source: A. Motivation · p. 4

B. Benchmark Design

  • IRBench selects simulated incident tasks from three platforms, decomposes them according to NIST guidance, and iteratively validates the decomposition with expert feedback.It contains 12 IR objectives and 130 sub-tasks across 27 categories; publicly available high-difficulty cases remain limited, and the authors plan to expand the benchmark.Source: B. Benchmark Design · pp. 4–5

C. Evaluation on Contemporary LLMs

  • The baseline evaluation uses identical prompts and controlled environments without external tools, and counts success only when every sub-task across detection, response, and recovery is completed.The study evaluates six contemporary models, including commercial, open-source, and reasoning-focused systems.Source: C. Evaluation on Contemporary LLMs · p. 5
  • GPT-O1 has the highest IRBench sub-task completion rate at 86.92%, ahead of CLAUDE-3.5-SONNET at 83.08%, while all evaluated LLMs decline on harder tasks.Both models solve 5 of 12 overall tasks; GPT-4O, DEEPSEEK-V3, LLAMA3-70B, and GPT-4 achieve 72.31%, 65.38%, 63.85%, and 58.46%, respectively.Source: C. Evaluation on Contemporary LLMs · p. 5, Table I · p. 5
  • The most prevalent baseline failures are ineffective IR strategies and inaccurate or incomplete guidance, alongside overlooked details, context-retention problems, and privacy risks.The workflow figure assigns planning and analysis to reasoning, command generation to action, and review to reflection.Source: C. Evaluation on Contemporary LLMs · p. 5, Figure 2 · p. 6

V. IRCOPILOT

IRCopilot adapts real-world incident-response teamwork into three phases coordinated by four specialized LLM sessions. Its Planner uses a tracked Incident Response Tree to maintain task continuity and manage response paths.

A. Design Overview

  • Three phases—Reasoning, Action, and Reflection—coordinate four separate LLM sessions that divide planning, instruction generation, reflection, and execution-result analysis.Each phase maintains its own session history and context.Source: A. Design Overview · p. 6
  • The design combines CoT, ToT, Few-Shot, and Negative Prompt methods with refined responsibilities, an IR Tree, and a Reflector that reviews four information sources.These design choices target reasoning, overlooked details, hallucinations, and adaptive optimization.Source: A. Design Overview · p. 6

B. Planner

  • The Incident Response Tree hierarchically decomposes and tracks tasks using Completed, To-do, and N/A statuses to preserve continuity and mitigate contextual drift.The design is motivated by the continuous, multifaceted nature of incident response and the need to preserve long-term strategy.Source: B. Planner · p. 6
  • For clear goals, the Planner logs objectives in the IRT; for unclear goals, it prioritizes sub-tasks and maintains a procedures tree beneath the root.The Reflector verifies the IRT, protects its root and key objectives from modification, and can trigger rollback; the Planner updates task priorities and records results to refine later paths.Source: B. Planner · pp. 7–8

C. Generator

The Generator turns Planner-defined sub-tasks into executable guidance, while the Reflector checks component outputs and the Analyst feeds new findings into the IRT. Together, these components support stepwise execution, output review, and continued investigation.

C. Generator

  • The Generator converts each Planner sub-task into detailed steps and then into executable terminal commands or precise desktop-investigation descriptions.It can produce multiple distinct strategies when a task has several possible approaches, and standardizes command boundaries with the "$" symbol.Source: C. Generator · p. 8

D. Reflector

  • The Reflector reviews the IRT, Planner decisions, Generator guidance, and execution results to identify possible hallucinations, missing information, and erroneous steps.It can suggest modifications and conduct traceback analysis; its stated purpose also includes addressing privacy and security concerns and preventing error accumulation.Source: D. Reflector · p. 8

E. Analyst

  • The Analyst uses Tree-of-Thought reasoning to evaluate trajectories, break results into executable sub-tasks, and add further-investigation items to the IRT.It feeds its findings back to the Planner for subsequent decisions.Source: E. Analyst · pp. 8–9

VI. EVALUATION ON IRCOPILOT

The evaluation compares IRCopilot variants with six underlying LLMs under matching experimental settings and assesses performance alongside efficiency, ablation, practicality, and failure questions. On IRBench, IRCopilot variants outperform their respective base models, though high-complexity tasks remain difficult.

A. Experimental Settings

  • IRCopilot was evaluated using six underlying LLMs, with the other inputs and experimental settings held identical to those described for the benchmark setup.The evaluation questions cover performance, efficiency, component ablation, real-world practicality, and failure conditions.Source: A. Experimental Settings · p. 9, §VI · p. 9

B. Performance Evaluation (RQ1)

  • At most difficulty levels, IRCOPILOT-GPT-4O, IRCOPILOT-CLAUDE-3.5-SONNET, and IRCOPILOT-GPT-O1 outperform other models, with a notable advantage on moderate- or lower-complexity incidents.The authors attribute GPT-4's weaker showing relative to GPT-4O to a smaller training dataset and narrower incident coverage.Source: B. Performance Evaluation (RQ1) · p. 9
  • 150%, 147%, 138%, 136%, 119%, and 114% of base-model performance were achieved by IRCOPILOT-GPT-4, IRCOPILOT-LLAMA3-70B, IRCOPILOT-DEEPSEEK-V3, IRCOPILOT-GPT-4O, IRCOPILOT-CLAUDE-3.5-SONNET, and IRCOPILOT-GPT-O1, respectively.The paper also reports that IRCopilot substantially improves the IR performance of all tested LLMs.Source: Figure 5 · p. 9, B. Performance Evaluation (RQ1) · p. 9
  • High-complexity tasks remain difficult because the approach does not expand the LLMs’ training knowledge in attacks, investigations, analysis, or response.The authors identify this as an inherent LLM limitation that constrains further improvement in complex scenarios.Source: B. Performance Evaluation (RQ1) · p. 9

C. Efficiency Evaluation (RQ2)

The efficiency evaluation measures reasoning time and monetary cost, including time per completed sub-task to account for unequal task completion. IRCopilot can require more total time while improving per-sub-task efficiency, and the authors identify GPT-4O as the best overall trade-off among performance, speed, and cost.

C. Efficiency Evaluation (RQ2)

  • The evaluation uses LLM reasoning time as a controllable efficiency measure and averages task completion time over ten runs per benchmark task.The timing measure covers processing inputs and producing outputs on the platform, isolating the LLM’s reasoning time from network delays and incident complexity.Source: C. Efficiency Evaluation (RQ2) · p. 10
  • Average time per completed sub-task normalizes total time by the number of tasks completed, because IRCopilot may complete more sub-tasks than GPT-4O.The paper defines the completed-task counts as n_g and n_i for GPT-4O and IRCopilot, respectively.Source: C. Efficiency Evaluation (RQ2) · p. 10
  • In four of twelve benchmarks, IRCopilot is more efficient per sub-task despite often taking more total time; on Ransomware it completes 6 versus 4 sub-tasks for a 35% increase in total time.The authors attribute the extra total time to multi-step reasoning overhead.Source: Table III · p. 10, C. Efficiency Evaluation (RQ2) · p. 10
  • IRCOPILOT-GPT-4O offers the best trade-off among performance, speed, and cost for real-time incident response, while GPT-O1's marginal performance gain comes with significantly higher reasoning time and monetary cost.The comparison reports one extra completed sub-task in “Linux 2” for GPT-O1.Source: C. Efficiency Evaluation (RQ2) · p. 10

D. Ablation Study (RQ3)

The ablation study finds that IRCopilot outperforms component-removal variants, while practicality tests report performance across public platforms and real-world attack cases.

D. Ablation Study (RQ3)

  • Across benchmark evaluations, IRCopilot consistently surpasses each ablation baseline in sub-task completion rates.Source: Figure 6 · p. 11, D. Ablation Study (RQ3) · p. 11
  • 42.97% (55/128) for IRCOPILOT-NO-PLANNER and 48.44% (62/128) for IRCOPILOT-NO-ANALYST fall below baseline GPT-4O performance.The authors highlight the Planner’s strategy and decision-making, and the Analyst’s ToT method, as important to effective IR.Source: D. Ablation Study (RQ3) · p. 11
  • 70.31% (90/128): IRCOPILOT-NO-GENERATOR performs approximately on par with baseline GPT-4O.The authors say the Generator primarily produces IR guidance through the base LLM.Source: D. Ablation Study (RQ3) · p. 11
  • 85.94% (90/128): IRCOPILOT-NO-REFLECTOR completes fewer sub-tasks than the full variant, but marginally outperforms baseline GPT-4O.The Reflector rectifies occasional hallucinations and other minor errors in most cases.Source: D. Ablation Study (RQ3) · p. 11

E. Practicality Study (RQ4)

F. Failure Analysis (RQ5)

IRCopilot reduces IRBench failure rates, but remaining errors shift toward missed information and false commands, with cognitive load and model limitations implicated.

F. Failure Analysis (RQ5)

  • 28.3% to 5.4%: IRCopilot reduces the IRBench failure rate, with the improvement particularly pronounced for more foundational models.Source: F. Failure Analysis (RQ5) · p. 12
  • The dominant failure shifts from “False IR Strategy” to “Key Information Ignored” and “False Command Generation.”Source: F. Failure Analysis (RQ5) · p. 12
  • State maintenance and multi-turn interactions increase the information load, which can cause the model to overlook critical details.Source: F. Failure Analysis (RQ5) · p. 12
  • Static knowledge and susceptibility to hallucinations can lead to incorrect commands or guidance.Source: F. Failure Analysis (RQ5) · p. 12
  • The qualitative failure analysis cross-validates human experts’ assessments with Gemini 2.5 Pro, an independent LLM not used in the preceding experiments.It examines failed tasks’ reasoning processes and identifies causes including hallucinations and limitations in reverse.Source: F. Failure Analysis (RQ5) · pp. 12–13

VII. DISCUSSION

The discussion addresses benchmark contamination and model selection, acknowledges remaining hallucination and complex-analysis limits, and identifies adaptive defenses as future work. The conclusion presents the benchmark and IRCOPILOT as groundwork for LLM-based incident response.

VII. DISCUSSION

  • The benchmark mitigates contamination concerns by prioritizing recent cases, checking model knowledge cutoffs, querying for targeted training, and examining IRCOPILOT reasoning paths.A majority of tasks, 43/72, postdate model cutoff dates.Source: §VII · p. 13
  • GPT-4O-2024-08-06 costs 1/12 as much as GPT-4 and has a 128K-token context window, while the authors also cite its faster response.The selected APIs also include Claude-3.5-Sonnet, Llama3-70B, and DeepSeek-V3.Source: §VII · p. 13
  • Complex sample analysis and reverse engineering often remain incomplete, while IRCOPILOT uses the IRT and Reflector to mitigate hallucinations.The authors anticipate that evolving training data and model architectures may reduce hallucinations and improve handling of complex IR tasks.Source: §VII · p. 13
  • Integrating RAG reduced efficiency and raised overhead without anticipated performance gains, motivating future work on adaptive defenses with advanced agents, memory, or search.The authors describe cybersecurity threats as evolving and increasingly diverse and sophisticated.Source: §VII · p. 13

VIII. CONCLUSION

  • The conclusion presents a comprehensive IR benchmark and modular IRCOPILOT, evaluated in experiments and real-world scenarios, as groundwork for cybersecurity enhancement.The system simulates planning and response processes of professional teams and uses strategies to mitigate hallucinations and context limitations.Source: §VIII · p. 13

APPENDIX

The appendix captions survey attack and vulnerability topics, while case examples illustrate failures from keyword-only investigation strategies and incorrect command generation. IRCopilot's described responses emphasize ordered task procedures and alternative account-checking methods.

APPENDIX

A. Case I (False IR Strategy)

  • In the ZGSF Linux 1 case, a false response strategy illustrates that keyword searches alone can miss clues; the paper proposes prioritizing tasks through the IRT procedure order.The IRT lists objectives and procedures such as reviewing command history, sensitive directories, and system logs.Source: A. Case I (False IR Strategy) · pp. 15–16

B. Case II (False Commands Generation)

  • In the XuanJI hidden-accounts task, `net user` failed to provide complete information; IRCopilot's proposed alternatives include registry inspection or WMIC.The example describes incorrect command generation as a cause of task failure.Source: B. Case II (False Commands Generation) · p. 16

C. Case III (False Command Generation by IRCOPILOT)

In Task 9, IRCopilot failed to identify when a stored cryptographic key was read because it generated a forensic command using Event ID 4657 instead of the more specific Event ID 5058.

C. Case III (False Command Generation by IRCOPILOT)

  • Task 9 asked when the hacker read a stored key, but IRCopilot failed to generate the correct forensic command and the task was marked failed.Gemini 2.5 pro was used after the initial experiments to evaluate the generated content.Source: C. Case III (False Command Generation… · p. 16
  • Event ID 4657 tracks general object access or registry modifications, whereas Event ID 5058 logs key-file operations such as reading a stored cryptographic key.The task required finding the key's read time.Source: C. Case III (False Command Generation… · pp. 16–17
  • The suggested PowerShell filters searched Security logs for Event ID 4657, with an additional example matching a RegistryPath string.The example used a RegistryPath placeholder that was to be replaced with a specific path.Source: C. Case III (False Command Generation… · pp. 16–17
  • The error analysis attributes the mismatch to treating the task as a routine registry check and overlooking the “Reserved key” context.The analysis contrasts an event for modification with the requested read event.Source: C. Case III (False Command Generation… · p. 17

D. Case IV (Key Information Ignored by IRCOPILOT)

Case IV shows IRCopilot overlooking a webshell parameter in a POST request, while Case V describes a reverse-engineering challenge involving complex analysis and obfuscated command output.

D. Case IV (Key Information Ignored by IRCOPILOT)

  • The case analysis reports attacker traffic between 192.168.20.1 and 192.168.20.144, a flag1 response, and a POST request containing PHP code that suggests a possible webshell or malicious script.The POST request URL was /index.php?user-app-register.Source: D. Case IV (Key Information Ignored by… · p. 17
  • The model focused on the complex PHP payload, the value, and ignored the parameter name, the key; the analysis recommends parsing key=value before analyzing the payload.The error analysis says Network2020 did not match typical parameter patterns such as cmd or eval, which likely contributed to the oversight.Source: D. Case IV (Key Information Ignored by… · p. 17
  • Task 4 failed because IRCopilot left the webshell connection parameter as To-do instead of extracting Network2020 from the POST request's key=value structure.The parameter name was available in the request body.Source: D. Case IV (Key Information Ignored by… · p. 17

E. Case V (Limitation in Reverse)

  • The Squid Game malicious-file challenge requires dynamic and static analysis, data-flow analysis, VBA macro handling, obfuscated-content decoding, and multiple tools.Its command outputs are described as voluminous, obfuscated streams that current LLMs find challenging to parse and may require intricate manual analysis.Source: E. Case V (Limitation in Reverse) · p. 17
Loading 2505.20945v3…