CORTEX: Collaborative LLM Agents for High-Stakes Alert Triage

• SOCs face tens of thousands of daily alerts, few of which are genuine threats, alongside high false-positive rates and alert fatigue.The cited empirical false-positive rates approach 99%.Source: §1 · p. 1
• Single-agent LLM approaches combine log interpretation, context retrieval, and alert adjudication, yet can falter on long-horizon investigations and offer limited auditability.Source: §1 · pp. 1–2
• CORTEX assigns workflow identification, external evidence gathering, and evidence synthesis to specialized agents that communicate through structured messages.Source: §1 · p. 2
• The released dataset captures stepwise analyst actions, tool queries and outputs, and intermediate observations across more than ten production scenarios.These traces support training and evaluation of process-level reasoning.Source: §1 · p. 2
• CORTEX substantially reduces false positives and improves investigative quality over single-agent LLMs across diverse enterprise scenarios.Source: §1 · p. 2

2.1 LLMs in Cybersecurity

• Cybersecurity LLM work spans threat-intelligence extraction, knowledge-graph reasoning, vulnerability analysis, and attack automation, but fewer studies model end-to-end SOC investigation as a tool-grounded, role-specialized pipeline.The passage also identifies distillation of that process into a deployable single model as less explored.Source: §2.1 · p. 2

2.2 Alert Fatigue and Automated Triage in SOCs

• Automated triage includes supervised prioritization and learning-to-rank, while emerging agentic SOC platforms advertise end-to-end coverage and speedups but can incur notable computational costs.Source: §2.2 · pp. 2–3

2.3 Collaborative Multi-Agent LLM Systems

• Collaborative-agent research motivates task decomposition and cross-checking, while documenting specification errors, inter-agent misalignment, and weak termination or verification.The paper frames disciplined message passing and externally grounded verification as relevant design needs.Source: §2.3 · p. 3

3 Methods

• The methods pair a divide-and-conquer multi-agent triage architecture with a fine-grained SOC workflow dataset.Source: §3 · p. 3

3.1 CORTEX Architecture

• CORTEX routes alerts to relevant workflows, uses workflow-specific agents to query typed enterprise tools, and synthesizes evidence into a structured, auditable report.Its four stages are orchestration, workflow routing, evidence-grounded analysis, and synthesis; any workflow escalation yields an actionable verdict.Source: §3.1 · pp. 3–4, Figure 1 · p. 4

3.2 Fine-Grained SOC Workflow Dataset

• The dataset records production investigations across ten scenarios, including raw telemetry, analyst actions, tool queries, intermediate reasoning, and final adjudications.Sensitive fields are pseudonymized, and traces are serialized as JSON for supervised training and structured evaluation.Source: §3.2 · p. 4
• Each trace includes behavioral-rule evidence fields, while investigations are labeled as actionable or non-actionable, with four non-actionable subclasses.The records span identity, network, system, operational, artifact, and security-annotation attributes.Source: §3.2 · p. 5
• Dataset examples cover heterogeneous cloud, SaaS, and endpoint sources, normalized into a unified schema while retaining domain-specific detail.Source: §3.2 · p. 5

4.1 Experimental Setup

• The test dataset is the fine-grained SOC workflow dataset, and all PII is pseudonymized upstream.Systems must return a schema-valid report with an Actionable/Non-actionable verdict, applicable subclass, evidence-grounded rationale, and observables.Source: §4.1 · p. 5
• Single-agent baselines are prompt-only or ReAct-style tool users, with tool budgets and turn caps matched to CORTEX for the latter.Both baselines must ground claims in tool outputs and produce schema-valid reports.Source: §4.1 · p. 5
• CORTEX routes alerts to workflows, uses evidence agents for calibrated checks, and applies conservative escalate-on-any synthesis.Evaluation includes decision and subclass macro-F1, false-positive rate, actionable-alert recall, output tokens, tool calls, and latency.Source: §4.1 · pp. 5–6

4.2 Triage Performance

• 0.66→0.78 actionable F1 over the best single-agent baseline, while false-positive rate falls from 24.9% to 14.2%.Macro-F1 reaches 0.82 and subclass F1 increases by +0.15.Source: §4.2 · p. 6, Table 4 · p. 6

4.3 Efficiency

• 152.4 s median end-to-end time per ticket remains within the approximately 3-minute SLO, versus 44.6 s for ReAct and 28 s for prompt-only.CORTEX processes 23,600 tokens versus 4,152 for the tool-using baseline; average tool calls rise from 1.3 to 3.1.Source: §4.3 · p. 6, Table 5 · p. 6

5 Conclusion

• Actionable F1 rises from 0.66 to 0.78 and false positives fall from 24.9% to 14.2%, with operationally acceptable end-to-end latency.Structured reports include explicit evidence links for downstream review, compliance, and post-incident learning.Source: §5 · p. 7
• The workflow dataset captures alerts, tool queries, intermediate observations, and final adjudications across diverse enterprise scenarios.Process-level supervision supports training agents to follow playbooks and measuring reasoning fidelity, including step accuracy, tool-policy match, and grounding consistency.Source: §5 · p. 7
• Evaluation is limited by coverage of ten-plus scenarios and the availability and quality of upstream telemetry.The paper also identifies sensitivity to distribution shift, prompt injection, and incomplete tool-returned context.Source: §5 · p. 7

A Workflow Prompts

• The workflow prompts assign evidence retrieval to typed tools and decision policies to the Reasoning & Coordination Agent.CORTEX’s roles include orchestration, workflow routing, evidence acquisition, and synthesis into an auditable report.Source: Table 6 · p. 9

A.1 Add User Workflow; A.2 Authentication Change Workflow

• For user-add and authentication-change workflows, agents retrieve directory records; admin status or account age then informs actionability.Adding an admin is actionable; removing an authentication method is actionable, while adding one is non-actionable only for a new user.Source: §A.1 · p. 9, §A.2 · p. 10

A.3 Correlation (Coro) Workflow; A.4 Generic Workflow

• Coro events are escalated by policy, while the generic workflow retrieves raw behavior events only when direct evidence is insufficient.The generic workflow closes invalid tickets, escalates valid actionable ones, or requests additional information when evidence is insufficient or conflicting.Source: §A.3 · p. 10, §A.4 · p. 11

A.5 Multiple ISP (Impossible Travel) Workflow; A.6 Office 365 Guest Workflow

• Impossible-travel analysis uses recent login activity, distance and time, ISP diversity, and plausible VPN or mobile explanations; guest activity is actionable when the guest has admin status.The impossible-travel workflow evaluates logins within an eight-hour window.Source: §A.5 · pp. 11–12, §A.6 · p. 12

A.7 Office 365 Login Workflow; A.8 PowerShell Workflow; A.9 Salesforce Abnormal Login Workflow; A.10 SharePoint File Workflow

• The policies make an O365 login actionable only when its risk exceeds 1000 and recent high-risk activity exists; PowerShell requires malicious code and admin status, while Salesforce requires at least three recent rule events.Source: §A.7 · p. 13, §A.8 · p. 13, §A.9 · p. 14

B Case Studies

• The case studies cover five alert-processing examples.Each illustrates how CORTEX processes an alert.Source: B Case Studies · p. 15
• Each example includes an agent reasoning trace.Source: B Case Studies · p. 15
• Each example also provides the final JSON report.Source: B Case Studies · p. 15

B.1 Case Study 1: Add User Workflow

• The alert concerns a new user added to a tenant with elevated privileges.Source: §B.1 · p. 15
• The retrieved user record is found and identifies the target as an admin, with GlobalAdmin cited as the role evidence.Source: §B.1 · p. 15
• The report treats assigning admin privileges as actionable under policy and recommends escalation.Source: §B.1 · p. 15

B.2 Case Study 2: Authentication Change Workflow

• The event is an established user's switch from MFA to password login.The account was found, and the trace says it was created in 2019, older than 30 days.Source: §B.2 · pp. 15–16
• Removing an authentication method is always actionable under the stated reasoning.The summary flags the established user's MFA removal for potential account compromise.Source: §B.2 · p. 16

B.3 Case Study 3: Multiple ISP / Geo-Impossibility

• The Behavior Analysis Agent matches the alert to the Multiple ISPs workflow.Evidence Acquisition Agents retrieve recent login activity from external systems.Source: §B.3 · p. 16
• The reasoning classifies the pattern as impossible travel and actionable.The trace records impossible_travel as true and summarizes the London-to-New York activity as impossible travel.Source: §B.3 · p. 16
• Logins from London and New York about 5,500 km apart within 30 minutes exceed feasible travel limits.The queried activity showed London at 10:15 UTC and New York at 10:45 UTC.Source: §B.3 · p. 16

B.4 Case Study 4: Office 365 Guest Workflow

• The Behavior Analysis Agent matches the alert to the Guest user pattern.The event is a guest user invited into the Finance group.Source: §B.4 · p. 16
• The guest record is found, and its admin role is listed as User.The trace characterizes the guest's roles as standard user access, with no admin roles.Source: §B.4 · p. 16
• The reasoning says the guest lacks admin privileges under policy, so the invitation is not actionable.The summary describes the guest as added to Finance without admin privileges.Source: §B.4 · pp. 16–17

B.5 PowerShell Workflow

• Encoded PowerShell and registry-based persistence are classified as malicious behavior.The workflow inspects command attributes and registry persistence.Source: §B.5 · p. 17
• Endpoint telemetry indicates the detection was disinfected.The recorded disinfection status is Disinfect.Source: §B.5 · p. 17
• An Admin's malicious PowerShell execution meets escalation policy and is escalated for follow-up.The summary retains the persistence finding despite the disinfected status.Source: §B.5 · p. 17