PaperlayerView exact revision on arXiv

Source-linked AI summary

HOLMES: Real-time APT Detection through Correlation of Suspicious Information Flows

Sadegh M. Milajerdi, Rigel Gjomemo, Birhanu Eshete, R. Sekar, V. N. Venkatakrishnan

arXiv:1810.01594v2cs.CR

TL;DR

HOLMES addresses noisy and computationally challenging APT detection by correlating suspicious information flows and mapping activity to the attack kill chain. It produces real-time detection signals and high-level attack graphs, distinguishing attack from benign scenarios with high precision and low false alarm rates.

  • Problem

    APT detection must handle benign audit events that match TTP rules, spurious dependencies, and computationally expensive graph analyses in real-time settings.

  • Method

    HOLMES maps host-log activities and enterprise alerts to the APT kill chain, correlates information flows in versioned provenance graphs, and reduces noise through dependency pruning, benign-activity rules, ranking, and prioritization.

  • Results

    HOLMES distinguishes attack and benign scenarios in nine real-life APT scenarios and a two-week live experiment, achieving high precision and recall with low false alarm rates.

  • Takeaways & Limitations

    HOLMES generates compact, high-level graphs that summarize attacker steps in real time and can support cyber-response operations.

Abstract

from arXiv · show

In this paper, we present HOLMES, a system that implements a new approach to the detection of Advanced and Persistent Threats (APTs). HOLMES is inspired by several case studies of real-world APTs that highlight some common goals of APT actors. In a nutshell, HOLMES aims to produce a detection signal that indicates the presence of a coordinated set of activities that are part of an APT campaign. One of the main challenges addressed by our approach involves developing a suite of techniques that make the detection signal robust and reliable. At a high-level, the techniques we develop effectively leverage the correlation between suspicious information flows that arise during an attacker campaign. In addition to its detection capability, HOLMES is also able to generate a high-level graph that summarizes the attacker's actions in real-time. This graph can be used by an analyst for an effective cyber response. An evaluation of our approach against some real-world APTs indicates that HOLMES can detect APT campaigns with high precision and low false alarm rate. The compact high-level graphs produced by HOLMES effectively summarizes an ongoing attack campaign and can assist real-time cyber-response operations.

I. INTRODUCTION

HOLMES addresses the challenge of turning noisy, low-level host alerts into a reliable, high-level representation of ongoing APT campaigns. It correlates suspicious information flows across kill-chain stages and produces compact graphs for detection and response.

  • APT campaigns generally follow recurring kill-chain stages despite variation in concrete exploits, enabling high-level mapping of attacker behavior.
  • Existing IDS and SIEM systems generate and correlate low-level alerts, but struggle to represent complex relationships and assemble reliable attack scenarios.
  • HOLMES maps host activities and enterprise alerts directly to kill-chain steps, making alerts more semantically meaningful than raw audit events.
  • Around 3M low-level events from five days of audit logs are reduced to 86 suspicious activity steps.
  • HOLMES correlates attack steps through information flows among files, processes, and other entities to construct an emerging kill-chain.
  • The high-level scenario graph uses TTP nodes and information-flow edges, while dependency pruning, noise reduction, and prioritization reduce false alarms.
  • From 10M audit records, HOLMES summarizes a high-level campaign in a graph of just 16 nodes for rapid analyst comprehension.
  • Evaluation on nine real-life APT scenarios and a two-week live experiment showed clear attack-benign separation with high precision and recall.

II. A RUNNING EXAMPLE

The running example follows a red-team attack against a vulnerable FreeBSD Nginx server, reconstructed from system audit data as a provenance graph. Its stealth and event volume illustrate why HOLMES relies on kill-chain stages and connected information flows.

  • The example uses audit data from a vulnerable Nginx web server on FreeBSD to construct a provenance graph of system entities and activity.
  • The attacker’s goal is to exfiltrate sensitive information from the system.
  • The attack proceeds through initial compromise, command-and-control communication, privilege escalation, and internal reconnaissance.
  • The initial compromise executes in memory within the existing Nginx process, leaving minimal visible traces and blending with normal operation.
  • A single host can generate tens of millions of events daily, creating a needle-in-a-haystack challenge for attack detection.
  • The attack steps correspond to APT stages and are connected by information flows, motivating HOLMES’s detection approach.

III. APPROACH OVERVIEW

HOLMES detects APT campaigns by mapping low-level audit activity to kill-chain stages and correlating the information flows connecting those stages. It bridges the semantic gap with TTPs, provenance graphs, and ranked High-level Scenario Graphs.

  • HOLMES bases detection on essential high-level APT steps and the information-flow dependencies connecting them.
  • The approach addresses the semantic gap between low-level audit data and the high-level kill-chain view of attacker goals and capabilities.
  • An intermediate ATT&CK-based TTP layer maps lower-level audit actions to high-level APT capabilities.
  • HOLMES represents audit events as a provenance graph whose information-flow dependencies support efficient TTP matching.
  • A High-level Scenario Graph represents matched TTPs as nodes and their information-flow and causality dependencies as edges.
  • Benign-pattern learning and severity-weighted ranking help reduce false positives and prioritize HSGs for analysts.

IV. SYSTEM DESIGN

HOLMES converts multi-host audit data into an OS-neutral, in-memory provenance graph for real-time analysis. The design assumes the kernel, auditing engine, and produced logs are trusted.

  • HOLMES assumes the OS kernel, auditing system, and generated logs are outside the threat model, with the system benign initially.
  • Audit data from Linux, BSD, and Windows is collected into an OS-neutral format accepted by HOLMES.
  • The provenance graph contains versioned subject and object nodes connected by event-annotated dependency edges.
  • HOLMES can accept additional IDS/IPS events and alerts, but this capability is not discussed further in the paper.
  • The compact provenance representation supports in-memory graph construction and real-time event consumption over prolonged periods.

B. TTP Specification

HOLMES uses information-flow-based TTP specifications as an intermediate abstraction between audit logs and high-level APT stages. Prerequisites constrain matching, reduce false positives, and drive compact HSG construction.

  • TTPs map OS-neutral audit events and information-flow dependencies to high-level APT steps while remaining independent of many implementation details.
  • TTP prerequisites encode causal relationships and information flows between APT stages, helping reduce false positives and clarify campaign context.
  • Most TTPs are modeled using a single event with subject and object preconditions, avoiding expensive backtracking during matching.
  • TTP rules specify an APT stage, TTP and entities, event family, and severity level used to prioritize alarms.
  • Prerequisites can constrain TTP parameters, previously matched TTPs, process relationships, shared parents, and information flows.
  • A TTP enters the HSG only when all prerequisites are satisfied, reducing graph size enough for sophisticated real-time analysis.

D. Avoiding Spurious Dependencies

HOLMES suppresses benign or irrelevant provenance dependencies by measuring how strongly attacker-controlled ancestors account for information-flow paths. Minimum ancestral covers and path factors guide pruning and prioritization.

  • Spurious dependencies from benign processes can cause dependence explosion and obscure attack steps in analyst-facing HSGs.
  • Strong dependencies are associated with shared compromised ancestors, whereas other dependencies are treated as weak and deemphasized.
  • An ancestral cover contains an ancestor or the process itself for every process on an information-flow path.
  • The minimum ancestral cover estimates the minimum number of compromised ancestors needed to control an information-flow path.
  • For the nginx-to-cat path, the minimum ancestral cover has size 2, indicating that controlling cat would require an additional exploit.
  • Path factor uses the smallest minimum-ancestral-cover value across paths to quantify attacker control over a flow.
  • Using path factor mitigated dependency explosions by prioritizing attacker-influenced flows.

E. Noise Reduction

HOLMES reduces noisy TTP matches by learning benign prerequisites and information-flow quantities, then scores correlated HSG stages to distinguish attacks from benign activity.

  • Benign prerequisite matches are learned from training data and ignored at runtime when triggered TTP prerequisites match the learned benign context.
  • Filtering solely on benign prerequisites can miss malicious events that resemble learned benign behavior, such as a compromised nginx reading /etc/passwd.
  • HOLMES supplements prerequisite filtering with transferred-byte quantities, flagging flows whose observed volume substantially exceeds the benign amount.
  • Signal correlation and detection: Each HSG receives a seven-stage threat tuple whose entries represent the severity of the corresponding APT stages.
  • Signal correlation and detection: The weighted-product score combines stage severities, weights, and a detection threshold; absent stages receive severity 1.
  • Signal correlation and detection: Experiments comparing scoring schemes found that the weighted product produced the best normalized margin between benign and attack subgraphs.

V. IMPLEMENTATION

HOLMES normalizes cross-platform audit data, streams it for real-time provenance construction, and incrementally correlates provenance dependencies with TTPs to build HSGs.

  • Audit records from different operating systems are normalized into a common data representation and published to Kafka for real-time consumption.
  • The policy engine checks TTP rules, prerequisites, and path factors while the provenance graph is being constructed.
  • Incremental matching avoids backtracking by storing prior matches, creating HSG nodes, and propagating pointers through dependent low-level entities.
  • Path factors are updated incrementally across provenance edges, with process ancestry determining whether the factor remains one or increases.
  • Pointer propagation can add space overhead as provenance graphs grow, although many entities typically point to the same TTP set.
  • The learning model stores matched TTP prerequisites and benign byte-transfer thresholds for runtime comparison.

VI. EXPERIMENTAL EVALUATION

HOLMES was evaluated on diverse DARPA red-team scenarios and benign activity, separating attack and benign HSGs while showing stronger score margins with learning and path factors.

  • The evaluation used DARPA adversarial engagements, first analyzing an available dataset before selecting a threshold and measuring performance.
  • The datasets contain nine APT scenarios across seven hosts and three operating systems, covering 20 days of audit logs.
  • The attacks include drive-by compromise, backdoor injection, privilege escalation, reconnaissance, exfiltration, and cleanup, alongside sophisticated loading techniques.
  • HOLMES was configured with representative TTPs, path thres = 3, and stage weights weight = (10+i)/10.
  • The highest benign score was 338, while the lowest attack score was 608, forming disjoint score clusters.
  • Enabling learning and path-factor calculations increased the margin between attack and benign HSG scores; removing either increased noise and false positives or false negatives.

D. Attack Scenarios

HOLMES illustrates attack scenarios through HSGs and selects detection thresholds from score separation, including a drive-by download whose threat score is 1163881.

  • Scenario-1: Drive-by Download: Scenario-1 models a vulnerable Firefox visiting a malicious website, dropping a file named net during a drive-by download.
  • Scenario-1: Drive-by Download: The initial-compromise stage contains execution of an untrusted file matching a critical-severity TTP.
  • Scenario-1: Drive-by Download: The resulting threat tuple is ⟨C, M, −, H, −, H, M⟩, with quantitative values ⟨10, 6, 1, 8, 1, 8, 6⟩.
  • Scenario-1: Drive-by Download: 1163881 is the resulting threat score for the Scenario-1 graph.
  • Threshold selection: The maximum F-score occurs for thresholds in [338.25, 608.26], separating the maximum benign and minimum attack subgraph scores in the tested datasets.
  • Threshold selection: After nth-root transformation, the maximum-F-score interval becomes [2.01, 2.16], with 2.09 selected as the midpoint average severity.

F. Performance

HOLMES compresses detailed provenance into compact high-level scenario graphs while supporting real-time analysis of mixed benign and attack activity. Its evaluation shows low false-alarm behavior, with documented boundary cases involving hidden causal flows and multiple entry points.

  • Graph Size: 1875:1 edge ratio represents an 1875-fold reduction when mapping the provenance graph to the high-level scenario graph.The reduction is measured using graph edges.
  • Memory Use: HOLMES showed nearly linear memory growth with the number of audit records and was extrapolated to support enterprises with hundreds of hosts.The system operated on in-memory audit records during testing on an 8-core, 2.5GHz CPU with 150GB of RAM.
  • Live Detection: During a two-week live experiment, HOLMES analyzed Windows, Linux, and BSD audit logs in real time amid predominantly benign activity.The simulated enterprise included security-critical web, email, SSH, and SMB services.
  • False Alarms: Some benign HSGs exceeded the detection threshold, producing false positives attributed in part to SSH connections made by system administrators.The experiment used a threshold of 1378, and the cumulative distribution compared attack and benign HSG threat scores.
  • False Alarms: A separate two-week benign activity period produced no false alarms, while no false negatives were observed during the experiments.The authors nevertheless discuss potential false-negative scenarios and allow analysts to neutralize falsely constructed HSGs.
  • Limitations: HOLMES lacks direct visibility into causal relations carried by information flows that avoid system calls, although it can partially reconstruct attacks visible through later system-call activity.This limitation concerns implicit causality between tactics, techniques, and procedures.
  • Limitations: Multiple attacker entry points can create detached subgraphs that require additional analysis when no information flow connects their attack steps.HOLMES follows each entry point and correlates disjoint subgraphs when information flow exists between them.

VII. RELATED WORK

HOLMES addresses real-time alarm generation, alert correlation, and attack-scenario reconstruction by correlating TTPs through information flows rather than relying primarily on generic statistical log relationships. The paper positions this design against misuse, anomaly, specification, alarm-correlation, SIEM, and system-call-level approaches.

  • Contributions: HOLMES contributes to real-time alarm generation, alert correlation, and scenario reconstruction through a high-level attack scenario graph.The graph is the underlying basis for all three problems.
  • Alarm Generation: Misuse-based, anomaly-based, and specification-based host intrusion detectors trade off unknown-attack coverage, false positives, and policy-development effort.The paper notes that misuse detection cannot handle unknown attacks, anomaly detection can produce many false positives, and specification methods require expert policies.
  • Alert Correlation: Existing alarm-correlation and SIEM approaches cluster alerts or use causal relationships, often relying on third-party application logs and statistical features such as timestamps.Examples include BotHunter, HERCULE, Splunk, LogRhythm, and IBM QRadar.
  • Alert Correlation: HOLMES contrasts with timestamp-based correlation by using information flows between attack steps to support precise multi-stage APT detection.The paper states that APTs usually span long durations, limiting the usefulness of alert timestamps alone.
  • System-Call-Level Approaches: Compared with SLEUTH, HOLMES addresses dependence explosion using minimum ancestral cover and correlates exfiltration with other actions of the responsible process.The cited discussion also notes that finer-grained information-flow tracking can provide more precise provenance at performance cost.
  • Conclusion: HOLMES evaluates nine real-world APT threats and reports high-precision detection with low false-alarm rates while producing real-time high-level attack graphs.This conclusion summarizes the system’s detection and scenario-reconstruction capabilities.

APPENDIX

The appendix illustrates HOLMES across diverse attack scenarios, including Trojans, spyware, RATs, EternalBlue, web shells, and Nginx exploitation. The corresponding HSGs encode attack entities, processes, commands, files, credentials, and network endpoints, and can flag an attack even when a critical step is absent.

  • Scenario-2: Trojan: Scenario-2 follows a malicious download and execution through C&C communication, shell-based reconnaissance, and secret-file exfiltration.The HSG notation includes an untrusted external address, Firefox, a Trojan file and process, shell, reconnaissance commands, and password.txt.
  • Scenario-3: Trojan: Scenario-3 adds file movement and renaming before Trojan execution, followed by C&C, a reverse shell, reconnaissance commands, and another deployment stage.Its HSG includes Firefox, tedit, shell, malicious executable and process, password.txt, and reconnaissance utilities.
  • Scenario-4: Spyware: Scenario-4 begins with a compromised Firefox and remote URL, then uses shellcode, a downloaded binary, reverse-shell communication, command execution, and footprint deletion.The HSG includes Firefox.exe, malicious files and processes, cmd.exe, a batch file, and host-information commands.
  • Scenario-5.1: Eternal Blue: Scenario-5.1 exploits vulnerable SMB using EternalBlue and Double Pulsar, injects into lsass.exe, launches rundll32.exe, contacts C&C, and exfiltrates a sensitive file.The HSG includes the external address, lsass.exe, rundll32.exe, password.txt, and Winevt logs.
  • Scenario-5.2: RAT: Scenario-5.2 shows an incomplete RAT attack in which Firefox is exploited, the RAT executes and connects to C&C, then is deleted without causing harm.Its HSG contains the external address, Firefox.exe, the dropped malware file, and the executed malware process.
  • Scenario-6: Web-Shell: Scenario-6 uses an Nginx vulnerability to permit arbitrary commands and exfiltrate a sensitive file, while its HSG records the server, root user, and passwd.txt.The example omits initial compromise, yet strong TTP-matching signals allow HOLMES to flag the APT.
  • Scenario-7.1: RAT: Scenario-7.1 exploits Nginx with shellcode, downloads and executes a RAT, grants remote administrative privileges, runs commands, deploys Python scripts, and exfiltrates information.The HSG notation includes memory, root userID, malicious files and processes, uname, and /etc/shadow.
Loading 1810.01594v2…