HOLMES: Real-time APT Detection through Correlation of Suspicious Information Flows

• APT campaigns generally follow recurring kill-chain stages despite variation in concrete exploits, enabling high-level mapping of attacker behavior.Source: §I · p. 1
• Existing IDS and SIEM systems generate and correlate low-level alerts, but struggle to represent complex relationships and assemble reliable attack scenarios.Source: §I · pp. 1–2
• HOLMES maps host activities and enterprise alerts directly to kill-chain steps, making alerts more semantically meaningful than raw audit events.Source: §I · p. 2
• Around 3M low-level events from five days of audit logs are reduced to 86 suspicious activity steps.Source: §I · p. 2
• HOLMES correlates attack steps through information flows among files, processes, and other entities to construct an emerging kill-chain.Source: §I · p. 2
• The high-level scenario graph uses TTP nodes and information-flow edges, while dependency pruning, noise reduction, and prioritization reduce false alarms.Source: §I · p. 2
• From 10M audit records, HOLMES summarizes a high-level campaign in a graph of just 16 nodes for rapid analyst comprehension.Source: §I · p. 2
• Evaluation on nine real-life APT scenarios and a two-week live experiment showed clear attack-benign separation with high precision and recall.Source: §I · p. 2

• The example uses audit data from a vulnerable Nginx web server on FreeBSD to construct a provenance graph of system entities and activity.Source: §II · p. 2
• The attacker’s goal is to exfiltrate sensitive information from the system.Source: §II · p. 2
• The attack proceeds through initial compromise, command-and-control communication, privilege escalation, and internal reconnaissance.Source: §II · pp. 2–3
• The initial compromise executes in memory within the existing Nginx process, leaving minimal visible traces and blending with normal operation.Source: §II · p. 3
• A single host can generate tens of millions of events daily, creating a needle-in-a-haystack challenge for attack detection.Source: §II · p. 3
• The attack steps correspond to APT stages and are connected by information flows, motivating HOLMES’s detection approach.Source: §II · p. 3

• HOLMES bases detection on essential high-level APT steps and the information-flow dependencies connecting them.Source: §III · p. 3
• The approach addresses the semantic gap between low-level audit data and the high-level kill-chain view of attacker goals and capabilities.Source: §III · p. 3
• An intermediate ATT&CK-based TTP layer maps lower-level audit actions to high-level APT capabilities.Source: TTPs & HSG · p. 4
• HOLMES represents audit events as a provenance graph whose information-flow dependencies support efficient TTP matching.Source: TTPs & HSG · p. 4
• A High-level Scenario Graph represents matched TTPs as nodes and their information-flow and causality dependencies as edges.Source: TTPs & HSG · p. 4
• Benign-pattern learning and severity-weighted ranking help reduce false positives and prioritize HSGs for analysts.Source: TTPs & HSG · p. 4

• HOLMES assumes the OS kernel, auditing system, and generated logs are outside the threat model, with the system benign initially.Source: §IV · p. 4
• Audit data from Linux, BSD, and Windows is collected into an OS-neutral format accepted by HOLMES.Source: A. Data Collection and Representation · p. 4
• The provenance graph contains versioned subject and object nodes connected by event-annotated dependency edges.Source: A. Data Collection and Representation · p. 4
• HOLMES can accept additional IDS/IPS events and alerts, but this capability is not discussed further in the paper.Source: A. Data Collection and Representation · p. 4
• The compact provenance representation supports in-memory graph construction and real-time event consumption over prolonged periods.Source: A. Data Collection and Representation · p. 5

• TTPs map OS-neutral audit events and information-flow dependencies to high-level APT steps while remaining independent of many implementation details.Source: B. TTP Specification · p. 5
• TTP prerequisites encode causal relationships and information flows between APT stages, helping reduce false positives and clarify campaign context.Source: B. TTP Specification · p. 5
• Most TTPs are modeled using a single event with subject and object preconditions, avoiding expensive backtracking during matching.Source: B. TTP Specification · p. 5
• TTP rules specify an APT stage, TTP and entities, event family, and severity level used to prioritize alarms.Source: B. TTP Specification · p. 5
• Prerequisites can constrain TTP parameters, previously matched TTPs, process relationships, shared parents, and information flows.Source: B. TTP Specification · p. 5
• A TTP enters the HSG only when all prerequisites are satisfied, reducing graph size enough for sophisticated real-time analysis.Source: C. HSG Construction · pp. 5–6

• Spurious dependencies from benign processes can cause dependence explosion and obscure attack steps in analyst-facing HSGs.Source: D. Avoiding Spurious Dependencies · p. 6
• Strong dependencies are associated with shared compromised ancestors, whereas other dependencies are treated as weak and deemphasized.Source: D. Avoiding Spurious Dependencies · p. 6
• An ancestral cover contains an ancestor or the process itself for every process on an information-flow path.Source: D. Avoiding Spurious Dependencies · p. 6
• The minimum ancestral cover estimates the minimum number of compromised ancestors needed to control an information-flow path.Source: D. Avoiding Spurious Dependencies · p. 6
• For the nginx-to-cat path, the minimum ancestral cover has size 2, indicating that controlling cat would require an additional exploit.Source: D. Avoiding Spurious Dependencies · p. 6
• Path factor uses the smallest minimum-ancestral-cover value across paths to quantify attacker control over a flow.Source: D. Avoiding Spurious Dependencies · p. 6
• Using path factor mitigated dependency explosions by prioritizing attacker-influenced flows.Source: D. Avoiding Spurious Dependencies · p. 6

• Benign prerequisite matches are learned from training data and ignored at runtime when triggered TTP prerequisites match the learned benign context.Source: E. Noise Reduction · p. 6
• Filtering solely on benign prerequisites can miss malicious events that resemble learned benign behavior, such as a compromised nginx reading /etc/passwd.Source: E. Noise Reduction · p. 7
• HOLMES supplements prerequisite filtering with transferred-byte quantities, flagging flows whose observed volume substantially exceeds the benign amount.Source: E. Noise Reduction · p. 7

Signal correlation and detection

• Each HSG receives a seven-stage threat tuple whose entries represent the severity of the corresponding APT stages.Source: F. Signal Correlation and Detection · p. 7
• The weighted-product score combines stage severities, weights, and a detection threshold; absent stages receive severity 1.Source: F. Signal Correlation and Detection · p. 7
• Experiments comparing scoring schemes found that the weighted product produced the best normalized margin between benign and attack subgraphs.Source: F. Signal Correlation and Detection · p. 7

• Audit records from different operating systems are normalized into a common data representation and published to Kafka for real-time consumption.Source: §V · p. 7
• The policy engine checks TTP rules, prerequisites, and path factors while the provenance graph is being constructed.Source: §V · p. 8
• Incremental matching avoids backtracking by storing prior matches, creating HSG nodes, and propagating pointers through dependent low-level entities.Source: §V · p. 8
• Path factors are updated incrementally across provenance edges, with process ancestry determining whether the factor remains one or increases.Source: §V · p. 8
• Pointer propagation can add space overhead as provenance graphs grow, although many entities typically point to the same TTP set.Source: §V · p. 8
• The learning model stores matched TTP prerequisites and benign byte-transfer thresholds for runtime comparison.Source: §V · p. 9

• The evaluation used DARPA adversarial engagements, first analyzing an available dataset before selecting a threshold and measuring performance.Source: §VI · p. 9
• The datasets contain nine APT scenarios across seven hosts and three operating systems, covering 20 days of audit logs.Source: A. Datasets · p. 9, Table 10 · p. 9
• The attacks include drive-by compromise, backdoor injection, privilege escalation, reconnaissance, exfiltration, and cleanup, alongside sophisticated loading techniques.Source: A. Datasets · p. 9
• HOLMES was configured with representative TTPs, path thres = 3, and stage weights weight = (10+i)/10.Source: B. Setup · p. 9
• The highest benign score was 338, while the lowest attack score was 608, forming disjoint score clusters.Source: C. Results in a Nutshell · p. 10, Table 11 · p. 10
• Enabling learning and path-factor calculations increased the margin between attack and benign HSG scores; removing either increased noise and false positives or false negatives.Source: C. Results in a Nutshell · p. 10

Scenario-1: Drive-by Download

• Scenario-1 models a vulnerable Firefox visiting a malicious website, dropping a file named net during a drive-by download.Source: D. Attack Scenarios · p. 10
• The initial-compromise stage contains execution of an untrusted file matching a critical-severity TTP.Source: D. Attack Scenarios · p. 10
• The resulting threat tuple is ⟨C, M, −, H, −, H, M⟩, with quantitative values ⟨10, 6, 1, 8, 1, 8, 6⟩.Source: D. Attack Scenarios · p. 11
• 1163881 is the resulting threat score for the Scenario-1 graph.Source: D. Attack Scenarios · p. 11

Threshold selection

• The maximum F-score occurs for thresholds in [338.25, 608.26], separating the maximum benign and minimum attack subgraph scores in the tested datasets.Source: E. Finding the Optimal Threshold Value · p. 11
• After nth-root transformation, the maximum-F-score interval becomes [2.01, 2.16], with 2.09 selected as the midpoint average severity.Source: E. Finding the Optimal Threshold Value · p. 11

Graph Size

• 1875:1 edge ratio represents an 1875-fold reduction when mapping the provenance graph to the high-level scenario graph.The reduction is measured using graph edges.Source: F. Performance · p. 11

Memory Use

• HOLMES showed nearly linear memory growth with the number of audit records and was extrapolated to support enterprises with hundreds of hosts.The system operated on in-memory audit records during testing on an 8-core, 2.5GHz CPU with 150GB of RAM.Source: F. Performance · p. 11

Live Detection

• During a two-week live experiment, HOLMES analyzed Windows, Linux, and BSD audit logs in real time amid predominantly benign activity.The simulated enterprise included security-critical web, email, SSH, and SMB services.Source: G. Live Experiment · pp. 11–12

False Alarms

• Some benign HSGs exceeded the detection threshold, producing false positives attributed in part to SSH connections made by system administrators.The experiment used a threshold of 1378, and the cumulative distribution compared attack and benign HSG threat scores.Source: G. Live Experiment · p. 12
• A separate two-week benign activity period produced no false alarms, while no false negatives were observed during the experiments.The authors nevertheless discuss potential false-negative scenarios and allow analysts to neutralize falsely constructed HSGs.Source: G. Live Experiment · p. 12

Limitations

• HOLMES lacks direct visibility into causal relations carried by information flows that avoid system calls, although it can partially reconstruct attacks visible through later system-call activity.This limitation concerns implicit causality between tactics, techniques, and procedures.Source: G. Live Experiment · p. 12
• Multiple attacker entry points can create detached subgraphs that require additional analysis when no information flow connects their attack steps.HOLMES follows each entry point and correlates disjoint subgraphs when information flow exists between them.Source: G. Live Experiment · p. 12

Contributions

• HOLMES contributes to real-time alarm generation, alert correlation, and scenario reconstruction through a high-level attack scenario graph.The graph is the underlying basis for all three problems.Source: §VII · p. 12

Alarm Generation

• Misuse-based, anomaly-based, and specification-based host intrusion detectors trade off unknown-attack coverage, false positives, and policy-development effort.The paper notes that misuse detection cannot handle unknown attacks, anomaly detection can produce many false positives, and specification methods require expert policies.Source: §VII · p. 12

Alert Correlation

• Existing alarm-correlation and SIEM approaches cluster alerts or use causal relationships, often relying on third-party application logs and statistical features such as timestamps.Examples include BotHunter, HERCULE, Splunk, LogRhythm, and IBM QRadar.Source: §VII · pp. 12–13
• HOLMES contrasts with timestamp-based correlation by using information flows between attack steps to support precise multi-stage APT detection.The paper states that APTs usually span long durations, limiting the usefulness of alert timestamps alone.Source: §VII · p. 13

System-Call-Level Approaches

• Compared with SLEUTH, HOLMES addresses dependence explosion using minimum ancestral cover and correlates exfiltration with other actions of the responsible process.The cited discussion also notes that finer-grained information-flow tracking can provide more precise provenance at performance cost.Source: §VII · p. 13

Conclusion

• HOLMES evaluates nine real-world APT threats and reports high-precision detection with low false-alarm rates while producing real-time high-level attack graphs.This conclusion summarizes the system’s detection and scenario-reconstruction capabilities.Source: §VIII · p. 13

Scenario-2: Trojan

• Scenario-2 follows a malicious download and execution through C&C communication, shell-based reconnaissance, and secret-file exfiltration.The HSG notation includes an untrusted external address, Firefox, a Trojan file and process, shell, reconnaissance commands, and password.txt.Source: APPENDIX · p. 15, Figure 19 · p. 15

Scenario-3: Trojan

• Scenario-3 adds file movement and renaming before Trojan execution, followed by C&C, a reverse shell, reconnaissance commands, and another deployment stage.Its HSG includes Firefox, tedit, shell, malicious executable and process, password.txt, and reconnaissance utilities.Source: APPENDIX · p. 15, Figure 20 · p. 15

Scenario-4: Spyware

• Scenario-4 begins with a compromised Firefox and remote URL, then uses shellcode, a downloaded binary, reverse-shell communication, command execution, and footprint deletion.The HSG includes Firefox.exe, malicious files and processes, cmd.exe, a batch file, and host-information commands.Source: APPENDIX · p. 15, Figure 21 · p. 15

Scenario-5.1: Eternal Blue

• Scenario-5.1 exploits vulnerable SMB using EternalBlue and Double Pulsar, injects into lsass.exe, launches rundll32.exe, contacts C&C, and exfiltrates a sensitive file.The HSG includes the external address, lsass.exe, rundll32.exe, password.txt, and Winevt logs.Source: APPENDIX · pp. 15–16, Figure 22 · p. 16

Scenario-5.2: RAT

• Scenario-5.2 shows an incomplete RAT attack in which Firefox is exploited, the RAT executes and connects to C&C, then is deleted without causing harm.Its HSG contains the external address, Firefox.exe, the dropped malware file, and the executed malware process.Source: APPENDIX · p. 16, Figure 23 · p. 16

Scenario-6: Web-Shell

• Scenario-6 uses an Nginx vulnerability to permit arbitrary commands and exfiltrate a sensitive file, while its HSG records the server, root user, and passwd.txt.The example omits initial compromise, yet strong TTP-matching signals allow HOLMES to flag the APT.Source: APPENDIX · p. 16, Figure 24 · p. 16

Scenario-7.1: RAT

• Scenario-7.1 exploits Nginx with shellcode, downloads and executes a RAT, grants remote administrative privileges, runs commands, deploys Python scripts, and exfiltrates information.The HSG notation includes memory, root userID, malicious files and processes, uname, and /etc/shadow.Source: APPENDIX · p. 16, Figure 25 · p. 16