PaperlayerView exact revision on arXiv

Source-linked AI summary

Enabling Efficient Cyber Threat Hunting With Cyber Threat Intelligence

Peng Gao, Fei Shao, Xiaoyuan Liu, Xusheng Xiao, Zheng Qin, Fengyuan Xu, Prateek Mittal, Sanjeev R. Kulkarni, Dawn Song

arXiv:2010.13637v2cs.CRcs.CLcs.DB

TL;DR

Threat hunting often requires labor-intensive manual query construction and overlooks OSCTI. ThreatRaptor uses OSCTI to synthesize TBQL hunts over system audit logs; evaluations report strong extraction scores and faster query execution than native database schedulers.

Abstract

from arXiv · show

Log-based cyber threat hunting has emerged as an important solution to counter sophisticated attacks. However, existing approaches require non-trivial efforts of manual query construction and have overlooked the rich external threat knowledge provided by open-source Cyber Threat Intelligence (OSCTI). To bridge the gap, we propose ThreatRaptor, a system that facilitates threat hunting in computer systems using OSCTI. Built upon system auditing frameworks, ThreatRaptor provides (1) an unsupervised, light-weight, and accurate NLP pipeline that extracts structured threat behaviors from unstructured OSCTI text, (2) a concise and expressive domain-specific query language, TBQL, to hunt for malicious system activities, (3) a query synthesis mechanism that automatically synthesizes a TBQL query for hunting, and (4) an efficient query execution engine to search the big audit logging data. Evaluations on a broad set of attack cases demonstrate the accuracy and efficiency of ThreatRaptor in practical threat hunting.

I. INTRODUCTION

Threat hunting over system audit logs is labor-intensive because existing approaches require manual query construction and overlook external threat knowledge. ThreatRaptor uses OSCTI to extract threat behaviors and synthesize queries for hunting in audit data.

  • Existing threat hunting requires substantial manual query construction and overlooks rich external threat knowledge, making the process labor-intensive and error-prone.Cyber threat hunting proactively and iteratively searches logs for malicious actors and indicators, supporting early-stage detection.Source: §I · p. 1
  • Unstructured OSCTI reports connect indicators into multi-step threat behaviors, unlike fragmented IOC feeds, which can be evaded when attackers change tools or signatures.Prior approaches do not automate harvesting and use of this connected threat knowledge for hunting.Source: §I · p. 1
  • ThreatRaptor targets both nuanced IOC extraction and efficient search across large audit logs, which can reach 0.5–1 GB per enterprise host daily.It uses auditing frameworks and database backends, and its execution engine reduces events, decomposes queries, and schedules them by estimated pruning power and semantic dependencies.Source: §I · pp. 1–2
  • ThreatRaptor extracts IOCs and their relations from OSCTI with a specialized NLP pipeline, then represents them as a structured threat behavior graph.Its techniques include IOC protection and dependency parsing-based relation extraction.Source: §I · p. 2
  • TBQL provides declarative constructs for system entities, events, operations, and event paths to specify and search complex multi-step behaviors.A query synthesis mechanism automatically converts the threat behavior graph into a TBQL query; analysts can edit synthesized queries with enterprise-specific knowledge.Source: §I · p. 2
  • Evaluations report 96.64% F1 for IOC extraction, 92.59% F1 for IOC relation extraction, and 98.34% F1 for finding malicious system activities.Threat behavior extraction and query synthesis take 0.52 seconds on average.Source: §I · p. 2
  • TBQL queries run 22.7x faster than SQL on PostgreSQL and 9.1x faster than Cypher on Neo4j.The system also reports TBQL queries are more concise than SQL (> 2.8x) and Cypher (> 2.2x).Source: §I · p. 2

II. SYSTEM OVERVIEW

ThreatRaptor turns OSCTI into a behavior graph and synthesized TBQL query, then searches system audit logs. Its overview describes the audited entities and events, database storage, and event reduction.

II. SYSTEM OVERVIEW

  • ThreatRaptor extracts IOCs and their relations from OSCTI, builds a threat behavior graph, synthesizes a TBQL query, and executes it against collected audit logs.Analysts can revise synthesized queries, or hunt proactively with manually constructed TBQL when an OSCTI report is unavailable.Source: §II · p. 3
  • The threat model excludes attacks that bypass kernel-layer auditing and assumes the kernel and auditing frameworks are trusted and collected logs are untampered.Examples outside the scope include side-channel and memory-based attacks.Source: §II · p. 3

A. System Auditing

  • System auditing models interactions as events ⟨subject_entity, operation, object_entity⟩ among files, processes, and network connections.Subjects are processes; objects may be files, processes, or network connections, yielding file, process, and network events.Source: A. System Auditing · p. 3

B. Data Storage

  • THREATRAPTOR stores entities and events in relational and graph databases, matching join-heavy queries to relational storage and graph-pattern searches to graph storage.It also merges repeated events between the same entities when operation types match and the gap is at most one second, preserving the combined time span and data amount.Source: B. Data Storage · p. 4

C. Threat Behavior Extraction

ThreatRaptor uses an unsupervised, lightweight NLP pipeline to extract IOCs and relations from OSCTI and represent them as an ordered threat behavior graph. It protects IOCs during parsing and applies dependency-based rules to identify relations.

C. Threat Behavior Extraction

  • The specialized pipeline is unsupervised and lightweight, addressing OSCTI-specific text nuances to extract IOCs and their relations into a threat behavior graph.The paper identifies dots and underscores in IOCs as examples of nuances that challenge general NLP modules.Source: C. Threat Behavior Extraction · p. 4
  • Regex rules recognize IOC types, then replace IOCs with a dummy word during NLP processing and restore them afterward to preserve security context.The rules extend an open-source parser, including improvements to distinguish Linux and Windows file paths.Source: C. Threat Behavior Extraction · p. 4
  • After resolving IOC coreferences within blocks and merging similar IOCs across blocks, the pipeline orders extracted triplets by relation-verb position and numbers graph edges to represent step order.Cross-block merging uses character-level overlap and semantic similarity of word vectors.Source: C. Threat Behavior Extraction · pp. 4–5
  • The pipeline enumerates IOC pairs in dependency trees, checks subject-object relations along paths through their lowest common ancestor, and selects the nearest eligible relation verb to form triplets.A final verb must appear in the curated keyword list and form the correct subject-verb-object relation with the IOC pair.Source: C. Threat Behavior Extraction · pp. 4–5

D. Threat Behavior Query Language (TBQL)

TBQL is a domain-specific language for searching audit logs for complex system behaviors. It expresses event patterns and their relationships, and adds variable-length event paths to bridge threat steps with paths in system logs.

D. Threat Behavior Query Language (TBQL)

  • TBQL’s concise primitives make complex multi-step behaviors easier to specify than low-level, verbose SQL or Cypher queries.Length-one path patterns are compiled to Cypher on Neo4j, while basic event patterns are compiled to SQL on PostgreSQL.Source: D. Threat Behavior Query Language (TBQL) · pp. 5–6
  • TBQL specifies typed system entities and event patterns with optional attribute filters, then relates patterns through temporal or attribute constraints.The resulting patterns represent a subgraph of system events to search.Source: D. Threat Behavior Query Language (TBQL) · p. 5
  • TBQL infers default entity attributes and allows reused entity IDs across patterns to indicate that those patterns share the same entity.Defaults are name for files, exename for processes, and dstip for network connections.Source: D. Threat Behavior Query Language (TBQL) · p. 5
  • Variable-length event path patterns let TBQL search paths of system events between entities, including cases where intermediate processes are omitted from OSCTI descriptions.This can bridge a threat-graph edge to a path in audit logs; path length and final-hop operation can be constrained.Source: D. Threat Behavior Query Language (TBQL) · pp. 5–6

E. TBQL Query Synthesis

ThreatRaptor synthesizes TBQL queries from threat behavior graphs through screening, relation mapping, pattern construction, and return-clause generation. User-defined plans can override the defaults to add attributes not captured in the graph.

E. TBQL Query Synthesis

  • Pre-synthesis screening removes graph nodes whose IOC types are not captured by system auditing, then maps remaining relations to TBQL operations using IOC types and relation semantics.For example, “download” between filepath IOCs maps to write, while filepath-to-IP “download” maps to read.Source: E. TBQL Query Synthesis · p. 6
  • For each graph node, synthesis assigns a TBQL entity ID, entity type, and IOC-derived attribute, then connects entities with the mapped operation in an event pattern by default.Administrators can configure synthesis to use variable-length event path patterns instead.Source: E. TBQL Query Synthesis · p. 6
  • For event patterns, synthesis orders temporal relationships according to the sequence numbers of their corresponding threat-graph edges.This relationship-synthesis step is omitted for variable-length event path patterns.Source: E. TBQL Query Synthesis · p. 6
  • The default return clause includes all synthesized entity IDs, while user-defined synthesis plans can add supported attributes absent from the threat behavior graph.Examples include hostname and time window.Source: E. TBQL Query Synthesis · p. 6

F. TBQL Query Execution

ThreatRaptor compiles TBQL patterns into SQL or Cypher queries and schedules them across database backends using pruning estimates and pattern dependencies. It also offers fuzzy graph matching when exact search does not return meaningful results.

F. TBQL Query Execution

  • Event patterns become SQL queries, while variable-length event paths become Cypher queries suited to graph-pattern searches.SQL leverages relational indexing and joins; Cypher provides path-pattern syntax.Source: F. TBQL Query Execution · p. 6
  • The scheduler prioritizes constrained patterns, then uses their results to filter dependent queries; shorter maximum path lengths raise path-pattern priority.Pruning scores count constraints, with smaller maximum path lengths additionally favored for variable-length paths.Source: F. TBQL Query Execution · pp. 6–7
  • Fuzzy search uses IOC-string similarity and graph alignment when exact matching fails, trading longer execution time for broader searching.It uses Levenshtein distance for IOC strings and Poirot’s graph-alignment approach for system-event subgraphs.Source: F. TBQL Query Execution · p. 7

IV. EVALUATION

The evaluation examines threat-behavior extraction, end-to-end hunting, query synthesis and execution, and TBQL conciseness. It uses real audit logs and a benchmark of 18 attack cases.

IV. EVALUATION

  • ThreatRaptor combines Sysdig auditing, PostgreSQL and Neo4j storage, Python and spaCy extraction, an ANTLR 4 TBQL parser, and Java system code.The system is approximately 25K lines of code.Source: §IV · p. 7
  • 18 attack cases form the evaluation benchmark, while the evaluation logs contain 47,688,033 system entities and 55,840,381 system events.The logs were collected on a physical testbed; Table IV lists the attack cases.Source: §IV · p. 7, Table IV · p. 7
  • The research questions assess extraction and hunting accuracy, extraction-to-query-synthesis efficiency, query-execution efficiency, and TBQL conciseness.The efficiency questions also examine the TBQL query scheduler’s speedup.Source: §IV · p. 7

A. Evaluation Setup

The benchmark combines DARPA TC cases with multi-step attacks conducted on a shared testbed, where benign activity substantially outnumbers attack activity.

A. Evaluation Setup

  • The benchmark contains 18 cases: 15 from DARPA TC and three multi-step attacks performed by the authors using a physical testbed.Monitoring covered 41 days for DARPA TC cases and 16 hours for the three authored attacks.Source: A. Evaluation Setup · p. 7
  • The testbed continued serving more than 15 active users during attacks and evaluations, capturing benign background activity alongside malicious events.Benign activity significantly outnumbered attack activity: 55 million versus thousands.Source: A. Evaluation Setup · p. 7
  • The 15 DARPA TC cases span operating systems, vulnerabilities, and exploit types, while the three authored cases broaden benchmark coverage with multi-step intrusions.The authored attacks were based on CVEs and traits from the Cyber Kill Chain framework, including initial penetration and data exfiltration.Source: A. Evaluation Setup · p. 7

1) DARPA TC Attack Cases:

ThreatRaptor’s extraction pipeline outperforms general open information extraction approaches on IOC entities and relations. The evaluation also uses selected DARPA TC cases and three authored multi-step attacks.

1) DARPA TC Attack Cases:

  • The DARPA TC selection includes 15 cases from four performer systems; TA5.2 logs were missing and CADETS logs lacked key attributes, preventing faithful evaluation.The cases cover varied operating systems, vulnerabilities, and exploits.Source: 1) DARPA TC Attack Cases · pp. 7–8

2) Multi-Step Intrusive Attack Cases:

  • Three authored multi-step intrusions extend benchmark coverage with CVE-based attacks reflecting Cyber Kill Chain traits, including initial penetration and data exfiltration.Attack descriptions were constructed to match how the attacks were performed.Source: 2) Multi-Step Intrusive Attack Cases · p. 8

1) RQ1: Accuracy of Threat Behavior Extraction:

  • 96.64% F1 for IOC entity extraction and 92.59% F1 for IOC relation extraction were the highest; Stanford Open IE and Open IE 5 scored below 5% and 0%, respectively.Without IOC Protection, ThreatRaptor’s F1 fell to 59.26% for IOC entity extraction and 16.39% for IOC relation extraction.Source: 1) RQ1: Accuracy of Threat Behavior… · p. 8
  • IOC Protection prevents NLP segmentation and tokenization from breaking security-specific indicators, supporting accurate entity and relation extraction from OSCTI text.Directly processing the text without protection can break file paths, executable names, and IPs into pieces.Source: 1) RQ1: Accuracy of Threat Behavior… · p. 8

2) RQ2: Accuracy of Threat Hunting:

ThreatRaptor finds malicious system events with 100% precision, 96.74% recall, and 98.34% F1. Its evaluations also report fast threat-behavior extraction and query execution speedups over native database schedulers.

2) RQ2: Accuracy of Threat Hunting:

  • 100% precision, 96.74% recall, and 98.34% F1 were achieved in finding malicious system events from OSCTI texts.In three ClearScope Android cases, the system also handled Android package names as process executable names.Source: 2) RQ2: Accuracy of Threat Hunting · pp. 8–9, Table VI · p. 9
  • 37 system events were missed in one case when query synthesis mapped an ambiguous IOC relation to the wrong event pattern.The authors identify semantic ambiguity in query synthesis as the primary reason for false negatives and suggest analyst query revision to improve coverage.Source: 2) RQ2: Accuracy of Threat Hunting · p. 9

3) RQ3: Efficiency of Threat Behavior Extraction:

4) RQ4: Efficiency of TBQL Query Execution:

  • 22.7x faster on PostgreSQL and 9.1x faster on Neo4j were the TBQL scheduler's query-execution speedups over the respective native schedulers.The authors note that TBQL can be slower for one-pattern queries, while SQL and Cypher can slow substantially as pattern counts grow.Source: 4) RQ4: Efficiency of TBQL Query… · pp. 9–10, Table VIII · p. 10
  • THREATRAPTOR-Fuzzy trades efficiency for generality, and some dense, high-alignment cases did not finish within one hour; the authors recommend exact search when possible.For fuzzy searches that finished within one hour, all ground-truth attack activities were found.Source: 4) RQ4: Efficiency of TBQL Query… · p. 10

5) RQ5: Conciseness of TBQL:

TBQL expresses system behaviors more concisely than SQL and Cypher across the evaluated cases. Its conciseness advantage grows with more patterns, while Cypher is generally more concise than SQL.

5) RQ5: Conciseness of TBQL:

  • Across four RQ4 query types, TBQL used 3.4x fewer characters than SQL and 2.9x fewer than Cypher, and 2.8x fewer words than SQL and 2.2x fewer than Cypher.Character counts exclude spaces and comments.Source: 5) RQ5: Conciseness of TBQL · pp. 10–11
  • TBQL's conciseness saving over SQL and Cypher increases when more patterns are declared.The authors attribute TBQL's conciseness to directly modeling system entities and events rather than lower-level tables or nodes and relationships.Source: 5) RQ5: Conciseness of TBQL · pp. 10–11
  • Cypher queries are generally more concise than SQL queries in the evaluated cases.The paper attributes this to Cypher's concise syntax for linked nodes and relationships, while SQL requires explicit table joins.Source: 5) RQ5: Conciseness of TBQL · p. 11

V. DISCUSSION

The discussion bounds ThreatRaptor by audit coverage and usable OSCTI, outlines search and scope extensions, and contrasts it with related work; the conclusion states that it facilitates hunting using OSCTI.

V. DISCUSSION

  • ThreatRaptor excludes attacks on kernels, auditing frameworks, and databases, as well as activity not captured by system auditing.Its extraction pipeline also depends on available OSCTI text containing useful information, such as IOCs and sentences expressing IOC relations.Source: §V · p. 11
  • When OSCTI and ground truth differ, exact search may miss activities; fuzzy search can find candidates before users revise queries and return to exact search.The work targets extraction from input reports, not CTI collection; enterprises may also have proprietary reports that better reflect their environments.Source: §V · p. 11
  • Threat actors and security tools are possible threat-behavior entities beyond the current IOC-focused extraction; supervised NER and relation extraction would require costly annotated corpora.The authors plan to explore programmatically synthesized annotations to support learning-based expansion.Source: §V · p. 11
  • Query synthesis screens out threat-graph nodes whose IOC types are not captured by auditing; planned monitoring additions include Windows registry entries and Linux pipes.Source: §V · p. 11

VI. RELATED WORK

  • Prior audit-log query work addresses investigation and anomaly detection, whereas ThreatRaptor extracts OSCTI knowledge and synthesizes hunting queries; its TBQL also supports variable-length event paths.Source: §VI · p. 11
  • Poirot searches for an acceptable provenance-subgraph alignment and requires analysts to build query graphs manually, unlike ThreatRaptor’s automated graph construction and query synthesis.Poirot’s inexact matching can inform ThreatRaptor’s fuzzy search, which supports exhaustive search.Source: §VI · p. 11
  • OSCTI analysis work includes IOC extraction and management standards, while Open IE extracts general relations; ThreatRaptor targets IOC relations and threat behaviors in security text.Source: §VI · p. 12

VII. CONCLUSION

  • The conclusion presents ThreatRaptor as a system for facilitating cyber threat hunting in computer systems using OSCTI.Source: §VII · p. 12
Loading 2010.13637v2…