Source-linked AI summary
AttacKG: Constructing Technique Knowledge Graph from Cyber Threat Intelligence Reports
Zhenyuan Li, Jun Zeng, Yan Chen, Zhenkai Liang
TL;DR
CTI reports are unstructured, manually intensive to analyze, and often cover only limited aspects of attack techniques. AttacKG extracts attack graphs, identifies techniques, and aggregates intelligence into technique knowledge graphs, achieving accurate extraction and large-scale collection across real-world reports.
Problem
CTI reports are natural-language documents that require tedious manual recovery and typically provide only limited coverage of attack techniques.
Method
AttacKG extracts attack-relevant entities and dependencies as attack graphs, aligns them with technique templates, and aggregates technique-level intelligence into technique knowledge graphs.
Results
AttacKG outperforms EXTRACTOR and TTPDrill, with F1-scores of 0.887 for entities, 0.896 for dependencies, and 0.789 for adversarial techniques, while collecting 28,262 techniques and 8,393 unique IoCs from 1,515 reports.
Takeaways & Limitations
Technique-level intelligence from aggregated CTI reports can support downstream tasks such as APT detection and cyber attack reconstruction.
Takeaways & Limitations
Existing CTI parsing approaches may extract separate techniques without the whole attack picture, generate false positives, or lose structural information needed for accurate technique identification.
Abstract
from arXiv · showhide
Cyber attacks are becoming more sophisticated and diverse, making detection increasingly challenging. To combat these attacks, security practitioners actively summarize and exchange their knowledge about attacks across organizations in the form of cyber threat intelligence (CTI) reports. However, as CTI reports written in natural language texts are not structured for automatic analysis, the report usage requires tedious manual efforts of cyber threat intelligence recovery. Additionally, individual reports typically cover only a limited aspect of attack patterns (techniques) and thus are insufficient to provide a comprehensive view of attacks with multiple variants. To take advantage of threat intelligence delivered by CTI reports, we propose AttacKG to automatically extract structured attack behavior graphs from CTI reports and identify the adopted attack techniques. We then aggregate cyber threat intelligence across reports to collect different aspects of techniques and enhance attack behavior graphs into technique knowledge graphs (TKGs). In our evaluation against 1,515 real-world CTI reports from diverse intelligence sources, AttacKG effectively identifies 28,262 attack techniques with 8,393 unique Indicators of Compromises (IoCs). To further verify the accuracy of AttacKG in extracting threat intelligence, we run AttacKG on 16 manually labeled CTI reports. Empirical results show that AttacKG accurately identifies attack-relevant entities, dependencies, and techniques with F1-scores of 0.887, 0.896, and 0.789, which outperforms the state-of-the-art approaches Extractor and TTPDrill. Moreover, the unique technique-level intelligence will directly benefit downstream security tasks that rely on technique specifications, e.g., APT detection and cyber attack reconstruction.
1 Introduction
AttacKG addresses the difficulty of recovering structured attack behavior and technique knowledge from natural-language CTI reports. It parses reports into attack graphs, aligns them with technique templates, and aggregates intelligence across reports into technique knowledge graphs.
- Natural-language CTI reports contain attack entities and dependencies but require substantial manual effort to recover structured attack behavior.
- AttacKG parses CTI reports into attack graphs, generates technique templates, and uses graph alignment to identify and refine techniques.
- 28,262 techniques and 8,393 unique IoCs were collected from 1,515 CTI reports.
- AttacKG achieved F1-scores of 0.887 for entities, 0.896 for dependencies, and 0.789 for adversarial techniques.
- The work presents technique-level aggregation across multiple CTI reports as a contribution intended to support downstream security tasks.
2 Background and Related Work
The background motivates technique-level intelligence because attack techniques have varied implementations while existing extraction methods lose semantic or structural information. AttacKG addresses this gap by aggregating implementation details and enriching attack graphs with technique knowledge.
- 2.1 Cyber Attacks and Reports: MITRE ATT&CK techniques can have multiple implementations, and implementation details vary more than the underlying ways of performing techniques.
- 2.2 Threat Intelligence Extraction: CTI reports provide detailed implementation knowledge, but manually extracting attack-relevant information from unstructured text is labor-intensive and error-prone.
- 2.2 Threat Intelligence Extraction: Structured IoC formats are machine-readable but lack semantic information about how entities interact to form an attack chain.
- 2.3 Motivating Example: Existing methods may identify separate techniques, provide coarse-grained classifications, or merge same-type entities and lose attack structure.
- 2.3 Motivating Example: AttacKG aggregates technique-level knowledge across reports and uses it to enrich attack graphs with more comprehensive technique information.
3 Approach
AttacKG uses two connected subsystems to parse CTI reports into attack graphs, identify techniques, and aggregate knowledge into technique templates and TKGs.
- Architecture: AttacKG parses CTI reports and builds multi-technique attack graphs, while MITRE ATT&CK procedure examples provide single-technique graphs.The two inputs are treated as isomorphic, producing graph representations for individual techniques and attack campaigns.
- Architecture: Technique templates aggregate technique-level intelligence across CTI reports, addressing the limited attack-pattern coverage of individual reports.The templates are paired with a revised graph-alignment algorithm for identifying techniques in extracted attack graphs.
- Outputs: The system outputs technique templates and TKGs, while repeated observations across reports allow the template to tolerate some parsing errors.Templates collect aggregated attack knowledge, whereas TKGs summarize complete attack chains in CTI reports.
- CTI report parsing: The pipeline addresses domain-specific terms, entity and dependency extraction, co-reference resolution, and attack-graph construction challenges in natural-language CTI.CTI reports contain specialized terms and summarized, coarse-grained dependencies that complicate automated recovery of attack behavior.
- CTI report parsing: AttacKG extracts entities with learning-based and customized rule-based recognizers, resolves explicit co-references, and constructs dependencies from parsed sentences.The entity taxonomy includes Actor, Executable, File, Network connection, Registry, and Others.
Attack Graph Generation and Simplification [C3, C4].
AttacKG constructs attack graphs by merging entity dependencies across sentences, then uses fuzzy graph alignment to identify techniques and enrich those graphs into TKGs.
- Attack graph construction: An attack graph Ga represents attack-relevant entities as nodes and their dependencies as edges, with co-reference merging joining sentence-level subgraphs.Explicit and implicit co-references support cross-sentence dependency construction and removal of redundant nodes.
- Attack graph construction: The parser generates concise attack graphs, and evaluation indicates that extraction errors have limited impact on subsequent fuzzy alignment-based technique identification.The reported pipeline is described as accurate and effective on a wide range of CTI reports.
- Technique templates: AttacKG represents each technique as a graph template containing aggregated entity knowledge and possible dependencies among entities.Template graphs support a universal description of attack techniques for matching against extracted attack graphs.
- Graph alignment: The revised alignment algorithm uses node and graph alignment to fuzzy-match technique templates with subgraphs of attack graphs despite false positives and false negatives.Node alignment compares candidate entities, while graph alignment measures overall similarity between a template and an attack-graph subgraph.
- Graph alignment: Node alignment assigns zero score to different entity types; matching types receive a type score combined with character-level IoC and natural-language similarity.Candidates are retained when their alignment scores reach a predefined threshold.
- Graph alignment: Graph alignment combines occurrence-weighted node scores with edge scores based on endpoint alignments, minimum hop distance, and dependency occurrence counts.The resulting scores are normalized to [0, 1], and disconnected nodes receive infinite minimum hop distance.
- TKG construction: Alignment results attach alternative entities and techniques from templates to attack graphs, producing TKGs that describe complete attack chains.The same alignment results support both TKG construction and technique-template initialization and updating.
Initialization and Updating of Technique Templates
Technique templates are initialized from MITRE-derived examples and updated with aligned CTI evidence to aggregate variants across reports.
- Updating: Template updating adds new IoC and natural-language terms from aligned attack-graph nodes, aggregating threat intelligence at the technique level.Alignment maps attack-graph nodes to corresponding template nodes before updating their attribute sets.
- Initialization: Initialization begins with a random single-technique attack graph from MITRE examples and merges aligned graphs of the same technique.Unaligned nodes are added as new template nodes, distinguishing initialization from later updating.
- Result: The resulting templates aggregate threat intelligence and cover multiple technique variants across multiple reports.The paper illustrates this process with a technique template example.
4 Evaluation
AttacKG is evaluated as both a CTI parser and a technique-level intelligence collector using manually labeled reports and a large multi-source report corpus. It extracts attack graphs and techniques accurately, while aggregating technique and IoC knowledge for attack analysis and reconstruction.
- Evaluation setup: AttacKG is evaluated on 1,515 real-world CTI reports and 7,373 procedure examples covering 179 MITRE ATT&CK techniques.Sixteen reports are manually labeled for entities, dependencies, and techniques to establish ground truth.
- RQ1: Attack graph extraction: 0.12 is AttacKG’s average F1-score improvement over EXTRACTOR for attack-graph extraction.AttacKG has slightly lower precision but a much lower false-negative rate; EXTRACTOR’s aggregation design loses structural information.
- RQ2: Technique identification: 0.462 is AttacKG’s technique-identification F1-score improvement over TTPDrill, while its precision improvement is 0.575.TTPDrill produces 15.5 false-positive techniques per report on average, whereas AttacKG aligns techniques with full attack-graph contexts.
- Case studies: TKGs aggregate technique-level knowledge to enrich reports, support attack reconstruction, and enhance detection of attack variants.In the Frankenstein case, a TKG identifies nine techniques across six tactics and helps infer the environment needed for reproduction.
5 Conclusion
The paper presents AttacKG as a solution for retrieving structured threat intelligence from CTI reports. Its evaluation indicates accurate attack-graph extraction and effective aggregation of technique-level intelligence across 1,515 real-world reports.
- Conclusion: AttacKG uses technique templates to identify and aggregate technique-level threat intelligence across massive CTI-report collections.The aggregated knowledge enhances extracted attack graphs and generates technique knowledge graphs.
- Conclusion: AttacKG extracts attack graphs accurately and aggregates technique-level threat intelligence effectively across 1,515 real-world CTI reports.
A Another Case Study
The case study examines how technique-level intelligence supports detection of attack variants. Aggregated knowledge enables independent technique detection and improves coverage across mutated implementations.
- Variant construction: The Frankenstein two-stage attack contains variants where a file-server URL changes and Boot Autostart switches from Registry Run Keys to DLL Side-loading.These mutations do not affect the attack’s functionality.
- Detection comparison: The comparison includes node-level intelligence, manually extracted attack graphs, and AttacKG’s aggregated technique-level intelligence.The schemes are represented by iACE, Poirot, and the proposed approach, respectively.
- Detection implications: Technique-level intelligence enables detectors to detect different attack techniques independently.Pooled knowledge from multiple reports can improve detection of various attack variants and can be merged with Eiger for greater generality.
B Selecting the Threshold Value
AttacKG selects node- and graph-alignment thresholds by balancing matching accuracy and analysis efficiency. The reported optimal values are 0.65 for node alignment and 0.85 for graph alignment.
- Threshold trade-offs: Too-low graph-alignment thresholds can cause premature false-positive matches, whereas too-high thresholds can miss reasonable matches.Node thresholds similarly affect unnecessary alignment candidates and analysis cost.
- Threshold selection: 0.65 for node alignment and 0.85 for graph alignment are the selected optimal thresholds.The values are chosen using F-score and analysis-time measurements across varying thresholds.
C Ablation Study of AttacKG.
The ablation study evaluates how removing entity attributes, dependencies, or graph simplification affects AttacKG’s technique-identification performance. Removing any component degrades overall performance, with dependency removal performing worst across metrics.
- AttacKG variants remove IoC information, natural-language text, dependencies, or graph simplification to test individual components.
- Removing any component degrades AttacKG’s overall performance in technique identification.The variants’ identification thresholds are adjusted separately using optimal F1-scores.
- AttacKG without dependencies performs worst consistently across all evaluation metrics.
- Table 6 summarizes the ablation study of components used in technique identification.
D Efficiency of AttacKG
AttacKG is evaluated for runtime efficiency against Extractor and TTPDrill on 16 CTI report samples. It is slower than TTPDrill but substantially faster than Extractor while also performing graph extraction and technique identification.
- The evaluation uses 16 CTI reports ranging from 61 to 1029 words, averaging 278.2 words.
- 24.0 seconds is AttacKG’s average total runtime per report, including 8.9 seconds for graph extraction and 15.1 seconds for technique identification.
- 239.70 seconds is Extractor’s average runtime per report, the highest among the three models.Extractor uses multiple NLP models and has the highest runtime overhead.
- 5.9 seconds is TTPDrill’s average runtime per report, making it the fastest model in the comparison.TTPDrill does not construct attack graphs but has a high false-positive rate.