Looking Beyond IoCs: Automatically Extracting Attack Patterns from External CTI

1 INTRODUCTION

• Static indicators such as malware hashes and IP addresses can be altered to evade detection, motivating more robust tactical intelligence from CTI.CTI is also noisy and unstructured, complicating extraction and integration into defense infrastructure.Source: §1 · p. 1
• Natural-language TTP descriptions and evolving sub-techniques make standardization difficult, while rule-based extraction may miss patterns without trigger words.The paper argues for machine-learning extraction of higher-level tactical intelligence from unstructured CTI.Source: §1 · p. 2
• LADDER automatically extracts and restructures attack patterns and IoCs from unstructured CTI, then classifies patterns according to MITRE ATT&CK techniques.It uses an ontology and TTPClassifier to organize information in a knowledge graph for predictive analysis.Source: §1 · p. 2
• The paper presents the framework as supporting analyst applications including attack-pattern extraction, malware-behavior prediction, threat hunting, and APT attribution.The authors describe this as the first work to use a knowledge graph for malware attack-pattern prediction.Source: §1 · p. 2
• The open-access benchmark malware dataset contains 140,447 tokens, 11,555 manually annotated named entities, and 5,499 relations.The dataset and code are publicly available.Source: §1 · p. 2

2 MOTIVATING EXAMPLE

• The example uses Cerberus, an Android banking trojan whose CTI describes its capabilities, tactics, techniques, and procedures.The report also covers vulnerabilities across technologies including email, domains, and mobile devices.Source: §2 · p. 2
• The Cerberus report is transformed from an annotated excerpt into triples and then a knowledge graph.The example includes triples for malware type, targets, and attack patterns such as bypassing security measures and stealing bank credentials.Source: §2 · p. 2
• The knowledge graph uses adapted ontology classes and predefined relations, following STIX2.1 for TTP and IoC exchange.Historical malware information, including attack patterns, is extracted from CTI reports to train a threat-intelligence model.Source: §2 · p. 2
• Analysts can query the graph to infer attack patterns a malware might attempt even if those patterns were not previously reported or observed.The paper links ATT&CK knowledge to mitigation techniques that can inform analyst action.Source: §2 · p. 2

3.1 Cyber Threat Intelligence

• CTI can be collected internally or externally, but public reports are often unstructured, making extraction of relevant tactical information challenging.Structured sharing standards support automation, but generating structured reports is time-consuming and manual; the section motivates automatic extraction from diverse unstructured sources.Source: §3.1 · pp. 2–3

3.2 Collecting and Structuring CTI Concepts

• The knowledge graph represents CTI as RDF triples linking domain concepts such as malware, operating systems, locations, and attack patterns.Its ten relations include targets, uses, exploits, and variantOf; triples can capture both low-level indicators and higher-level intelligence.Source: §3.2 · pp. 3–4, Figure 3 · p. 4

3.3 Attack Patterns

• Attack patterns describe an attacker’s methods for achieving tactical goals, while ATT&CK enumerates common patterns, including 66 techniques for mobile platforms.Unlike potentially short-lived IoCs, TTPs provide long-term intelligence for threat analysis and can support evaluation of security measures and defense testing.Source: §3.3 · p. 4

4.1 Dataset Collection

• LADDER’s corpus covers 36 Android malware from 2015–2022; annotations include nested attack patterns and context-sensitive entity classes.Reports were authored by security analysts at organizations including McAfee, Symantec, and Kaspersky.Source: §4.1 · p. 4

4.2 Information Extraction from CTI for Threat Intelligence Graph

• A crawler collected over 12,000 open-access CTI reports published from 2010–2021; transformer models extract contextual entities, while pattern matching extracts IoCs.The entity model covers Malware, AttackPattern, Application, OS, Organization, Person, Time, and Location.Source: §4.2 · pp. 4–5

4.3 Attack Pattern Extraction

• The attack-pattern sentence classifier and phrase extractor identify contiguous attack descriptions, then map each phrase to the nearest ATT&CK technique using title and description similarity when it meets a threshold.The mapping considers techniques, not sub-techniques.Source: §4.3 · pp. 5–6
• Standardized ATT&CK mapping fixes the pattern classes across reports, and the platform-agnostic extractor can transfer beyond mobile by adapting the target platform’s mapping list.The paper demonstrates extraction from enterprise CTI reports.Source: §4.3 · p. 6

4.4 Adding Relationship to Concepts

• A transformer relation classifier uses entity-position markers and ontology-valid entity pairs to predict relations, after which extracted triples form the threat-intelligence graph.The ontology excludes invalid pair types, such as Application and Time.Source: §4.4 · p. 6

4.5 Querying LADDER

• For graph queries, LADDER predicts a missing tail entity with a scoring function and uses TuckER link prediction to rank candidates.The graph represents facts as head–relation–tail triples.Source: §4.5 · p. 6

5.1 Information Extraction

• The information-extraction datasets were split by malware to avoid repeated reports, using 26 of 36 malware for training and five each for validation and testing.The splits contained 104 training, 21 validation, and 25 test CTI reports.Source: §5.1 · p. 7

5.1.1 Entity Extraction.

• 78.98% average F1 for XLM-RoBERTa-large on named entity extraction; ThreatActor is the most challenging class because it requires contextual interpretation and has limited training examples.Application and Organization overlap in some cases, while OS and Location yield better results with less variation in form.Source: §5.1.1 · p. 7

5.1.2 Attack Pattern Extraction.

• 89.62% average F1 for RoBERTa-large on relevant-sentence classification and 90.64% average F1 for RoBERTa-large on attack phrase extraction.All sentence-classification models exceeded 85% F1; mapping parameters were tuned using 80 manually mapped attack-pattern descriptions.Source: §5.1.2 · p. 7, Table 3 · p. 7, Table 4 · p. 7

5.2 Relation Extraction

• 92.62% average F1 for BERT-large on relation extraction; performance declines on noisier text than the manually cleaned test dataset.Attack patterns are excluded from relation extraction because they use one relation type, Malware uses AttackPattern, which can be inferred once the malware is identified.Source: §5.2 · pp. 7–8, Table 5 · p. 7

5.3 Threat Intelligence Knowledge Graph

• The trained models and heuristics extract concepts, classify ontology-defined relations, map attack patterns to MITRE IDs, and associate them with the malware discussed in each report.The most frequently mentioned malware is used when a report discusses more than one.Source: §5.3 · p. 8

5.4 Inferring Entities

• The study tested tail-entity prediction on 25% and 40% triple holdouts, comparing remaining annotated triples (KG1) with LADDER-generated triples from 12,000 documents (KG2).TuckER was trained for 1,000 iterations using 50 embedding dimensions and a mini-batch size of 64.Source: §5.4 · p. 8
• 16.4% Hits@10 separated KG1 and KG2 on TestSet2, while KG2 performed similarly across both test sets.TestSet1 performed similarly with the different training datasets across all Hits@n values.Source: Table 7 · p. 8, §5.4 · p. 8
• AttackPattern prediction was most promising with 66 unique patterns, whereas other classes had many more possible tail entities.The authors infer that malware with similar properties may exhibit similar attack patterns.Source: Table 8 · p. 8, §5.4 · p. 8

5.5 Comparison with state-of-the-art for TTP Classifier

• MITRE-listed patterns had the best F1; TTPClassifier had better recall, while TTPDrill and AttackKG had similar F1 and TTPDrill more false positives.The comparison used annotations from five malware reports totaling 9,360 tokens.Source: Table 9 · p. 9, §5.5 · p. 9

5.6 Error Analysis

• Most false negatives arose in the first step and most false positives in the last; aggregating reports for the same malware may reduce false negatives.Errors in an earlier stage may influence later stages, while repeated descriptions can allow missed patterns to be captured elsewhere.Source: §5.6 · p. 9

6.1 Attack Pattern Extraction and Trend Analysis

• LADDER extracted 3,159 attack patterns from reports on 433 malware instances, enabling trend analysis for 2015–2021, but the trends reflect analyzed CTI sources, not malware in the wild.The study also reports rising T1417 usage peaking in 2021, declining T1626, and an almost flat T1582 trend.Source: §6.1 · p. 10, Figure 5 · p. 10

6.2 Threat Hunting

• Attack patterns can help correlate CTI with variable kernel logs, but extracting patterns from logs is outside this study’s scope.The authors report Sigma rules for 60 of 193 enterprise MITRE patterns on Linux, or 31% coverage.Source: §6.2 · pp. 10–11

6.3 Attack Pattern Prediction

• Top-10 predictions correctly identified 9 Anubis and 7 Flubot attack patterns; top-15 predictions identified 12 for each malware.The authors also describe the predictions as useful for inferring future patterns and filling gaps in knowledge graphs.Source: Table 12 · p. 11, §6.3 · p. 11

6.4 Identifying Similar Malware and APT Groups

• A Jaccard comparison of neighboring knowledge-graph nodes identified TeaBot as FluBot’s most similar malware, with shared and distinct targets and behaviors.The analysis also found GREF and Ke3chang among the actors most similar to APT15; both are associated with APT15 on MITRE.Source: §6.4 · p. 12, Figure 7 · p. 12

TTP Extraction from CTI

• Prior work tracked IoCs or used internal enterprise logs, while LADDER targets tactical intelligence from unstructured CTI reports.The authors note that internal-log approaches are not scalable and that IoC tracking offers little shared information or long-term analysis [38].Source: §7 · p. 12
• Earlier TTP approaches classified kill-chain phases, used dependency rules, sequence tagging, or graph alignment to map patterns to MITRE ATT&CK IDs.LADDER does not rely on predefined static patterns and can discover descriptions not yet included in MITRE; the authors also report fewer false positives than relevant works.Source: §7 · p. 12

Named Entity Extraction and Relation Extraction

• Cybersecurity NER datasets and systems vary in label coverage and reported F1, while relation-extraction studies report scores from 59.4% to 92%.The cited NER datasets include automatically generated labels or narrower entity classes than LADDER’s ontology.Source: §7 · pp. 12–13

Threat Intelligence Knowledge Graph

• Existing cybersecurity knowledge graphs integrate heterogeneous sources, but EXTRACTOR focuses on IoCs and does not include attack-pattern extraction.The related work also notes an open-access CTI graph and periodically updated SEPSES.Source: §7 · p. 13

8 CONCLUSION

• LADDER is designed to infer attack patterns and other threat intelligence for existing and emerging threats.The authors say attack patterns combined with location, application, and OS information provide evidence for analysts assessing emerging threats.Source: §8 · p. 13
• Future work includes temporal analysis, evaluation on a large CTI corpus, and improved integration with system logs.The stated aim is a unified interface for threat intelligence.Source: §8 · p. 13

Threat Intelligence Concepts

• The ontology represents malware as a central concept connected to information such as attack patterns, location, and time.For example, Cerberus is described as stealing banking credentials and targeting banking users in Spain.Source: Threat Intelligence Concepts · p. 15
• The ontology distinguishes entity classes including malware type, application, operating system, organization, person, time, threat actor, location, and IoC.Entity classification can depend on context, such as whether a company name refers to an application or an organization.Source: Threat Intelligence Concepts · p. 15
• Relationships encode malware types, targets, uses, authorship, aliases, indicators, discovery time, exploited vulnerabilities, variants, and other connections.The hasAlias relationship is transitive, so its head and tail entities are interchangeable.Source: Threat Intelligence Concepts · p. 15

• The crawler takes a seed_url and outputs web_text and relevant_urls.Source: Web crawler · p. 16
• The crawler repeats queue processing for N generations.Source: Web crawler · p. 16
• For each URL in the queue, it scrapes the URL and checks the scraped URLs for keywords.Source: Web crawler · p. 16
• When a URL matches a keyword, the algorithm adds its text to web_text and adds the URL to relevant_urls.Source: Web crawler · p. 16

• Table 13 presents regular expressions used to extract IoCs.Source: Table 13 · p. 16

• Table 14 lists attack patterns for Cerberus extracted from CTI sources.Source: Table 14 · p. 17
• The Cerberus attack patterns are mapped to the MITRE ATT&CK framework.Source: Table 14 · p. 17
• Table 14 ranks the extracted patterns in order of occurrence and provides a complete list of MITRE attack techniques.Source: Attack Patterns for trojan horse,… · p. 16, Table 14 · p. 17