PaperlayerView exact revision on arXiv

Source-linked AI summary

Merging Cyber Threat Intelligence Through Retrieval-Augmented Generation and Small Language Models for Rich Threat Representation

Nicola Deidda, Leonardo Regano, Alessandro Sanna, Davide Maiorca, Giorgio Giacinto

arXiv:2609.07280v1cs.CR

TL;DR

Individual CTI artifacts often do not establish attack progression, step feasibility, and observable traces, while analysts must correlate evidence manually. The paper combines RAG with a Small Language Model to produce enriched, tactic-aligned Attack Graphs; across case studies, graph structure was manually verified and description similarity was stable for most cases. The authors note that absent step-level ground truth and proxy-based evaluation limit what can be established about correctness and utility.

  • Problem

    Heterogeneous CTI artifacts are insufficient alone to reconstruct attack progression, step feasibility, and resulting traces, requiring analysts to correlate sources manually.

    Source: §1 · p. 2

  • Method

    A RAG architecture and locally deployable Small Language Model generate a coarse, tactic-aligned Attack Graph with step-level descriptions, pre-conditions, and post-conditions.

    Source: §1 · pp. 2–3, §3.1 · p. 5

  • Results

    The generated graph's nodes and edges were manually verified for correctness, and BERTScore similarity to references was stable across runs for most case studies.

    Source: §5 · p. 13

  • Takeaways & Limitations

    The enriched graph represents coarse attack progression and exposes step feasibility constraints and observable consequences relevant to prevention, detection, and response.

    Source: §5 · p. 13, §7 · p. 16

    Without step-level ground truth, manual verification may be subjective, and BERTScore does not fully measure graph correctness, actionability, or analyst utility.

    Source: §5.1 · p. 14

Abstract

from arXiv · show

Modern cybersecurity operations rely on CTI collected from heterogeneous sources, including semi-structured threat representations, IoCs, and narrative technical reports. However, these artifacts are often insufficient in isolation to reconstruct how an attack unfolds, under which conditions each step is feasible, and which traces it leaves behind. In practice, analysts must manually correlate partial evidence scattered across multiple and only partially structured sources, delaying the design of effective prevention, detection, and response actions. To address this gap, we propose an automated pipeline that derives an actionable representation of a cyberattack from heterogeneous CTI sources. The pipeline combines a RAG architecture with a locally deployable SLM, used to consolidate such evidence and infer missing operational details. Starting from a semi-structured threat representation and auxiliary CTI documents, the pipeline produces an enriched Attack Graph that captures a coarse, tactic-aligned progression of the attack and annotates each step with explicit pre-conditions and post-conditions, and an enriched description. This representation supports prevention by exposing execution requirements, detection by highlighting observable traces, and response by clarifying the temporal progression of the attack. Then, due to the lack of validated datasets with ground-truth information on the temporal evolution of real-world attacks, we test the complete pipeline on 10 real-world case studies spanning multiple threat types, including backdoors and staged downloaders delivered via phishing. A manual assessment across 10 real-world case studies provides initial evidence that the generated graphs are consistent with expected attack progressions, indicating that the proposed approach can support analysts by consolidating dispersed CTI evidence into a structured and actionable view of attacks.

1 Introduction

CTI artifacts rarely reveal attack progression, step feasibility, and resulting traces together, leaving analysts to reconstruct these manually. The paper proposes a RAG/SLM pipeline for a coarse, tactic-aligned Attack Graph enriched with step-level details and evaluates it across 10 case studies.

1 Introduction

  • IoCs and standards such as STIX support observation and intelligence sharing, but no single artifact supplies a complete operational view of an attack.Analysts manually correlate heterogeneous sources to reconstruct execution flow and step conditions, a costly and error-prone task.Source: §1 · p. 2
  • Unlike entity-centric LLM and knowledge-graph representations, the paper focuses on execution flow and step-level operational semantics.The stated aim is to reconstruct attack progression and enrich each step with information useful to analysts.Source: §1 · p. 2
  • The study asks whether NLP and AI can derive coherent temporal Attack Graphs, enrich their steps, and generalize across threats, architectures, and TTPs.Source: §1 · p. 2
  • A RAG architecture and Small Language Model derive a coarse, tactic-aligned attack progression represented by Milestones and atomic Attack Steps.Milestones are tactic-aligned execution checkpoints; Attack Steps are atomic attacker actions.Source: §1 · pp. 2–3
  • Each Attack Step receives grounded descriptions, pre-conditions, and post-conditions, supporting feasibility assessment, prevention, detection, and investigation.Pre-conditions expose feasibility constraints; post-conditions identify observable consequences.Source: §1 · p. 3
  • The pipeline is evaluated on 10 real-world case studies because public datasets with step-level pre- and post-conditions are unavailable.With limited validated ground-truth data, the authors validate components individually and assess complete executions across the cases.Source: §1 · p. 3

2 Background

The background describes retrieval methods for grounding generation in predominantly textual CTI, then outlines chunking and hybrid search choices that shape evidence retrieval.

2 Background

  • The pipeline relies on retrieval-centric methods to ground generation in predominantly textual CTI, with effectiveness depending on document chunking and retrieval quality.Source: §2 · p. 3

2.1 Retrieval-Augmented Generation

  • RAG retrieves relevant text chunks from a document store and conditions generation on them, consolidating dispersed CTI evidence for step descriptions and conditions.Its effectiveness is sensitive to chunking, index maintenance, and retrieval noise.Source: §2.1 · pp. 3–4
  • Chunking divides documents into retrieval units: small chunks can separate related content, while large chunks can mix relevant and irrelevant information.Semantic boundaries and overlap can help preserve coherent evidence and mitigate boundary effects.Source: §2.1 · p. 4

2.2 Hybrid Search

  • Hybrid search combines lexical matching for exact technical artifacts with embedding-based semantic search for paraphrases and vocabulary mismatches.The two retrieval scores can be combined to rank candidate chunks.Source: §2.2 · p. 4

3 Methodology

The local-first pipeline combines a RAG architecture and on-device SLM to turn semi-structured threat representations and heterogeneous CTI documents into an enriched, tactic-aligned Attack Graph. It grounds step descriptions and infers pre-conditions and post-conditions from retrieved evidence, while representing progression coarsely rather than as a fully ordered execution trace.

3 Methodology

  • The pipeline uses an on-device SLM to consolidate heterogeneous CTI evidence into an enriched Attack Graph, targeting deployments with sensitive data and limited computational resources.It starts from a semi-structured threat representation such as STIX and auxiliary CTI documents.Source: §3 · p. 4

3.1 Pipeline Overview

  • For each technique instance derived from the threat representation, the graph records a tactic-aligned milestone and an attack-step description with evidence-inferred pre-conditions and post-conditions.The pipeline normalizes documents, chunks and indexes them, retrieves relevant chunks, assembles bounded context, and queries the SLM for each step.Source: §3.1 · pp. 4–5, Figure 1 · p. 5

3.2 User-provided Threat Representation and Heterogeneous Documents

  • The pipeline assumes valid, standard-compliant STIX, sanitized auxiliary documents, and trustworthy expert-produced sources; knowledge-base poisoning is out of scope.MISP may be supported with limited engineering effort.Source: §3.2 · pp. 5–6
  • From STIX, the pipeline extracts malware and attack-pattern metadata, uses kill-chain phases as a coarse sequencing reference, and normalizes and chunks other documents with source metadata for traceability.It explicitly parses JSON, XML, YAML, and CSV; other formats are processed as text.Source: §3.2 · p. 6

3.3 RAG Architecture

  • For each attack step, queries combine the malware name, MITRE technique name, and original description; a hybrid retriever ranks chunks using BM25 and embedding similarity before deduplication and bounded-context assembly.FAISS performs vector retrieval using an L2 index, and ensemble weights control the lexical and vector contributions.Source: §3.3 · p. 6

3.4 Enrichment Process and Attack Graph

  • The graph groups attack steps under tactic-aligned milestones ordered by kill-chain phase, but steps within a milestone remain unordered unless the input provides finer sequencing.Edges represent tactic-level progression and step membership, not prerequisites or causal relations between techniques.Source: §3.4 · pp. 6–7
  • For each step, the SLM produces a context-grounded description, feasibility pre-conditions, and observable post-conditions; the example notes that sparse or high-level evidence may yield generic statements.In the service-creation example, pre-conditions include OS and access constraints, while post-conditions describe artifacts and effects relevant to investigation.Source: §3.4 · pp. 7–9
  • The Goofy Guineapig example used 172 chunks from a 28-page behavior report, an IoC CSV, and five YAML files; its graph depicts required actions between pivot points, while enriched descriptions integrate details from the document pool.The example labels steps with corresponding TTPs; moving from the first to the second pivot point requires successful execution of steps a1 to a12.Source: §3.4 · pp. 8–9

4 Experimental Results

The experiments configure retrieval and preprocessing, then evaluate the pipeline on ten NCSC malware case studies using manual checks of graph steps and enrichments.

4 Experimental Results

  • Retrieval experiments ran on Ubuntu 22.04.5 with 250 GB RAM and an NVIDIA RTX 5000 Ada Generation 32GB GPU.Case-study experiments were excluded from this machine setup.Source: §4 · p. 9

4.2 RAG Hyperparameters

  • On SQuAD-derived retrieval checks, selected defaults were kbm25 = 50, kvdb = 30, and equal ensemble weights (0.5, 0.5).Because SQuAD is not CTI-specific, the settings are practical defaults rather than task-optimal CTI hyperparameters.Source: Figure 3 · p. 10, §4.2 · pp. 10–11, Figure 4 · p. 11

4.1 Preprocessing Steps

  • Preprocessing used ClusterSemanticChunker with a 200-token maximum and no overlap, preserving semantically related content in compact retrieved context.all-MiniLM-L6-v2 produced 384-dimensional embeddings for chunking and similarity computations.Source: §4.1 · p. 10

4.3 External Documents and STIX Files

  • The case studies used NCSC analyses and accompanying document pools, with the correctness of NCSC files assumed based on the source’s public reputation.Assessing CTI-feed trustworthiness and quality was outside the proposal’s scope.Source: §4.3 · p. 11
  • Reviewers checked each generated graph’s steps, edges, preconditions, postconditions, and evidence grounding against source documents across ten selected NCSC malware reports.The case studies span different threat families; runs used a commercial laptop with 32 GB RAM and an NVIDIA GeForce RTX 4060 8GB.Source: §4.3 · pp. 11–12, Table 3 · p. 12
  • Case-study enrichment used gemma2:9b locally through Ollama for precondition and postcondition inference and enriched-description assembly.The model was chosen for its lightweight nature; parameters remained configurable.Source: §4.3 · pp. 12–13

5 Discussion

The pipeline produces tactic-aligned attack graphs with enriched steps, while description similarity is stable across most cases and repeated runs. The discussion frames the evaluation as feasibility evidence bounded by proxy metrics, manual checks, and a limited case set.

5 Discussion

  • Generated graphs encode coarse attack progression through tactic-aligned milestones and attack steps, with preconditions exposing feasibility constraints and postconditions supporting detection and investigation.The graph structure and step enrichments were generated with the same prompting strategy across case studies.Source: §5 · p. 13
  • BERTScore F1 distributions showed stable similarity to reference descriptions across repeated runs for most case studies.The references were assembled from external documents; BERTScore does not directly measure graph correctness, grounding, or analyst utility.Source: Figure 5 · p. 13, §5 · p. 13
  • Execution time varied with the number of Attack Steps and retrieved-context size; retrieval misses, noise, conflicting sources, and sparse evidence can weaken enrichments.The authors recommend associating enrichments with provenance and using conservative fallback behavior, such as UNKNOWN when evidence is insufficient.Source: Figure 6 · p. 14, §5 · p. 14

5.1 Threats to Validity

  • Manual graph checks may be subjective, and BERTScore does not fully assess actionability, completeness, faithfulness, or analyst utility.The score is also sensitive to reference wording, and no benchmark with step-level ground truth is available.Source: §5.1 · p. 14
  • Curated document pools reduce ambiguity relative to open-world CTI, while the ten-case evaluation omits broader operational sources and remains feasibility evidence, not proof of general performance.The reported threat-coverage boundary includes multilingual reports, fragmented social media, and organization-internal intelligence.Source: §5.1 · p. 15

6 Related Works

The pipeline combines RAG, a local SLM, and an Attack Graph with explicit pre- and post-conditions to represent coarse attack progression. Unlike entity-centric knowledge graphs or TTP extraction, it emphasizes execution flow and operational details.

6 Related Works

  • The authors combine RAG, a local SLM, and an Attack Graph with explicit pre- and post-conditions to derive an actionable view of coarse attack progression.Source: §6 · p. 15
  • Unlike Hu et al.’s entity–relationship knowledge graph, the revised Attack Graph emphasizes coarse progression and operational details such as step pre- and post-conditions.The pipeline also avoids fine-tuning, which the authors describe as resource-intensive.Source: §6 · p. 15
  • Unlike Liu and Zhan’s entity–relationship graph, the pipeline uses an open model rather than a closed-source model to process CTI reports.The authors connect the open model choice with data protection for sensitive data and closed-source intelligence.Source: §6 · p. 15
  • Unlike IntelEx, which extracts TTPs from CTI using LLMs and RAG, this pipeline targets an enriched representation of attack progression and does not rely on commercial LLMs.Source: §6 · p. 16
  • Compared with Zhang et al.’s Attack Graph, this approach supplies step preconditions for feasibility assessment and uses execution flow to identify feasible and infeasible attack stages.The authors link this information to prioritization and post-incident response.Source: §6 · p. 16

7 Conclusion and Future Work

The pipeline turns heterogeneous CTI into a tactic-aligned representation of coarse attack progression, with Milestones, Attack Steps, and step-level operational details. Future work targets reliability and grounding, and plans to use pre- and post-conditions in feasibility assessment, detection, and response.

7 Conclusion and Future Work

  • The pipeline represents coarse attack progression from heterogeneous CTI with Milestones as attack-flow pivots and Attack Steps as attacker actions, each with preconditions, postconditions, and an enriched description.The representation aims to reduce analyst effort in moving from threat reporting to prevention, detection, and response planning.Source: §7 · p. 16
  • The RAG architecture supplies retrieved evidence as context for a locally executing SLM, keeping closed-source intelligence from being shared with third parties.Source: §7 · p. 16
  • Future work targets inference faithfulness and evidence grounding, and plans to use preconditions for feasibility assessment and postconditions for response, detection engineering, and investigation.Source: §7 · p. 16
Loading 2609.07280v1…