PaperlayerView exact revision on arXiv

Source-linked AI summary

From Threat Intelligence to Detection: Knowledge-driven Enrichment and Template-based Rule Grounding for Automated Sigma Rule Generation

Sepehr Ghaffarzadegan, Boubakr Nour, Makan Pourzandi, Mourad Debbabi, Chadi Assi

arXiv:2608.19011v1cs.CRcs.AI

TL;DR

Converting heterogeneous CTI narratives into validated, actionable Sigma rules remains difficult because existing approaches provide limited structured outputs and rule coverage. AUTOSIGMA enriches CTI, grounds generation in retrieved Sigma templates, and iteratively validates rules, achieving higher relevancy than a state-of-the-art solution.

  • Problem

    Converting heterogeneous attack narratives into formal, relevant Sigma rules remains difficult, while existing rule coverage is incomplete and prior approaches require manual interpretation.

  • Method

    AUTOSIGMA enriches sparse CTI with cybersecurity knowledge, retrieves validated Sigma templates, generates tailored rules, and iteratively validates them with an LLM-as-a-Judge.

  • Results

    80.4% and 77.7% relevancy scores for AUTOSIGMA’s cloud-based and local-based versions exceeded the state-of-the-art solution’s 76.4%.

  • Takeaways & Limitations

    AUTOSIGMA streamlines operationalizing threat intelligence for security operations without requiring expert rule engineers for every new threat report.

  • Takeaways & Limitations

    AUTOSIGMA’s multi-stage enrichment, retrieval, and iterative validation pipeline has non-trivial latency, limiting suitability for time-sensitive environments.

Abstract

from arXiv · show

Mechanisms for dynamically converting cyber threat intelligence (CTI) into actionable detection capabilities are necessary due to the rapid evolution of Advanced Persistent Threats (APTs). Sigma rules are an essential part of contemporary threat detection workflows because they offer a platform-independent framework for expressing detection logic that can be converted into particular queries across SIEM systems. Conventional techniques for manually crafting Sigma rules are prone to mistakes, and necessitate extensive knowledge, which restricts their scalability. Although there are open-source and industry-maintained Sigma rule repositories, they often fail to keep pace with emerging threats and require frequent customization to fit diverse operational environments. This emphasizes the necessity of dynamic rule generation that is adapted to evolving attack techniques as well as particular use cases. In this work, we design AUTOSIGMA, an automated solution for transforming unstructured CTI reports into relevant Sigma rules. Rather than relying solely on language models, AUTOSIGMA leverages a structured knowledge base to enrich partial inputs, matches the enriched content against a repository of existing Sigma rules, and then employs an LLM-as-a-Judge mechanism to iteratively validate the rules. By combining knowledge-driven enrichment, template-based rule grounding, and a multi-stage solution, AUTOSIGMA enables accurate, context-aware, and relevant rule generation. Evaluations across multiple real-world APT reports and multiple security blogs demonstrate that AUTOSIGMA outperforms alternative solutions and LLM models in rule validity, rule relevancy, MITRE ATT&CK technique coverage, and robustness to input quality. AUTOSIGMA's Demo: https://youtu.be/iSr6IurQ6BM

I. INTRODUCTION

AUTOSIGMA addresses the gap between unstructured CTI reports and deployable Sigma rules through an end-to-end, knowledge-enriched pipeline. It grounds generation in existing rules and iteratively validates candidates to support accurate, context-aware detection with reduced manual effort.

  • Motivation: CTI reports provide detailed attacker context but are typically unstructured, forcing security teams to manually translate threat descriptions into adaptable Sigma detection logic.This limits scalable conversion of threat intelligence into actionable outputs for diverse operational contexts.
  • AUTOSIGMA: AUTOSIGMA transforms raw narrative CTI into precise, deployable Sigma rules through a unified, fully automated pipeline.The pipeline combines contextual analysis, template-driven retrieval, and generative refinement to reduce manual effort.
  • Knowledge-driven enrichment: External cybersecurity knowledge bases enrich sparse attack descriptions, while decomposition into atomic steps improves the accuracy, granularity, and modularity of generated rules.Enrichment compensates for incomplete intelligence and supports robust rule generation from less detailed inputs.
  • Rule grounding and refinement: Validated Sigma repositories ground generation with similar templates, and a dual LLM-as-a-Judge feedback loop iteratively validates and improves candidate rules.Together, these mechanisms promote syntactic correctness, contextual alignment, consistent formatting, and reduced hallucination and human error.
  • Evaluation: AUTOSIGMA is evaluated on public security blogs and APT41, APT28, and APT29 reports against state-of-the-art solutions and LLM models.The evaluation targets high-fidelity rule generation, accurate threat-behavior capture, and operational SIEM readiness.

II. MOTIVATION AND PROBLEM STATEMENT · A. Motivation · B. Problem Statement

Sigma’s portability makes context-relevant rule creation important, yet heterogeneous CTI and current extraction methods make reliable automation difficult. Existing approaches lack robustness, contextual enrichment, and the structure needed to produce ready-to-use Sigma rules from unstructured reports.

  • A. Motivation: Sigma is a widely adopted, vendor-agnostic YAML format for encoding CTI as detection logic across diverse SIEM platforms.Its portability increases the importance of creating relevant rules for proactive threat hunting and varied operational contexts.
  • A. Motivation: Security teams need new Sigma rules for proactive threat hunting and diverse operational contexts, making relevant rule creation critical.The supplied passage identifies manual crafting as time-consuming, though its statement is truncated.
  • A. Motivation: CTI reports blend attack steps, benign observations, and inconsistent formatting, making automated conversion into Sigma’s structured conditions extremely challenging.The cited figure exemplifies CTI’s unstructured, narrative style through an excerpt from an APT41 threat report.
  • A. Motivation: Prior NLP extraction techniques generally produce limited outputs, such as IoCs or MITRE ATT&CK techniques, that still require extensive manual interpretation.These outputs do not directly resolve the conversion from narrative CTI into complete Sigma detection logic.
  • B. Problem Statement: Many existing methods depend heavily on report quality and structure, producing inconsistent results for unstructured or poorly formatted documents.This input sensitivity limits reliability across heterogeneous CTI reports.
  • B. Problem Statement: Other approaches lack contextual enrichment, omitting additional knowledge sources that could improve the accuracy and relevance of generated rules.The problem is not only extraction, but also the absence of information that augments the report’s content.
  • B. Problem Statement: Extracted insights often lack the granularity and structured format required for direct SIEM integration, forcing teams to refine or complete rules manually.This manual intervention undermines the goal of full automation.
  • B. Problem Statement: Research has not adequately bridged narrative threat intelligence and actionable content because attacks must be formalized, sequenced, and correlated with existing detection patterns.These obstacles have prevented solutions from producing high-quality, ready-to-use Sigma rules from unstructured threat reports.

III. RELATED WORK … 1) Overview:

AUTOSIGMA addresses gaps in CTI knowledge extraction and Sigma rule generation through an end-to-end, knowledge-enriched pipeline. It targets multi-stage APT reports and produces validated, context-rich rules robust to incomplete or variable-quality inputs.

  • III. RELATED WORK: CASIE and OPEN-CYKG extract entities and relations into threat knowledge graphs, while LADDER and SHIELD apply contextual LLM reasoning and refinement.
  • III. RELATED WORK: Existing systems often address extraction, alert explanation, or candidate generation without providing a complete, correctness-assessed pipeline for Sigma rule synthesis.
  • III. RELATED WORK: Unlike one-pass or report-constrained systems, AUTOSIGMA tolerates formatting variation and constructs context-rich attack scenarios beyond the report using known detection patterns.
  • III. RELATED WORK: AUTOSIGMA retrieves relevant validated Sigma templates, enriches them with external intelligence, completes rules with generative LLMs, and iteratively validates syntax and semantics.
  • IV. THREAT MODEL AND ASSUMPTIONS: AUTOSIGMA targets sophisticated multi-stage cyberattacks, especially APTs described in unstructured threat reports and involving tactics such as initial access, lateral movement, and impact.
  • IV. THREAT MODEL AND ASSUMPTIONS: The system assumes reports may contain multiple discrete attack steps, requiring sequences of Sigma rules whose granularity depends on available detail.
  • V. AUTOMATED SIGMA RULES GENERATION: AUTOSIGMA uses a modular four-component pipeline to transform APT reports into high-quality, actionable Sigma rules.
  • A. System Overview: Its overview combines contextual extraction, external enrichment, atomic attack-step decomposition, and semantic matching against a curated repository of validated Sigma rules.

2) Novelty: … 2) TTP and NER Extraction:

AUTOSIGMA combines end-to-end Sigma rule generation with external knowledge enrichment, structured contextual analysis, and LLM-based extraction and refinement. Its pipeline preprocesses unstructured CTI, extracts cybersecurity entities and ATT&CK techniques, and addresses limited labeled data and LLM reliability challenges.

  • 2) Novelty:: AUTOSIGMA automates Sigma rule generation end to end, from raw unstructured threat reports to relevant, deployable rules.It also integrates MITRE ATT&CK and NVD knowledge to enrich attack context and fill gaps in incomplete reports.
  • 3) Design Challenges:: AUTOSIGMA addresses free-form CTI, limited labeled cybersecurity data, and LLM hallucinations or syntactically incorrect outputs as core design challenges.These constraints complicate direct entity and behavior-sequence extraction and limit traditional supervised NER models.
  • 3) Design Challenges:: An LLM-as-a-Judge refinement loop improves the quality of AUTOSIGMA’s final results.The loop is presented as part of the system’s response to LLM output reliability challenges.
  • B. Contextual Analysis: Contextual analysis maps each unstructured CTI document into a representation of preprocessed segments and extracted cybersecurity entities, including NERs, IoCs, and MITRE ATT&CK TTPs.The formal representation defines documents as D, segments as S, entities as X, and contextual space as C ⊆ 2^S × 2^X.
  • B. Contextual Analysis: The first contextual-analysis component preprocesses raw reports to ensure consistency, remove irrelevant content, and isolate information for subsequent analysis.This component extracts meaningful threat intelligence despite variation in report structure and format.
  • 1) Text Preprocessing:: Text preprocessing extracts report text, cleans irrelevant data, normalizes content, and segments inputs into manageable pieces for language models.Segment-by-segment processing provides context while optimizing LLM performance compared with feeding one large block.
  • 2) TTP and NER Extraction:: AUTOSIGMA uses an LLM for NER extraction because it generalizes from limited data and handles complex contexts better than the experimented BERT-based model.Existing approaches such as PELAT, SMET, and Adema often rely on annotated datasets and may not adapt well to diverse unstructured reports.
  • 2) TTP and NER Extraction:: From an APT41 example, AUTOSIGMA identifies the threat actor, exploited vulnerability, and malware, then infers three corresponding MITRE ATT&CK techniques.The extracted entities are APT41, CVE-2019-19781, and Cobalt Strike; inferred techniques include T1190, T1071.001, and T1021.

C. Attack Understanding … 3) Attack Step Decomposition:

AUTOSIGMA transforms fragmented threat-report information into structured attack scenarios, enriches them with external cybersecurity knowledge, and decomposes complex scenarios into discrete steps for focused Sigma rule generation. This attack-understanding process improves the context, modularity, and scalability of downstream detection generation.

  • 2) Attack Enrichment:: AUTOSIGMA queries multiple cybersecurity knowledge bases to add context beyond basic entity and technique extraction.This enrichment addresses missing context, depth, or completeness in vague or incomplete reports.
  • 1) Attack Scenario Extraction:: Structured attack scenarios make extracted intelligence suitable for subsequent enrichment and detection-rule generation.
  • 2) Attack Enrichment:: CVE enrichment retrieves vulnerability descriptions, CVSS scores, severity ratings, affected software, and versions from external repositories.The repository used in this work contains over 277,000 CVE records.
  • 2) Attack Enrichment:: Threat-actor enrichment queries MITRE ATT&CK for related tools, malware, and software used by an actor.
  • 3) Attack Step Decomposition:: AUTOSIGMA decomposes each enriched scenario into discrete attack steps because a single scenario may contain multiple behaviors or subgoals.The decomposition aligns individual steps with existing detection logic more effectively.
  • 3) Attack Step Decomposition:: Breaking complex threats into distinct components enables multiple focused Sigma-rule flows, improving rule precision, modularity, and scalability.This contrasts with approaches that encapsulate an entire attack scenario in one rule.

D. Template Matching … 1) Rule Generator:

AUTOSIGMA matches enriched attack steps with semantically similar SigmaHQ rules in a shared BERT-based vector space, then uses the retrieved templates to generate tailored, executable detection rules. Its retrieval and refinement process emphasizes current community-curated patterns, syntactic validity, attack alignment, IoC coverage, and adversary behavior.

  • D. Template Matching: Template matching retrieves a non-empty, finite set of SigmaHQ rules whose BERTScore similarity to each enriched attack step exceeds threshold τ.The retrieval function operates over the SigmaHQ rule set and returns candidate templates from a shared vector space.
  • D. Template Matching: AUTOSIGMA embeds attack steps and known valid Sigma rules into a common vector space to retrieve templates for new rule generation.This step introduces existing detection-rule knowledge into the generation process.
  • 1) Attack Embedding Construction:: Each attack description is converted into a high-dimensional BERT-based embedding that captures its semantic meaning for comparison with Sigma rule embeddings.The embedding construction enables semantic comparison between extracted attack descriptions and existing rules.
  • 2) Rule Embedding Construction:: AUTOSIGMA indexes an up-to-date SigmaHQ rule collection using a graph-based representation to accelerate subsequent rule retrieval.SigmaHQ organizes rules hierarchically into categories and subcategories, supporting repository organization for search.
  • 3) Similar Rule Discovery:: The indexed and embedded repository enables rapid searches against current, community-curated detection patterns relevant to each attack step.Keeping the repository current helps align generated rules with the latest known detection techniques.
  • 3) Similar Rule Discovery:: BERTScore is used because it captures word matches, meaning, fluency, and order when identifying contextually similar rules.The approach is described as more effective than traditional n-gram-based metrics such as BLEU for scanning the rule corpus.
  • E. Rule Generation: The final generation phase refines candidate templates into high-fidelity Sigma rules tailored to specific attack scenarios.This phase transforms enriched attack steps into executable detection rules.
  • 1) Rule Generator:: Generated rules must reflect reported attack techniques, cover relevant IoCs and adversary behaviors, and satisfy validity and alignment conditions.The formal refinement function outputs finalized rules only when the validity and attack-alignment predicate is satisfied.

2) LLM-as-a-Judge: · VI. IMPLEMENTATION · A. LLM Models

AUTOSIGMA uses a generator–validator loop in which one LLM creates template-grounded Sigma rules and another evaluates and improves them for accuracy, logic, and format. The implementation uses Python on specified hardware and evaluates local and cloud LLM configurations with low temperature and bounded outputs.

  • 2) LLM-as-a-Judge:: LLM-1 generates an initial Sigma rule by adapting a retrieved template to the attack-step context and scenario.It fills or modifies fields such as the title, description, detection filters, and tags.
  • 2) LLM-as-a-Judge:: LLM-2 evaluates whether the rule detects the described technique and behaviors, uses valid detection logic, and follows YAML and Sigma specifications.Its assessment covers accuracy, sensible conditions, field names, gaps, mistakes, and formatting.
  • 2) LLM-as-a-Judge:: The validator returns a quantitative score or verdict plus structured feedback, enabling LLM-1 to revise overly broad conditions, missing fields, or formatting errors.This generator–validator interaction forms an adversarial learning loop.
  • 2) LLM-as-a-Judge:: AUTOSIGMA repeats drafting, validation, scoring, and feedback-driven revision until each rule is syntactically valid and semantically robust for detection.Revisions can tighten detection conditions, add missing IoCs, or correct formatting errors.
  • 2) LLM-as-a-Judge:: The LLM-as-a-Judge mechanism improves generated-output quality but does not handle hallucinations, which are addressed through low temperatures and fine-tuned LLMs.The authors observed only 1–2 such cases.
  • VI. IMPLEMENTATION: AUTOSIGMA’s proof of concept was implemented in Python 3 and evaluated on an Intel Xeon E312xx system with 6 vCPUs, 2.69 GHz, 32 GB RAM, and Ubuntu 20.04.Further proof-of-concept details are provided in Appendix C.
  • A. LLM Models: The study distinguishes cloud-based and locally hosted LLMs, using Lily-Cybersecurity-7Bv0.25 locally and ChatGPT-4o-mini and Llama-3.3-70b through APIs.Both configurations used temperature 0.1 and a maximum token length of 512; local execution supports privacy but requires substantial computing resources, whereas cloud access reduces resource demands but raises cost and confidentiality concerns.

B. Datasets

The evaluation uses threat reports from APTNotes, focusing on APT41, APT28, and APT29 because no comprehensive dataset is available. It also includes the LLMCLOUDHUNTER dataset of 20 cloud-security blogs.

  • Threat Reports: The threat-report dataset comes from APTNotes, which aggregates intelligence from vendors including Mandiant and CrowdStrike.The paper notes that AUTOSIGMA can work with any threat report despite evaluating only selected groups.
  • Threat Reports: The evaluation focuses on reports concerning APT41, APT28, and APT29 because a comprehensive dataset is unavailable.Appendix D summarizes the CTI reports used in the evaluation.
  • Additional Dataset: 20 cloud-security blogs from the LLMCLOUDHUNTER dataset are also used to evaluate different solutions.The passage introduces this dataset as an additional evaluation resource.

VII. EVALUATION

AUTOSIGMA consistently outperforms baseline and alternative configurations in rule validity, semantic relevancy, IoC and MITRE ATT&CK coverage, and detection breadth. Its knowledge-driven decomposition and iterative evaluation produce context-tailored rules while reducing hallucination risk through automated critique and refinement.

  • Rule validity: 100% of cloud-based AUTOSIGMA’s generated rules are syntactically valid and directly convertible into SIEM queries.The local configuration reaches 76.2% validity, while its larger output volume still yields more valid rules than LLMCLOUDHUNTER.
  • Rule relevancy: 80.4% and 77.7% semantic similarity scores for cloud-based and local AUTOSIGMA exceed the state-of-the-art solution’s 76.4%.All evaluated models generate rules semantically aligned with their corresponding CTI reports.
  • Threat coverage: 97.4% and 84.4% partially covered rules and 83.0% fully covered rules exceed LLMCLOUDHUNTER’s 80.9% and 74.0% coverage results.AUTOSIGMA’s attack scenario extraction and decomposition also produce an average of 12 newly discovered rules versus 2 for LLMCLOUDHUNTER.
  • IoC coverage: 95.6% and 95.2% of reported IoCs are covered on average by cloud-based AUTOSIGMA configurations.The evaluation manually checks whether each extracted IoC appears in at least one generated Sigma rule.
  • Threat coverage: 91.2% and 90.8% MITRE ATT&CK technique coverage for cloud-based AUTOSIGMA OpenAI and Llama exceed ChatGPT-off-the-shelf’s 20.2%.The local LLM configuration achieves 70.8% coverage, attributed to AUTOSIGMA’s contextual analysis stage.
  • Threat coverage: AUTOSIGMA’s cloud-based rules introduce 91 new MITRE techniques across five APT reports, averaging 18 additional techniques per report, versus 12 for ChatGPT-off-the-shelf.The rules retain most techniques explicitly mentioned in the reports while inferring additional techniques.

VIII. DISCUSSION

The discussion identifies AUTOSIGMA’s strong performance in generating high-quality, deployable Sigma rules while emphasizing robustness and execution time as areas for improvement. It proposes combining generative reasoning with rule-based validation and notes that the multi-stage pipeline can introduce non-trivial latency, particularly outside offline threat hunting.

  • Design assessment: AUTOSIGMA demonstrates strong performance in generating high-quality, deployable Sigma rules, but its design requires further critical assessment.The discussion distinguishes this assessment from numerical evaluation alone.
  • Robustness: Combining generative reasoning with structured grammars, detection logic templates, and rule-based validation could enhance robustness.The proposed direction targets hallucinations and stronger alignment with security semantics through guided refinement.
  • Robustness: Evaluating intermediate rule candidates with static analyzers or predefined criteria, in addition to another model, could reduce hallucinations and enforce stronger security-semantic alignment.This guided refinement process supplements model-based validation with explicit checks.
  • Execution time: AUTOSIGMA’s enrichment, template retrieval, and iterative validation stages can make overall execution latency non-trivial.The discussion considers this latency acceptable for offline threat hunting use cases.

IX. CONCLUSION

AUTOSIGMA is a fully automated framework that transforms unstructured CTI reports into actionable Sigma rules through knowledge-based enrichment, template-guided generation, and a structured multi-phase pipeline. It produces contextually relevant and semantically accurate rules for threat-hunting activities.

  • Framework contribution: AUTOSIGMA automatically generates actionable Sigma rules from unstructured CTI reports.The framework is designed for fully automated rule generation.
  • Framework contribution: Its multi-phase pipeline integrates knowledge-based enrichment with template-guided rule generation.The approach leverages external threat knowledge and aligns generated rules with existing detection templates.
  • Framework contribution: The resulting rules are contextually relevant and semantically accurate for hunting activities.The framework also enforces rule correctness within its structured generation process.

APPENDIX A EXAMPLE OF OUTPUTS … APPENDIX D CTI REPORTS SUMMARY

The appendices illustrate AUTOSIGMA’s enrichment, prompting, validation, interface, and evaluation-input design. Together, they show how reports are structured into attack steps, transformed into reviewed Sigma rules, and tested on diverse CTI sources.

  • APPENDIX A EXAMPLE OF OUTPUTS: AUTOSIGMA enriches attack scenarios by extracting entities such as CVEs and threat actors before decomposing the scenario into actionable attack steps.The example separates vulnerability identification, exploitation, and malicious-payload downloading for independent rule generation.
  • APPENDIX A EXAMPLE OF OUTPUTS: The decomposed example identifies Citrix ADC vulnerability discovery as Initial Access, exploitation as a subsequent step, and FTP payload downloading as Execution.Each step is prepared for separate handling by the rule generation module.
  • APPENDIX B LLM PROMPTING: Basic extraction prompts were inadequate for cybersecurity tasks, so AUTOSIGMA uses explicit instructions, relevant examples, and defined output formats.The structured prompt-engineering approach is illustrated in Fig. 9.
  • APPENDIX B LLM PROMPTING: AUTOSIGMA’s LLM-as-a-Judge loop iteratively evaluates generated Sigma rules for correctness, completeness, and syntax compliance.A separate validator assesses the generator’s initial rule, and scores below 8/10 trigger further processing.
  • APPENDIX C PROOF OF CONCEPT (POC): The proof-of-concept interface lets users choose local or cloud LLMs and upload or paste threat reports for analysis.This input area initiates the end-to-end AUTOSIGMA workflow.
  • APPENDIX C PROOF OF CONCEPT (POC): The interface visualizes extracted entities, ATT&CK techniques and tactics, attack tests, similar SigmaHQ rules, contextual information, and reconstructed attack scenarios.These views support inspection of both knowledge extraction and similar-rule discovery.
  • APPENDIX D CTI REPORTS SUMMARY: The evaluation uses CTI reports from multiple security vendors that vary in technical depth, length, structure, and included information types.Reports combine executive summaries, technical analyses, command-line snippets, screenshots, figures, and narrative descriptions of attacker behavior.

APPENDIX E LOCAL LLMS VERSUS CLOUD LLMS · APPENDIX F LLM SPECIFICATIONS · APPENDIX G AVERAGE EVALUATION ACROSS MULTIPLE APTS

AUTOSIGMA’s results reflect complementary strengths across local and cloud LLMs: local models support upstream analysis, while cloud models perform better in final Sigma-rule generation. Across multiple APTs, AUTOSIGMA substantially improves IoC coverage in both deployment settings, using fixed and reproducible evaluation configurations.

  • APPENDIX E LOCAL LLMS VERSUS CLOUD LLMS: AUTOSIGMA is evaluated as an end-to-end pipeline, so observed cloud–local performance gaps reflect stage-specific model capabilities rather than pipeline design shortcomings.The local cybersecurity-specialized model performed strongly during early analytical stages, while cloud models offered distinct strengths elsewhere.
  • APPENDIX E LOCAL LLMS VERSUS CLOUD LLMS: Cloud models outperform local models in final Sigma-rule generation because those stages demand generative capacity, instruction adherence, and formatting consistency.Validation and scoring occur after generation, making the quality of initial drafts decisive for downstream outcomes.
  • APPENDIX F LLM SPECIFICATIONS: Three LLMs were tested across cloud and local configurations with temperature 0.1 and a maximum generation length of 512 tokens for deterministic, reproducible outputs.The models varied in architecture, parameter scale, and domain specialization.
  • APPENDIX F LLM SPECIFICATIONS: GPT-4o-mini represents a commercial closed-weight, general-purpose model for reasoning, summarization, and text generation.Llama 3.3-70B provides an open-weight, 70-billion-parameter comparison across model families under identical prompting conditions.
  • APPENDIX F LLM SPECIFICATIONS: Lily-Cybersecurity-7B-v0.2 is a Mistral-7B variant fine-tuned for cybersecurity using 22,000 hand-crafted cybersecurity and hacking-related data pairs.The dataset was additionally processed by an LLM to provide context, personality, and styling to outputs.
  • APPENDIX G AVERAGE EVALUATION ACROSS MULTIPLE APTS: 90.4% average IoC coverage is achieved with AUTOSIGMA and cloud LLMs, up from 20.1% for the ChatGPT-off-the-shelf baseline across APTs.With local LLMs, average IoC coverage rises from 20.1% to 60.7%, an increase of 40.6 percentage points.
  • APPENDIX G AVERAGE EVALUATION ACROSS MULTIPLE APTS: AUTOSIGMA consistently improves IoC coverage for APT41, APT28, and APT29 across both cloud-based and local LLM settings.The results also align with MITRE ATT&CK coverage findings, broadening technique coverage into more comprehensive Sigma rule sets.
Loading 2608.19011v1…