PaperlayerView exact revision on arXiv

Source-linked AI summary

From Requirements to Attack Trees: Grounded LLM Agents for Design-Time Security Review

Akash Iyer, Taha Demirkan, Keerthi Koneru, Aaryan Siddharthan, Sheethal Kumar, Ramesh Radhakrishnan

arXiv:2610.03820v1cs.CRcs.AI

TL;DR

Design-time security weaknesses in requirements and architecture receive limited support from post-implementation practices. The paper presents a multi-agent framework that grounds security analysis in PRDs and architecture diagrams, and reports better review quality across its evaluation settings than single-shot and ablation baselines.

  • Problem

    Post-implementation security techniques provide limited support for identifying design-level weaknesses before code is written.

    Source: §1 · p. 1

  • Method

    A multi-agent framework parses architecture diagrams into graphs, derives misuse cases and attack trees, validates findings, and produces mitigations and candidate redesign recommendations.

    Source: §1 · p. 2, §3.2.2 · p. 6, §3.2.6 · p. 8

  • Results

    Architecture-informed, misuse-driven reasoning improves grounding, traceability, actionability, and attack-tree quality compared with single-shot and ablation baselines across the evaluated systems.

    Source: §8 · p. 21

  • Takeaways & Limitations

    Misuse-case intermediates and validation help produce more reviewable findings, while larger threat lists are not necessarily better because STRIDE-style enumeration can increase redundancy.

    Source: §8 · p. 21

    Input artifacts are treated as trusted, leaving prompt-injection defenses such as sanitization, source verification, and sandboxed execution unimplemented.

    Source: §7 · p. 20

Abstract

from arXiv · show

Design-level security weaknesses can arise from requirements, trust assumptions, missing controls, and data flows before implementation begins. Existing security practices often identify these issues after code is written. We present a multi-agent LLM framework for design-time security analysis from product requirement documents and architecture diagrams. The proposed framework parses architecture diagrams into graph representations, generates misuse and failure cases, constructs attack trees, checks governance and compliance gaps, recommends mitigations, assigns enterprise security-domain tags, and produces a candidate revised architecture recommendation for expert review. The framework does not retrieve from Common Weakness Enumeration (CWE) databases at inference time. Instead, it analyzes system behavior, trust boundaries, component interactions, and data-flow assumptions. Misuse cases act as intermediate representations that link findings to system components and attack paths, while a validation and refinement loop filters unsupported findings and improves grounding, traceability, and actionability. We evaluate the framework on a Microsoft reference-labeled threat-modeling example, labeled synthetic PRD--architecture pairs, and two open-ended systems: Berty and Gas Town. The reference-labeled case supports threat-recovery and actionability analysis, while the open-ended cases evaluate validity, noise, traceability, actionability, redundancy, and attack-tree quality. Results show that architecture-informed, misuse-driven reasoning improves review quality compared with single-shot and ablation baselines. Keywords: LLM Multi-Agent Systems, Design-Time Security, Threat Modeling, Vulnerability Discovery, Architecture Diagrams, Security Analysis, Misuse Case Derivation, Attack Trees, Iterative Reasoning, Security Governance.

1. Introduction

Design-time weaknesses can emerge in requirements, trust assumptions, controls, and data flows, but common post-implementation security techniques offer limited support for identifying them before code exists. The paper proposes a grounded multi-agent workflow that links architecture-informed findings to attack paths and review actions.

1. Introduction

  • Design-level weaknesses such as unsafe trust assumptions, missing controls, and insecure data flows can arise before code is written, beyond the reach of post-implementation techniques alone.The paper situates this need within guidance to integrate security during requirements analysis and architecture design.Source: §1 · p. 1
  • Direct LLM prompting can yield unsupported or generic threats and miss coverage gaps, while practical threat modeling also requires security expertise and architecture understanding.Enterprise review spans teams including IAM, network, application, data, cloud, DevSecOps, and governance.Source: §1 · p. 2
  • The framework analyzes PRDs and architecture diagrams through graph representations, misuse and failure cases, attack trees, governance checks, mitigations, domain tags, and candidate architecture redesign.It reasons over system behavior, trust assumptions, component interactions, data flows, and control placement rather than retrieving from CVE databases at inference time.Source: §1 · p. 2
  • Misuse cases connect requirements and architecture elements to adversarial goals, attack paths, and mitigation actions; iterative critique and revision filters unsupported artifacts and improves grounding, traceability, and actionability.The introduction frames evaluation around threat identification, the value of misuse-case intermediates and validation, and support for mitigation planning and redesign.Source: §1 · p. 2
  • The paper contributes a grounded multi-agent framework, a misuse-case-centered workflow, enterprise security-domain tagging, and evaluation against a reference-labeled example and ablation baselines.The contributions claim improved grounding, traceability, actionability, and noise reduction over single-shot baselines.Source: §1 · p. 2

2. Related Work

Prior threat-modeling and architecture-analysis approaches offer structured reasoning or documentation, but often require expert facilitation, structured inputs, or separate workflows. The proposed framework combines architecture grounding, validated misuse-case intermediates, iterative refinement, and enterprise review support.

2. Related Work

  • Architecture-analysis approaches use structured models or extracted data-flow diagrams, rather than end-to-end reasoning across heterogeneous PRDs and architecture diagrams.Prior work includes design-flaw detection and security-rich data-flow diagram extraction Tuma et al. (2020); Schneider & Scandariato (2023); Schneider et al. (2023).Source: §2 · p. 3
  • Manual methods, tools, design-flaw detection, and LLM-assisted threat modeling each address parts of security review, but existing approaches leave gaps in heterogeneous input handling and end-to-end grounding.Manual threat modeling, tools, and prior LLM systems are discussed as distinct approach families.Source: §2 · pp. 3–4

2.1. Key Differentiators of the Proposed Framework

  • The framework parses diagrams into graphs, uses misuse cases to connect requirements, components, attack paths, and mitigations, then iteratively critiques and revises artifacts before downstream analysis.It distinguishes itself from systems that generate threats from unstructured text without validated intermediate artifacts.Source: §2.1 · p. 4
  • Misuse-case-centered reasoning links findings to requirements and architecture elements, while enterprise review integration adds governance, mitigation, security-domain routing, and candidate redesign recommendations.The routing is intended to connect findings with responsible teams and support human-in-the-loop decisions.Source: §2.1 · pp. 4–5

3. Framework and Methodology

The framework aims to support iterative security analysis across software-development lifecycle checkpoints using evolving requirements and architecture artifacts, with findings grounded, linked to relevant elements, paired with mitigations, and routed for review.

3.1. Objective

  • For enterprise review, findings should be grounded in available artifacts, linked to affected requirements or architecture elements, associated with a mitigation, and routed to the responsible security team.The stated objective accounts for approval distributed across multiple security domains.Source: §3.1 · p. 5

3.2. Pipeline Overview

The pipeline transforms PRDs and architecture artifacts into grounded security findings, validates and refines misuse cases and attack trees, then produces governance, mitigation, and redesign outputs. Its stages preserve links to source requirements and architecture elements for review.

3.2. Pipeline Overview

  • The pipeline has four stages: input processing; threat reasoning and refinement; governance and mitigation; and output representation and architecture redesign.The stages are organized across the framework's subsections.Source: §3.2 · p. 6

3.2.1. Requirement and Architecture Processing

  • PRD processing extracts requirements, actors, dependencies, security expectations, compliance constraints, and data assumptions; available diagrams become graphs of components, paths, and trust or deployment zones.Nodes represent components, edges represent communication and data flow, and zones represent boundaries or logical groups.Source: §3.2.1 · p. 6

3.2.2. Misuse and Failure Case Generation

  • Misuse and failure cases capture adversarial or accidental conditions and link affected requirements or architecture elements to actors, entry points, violated properties, conditions, evidence, confidence, and review domains.In architecture-informed mode, each case links to one or more graph nodes, edges, or zones; PRD-only cases link to requirement text.Source: §3.2.2 · p. 6

3.2.3. Planner–Worker Attack Tree Construction

  • A planner derives bounded attack objectives from validated misuse cases, and workers expand each objective into attack paths using the associated cases, PRD context, and architecture graph.Attack-tree nodes use AND, OR, and LEAF types and link back to supporting misuse cases and, when available, architecture elements.Source: §3.2.3 · pp. 6–7

3.2.4. Validation and Iterative Refinement

  • Validation scores misuse cases and attack trees, classifies artifacts for acceptance or revision, and jointly refines related cases and branches for up to three iterations or until convergence.Only affected artifacts are regenerated, while identifiers are preserved to maintain traceability.Source: §3.2.4 · p. 7

3.2.5. Governance, Mitigation, and Domain Tagging

  • Governance analysis identifies policy, compliance, and control gaps; the mitigation engine proposes design-level controls and implementation steps, then critiques recommendations for feasibility and architectural specificity.Priority reflects linked finding severity and affected architecture breadth, including trust boundaries and sensitive data flows.Source: §3.2.5 · pp. 7–8

3.2.6. Output Representation and Architecture Redesign

  • The final report contains validated misuse and failure cases, attack trees, governance findings, mitigations, domain tags, and reasoning traces, alongside a candidate revised architecture for human review.Recommendations may add controls, components, edges, or trust-boundary changes, but are not treated as approved final architectures.Source: §3.2.6 · p. 8

4. Experimental Setup

The evaluation combines controlled synthetic designs, a reference-labeled Microsoft example, and open-ended real-world cases. It compares pipeline configurations and uses ground-truth metrics where labels exist alongside review-quality metrics across all cases.

4. Experimental Setup

  • The evaluation uses controlled flawed synthetic designs, a reference-labeled Microsoft threat model, and open-ended architecture/PRD case studies.Synthetic cases have manually reviewed ground truth; open-ended cases lack complete ground-truth annotations.Source: §4.1 · p. 8, §4 · p. 8

4.1. Input Artifacts

  • Synthetic PRD–architecture pairs contain explicit or implicit security gaps, and experts reviewed their expected findings to create ground-truth threat sets.The injected gaps support testing detection, multi-step reasoning, mitigation generation, and architecture redesign.Source: §4.1 · pp. 8–9
  • The Microsoft IoT Edge case includes existing threats and mitigations, enabling analysis of threat recovery, actionability, and novelty beyond its reference threats.Berty and Gas Town are open-ended cases assessed on validity, noise, traceability, actionability, redundancy, and attack-tree quality rather than gold-set coverage or precision.Source: §4.1 · p. 9

4.2. Pipeline Configurations

  • The full framework is compared with two single-shot baselines and three ablations; configurations share evaluation procedures, and each is run three times per system.Arch-Informed synthetic-case standard deviations for coverage, validity, and actionability were each below 0.03.Source: §4.2 · pp. 9–10

4.3. Evaluation Metrics

  • Ground-truth cases are evaluated with coverage, precision, and novelty, while all cases also receive review-quality measures such as validity, noise, traceability, actionability, and attack-tree quality.Open-ended cases without complete ground truth are not scored on coverage, grounded coverage, or grounded precision.Source: §4.3 · p. 10
  • One enterprise security expert scored attack-tree quality, and inter-rater reliability was not formally assessed.The evaluator scored each system's trees in one blinded session with configurations in random order to reduce within-rater inconsistency.Source: §4.3 · pp. 10–11

5. Results and Ablation Study

Across controlled, reference-labeled, and open-ended settings, Arch-Informed provides the most consistent balance across review-quality measures. Results include reference-threat recovery, expert-judged novel findings, and routed artifacts and candidate redesigns.

5. Results and Ablation Study

  • Across settings, Arch-Informed provides the most consistent, reviewable balance; ablations can improve isolated metrics but often weaken grounded precision or traceability, or increase redundancy.Source: §5 · pp. 11–12, §5.4 · p. 17

5.1. Controlled Synthetic Results

  • EHS Arch-Informed reaches 0.97 raw and grounded coverage, 0.84 grounded precision, 1.00 traceability, 0.91 actionability, and 4.40 attack-tree quality.Source: §5.1 · p. 12, Table 4 · p. 12
  • On OLB, No-MC has slightly higher raw and grounded coverage, but grounded precision is 0.68 versus 0.81 for Arch-Informed.Source: §5.1 · p. 12

5.2. Reference-Labeled Microsoft Threat Model

  • On Microsoft, Arch-Informed achieves 0.83 raw and grounded coverage and the highest validity, traceability, actionability, and attack-tree quality.Source: §5.2 · p. 13, Figure 2 · p. 13, Figure 3 · p. 13
  • Twenty-five percent of Arch-Informed’s validated findings are novel, plausible design-time threats absent from the reference example and judged relevant by a security expert.Source: §5.2 · p. 13
  • Microsoft findings were routed across seven enterprise domains, and the candidate redesign maps validated findings to controls including mutual TLS, RBAC-protected uploads, and encrypted paths.Source: §5.2 · p. 14, Table 6 · p. 15, Table 7 · p. 16

5.3. Open-Ended Real-World Case Studies

  • On Berty, Arch-Informed has 0.92 validity, 0.08 noise, and 0.90 traceability; on Gas Town it reaches 0.97 validity, 0.03 noise, 1.00 traceability, 0.96 actionability, and 4.30 attack-tree quality.Source: §5.3 · p. 14, Table 8 · p. 16

5.4. Cross-Setting Analysis

  • Across settings, architecture grounding supports links between findings and system elements, while misuse-case removal can raise apparent coverage but lower grounded precision; STRIDE often adds redundancy.Source: §5.4 · pp. 16–17

6. Discussion

In the evaluated settings, the framework identifies design-time weaknesses and supports downstream review, while misuse cases connect design artifacts to security reasoning. Its outputs are structured aids for human security review, not approved designs.

RQ1: Can LLM agents identify design-time security weaknesses from PRDs and architecture

  • Across synthetic, Microsoft, Berty, and Gas Town cases, architecture-informed analysis recovers known flaws or maintains high validity, traceability, and actionability, depending on the evaluation setting.The open-ended cases have no complete ground truth; the synthetic cases are controlled detection tests.Source: RQ1: Can LLM agents identify… · pp. 18–19, §6.3 · p. 19
  • Without misuse-case intermediates, apparent coverage can rise in some controlled cases while grounded precision and traceability weaken.The full pipeline uses misuse cases to connect requirements and components to attack paths, mitigations, and domain tags.Source: RQ1: Can LLM agents identify… · p. 19
  • Generated findings support mitigation planning, enterprise-domain routing, and candidate architecture redesign, including concrete Microsoft design changes across seven security domains.The revised architecture is a structured review aid, not an automatically approved design.Source: RQ1: Can LLM agents identify… · p. 19

6.2. Why Misuse Cases Matter

  • Each misuse case links an actor, affected component, violated security property, precondition, impact, and supporting evidence to downstream security reasoning.This structure supports validation, refinement of weak artifacts, and targeted mitigations.Source: §6.2 · p. 19

6.3. Controlled vs. Open-Ended Evaluation

  • Synthetic cases enable controlled measurement, the Microsoft case provides reference-labeled validation, and Berty and Gas Town provide open-ended evidence without complete ground truth.Synthetic recovery scores should be interpreted as controlled detection results, not broad real-world generalization.Source: §6.3 · p. 19

6.4. Practical Use in Enterprise Security Review

  • The framework is designed to accelerate early design analysis and route findings to enterprise security teams, while expert reviewers retain responsibility for assessment.Validity, actionability, novelty, and attack-tree quality were judged by one enterprise security expert using a fixed rubric.Source: §6.4 · p. 19

7. Limitations and Threats to Validity

The evaluation and deployment scope impose boundaries on the framework's evidence and use: seeded tests do not establish broad generalization, and generated redesigns require expert review.

Synthetic evaluation

  • Seeded synthetic systems may inflate recovery metrics, so their results are controlled stress tests rather than evidence of broad generalization.The evaluation also includes a reference-labeled Microsoft threat model and two open-ended systems.Source: §7 · p. 20

Ground-truth completeness

  • Because complete ground-truth threat sets are difficult to define for Berty and Gas Town, the paper reports review-quality metrics there instead of coverage or precision.Reported measures include validity, noise, traceability, actionability, redundancy, and attack-tree quality.Source: §7 · p. 20

Scope of design artifacts

  • The current evaluation uses PRDs and architecture diagrams; the framework does not yet extract architecture directly from codebases or runtime telemetry.Potential future inputs include code repositories, API specifications, deployment manifests, and operational traces.Source: §7 · p. 20

Adversarial Input Artifacts

  • The framework treats input artifacts as trusted and does not yet address prompt injection through crafted PRD text or diagram labels.Input sanitization, source-artifact verification, and sandboxed execution are not yet implemented.Source: §7 · p. 20

Architecture redesign

  • The revised Microsoft architecture is a candidate design artifact, not a deployed implementation, and its diagram does not establish completeness, optimality, or implementation readiness.It demonstrates translation of validated findings into design-change recommendations.Source: §7 · p. 20

Human approval

  • Generated architecture diagrams remain recommendations until human architects and security reviewers validate feasibility, cost, compliance impact, and implementation priority.Source: §7 · p. 20

Evaluator subjectivity

  • A single enterprise security expert assessed judgment-based metrics, and inter-rater reliability was not formally measured.The paper identifies independent evaluators and agreement scores as future work.Source: §7 · p. 20

Refinement loop isolation

  • The ablations do not isolate iterative refinement from misuse-case guidance; a no-refinement ablation retaining misuse cases is planned.Thus, full-pipeline gains reflect the combined effect of architecture grounding, misuse cases, and iterative validation.Source: §7 · p. 20

8. Conclusion

The paper presents a grounded multi-agent framework for design-time security analysis from PRDs and architecture diagrams. Across controlled, reference-labeled, and open-ended cases, architecture-informed, misuse-driven reasoning improves review quality over single-shot and ablation baselines, with human review still required.

8. Conclusion

  • The framework parses architecture into graphs, generates misuse and failure cases and attack trees, validates outputs, identifies governance gaps, recommends mitigations, and proposes revised architectures.Source: §8 · p. 20
  • Across synthetic, Microsoft, Berty, and Gas Town cases, architecture-informed, misuse-driven reasoning improves grounding, traceability, actionability, and attack-tree quality over single-shot and ablation baselines.STRIDE-style enumeration can increase redundancy, while misuse cases and validation help produce more reviewable findings.Source: §8 · p. 21
  • Revised architectures are structured recommendations requiring expert approval, and future evaluation is planned across more systems, LLM backends, and richer inputs.Source: §8 · p. 21

A. Microsoft Redesign and Domain-Routing Details

The appendix describes supporting artifacts for the Microsoft reference-labeled run and defines abbreviations used for its domain tags.

A. Microsoft Redesign and Domain-Routing Details

  • The Microsoft-run appendix includes original and candidate revised architectures, domain-routing assignments, and representative token usage; domain tags abbreviate seven enterprise security areas.IAM means Identity and Access Management; NET, Network Security; APP, Application Security; DATA, Data Security; CLOUD, Cloud Security; DEV, DevSecOps; and GRC, Governance, Risk, and Compliance.Source: A. Microsoft Redesign and… · p. 24

B. Synthetic Architectures

The synthetic architectures model banking and clinical systems with labeled security weaknesses. Editable diagrams also let the framework produce candidate redesigns alongside threat findings and mitigation recommendations.

B. Synthetic Architectures

  • Each OLB and EHS ground-truth set contains 12 labeled threats spanning misuse cases and workflow or service-level threats.The synthetic cases use editable diagrams for candidate redesigned architectures.Source: B. Synthetic Architectures · p. 24
  • OLB models a retail banking platform with missing risk screening, unverified PSP webhooks, and missing resource-level authorization.Its architecture connects web and mobile clients through an API gateway to payment and account services, a ledger, a database, and external payment providers.Source: B. Synthetic Architectures · p. 24, Figure 10 · p. 25
  • EHS models a clinical platform with a read path lacking consent checks, unvalidated stored data, and missing service-to-service authentication.The passage notes that Figure 9 shows a candidate redesigned EHS architecture generated after mitigation recommendations.Source: B. Synthetic Architectures · p. 24

C. Cost Estimation

The cost estimate uses input and output token counts with provider prices per million tokens. Table 10 presents representative token usage for agents in the Microsoft Arch-Informed run.

C. Cost Estimation

  • For the Microsoft Arch-Informed run, cost is estimated using input and output token counts and provider prices per million input and output tokens at the time of the experiment.Source: C. Cost Estimation · p. 25
  • The cited passage identifies T_in and T_out as input and output tokens, and P_in and P_out as provider prices per million input and output tokens at the time of the experiment.Source: C. Cost Estimation · p. 25
  • Table 10 presents representative token usage for different agents in the Microsoft Arch-Informed run.The supplied table caption identifies the table's scope but does not provide its cell values.Source: Table 10 · p. 26

Related papers

Loading 2610.03820v1…