PaperlayerView exact revision on arXiv

Source-linked AI summary

Cost-Aware Hierarchical Multi-Agent Ransomware Detection and Family Attribution

Mubashar Iqbal, Asifullah Khan

arXiv:2609.04820v1cs.CRcs.AI

TL;DR

The paper addresses the challenge of balancing sufficient ransomware-analysis modalities with analysis cost and reports a Cost Aware Hierarchical Multi-Agent System. Its complete configuration achieved 96.57% accuracy and 0.96 F1-score, while the evaluation identified differences between fixed evidence-acquisition strategies and the proposed approach.

  • Problem

    Ransomware analysis must balance modality sufficiency with analysis cost, while existing systems lack dynamic decisions about which modality is needed and when.

  • Method

    The paper presents a Cost Aware Hierarchical Multi-Agent System for ransomware detection and family attribution, with evidence acquisition and processing across modalities.

  • Results

    96.57% accuracy and 0.96 F1-score were achieved by the complete HMAS configuration, while fixed evidence-acquisition strategies showed substantial differences from the proposed approach.

  • Takeaways & Limitations

    The results support comparing adaptive evidence acquisition with fixed configurations when evaluating ransomware-analysis systems.

  • Takeaways & Limitations

    The work depends on the evaluated ransomware families and datasets and on obtainable evidence, with anti-analysis evasion or incomplete memory acquisition remaining constraints.

Abstract

from arXiv · show

Ransomware detection and family attribution require analysis of different modalities because it can use packing, obfuscation, process manipulation and runtime evasion techniques. However, conventional multimodal usually uses all available modalities for every sample resulting in unnecessary computational cost and increased latency. In this paper, we present a Cost Aware Hierarchical Multi-Agent System (HMAS) for adaptive ransomware detection. The proposed architecture organizes specialized agents into hierarchical domain controllers coordinated by a Meta Orchestrator. Static analysis is used as the initial low-cost modality while additional dynamic and memory modality is selectively used when confidence is insufficient or specialist agents exhibit disagreement. A cost model incorporates modality use and processing overhead. It enables the orchestration policy to balance analysis performance against computational cost. A locally deployed large language model provides verification for selected difficult cases without replacing the deterministic pipeline. Experimental evaluation compares adaptive HMAS with static only, static plus dynamic and exhaustive analysis policies across binary ransomware detection and multiclass family attribution. The complete HMAS achieved 96.57% accuracy, 0.96 F1-score and 0.99 ROC-AUC for binary detection. It also achieved 0.90 macro-F1 for family attribution. At the same time, the HMAS reduced average analysis cost by 43.97% relative to exhaustive analysis and substantially reduced average analysis latency except for the case where LLM is used. Routing analysis showed that 56.05% of cases were resolved using static evidence alone. Only 4.33% required the complete evidence pipeline. These findings demonstrate that adaptive HMAS can provide accuracy cost tradeoff for ransomware analysis while retaining support for heterogeneous and incomplete modalities.

1.1 Problem Statement

Ransomware analysis must balance sufficient evidence for reliable detection and attribution against the cost and latency of acquiring additional modalities. Existing approaches leave this balance unresolved, motivating adaptive acquisition based on uncertainty, disagreement, and expected cost benefit.

  • Problem motivation: The central challenge is balancing modality sufficiency with analysis cost for ransomware detection and family attribution.The paper frames the task as choosing when additional evidence is justified.
  • Existing modality limitations: Static, dynamic, and memory analysis each have distinct limitations, including evasion, latency, and costly forensic processing.Static analysis can fail against packing and obfuscation; dynamic analysis is vulnerable to sandbox evasion; memory analysis is impractical for every sample.
  • Existing modality limitations: Fixed multimodal systems process all evidence types indiscriminately, creating unnecessary costs when cheap static evidence is sufficient.The paper also notes that machine-learning systems generally optimize accuracy without controlling evidence acquisition.
  • Research gap: Current systems do not jointly address adaptive evidence acquisition, analysis cost, processing latency, uncertainty, and conflicting outputs.This gap is especially important when individual modalities are incomplete or ambiguous.
  • Proposed direction: The proposed framework treats static, dynamic, and memory evidence as conditional sources and escalates only when current evidence is insufficient.Its stated requirements include estimating uncertainty, judging whether added evidence justifies its cost, coordinating specialists, and defining a stopping condition.
  • Proposed direction: The contribution is a cost-aware hierarchical multi-agent framework that balances confidence gain against acquisition cost for unified detection and attribution.Specialized agents provide structured findings to higher-level agents for reconciled attribution.

2. Background and Related Work

Prior ransomware analysis spans signatures, static, dynamic, memory, multimodal, and adaptive methods, but these approaches face evasion, availability, cost, and fixed-workflow limitations. The paper positions adaptive modality selection as a cost-benefit decision over heterogeneous evidence.

  • Prior analysis methods: Early signature and rule-based methods were effective for known threats but are challenged by packing, encryption, and obfuscation.These transformations can hide behavioral patterns from fixed approaches.
  • Static analysis: Static analysis is fast and cheap, making it suitable as a first step whose confidence determines whether more expensive analysis is needed.It uses file-level features such as PE structure, imports, strings, bytes, opcodes, and entropy.
  • Dynamic analysis: Dynamic analysis exposes runtime behavior but incurs substantial time and resource overhead and remains vulnerable to sandbox evasion.The paper therefore motivates triggering it only when static evidence is insufficient.
  • Memory analysis: Memory forensics can reveal unpacked or injected ransomware activity invisible to earlier modalities, but acquisition and processing are computationally heavy and availability is limited.Its value is greatest when earlier evidence is inconclusive.
  • Multimodal analysis: Fixed multimodal pipelines combine complementary evidence but often assume every modality is available and processed together, wasting resources on easy cases.Static, dynamic, and memory modalities differ substantially in cost and operational requirements.
  • Adaptive analysis: Adaptive analysis treats modality acquisition as a sequential decision in which confidence, uncertainty, expected information gain, benefit, and cost guide the next action.The paper identifies ransomware analysis as a suitable setting because modality costs differ greatly.

3. Problem Formulation and Threat Model

The paper formulates ransomware detection and family attribution as selecting an analysis strategy that balances decision utility against explicit acquisition and processing costs. Its threat model assumes adversaries can make each individual modality incomplete, motivating adaptive coordination under defined operational assumptions.

  • 3. Problem Formulation and Threat Model: The framework minimizes evidence-acquisition cost while maintaining reliable ransomware detection and family attribution.The formulation treats analysis strategy as the decision variable.
  • 3. Problem Formulation and Threat Model: For a sample, the system produces binary detection and family-attribution outputs from acquired evidence under a selected strategy.Available strategies can include static, dynamic, and memory analysis.
  • 3. Problem Formulation and Threat Model: The optimal strategy maximizes decision utility minus a cost penalty, with the cost term explicitly incorporated into evidence acquisition.The cost model includes modality-specific processing factors.
  • 3.1 Analysis Cost Model: For modality i, analysis cost combines time, computational-resource consumption, execution or analysis events, and additional processing overhead.The evaluation reports absolute cost, cost reduction, and modality utilization.
  • 3.2 Confidence, Risk and Decision Criteria: The adaptive mechanism escalates when additional evidence is expected to justify its cost and rejects it when expected benefit is insufficient.Thresholds are selected on validation data, with sensitivity analysis examining their effect on the accuracy-cost trade-off.
  • 3.2 Confidence, Risk and Decision Criteria: Analysis terminates when detection and family confidence exceed required thresholds while uncertainty remains sufficiently low.Otherwise, the framework considers additional evidence using expected improvement in decision quality.
  • 3.3 Threat Model and Assumptions: The threat model assumes packing, obfuscation, sandbox evasion, and memory-artifact tampering can foil any single modality.This assumption motivates adaptive acquisition and hierarchical coordination.
  • 3.3 Threat Model and Assumptions: The system analyzes samples in an isolated monitored environment, excludes infrastructure attacks and LLM prompt injections, and assumes reliable family labels.The LLM is limited to grounded reasoning support rather than direct evidence.

4. Proposed Cost-Aware Hierarchical Multi-Agent Framework

The proposed HMAS uses hierarchical specialist agents and a Meta-Orchestrator to acquire ransomware evidence adaptively. It starts with low-cost static analysis, escalates on insufficient confidence or disagreement, and combines available modality outputs into detection and attribution decisions.

  • Architecture: The framework organizes a Meta-Orchestrator, domain controllers, specialist agents, and an optional LLM reasoning layer into hierarchical levels.It processes heterogeneous static, dynamic, network, and memory evidence.
  • Architecture: Specialist agents extract domain-specific risk signals, while domain controllers aggregate them into modality-level scores.Signals include entropy, imports, opcode behavior, API activity, process behavior, persistence artifacts, and memory indicators.
  • Adaptive orchestration: The Meta-Orchestrator begins with the cheapest broadly available evidence and adaptively selects whether to acquire additional modalities.The framework typically begins with static analysis and escalates when confidence is insufficient or agents disagree.
  • Adaptive orchestration: Static analysis can short-circuit the pipeline when scores are highly confident, while ambiguous scores trigger escalation.The implementation treats scores below 0.20 as benign and above 0.70 as sufficiently malicious for downstream avoidance.
  • Decision fusion: Controller disagreement among child agents is an additional escalation condition, and the Meta-Orchestrator fuses executed modality scores using renormalized weights.Final scores map to benign, suspicious, malicious, or ransomware verdict bands.
  • Agent coordination: Level-1 domain controllers coordinate modality specialists and return aggregate risk scores, child outputs, and escalation flags.Static, dynamic, and memory controllers combine their respective specialist evidence.
  • Agent coordination: Specialist agents execute concurrently, and excessive variance among risk scores triggers escalation while outputs remain schema-validated and auditable.Each output carries identity, version, risk score, latency, and agent-specific evidence fields.
  • Outputs and auditability: The pipeline produces binary and family verdicts, preserves deterministic and LLM-modified decisions separately, and falls back to a generic ransomware label when no family classifier is available.Family classification uses available static, dynamic, or memory models after a positive ransomware decision.

5. Implementation

The implementation uses an asynchronous hierarchical execution model with deterministic analytical agents, adaptive modality escalation, and bounded local language-model review. The pipeline records execution paths, agent use, latency, and cost for evaluation.

  • Software architecture: The HMAS is implemented in Python using asynchronous hierarchical execution and Microsoft AutoGen for agent orchestration.Pandas, NumPy, scikit-learn, XGBoost, and LightGBM support data processing and modeling.
  • Bounded language-model review: The optional locally hosted language model reviews selected difficult or conflicting cases without replacing deterministic analytical agents.Both deterministic and reviewed decisions are retained.
  • Adaptive evidence acquisition: Static analysis begins the pipeline, with dynamic and memory evidence acquired when confidence is insufficient or evidence conflicts.Available modality scores are normalized and fused for ransomware detection and family attribution.
  • Instrumentation: The system records executed analysis paths, agent invocations, latency, and cost for subsequent evaluation.The ransomware-analysis pipeline is depicted in Figure 3.

6. Experimental Methodology

The evaluation tests whether adaptive hierarchical coordination can support ransomware detection and family attribution while reducing unnecessary evidence acquisition and computational cost. It uses multimodal evidence, multiple analysis policies, and metrics spanning performance, calibration, routing, and efficiency.

  • Evaluation objectives: The evaluation examines binary detection, family attribution, adaptive routing efficiency, hierarchical coordination, and bounded LLM review.It compares adaptive HMAS with alternative evidence configurations and tests LLM review without replacing the deterministic pipeline.
  • Dataset and evidence: The dataset contains ransomware, non-ransomware, and benign samples with static, dynamic, and memory evidence features.Sixteen ransomware families are represented, with 8013 ransomware samples and 8015 benign samples; training data are stratified and balanced.
  • Evidence representation: Features from static, dynamic, and memory modalities are normalized, encoded into latent vectors, and fused for detection and attribution.This representation allows the framework to reason over whatever evidence is available per sample.
  • Compared policies: The study compares static-only, dynamic-only, memory-only, fixed multimodal, and cost-aware hierarchical multi-agent systems.The framework is evaluated across detection, family attribution, decision reliability, and analysis efficiency.
  • Performance metrics: Binary detection uses accuracy, precision, recall, F1-score, ROC-AUC, and error rates, while family attribution uses macro-averaged F1-score.Family attribution is evaluated separately so correct ransomware detection does not receive credit for an incorrect family label.
  • Reliability metrics: Calibration is evaluated with Expected Calibration Error because overconfident errors can prevent necessary evidence acquisition.Lower ECE indicates better calibration.
  • Routing metrics: Routing efficiency is measured through modality acquisition rates, average acquired modalities, and specialist-agent utilization per case.These measures quantify how often evidence and agents are invoked.

7. Results & Discussion

The results show that adaptive HMAS maintains strong ransomware detection and family-attribution performance while reducing analysis cost, latency, and unnecessary evidence acquisition. Routing and ablation findings indicate that evidence acquisition strategy and hierarchical components affect both predictive performance and computational efficiency.

  • Discussion: Fixed evidence acquisition did not consistently improve results: exhaustive analysis achieved lower recall than static-only analysis.The findings emphasize evidence acquisition strategy and decision mechanism alongside heterogeneous evidence availability.
  • Binary Detection: 96.57% accuracy, 0.94 precision, 0.99 recall, and 0.96 F1-score were achieved by complete HMAS for binary ransomware detection.The high recall indicates that the adaptive framework identified most ransomware cases while maintaining a relatively low false positive rate.
  • Family Attribution: 98.07% accuracy and 0.90 macro F1 were achieved for multiclass ransomware family attribution by the full HMAS configuration.Macro precision was 92.71% and macro recall was 0.89, with performance remaining strong across evaluated family classes.
  • Cost and Efficiency: 243.72 ms versus 107.34 ms average latency shows that HMAS reduced end-to-end analysis time by approximately 56% relative to exhaustive analysis.Average agent utilization also decreased from 1.00 for exhaustive analysis.
  • Routing Behavior: 56.05% of cases were resolved with static evidence alone, 39.62% escalated to static plus dynamic analysis, and 4.33% required the complete pipeline.High-confidence cases terminate after static analysis, while insufficient confidence or conflicting evidence triggers additional analysis.
  • Ablation: Ablation results showed that HMAS components contribute differently to predictive performance and computational efficiency.The study separately examined cost-aware routing, adaptive escalation, hierarchical coordination, memory evidence, disagreement-based escalation, and LLM-assisted verification.
  • Limitations: The evaluation is limited by dependence on selected ransomware families and datasets, obtainable evidence, memory-model performance, and multi-agent coordination overhead.Optional LLM inference also introduces risks from generative-model inconsistency and hallucination, while deployment-specific costs and scalability remain unresolved.

8. Conclusion

The Cost Aware Hierarchical Multi Agent System treats ransomware detection and family attribution as a sequential decision process, adaptively acquiring evidence according to confidence, disagreement, availability, and cost. It achieved strong detection and attribution performance while reducing analysis cost through selective evidence acquisition and bounded LLM review.

  • 8. Conclusion: The system adaptively acquires static, dynamic, and memory evidence according to confidence, disagreement, availability, and cost.Analysis is organized as a sequential decision process rather than a fixed multimodal pipeline.
  • 8. Conclusion: 96.57% binary accuracy, 0.99 recall, 0.96 F1-score and 0.90 multiclass macro-F1 were achieved by the complete system.These results cover binary ransomware detection and multiclass family attribution.
  • 8. Conclusion: 43.97% average analysis cost reduction was achieved through adaptive routing and selective acquisition of expensive evidence.The complete pipeline was required for only 4.33% of cases.
  • 8. Conclusion: 56.05% of samples were resolved using static evidence alone, while only 4.33% required the full analysis pipeline.This routing pattern indicates that expensive modalities were reserved for a minority of cases.
  • 8. Conclusion: The LLM was restricted to difficult or conflicting cases, and the reported gains were attributed to adaptive routing and hierarchical coordination rather than generative output.The LLM served as bounded verification within the analysis process.
Loading 2609.04820v1…