PaperlayerView exact revision on arXiv

Source-linked AI summary

EGAMA-RC: Risk-Calibrated Evidence-Gated Adaptive Malware Analysis for Robust and Interpretable Memory-Forensic Triage

Isaac Kofi Nti

arXiv:2608.22721v1cs.CRcs.LG

TL;DR

Clean accuracy alone does not answer whether malware predictions are reliable enough for automatic operational action. EGAMA-RC adds calibrated evidence-gated routing, novelty awareness, and explanation-conditioned signals, achieving broad automated coverage while preserving review and escalation pathways.

  • Problem

    Operational malware triage requires evidence about uncertainty, novelty, robustness, interpretability, latency, and review cost beyond clean-data classification accuracy.

    Source: §I · p. 1, E. Selective Prediction, Calibration,… · p. 3

  • Method

    EGAMA-RC combines SHAP-guided feature refinement, model-pool evaluation, calibrated evidence gating, novelty scoring, explanation-conditioned evidence, and runtime-aware routing.

    Source: §I · pp. 1–2

  • Results

    93.12% of pooled samples were accepted with 99.864% accepted-set accuracy and a 0.136% false-accept rate.

    Source: §I · p. 2

  • Takeaways & Limitations

    Dependable malware analysis depends on controlling when automated decisions are trusted through acceptance, review, escalation, and novelty-aware pathways.

    Source: §V · p. 12, §VI · p. 12

    Robustness findings are limited to the evaluated datasets, attacks, and perturbation contracts, while analyst impact is represented by review-load proxies rather than direct user studies.

    Source: §V · p. 12

Abstract

from arXiv · show

Machine-learning malware detectors often achieve high clean-data accuracy, but operational triage also requires evidence about uncertainty, novelty, robustness, interpretability, latency, and review cost. This paper presents EGAMA-RC, a risk-calibrated evidence-gated framework for memory-forensic malware triage. Building on SHAP-guided feature refinement, EGAMA-RC combines dataset-specific refinement, model-pool evaluation, adversarial and open-family testing, novelty scoring, explanation-conditioned evidence, and runtime-aware routing. Low-risk samples are accepted automatically, while uncertain, high-risk, or potentially novel cases are routed to review, escalation, or novelty-aware handling. Across three malware datasets and a frozen multi-seed protocol, the selected hybrid gate accepts 93.12% of pooled samples with 99.86% accepted accuracy and a 0.136% false-accept rate. Novelty calibration reduces over-restrictive review behavior while preserving a low unsafe-accept profile. XGBoost provides lightweight fast-path inference with p50/p95 latency of 0.0054/0.0059 ms per sample. The results show that dependable malware analysis requires risk-calibrated routing, novelty awareness, and controlled analyst review, not classification accuracy alone.

I. INTRODUCTION

EGAMA-RC addresses the gap between clean-data classification and operationally reliable malware triage. It routes uncertain, high-risk, adversarially sensitive, or potentially novel cases toward review, escalation, or novelty-aware handling.

  • Memory-forensic analysis complements static detection by exposing volatile execution evidence such as processes, modules, handles, and injection traces.Source: §I · p. 1
  • Clean-data performance alone does not establish reliability under perturbation, family shift, latency, explanation instability, or review-cost constraints.Source: §I · p. 1
  • EGAMA-RC extends SHAP-GFR from model evaluation to runtime decision routing across automatic acceptance, review, escalation, and novelty-aware handling.Source: §I · pp. 1–2
  • SHAP-derived evidence, explanation stability, and forensic consistency inform whether predictions are accepted, escalated, reviewed, or flagged as novel.Source: §I · p. 2
  • The framework evaluates three datasets using clean, robustness, open-family, calibration, attribution-stability, and deployment-cost analyses.Source: §I · p. 2

B. Explainable AI for Malware Detection

The paper positions explainability, adversarial robustness, concept drift, novelty, and calibration as complementary evidence for operational malware triage. EGAMA-RC uses these signals to decide when predictions should be accepted, deferred, escalated, or reviewed.

III. METHODOLOGY

The methodology integrates dataset-specific preprocessing and feature refinement with model-pool evaluation, sample-level evidence construction, and risk-calibrated runtime routing. It evaluates the framework across three heterogeneous malware datasets while preserving dataset-native feature spaces and leakage controls.

C. Baseline Model Pool

EGAMA-RC evaluates a five-family model pool and derives runtime evidence from confidence, uncertainty, margins, cross-model variance, and disagreement. XGBoost is positioned as the fast path based on validated accuracy and latency evidence.

E. Risk-Calibrated Acceptance and Selective Prediction

EGAMA-RC combines heuristic and conformal evidence gates before assigning operational routes. It then applies severe-risk escalation and novelty-aware handling to samples that should not be automatically accepted.

Risk-Calibrated Acceptance

  • The hybrid gate accepts a sample only when both heuristic evidence thresholds and conformal acceptance conditions pass.The layered policy uses a default conformal risk level of α = 0.05 and routes failures to review.Source: E. Risk-Calibrated Acceptance and… · p. 6

Selective Routing

  • Samples failing acceptance can be routed to review, while severe-risk conditions trigger escalation for higher-assurance handling.Escalation is a routing label rather than an automatic invocation of a separate specialist model.Source: F. Adaptive Routing Policy · pp. 6–7

Routing Procedure

  • Algorithm 1 sequences calibrated model outputs, fast-path prediction, evidence construction, gating, escalation, novelty scoring, and report generation.The final report records prediction, routing outcomes, evidence fields, route reasons, and available diagnostic artifacts.Source: F. Adaptive Routing Policy · pp. 6–7

Risk-Calibrated Acceptance

Novelty-Aware Handling

  • Novelty evidence uses nearest known-family centroid distance and normalized novelty scoring to identify potential open-family risk.The default novelty percentile threshold is 95, and novelty flags remain review-level in the collapsed binary route.Source: F. Adaptive Routing Policy · pp. 6–7

G. Explanation-Conditioned Evidence Generation

EGAMA-RC uses SHAP-derived evidence to summarize the forensic basis of predictions and assess explanation stability. Explanations support decisions but are not treated as causal proof.

Evidence Generation

  • SHAP-derived evidence organizes prediction support into forensic categories for operational triage.TreeSHAP is used for supported tree models, while other explanation methods are considered when stability and computational cost are acceptable.Source: G. Explanation-Conditioned Evidence… · p. 7

Stability Diagnostics

  • Explanation stability compares the overlap between top-k SHAP features before and after perturbation.The measure uses the intersection-over-union of the two top-k feature sets.Source: G. Explanation-Conditioned Evidence… · p. 7

H. Threat Model and Adversarial Evaluation Design

The threat model evaluates whether attackers can cause malicious memory samples to be accepted as benign by manipulating classifier and routing evidence. Evaluation spans multiple attacker knowledge levels and perturbation strategies.

Threat Model

  • The primary attacker objective is an integrity violation: accepting a malicious memory sample as benign.Attackers may target malware probability, risk, novelty, or model-disagreement signals.Source: H. Threat Model and Adversarial… · p. 7
  • The evaluation distinguishes black-box, gray-box, and adaptive attackers by their access to labels, feature categories, model behavior, and the gate.The main attack stage is test-time evasion.Source: H. Threat Model and Adversarial… · p. 7

Adversarial Evaluation

  • Adaptive attacks optimize evasion of both classification and routing evidence subject to feasible memory-forensic constraints.The constraints include non-negative counts, percentile-bounded values, and plausible feature-group relationships.Source: H. Threat Model and Adversarial… · p. 7
  • The protocol includes gradient-based attacks such as PGD, FGSM momentum, and MI-FGSM, alongside generic perturbation and transfer evaluations where supported.The stated adversarial settings include epsilon = 0.02 for FGSM and PGD, with PGD using alpha = 0.005 for 10 steps.Source: H. Threat Model and Adversarial… · p. 7

I. Latency, Efficiency, and Deployment Protocol

The deployment protocol measures fast-path efficiency alongside routing and review costs. It also identifies incomplete observability of route-specific computation as a deployment limitation.

Efficiency Metrics

  • The efficiency protocol measures fast-path latency, p50/p95 per-sample latency, model size, analyst-review rate, escalation rate, novelty burden, and safety–throughput trade-offs.These measurements are intended to provide deployability evidence for EGAMA-RC.Source: §I · p. 7

Deployment Limitation

  • The study does not separately measure gate, novelty-scoring, explanation-generation, or route-specific review and escalation latency.Complete decision-path observability is therefore treated as a future deployment extension.Source: §I · pp. 7–8

J. Standard Classification Evaluation Metrics

The paper defines standard classification metrics for evaluating malware detectors and routing metrics for measuring selective automatic acceptance and unsafe malware acceptance.

K. Reproducibility and Implementation Details

The evaluation uses a configured, auditable pipeline spanning multi-dataset refinement, model-pool benchmarking, evidence-gate comparisons, dataset-level routing, and severe-risk trigger analysis. Results favor hybrid gating as a conservative operating point that maintains high automation while controlling unsafe acceptance and adapting review to workload difficulty.

Reproducibility and implementation

Baseline model performance

Risk-calibrated routing

Dataset-level routing

Severe-risk triggers

C. Novelty-Aware Routing and Calibration Effects

Novelty calibration substantially increases acceptance coverage while preserving a low unsafe-accept profile, but robustness and deployment results show that routing must remain risk-aware and dataset-conditioned.

Novelty calibration

  • 93.12% accept coverage followed novelty calibration, up from 64.65%, while accepted accuracy improved slightly from 99.80%.The selected policy used novelty quantile 0.80, conformal alpha 0.01, and risk-conditioned novelty.Source: C. Novelty-Aware Routing and… · p. 10
  • 0 conformal rejections occurred under the selected configuration, indicating that calibrated novelty and risk-conditioned evidence drove routing.Source: C. Novelty-Aware Routing and… · p. 10
  • 28.47 percentage points increased accept coverage without degrading safety, while false-accept rate decreased.The final routing behavior was driven primarily by calibrated novelty and risk-conditioned evidence rather than conformal rejection.Source: C. Novelty-Aware Routing and… · p. 10

Robustness and family shift

  • 61.96% attack success accompanied MLP FGSM testing, while FGSMtransfer reduced XGBoost accuracy to 0.4829 with 51.71% attack success.These findings support deferral and escalation because classifiers are not uniformly attack-invariant.Source: D. Robustness, Open-Family… · p. 10
  • 0.9988 mean F1 was achieved by RF under LOFO, compared with 0.9987 for XGBoost; neural models were more sensitive to family shift.On the hardest held-out family, TIBS, MLP, CNN1D, and CNN-LSTM reached F1-scores of 0.8741, 0.8714, and 0.8552, respectively.Source: D. Robustness, Open-Family… · p. 10

Novelty calibration

  • All 8,790 CIC-MalMem-2022 test samples were flagged at the fixed 95th-percentile novelty threshold, unlike mean open-family rates of 0.0515 for BODMAS and 0.0565 for BCCC.The review-all outcome reflects an over-restrictive fixed threshold rather than universal novelty.Source: D. Robustness, Open-Family… · p. 10

Deployment trade-offs

  • 0.0054/0.0059 ms p50/p95 latency made XGBoost the fastest fast-path model, followed by RF at 0.1839/0.3715 ms.Source: E. Runtime Overhead and Deployment… · p. 11
  • 93.12% hybrid-gate acceptance paired approximately 0.14% false-accept rate, whereas conformal-only gating reached 96.31% acceptance with 0.640% false acceptance.Hybrid gating routed 6.88% of pooled samples to review, compared with 3.69% for conformal-only gating.Source: E. Runtime Overhead and Deployment… · p. 11

F. Comparison with Related Work

EGAMA-RC differs from related malware-detection and adversarial-defense studies by treating deployment reliability as a decision-governance problem rather than claiming universal predictive superiority.

Comparison with related work

  • Related studies address separate concerns including unseen-family detection, temporal drift, adversarial defense, and operational malware analysis.Source: F. Comparison with Related Work · p. 11
  • Direct numerical comparison is inappropriate because datasets, representations, splits, attack models, and evaluation objectives differ.EGAMA-RC’s distinction is methodological rather than a claim of universal performance superiority.Source: F. Comparison with Related Work · p. 11

EGAMA-RC distinction

Operational implications

  • Reliable malware triage is framed as a decision-governance problem in which predictive performance becomes operationally useful when paired with routing logic.Source: §V · p. 12
  • Novelty-aware routing can trigger review for unfamiliar evidence, while explanation stability supplies an additional reliability signal under perturbation.The appropriate operating point depends on organizational risk tolerance and the importance of unknown-family exposure.Source: §V · p. 12

Limitations

  • The findings are bounded by evaluated datasets, attacks, perturbation contracts, dataset-dependent novelty behavior, incomplete footprint metadata, and review-load proxies instead of user studies.Source: §V · p. 12

Conclusion

  • Hybrid routing provides a strong safety-throughput operating point, while calibrated novelty reduces over-restrictive review without weakening the safe-acceptance profile.Source: §VI · p. 12

Related papers

Loading 2608.22721v1…