PaperlayerView exact revision on arXiv

Source-linked AI summary

SENTINEL: A Multi-Pathway Architecture for Detecting Living-Off-the-Land APT Attacks on Windows Command Lines

Ahad Bin Islam Shoeb, Kamrul Hasan, Jamal Uddin Tanvin, Liang Hong, Imtiaz Ahmed, Md Arif Billah, Al Amin

arXiv:2609.14593v1cs.CRcs.AI

TL;DR

LOTL detection is difficult because trusted Windows utilities can be used for malicious operations, while existing approaches struggle with obfuscation and novel techniques. SENTINEL combines semantic, character-level, sequence, and anomaly pathways; on Volt Typhoon and obfuscated inputs, it outperforms standalone BERT, while balanced adversarial evaluation reveals weak malicious recall despite high validation accuracy.

  • Problem

    Trusted Windows utilities are used by both administrators and attackers, while supervised detection can confidently misclassify novel LOTL techniques absent from training.

    Source: §I · pp. 1–2

  • Method

    SENTINEL combines BERT semantic encoding, character-level CNN analysis, inter-command attention, and autoencoder-based anomaly detection in a multi-pathway detector.

    Source: §I · p. 2, A. Overview · p. 2

  • Results

    SENTINEL reached 92.0% accuracy on Volt Typhoon and 91.2% on obfuscated inputs, versus BERT’s 74.0% and 72.0%; over 98% validation accuracy co-occurred with 44–58% malicious recall.

    Source: §I · p. 2, B. Comparative Results · p. 4

  • Takeaways & Limitations

    The reported 8.0 percentage-point gap over BERT with augmentation indicates architectural robustness beyond data augmentation alone.

    Source: B. Comparative Results · p. 4

    The effective Volt Typhoon test size is approximately 50, 24% of malicious samples share semantic patterns with training data, and enterprise-scale telemetry remains unvalidated.

    Source: G. Limitations · p. 6

Abstract

from arXiv · show

Living-Off-the-Land (LOTL) is the dominant evasion technique of Advanced Persistent Threat (APT) actors, exploiting legitimate Windows utilities to conduct malicious operations without deploying custom malware and enabling state-sponsored campaigns to maintain persistent access within military and critical defense infrastructure for extended periods. Existing detection methods fail against obfuscated commands and multi-stage attack sequences, as demonstrated by the Volt Typhoon APT campaign, which maintained undetected access to U.S. critical infrastructure for over 18 months using exclusively signed Windows utilities. We present SENTINEL, a multi-pathway architecture integrating BERT-based semantic encoding, character-level CNN for obfuscation invariance, inter-command attention for multi-stage pattern recognition, and autoencoder-based anomaly scoring. Evaluated on a balanced Volt Typhoon benchmark derived from Microsoft and CISA threat intelligence advisories, SENTINEL achieves 92.0% accuracy on documented state-sponsored attack commands and 91.2% on obfuscated variants, compared to 74.0% and 72.0% for standalone BERT. Per-class analysis reveals that models achieving over 98% overall validation accuracy on imbalanced data exhibit only 44-58% malicious recall on balanced adversarial sets. Character-level processing contributes 5.6 percentage points of obfuscation invariance, and the 8.0 percentage point gap over augmentation-only baselines confirms structural architectural value beyond data-driven robustness alone.

I. INTRODUCTION

LOTL attacks misuse trusted Windows utilities, creating detection challenges because adversaries share administrative tools and can obfuscate commands or coordinate benign-looking commands into attack progressions. SENTINEL combines four pathways to address these failure modes, and the paper’s balanced Volt Typhoon benchmark highlights weaknesses hidden by conventional validation accuracy.

I. INTRODUCTION

  • LOTL abuses trusted Windows utilities, making malicious operations resemble routine administration and structurally resisting signature-based detection, binary inspection, and sandbox analysis.These binaries are whitelisted and used by both administrators and adversaries,.Source: §I · p. 1
  • Volt Typhoon evaded commercial EDR products for over 18 months while compromising U.S. communications, energy, and water infrastructure.Joint advisories identified systematic obfuscation and multi-stage command sequencing as primary evasion mechanisms.Source: §I · p. 1
  • Token methods lose accuracy on adversarial commands, independent-command classifiers miss attack progressions, and supervised models can confidently misclassify novel LOTL techniques.Accuracy drops from over 98% on clean validation data to 52–74% on adversarial inputs; coordinated benign commands can mirror reconnaissance-to-credential-access workflows.Source: §I · pp. 1–2
  • SENTINEL combines BERT semantic encoding, character-level CNN analysis, inter-command attention, and autoencoder anomaly detection to address the identified failure modes.The four pathways target contextual intent, obfuscation invariance, multi-stage correlation, and distributional-shift triage, respectively.Source: §I · p. 2
  • The balanced Volt Typhoon benchmark exposes 44–58% malicious recall despite over 98% validation accuracy, while ablation attributes an 8.0-point gain beyond augmentation-only baselines to architecture.The paper also presents this benchmark and per-class analysis as a way to expose failures concealed by conventional imbalanced metrics.Source: §I · p. 2

A. Traditional Machine Learning Approaches

Prior LOTL detectors face limitations in lexical robustness, semantic interpretation, temporal modeling, or coverage across utilities. SENTINEL combines complementary pathways in one fused representation for command classification.

A. Traditional Machine Learning Approaches

  • TF-IDF and related lexical approaches can achieve 91–94% F1 on PowerShell detection, but syntactic transformations defeat lexical features and bag-of-words methods miss cross-command dependencies and semantic equivalence.These methods operate on surface form despite legitimate tools being shared by defenders and adversaries,,.Source: A. Traditional Machine Learning… · p. 2

B. Deep Learning and Transformer Methods

  • LSTMs can lose early context in long command chains, CharCNNs cannot distinguish commands with identical character distributions, and BERT tokenization fragments encoded payloads.BERT’s adversarial accuracy drop is 25–27 percentage points; security-adapted BERT variants retain tokenization vulnerability and lack character-level obfuscation invariance,, –.Source: B. Deep Learning and Transformer Methods · p. 2

C. Specialized LOTL Systems

  • Specialized systems have narrower or fragile coverage: LOLWTC falls to 52.0% on Volt Typhoon and 48% on obfuscated variants, while PowerPeeler covers PowerShell-specific obfuscation.No prior architecture jointly addresses obfuscation invariance, semantic discrimination, temporal modeling, and anomaly detection in an end-to-end framework evaluated on documented state-sponsored campaigns –,.Source: C. Specialized LOTL Systems · p. 2

A. Overview

  • SENTINEL dual-encodes commands as WordPiece tokens and characters, then fuses BERT, CharCNN, inter-command attention, and autoencoder outputs into a 1856-dimensional representation for classification.The pathways model semantic intent, character-level obfuscation patterns, multi-stage progressions, and distributional shifts [Fig. 1].Source: A. Overview · pp. 2–3, Figure 1 · p. 3

B. BERT Semantic Encoder

The BERT pathway forms contextual command representations from token, position, and segment embeddings. The CharCNN pathway processes raw character sequences to capture obfuscation patterns that token-based methods can miss.

B. BERT Semantic Encoder

  • Each BERT token embedding combines token, position, and segment information; twelve transformer layers produce 768-dimensional contextual outputs for downstream pathways and feature fusion.The encoder is used to capture command intent.Source: B. BERT Semantic Encoder · p. 3

C. Character-Level CNN

  • CharCNN converts commands to fixed-length ASCII sequences and applies character n-gram filters of sizes 3, 4, and 5, then max-pools features into a 192-dimensional representation.The representation captures patterns such as base64 padding, PowerShell concatenation operators, and bracket nesting regardless of their position in the command string.Source: C. Character-Level CNN · p. 3

D. Inter-Command Attention

Inter-command attention models dependencies across commands grouped into short session windows. The training setup combines single-command and sequence-level supervision and uses labeled benign workflows and malicious progressions.

D. Inter-Command Attention

  • Commands are grouped into 1-hour windows of at most five, BERT-encoded, and mean-pooled into 768-dimensional vectors for session-level dependency modeling.The module models dependencies across commands within a session.Source: D. Inter-Command Attention · p. 3
  • Four attention heads with d_k = 192 compute dependencies among commands in each sequence.Source: D. Inter-Command Attention · p. 3
  • Dual supervision weights single-command loss at α = 0.6 and sequence-level loss at 0.4; training uses 1,247 sequences, including 892 benign workflows and 355 malicious progressions.Malicious sequences are labeled by MITRE ATT&CK phases, with Cohen’s κ = 0.87 inter-rater agreement.Source: D. Inter-Command Attention · p. 3

E. Anomaly Detection

SENTINEL uses an autoencoder branch to flag out-of-distribution commands for analyst triage, then fuses pathway features for classification. Its combined training objective supports stable performance across five random seeds.

E. Anomaly Detection

  • The autoencoder flags commands whose reconstruction error exceeds 0.11, a threshold set at the 95th percentile of benign validation data.Pretrained only on benign training data and frozen during joint training, the branch serves operational triage; its removal changes classification by only 0.4–0.6%.Source: E. Anomaly Detection · p. 3

F. Feature Fusion and Training

  • Feature concatenation forms a unified 1,856-dimensional representation (1,856 = 768 + 192 + 768 + 128), then LayerNorm, ReLU, and dropout (p = 0.3) feed two dense layers for final binary classification.Source: F. Feature Fusion and Training · p. 3
  • Across five random seeds, SENTINEL achieves 92.0% ± 1.2% accuracy on Volt Typhoon and 91.2% ± 0.8% on obfuscated inputs.Training uses class-weighted cross-entropy to address the 15:1 training imbalance and optimizes it with the reconstruction loss.Source: F. Feature Fusion and Training · p. 4

IV. EXPERIMENTAL EVALUATION

The evaluation combines a command dataset with a balanced, held-out Volt Typhoon set and a transformed obfuscated set. The setup includes multiple baseline families and separates all Volt Typhoon samples from training.

A. Dataset and Setup

  • All Volt Typhoon samples are held out from training, which uses a 70/15/15 split of 7,049 commands after exact- and near-duplicate filtering.The corpus contains 6,613 benign and 436 malicious commands; filtering removes 312 exact duplicates and 89 near-duplicates.Source: A. Dataset and Setup · p. 4
  • The balanced Volt Typhoon evaluation set contains 100 commands: 50 malicious examples spanning eight attack phases and 50 utility-matched benign commands.Two analysts achieve Cohen’s κ = 0.91 inter-rater agreement.Source: A. Dataset and Setup · p. 4
  • The obfuscated set contains 250 test-split commands transformed using eight deterministic techniques, including base64 encoding, alias substitution, and whitespace insertion.The set also includes string concatenation, backtick injection, case randomization, character encoding, and format operators.Source: A. Dataset and Setup · p. 4
  • Baselines cover classical machine learning, deep learning, and specialized LOTL systems, with experiments using PyTorch 2.0.1, bert-base-uncased, and an NVIDIA Tesla T4 GPU.The baseline set includes BoW, TF-IDF, Random Forest, XGBoost, Char-CNN, BERT variants, PowerPeeler, and LOLWTC.Source: A. Dataset and Setup · p. 4

B. Comparative Results

SENTINEL’s accuracy on adversarial sets exceeds the reported BERT and LOLWTC results, and its advantage over augmentation-only BERT is reported on Volt Typhoon. The structural analysis attributes model weaknesses to obfuscation, sequence, and distribution-shift challenges.

B. Comparative Results

  • 92.0% accuracy on Volt Typhoon and 91.2% on obfuscated inputs exceed BERT’s 74.0% and 72.0%, while the Volt Typhoon result is 8.0 points above BERT with augmentation at 84.0%.Table I covers accuracy across all evaluation datasets; Fig. 2 visualizes validation, Volt Typhoon, and obfuscated-set accuracy.Source: Table I · p. 4, B. Comparative Results · p. 4, Figure 2 · p. 4

C. Structural Failure Analysis

  • The five-dimension structural analysis identifies weaknesses in token-based obfuscation handling, independent command classification, and detection of distributional shift.SENTINEL’s character-level CNN captures patterns independent of tokenization; inter-command attention models sequence relationships, while reconstruction error flags out-of-distribution inputs.Source: C. Structural Failure Analysis · pp. 4–5

D. Per-Class Analysis

Per-class results on the balanced Volt Typhoon set expose low malicious recall among models with high validation accuracy. SENTINEL reports 90% benign recall and 94% malicious recall on that set.

D. Per-Class Analysis

  • TF-IDF’s 98.72% validation accuracy accompanies 50% malicious recall on Volt Typhoon, while BERT’s 99.29% validation accuracy accompanies 58% malicious recall.Fig. 4 presents per-class precision and recall bars for each model on the balanced Volt Typhoon benchmark.Source: D. Per-Class Analysis · p. 5, Figure 4 · p. 5
  • SENTINEL achieves 90% benign recall and 94% malicious recall on the Volt Typhoon set.For defense network monitoring, the passage identifies malicious recall as the primary operational metric because missing a coordinated state-sponsored command sequence carries higher cost than a false alarm.Source: D. Per-Class Analysis · p. 5

E. Ablation Study

The ablation study identifies CharCNN as SENTINEL’s largest single-pathway gain, while inter-command attention addresses coordinated progressions and anomaly detection primarily supports triage. SENTINEL also maintains higher true-positive rates than the cited baselines at low false-positive rates.

E. Ablation Study

  • Removing CharCNN degrades Volt Typhoon and obfuscated accuracy by 5.6 percentage points, from 91.6% to 86.0%.The ablation identifies character-level obfuscation invariance as the largest single-pathway gain.Source: E. Ablation Study · p. 5, Figure 5 · p. 5
  • Disabling inter-command attention reduces Volt Typhoon accuracy by 2.0 percentage points while targeting multi-stage pattern recognition.The reported contribution is smaller in absolute accuracy terms than CharCNN’s, but addresses a distinct failure mode.Source: E. Ablation Study · p. 5
  • Removing anomaly detection changes classification performance by only 0.4–0.6%, supporting its role as operational triage rather than primary detection.The passage describes the branch as intended for triaging novel techniques.Source: E. Ablation Study · p. 5

F. Low False-Positive Rate Performance

  • At FPR = 0.1%, SENTINEL maintains 81.8% TPR versus 69.7% for BERT, a 12.1 percentage point improvement.The passage attributes this gain to CharCNN maintaining high-confidence scores on obfuscated variants.Source: F. Low False-Positive Rate Performance · p. 5

G. Limitations

The paper’s limitations include Windows-only scope, a small effective Volt Typhoon test set, fixed one-hour temporal windows, and anomaly-threshold calibration needs. It concludes that LOTL detection should model syntactic and temporal structure, while identifying longer-horizon modeling and broader deployment as future work.

G. Limitations

  • The study addresses Windows command-line LOTL exclusively; Linux LOTL extension is left to future work because no annotated benchmark at LOLBAS fidelity exists.The passage notes that Linux-based LOTL presents different challenges.Source: G. Limitations · p. 5
  • At 0.1% FPR, SENTINEL achieves 81.8% TPR, compared with 69.7% for BERT and 56.1% for LOLWTC.The figure caption reports that SENTINEL consistently outperforms baselines at low-FPR operating points on the validation set.Source: Figure 6 · p. 6
  • The effective Volt Typhoon test size is approximately 50, with wide 95% confidence intervals and 24% of malicious samples sharing semantic patterns with LOLBAS training data.Performance on enterprise-scale telemetry remains unvalidated.Source: G. Limitations · p. 6
  • Inter-command attention uses fixed 1-hour windows, so strategic phases separated by days or weeks fall outside its scope.The passage says longer-horizon tracking would require hierarchical attention or external memory.Source: G. Limitations · p. 6
  • At τ = 0.11, anomaly scoring flags roughly 5% of benign commands, translating to thousands of daily triage alerts at enterprise scale without recalibration.The passage recommends optional analyst-triage deployment with independent FPR/TPR calibration.Source: G. Limitations · p. 6

V. CONCLUSION

  • The conclusion argues that robust LOTL detection requires explicitly modeling syntactic and temporal structure rather than relying on single-view representations.Future work includes Linux LOTL, campaign-level temporal modeling, and richer system and process context signals.Source: §V · p. 6
Loading 2609.14593v1…