Source-linked AI summary
DUPIN: Attack Learning Is Still Needed! Demonstrating Few-Shot after Unsupervised Pretraining Is A Nimble Forensics Learner
Chanwoo Bae, Hailun Ding, Shiqing Ma, Xiangyu Zhang
TL;DR
Attack forensics lacks abundant, reliable labeled attacks, while benign-heavy logs make conventional supervised and one-class approaches difficult. DUPIN pretrains a GAT on provenance graphs with masked prediction and then fine-tunes it using few-shot attack samples. Across 25 APT attack sets from four sources, DUPIN outperforms one-class learners, alternative GNNs, and ablated models, with extensive pretraining improving accuracy by 16%.
Problem
Attack data is sparse and labels are costly and error-prone, limiting supervised training; benign-like malicious behavior also challenges one-class learners.
Method
DUPIN transforms audit logs into provenance graphs, pretrains a foundation GAT through masked prediction on benign logs, and fine-tunes it with few-shot attack samples.
Results
DUPIN outperforms one-class learners, alternative GNN architectures, and ablated models across 25 APT attack sets from four sources; pretraining improves accuracy by 16%.
Takeaways & Limitations
Attack learning remains useful when threat techniques recur across actors and sparse attack samples can be combined with broad benign-log pretraining.
Takeaways & Limitations
DUPIN detects only previously observed attack types, so effectiveness depends on attack coverage and requires some attack-data collection.
Abstract
from arXiv · showhide
We propose a novel approach to learning-based attack forensics called DUPIN. DUPIN performs unsupervised pre-training on an enormous amount of audit events in the form of provenance graphs. It then proceeds to a few-shot learning stage, leveraging a small number of labeled attack examples to fine-tune its detection capabilities. We pretrain DUPIN on up to 38 - 52 days of audit logs (7.3TB total) and evaluate it against various baselines on 25 APT campaigns across four different data sources, facilitating the scalable evaluation.
1 Introduction
DUPIN addresses the scarcity and complexity of attack data by combining extensive unsupervised pretraining on provenance graphs with few-shot learning from labeled attacks. It is evaluated across diverse APT datasets and reports improved accuracy with extensive pretraining.
- Attack forensics is difficult because complex APTs hide within extensive benign logs, while trained analysts are limited.
- Sparse attack data makes traditional supervised training difficult, while one-class methods can mistake malicious behaviors for benign activity.
- 85% of attack techniques are used by at least two known threat actors, motivating learning from shared and recurring attack knowledge.
- DUPIN pretrains a Graph Attention Network on benign provenance graphs using masked node, edge, and classification predictions, then fine-tunes it with attack samples.
- 16% accuracy improvement results from extensive DUPIN pretraining in the ablation study.
- Evaluation covers 25 representative APT attack sets from four dataset sources using cross-validation separated by red-team origin.
2 Motivation
The motivation section argues that benign-heavy, noisy audit data limits supervised and one-class learning for attack forensics. DUPIN instead uses massive benign-log pretraining followed by few-shot attack fine-tuning to learn attack context beyond superficial activity patterns.
- 2 Motivation: Analysts cannot review entire logs because overwhelming benign activity obscures ongoing attacks.
- 2 Motivation: Supervised methods require large, accurately labeled datasets, but attacks are rare and labeling is costly and error-prone.
- 2 Motivation: One-class learners profile benign activity, but benign-like attack behavior can prevent them from recognizing malicious context.
- 2 Motivation: DUPIN pretrains on massive benign audit logs to learn behavioral context, then uses relatively small attack samples for few-shot fine-tuning.
- 2 Motivation: Fine-tuning can abstract attack context shared across examples even when individual activities, such as writing multiple files, also occur benignly.
3 System Design
DUPIN transforms noisy audit logs into provenance-graph representations and uses a GAT-based architecture with edge-aware attention. Its design combines dynamic file-path tokenization, bounded subgraphs, unsupervised graph pretraining, and downstream classifier adaptation.
- System overview: DUPIN comprises a tokenizer, graph constructor, base graph model, and attack classifier.These components form the system’s main processing and learning pipeline.
- 3.1 Tokenizer: Dynamic token extraction merges redundant file names into format tokens while preserving significant names and bounding vocabulary size.Names recurring in the same directory beyond κ are normalized, with vocabulary constrained by τ.
- 3.1 Tokenizer: Path embeddings split file paths into directory, file-name, and extension tokens, then aggregate their embeddings with a recurrent network.The aggregation preserves token order and context; DUPIN uses an LSTM for this purpose.
- 3.2 Graph Construction and Subgraphs Extraction: DUPIN constructs heterogeneous provenance graphs whose nodes represent entities and whose edges represent interactions, with names and edge types annotated.The full graph is divided into bounded-size subgraphs that serve as training and inference units.
- 3.3 Architecture of DUPIN: The base model is a graph attention network that weights neighboring entities and incorporates edge-type embeddings derived from event embeddings.For multiple events connecting two nodes, the edge representation is the average of their event embeddings.
- 3.3 Architecture of DUPIN: After graph masked pretraining, DUPIN attaches a classifier for downstream tasks and lowers the base-model learning rate to preserve pretrained context representations.The classifier adapts to the downstream task while alterations to pretrained cells are minimized.
4 Evaluation
DUPIN is evaluated across diverse attack sources and baselines, with results showing strong detection, few-shot performance, efficiency, and practical forensic utility, alongside graph-temporal limitations.
- Evaluation Design: Cross-validation separates attack campaigns by dataset source to test whether models focus on fundamental attack contexts rather than shared red-team or victim-environment cues.The evaluation is designed to better reflect evolving and diverse real-world surveillance conditions.
- RQ1: Effectiveness: DUPIN outperforms all baselines in TPR, TNR, and AUROC, improving AUROC by 21% over the best baseline and leading in 19 of 25 attack sets.It also achieves the highest true positives in 15 attack sets across all four dataset sources.
- RQ1: Effectiveness: With training on remaining attack sets from the same dataset source, DUPIN achieves a TPR of 86% and a TNR of 80%.This relaxed evaluation excludes the target attack set while retaining the dataset source.
- RQ2: Architecture Comparison: DUPIN improves AUROC by 5% over GCN and maintains more balanced TPR and TNR than the compared non-attentive GNNs.GIN has higher false negatives, while GCN has higher false positives in the reported comparison.
- RQ2: Baseline Comparison: DUPIN outperforms VELOX by 17% in AUROC, despite VELOX outperforming existing unsupervised baselines.Temporal features can improve performance, but DUPIN still achieves the highest overall accuracy in the temporal-feature comparison.
- RQ3: Ablation Study: Pretraining and few-shot learning remain effective with limited attack data: DUPIN can outperform baselines using as few as 3 to 5 APT sets.In some cases, a single attack set is sufficient to exceed baseline performance.
- RQ4: Efficiency: Pretraining adds little runtime complexity, while total processing remains faster than log accumulation; log parsing and graph construction are the main bottlenecks.The reported accumulated runtime rates are 25.58% and 15.55%, and masked learning rates are 0.24% and 0.38%.
- Case Study: Case studies show DUPIN can identify compromise vectors, continuing attack behaviors, and data leakage from raw logs, but graph representations can lose event-time order.The lost temporal relation reduces confidence in at least one attack case, while temporal ordering embeddings are left out of scope.
5 Related Work
Attack forensics has shifted from rule-based semantic bridging toward data-driven learning, including sequential, graph, and embedding models. DUPIN addresses dataset imbalance by combining large-scale benign provenance pretraining with few-shot supervised adaptation on limited attack data.
- Audit analysis research increasingly emphasizes effectiveness after improvements in scalability made forensic analysis more feasible.
- Rule-based systems bridge low-level artifacts and human threat knowledge by defining prerequisites or clustering related behaviors.
- Data-driven attack-forensics systems use sequential models, graph learning, and behavioral embeddings, but dataset imbalance remains a major limitation.
- DUPIN combines large-scale self-supervised pretraining on benign provenance with few-shot supervised adaptation on a small attack corpus.
6 Limitations
DUPIN’s supervised learning limits detection to attack types observed during training, making performance dependent on attack coverage and requiring attack-data collection despite extensive pretraining.
- DUPIN can detect only attack types previously observed during supervised learning, leaving rare attacks difficult to identify.
- DUPIN’s effectiveness depends on attack coverage in the dataset, although extensive pretraining reduces the data-collection effort.
7 Conclusion
DUPIN combines massive self-supervised pretraining on benign audit logs with few-shot supervised fine-tuning on attack logs, and outperforms several baseline and ablated approaches across 25 APT attack sets from four sources.
- DUPIN applies massive self-supervised pretraining on benign audit logs followed by few-shot supervised fine-tuning on attack logs.
- Across 25 APT attack sets from four sources, DUPIN outperforms one-class learners, alternative GNN architectures, and ablated models.
9 Ethical Considerations
The authors identify ethical risks because attack datasets could be misused, and restrict their stated use to defense research under a code of conduct.
- Attack datasets carry a risk of misuse, so the authors state that acquired attack knowledge is used solely for defense research.
- Subsequent use of the code, datasets, or artifacts is expected to follow proper attribution and ethical-use requirements.
10 Open Science
The authors release code, replication instructions, and an audit-analysis API toolkit, while requiring users to obtain original audit-log datasets from their repositories.
- The authors release source code and instructions for replicating their evaluation results.
- Original audit-log datasets must be downloaded from their original repositories.
- An API toolkit supports automated parsing of raw logs and provenance-graph construction with limited configuration effort.
A The performance of DUPIN on the relaxed cross-validations.
Relaxed cross-validation evaluates each attack set with a model trained on other data from the same source. DUPIN and AirTag improve substantially, while the setup raises evaluation-design concerns.
- Relaxed cross-validation trains each evaluation model on same-source data while excluding the target attack set.
- +13% TPR and +14% TNR are reported for DUPIN, while AirTag gains +38% TPR and +22% TNR.
- +6% TPR and -1% TNR are reported for MAGIC, whose scores are described as slightly affected.
- DUPIN and all baselines achieve higher true positive rates in this training configuration, underscoring the impact of cross-validation design.
B The performance of DUPIN on the non-pretrained model.
DUPIN’s pretrained model improves AUROC across most evaluated cases, while the non-pretrained model trails the strongest baseline in TPR and shows only a small AUROC difference.
- 18 of 25 cases show increased AUROC, including 11 with improvements exceeding 20%.
- The non-pretrained model achieves 61% TPR, below Magic’s 65% TPR.
- The non-pretrained model differs from the strongest baseline by only 3% in AUROC.
C Usage of Darpa Trace Dataset.
The DARPA TRACE data provide Linux and Windows tracepoints for attack and benign sets, while separate trace durations are used for DUPIN pretraining.
- The DARPA TRACE datasets include 9 Linux tracepoints and 6 Windows tracepoints across the E3 and E5 versions.
- A dozen attack sets are collected from official DARPA TRACE documentation, with each benign set derived from its corresponding attack tracepoint.
- DUPIN pretraining uses 5 Linux and 6 Windows tracepoints, each spanning over one week.
- Pretraining excludes trace durations overlapping the periods used to construct attack and benign sets, ensuring strict separation.
D Subdivided Results in Our Evaluations
The evaluation materials describe full results for several baseline models and a random provenance-graph subgraph discovery procedure with bounded growth and coverage over all audit-log nodes.
- Full results are provided for Flash, Kairos, GIN, GCN, VELOX, and TAPAS models.
- Each discovered subgraph grows by randomly selecting neighboring nodes until reaching size γ, or stops after δ failed expansion attempts.DUPIN uses γ = 40 and δ = 100.
- The random subgraph discovery algorithm iterates over every provenance-graph node and attempts one single-graph discovery from each starting node.
- The procedure returns the discovered subgraphs and labels attack and benign subgraphs only afterward.