PaperlayerView exact revision on arXiv

Source-linked AI summary

Know the Normal, Track the Attack: Context-Grounded and Stateful LLM Investigation over System Provenance

Lijie Zheng, Ji He, Ying Wang, Huang Zhang, Yulong Shen

arXiv:2609.36494v1cs.CR

TL;DR

Provenance-based intrusion detection outputs are difficult to turn into coherent attack narratives, and direct LLM analysis lacks deployment-specific normalcy and accumulated case context. ANCHOR curates cross-window evidence and grounds investigation in deployment and case context. Across six DARPA TC E3/E5 datasets, it improves IoC recovery and attack-stage attribution over provenance baselines, with dollar-level API cost for full-day investigation.

  • Problem

    PIDS outputs leave analysts to identify critical evidence and organize scattered findings into coherent narratives, while direct LLM analysis lacks deployment-specific normalcy and accumulated case context.

    Source: §1 · pp. 1–2

  • Method

    ANCHOR curates provenance evidence into structured cross-window queues and investigates it using deployment baselines and a confidence-gated Attack-Tracking Cache.

    Source: §1 · p. 2, §5.6 · p. 12

  • Results

    On six DARPA TC E3/E5 datasets, ANCHOR improves overall IoC recovery and attack-stage attribution over state-of-the-art provenance baselines, with the advantage persisting under a fixed LLM backbone.

    Source: §1 · p. 2, §7 · p. 13

  • Takeaways & Limitations

    Full-day investigations at dollar-level API cost support practical, context-grounded investigation across windows.

    Source: §7 · p. 13

    The threat model excludes hardware-level side channels and assumes trusted benign training data, protected system components, and an LLM service not subject to adversarial manipulation.

    Source: §3 · p. 4

Abstract

from arXiv · show

Provenance-based intrusion detection systems (PIDSs) identify suspicious activity in audit streams, but their outputs remain difficult to turn into coherent attack narratives. Direct LLM analyses of local anomalous subgraphs lack deployment-specific normal-behavior knowledge and validated attack state across evidence fragments. This can cause unsupported attack interpretations of routine activities and incorrect attribution of temporally dispersed evidence to attack stages. We present ANCHOR, an investigation-oriented provenance system that combines evidence curation with context-grounded LLM reasoning. It calibrates anomaly judgments by relation type and links anomalous windows through rare relation-role patterns. The resulting evidence queues preserve causal structure, temporal boundaries, and cross-window continuity. The investigator interprets process-centered evidence using two complementary forms of context. Deployment Context combines environment-specific interaction and object baselines with high-risk security knowledge. Case Context uses a confidence-gated Attack-Tracking Cache to maintain investigation state across windows. Correlating current evidence with high-confidence prior findings, ANCHOR incrementally reconstructs attack narratives organized by kill-chain stages. We evaluate ANCHOR on six DARPA Transparent Computing E3/E5 datasets across three operating systems. Controlled evidence-level and end-to-end comparisons show improved overall IoC recovery and attack-stage attribution over state-of-the-art provenance-based baselines. These gains persist under a fixed LLM backbone in our evaluation. ANCHOR processes a full audit day at dollar-level API cost, supporting practical, context-grounded investigation across windows.

1 Introduction

PIDS outputs leave analysts to turn anomalies into attack narratives, while direct LLM investigations lack deployment-specific normalcy and accumulated case context. ANCHOR curates cross-window evidence and grounds investigation in deployment and case context, improving IoC recovery and attack-stage attribution in evaluation.

1 Introduction

  • Direct LLM analysis lacks deployment-specific normalcy and validated prior attack state, contributing to unsupported interpretations of routine activity and incorrect stage attribution.The paper identifies these as two missing forms of context in prior LLM-based investigation.Source: §1 · pp. 1–2
  • Evidence curation addresses massive audit streams and heterogeneous anomaly signals by preserving local provenance structure, temporal boundaries, and continuity across windows.It identifies suspicious interactions and links cross-window activity through relation-role rarity.Source: §1 · p. 2
  • ANCHOR combines an evidence-curation layer with a dual-context investigator that produces stage-structured attack narratives across windows.Deployment Context combines normal interaction and object baselines with high-risk security knowledge; a confidence-gated Attack-Tracking Cache supplies previously admitted evidence and stage assignments.Source: §1 · p. 2
  • Across six DARPA TC E3/E5 datasets, ANCHOR improves overall IoC recovery and attack-stage attribution over state-of-the-art provenance baselines, including under the same LLM backbone.End-to-end investigation remains at dollar-level API cost.Source: §1 · p. 2

Contributions

  • The contributions include relation-aware evidence queues, context-grounded stateful investigation, evaluation across six datasets, and an open-source implementation and labels.The evaluation reports improved IoC recovery and stage attribution and dollar-level end-to-end investigation costs.Source: Contributions · p. 2

2 Background and Motivation

The section examines why provenance detection outputs and direct LLM analysis of anomalous subgraphs do not by themselves produce reliable attack narratives, then derives ANCHOR’s evidence-organization and context-grounding principles.

2 Background and Motivation

  • The section frames the background analysis around PIDS output granularity and OCR-APT’s limitations as an LLM investigation case study.It uses those analyses to derive design principles for bridging the semantic gap.Source: §2 · p. 2

2.1 Why Not Just Feed a PIDS to an LLM?

  • Graph-, node-, and subgraph-level outputs respectively lack localization, causal and temporal associations, or readable narratives; useful investigation evidence needs causal links, temporal boundaries, and cross-window continuity.The section identifies no existing output form as satisfying all three requirements at once.Source: §2.1 · p. 3, Table 1 · p. 3

2.2 The Necessity of Context-Grounded Investigation

  • OCR-APT analyzes subgraphs with generic knowledge and no deployment baseline or confirmed case state, producing unsupported interpretations and weak IoC stage attribution.Its report correctly attributes 3/16 E3-CADETS IoCs and 3/7 E3-THEIA IoCs to their attack stages.Source: §2.2 · pp. 3–4, Table 2 · p. 3

2.3 Design Principles

  • ANCHOR’s design principles pair relation-aware, temporally bounded cross-window evidence queues with retrieved deployment baselines and a queue-scoped cache updated only by highly suspicious findings.The cache retains high-confidence evidence, interactions, the evolving story, and stage assignments to limit uncertainty propagation.Source: §2.3 · p. 4

3 Threat Model

The threat model covers APT activity against a target host captured by standard kernel-level auditing, under assumptions about trusted system components and the LLM service.

3 Threat Model

  • The scope is APT activity seeking extended unauthorized access or control of a target host, as captured by standard kernel-level auditing; hardware-level side channels are excluded.The detector’s benign training data and normalcy baselines are assumed to come from a trusted environment, excluding data and model poisoning.Source: §3 · p. 4
  • ANCHOR software, provenance capture, and the operating system are assumed protected from direct compromise by system-hardening techniques.These protections are described as orthogonal and complementary to the work.Source: §3 · p. 4
  • The threat model assumes that the LLM service is not subject to adversarial manipulation.Source: §3 · p. 4

4 ANCHOR’s Architecture

ANCHOR combines provenance evidence curation with stateful, dual-context LLM investigation to produce attack narratives organized by stage. Its pipeline converts audit logs into evidence queues and fragments, then carries high-confidence findings across windows.

4. ANCHOR’s Architecture

  • ANCHOR converts audit logs into provenance graphs and interaction features, then curates evidence and applies dual-context investigation to produce stage-structured narratives.Interactions are represented with entity types and semantic embeddings; relation type is retained separately. Graph events are partitioned into 15-minute windows.Source: §4 · p. 4, §4.1 · p. 4, §4.1.1 · p. 4

4.2.1 Learning Normal Interaction Patterns

  • A temporal relational encoder learns benign interaction patterns, using entity memory and relation-conditioned neighborhood aggregation to reconstruct interaction relation types.Higher reconstruction loss signals greater deviation from learned benign patterns.Source: §4.2.1 · p. 5

4.2.2 Relation-Aware Anomaly Judgment

  • Relation-specific loss thresholds and relation-role rarity jointly identify suspicious interactions while accounting for different normal loss distributions across relation types.The threshold is the relation-specific mean plus a benign-calibrated multiple of its standard deviation.Source: §4.2.2 · p. 5

4.2.3 Constructing Evidence Queues

  • Rare relation-role keys link windows into evidence queues, and short-gap intermediate windows may be added to preserve context across low-anomaly transitions.Queues are submitted when their aggregate score exceeds a threshold calibrated on benign validation data.Source: §4.2.3 · p. 6

4.3.1 Constructing Deployment Context

  • Deployment Context combines benign interaction and object baselines with threat-intelligence-derived high-risk security knowledge.Rather than build benign Process–File baselines, ANCHOR uses a separate Process–File rule set; network targets are normalized as internal or external.Source: §4.3.1 · p. 6

4.3.2 Maintaining Case Context with the Attack-Tracking Cache

  • For each queue, the Attack-Tracking Cache retains highly_suspicious fragment cards in an Evidence View and pairs them with accumulated case summaries and confirmed stages in a Progression View.The cache is read during current-window analysis, and only highly_suspicious findings are admitted as historical attack evidence.Source: §4.3.2 · pp. 6–7

4.3.3 Context-Grounded Attack Investigation

  • Two-round analysis combines current fragments with retrieved deployment and case context, then confirms suspicious fragments using object baselines before admitting high-confidence cards.Fragments lacking an object baseline are directly upgraded to highly_suspicious.Source: §4.3.3 · p. 7

4.3.4 Stage-Structured Attack Report

  • Confidence-gated evidence supports evolving kill-chain-stage narratives, while analyst-confirmed attacks can enrich high-risk rules and false alarms can refine normal baselines.The narrative maintains accumulated stage assignments for later updates.Source: §4.3.4 · pp. 7–8, §4.3.5 · p. 8

5 Evaluation

ANCHOR is evaluated on six DARPA TC datasets spanning E3/E5 and three operating systems, using controlled comparisons of investigation quality, evidence, curation, cost, ablations, and sensitivity. Across these evaluations, curated evidence and dual-context investigation improve attack narratives, while experiments examine the cost and conditions of those gains.

5.1 Experimental Setup

  • The evaluation uses six E3/E5 datasets across CADETS, THEIA, and ClearScope, and measures narrative IoC coverage and stage-attribution accuracy alongside candidate-generation precision, recall, and F1.Labels are checked against official reports; controlled narrative comparisons include OCR-APT on the two shared datasets and a rerun using its GPT-4o-mini backbone.Source: §5.1 · pp. 8–9, §5.2 · p. 9

5.2 RQ1: Dual-Context Investigation Quality

  • On E3-CADETS, ANCHOR raises IoC coverage from 0.438 to 0.875 and stage accuracy from 0.250 to 0.375; on E3-THEIA, coverage rises from 0.714 to 1.000 and stage accuracy matches at 0.571.With GPT-4o-mini, IoC coverage is 0.875 vs. 0.688 on E3-CADETS and 0.857 vs. 0.714 on E3-THEIA, with higher stage-attribution accuracy.Source: §5.2 · pp. 9–10, Table 3 · p. 9

5.3 RQ2: Impact of Investigation Evidence

  • Across six datasets, ANCHOR’s curated evidence has the highest IoC coverage on four, ties on E5-THEIA, and reaches the highest or tied-highest stage accuracy on every dataset.Under a fixed process-centered one-hop format and investigation protocol, ClearScope coverage drops amid sparse benign semantics and short-lived APK IoCs, but ANCHOR still matches or exceeds alternatives.Source: §5.3 · p. 10, Table 4 · p. 10

5.4 RQ3: Evidence Curation Effectiveness

  • ANCHOR recalls 1.000 on five of six datasets and 0.700 on E3-THEIA, matching or exceeding Kairos recall across all six and achieving higher F1 on five.Kairos has higher precision on both ClearScope datasets; ANCHOR’s three E3-THEIA false negatives reflect sparse evidence or changing relation-role keys across adjacent windows.Source: §5.4 · p. 11, Table 6 · p. 11

5.5 RQ4: Investigation Cost

  • A full-day investigation costs $0.67 on E3-CADETS and $1.31 on E3-THEIA under DeepSeek V4 Flash pricing.The cost exceeds OCR-APT’s cent-level cost because normalcy retrieval, cache-conditioned analysis, and progression updates require additional LLM processing.Source: §5.5 · p. 11, Table 7 · p. 11

5.6 RQ5: Ablation

  • On E3-CADETS, global-threshold anomaly judgment lowers precision from 1.000 to 0.031 and raises false positives from 0 to 124; adding deployment and case context improves narrative results.Normal reconstruction losses differ by relation type, so a global threshold flags benign behavior and misses anomalies. Interaction baselines provide the largest single-step IoC-coverage gain, object baselines add further gain, and the confidence-gated cache reaches full-system results; RGAT and UniMP also produce no false-positive windows while all detector variants retain recall 1.000.Source: §5.6 · p. 12, Figure 5 · p. 12, Figure 6 · p. 12

5.7 RQ6: Hyperparameter Sensitivity

  • Across tested hyperparameter settings, recall remains 1.000; precision falls only with extreme learning rates or undersized GNN/TGN dimensions or neighborhoods, while Word2Vec dimension has no observed effect.The evaluation varies one hyperparameter at a time on E3-CADETS, retaining all four labeled attack windows under every tested setting.Source: §5.7 · pp. 12–13

6 Related Work

Prior provenance detectors and attack-reconstruction systems generally leave analysts with localized anomalies, graphs, or ranked events rather than a cross-window narrative. ANCHOR combines relation-role evidence curation with deployment grounding and persistent case state to address that gap.

Provenance-based intrusion detection

  • Provenance detectors range from graph-level anomaly scoring to node localization and temporal edge reconstruction, but their outputs remain fragmented local anomalies rather than investigation-organized evidence.Examples include StreamSpot [29], Unicorn [13], ProGrapher, ThreaTrace [43], Flash, MAGIC [19], Orthrus, Kairos [5], and VELOX.Source: §6 · p. 13

Attack investigation and reconstruction

  • Attack-reconstruction systems correlate flows, prioritize events, or summarize anomalous windows, yet generally leave human analysts to interpret outputs without accumulated case state across windows.Examples include SLEUTH [16], HOLMES [32], NoDoze [15], PrioTracker [28], ATLAS, NodLink [26], Kairos [5], and Orthrus.Source: §6 · p. 13

LLM-assisted security investigation

  • OCR-APT [2] generates LLM-based reports from provenance subgraphs but lacks deployment-normal behavior grounding and validated cross-window attack state; ANCHOR supplies both and reports better results under the same backbone.ANCHOR extends anomaly rarity to (entity, relation, role) keys and organizes findings into stage-structured narratives.Source: §6 · p. 13

7 Conclusion

ANCHOR curates cross-window evidence and grounds investigation in deployment and case context, improving IoC recovery and attack-stage attribution across six datasets. The study reports no known ethical issues and plans to release its implementation and evaluation artifacts.

7 Conclusion

  • Across six DARPA TC E3/E5 datasets, ANCHOR improves overall IoC recovery and attack-stage attribution over state-of-the-art provenance-based baselines.The advantage persists under a fixed LLM backbone, and full-day investigations have dollar-level API cost.Source: §7 · p. 13

A Ethics Considerations

  • The authors report no known ethical issues and state that experiments use publicly released benchmark datasets collected ethically and containing no sensitive information.Source: A Ethics Considerations · p. 16

B Compliance with the Open Science Policy

  • The authors plan to release ANCHOR, evaluation labels, configurations, and reproduction scripts upon paper acceptance.The labels include corrected window-level annotations, IoC labels, and stage-attribution labels.Source: B Compliance with the Open Science… · p. 16

C Ground Truth Label Corrections

The ground-truth window labels are supplemented or adjusted using the official engagement reports, with changes classified by action type.

C Ground Truth Label Corrections

D Provenance Graph Schema

The provenance graph schema specifies the entity types, interaction relations, and entity attributes that ANCHOR models or retains for feature extraction.

D Provenance Graph Schema

E Dataset Splitting

Dataset days are split into training, validation, and evaluation sets, with each split serving a distinct role in detection, baseline construction, calibration, or performance measurement.

E Dataset Splitting

F Hyperparameter Sensitivity Results

Figure 7 reports the complete per-parameter hyperparameter sensitivity results on E3-CADETS, varying one parameter at a time while keeping the others at their defaults.

F Hyperparameter Sensitivity Results

G Case Study on E5-CADETS

The E5-CADETS case study examines an attack across two days and shows how ANCHOR joins temporally dispersed evidence into one kill-chain narrative. The appendix also specifies the prompts used to investigate fragments and update the global attack story.

G Case Study on E5-CADETS

  • The case study describes an nginx-backdoor attack on two FreeBSD hosts, with reconnaissance and credential-file theft followed by a rerun and C2 establishment.The May 17 activity first fails to deliver a payload via 98.23.182.25, then establishes C2 through 128.55.12.233.Source: G Case Study on E5-CADETS · p. 17
  • Among 15 ground-truth IoCs, ANCHOR recovered 13 and correctly attributed 10, reporting 0.867 coverage and 0.667 stage-attribution accuracy.It investigated 23 candidate windows; 14 contributed suspicious evidence to the final narrative and nine were excluded.Source: G Case Study on E5-CADETS · p. 17
  • Across the May 16–17 attack, the Attack-Tracking Cache linked evidence across a pause, connecting nginx and the new C2 address to one kill-chain case.The report flags nginx on May 16 and attributes its communication with 128.55.12.233 on May 17 to the command-and-control stage.Source: G Case Study on E5-CADETS · p. 17

H LLM Prompt Templates

  • Each LLM request combines a fixed system prompt with a JSON user message carrying runtime evidence, using two placeholders for runtime concatenations.Source: H LLM Prompt Templates · p. 18

Core Prompt 1. FRAGMENT_SYSTEM_PROMPT

  • The fragment prompt compares suspicious evidence with normal interaction baselines and high-risk rules, using prior high-confidence memory for continuity rather than as a substitute for current evidence.Object baselines are supplementary, time is weak evidence, and a common process name alone should not make an action benign.Source: Core Prompt 1. FRAGMENT_SYSTEM_PROMPT · p. 18

Core Prompt 2. STORY_UPDATE_SYSTEM_PROMPT

  • The story-update prompt incrementally decides whether a window continues an attack or adds a stage, retains only evidence-confirmed stages, and updates key entities and the global story.Source: Core Prompt 2.… · p. 19

Related papers

Loading 2609.36494v1…