PaperlayerView exact revision on arXiv

Source-linked AI summary

CyberClear: A Benchmark for LLM Agent Systems on APT Attack Chain Provenance

Qi Chen, Fushuo Huo, Hangli Shen, Jingcai Guo, Shuhao Li, Guang Cheng

arXiv:2609.32424v1cs.CRcs.AI

TL;DR

Existing cybersecurity benchmarks leave agents’ ability to reconstruct complete APT attack campaigns from complex security logs insufficiently evaluated. CyberClear tests this task, and CyberProvenance adds evidence accumulation, execution-based validation, and feedback-guided refinement. Evaluations report improved attack-chain reconstruction, while complete reconstruction remains challenging for current systems.

  • Problem

    Existing agent benchmarks mainly target vulnerability discovery, exploitation, and security analysis, leaving complete APT attack-chain provenance from realistic long-context logs insufficiently evaluated.

    Source: §1 · p. 1

  • Method

    CyberClear benchmarks single-step and multi-stage APT provenance from long-context logs, while CyberProvenance augments agents with evidence accumulation, execution-based validation, and feedback-guided refinement.

    Source: §1 · pp. 2–3

  • Results

    Evaluations report that CyberProvenance validates attack behaviors continuously and reconstructs complete attack chains, while advanced agent systems still face substantial challenges in evidence reasoning and reconstruction.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    CyberClear assesses whether agents can identify attack evidence, infer progression, and represent complete chains in provenance graphs without prior attack clues.

    Source: §1 · p. 2, §3.1 · p. 4

    Complete APT attack-chain reconstruction remains challenging for current agent systems.

    Source: §5.2 · p. 9

Abstract

from arXiv · show

Large language model agents have demonstrated promising capabilities in cybersecurity tasks, yet their ability to reconstruct complete Advanced Persistent Threat attack campaigns from complex security logs remains largely unexplored. Existing cybersecurity benchmarks for agents mainly focus on vulnerability discovery, exploitation, and security analysis tasks, leaving the evaluation of attack chain provenance under realistic security logs insufficiently studied. To address this gap, we introduce CyberClear, a benchmark for evaluating LLM agents and advanced agent systems on APT attack chain provenance from long-context security logs. CyberClear covers both single-step attacks and multi-stage attack chains, requiring agents to identify attack evidence, infer attack progression, and generate provenance graphs containing entities, causal relationships, MITRE ATT&CK techniques, and forensic evidence. To enable comprehensive evaluation, we develop an evaluation method tailored to APT attack chain provenance. Unlike conventional text similarity metrics that focus on surface-level matching, our evaluation examines whether reconstructed graphs preserve the semantics of attack chains across single-step behavior correctness, multi-step behavior identification, temporal and causal consistency, entity and relationship fidelity, and overall attack narrative consistency. Advanced multi-agent systems powered by state-of-the-art LLMs still struggle on CyberClear, motivating us to propose CyberProvenance, an agent cyber harness designed for multi-agents that augments LLM agents with evidence accumulation, execution-based validation, and feedback-guided refinement mechanisms for reliable attack-chain provenance. Extensive evaluations on CyberClear demonstrate the effectiveness of CyberProvenance in improving evidence reasoning, execution-grounded validation, and complete APT attack chain reconstruction.

1 INTRODUCTION

Existing agent benchmarks and evaluations do not directly test whether agents can independently reconstruct complete APT attack chains from long-context security logs. CyberClear evaluates that capability, and CyberProvenance is proposed to address the challenges current agents face.

1 INTRODUCTION

  • Prior evaluations largely emphasize attack detection or investigations guided by prior clues, leaving independent discovery of dispersed evidence and complete chain reconstruction without a unified evaluation setting.Reconstructing provenance requires finding relevant evidence in long, noisy logs, relating entities and activities, and inferring temporal dependencies across attack stages.Source: §1 · pp. 1–2
  • CyberClear evaluates agents on 450 instances averaging 531K log characters, spanning single-step and multi-stage attacks without prior attack clues.Agents must identify evidence, infer progression, and generate provenance graphs containing entities, causal relationships, MITRE ATT&CK techniques, and forensic evidence.Source: §1 · p. 2
  • Its DOT-based LLM judge evaluates reconstructed graphs across five dimensions, including behavior correctness, temporal and causal consistency, entity and relationship fidelity, and narrative consistency.The method ignores irrelevant differences in graph identifiers and visual styles.Source: §1 · p. 2
  • CyberProvenance combines evidence accumulation, execution-based validation, and feedback-guided refinement; evaluations report continuous behavior validation and complete attack-chain reconstruction.The evaluation also identifies iterative refinement, evidence consistency, and effective use of cybersecurity-specific verification tools as limitations of current agents.Source: §1 · pp. 2–3

2 RELATED WORK

Prior attack-provenance approaches range from rule-based systems to graph learning and LLM agents, but leave challenges in context, granularity, or clue dependence. Existing agent benchmarks mainly assess vulnerability-related tasks rather than reconstruction of attack chains from security logs.

APT Attack Chain Provenance

  • Rule-based provenance systems use expert rules or TTP mappings, while graph-learning methods capture entity dependencies but may provide insufficient context or overly coarse investigation results.The rule-based systems discussed include SLEUTH, HOLMES, and CONAN.Source: §2 · p. 3
  • Recent LLM-based provenance approaches either identify only relationships tied to attack indicators or rely on predefined attack clues during evaluation.The paper notes that these constraints make them difficult to apply to realistic scenarios requiring reconstruction of the complete attack process from logs.Source: §2 · p. 3

Cybersecurity Benchmarks for Agents

  • Existing agent benchmarks mainly assess CTF problem solving and real-world vulnerability discovery, exploitation, or remediation, rather than attack-chain reconstruction from security logs.CyberClear is presented as a dedicated benchmark for long-context logs and complex multi-stage APT provenance scenarios.Source: §2 · p. 3, Table 1 · p. 3

3 CYBERCLEAR BENCHMARK

CyberClear formulates APT provenance as reconstructing attack entities, relationships, dependencies, and a timeline from defender logs without prior attack clues. Its four-stage construction pipeline produces and reviews provenance graphs, while the benchmark statistics and semantic evaluation dimensions describe its scope and assessment.

3.1 TASK FORMULATION

  • CyberClear requires agents to reconstruct an attack process from defender logs as a provenance graph of entities, relationships, dependencies, and a narrative timeline with behaviors, ATT&CK techniques, and forensic evidence.Agents analyze dispersed log evidence without access to prior attack clues.Source: §3.1 · p. 4

3.2 BENCHMARK CONSTRUCTION

  • Four stages—attack evidence extraction, provenance graph generation, automatic verification, and human expert review—construct each sample from continuous-campaign defender logs.Generated graphs use unified entity formats, are compiled from DOT code, and can be returned for iterative correction when compilation fails.Source: §3.2 · pp. 4–5

3.3 DATA ANALYSIS

  • 450 samples comprise 318 easy, 122 medium, and 10 hard instances; hard samples average 83.2 log files and approximately 7.80 × 10^5 characters.Easy instances are single-step attacks; medium and hard instances contain multi-step chains requiring reasoning over dependencies among attack stages.Source: §3.3 · p. 5

3.4 EVALUATION METRICS

  • The semantic judge scores reconstructed provenance graphs from 0 to 1, averaging across samples, and assesses single-step attack behavior, unordered multi-step identification, temporal and causal chain consistency, and graph-level element consistency.Source: §3.4 · p. 6

4 METHODOLOGY

CyberProvenance assigns evidence gathering, attack validation, and graph checking to specialized collaborating agents. Their workflow accumulates evidence from logs, tests reconstructed steps in execution environments, and iteratively refines unreliable parts of the provenance graph.

CyberProvenance

  • Specialized Log Analysis, Attack Validation, and Check Agents collaborate on evidence accumulation, execution-based validation, and iterative provenance-graph refinement.These capabilities define CyberProvenance’s multi-agent harness.Source: §4 · p. 6

Log Analysis Agent

  • The Log Analysis Agent chunks defender logs, accumulates attack evidence in Evidence Memory, and uses global reasoning to produce an initial provenance hypothesis.Evidence includes temporal events, processes, commands, network activities, and potential ATT&CK techniques; the hypothesis encodes scenario, stages and dependencies, technique mappings, and a graph serialized as DOT code.Source: §4 · p. 6

Attack Validation Agent

  • The Attack Validation Agent retrieves execution specifications for predicted ATT&CK techniques and tests reconstructed steps in isolated environments.Independent techniques use Atomic Red Team; dependency-aware multi-step chains use CALDERA, with execution traces used to report whether each step can be reproduced.Source: §4 · pp. 6–7

Check Agent

  • The Check Agent integrates graph structure, dependencies, and ATT&CK mappings with validation feedback, preserving validated nodes while refining unreliable steps until the graph is consistent with evidence and reproduced steps.The updated hypothesis is revalidated in an iterative prediction, validation, and refinement loop.Source: §4 · p. 7

5 EXPERIMENTS

The experiments compare CyberProvenance with agent frameworks across code-level and semantic metrics, log difficulty, and attack-stage complexity. Results show stronger semantic performance and more robust reconstruction as log volumes and attack chains grow, while a case study illustrates correction of unsupported causal links.

5.1 EXPERIMENTAL SETUP

  • The comparison includes single-agent and multi-agent frameworks, evaluated with multiple LLM backbones, including GPT-5.6-Terra, DeepSeek-V4-Flash, and Qwen3.8-Max.The main experiments also include results with GPT-4o-mini and Gemini-3.5-Flash-Lite in the appendix.Source: §5.1 · p. 7

5.3 PERFORMANCE ANALYSIS ACROSS DIFFICULTY LEVELS

  • All frameworks’ average Semantic Metrics scores decline as input log-file counts rise across CyberClear’s Easy, Medium, and Hard subsets.The trend is associated with increasing difficulty identifying relevant evidence and maintaining coherent attack progression.Source: Figure 5 · p. 7, §5.3 · p. 9

5.2 MAIN RESULTS

  • CyberProvenance achieves the highest Single (0.8897), Strict (0.6477), Detail (0.6124), and Scenario (0.7263) semantic scores.The remaining Strict and Detail gap indicates that complete APT attack-chain reconstruction remains challenging.Source: Table 3 · p. 8, §5.2 · pp. 8–9
  • AgentScope-V2 leads BLEU (0.5035) and CodeBLEU (0.7240), while CyberProvenance leads ROUGE-L (0.6968) and Pass@1 (0.9933).The authors suggest iterative refinement and cybersecurity-specific validation tools contribute to more reliable graph reconstruction.Source: §5.2 · pp. 7–8, Table 3 · p. 8

5.5 CASE STUDY

  • In the case study, CyberProvenance removes unsupported dependencies and reconstructs causal relations more consistently with the ground-truth graph than AgentVerse.Audit evidence shows that Bash, not Python, spawns stty; chmod independently modifies linpeas.sh.Source: Figure 6 · p. 9, §5.5 · p. 9

5.4 PERFORMANCE ANALYSIS ACROSS ATTACK STAGE LEVELS

  • Nearly all methods perform substantially worse on multi-stage than single-stage attacks, while CyberProvenance remains more robust across three LLM backbones.The decline is especially pronounced for CAMEL, MacNet, and LLM-Debate; the analysis reports average Semantic Metrics across five dimensions.Source: Figure 7 · p. 9, §5.4 · p. 9

6 CONCLUSION

The paper concludes that CyberClear evaluates APT attack-chain provenance from long-context security logs and that CyberProvenance combines evidence accumulation, execution-based validation, and feedback-guided refinement. The accompanying statements describe the paper’s AI use, safety scope, reproducibility materials, and appendix contents.

6 CONCLUSION

  • CyberClear evaluates LLM agents on single-step and multi-stage APT attack provenance, while CyberProvenance combines evidence accumulation, execution-based validation, and feedback-guided refinement.The conclusion reports that advanced agent systems still face substantial challenges in evidence reasoning and complete attack-chain reconstruction.Source: §6 · p. 10

AI USE STATEMENT

  • The authors report using generative AI for research-method refinement, coding, literature work, manuscript editing, and figure and table preparation, with outputs reviewed by the authors.They state that reported results were checked against execution outputs and evaluation scripts, and that authors take responsibility for the final content.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • CyberClear uses publicly available security datasets, collects no private user data, and conducts execution validation in authorized, isolated environments with safeguards against destructive or unauthorized external actions.The authors describe the benchmark and framework as intended for defensive analysis, provenance, and agent evaluation.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper says its main text and appendix provide benchmark, protocol, configuration, implementation, data-processing, prompt, and evaluation details intended to support reproduction.Source: REPRODUCIBILITY STATEMENT · p. 10

APPENDIX OVERVIEW

  • The appendix covers additional experiments, experimental details, LLM-as-a-judge evaluation, attack-behavior taxonomy and boundaries, and further case studies.Source: APPENDIX OVERVIEW · p. 16

A.1 EVALUATION WITH ADDITIONAL LLM BACKBONES

Additional experiments examine backbone robustness, harness-component ablations, and agreement between automatic semantic scores and human judgments. Results show variation across LLM backbones, contributions from all three harness components, and strong evaluator-human correlations.

A.3 CONSISTENCY WITH HUMAN JUDGMENTS

  • Automatic evaluator scores correlate with human judgments at 0.9425, 0.9351, 0.9445, 0.9426, and 0.9376 for Single, Loose, Strict, Detail, and Scenario, respectively.Two experts each evaluated graphs three times, with their judgments averaged.Source: §A.3 · p. 16

A.1 EVALUATION WITH ADDITIONAL LLM BACKBONES

  • With Gemini-3.5-Flash-Lite, CyberProvenance achieves the strongest performance across all five Semantic Metrics, while GPT-4o-mini results are generally lower, especially on Strict, Detail, and Scenario.The authors report substantial performance variation across backbones, indicating provenance performance is sensitive to underlying model capability.Source: §A.1 · p. 16, Table 4 · p. 17

A.2 ABLATION STUDIES

  • Removing Execution Validation lowers average Semantic Metrics from 0.7212 to 0.6472; removing Evidence Memory yields 0.6516, and removing Feedback Refinement yields 0.6897.Execution Validation causes the largest average drop; Evidence Memory notably reduces Strict and Detail by 0.0759 and 0.0888 when removed.Source: §A.2 · p. 16, Table 5 · p. 17

A.4 ADDITIONAL PERFORMANCE ANALYSIS ACROSS DIFFICULTY LEVELS

The appendix reports that increasing task difficulty degrades code-level similarity, with larger framework gaps on hard instances. It also finds that AgentScope-V2 and CyberProvenance retain stronger CodeBLEU structural fidelity across single- and multi-stage attacks and different backbones.

A.4 ADDITIONAL PERFORMANCE ANALYSIS ACROSS DIFFICULTY LEVELS

  • BLEU, ROUGE-L, and CodeBLEU are relatively stable on easy and medium instances but decline substantially on hard instances, where framework performance gaps also widen.The decline appears across three LLM backbones, with its extent varying by model; increasing difficulty challenges semantic reconstruction and graph structural consistency.Source: §A.4 · pp. 17–18

A.5 ADDITIONAL PERFORMANCE ANALYSIS ACROSS ATTACK STAGE LEVELS

  • AgentScope-V2 and CyberProvenance maintain consistently high CodeBLEU scores across backbones, while several general-purpose frameworks score lower, especially on multi-stage attacks.BLEU and ROUGE-L are relatively stable across single- and multi-stage settings for most methods, while CodeBLEU varies more.Source: §A.5 · p. 18

B EXPERIMENTAL DETAILS

The experimental details specify configurations for baseline agents and an environment for validating predicted attack steps and chains. The validation setup combines attack execution with host and network telemetry.

B.1 BASELINES

  • OpenClaw is tested as a single agent with up to three self-review rounds and as three agents for evidence extraction, DOT generation, and verification.The multi-agent version restricts web search to ATT&CK identifier and name verification and allows up to three rounds of review and refinement.Source: §B.1 · pp. 18–19
  • AgentVerse uses four expert agents, LLM-Debate uses three agents over two debate rounds, and MultiPersona simulates three task-specific personas with one LLM instance.Source: §B.1 · pp. 19–20
  • LLM-Blender ranks 11 reconstructions from three generator LLMs and fuses the top three, while DyLAN uses dynamic selection, multi-round interaction, and early stopping.Figures 9 and 10 compare BLEU, ROUGE-L, and CodeBLEU across difficulty levels and single-stage versus multi-stage instances.Source: Figure 9 · p. 19, Figure 10 · p. 19, §B.1 · p. 20
  • MacNet uses a directed acyclic collaboration network, while CAMEL and AgentScope-V2 use role-based agents, with AgentScope-V2 exchanging intermediate evidence for refinement.All baselines use temperature 0.2 and top-p 1.0.Source: §B.1 · pp. 18, 20

B.2 ATTACK VALIDATION ENVIRONMENT

  • Atomic Red Team reproduces individual predicted techniques using compatible catalog tests, while CALDERA orchestrates multi-step chains and records execution traces and dependencies.Platform, runtime, and execution settings come from predefined environment configurations.Source: §B.2 · p. 20
  • Ludus provisions isolated virtual machines from predefined templates, while Sysmon, Linux Audit, and Zeek collect host and network observations for comparison with reconstructed steps.Ludus uses snapshots for reproducible starting states; Zeek records traffic on the designated capture interface.Source: §B.2 · pp. 20–21

C DETAILS OF SEMANTIC EVALUATION VIA LLM-AS-A-JUDGE

The evaluation combines step-level semantic scoring with judges for graph details and attack-chain meaning, including temporal progression. Its protocol treats the reference graph as authoritative and evaluates substantive content rather than visual formatting.

C.1 SCORING CRITERIA

  • Each reference attack step is matched to predicted content, classified from Correct to Missing, assigned a raw score from 0 to 100, and normalized to [0, 1].Correct preserves the actor, core action, target, and attack meaning; partial categories mark progressively greater semantic omissions or errors.Source: §C.1 · p. 21
  • Missing means no predicted behavior has meaningful semantic correspondence to the reference event.Source: §C.1 · p. 21

C.2 MULTI-DIMENSIONAL SEMANTIC ASSESSMENT

  • The Single-Step Judge scores preservation of an event’s core action, target, and overall meaning, while the Loose Judge averages semantic coverage of individual multi-step events without temporal order.Loose matching prevents reuse of the same predicted sub-action across multiple reference steps and excludes unsupported actions.Source: §C.2 · p. 22
  • The Strict Judge evaluates temporal progression using the longest consecutively correct prefix of reference steps, stopping at the first partial or incorrect event or an unsupported action inserted before completion.A fully covered chain is penalized for unsupported actions appended after completion; a complete chain without such additions scores 1.Source: §C.2 · p. 22
  • The Detail Judge aligns entities, directed relations, and attributes, weighting them 0.35, 0.45, and 0.20 respectively; visual graph properties are ignored.Partial semantic matches receive weights of 0.75, 0.50, or 0.25, while incorrect, uncovered, or unsupported elements receive zero.Source: §C.2 · pp. 21–22
  • The Scenario Judge scores whether reference and predicted descriptions preserve the same attack process, target, method, progression, and outcome, allowing paraphrase but penalizing contradictions or fabricated objectives.Source: §C.2 · p. 22

C.3 LLM-AS-A-JUDGE PROMPTS

  • Figure 11 presents the evaluator prompt’s role, evidence constraints, and JSON-only response requirements.Source: Figure 11 · p. 23
  • The judge compares reference and predicted DOT graphs, treating the reference as authoritative and requiring judgments to provide supporting evidence from both graphs when available.Source: §C.2 · p. 22, §C.3 · p. 23

I. GENERAL EVALUATION PRINCIPLES

CyberClear’s evaluation protocol assesses evidence-grounded attack provenance beyond wording or graph appearance, combining attack-step judgments with structural graph audits and treating unsupported claims as errors. It measures attack coverage, ordering, structural fidelity, holistic scenario agreement, and provenance-graph compilation success alongside aggregate Pass@1 results.

I. GENERAL EVALUATION PRINCIPLES

  • Reference and predicted graphs are compared for semantic equivalence, ignoring harmless wording and visual differences while rejecting unsupported substantive claims and requiring evidence from both graphs.Step-level judgments allow reasonable paraphrases and generalizations without requiring exact forensic identifiers; equivalent time zones must preserve the event and temporal order.Source: §I · p. 23

II. EVALUATION ROUTING

  • Samples with one reference step use Single-Step, Detail, and Scenario judges; samples with two or more use Loose, Strict, Detail, and Scenario judges.Step metrics assess attack-step coverage and temporal organization, while Detail Judge audits system entities, provenance relations, and attributes; timeline boxes are not system entities.Source: §II · p. 23

III. COMMON 0−−100 SCORING PROTOCOL

  • The shared 0–100 rubric uses five alignment bands, reserves 100 for complete semantic equivalence, and assigns 0 when no meaningful semantic correspondence exists.For Strict and Detail the bands interpret computed scores; for Single-Step, Loose reference steps, and Scenario they guide direct score selection.Source: §III · p. 24

IV. SEMANTIC ALIGNMENT AND FALSE−POSITIVE ACCOUNTING

  • Correct reference coverage and unsupported predictions are judged independently; unsupported substantive actions, entities, relations, objectives, timestamps, or ATT&CK techniques are audited separately.Detail Judge uses weighted semantic precision, recall, and F1, with one-to-one alignment credits of 1.0 for correct matches and 0.75, 0.50, or 0.25 for minor, moderate, or major partial matches.Source: §IV · p. 24

LOOSE JUDGE

  • Loose Judge scores unordered semantic coverage of multi-step reference attacks, whereas Strict Judge evaluates the consecutive correct prefix in chronological order and treats missing, misordered, or early-ended behavior as incorrect.A predicted compound step may cover multiple distinct reference sub-actions; Loose permits nonconsecutive coverage and does not penalize reordering, while Strict requires the next required behavior and no unsupported action intervening.Source: LOOSE JUDGE · pp. 26–27
  • Detail Judge inventories entities, directed provenance relations, and investigation-relevant attributes, then aligns reference and predicted items one-to-one and records remaining substantive prediction items as unsupported false positives.Graph semantics, endpoint types, and labels—not visual style—determine entity and relation types; timeline boxes, step-to-entity evidence links, and step-to-step order links are not system entities or provenance relations.Source: LOOSE JUDGE · pp. 28–29
  • Scenario Judge directly scores the overall meaning of the two scenario descriptions, penalizing unsupported stages, objectives, or techniques according to their impact on the attack narrative.It uses a holistic 0–100 rubric rather than claim counts or precision, recall, and F1; fabricated core stages, targets, objectives, or outcomes should substantially reduce the score.Source: LOOSE JUDGE · p. 30

D.1 COMPLETE LIST OF ATTACK BEHAVIORS

  • CyberClear’s normalized attack-behavior set contains 89 distinct behaviors from 450 instances: 46 single-technique and 43 composite behaviors.Composite behaviors combine multiple MITRE ATT&CK techniques and represent more complex attacker activity patterns.Source: §D.1 · p. 33
  • A behavior is supported when at least 80% of its instances compile successfully, providing a finer-grained view of capability beyond aggregate Pass@1.A generated provenance graph counts as successful when its DOT code compiles correctly.Source: §D.1 · p. 33

D.2 METHOD BOUNDARIES

The method boundaries specify evidence- and execution-grounded validation constraints, while future work proposes adaptive harnesses and broader benchmark coverage.

D.2 METHOD BOUNDARIES

  • Validation uses log evidence and ATT&CK techniques to choose context-compatible implementations, checks dependencies only with current-execution evidence, and replays from the initial snapshot after refinement.Validation is conducted in authorized private environments with blocked external network access, predefined target scopes, and non-destructive execution constraints.Source: §D.2 · p. 33

D.3 FUTURE WORK

  • Future work will explore adaptive harnesses that optimize evidence retrieval, validation, tool selection, and agent collaboration based on accumulated experience.Source: §D.3 · p. 33
  • Planned benchmark extensions include more campaigns, behaviors, longer chains, diverse environments, noisier and inconsistent observations, and finer-grained dependency and evidence-alignment annotations.The authors also plan to evaluate emerging agent frameworks and foundation models.Source: §D.3 · p. 34

E MORE CASES

Additional cases contrast CyberProvenance’s evidence-supported reconstructions with other systems’ omissions or misalignments; accompanying tables summarize behavior coverage.

E MORE CASES

  • CyberProvenance preserves the bash-to-stty dependency and terminal-configuration step that LLM-Debate omits despite explicit audit evidence.The evidence includes /usr/bin/stty and the recorded terminal dimensions, rows 38 and columns 116.Source: Figure 18 · p. 35
  • CyberProvenance retains the service-modification, privileged-activation, and execution dependencies that MacNet misses, along with important host, service, and network entities.The logs record editing and enabling auditf.service, followed by execution of /usr/bin/auditf.Source: Figure 20 · p. 37
  • CyberProvenance reconstructs the process relations and attack-step progression for terminal configuration that OpenClaw fails to preserve.The logs show python3 spawning bash, followed by /usr/bin/stty with rows 38 and columns 116.Source: Figure 21 · p. 38
  • CyberProvenance corrects AgentScope-V2’s entity misclassification and restores dependencies from a Puppet-triggered download through a cron script to an lftp connection.The final connection is to 172.17.100.122.Source: Figure 22 · p. 39
  • CyberProvenance restores the reconnaissance chain that LLM-Blender incompletely represents, including the connection, chmod operation, and subsequent nmap scan.The logs record a zmcontroller connection to 192.42.1.174:443, chmod 755 /tmp/nmap, and a scan of 172.17.100.0/24.Source: Figure 23 · p. 40
  • 450 instances yield 89 distinct technique sets: 46 single-technique and 43 composite behaviors.Source: Table 6 · p. 41
  • Table 7 compares agent-framework coverage across attack behaviors, marking successful handling with checkmarks and failures with crosses.Source: Table 7 · p. 42

Related papers

Loading 2609.32424v1…