PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Predefined Sinks: Security-Aware Dependency Analysis for LLM Agents

Hang Cui

arXiv:2610.03014v1cs.CRcs.AI

TL;DR

Sensitive-operation identity alone cannot determine the security implications of LLM-agent code. AgentSecGraph builds candidate-centered dependency graphs that add agent-specific and security-context evidence. In held-out cases, these graphs preserve more reference context and expose observed guards more completely than reduced views.

  • Problem

    Sensitive-operation identity alone cannot determine whether an operation is influenced by agent-facing inputs or how its security context affects interpretation.

    Source: §1 · p. 2

  • Method

    AgentSecGraph treats each sensitive operation as an analysis anchor and builds a candidate-centered Security-ADG containing agent relevance, provenance, dependencies, trust context, guards, privilege information, and external-effect semantics.

    Source: §1 · p. 3, §3.1 · p. 5

  • Results

    In nine held-out cases, Security-ADG preserves 91.1% of reference context and all five observed guards, versus 20.0% and 0/5 for sink-only and 40.0% and 0/5 for simplified ADG.

    Source: §7 · p. 18

  • Takeaways & Limitations

    Sensitive operations remain useful analysis anchors, but dependency and security-context evidence yields a more faithful interpretation of security-critical behavior.

    Source: §7 · p. 18

    The analysis is intentionally incomplete: Python recovery is local and AST-based, TypeScript and JavaScript use weaker lexical tracing, and deployment-dependent exposure and privilege may be missing or imprecise.

    Source: §5.3 · p. 15

Abstract

from arXiv · show

Large language model (LLM)-based agents increasingly connect model-generated decisions to security-sensitive software capabilities such as command execution, filesystem access, network communication, browser control, and external tools. Existing analyses often use predefined sensitive operations as anchors, but operation identity alone is insufficient to determine security implications. We present AgentSecGraph, a security-aware static analysis framework that constructs a candidate-centered Security-Aware Agent Dependency Graph (Security-ADG) for each security-sensitive operation. It augments operation identity with agent relevance, source and dependency evidence, trust-boundary context, guard evidence, and external-effect semantics. We further introduce AgentSecBench, a corpus of 67 real-world LLM-agent repositories spanning 11 ecosystems and 37,542 source files. The current analyzer identifies 23,866 static security-sensitive operation candidates across 65 repositories and emits one Security-ADG artifact per candidate. Corpus-wide analysis recovers source-to-operation dependency evidence for 9,821 candidates (41.15%) and potential guard evidence for 3,075 (12.88%), completing in 50.8 minutes. Using a separate reproduction-backed evaluation layer, we establish 22 security-sensitive behaviors across 13 repositories: one confirmed vulnerability, one pending disclosure candidate, and 20 guarded behaviors. In nine held-out cases, Security-ADG preserves 91.1% of the reference context and all five observed guards, compared with 20.0% for a sink-only view and 40.0% for a simplified ADG. These results show that security-aware dependency and contextual evidence enable distinctions that cannot be recovered from sensitive-operation identity alone.

1 Introduction

AgentSecGraph evaluates security-sensitive operations in context rather than treating operation identity as sufficient evidence. The paper introduces a repository corpus and a reproduction-backed evaluation to measure the evidence recovered by this approach.

1 Introduction

  • LLM agents connect model-generated decisions to executable capabilities, so untrusted information may cross software and trust boundaries before influencing external operations.The introduction situates this risk in a growing ecosystem of agent frameworks and external-tool protocols.Source: §1 · p. 1
  • Operation identity alone is insufficient because the same operation may depend on agent-facing input, be disconnected from it, or execute within a security context that changes its interpretation.Framework abstractions can also hide relevant relationships, while security interpretation requires provenance, trust boundaries, guards, privilege context, and external effects.Source: §1 · p. 2
  • AgentSecGraph treats each sensitive operation as an analysis anchor and builds a candidate-centered Security-ADG with agent relevance, provenance, dependencies, trust context, guards, privilege information, and external-effect semantics.It separates evidence about security-critical behavior from a vulnerability verdict.Source: §1 · p. 3
  • AgentSecBench spans 67 repositories, and the analyzer emits 23,866 candidate graphs across 65 repositories, recovering dependency evidence for 41.15% and potential guard evidence for 12.88% of candidates.The full-corpus run completes in 50.8 minutes.Source: §1 · p. 3
  • 91.1% of reference context and all five observed guards are preserved by Security-ADG, versus 20.0% and 0/5 for sink-only and 40.0% and 0/5 for simplified ADG.The held-out comparison covers nine evidence-backed cases.Source: §1 · p. 3

2 Motivation and Problem Formulation

LLM-agent security analysis must interpret sensitive operations alongside agent-specific dependencies and security context. The section distinguishes operation candidates, security-critical behaviors, and vulnerabilities, and defines the evidence a static analysis should recover.

2.1 Agent-Specific Security Semantics

  • Agent-facing parameters have different security implications from unrelated internal parameters even when both reach the same low-level API, so framework-induced relationships must be interpreted with ordinary program dependencies.Tool decorators, workflow nodes, dynamic registration, plugins, and inter-agent handoffs can hide those relationships.Source: §2.1 · p. 4, §2 · p. 4

2.2 Why Sink Identity Is Insufficient

  • Three examples share a command-execution operation but differ in whether agent-input dependency remains and whether an allowlist guard is present.One example severs the dependency through a constant overwrite; another retains it while adding a guard.Source: §2.2 · p. 4
  • A security-critical behavior combines a sensitive-operation candidate with agent-specific dependency and contextual evidence; a vulnerability additionally depends on controllability, protections, exposure, authorization, privilege, impact, and exploitability.The three levels are operation candidate, security-critical behavior, and vulnerability.Source: §2.2 · p. 4

2.3 Security Context and Analysis Objective

  • The operation-candidate set is intentionally permissive: matching a sensitive operation does not establish agent control, untrusted-source reachability, inadequate guards, or exploitability.A predefined operation is an analysis anchor, not a vulnerability verdict.Source: §2.2 · p. 4, §2.3 · p. 5
  • For each candidate, the security context records agent relevance, source provenance, source-to-operation dependency, trust boundaries, available privilege context, candidate guards, and external-effect category.These dimensions characterize statically available evidence.Source: §2.3 · p. 5
  • Dependency or trust-boundary evidence does not prove exploitability, and observed guard evidence does not prove that a guard is complete or effective.The objective is to recover context needed to interpret candidates, rather than to make an automatic vulnerability determination.Source: §2.3 · p. 5
  • The analysis objective is to recover auditable evidence about how agent-relevant values may influence sensitive capabilities and the security context around that relationship, not to verify vulnerabilities automatically.Stronger vulnerability determinations are made separately using reproduction and security evidence.Source: §2.3 · p. 5

3 AgentSecGraph

AgentSecGraph treats each security-sensitive operation as an analysis anchor and builds a candidate-centered Security-ADG from agent relevance, dependencies, trust-boundary context, and guard evidence. It separates static evidence recovery from vulnerability assessment and retains uncertainty where runtime or deployment evidence is unavailable.

3.1 Overview

  • AgentSecGraph treats each sensitive operation as an analysis anchor, recovering agent exposure, dependencies, trust-boundary and guard evidence while separating evidence recovery from vulnerability assessment.Its output retains uncertainty when conclusions depend on runtime, deployment, or threat-model evidence.Source: §3.1 · p. 5

3.2 Candidate Anchoring and Framework Normalization

  • Candidate generation permissively anchors potentially relevant operations; supported framework constructs are normalized so exposed tool parameters can be treated as agent-relevant sources.The current implementation supports representative constructs from MCP/FastMCP, LangChain, CrewAI, AutoGen, OpenAI Agents SDK, Semantic Kernel, and LlamaIndex.Source: §3.2 · pp. 5–6

3.3 Security-ADG Construction

  • Each typed, candidate-centered graph retains operation-specific provenance and context, while may_* relations denote static evidence—not reachability, attacker-control, exploitability, or guard-sufficiency proofs.The graph retains evidence needed to interpret one operation rather than reconstructing every program relationship.Source: §3.3 · pp. 6–7, Figure 3 · p. 7, Table 1 · p. 7

3.4 Dependency, Trust-Boundary, and Guard Analysis

  • Backward dependency recovery identifies whether recognized sources may influence an operation, while the analysis also records safeguards and trust-boundary evidence.Python tracing follows local AST dependencies and ends when a value is overwritten by a constant; TypeScript/JavaScript tracing is more conservative and lacks complete alias, type, or control-flow reasoning. Visible may_guard evidence does not establish guard effectiveness, and privilege context does not infer deployment identities or runtime authorization state.Source: §3.4 · p. 8

3.5 Implementation and Analysis Contract

  • The modular pipeline emits candidate-linked graphs, but Security-ADG is an auditable evidence representation, not proof of reachability, exploitability, guard sufficiency, or vulnerability.The current local analysis may leave evidence incomplete for reflection, dynamic registration, metaprogramming, or deployment-dependent exposure and privilege.Source: §3.5 · pp. 8–9

4 Evaluation

The evaluation combines corpus-wide static analysis with reproduction-backed behavior evidence, keeping operation candidates distinct from vulnerability findings. Across held-out cases, the full Security-ADG preserves more reference context than reduced representations, while the corpus analysis shows uneven recovery of dependency and guard evidence.

4.1 Evaluation Populations and Experimental Setup

  • The corpus contains 67 repositories; 51 (76.1%) primarily use Python, and the analyzer has stronger AST-based dependency recovery for Python than lexical analysis for TypeScript/JavaScript.The current full-corpus scan yields 23,866 candidates from 7,214 source files, with candidates in 65 repositories; the frozen pilot cohort and current five-repository scan are separate populations.Source: §4.1 · pp. 9–10, Table 2 · p. 11

4.2 Reproduction-Confirmed Ground Truth

  • The reference set contains 22 reproduction-confirmed behaviors across 13 repositories, with each case linked to its frozen source location, graph instance, and recovered dependency path.Cases were grounded in source inspection and controlled behavioral confirmation; model judgments were not treated as ground truth, and incomplete independent adjudication precludes exhaustive repository-wide claims.Source: §4.2 · p. 11

4.3 RQ1: Corpus-Wide Coverage and Scalability

  • 41.15% of candidates contain recovered source-to-operation dependency evidence and 12.88% expose potential guard evidence across the full corpus.The analyzer produced artifacts for all 23,866 candidates across 67 repositories without reported construction failures; these rates do not estimate vulnerability prevalence or repository-wide recall.Source: §4.3 · pp. 12–13, Table 3 · p. 12

4.4 RQ2: Reproduction-Confirmed Evidence

  • One of 22 reproduced behaviors met the vulnerability criteria, one remained a pending disclosure candidate, and 20 were guarded under evaluated conditions.Reproducing a sensitive behavior alone does not establish a vulnerability; deployment exposure, authorization, privilege, guards, and impact remain relevant.Source: §4.4 · p. 13

4.5 RQ3: Beyond Sink Identity

  • Security-ADG distinguishes similar operations through source dependencies, guards, validation or effectful paths, and differing agent or trust contexts.Examples include an agent-facing parameter reaching command execution versus being overwritten by a constant, and paths accompanied by allowlists or sandboxing.Source: §4.5 · p. 13

4.6 RQ4: Context Preservation

  • 91.1% of applicable reference context and all five observed guards are preserved by Security-ADG, versus 20.0% for sink-only and 40.0% for simplified ADG.These are context-preservation measures, not vulnerability-detection accuracy; guard visibility does not establish that the guards are sufficient.Source: Table 5 · p. 14, §4.6 · p. 14

5 Discussion and Threats to Validity

Predefined sensitive operations remain useful anchors, but operation identity alone cannot establish agent influence, trust-boundary crossings, or protections. Security-ADG is an evidence representation for interpreting behavior, not a vulnerability verdict; its findings are bounded by incomplete static analysis, corpus and case selection, and the evaluated samples.

5.1 Interpreting “Beyond Predefined Sinks”

  • Security-ADG augments predefined operation anchors with agent semantics, provenance, dependencies, trust-boundary, guard, and external-effect evidence rather than replacing sink specifications.The representation study found that operation identity alone preserves substantially less independently supported security context than the full Security-ADG.Source: §5.1 · p. 15

5.2 From Security Evidence to Vulnerability

  • A reproduced security-sensitive behavior is not necessarily a vulnerability, because validation, authorization, sandboxing, argument construction, deployment assumptions, or intended use may constrain it.Security-ADG records evidence for staged review; reproduction and security review examine cases that warrant further investigation.Source: §5.2 · p. 15

5.3 Limitations and Threats to Validity

  • Static-analysis results are incomplete: Python recovery is local and AST-based, TypeScript/JavaScript tracing is weaker, and dynamic constructs or deployment context can yield missing or imprecise evidence.No explicit privilege-context evidence was recovered corpus-wide, and identical exported source, dependency, and trust-crossing counts are not treated as independent recovery measurements.Source: §5.3 · p. 15
  • AgentSecBench’s 67 repositories are not statistically representative, and Python’s dominance plus stronger Python analysis means aggregate results are not language-independent performance estimates.Repository availability, selection criteria, and open-source access shape the corpus.Source: §5.3 · p. 16
  • The 23,866-candidate full-corpus population cannot be compared directly with the historical 531-candidate pilot cohort because scanner versions differ.Re-running the current scanner on the same five pilot snapshots yields 508 candidates after language-scoping and AST-filtering changes.Source: §5.3 · p. 16
  • Corpus evidence frequencies do not estimate vulnerability prevalence or recall, and the 22-behavior reference set and nine-case representation study do not support exhaustive detection claims.The 91.1% context completeness and 5/5 guard visibility figures describe evaluated cases; timing reflects one run, and the reproduction set is not an unbiased prevalence estimate.Source: §5.3 · p. 16

6 Related Work

Prior work studies agent attacks and evaluations, program analysis, enforcement, and dependency representations. AgentSecGraph addresses a complementary objective: static recovery of auditable, candidate-centered dependency and security-context evidence for sensitive behavior in real-world agent source code.

6.1 Security Evaluation of LLM Agents

  • Agent-security studies have emphasized attack generation, adversarial evaluation, and vulnerability benchmarking, while AgentSecGraph recovers static dependency and security-context evidence from real-world source code.The related work also covers indirect prompt injection [13] [24] [37], agent-environment attacks and defenses [9], unsafe agent–tool interactions [26], and broader security benchmarks [38] [1] [19].Source: §6.1 · p. 16

6.2 Program Analysis for Agent Security

  • Unlike approaches centered on predefined source–sink relations, AgentSecGraph uses sensitive operations as candidate anchors and adds agent semantics, provenance, dependency, trust-boundary, guard, and external-effect evidence.Related approaches include dynamic AgentFuzz, static TaintP2X, and Agent Audit’s agent-aware data-flow and credential, configuration, and privilege analysis [39].Source: §6.2 · p. 17

6.3 Security Enforcement for LLM Agents

  • Runtime enforcement systems impose policies, whereas AgentSecGraph statically records evidence about trust boundaries, guards, available privilege context, and external effects for auditing existing software.The compared systems include isolation [34], trusted-control-flow separation [8], least-privilege tool-call policies [28], execution boundaries [21], and runtime attribution [15].Source: §6.3 · p. 17

6.4 Agent Dependency Representations and Positioning

  • The candidate-centered Security-ADG differs in granularity from AgentFlow and Agent-BOM by augmenting one sensitive operation with context needed to interpret it.The comparison positions taint analyses as recovering paths, agent-aware analyzers as adding security semantics, AgentFlow as recovering framework dependencies, and Agent-BOM as reconstructing runtime context.Source: §6.4 · pp. 17–18, Table 6 · p. 17

7 Conclusion

AgentSecGraph treats sensitive operations as anchors for candidate-centered Security-ADG representations that separate detection from security interpretation. In held-out cases, it preserved more reference context than reduced representations, while corpus-wide and reproduction-backed analyses characterized candidates and behaviors without equating them with vulnerabilities.

7 Conclusion

  • AgentSecGraph constructs candidate-centered Security-ADG representations that preserve agent relevance, provenance, dependencies, trust boundaries, candidate guards, available privilege information, and external effects.This separates operation detection from security interpretation.Source: §7 · p. 18
  • 41.15% of candidates had source-to-operation dependency evidence and 12.88% had potential guard evidence in the 23,866-candidate corpus-wide analysis.The analysis covered AgentSecBench’s 67 frozen repositories and completed in 50.8 minutes.Source: §7 · p. 18
  • The reproduction-backed reference set established 22 behaviors: one confirmed vulnerability, one pending disclosure candidate, and 20 guarded behaviors that did not meet vulnerability criteria.The separate evaluation layer distinguishes behavior evidence from vulnerability conclusions.Source: §7 · p. 18
  • 91.1% context completeness and all five observed guards were preserved by Security-ADG, versus 20.0% context completeness for a sink-only view and 40.0% for a simplified ADG, neither of which exposed guards.These are held-out representation results.Source: §7 · p. 18

Data and Artifact Availability

The appendix documents the released artifacts and reproducible corpus, then details framework normalization, model-assisted triage, held-out representation evaluation, and full-corpus characterization. The corpus aggregates describe static evidence and candidate populations, not vulnerability counts or independently validated recovery dimensions.

Data and Artifact Availability

  • The implementation, pipeline, experiment configuration, evaluation scripts, and documentation are publicly released under Apache License 2.0, with reproduction details for ongoing coordinated disclosures withheld pending remediation or disclosure.The corpus inventory uses immutable repository commits and a frozen manifest; repositories were selected for substantive agent logic and analyzable source code, excluding duplicates, mirrors, and trivial examples.Source: Data and Artifact Availability · p. 18, A AgentSecBench Repository Inventory… · p. 21

B Framework Normalization and Matching Rules

  • Framework adapters normalize matched constructs into records containing framework family, semantic role, callable or program symbol, exposed parameters, and matching evidence and confidence.Frozen selectors cover constructs such as decorators, calls, registrations, wrappers, workflow nodes, and tool classes; adding adapters does not change the candidate-centered graph schema.Source: B Framework Normalization and Matching… · pp. 21–22, Table 8 · p. 22

C Generative-AI-Assisted Candidate Triage

  • Five model raters independently processed 150 candidate tasks, producing 750 valid task–rater responses used only for pre-annotation and disagreement-based triage, not ground truth.Tools, browsing, retrieval, memory, and code execution were disabled; panel outputs were excluded from graph construction and vulnerability adjudication.Source: C Generative-AI-Assisted Candidate… · pp. 22–23

D Held-Out Context-Preservation Study

  • Across nine held-out cases with five applicable context dimensions each, the detailed study reports micro totals of 9/45 = 20.0%, 18/45 = 40.0%, and 41/45 = 91.1%, with concrete guard evidence exposed in all five guarded cases.Cases were supported by controlled local monkeypatch proofs or source-level construction proofs; these measurements concern representation retention, not vulnerability-detection accuracy.Source: D Held-Out Context-Preservation Study · p. 23, Table 12 · p. 24

E Corpus-Wide Characterization and Scalability

  • The current full-corpus run identifies 23,866 static candidates from 7,214 source files; 65 of 67 repositories contain candidates, and all 67 complete extraction and graph construction without reported skips or failures.The candidates are an analysis population, not vulnerability counts; this run is distinct from the historical pilot cohort.Source: E Corpus-Wide Characterization and… · p. 23

E.1 Aggregate Security-ADG Evidence

  • Each candidate receives a candidate-centered Security-ADG artifact, while exported source provenance, dependency-path, and trust-crossing fields share the same count of 9,821 and are reported as one consolidated dependency measure.The run recovered no explicit privilege-context evidence; every candidate’s abstract external-effect node is assigned by operation category, so its 100% coverage is a schema property, not runtime evidence.Source: §E.1 · p. 25, Table 13 · p. 25

E.2 Evidence by Operation Category

Recovered dependency and guard evidence varies substantially by operation category, reflecting what the current analyzer can recover from this corpus rather than differences in vulnerability risk.

E.2 Evidence by Operation Category

  • Dependency evidence appears in 84.14% of filesystem-write candidates and 65.28% of filesystem-read candidates, versus 11.28% of external-tool invocations and 1.87% of browser-control candidates.The category-level percentages describe the current analyzer and corpus only.Source: §E.2 · pp. 25–26
  • Potential guard evidence occurs in 51.29% of filesystem-delete candidates and 41.82% of dynamic-code-execution candidates, but only 1.37% of external-tool invocations.These counts do not estimate category-specific security risk.Source: §E.2 · pp. 25–26
  • The exported trust-crossing and dependency counts are identical, so the trust-crossing column is retained for auditability rather than treated as an independently validated recovery rate.The section cautions that category differences reflect source-analysis rules, coding patterns, and statically recoverable evidence—not relative vulnerability.Source: §E.2 · p. 26

E.3 Scalability

A single-process full-corpus timing run reports a skewed per-repository distribution; formal peak-memory data were not collected for that run.

E.3 Scalability

  • The full-corpus run used one analysis process without a worker pool, and timing excluded tracemalloc profiler overhead.Candidate extraction and graph construction were timed.Source: §E.3 · p. 26
  • The median combined analysis time was 9.562 seconds, while P90 was 203.008 seconds and P95 was 236.191 seconds.The distribution is strongly skewed, so the quantiles show a large-repository tail that an arithmetic mean alone would obscure.Source: §E.3 · p. 26
  • No peak-memory result is reported because formal process-level peak-RSS collection postdated the completed timing run.An earlier tracemalloc pilot measured Python allocation behavior, but its profiler-inflated wall time is not comparable to the full-corpus timing.Source: §E.3 · p. 26

E.4 Population and Reproducibility Notes

The study keeps historical pilot, current five-repository, and current full-corpus candidate populations separate because scanner changes altered extraction on identical snapshots.

E.4 Population and Reproducibility Notes

  • The current scanner finds 508 candidates on the same five repository snapshots represented by the historical 531-candidate pilot, following changes to language scoping and AST filtering.The pilot manifest used an earlier scanner configuration.Source: §E.4 · p. 27
  • The full-corpus run covers 23,866 candidates from 67 frozen repositories; none of the three populations is a vulnerability count or establishes repository-wide recall.The 531-candidate pilot is retained for compatibility with historical pilot and annotation studies, while the current five-repository scan documents scanner-version drift.Source: §E.4 · p. 27

F Inventory-Bounded Evaluation

This auxiliary evaluation uses a finite, held-out source-level operation inventory and distinguishes source evidence from stronger behavioral validation. Its incomplete independent adjudication prevents aggregate recovery metrics, and the inventory cannot establish exhaustive repository-wide recall.

F Inventory-Bounded Evaluation

  • The auxiliary inventory provides a bounded source-level denominator on two held-out repositories, complementing corpus-wide measurements, reproduction-confirmed behavior evidence, and a held-out representation-preservation study.The inventory does not establish that every relevant operation in either repository has been enumerated.Source: F Inventory-Bounded Evaluation · p. 27, §F.5 · p. 29

F.1 Held-Out Operation Inventory

  • The frozen inventory contains 344 operation records across eight categories: 148 from dddabtc/winremote-mcp and 196 from nexu-io/html-anything.Both repositories were held separate from development-time method tuning; the inventory is not intended to estimate ecosystem prevalence.Source: §F.1 · pp. 27–28

F.2 Reference Evidence

  • Reference evidence ranges from source-level constructions and controlled stub or monkeypatch calls to isolated external-effect execution and attack–defense paired tests.These evidence levels are not interchangeable, and model-generated judgments are not reference labels.Source: §F.2 · p. 28

F.3 Prediction-to-Reference Matching

  • A record counts as recovered only when an output matches the frozen repository, commit, file, operation-category, and line-overlap rule.The protocol adds no label-dependent line tolerance or manual per-item matching; the finite inventory does not support repository-wide recall.Source: §F.3 · p. 28

F.4 Evaluation Status and Claim Boundary

  • Because independent adjudication does not cover all 344 records, the evaluator emits no aggregate or category-level recovery metrics, and smaller reviewed samples are not substituted.Completing independent adjudication remains future work.Source: §F.4 · p. 28

F.5 Interpretation and Limitations

  • Inventory-relative recovery would not establish exhaustive repository recall, and a matched operation need not be agent-controlled, exploitable, or vulnerable.Behavior-level and vulnerability conclusions instead depend on the paper’s separate reproduction-backed evidence layer.Source: §F.3 · p. 28, §F.5 · p. 29
Loading 2610.03014v1…