PaperlayerView exact revision on arXiv

Source-linked AI summary

Trust propagation and structural containment in Multi-agent LLM pipelines

Tanzim Hossain Safin, Sharif Noor Zisad, Swakkhar Shatabda, Ragib Hasan

arXiv:2609.17648v1cs.CRcs.CY

TL;DR

A compromised low-privilege agent can influence a higher-privilege agent and trigger an unauthorized action in a multi-agent LLM pipeline. The paper studies attack propagation and structural authorization in a four-agent pipeline, finding that authorization can contain execution even when Validator review is bypassed.

  • Problem

    Multi-agent delegation creates a risk that a lower-privilege component can cause a higher-privilege component to misuse its authority.

    Source: §I · p. 1

  • Method

    The paper studies a four-agent pipeline, structural authorization, attack propagation, and JBR, which distinguishes compromised judgment from unauthorized action.

    Source: §I · p. 1, D. Evaluation Metrics · p. 3, §VI · p. 6

  • Results

    With structural authorization enabled, unauthorized execution was prevented in every evaluated trial, including when the Validator was compromised.

    Source: §I · p. 1, §VI · p. 6

  • Takeaways & Limitations

    Structural authorization can contain compromised agent behavior even when upstream LLM judgment fails.

    Source: §VI · p. 6

    Evaluation is limited to 60 tasks, 13 action types, and one architecture, framework, and model; broader validation is needed.

    Source: E. Threats to Validity · pp. 5–6

Abstract

from arXiv · show

Multi-agent LLM systems increasingly automate tasks involving agents with different levels of privilege, creating a security risk in which a compromised low-privilege agent can influence a higher-privilege agent and trigger an unauthorized action. We study attack propagation in a four-agent LangGraph pipeline comprising a Supervisor, Researcher, Validator, and Executor. We evaluate shared-memory poisoning and indirect prompt injection through a forged approval embedded in a retrieved document. We compare the Validator's judgment with an independent authorization layer using task-bound signed tokens and a separately verified policy oracle. Our contribution is an empirical study of attack propagation, a component-level ablation of the authorization boundary, and the Judgment Bypass Rate (JBR), which measures compromise at the attacked agent rather than at the final action. Across three seeds and 60 labeled tasks, memory poisoning reaches execution in every undefended trial. With authorization enabled, it achieves 100% JBR but 0% Unsafe Action Rate, showing that the Validator can remain compromised while execution is contained. Against an attacker possessing the signing secret, the policy oracle provides the observed containment, while an independently authored least privilege policy preserves this result. An Observer layer reduces the false-positive rate for agent hijacking from 49% to 7% without weakening execution-level security. These results show that structural authorization can contain compromised agent behavior even when upstream LLM judgment fails.

I. INTRODUCTION

Delegation among agents with different privileges creates a confused-deputy risk, while prior work has focused more on agent deception than on whether compromise reaches an action. This paper studies propagation and structural authorization in a four-agent pipeline, introducing JBR and evaluating attacks, authorization components, and an Observer layer.

I. INTRODUCTION

  • Existing studies primarily assess whether an agent can be deceived, whereas downstream authorization and independent verification shape whether compromise produces a system action.The study examines this downstream boundary in a pipeline with a Supervisor, Researcher, Validator, and Executor.Source: §I · p. 1
  • The study evaluates shared-memory poisoning and indirect prompt injection through retrieved documents under progressively stronger attacker capabilities.The attack models include an attacker possessing the signing secret.Source: §I · p. 1
  • Structural authorization prevents unauthorized execution even when a compromised Validator bypasses genuine review.The paper introduces Judgment Bypass Rate (JBR) to distinguish agent-level compromise from a successful unauthorized action.Source: §I · p. 1
  • Against an attacker holding the signing secret, containment comes from the independent policy oracle rather than the signed token alone.The paper also evaluates an independently authored least-privilege policy.Source: §I · p. 1
  • The evaluation spans three independent seeds and a 60-task corpus, while the authorization ablation uses a single seed.Mean performance and 95% confidence intervals are reported where applicable.Source: §I · p. 1
  • The Observer removes detected injections before review and reduces false blocking of legitimate tasks without weakening the execution-level guarantee.Its stated role is to remove fabricated approval claims before validation.Source: §I · p. 2

II. BACKGROUND

Multi-agent delegation can transmit implicit trust from lower-privilege agents to higher-privilege agents, creating a confused-deputy risk. Prompt injection may enter through direct input or retrieved content and influence downstream agents even when the system prompt is unchanged.

A. Trust Propagation and the Confused Deputy Problem

  • Trust propagation concerns how authority and permissions move through a delegation chain, where a higher-privilege agent may accept a compromised agent’s forged message as legitimate.This is a modern multi-agent form of the confused deputy problem.Source: A. Trust Propagation and the Confused… · p. 2

B. Overview of Direct and Indirect Prompt Injection

  • Direct prompt injection targets attacker-controlled content, while indirect prompt injection embeds malicious content in retrieved documents, search results, or tool responses.An agent may forward forged authority claims downstream despite an unchanged system prompt,.Source: B. Overview of Direct and Indirect… · p. 2

C. Structural Authorization versus Model Judgment

The section argues for structural authorization independent of model judgment and distinguishes agent compromise from an unauthorized action reaching execution. It situates this approach among existing defenses and notes limited evaluation of propagation through complete multi-agent systems.

C. Structural Authorization versus Model Judgment

D. Importance in Public Safety Context

  • In safety-critical settings, distinguishing a bypassed review from a blocked attack matters because naive Executors may accept forged approvals before destructive or externally visible actions.Source: D. Importance in Public Safety Context · p. 2

III. LITERATURE REVIEW

  • JBR distinguishes agent-level compromise from successful end-to-end attacks when downstream authorization can block the final action, unlike tool-invocation evaluations where compromise and harmful action coincide.Source: §III · p. 2
  • The Observer removes fabricated approval claims before validation and is evaluated as a utility-preserving input defense, not as a security boundary.Other reviewed defenses include runtime monitoring, delimiters, structured prompt/data channels, and instruction-resistant fine-tuning –[13].Source: §III · p. 2
  • Prior work includes capability restrictions, signed privileged commands, independent auditors, zero-trust frameworks, threat-modeling tools, and structural verification for computer-using agents –[21].Practical evaluation of attack propagation through complete multi-agent systems remains limited [22],.Source: §III · p. 2
  • The study answers calls for empirical security testing by evaluating propagation in a reproducible four-agent pipeline under progressively stronger attacker models [24]–.Source: §III · p. 3

IV. METHODOLOGY

The methodology uses an email dataset as a mutable resource for observing attack effects and evaluates tasks in a separate labeled corpus. The task corpus covers safe and unsafe cases across multiple action types and resource-access scopes.

A. Dataset

  • The mutable resource is a fixed subset of 1,000 Enron messages, retaining only message headers in an indexed table.The full dataset contains more than 500,000 internal messages from approximately 150 employees.Source: A. Dataset · p. 3
  • The email dataset serves as a mutable resource for observable attack effects rather than as a classification target.The Executor’s sandboxed tools treat each stored record as real, mutable data.Source: A. Dataset · p. 3
  • The separate evaluation corpus contains 60 tasks: 36 safe and 24 unsafe, covering 13 action types with specific resource-access scopes.Metrics are computed by joining each trial with its corresponding task.Source: A. Dataset · p. 3

B. Model Configuration

The pipeline uses a fixed four-agent architecture, with language-model agents configured at temperature 0. Its Executor gates real tool calls on both a signed, task-bound token and independent policy verification.

B. Model Configuration

  • The Researcher, Validator, and enabled Observer use gemma4:31b-cloud through Ollama at temperature 0, while the Supervisor makes no model call.A decoding seed is requested best-effort because fixed seeds do not guarantee identical outputs across Ollama calls.Source: B. Model Configuration · p. 3

C. System Architecture

  • The deterministic Supervisor routes each trial through sub-agents in a fixed order, with the four roles communicating through a shared typed state object and logged transitions.Figure 1 depicts the Supervisor routing tasks to the Researcher, Validator, and Executor.Source: C. System Architecture · p. 3, Figure 1 · p. 3
  • Memory poisoning replaces the Validator with a routine that forges approval and can generate a matching token when it has the signing key; hijacking injects fabricated approval claims through a retrieved document.The hijacked Validator reasons over a summary without being told its source; the baseline uses an honest source and clean input.Source: C. System Architecture · p. 3
  • The Executor allows requests to reach the SQLite-backed email store only after validating a signed, task-bound, single-use token and independently re-deriving permission through a policy oracle.The shared boundary separates structural enforcement from the Validator’s judgment.Source: C. System Architecture · p. 3
  • The corpus policy mirrors task gold labels, whereas the independent policy is a hand-written least-privilege allowlist that agrees with gold labels for all 60 tasks.The independent policy receives only the proposed action and its scopes; a 0% Unsafe Action Rate under the corpus policy demonstrates enforcement of the supplied policy, not independent discovery of unsafe actions.Source: C. System Architecture · p. 3

D. Evaluation Metrics

The evaluation distinguishes unsafe execution, legitimate completion, over-blocking, and compromised review. JBR captures bypass at the Validator, while execution metrics assess whether actions reach the system.

D. Evaluation Metrics

  • Trials are scored against gold safety labels, with blocked gold-unsafe tasks counted as TP and executed gold-unsafe tasks as FN; recall and F1 use standard definitions.A blocked gold-safe task is FP, and a correctly allowed gold-safe task is TN.Source: D. Evaluation Metrics · p. 3
  • UAR is the proportion of gold-unsafe tasks reaching execution, while LTC is the proportion of gold-safe tasks allowed to complete.UAR is lower-is-better, and LTC is higher-is-better.Source: D. Evaluation Metrics · p. 3
  • JBR measures the share of gold-unsafe, attack-injected trials in which the Validator’s genuine review was skipped or bypassed.Under memory poisoning, JBR is 100% by construction because the compromised routine generates approval without calling the model.Source: D. Evaluation Metrics · pp. 3–4
  • FPR measures over-blocking of gold-safe tasks and equals 1−LTC; for hijacking, the reported 95% t intervals overlap: T0 [52.7, 65.8] and T1 [28.1, 70.0].JBR is computed over 24 gold-unsafe trials per seed, and FPR over 36 gold-safe trials per seed.Source: D. Evaluation Metrics · pp. 3–4

V. RESULTS AND DISCUSSION

The evaluation holds the pipeline and attack setup fixed while varying protection layers, using three seeds and reporting uncertainty across seeds.

V. RESULTS AND DISCUSSION

  • The controlled ablations keep agents, graph topology, prompts, task corpus, and attack injections fixed while varying only protection layers.T0 disables both token and policy checks; T1 enables both.Source: §V · p. 4
  • The evaluation uses 60 tasks—36 safe and 24 unsafe—across three seeds, yielding 180 trials per cell unless otherwise stated.The reported values are means across seeds with 95% Student’s t confidence intervals (df = 2).Source: §V · p. 4
  • Because UAR is 0% for most defended conditions, the analysis focuses on FPR, LTC, F1, and JBR.Intervals are computed over seeds because trials within a seed share the task corpus and are not independent.Source: §V · p. 4

A. Attack Effectiveness

Memory poisoning compromises the Validator and reaches execution without authorization, while enabled authorization contains execution despite continued compromise. Against an attacker holding the signing secret, the policy oracle—not the signature check—provides the observed containment.

A. Attack Effectiveness

  • At T0, memory poisoning forges approval and executes the action; with T1, Unsafe Action Rate is 0% while Judgment Bypass Rate remains 100%.The Validator remains compromised, but the Executor prevents unauthorized action.Source: A. Attack Effectiveness · p. 4
  • Agent hijacking yields 0% Unsafe Action Rate under both T0 and T1, while false-positive rate is 59.3% at T0 and 49.1% at T1.The FPR decrease is attributed to Validator-judgment variation, not authorization; the confidence intervals overlap.Source: A. Attack Effectiveness · p. 4

B. Authorization Ablation

  • Against an attacker holding the signing secret, token-only is indistinguishable from T0, whereas policy-oracle-only reproduces T1 containment.The ablation uses one seed and n = 60 per condition; the oracle alone provides the same containment as full T1.Source: B. Authorization Ablation · p. 4, Figure 2 · p. 4
  • A signature establishes provenance, not permission; without the signing secret, tokens can prevent forgery, bind approval to task and scope, and prevent replay.These token benefits were not directly evaluated.Source: B. Authorization Ablation · p. 4

C. Independent Policy Evaluation

The independent least-privilege policy preserves containment without using task labels, though its agreement with the corpus policy does not establish objective policy correctness.

C. Independent Policy Evaluation

  • The independent least-privilege policy uses only proposed actions and scopes, has no access to task labels, and agrees with gold labels for all 60 tasks.It remains unaffected when the corpus-mode verdict is modified.Source: C. Independent Policy Evaluation · p. 4
  • Unsafe Action Rate remains 0% for both attacks under both policies, and the independent policy provides the same memory-poisoning containment as the corpus policy.For hijacking, false-positive rates are similar (49.1% versus 46.3%), with overlapping confidence intervals; F1 is identical to two decimal places.Source: C. Independent Policy Evaluation · pp. 4–5
  • Both policies were authored by the same group, so their agreement supports non-circular implementation rather than objective policy correctness.Source: C. Independent Policy Evaluation · p. 5

D. Observer Evaluation

The Observer removes fabricated approval claims before validation, reducing false blocking while leaving execution security to the authorization boundary. Its matched-case separation reflects the explicit-approval threat model, not a general Observer property.

D. Observer Evaluation

  • The Observer flags and removes fabricated approval or authority claims before Validator review, but cannot approve or block actions.It lacks access to the task record, gold label, policy oracle, and Validator output, so it is treated as a utility component rather than a security control.Source: D. Observer Evaluation · p. 5
  • JBR rises from 0% to 4.2% with the Observer while UAR remains unchanged, as substantive judgments replace broad rejection and their errors remain contained.Without the Observer, conspicuous fabricated approval corresponds to a 49.1% FPR and 0% JBR; the unchanged UAR supports the interpretation that review was not weakened.Source: D. Observer Evaluation · p. 5
  • FPR falls from 49.1% to 7.4% and F1 rises from 0.74 to 0.95 with the Observer, while UAR remains 0%.The execution-level security guarantee is unchanged.Source: Figure 3 · p. 5, D. Observer Evaluation · p. 5
  • Across 120 matched clean and hijacked cases, the Observer perfectly separated the explicit approval injections, but that operating point is threat-model-specific.All clean cases had confidence 0.0 and CLEAN verdicts; all hijacked cases had confidence 1.0 and INJECTION verdicts. Live in-pipeline detection was 97.2% because Researcher summaries differed from the matched set.Source: D. Observer Evaluation · p. 5

E. Threats to Validity

The evaluation is bounded by a small task set, imprecise three-seed estimates, conspicuous injections, and a single architecture, framework, and model. The conclusion reports containment even against an attacker with the signing secret, while identifying broader model and attacker evaluations as future work.

E. Threats to Validity

  • The study covers 60 tasks and 13 action types; hijacking FPR varies across three seeds, and the authorization ablation uses a single seed.UAR and JBR were stable across three seeds, but the hijacking FPR 95% interval was [28.1%, 70.0%], and intervals over three seed means are imprecise.Source: E. Threats to Validity · p. 5
  • Because the hijacking injection conspicuously asserts approval, confidence, and provenance, the Observer operating point should not be extrapolated to adaptive adversaries.Less conspicuous injections may produce overlapping score distributions and non-zero JBR; the policy oracle does not read retrieved content, so execution containment does not depend on detectability.Source: E. Threats to Validity · p. 5
  • Results use one architecture, framework, and model, so containment should be validated more broadly across models.Future work also proposes additional models and heterogeneous agent configurations.Source: E. Threats to Validity · p. 6, §VI · p. 6

VI. CONCLUSION

  • Against an attacker holding the signing secret, the policy oracle—not the signed token—provided observed containment, and the result persisted under an independently derived policy.The conclusion presents the empirical propagation study, component-level ablation, and JBR as the contributions rather than the standard authorization primitives.Source: §VI · p. 6
  • Future work will test weaker attackers and extend the authorization ablation across seeds, models, and heterogeneous agent configurations.The paper expects signed tokens to be load-bearing against weaker attackers.Source: §VI · p. 6

Related papers

Loading 2609.17648v1…