PaperlayerView exact revision on arXiv

Source-linked AI summary

MetaPermit: Scalable and Auditable Access Control for AI Agents via LLM-Inferred Meta-Attributes

Hanzhang Ma, Ali Hariri, Tianxiang Shen, Bohua Zou, Qianjun Zheng, Ji Wang, Li Yi, Ning Jia, Yutao Liu, Haibo Chen, Lin Wang, Debayan Roy

arXiv:2609.31039v1cs.CRcs.SE

TL;DR

Static policies struggle to cover open-ended intents, while LLM-driven authorization can vary across similar tasks and be steered by injected content. MetaPermit has an LLM infer meta-attributes for each tool call and a fixed policy decide authorization; it improves benchmark utility and attack robustness over evaluated defenses.

  • Problem

    Static policies require anticipated intents, while LLM-driven authorization can give inconsistent verdicts and injected content may be followed as if it came from the user.

    Source: §1 · p. 1

  • Method

    MetaPermit uses an LLM to infer meta-attributes describing a proposed tool call and a fixed policy to make auditable authorization decisions.

    Source: §1 · pp. 1–2, §3.3 · p. 4

  • Results

    MetaPermit improves utility by up to 109%, reduces attack success from as high as 3.14% to 0%, and provides 31% more consistent authorization decisions than a direct LLM guard.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Policy feedback helps agents recover from denied calls and revise actions without weakening enforcement.

    Source: §5 · p. 7

    The deterministic policy cannot verify that inferred meta-attributes are correct, so the safety claims are empirical and enforcement-oriented rather than a formal guarantee of semantic correctness.

    Source: §4.5 · p. 7

Abstract

from arXiv · show

The rise of autonomous AI agents equipped with tools has introduced significant security risks, ranging from unintended tool misuse to adversarial manipulation through Indirect Prompt Injection (IPI) attacks. In practice, deployed agent systems such as OpenAI Codex and Claude Code protect tool invocations through a combination of coarse-grained permission rules and LLM-based judgments about individual proposed actions. Both components, however, have important limitations: static policies must anticipate possible user intents and therefore do not scale to open-ended tasks, while LLM-driven authorization supports dynamic decisions but produces inconsistent outcomes and remains vulnerable to targeted IPI attacks. To provide scalable and more consistent authorization, we propose MetaPermit, a policy-based tool access-control framework that decouples semantic inference from security enforcement. By analyzing agent-user interactions, we derive a compact, task-independent set of meta-attributes that capture the relationships among the user's intent, the execution context, and the proposed tool call. These meta-attributes allow MetaPermit to authorize tool use without enumerating user intents. At runtime, an LLM infers the meta-attribute values for each proposed tool call, while a fixed policy evaluates these values to allow or deny the call, making each decision auditable through the inferred values and the applied policy rule. We evaluate MetaPermit on the AgentDojo and AgentDyn benchmarks, across seven task suites and five attack methods, using two widely deployed open-weight LLMs. The results show that MetaPermit produces 31% more consistent authorization decisions than LLM-driven authorization and outperforms the state-of-the-art defenses CaMeL and IPIGuard in both task completion, with improvements of up to 109%, and robustness to IPI attacks, with no malicious tool calls executed.

1 Introduction

Static tool policies struggle with open-ended intents, while LLM-driven authorization can vary across similar tasks and be manipulated by injected instructions. MetaPermit separates semantic inference from policy enforcement, using inferred meta-attributes to support auditable decisions without enumerating intents.

1 Introduction

  • Static policies require anticipated intents, whereas LLM-driven authorization may give inconsistent verdicts and injected content can be mistaken for user instructions.In the motivating refund example, recipient-specific rules would be needed to distinguish the legitimate transfer to Bob from the malicious transfer to Eve.Source: §1 · p. 1
  • MetaPermit combines static policies with LLM-inferred meta-attributes that capture semantic relationships among the request, proposed tool call, and context.Its compact, general policies use explicit inferred values and rules, enabling decisions to be inspected and audited without enumerating user intents.Source: §1 · pp. 1–2
  • LLMs infer bounded meta-attribute values, while fixed policies retain authorization control and the schema can be extended without changing the access-control architecture.The introduction notes that some values can depend only on trusted inputs, such as whether the user’s intent is an action.Source: §1 · p. 2
  • Policy feedback gives the agent a denial reason or next action, helping it pursue the user’s intent rather than leaving it with an unexplained denial.The mechanism is intended to guide task completion without delegating the final authorization decision to the LLM.Source: §1 · p. 2
  • 31% more consistent authorization decisions than a direct LLM guard, while MetaPermit improves utility by up to 109% and reduces attack success from as high as 3.14% to 0%.The evaluation covers AgentDojo and AgentDyn, seven task scenarios, and five prompt-injection attacks.Source: §1 · p. 2

2 Related Work

Prior agent-authorization defenses use predefined static policies or runtime LLM-driven authorization, with different trade-offs in scalability, consistency, and utility. The related-work comparison also covers hybrid designs and task-specific policy generation.

2 Related Work

  • Table 1 compares defenses by support for open-ended intents, whether identical inputs yield the same verdict, and preservation of task utility.Its markers distinguish static, LLM-based, and hybrid approaches; CaMeL and IPIGuard utility figures come from the paper’s evaluation, while Conseca and Progent figures are reported results.Source: Table 1 · p. 2
  • Predefined static policies struggle with open-ended tasks and unanticipated actions, while intent-agnostic rules may not distinguish legitimate from malicious uses of the same permission.The related-work discussion describes these approaches as dependent on complete pre-authored policies.Source: §2 · p. 2
  • Runtime LLM-driven approaches include just-in-time or per-task policies, but their correctness depends on the LLM’s task understanding and stochasticity can vary decisions across similar tasks.Conseca reports 60% utility on 20 Linux tasks; Progent reduces attack success on AgentDojo and ASB but requires user approval to expand permissions.Source: §2 · p. 2

3 MetaPermit

MetaPermit authorizes tool calls by separating model-based semantic interpretation from fixed-policy decisions and deterministic enforcement. Its reusable schema and feedback support varied tasks while constraining model authority.

3.1 Threat Model and Design Requirements

  • The threat model allows an adversary to control tool-returned content and inject instructions, but not to change the request, tools, policy, or runtime.MetaPermit authorizes each call and its arguments before execution, aiming to block injected instructions while allowing the benign request to be completed.Source: §3.1 · pp. 2–3

3.2 Design Rationale and End-to-End Workflow

  • The design requires intent-independent policies, a constrained model vocabulary, and deterministic enforcement so the LLM neither creates rules nor issues the final verdict.The workflow maps each candidate call’s context to a meta-attribute vector, evaluates it against fixed ABAC rules, and applies the resulting verdict before execution.Source: §3.2 · pp. 3–4

3.3 Authorizing a Tool Call

  • For each proposed call, the inference module uses the request, trusted tool description, arguments, and prior tool results to infer finite-domain meta-attributes; some rely only on the trusted request.For the injected transfer to Eve, the policy denies the call because its recipient is not justified by the user’s request; the default rule also denies unmatched vectors.Source: §3.3 · p. 4, Table 2 · p. 4
  • The policy engine applies the first matching ordered rule and returns a verdict with feedback; the enforcement gate blocks denied calls and returns that feedback so the agent can replan.A call to Bob is permitted when its argument aligns with the request, whereas an injection must induce a valid combination of values satisfying an allow rule.Source: §3.3 · p. 4

3.4 Building and Extending the Policy

  • Designers derive reusable meta-attributes from security goals and encode those goals as ordered rules; the evaluated deployment uses five attributes and one rule set across task suites and attacks.New requirements can add a security dimension and update dependent rules without changing the interface, enforcement path, or unrelated rules; invalid or incomplete outputs are denied by default.Source: §3.4 · pp. 4–5

4 Experimental Evaluation

Across seven task suites and five injection attacks, MetaPermit combines low attack success with preserved task utility; the evaluation also probes cross-model transfer, authorization design, decision stability, and the limits of its guarantee.

4.1 Experimental Setup

  • The evaluation covers seven suites, five attack templates, and two open-weight models, using AgentDojo and AgentDyn tasks.AgentDojo contributes workspace, travel, banking, and slack; AgentDyn adds Shopping, Github, and Dailylife.Source: §4.1 · p. 5

4.2 Main Results

  • 0.00% ASR with 50.13% UA on Qwen3-235B; MetaPermit has the lowest ASR and highest utility among evaluated defenses on both backends.On MiniMax-M2.7, it records 0.08% ASR and 53.92% UA; the table reports macro-averaged results across seven suites.Source: §4.2 · p. 5, Table 3 · p. 6
  • 53.9% and 50.1% Secure-Usable shares on MiniMax-M2.7 and Qwen3-235B are the highest among defenses and close to undefended levels.An audit found no executed attack-induced tool calls among 7,545 MiniMax-M2.7 attacks, although the official checker reports 0.08% ASR.Source: §4.2 · pp. 5–6, Figure 2 · p. 6

4.3 Ablation Study

  • On Qwen3-8B, MetaPermit reduces checker-measured ASR from 9.01% to 1.50% while achieving 31.9% UA, with the schema and policy unchanged.The undefended agent achieves 45.6% UA; MetaPermit also achieves Secure-Usable execution in 31.70% of cases.Source: §4.3 · p. 6
  • 0.20% ASR and 45.66% UA on MiniMax-M2.7 outperform direct LLM verdicts at 0.61% ASR and 41.96% UA.Removing intent_input_alignment doubled attack positives (one to two) while completed tasks changed from 54 to 53; in a white-box probe, none of 95 malicious calls was permitted.Source: §4.3 · pp. 6–7, Table 4 · p. 6, Table 6 · p. 7

4.4 Decision Stability under Repeated Inference

  • 42% of MetaPermit decisions were fully consistent across 20 runs, versus 32% for the Direct LLM Guard; majority agreement was 90% versus 87%.The static policy mapped 18 of 47 queries with varying meta-attribute vectors to consistent verdicts.Source: Figure 3 · p. 7, §4.4 · p. 7

4.5 Limitations

  • The deterministic policy guarantees the same verdict for a given meta-attribute vector, but does not verify that inferred attributes are semantically correct.The paper characterizes its safety claims as empirical and enforcement-oriented, not a formal guarantee of semantic correctness.Source: §4.5 · p. 7

5 Conclusion

MetaPermit separates semantic interpretation from authorization: an LLM maps candidate tool calls to meta-attributes, and a fixed policy evaluates them. On AgentDojo and AgentDyn, it reduces attack success to near zero while preserving more utility than state-of-the-art defenses.

5 Conclusion

  • MetaPermit maps candidate tool calls to meta-attributes, which a fixed policy evaluates and an enforcement gate enforces.The design supports open-ended tasks without enumerating intents and generalizes across task suites and agent models.Source: §5 · p. 7
  • On AgentDojo and AgentDyn, MetaPermit reduces attack success to near zero while preserving substantially more utility than state-of-the-art defenses.Policy feedback also helps the agent recover from denied calls and revise actions without weakening enforcement.Source: §5 · p. 7
  • Future work will explore local meta-attribute-specific models to reduce inference latency and cost, improve prediction consistency through task-specific fine-tuning, and strengthen inference against IPI attacks.Source: §5 · p. 7

A.1 Benchmark Instantiation

The benchmark implementation holds a five-attribute schema and ordered policy rules fixed across suites and attack templates. Its LLM infers attributes from the request and proposed call, while the policy determines authorization and denial feedback can guide revised calls.

A.1 Benchmark Instantiation

  • The five attributes and ordered policy rules are held fixed across all seven task suites and attack templates.They cover whether the request calls for action, goal clarity, risk, tool necessity, and whether proposed arguments align with the goal.Source: §A.1 · p. 9
  • Tool alignment and input alignment are separate because a necessary retrieval tool can still be followed by a call using an unauthorized recipient, account, path, or URL.The inference module jointly infers the five values, and the policy engine evaluates ordered rules.Source: §A.1 · p. 9

A.2 Meta-Attribute Value Definitions

  • The attribute definitions derive from security questions such as whether the user expects an action, whether the goal is clear, and whether the tool and its arguments fit that goal.The framework allows stakeholders to define further attributes for their application domain.Source: §A.2 · p. 9

A.3 Inference-Module Prompt Template

  • The inference prompt supplies the original request, tool description, proposed arguments, and prior tool-role messages, treating retrieved content as untrusted evidence.An unparsable response maps to a fail-closed vector that the policy denies.Source: §A.3 · p. 10

A.4 Direct LLM Guard Prompt

  • The direct LLM guard ablation uses the same authorization query but outputs a binary Allow-or-Deny verdict instead of typed attributes.Its prompt requires PERMIT for a reasonable invocation or DENY when there is a security risk.Source: §A.4 · p. 11

A.5 Task-Agent Configuration

  • The task agent uses AgentDojo’s default tool-calling pipeline and system prompt, with at most 15 tool-execution iterations per task.Source: §A.5 · p. 11

A.6 Policy Rule Listing

  • The policy applies ordered rules, and a denial returns rule-specific obligations to the agent as a synthetic tool result.Repeated proposals are evaluated under the same ordered rules.Source: §A.6 · p. 11

A.7 Worked Policy Construction Example

  • The Banking example denies a transfer to an attacker when the tool is relevant but its recipient and disclosure-bearing subject are unsupported by the user’s goal.The intended refund to the requested friend proceeds, illustrating how reusable attributes distinguish calls to the same tool without task-specific bans.Source: §A.7 · p. 11

A.9 Experimental Setup and Hyperparameters

The reported experiments use the same model for the task agent and inference module, with task-level iteration limits and benchmark-specific baseline implementations. The supplied setup passages describe configurations but do not provide model-table values.

A.9 Experimental Setup and Hyperparameters

  • The task agent and inference module always use the same model, with Table 8 listing the models and serving configurations.Source: §A.9 · p. 11, Table 8 · p. 12
  • Tasks allow at most 15 tool-execution iterations and are evaluated once per case, except repeated decision-stability queries.That study repeats each authorization query 20 times.Source: §A.9 · p. 11
  • Table 7 lists the exact ordered ABAC rules used in the reported evaluation; earlier rules take precedence.The table caption says an asterisk denotes any value and a dash denotes a condition already constrained by an earlier rule.Source: Table 7 · p. 12
  • The reported CaMeL and IPIGuard runs use vendored implementations with backend compatibility adapters and latency instrumentation, leaving their authorization logic unchanged.CaMeL uses its default AgentDojo tools pipeline; IPIGuard uses its plan-construction and traversal modules.Source: §A.9 · p. 12

A.10 Diagnostic Analysis of Baseline Outcomes

The checker-artifact audit finds every official MetaPermit positive was a think-only match; after excluding that artifact, MetaPermit remains at 0% attack success while each baseline retains observable successes. The baseline diagnostics show that low attack success can coincide with unfinished benign tasks, and the comparisons reflect performance under the MiniMax-M2.7 backend.

A.10 Diagnostic Analysis of Baseline Outcomes

  • Low ASR alone does not establish that a defense preserves useful behavior; the diagnostic distinguishes successful benign tasks with failed injections from unfinished benign tasks and successful injections.Source: §A.10 · p. 12
  • Of 5,581 Secure-Unusable CaMeL attack cases, 4,706 had at most two executable calls and ended with an assistant message rather than another executable call.The traces often retrieved evidence but omitted a final answer or benign side effect needed for utility credit.Source: §A.10 · p. 12
  • For IPIGuard, 2,823 traces contained textual planning artifacts that AgentDojo could not execute, while other traces stopped after retrieval or a security warning.Only 86 of 5,226 Secure-Unusable cases had no executable tool call; 2,170 had five or more.Source: §A.10 · pp. 12–13
  • The CaMeL and IPIGuard comparisons represent out-of-the-box performance under MiniMax-M2.7, where expected intermediate artifacts and execution behavior vary by backend.The paper reports ASR, UA, and Secure-Usable together because successful defense requires both blocking injection and completing the benign task.Source: §A.10 · p. 13

A.11 Checker-Artifact Audit Across Defenses

  • After excluding think-only checker artifacts, MetaPermit remains at 0% ASR while each baseline retains observable attack successes.All official MetaPermit positives were think-only; 11 of 215 No Defense positives and 17 of 69 IPIGuard positives were also think-only, while CaMeL’s 20 positives had no hidden-reasoning artifacts.Source: §A.11 · p. 13

A.12 Denial-Feedback Ablation

Structured, policy-grounded denial feedback supports recovery without relaxing the fixed policy. A separate white-box probe tests whether disclosed inference details and the allow rule can manipulate enforcement-layer attribute judgments.

A.12 Denial-Feedback Ablation

  • The ablation compares full structured feedback, a generic denial message, and silent denial on two denial-exercising cases per suite–attack cell.It measures UA, ASR, task completion after the first denial, repeated denials, and subsequent tool calls.Source: §A.12 · p. 13
  • 60.00% UA and 50.00% recovery with 0.00% ASR: full structured feedback outperforms generic or silent denial on recovery while preserving security.The ablation covers 70 targeted MiniMax-M2.7 denial cases; repeated denials also decrease relative to the other modes.Source: §A.12 · p. 13, Table 9 · p. 13

A.13 White-Box Enforcement-Layer Probe

  • The white-box probe replays 105 matched cases with an injected malicious-call task, excluding 10 cases without a ground-truth malicious call and leaving 95.Injected content reveals inference inputs, attribute definitions, and the deterministic allow rule; the probe checks whether the inferred vector satisfies that rule and counts attributes flipped to allow versus control.Source: §A.13 · p. 13

A.14 Statistical Significance of the Main Comparison

Wilson intervals over the full benchmark matrix separate MetaPermit’s ASR from every baseline and its UA from the structural defenses. The section also describes paired significance testing, evaluation metrics, and the full per-attack breakdown.

A.14 Statistical Significance of the Main Comparison

  • Wilson 95% confidence intervals separate MetaPermit’s ASR from every baseline and its UA from both structural defenses, while the undefended agent retains higher UA.The intervals cover UA, ASR, and Secure-Usable over the full AgentDojo and AgentDyn matrix for MiniMax-M2.7.Source: §A.14 · p. 13, Table 10 · p. 14
  • Paired McNemar exact tests assess matched per-case differences between MetaPermit and each baseline; negative ASR differences and positive UA differences favor MetaPermit.Table 11 reports percentage-point differences, with significance stars based on discordant outcomes.Source: §A.14 · p. 13, Table 11 · p. 14

B.1 Evaluation Protocol and Metrics

  • UA measures completed benign tasks, ASR measures achieved attack objectives, and Secure-Usable measures cases that are both secure and task-complete.Rates use official benchmark checkers; an execution-aware audit additionally excludes matches solely inside a think block and finds no executed unauthorized effects.Source: §B.1 · p. 14

B.2 Per-Attack Breakdown (MiniMax-M2.7)

  • The cited passage identifies Table 12 as giving the full per-attack, per-suite MiniMax-M2.7 results and says these are summarized by the main-text averages in Table 3.Source: §B.2 · p. 14

B.3 Cross-Model Results

The appendix provides a complete seven-suite Qwen3-235B comparison across AgentDojo and AgentDyn, with outcome decomposition shown alongside MiniMax-M2.7.

B.3 Cross-Model Results

  • Table 13 reports the complete seven-suite Qwen3-235B comparison across AgentDojo and AgentDyn.The comparison includes No Defense, CaMeL, IPIGuard, and MetaPermit.Source: §B.3 · p. 14
  • Figure 2 presents the outcome decomposition for Qwen3-235B alongside MiniMax-M2.7.The accompanying table identifies lower ASR and higher UA as better.Source: §B.3 · p. 14
  • The reported cross-model results span seven task suites on both AgentDojo and AgentDyn.Table 13 reports performance in percentages.Source: §B.3 · p. 14

B.4 Qwen3-8B Detailed Results

The Qwen3-8B results cover a complete seven-suite attack matrix and show that the typed authorization boundary transfers to a smaller locally served open-weight model without changing the framework setup.

B.4 Qwen3-8B Detailed Results

  • The typed authorization boundary transfers to a smaller locally served open-weight model without changing the attribute schema, policy, environments, or evaluators.The results are reported for Qwen3-8B.Source: §B.4 · p. 14
  • Table 14 reports Qwen3-8B results on the complete seven-suite attack matrix in percentages.UA is computed on attacked user-task/injection-task pairs.Source: §B.4 · p. 14, Table 14 · p. 16
  • The table’s comparison criteria are lower ASR and higher UA.The appendix identifies the per-suite results as the detailed attack-case results summarized in Section 4.Source: §B.4 · p. 14, Table 14 · p. 16
Loading 2609.31039v1…