PaperlayerView exact revision on arXiv

Source-linked AI summary

Delegated Authorization for Agents Constrained to Semantic Task-to-Scope Matching

Majed El Helou, Chiara Troiani, Benjamin Ryder, Jean Diaconu, Hervé Muyal, Marcelo Yannuzzi

arXiv:2510.26702v1cs.AI

TL;DR

LLM-driven agents can seek access beyond their assigned tasks, while conventional delegated authorization does not inspect the original intent. The paper proposes proxy-mediated semantic matching and introduces ASTRA for benchmarking it; results show promise but increasing difficulty balancing over- and under-scoping as task complexity grows.

  • Problem

    Conventional delegated authorization can grant agents broader access than their tasks require, and datasets for evaluating appropriate and inappropriate task-to-scope matches are unavailable.

    Source: §I · p. 1

  • Method

    The paper proposes a trusted-proxy authorization model for semantic inspection of access requests and introduces ASTRA, a dataset and generation pipeline for benchmarking task-to-scope alignment.

    Source: §I · p. 2

  • Results

    As task complexity and required scopes increase, balancing over-scoping and under-scoping becomes more challenging, affecting agent usability and security.

    Source: §VII · p. 8

  • Takeaways & Limitations

    LLM-based semantic matchers show promise for constraining agent permissions, but further research is needed as task complexity increases.

    Source: §VII · p. 8

    In common delegated-authorization implementations, the protected resource may not be explicit in metadata steps, so the client must convey it in the access request.

    Source: Appendix A · p. 11

Abstract

from arXiv · show

Authorizing Large Language Model driven agents to dynamically invoke tools and access protected resources introduces significant risks, since current methods for delegating authorization grant overly broad permissions and give access to tools allowing agents to operate beyond the intended task scope. We introduce and assess a delegated authorization model enabling authorization servers to semantically inspect access requests to protected resources, and issue access tokens constrained to the minimal set of scopes necessary for the agents' assigned tasks. Given the unavailability of datasets centered on delegated authorization flows, particularly including both semantically appropriate and inappropriate scope requests for a given task, we introduce ASTRA, a dataset and data generation pipeline for benchmarking semantic matching between tasks and scopes. Our experiments show both the potential and current limitations of model-based matching, particularly as the number of scopes needed for task completion increases. Our results highlight the need for further research into semantic matching techniques enabling intent-aware authorization for multi-agent and tool-augmented applications, including fine-grained control, such as Task-Based Access Control (TBAC).

I. INTRODUCTION

LLM-driven agents can request permissions beyond their assigned tasks, while conventional authorization servers judge the agent’s request without seeing the original intent. The paper proposes proxy-mediated semantic authorization and introduces ASTRA to evaluate task-to-scope matching.

I. INTRODUCTION

  • Agents can invoke tools or request scopes that exceed the task’s intended scope, creating potential attack vectors when additional permissions are granted.The risk includes both unsuitable tool calls and requests that are technically permitted but outside the commissioned task.Source: §I · pp. 1–2
  • Current authorization servers typically apply static rules to agent requests without visibility into the original task or intent.An agent may therefore request additional scopes, which can be granted if they are allowable for that agent.Source: §I · p. 2
  • The proposed model uses a trusted proxy to capture the original intent and lets the authorization server semantically inspect requested scopes.The authors describe this as helping mitigate intentional or unintentional requests for additional scopes.Source: §I · p. 2
  • ASTRA is a dataset and generation pipeline for systematic, reproducible evaluation of semantic alignment between tasks and requested scopes.The paper presents it as addressing the lack of datasets for delegated authorization flows.Source: §I · p. 2
  • Experiments using ASTRA show both promise and limitations for semantic approaches as tool complexity and scale increase.The paper identifies this as a first study introducing a dataset-generation pipeline for benchmarking indirect task-to-scope matching.Source: §I · p. 2

A. LLM-based Tool Selection and Benchmarks

Prior tool-selection benchmarks and authorization approaches do not directly provide independent validation of whether requested tools or scopes match an agent’s task. The paper situates this gap across tool benchmarks, intent detection, and delegated authorization.

Tool Selection Benchmarks:

  • Tool-selection benchmarks evaluate tool choice in particular environments, while MCP supports discovery across larger tool sets.Toucan provides real-world MCP servers and multi-tool synthetic tasks, but the stated gap concerns independently validating task-to-scope matching.Source: Tool Selection Benchmarks · pp. 2–3
  • Existing benchmark data lacks realistic examples of both appropriate and inappropriate scope choices and corresponding tool calls for a given task.This limits independent validation of LLM task-to-scope matching.Source: Tool Selection Benchmarks · p. 3

B. Intent Detection

  • Intent detection includes identifying implicit intent and out-of-scope queries, with LLMs offering new opportunities beyond traditional supervised or similarity-based approaches.The paper connects task-to-tool appropriateness to intent detection in task-oriented conversational AI.Source: B. Intent Detection · p. 3

C. Delegated Authorization and Access-Control

  • Traditional delegated authorization adjudicates an agent’s explicit request rather than the original natural-language intent, creating a trust gap.Predefining all possible scopes can bound actions but does not bridge that intent gap and complicates policy management as scopes grow.Source: C. Delegated Authorization and… · p. 3
  • XAA extends OAuth with dynamic associations and added identity-provider visibility, but does not align agent-generated requests with original intent.The paper notes this remaining challenge particularly for ambiguous natural-language prompts.Source: C. Delegated Authorization and… · p. 3

III. METHOD

The method combines an authorization server with a trusted proxy so access decisions can compare requested tools or resources with the task intent. It describes a semantic matcher module and two contrasting approaches, while noting that strict task-based permissions risk under-scoping.

III. METHOD

  • The proposed semantic, task-centric model combines an authorization server with a trusted proxy to align access requests with the subject’s input intent.It is presented as enabling real-time, granular, transient access policies tailored to each request’s execution context.Source: §III · p. 4
  • The proxy captures the full natural-language prompt and relays its intent and contextual metadata to both the principal and authorization server.Authorization requests from the principal pass exclusively through the proxy.Source: §III · p. 4
  • The authorization server semantically compares task descriptions with requested-tool or resource details to grant scope and duration needed for the task.The method describes permissions as dynamically provisioned and automatically revoked on task completion.Source: §III · p. 4
  • The architecture can assign each agentic application a lightweight virtualized authorization-server instance to isolate scopes and support scaling.The passage links this segmentation to policy manageability and reduced potential blast radius.Source: §III · p. 4
  • Strictly inferring permissions for the immediate task creates an under-scoping risk that may block tools or resources needed for successful execution.The practical effectiveness therefore depends on the semantic technique used to align tasks and permissions.Source: §III · p. 4, A. Semantic Matching · p. 4

1) Semantic Similarity Matcher:

  • SemSimM generates an ideal tool description, embeds it alongside available-tool descriptions, and accepts a request when the requested tool is the closest match above a threshold.The approach evaluates tools in isolation and may be challenged by extensive registries or coordinated multi-tool tasks.Source: 1) Semantic Similarity Matcher · p. 5

2) LLM Reasoning Matcher:

The paper describes LLM-ResM as a direct task–tool judgment strategy, then outlines the data-generation and preprocessing pipeline used to evaluate matchers. Its synthetic tasks use up to three tools, and incorrect matches are sampled as either semantically closer or more distant alternatives.

2) LLM Reasoning Matcher:

  • LLM-ResM prompts a trusted LLM with task context and the requested tool’s name and description to return a structured suitability judgment.It assesses each task–tool pair without scanning the full tool registry, which the paper identifies as a scalability strength.Source: 2) LLM Reasoning Matcher · p. 5

IV. EXPERIMENTAL EVALUATION

  • The evaluation generates tasks and simulated incorrect tool requests, then tests preliminary matchers on the paper’s dataset and public data.The paper also processes Toucan data to run the same simulation pipeline on it.Source: §IV · pp. 5–6

A. Data Generation and Tool Matching Simulation

  • ASTRA’s task pipeline curates 12 enterprise-use-case MCP servers and uses GPT-4o to generate three tasks for each sampled set of one to three tools.Tool descriptions omit argument details, and prompts encourage realistic tasks that indirectly require the tools; the authors limit tasks to at most three tools after finding realism drops beyond four.Source: A. Data Generation and Tool Matching… · p. 6
  • Incorrect matches are sampled as same-server wrong matches or different-server null matches, with no overlap with correct matches for tasks requiring multiple tools.The simulation accepts desired correct-match counts, wrong/null ratios, and server subsets for independent validation and test sets.Source: A. Data Generation and Tool Matching… · p. 6

B. Toucan Data Pre-processing and Simulation

  • Toucan preprocessing filters non-English servers, empty tool descriptions, redundant tools, and servers with fewer than twice the task’s tool count.After filtering, the pipeline retains 118 servers and produces 1,056 processed tasks for each N from 1 to 3.Source: B. Toucan Data Pre-processing and… · p. 6

C. Task-Tool Matching

Task-tool matching asks whether each requested tool is necessary for a task, including in complex multi-tool workflows. The experiments compare two preliminary matchers and report stronger single-tool results for LLM-ResM, while multi-tool matching becomes harder as complexity rises.

C. Task-Tool Matching

  • Tool matching evaluates whether a requested tool is necessary and appropriate for the task, with the aim of limiting access to tools strictly required for completion.Source: C. Task-Tool Matching · p. 6
  • Multi-tool workflows add complexity because the matcher must assess both each tool’s appropriateness and the necessity of composite tool use and interdependencies.Source: C. Task-Tool Matching · pp. 6–7

D. Experiments

  • The experiments use dynamic tool availability and on-demand inference, with tools and MCP servers separated between validation and test; SemSimM was evaluated only on single-tool tasks.Both matchers were tested on the generated and processed Toucan datasets; multi-tool evaluation used LLM-ResM only.Source: D. Experiments · p. 7

V. RESULTS

  • The reported evaluation covers single-tool and multi-tool scenarios using standard classification metrics, with performance examined as task complexity changes.Source: §V · p. 7

A. Single-Tool Tasks

  • LLM-ResM outperforms SemSimM in accuracy, recall, and F1 across the single-tool datasets; SemSimM has substantially lower recall.Source: A. Single-Tool Tasks · p. 7

B. Multi-Tool Tasks

  • As tasks require more tools, LLM-ResM performance worsens from two-tool to three-tool tasks, especially in recall; three-tool recall on Toucan is 29 percentage points higher than on the generated dataset.The authors attribute the dataset gap to more implicit tool use in generated tasks and more explicit action lists in Toucan.Source: B. Multi-Tool Tasks · p. 7

C. Matcher Performance Across Task Complexities

For LLM-ResM, increasing task complexity makes it harder to balance over-scoping against under-scoping, with under-scoping rising sharply in three-tool tasks. The discussion and conclusion identify limits in scale and complex workflows, while presenting semantic matching as a route toward more fine-grained delegated authorization.

C. Matcher Performance Across Task Complexities

  • Over-scoping grants excessive permissions, whereas under-scoping can hinder task completion; balancing both becomes harder when multiple matcher decisions combine.Source: C. Matcher Performance Across Task… · pp. 7–8
  • As required tools increase from one to three, under-scoping grows, with the false negative rate reaching 0.78 on the three-tool dataset.Figure 4 compares false positive rate (over-scoping) and false negative rate (under-scoping) across task complexities and datasets.Source: Figure 4 · p. 8, C. Matcher Performance Across Task… · p. 8

VI. DISCUSSION AND FUTURE WORK

  • The experimental scope could expand to multi-turn contexts, tasks with four or more tools, more MCP servers, and semantically informed non-match selection.Source: §VI · p. 8
  • Accuracy and performance may degrade as the number of protected resources or available tools increases, so further study of this relationship is needed.Source: §VI · p. 8
  • Delegation across agent chains requires preserving requester context and enforcing appropriate scopes at multiple points in the chain.Source: §VI · p. 8

VII. CONCLUSION

  • The proposed architecture combines semantic matching with delegated authorization to give agents minimally necessary, task-relevant permissions.Source: §VII · p. 8
  • ASTRA addresses the lack of delegated-authorization benchmarks, while the experiments indicate that balancing over- and under-scoping becomes more difficult as task complexity and required scopes increase.Source: §VII · p. 8
  • Current semantic, intent-aware matching methods remain limited in accuracy and scalability, especially for complex multi-tool scenarios.Source: §VII · p. 9

APPENDIX A - DELEGATED AUTHORIZATION BACKGROUND AND LIMITATIONS

Delegated authorization lets one subject grant another access to protected resources without sharing credentials, but conventional flows separate the agent’s scope request from the task that prompted it. This leaves authorization servers unable to assess whether requested scopes match the task, motivating semantic inspection.

APPENDIX A - DELEGATED AUTHORIZATION BACKGROUND AND LIMITATIONS

  • A subject can delegate access to protected resources to a third party without revealing or sharing credentials.OAuth 2.0, OAuth 2.1, OpenID Connect, and SAML are cited as protocols used in delegated authorization.Source: Appendix A · p. 10
  • PKCE protects the authorization-code exchange by requiring the server to verify a hashed code verifier before issuing an access token.The verifier travels over a secure back channel rather than the public channel.Source: Appendix A · p. 12
  • Existing OAuth 2.X flows analyze the agent’s requested scopes without the original task, leaving authorization disconnected from user intent and relying on static, permissive delegations.The authorization request typically carries scopes and PKCE information, but not the tasks conveyed in the original prompt.Source: Appendix A · pp. 11–12
  • The proposed approach pairs the authorization server with a trusted proxy that captures the original intent and supports semantic inspection of access requests.The proxy forwards the original intent with metadata to the agent, and delegated authorization flows are proxied.Source: Appendix A · p. 12

APPENDIX B - DELEGATED AUTHORIZATION CONSTRAINED TO SEMANTIC TASK-TO-SCOPE MATCHING

The enhanced flow routes delegated authorization through a trusted proxy that links the original prompt to the agent’s scope request. The authorization server can then semantically match requested scopes against the prompt’s tasks before making its access decision.

APPENDIX B - DELEGATED AUTHORIZATION CONSTRAINED TO SEMANTIC TASK-TO-SCOPE MATCHING

  • The Trusted Proxy links each authorization request to the original prompt using a request ID, then forwards the prompt, requested scopes, PKCE information, and principal ID to the authorization server.The proxy removes the request ID before forwarding the authorization request.Source: Appendix B · pp. 12, 14
  • The authorization server semantically matches requested scopes against the tasks in the prompt when inspecting the access request.The described examples include SemSimM and LLM-ResM.Source: Appendix B · p. 14
  • If access is permitted and PKCE verification succeeds, the authorization server issues an access token through the Trusted Proxy, after which the agent can retry the resource request.The proxy can revoke or invalidate the associated request ID after the final response.Source: Appendix B · p. 14
  • The flow bridges the user’s original intent and the agent’s access request by making the prompt available to the authorization server.The relevant flow steps are (8b), (8c), and (9).Source: Appendix B · p. 14

APPENDIX C - DATA GENERATION SYSTEM PROMPTS

The data-generation prompts produce realistic natural-language tasks that indirectly require one or more tools. Separate prompt variants address single-tool and multi-tool tasks, with instructions emphasizing the user’s goal rather than directly naming the tools.

APPENDIX C - DATA GENERATION SYSTEM PROMPTS

  • The generation pipeline uses separate system prompts for single-tool and multi-tool tasks, paired with a user prompt that requests a chosen number of diverse examples.Tool names and descriptions are substituted into the corresponding prompt templates.Source: Appendix C · p. 16
  • Generated tasks should be realistic, natural-language requests focused on what the user wants to achieve, with tool use inferred rather than directly requested.The instructions call for plausible specifics and one or two fluid sentences rather than a parameter list or JSON object.Source: Appendix C · pp. 16–17
  • The prompts instruct the model to return only the generated request, without explanations, preambles, or markdown.This output constraint appears in the single-tool generation instructions.Source: Appendix C · p. 16
  • Multi-tool tasks must require all supplied tools, with their connection to the task made obscure, implicit, or indirect.The multi-tool system prompt supplies the tools and asks for a realistic user request requiring them.Source: Appendix C · p. 17

APPENDIX D - MATCHER SYSTEM PROMPTS

The matcher prompts define two approaches: SemSimM reformulates the user request as an ideal tool description, while LLM-ResM judges whether a requested tool is appropriate for that request.

APPENDIX D - MATCHER SYSTEM PROMPTS

  • SemSimM reformulates the dialog context into a concise ideal-tool description in a specified format, without explanations or commentary.The prompt instructs it to avoid detailed values.Source: Appendix D · p. 18
  • LLM-ResM acts as a guardrail that evaluates whether the requested tool is appropriate for the user request.Its prompt distinguishes appropriateness from sufficiency: a tool may be appropriate but not sufficient to fulfill the request.Source: Appendix D · p. 18
  • The SemSimM reformulation prompt takes the original prompt, requested tool name, and tool description as input fields..Source: Appendix D · p. 18

APPENDIX E - DATASETS ILLUSTRATIVE EXAMPLES

Matching validation has higher recall on Toucan-based data than on this dataset, with the gap most pronounced for three-tool tasks. Toucan tasks make tool mappings explicit, while this dataset’s tasks imply multiple tools more subtly.

APPENDIX E - DATASETS ILLUSTRATIVE EXAMPLES

  • Recall is higher on Toucan-based data than on this dataset, with the gap already present for two-tool tasks and largest for three-tool tasks.The authors manually inspect samples and provide examples to illustrate the recall and F1-score gap.Source: Appendix E · p. 18
  • The examples contrast a three-tool request for compound and assay data with a request to synthesize healthcare findings from preprint and peer-reviewed sources.The Toucan example names successive retrieval steps; the dataset example describes a comparative analysis task.Source: Appendix E · p. 18
  • Toucan tasks often map explicit sequential steps one-to-one to tools, whereas this dataset’s tasks more subtly imply multiple tools without naming them.The authors suggest this subtlety may also characterize real users unfamiliar with an MCP Server’s underlying tools.Source: Appendix E · p. 18

Related papers

Loading 2510.26702v1…