Source-linked AI summary
Report: Progressive Disclosure of Agent Skills
Guilin Zhang, Kai Zhao, Priyanka Mudgal, Waleed Ammar, Xiquan Cui, Xu Chu, Alet Blanken
TL;DR
Growing skills libraries make eager loading costly in tokens, while the effects of progressive disclosure on latency and retrieval quality are uncertain. The report evaluates lazy loading and finds lower token use and better reliability and retrieval quality, with greater latency.
Problem
As an agent’s skills library grows, eager loading increases token-based operational costs, while progressive disclosure’s effects on latency and skill-retrieval quality remain unclear.
Source: Agent Skills in Production · p. 1Method
The report contrasts eager loading with progressive disclosure, which exposes all skill frontmatter first and loads a selected skill’s body in a later invocation.
Source: Agent Skills Management in The Harness · p. 2Results
Progressive disclosure reduces token use and improves reliability and skill-retrieval quality at the expense of latency; eager-loading skill-retrieval quality for Qwen3-14B fell from 1.00 at N=5 to 0.125 at N=50.
Source: Conclusion · p. 3, Key Findings · p. 3Takeaways & Limitations
Skill-management regimes should account for retrieval quality as well as token usage.
Source: Key Findings · p. 3The setup limits each task to one skill and each library entry to SKILL.md, leaving multi-skill selection and loading additional files open.
Source: Open Questions · p. 3
Abstract
from arXiv · showhide
Users of Workday's deployed LLM-based agents often request features which can be addressed by defining named procedures, also known as skills, in the LLM context, effectively augmenting agents' capabilities. However, as an agent's skills library grows in size, so does the agent's operational cost. Progressive disclosure (lazy-loading) of skills as needed may reduce operational costs, but its impact on overall latency and skill-retrieval quality remains unclear. In this report, we investigate the impact empirically and find that progressive disclosure improves skill-retrieval quality but marginally degrades overall latency.
Agent Skills in Production
Skills let agents reuse specialized instructions to address feature requests and bug reports. As eager loading makes token costs grow with the library, the report evaluates progressive disclosure across cost, latency, and skill-retrieval quality.
Agent Skills in Production
- Workday reports serving over 65% of Fortune 500 enterprises and having more than 5,500 customers using one or more deployed AI agents as of August 2026.The reported customer count was 35% higher than the previous quarter.Source: Agent Skills in Production · p. 1
- As the skills library grows, eager loading raises operational costs because it discloses the library’s tokens in the LLM context.Skills package domain expertise such as workflows, scripts, reference documents, and templates into reusable directories.Source: Agent Skills in Production · p. 1
- Progressive disclosure incrementally adds skill details to the LLM context only as a task calls for them.It is also known as lazy loading and is used in agent frameworks such as LangChain.Source: Agent Skills in Production · p. 1
- The report measures how progressive disclosure affects token-based operational cost, latency, and skill-retrieval quality as library size grows.The investigation uses a controlled setup.Source: Agent Skills in Production · p. 1
Agent Skills Management in The Harness
The harness manages skills and prepares context for the LLM. The report contrasts loading every skill’s full definition at once with loading all frontmatter first and retrieving selected skill bodies as needed.
Agent Skills Management in The Harness
- The harness interfaces with the environment, manages the skills library, prepares context, invokes the LLM, and processes its outputs.A minimal agent consists of a harness and an LLM core.Source: Agent Skills Management in The Harness · p. 1
- A skills library contains structured skills whose SKILL.md file has YAML frontmatter and Markdown body content.The body can include step-by-step instructions, input-output examples, and edge cases.Source: Agent Skills Management in The Harness · p. 1
- Progressive disclosure places all skills’ frontmatter in the initial context, then loads the selected skill’s body in a later LLM invocation.The LLM emits a structured command naming the skill, which the harness uses to load its body.Source: Agent Skills Management in The Harness · p. 2
- Eager loading puts all skill details in the LLM context, while progressive disclosure starts with minimal information and adds relevant details when needed.Figure 1 describes retrieval degradation with larger libraries under eager loading and increased latency under progressive disclosure.Source: Agent Skills Management in The Harness · p. 2, Figure 1 · p. 2
Experimental Setup
The evaluation measures skill-retrieval success, token use, and overall latency. It varies library size, distractor difficulty, and LLM core in repeated rollouts.
Experimental Setup
- The study measures skill-retrieval success rate, total tokens used, and overall latency in seconds..Source: Experimental Setup · p. 2
- Skill retrieval is tested by asking the agent to return the activation code for the skill matching a named domain intent.The code is available only when the relevant skill body is loaded in the LLM context.Source: Experimental Setup · p. 2
- Progressive disclosure requires additional LLM-core invocations but reduces tokens used in the context..Source: Experimental Setup · p. 2
- The experiment uses library sizes N ∈{5, 20, 50, 100} and 24 task instances based on five relevant skills, with distractors at three difficulty tiers when N > 5.Three seeds yield means over 72 rollouts for each library-size and management-regime combination.Source: Experimental Setup · p. 2
- The evaluation uses three LLM cores with greedy decoding and a 32k context window, serving them on one NVIDIA L40S GPU.Each core runs 576 agent rollouts; a wrapper records prompt and completion tokens and wall-clock latency.Source: Experimental Setup · p. 2
Key Findings
Progressive disclosure reduces token use and improves skill-retrieval quality as the library grows, but may increase latency. Eager loading can exceed the maximum prompt size, while even the largest tested model remains susceptible to hard distractors.
Key Findings
- Up to 81.7% of eager-loading tokens were saved, and eager loading crashed in all rollouts at N=100 by exceeding the maximum prompt size.Token savings increased with library size; the report links them to lower operational costs at scale.Source: Key Findings · pp. 2–3, Figure 2 · p. 3
- Skill-retrieval quality under progressive disclosure matched or exceeded eager loading in all but one setup, while Qwen3-14B eager-loading quality fell from 1.00 at N=5 to 0.125 at N=50.The report attributes the decline to distractor skills dominating the context as the library grows.Source: Figure 2 · p. 3, Key Findings · p. 3
- Progressive disclosure corresponded to a 15% increase in overall latency, from 1.75 to 2.01 seconds, for Qwen3-8B with N=50.The latency delta reduces as N grows and may reverse with a larger prompt-token budget; multiple subtasks may also amortize the added invocation latency.Source: Key Findings · p. 3
- Qwen3-14B resisted easy and medium distractors but remained susceptible to hard distractors.Retrieving the wrong skill name was the dominant failure described in the report.Source: Key Findings · p. 3
Conclusion
Progressive disclosure exposes skill names and brief descriptions initially, then loads relevant full definitions as needed. The report concludes that this reduces token use and improves reliability and skill-retrieval quality, at the expense of greater latency.
Conclusion
- Progressive disclosure initially presents each skill’s name and brief description, then incrementally loads relevant full definitions as needed.The report contrasts this with loading all full skill definitions into the initial context.Source: Conclusion · p. 3
Open Questions
The report leaves open how to balance skill-definition verbosity against task complexity, which skill files to load and how to evaluate those strategies, and how to monitor risks from vulnerable skill definitions.
Open Questions
- The experiment limits retrieval to one skill per task, leaving open how many skills tasks need and how to balance definition verbosity with task complexity.In practice, the agent has no prior knowledge of the number of skills a task requires.Source: Open Questions · p. 3
- The experiment limits each skill definition to SKILL.md; strategies for loading additional files and evaluating them remain open questions.Skill directories may also contain scripts, references, or assets that use more LLM context.Source: Open Questions · p. 3
- Levy, Jacoby, and Goldberg (2024) also report reasoning-quality degradation as LLM prompt size increases.Source: Open Questions · p. 3
- The report asks how developers can monitor risk from serious vulnerabilities hidden in skill definitions as skill-library adoption grows.Source: Open Questions · p. 4