PaperlayerView exact revision on arXiv

Source-linked AI summary

SkillReducer: Optimizing LLM Agent Skills for Token Efficiency

Yudong Gao, Zongjie Li, Yuanyuan Yuan, Zimo Ji, Pingchuan Ma, Shuai Wang

arXiv:2603.29919v2cs.SE

TL;DR

Coding-agent skills can consume substantial context tokens, and a large-scale study finds inefficiencies in routing descriptions and skill bodies. SkillReducer compresses descriptions and restructures bodies for progressive disclosure, achieving 48% description compression and 39% body reduction while improving functional quality by 2.8%.

  • Problem

    A study of agent skills examines token inefficiencies in routing descriptions and instruction bodies, which are injected into context windows and incur costs.

    Source: §I · p. 1

  • Method

    SkillReducer combines delta-debugging-based description compression with taxonomy-driven progressive disclosure of skill bodies.

    Source: §I · p. 2

  • Results

    48% mean description compression and 39% mean body-token reduction, while compressed skills improved functional quality by 2.8% over originals.

    Source: §I · p. 1

  • Takeaways & Limitations

    The reported less-is-more effect indicates that removing non-essential content reduces context-window distraction, particularly for longer, more verbose skills.

    Source: §I · p. 2

    Evaluation uses Anthropic’s skill protocol; generalization to other platforms such as Cursor and Windsurf remains untested.

    Source: §VII · p. 11

Abstract

from arXiv · show

LLM-based coding agents rely on \emph{skills}, pre-packaged instruction sets that extend agent capabilities, yet every token of skill content injected into the context window incurs both monetary cost and attention dilution. To understand the severity of this problem, we conduct a large-scale empirical study of 55,315 publicly available skills and find systemic inefficiencies: 26.4\% lack routing descriptions entirely, over 60\% of body content is non-actionable, and reference files can inject tens of thousands of tokens per invocation. Motivated by these findings, we present \textsc{SkillReducer}, a two-stage optimization framework. Stage~1 optimizes the routing layer by compressing verbose descriptions and generating missing ones via adversarial delta debugging. Stage~2 restructures skill bodies through taxonomy-driven classification and progressive disclosure, separating actionable core rules from supplementary content loaded on demand, validated by faithfulness checks and a self-correcting feedback loop. Evaluated on 600 skills and the SkillsBench benchmark, \textsc{SkillReducer} achieves 48\% description compression and 39\% body compression while improving functional quality by 2.8\%, revealing a \emph{less-is-more} effect where removing non-essential content reduces distraction in the context window. These benefits transfer across five models from four families with a mean retention of 0.965, and generalize to an independent agent framework.

I. INTRODUCTION · II. BACKGROUND · A. LLM Agent Skills

SkillReducer reduces skill tokens while preserving or improving task performance, addressing inefficiencies in routing descriptions and skill bodies. It combines description optimization with restructuring skill content for progressive disclosure.

I. INTRODUCTION

  • Skills are reusable, pre-packaged instructions that extend coding-agent behavior, but their ecosystem lacks the mature optimization practices of traditional source code.They are authored, versioned, shared in marketplaces and forums, and maintained by developers.Source: §I · p. 1
  • Across 55,315 skills, 26.4% lacked routing descriptions, over 60% of body content was non-actionable, and reference-heavy skills could inject tens of thousands of tokens per invocation.The paper also estimates that a 10,000-token skill costs $0.03 to $0.15 per invocation at current API pricing.Source: §I · p. 1
  • SkillReducer uses adversarial-oracle delta debugging to compress or generate routing descriptions, then classifies bodies to keep actionable rules loaded and move supplementary content into on-demand modules.Body restructuring is checked for faithfulness and evaluated on tasks with a self-correcting feedback loop.Source: §I · pp. 1–2
  • 48% mean description compression and 39% mean body-token reduction accompanied an 86.0% task-evaluation pass rate, a 100% SkillsBench pass rate, and 2.8% better functional quality than originals.The reported quality gain suggests that removing non-essential content can reduce distraction, particularly in longer, more verbose skills.Source: §I · pp. 1–2
  • At equivalent token budgets, SkillReducer significantly outperformed LLMLingua, direct LLM compression, truncation, and random removal, with benefits transferring across five models from four families and an independent framework.The paper presents these comparisons as evidence of robustness across models and an agent framework.Source: §I · p. 2

A. LLM Agent Skills

  • A skill has a routing description, an injected instruction body, optional reference files, and executable scripts, which are outside this work’s token-focused scope.The runtime matches user requests against descriptions, then injects the selected skill’s body and references into the context window.Source: A. LLM Agent Skills · p. 2
  • Tiered skills keep a compact core loaded while making supplementary modules available on demand, following progressive disclosure rather than loading all content at once.SkillReducer’s Stage 2 automates conversion from a monolithic skill to this tiered architecture.Source: A. LLM Agent Skills · p. 2

B. Context Window Constraints and Prompt Compression … B. Body Content Taxonomy

The study finds that skill descriptions and bodies are poorly calibrated for context-window use: descriptions may be missing or verbose, and most body content is not core rules. These findings motivate structure-aware analysis and compression that preserve routing correctness and task performance.

B. Context Window Constraints and Prompt Compression

  • Skills can consume tens of thousands of tokens per invocation, and their mixed actionable rules, examples, and background make uniform pruning unsuitable.Context windows must also accommodate system prompts, conversation history, and codebase context, motivating structure-aware compression that preserves routing and task performance.Source: B. Context Window Constraints and… · p. 2

C. Delta Debugging

  • Stage 1 adapts delta debugging by treating routing-description clauses as changes and routing correctness as the predicate, seeking a minimal description that preserves routing behavior.DDMIN binary-partitions changes and discards unnecessary subsets until every remaining element is necessary.Source: C. Delta Debugging · p. 2

III. EMPIRICAL STUDY

  • The motivating empirical study covers 55,315 public GitHub skills, 100 curated SkillHub skills, and 620 community skills.The GitHub skills were filtered for valid SKILL.md files and deduplicated by content hash.Source: §III · p. 2

A. Description Quality

  • 26.4% of Wild skills lack descriptions, 44.1% are missing or under 20 tokens, while SkillHub descriptions average 47.78 tokens and often include non-routing content.Missing or very short descriptions waste router tokens; verbose descriptions can be trimmed without affecting routing accuracy.Source: A. Description Quality · p. 3

B. Body Content Taxonomy

  • The body taxonomy classifies 15,107 paragraph-level items from 90 stratified skills into five categories, using an LLM classifier and contextual cross-validation.The sample included 30 skills per source and exceeded the cited saturation threshold of 12–30 samples per stratum; the classification methodology follows.Source: B. Body Content Taxonomy · p. 3
  • Only 38.5% of body items are core rules, versus 40.7% background and 12.9% examples; over 60% is background, examples, or templates.Clustering also yielded five taxonomy-aligned clusters, with moderate separation (silhouette = 0.393).Source: Table I · p. 3, B. Body Content Taxonomy · p. 3, Figure 2 · p. 3

C. File Composition · IV. APPROACH · A. Stage 1: Routing Layer Optimization

SkillReducer treats skill bloat as a separation-of-concerns problem and uses staged optimization to reduce token cost while preserving routing and task performance. Stage 1 minimizes existing routing descriptions or generates missing ones, with validation against real agent behavior.

C. File Composition

  • 14.8% of Wild skills include reference files, and 100 SkillHub skills with references contain 505 files totaling 1.67M tokens.A single invocation can therefore inject tens of thousands of reference tokens even when only a fraction is relevant.Source: C. File Composition · p. 3

IV. APPROACH

  • SkillReducer frames mixed specifications, documentation, and data as a separation-of-concerns problem and targets them with staged optimization.Its objective is lower invocation token cost subject to routing equivalence and functional retention; Stage 1 minimizes descriptions, while Stage 2 classifies and compresses body content and enables task-dependent reference loading.Source: C. File Composition · p. 3, §IV · p. 4

A. Stage 1: Routing Layer Optimization

  • Stage 1 addresses both missing or too-short descriptions that impair routing and verbose descriptions by generating or compressing them to minimally sufficient routing content.For existing descriptions, it retains only content needed to distinguish a skill from competitors.Source: A. Stage 1: Routing Layer Optimization · p. 4
  • The simulated routing oracle tests queries against the target, four TF-IDF-similar distractors, and an LLM-generated adversarial skill, with candidate order randomized.The adversarial skill is topically similar but functionally distinct; the oracle passes only if the router consistently selects the target for every query.Source: A. Stage 1: Routing Layer Optimization · p. 4
  • DDMIN removes unnecessary semantic clauses until each remaining clause is individually necessary for routing, using O(n log n) oracle calls rather than O(2^n) exhaustive search.Retained clauses are paraphrased more briefly only when routing still passes, then polished for grammatical coherence.Source: A. Stage 1: Routing Layer Optimization · p. 4
  • Because the simulated oracle may be overly optimistic, Phase 2 checks whether the actual agent runtime triggers the compressed skill for the original-description validation queries.If any query fails, deleted units are greedily restored to improve the trigger rate, for at most three steps; the original description is the fallback.Source: A. Stage 1: Routing Layer Optimization · pp. 4–5
  • 100% routing preservation across all non-obsolete skills was achieved by the selective-restore mechanism in the two-phase description pipeline.The evaluation covered 600 skills.Source: A. Stage 1: Routing Layer Optimization · p. 5
  • For the 26.4% of skills with missing or short descriptions of ≤40 tokens, SkillReducer generates descriptions from the body using three routing signals.The signals are primary capability, trigger condition, and unique identifiers; the generated description is checked by the simulated oracle, with real-trigger validation on a sampled subset.Source: A. Stage 1: Routing Layer Optimization · p. 5

B. Stage 2: Body Restructuring via Progressive Disclosure

Stage 2 transforms monolithic skill bodies into an always-loaded core and references loaded on demand, addressing the finding that only 38.5% of body content is actionable. It uses taxonomy-based classification, type-specific compression, and quality checks with a feedback loop to preserve task performance.

B. Stage 2: Body Restructuring via Progressive Disclosure

  • Stage 2 keeps core rules in the always-loaded module, moves background, examples, and templates to on-demand references, and discards redundant content.A taxonomy-driven classifier assigns paragraph-level items to five categories; agents can select annotated references to load using a read_file call.Source: B. Stage 2: Body Restructuring via… · pp. 5–6
  • Type-specific compression tightens core rules, keeps one representative example per concept, deduplicates templates, summarizes background, and removes overlapping reference content.Example count typically falls by 60–70%; references shrinking below 30 tokens are discarded, and surviving references receive trigger and topic annotations.Source: B. Stage 2: Body Restructuring via… · p. 6
  • Faithfulness checks and task-based evaluation safeguard compression, while a feedback loop promotes needed non-core content into the core to recover task performance.When retention falls below the original skill’s score, identified items are promoted in their original form; the loop runs at most twice and trades a small token-cost increase for full functional retention.Source: B. Stage 2: Body Restructuring via… · pp. 6–7
  • Gate 2 tests five diverse tasks per skill under no-skill, original-skill, and compressed-skill conditions, using code assertions or rubric-based LLM judging.Code execution accounts for 52.3% of evaluations and rubric tasks 47.7%; on 1,646 overlapping tasks, the judge and deterministic verifier achieve Cohen’s κ=0.939.Source: B. Stage 2: Body Restructuring via… · p. 7
  • The marketing-strategy example compresses a 2,543-token body to a 540-token core and reports scoreC=1.0 versus scoreA=0.93.Three reference modules are created, and end-to-end invocation cost drops from 12,019 tokens to 540 for core-only tasks or 7,231 when all references are loaded.Source: B. Stage 2: Body Restructuring via… · p. 7

V. EVALUATION

The evaluation examines SkillReducer’s token efficiency, functional quality, quality-preservation components, and generalization across models and frameworks.

V. EVALUATION

  • The evaluation asks how much token reduction SkillReducer achieves, whether quality is preserved, what drives preservation, and whether results generalize across models and frameworks.It covers 600 skills: 87 official SkillHub skills, 464 community skills, and 49 wild samples from GitHub, plus one external benchmark.Source: §V · p. 7

A. Experimental Setup

The evaluation combines a curated 600-skill set with SkillsBench and compares original skills against SkillReducer’s compressed core and a no-skill lower bound. It measures compression and task performance using paired statistical tests, alongside large-scale task, routing, and generalization evaluations.

Datasets

  • The curated evaluation set includes 87 Official, 464 Community, and 49 Wild skills; SkillsBench supplies an independent benchmark of 87 tasks across 229 skills.The three-source set filters low-quality and duplicate content; SkillsBench uses deterministic pytest verifiers.Source: A. Experimental Setup · pp. 7–8

Conditions

  • Condition D provides no skill, Condition A provides the original body and all references, and Condition C provides SkillReducer’s compressed core with references loaded on demand.Source: A. Experimental Setup · p. 8

Metrics

  • The study reports compression ratio, pass rate, improvement rate, and retention, with bootstrap 95% confidence intervals.It uses Wilcoxon signed-rank tests for paired A/C comparisons and one-sided sign tests for the baseline comparison because of ceiling ties; Cohen’s d is an approximate effect size.Source: A. Experimental Setup · p. 8

Implementation

  • DeepSeek-V3 performs both stages’ compression and verification work, while Qwen3.5 generates, executes, and evaluates tasks in a separate session.DeepSeek-R1 serves as the simulated routing oracle, and real-trigger routing validation runs through Claude Code CLI.Source: A. Experimental Setup · p. 8

Scale

  • The evaluation compresses 600 skills and evaluates 3,000 tasks across no-skill, original-skill, and SkillReducer conditions.It also validates routing on 600 skills in real agent environments and tests generalization across five models from four families and an independent agent framework.Source: A. Experimental Setup · p. 8

B. RQ1: Token Reduction

SkillReducer reduces description and body tokens, with end-to-end input savings; compression also scales to a sample of wild skills. The results include distinct compression patterns for existing versus generated descriptions and for skills of different body lengths.

B. RQ1: Token Reduction

  • Existing descriptions longer than 40 tokens were reduced by 56.5% on average, while generated descriptions were reduced by 44.6%.The 44.6% reduction is relative to the generated description, not the original short or missing description; overall Stage 1 reduction was 48.0%.Source: B. RQ1: Token Reduction · p. 8
  • 39.0% mean body reduction saved about 1,000 tokens per skill, while SkillsBench bodies fell from 359K to 84K across 84 successfully compressed tasks.SkillsBench’s mean per-task reduction was 75.0%.Source: B. RQ1: Token Reduction · p. 8
  • On 198 successfully processed wild skills, mean core reduction was 77.5%, rising from 58.8% for tiny skills to 95.8% for extra-large skills.Faithfulness verification passed for 148 of 198 skills (74.7%); the higher reduction partly reflects the absence of Gate 2 feedback and the wild sample’s longer, more redundant bodies.Source: B. RQ1: Token Reduction · p. 8
  • At an estimated $14–18 one-time cost to compress 600 skills, the expense amortizes within a few hundred invocations per skill.The estimate assumes about 20–40 LLM calls per skill and an average low-cost open-source model input rate of about $0.3 per million tokens.Source: B. RQ1: Token Reduction · p. 8
  • 48.0% mean description reduction, 39.0% mean body reduction, and 26.8% mean end-to-end input savings summarize SkillReducer’s token savings.Mean agent output changed by +1.7% and median output by −0.3%.Source: B. RQ1: Token Reduction · p. 8

C. RQ2: Functional Quality · D. RQ3: Component Contribution

Compression preserved or improved functional quality for most skills, while taxonomy-driven classification made the largest contribution among tested components. Routing validation and a feedback loop addressed failures in compressed descriptions and skill bodies.

C. RQ2: Functional Quality

  • 86.0% of skills had scoreC ≥ scoreA, and condition C averaged 0.742 versus 0.722 for condition A (p = 0.002, d = 0.107).After excluding ceiling-effect skills (N=481), the advantage remained significant (p = 0.010).Source: Table III · p. 9, C. RQ2: Functional Quality · p. 9
  • At the same token budget, SkillReducer was compared on 50 skills against LLMLingua, LLM direct compression, truncation, and random sentence removal.The supplied passages describe the comparison setup but do not report its outcomes.Source: Table IV · p. 9, C. RQ2: Functional Quality · p. 9
  • 14.0% of skills regressed; among these 84 cases, 50% reflected obsolescence, 17% evaluation noise, and 33% true compression regression.The recurring failure mode was example-as-specification: examples implicitly defining expected behavior were moved to reference modules.Source: C. RQ2: Functional Quality · p. 9
  • 536/536 non-obsolete skills (100%) preserved routing after compressed descriptions were tested, with 245 requiring selective restore and 39 falling back to the original description.The remaining 64 skills did not trigger even with the original description, suggesting obsolescence.Source: Table V · p. 9, C. RQ2: Functional Quality · p. 9

D. RQ3: Component Contribution

  • Without taxonomy-driven classification, retention dropped to 0.919, a 6.8pp gap versus the full pipeline, making classification the key component in the ablation study.The ablation evaluated six conditions on 50 skills.Source: Table VI · p. 9, D. RQ3: Component Contribution · p. 9
  • Reference deduplication achieved retention 1.000 versus 0.939 for the original, while description compression had negligible task-performance effect (∆A = −0.014).Reference deduplication kept the full body untouched and removed only redundant reference content.Source: D. RQ3: Component Contribution · p. 9
  • 31/38 skills that triggered feedback recovered after one or two iterations, an 81.6% total recovery rate.Of 38 skills (6.3% of 600) triggering feedback, 22 recovered in the first round and 9 of the remaining 16 in the second.Source: D. RQ3: Component Contribution · p. 9

E. RQ4: Cross-Model and Cross-Framework Generalization … VIII. CONCLUSION

Compression transfers across tested models and an independent agent framework, while the paper positions SkillReducer as a structure-aware, build-time complement to prior methods. The evaluation has stated scope limitations, and the conclusion links skill obsolescence to adaptive lifecycle management.

E. RQ4: Cross-Model and Cross-Framework Generalization

  • Retention remained 0.897 (Qwen3-max) and 0.874 (Qwen2.5-7B) versus 0.896 (DeepSeek-V3) when the compressor varied.The evaluation fixed the evaluator and re-compressed the same 30 skills, supporting the authors’ interpretation that structure-aware design drives quality preservation.Source: E. RQ4: Cross-Model and… · p. 10
  • Retention averaged 0.965 across four model families, ranging from 0.939 (Qwen2.5-7B) to 0.986 (GLM-5).The evaluation covered 30 skills and five tasks per skill; GPT-OSS-120B achieved retention 0.982 despite using flat injection without tool-calling support.Source: Table VIII · p. 10, E. RQ4: Cross-Model and… · p. 10
  • OpenCode achieved mean retention of 0.944, while compressed skills scored 0.764 versus 0.751 for originals.The cross-framework evaluation used 30 skills, three tasks per skill, and conditions A and C; the retention was consistent with 0.949 in the main baseline comparison.Source: E. RQ4: Cross-Model and… · p. 10
  • 10.7% of Stage 1 skills did not trigger even with original descriptions, while condition D achieved 98.9% on SkillsBench.Qwen2.5-7B showed the largest gap between conditions A and D, motivating adaptive lifecycle management and Gate 2 identification of retirement candidates.Source: E. RQ4: Cross-Model and… · p. 10

VI. RELATED WORK

  • SKILLREDUCER adapts flat-sequence compression and context-management approaches by preserving actionable rules separately from supplementary skill content,,,, [21], [22], [23],, [25], [26], [27].It applies software debloating, delta debugging, and program slicing,,, as a training-free, post-authoring build-time pass that complements authoring-time and runtime approaches,,, [32],,.Source: §VI · pp. 10–11

VII. THREATS TO VALIDITY

  • The evaluation’s external validity is limited to Anthropic’s skill protocol, five models from four families, and 30 cross-model skills; Cursor, Windsurf, and additional families remain untested.A stratified study of 198 wild skills supports scalability but lacks Gate 2 evaluation due to cost; internal validity mitigations include temperature 0, conservative fallbacks, a feedback loop, and judge validation (κ=0.939 on 1,646 tasks).Source: §VII · p. 11

VIII. CONCLUSION

  • Across 55,315 skills, SkillReducer’s routing compression (48%) and progressive disclosure (39% body reduction) achieved an 86% pass rate, with results transferring across models and an independent framework.The conclusion reports retention of 0.965 across five models from four families and 0.944 in an independent framework; 4.7% of skills were true compression failures, while the remainder stemmed from obsolescence, motivating adaptive lifecycle management.Source: §VIII · p. 11

APPENDIX A RUNNING EXAMPLE: MARKETING-STRATEG Y-P M M … APPENDIX E THEORETICAL ANALYSIS

The appendices walk through a skill-compression example and clustering choice, then describe quality safeguards and report dataset statistics and theoretical properties.

APPENDIX A RUNNING EXAMPLE: MARKETING-STRATEG Y-P M M

  • The example compresses the description 87→32 tokens (63%), body 2,543→540 tokens (79%), and original references 9,476→6,691 tokens (29%).It routes templates, persona examples, and methodological background into reference modules loaded on demand.Source: Appendix A · p. 12
  • Invocation cost falls from 12,019 to 540 tokens (96%) for core-only tasks, or to 7,231 tokens (40%) with all references; Gate 2 scores scoreC=1.0 vs. scoreA=0.93.Source: Table IX · p. 12, Appendix A · p. 12

APPENDIX B GMM MODEL SELECTION

  • The GMM silhouette score first peaks locally at k = 5 (0.393), matching the five-category taxonomy; higher k values do not consistently improve separation.Source: Figure 5 · p. 12, Appendix B · p. 12

APPENDIX C DATASET STATISTICS

APPENDIX D QUALITY SAFEGUARDS

  • The safeguards combine a faithfulness gate, a feedback loop that promotes failed non-core items into the core, and a fallback to the best compression after two iterations.Gate 1 rolls back content if key operational concepts are missing; the loop recompresses and retests when scoreC < scoreA.Source: Appendix D · pp. 12–13
  • 86.0% of 600 skills achieve scoreC ≥scoreA; regressions typically occur when examples contain implicit rules that taxonomy-based classification moves out of the core.The authors identify deeply interleaved content and implicit dependencies as a limitation of automated separation.Source: Appendix D · p. 13

APPENDIX E THEORETICAL ANALYSIS

  • The theoretical analysis formalizes the feedback loop’s convergence guarantee and the expected token cost under progressive disclosure.Source: Appendix E · p. 13

A. Feedback Loop Convergence

Gate 2 feedback expands the skill core by promoting items when checks fail, ensuring finite termination; empirically, after at most two iterations, most triggered skills reach or exceed their original retention score.

A. Feedback Loop Convergence

  • On each Gate 2 failure, the loop adds at least one item, so the core grows monotonically and terminates within |I| − |I(0)_core| iterations.If Gate 2 is monotone in coverage, adding items cannot decrease scoreC.Source: A. Feedback Loop Convergence · p. 13
  • In practice, the feedback loop is capped at K = 2 iterations, although added content can distract and violate the assumption that more content cannot reduce task performance.Promotions are selected to address failed rubric criteria, which makes distraction unlikely but does not eliminate it.Source: A. Feedback Loop Convergence · p. 13
  • 71.1% (27/38) of triggered skills achieve scoreC ≥ scoreA after at most two iterations; the remaining 28.9% experience distraction that offsets informational benefits.The promoted items target failed rubric criteria, making distraction unlikely, but the monotonicity assumption can still be violated.Source: A. Feedback Loop Convergence · p. 13

B. Expected Cost Under Progressive Disclosure

Under progressive disclosure, expected invocation cost includes the description and compressed core on every call, plus reference modules only when loaded. Using empirical loading and compression estimates, the analysis reports expected body-cost reductions of 26.8%–43.2% across 490 skills.

B. Expected Cost Under Progressive Disclosure

  • Expected invocation cost counts the description and compressed core on every call, plus each reference module weighted by its independent probability of being loaded.The core compression factor α captures merging and tightening of core rules.Source: B. Expected Cost Under Progressive… · p. 13
  • The estimate uses ρ = 0.383, α ≈0.63, k ≈2.1 reference modules per skill, and p ≈0.30, with reference size ≈0.617·|s.b|/k.The analysis assumes a uniform reference-loading probability p for its empirical instantiation.Source: B. Expected Cost Under Progressive… · p. 13

Related papers

Loading 2603.29919v2…