PaperlayerView exact revision on arXiv

Source-linked AI summary

SkillReducer: Optimizing LLM Agent Skills for Token Efficiency

Yudong Gao, Zongjie Li, Yuanyuan Yuan, Zimo Ji, Pingchuan Ma, Shuai Wang

arXiv:2603.29919v2cs.SE

TL;DR

LLM agent skills can impose substantial token costs because routing descriptions and injected bodies or references contain systemic inefficiencies. SkillReducer optimizes both layers using delta-debugging-based routing compression and taxonomy-driven progressive disclosure. Across 600 skills and SkillsBench, it reduces tokens while preserving or improving functional quality, with benefits transferring across models and an independent framework.

  • Problem

    55,315 skills exhibit systemic inefficiencies: 26.4% lack routing descriptions, over 60% of body content is non-actionable, and references can inject tens of thousands of tokens.

  • Method

    SkillReducer uses delta debugging to compress or generate routing descriptions and taxonomy-driven progressive disclosure to separate actionable body content from supplementary material.

  • Results

    48% description compression and 39% body-token reduction accompany a 2.8% functional-quality improvement, with mean retention of 0.965 across five models from four families.

  • Takeaways & Limitations

    Removing non-essential skill content can reduce context distraction while retaining or improving functional quality, and the optimized skills transfer across models and an independent agent framework.

  • Takeaways & Limitations

    The evaluation uses skills following Anthropic’s protocol; generalization to Cursor and Windsurf remains untested, and additional model families remain uncovered.

Abstract

from arXiv · show

LLM-based coding agents rely on \emph{skills}, pre-packaged instruction sets that extend agent capabilities, yet every token of skill content injected into the context window incurs both monetary cost and attention dilution. To understand the severity of this problem, we conduct a large-scale empirical study of 55,315 publicly available skills and find systemic inefficiencies: 26.4\% lack routing descriptions entirely, over 60\% of body content is non-actionable, and reference files can inject tens of thousands of tokens per invocation. Motivated by these findings, we present \textsc{SkillReducer}, a two-stage optimization framework. Stage~1 optimizes the routing layer by compressing verbose descriptions and generating missing ones via adversarial delta debugging. Stage~2 restructures skill bodies through taxonomy-driven classification and progressive disclosure, separating actionable core rules from supplementary content loaded on demand, validated by faithfulness checks and a self-correcting feedback loop. Evaluated on 600 skills and the SkillsBench benchmark, \textsc{SkillReducer} achieves 48\% description compression and 39\% body compression while improving functional quality by 2.8\%, revealing a \emph{less-is-more} effect where removing non-essential content reduces distraction in the context window. These benefits transfer across five models from four families with a mean retention of 0.965, and generalize to an independent agent framework.

I. INTRODUCTION

LLM agent skills are reusable instruction packages intended to save tokens, but current authoring practices often create routing failures, excessive context costs, and non-actionable content. SkillReducer addresses this bloat with two-stage optimization and achieves substantial compression while preserving or improving functional quality.

  • Motivation: Skills package reusable rules, templates, and references, but their descriptions and bodies are loaded differently for routing and execution.Descriptions guide invocation, while bodies and references consume context tokens after selection.
  • Motivation: 26.4% of 55,315 skills lack descriptions, over 60% of body content is non-actionable, and reference-heavy skills can inject tens of thousands of tokens.These inefficiencies affect both the routing layer and the content injected into the context window.
  • Approach: SkillReducer combines description compression and generation through delta debugging with taxonomy-driven progressive disclosure for skill bodies.The framework transforms monolithic content into a compact core with supplementary material loaded on demand.
  • Results: 48% mean description compression and 39% mean body-token reduction preserve functional quality across 600 skills and SkillsBench evaluations.The reported evaluation includes an 86.0% task-evaluation pass rate and a 100% SkillsBench pass rate.
  • Results: Compressed skills improve functional quality by 2.8%, supporting a less-is-more effect in which removing non-essential content reduces context distraction.The benefits transfer across five models from four families and an independent agent framework.

B. Context Window Constraints and Prompt Compression

LLM agent skills can strain fixed context windows because bodies, references, and poorly calibrated descriptions consume tokens beyond what routing and task execution require. The paper studies these costs and motivates structured compression through content taxonomy and delta debugging.

  • B. Context Window Constraints and Prompt Compression: 128K–200K-token context windows must accommodate system prompts, skill content, conversation history, and codebase context.Large skills and references can consume tens of thousands of tokens per invocation.
  • C. Delta Debugging: Delta debugging minimizes semantic routing clauses under a routing-correctness predicate, adapting software test-case minimization to skill descriptions.The method seeks a 1-minimal subset that preserves routing behavior.
  • III. EMPIRICAL STUDY: 55,315 public skills were analyzed across Wild, SkillHub, and Community sources to characterize token-length and content patterns.The sources include 55,315 Wild skills, 100 SkillHub skills, and 620 Community skills.
  • A. Description Quality: 26.4% of 55,315 Wild skills have no description, while 44.1% are missing or under 20 tokens.These descriptions remain in the router candidate pool but cannot match relevant requests.
  • A. Description Quality: SkillHub descriptions average 47.78 tokens and often include feature lists, trigger enumerations, and examples that do not aid routing.The study identifies a bimodal quality problem: descriptions may be too short or unnecessarily verbose.
  • B. Body Content Taxonomy: Only 38.5% of sampled body items are core rules, while 40.7% are background and 12.9% are examples.The taxonomy classifies body items as core rule, background, example, template, or redundant.
  • B. Body Content Taxonomy: GMM clustering with k = 5 produced five UMAP clusters aligned with the taxonomy categories, with silhouette = 0.393.The alignment suggests the content distinctions are structurally grounded, although separation is moderate.
  • B. Body Content Taxonomy: Over 60% of skill body content is background, examples, or templates injected regardless of task relevance.This finding identifies substantial non-actionable content in the invoked body.

C. File Composition

Skills combine routing descriptions, instruction bodies, references, and scripts, creating distinct token-cost and optimization targets. SkillReducer addresses these components with staged, validated reduction while preserving routing and functional behavior.

  • File Composition: 14.8% of 55,315 Wild skills include reference files, and 8.3% include scripts.Most Wild skills are single-file, but auxiliary files create additional invocation costs.
  • File Composition: The 100 SkillHub skills with references contain 505 files totaling 1.67M tokens.A single invocation can inject tens of thousands of reference tokens even when only a fraction is relevant.
  • File Composition: Monolithic authoring mixes core rules, documentation, examples, templates, and references without separating essential from on-demand content.The paper frames this as an automated separation-of-concerns problem.
  • Design Requirements: SkillReducer removes non-essential skill content while targeting both routing efficiency and functional-quality retention.Its design is driven by the empirical findings about descriptions, bodies, and references.
  • Stage 1: Routing Layer Optimization: Stage 1 generates missing descriptions and compresses verbose descriptions to minimally sufficient routing forms.It uses semantic clauses, an adversarial simulated oracle, delta debugging, and real-environment validation.
  • Stage 2: Body and References: Stage 2 classifies body content, applies type-specific compression, deduplicates references, and validates faithfulness and task quality.A feedback loop supports correction when validation gates are not satisfied.
  • Optimization Objective: The optimization objective minimizes Cost(s) = |s.d| + |s.b| + |r| while preserving routing equivalence and functional retention.The retained skill should trigger for the same queries and achieve the same task performance as the original.
  • Stage 1: Routing Layer Optimization: Real-environment validation tests compressed descriptions through the Claude Code CLI, with selective restoration or fallback to the original description when needed.The two-phase design addresses the simulated oracle’s controlled-candidate-pool limitation.

B. Stage 2: Body Restructuring via Progressive Disclosure

Stage 2 restructures monolithic skill bodies into an always-loaded core and reference modules loaded on demand. It classifies content, applies type-specific compression and deduplication, then uses faithfulness and task-based gates with feedback to preserve functionality.

  • Motivation: 38.5% of body content is actionable, motivating separation of core rules from examples, templates, and background material.The taxonomy classifies paragraph-level items as core rule, background, example, template, or redundant.
  • Progressive disclosure: Stage 2 transforms each skill into an always-loaded core module b* and on-demand reference modules R*.References are loaded only when explicitly requested by the agent.
  • Type-specific compression: Content classification enables tailored compression: core rules are tightened, examples deduplicated, templates deduplicated, and background summarized.Examples typically decrease by 60–70% after retaining one representative example per concept.
  • Cross-file deduplication: External references are overlap-checked against the original body, compressed when needed, and discarded when fewer than 30 tokens remain.The process removes material already captured in core or body-derived reference modules.
  • Quality safeguards: Gate 1 checks structural faithfulness per content type, while Gate 2 tests real tasks and promotes missed dependencies through a feedback loop.The loop runs at most twice; if retention remains below 1.0, the best current compression is retained.

V. EVALUATION

The evaluation measures token reduction, functional-quality preservation, component contributions, and generalization across models and agent frameworks.

  • Research questions: The evaluation addresses four questions covering token reduction, functional quality, quality-preserving components, and generalization.These questions organize the empirical assessment of SKILLREDUCER.
  • RQ1: Token reduction: RQ1 asks how much token reduction SKILLREDUCER achieves.This question concerns the framework’s compression effectiveness.
  • RQ2: Functional quality: RQ2 asks whether compression preserves functional quality.The evaluation compares task performance after optimization with the original skills.

A. Experimental Setup

The study evaluates SKILLREDUCER on a curated 600-skill set and SkillsBench using three task conditions, multiple metrics, separate evaluation models, and broad generalization tests.

  • Datasets: The evaluation set contains 600 skills from official, community, and wild sources, plus SkillsBench as an external benchmark.SkillsBench includes 87 tasks across 229 skills with deterministic pytest verifiers.
  • Conditions: Tasks compare no skill, the original skill with all references, and the compressed core with references loaded on demand.The compressed condition permits up to six read_file calls per task.
  • Metrics: Metrics include compression ratio, pass rate, improvement rate, and retention, with paired non-parametric comparisons and bootstrap 95% confidence intervals.The paired design compares original and compressed conditions for each skill.
  • Implementation and scale: Evaluation uses separate models for compression and assessment and tests generalization across five models from four families and an independent framework.The full evaluation includes 3,000 tasks across three conditions.

B. RQ1: Token Reduction

SKILLREDUCER reduces tokens substantially at both routing and body layers, with especially large reductions on SkillsBench and additional compression on wild skills.

  • Description compression: 48.0% mean description token reduction is achieved overall, with a 59.0% median and a 95% CI of [45.2%, 50.8%].Existing descriptions had 56.5% mean reduction, while generated descriptions had 44.6% reduction relative to generated text.
  • Body compression: 39.0% mean body token reduction saves approximately 1,000 tokens per skill.The median body reduction is 43.0%, with a 95% CI of [36.2%, 41.8%].
  • SkillsBench: 75.0% mean per-task body reduction occurs on SkillsBench, where body tokens fall from 359K to 84K.This result covers 84 of 87 successfully compressed tasks.
  • Overall and wild-skill results: 26.8% end-to-end savings are reported across the evaluation, while Stage 2 compression scales to 77.5% on 198 wild-skill samples.The wild-skill result uses successful samples from a stratified validation set.

C. RQ2: Functional Quality

SKILLREDUCER preserves functional quality while compressing skills, and often improves performance by removing distracting non-essential content. Its feedback and component analyses identify which mechanisms support this outcome.

  • Routing Preservation: 100% of non-obsolete skills preserved routing after selective restoration or fallback, while 64 skills were classified as obsolete.Among 600 skills, 252 passed directly, 245 required selective restore, and 39 fell back to the original description.
  • Task Evaluation: 86.0% of skills achieved scoreC ≥ scoreA, with 25.3% improving and 14.0% regressing.The result remained favorable after excluding ceiling-effect skills (N=481, p = 0.010).
  • Task Evaluation: Condition C outperformed condition A on average, scoring 0.742 versus 0.722 (p = 0.002, d = 0.107).This supports a less-is-more effect in which non-essential content can distract the agent from core instructions.
  • Ablation Study: Taxonomy-driven classification was the key component: removing it reduced retention to 0.919, a 6.8pp gap versus the full pipeline.Reference deduplication achieved retention 1.000, while description compression had negligible task-performance effect (∆A = −0.014).
  • Feedback Loop: 81.6% of skills triggering feedback recovered after one or two iterations, with 31 of 38 triggered skills recovering.Twenty-two recovered in the first round and nine of the remaining sixteen in the second.

E. RQ4: Cross-Model and Cross-Framework Generalization

SKILLREDUCER’s compression benefits generalize across models, compressors, and an independent agent framework. The reported evaluations indicate that preservation depends more on the structure-aware pipeline than on any single compressor or model.

  • Cross-Model Generalization: Mean retention was 0.965 across five models from four model families, ranging from 0.939 to 0.986.Compression transferred to the smaller Qwen2.5-7B model and to GPT-OSS-120B under flat injection without tool calling.
  • Cross-Compressor Validation: Retention remained stable across compressors: 0.897 with Qwen3-max and 0.874 with Qwen2.5-7B versus 0.896 with DeepSeek-V3.The evaluation fixes the evaluator while varying the compressor.
  • Cross-Framework Validation: OpenCode achieved mean retention of 0.944, consistent with the 0.949 observed in the main baseline comparison.This evaluation used OpenCode’s native skill mechanism and a different system prompt, tool registry, and routing tool.
  • Scope and Validity: The paper reports transfer across four model families and to OpenCode, while identifying untested platforms and additional model families as external-validity boundaries.Generalization to Cursor and Windsurf remains untested, and the cross-model study covers 30 skills.
  • Design Context: SKILLREDUCER is training-free and applies type-specific compression to skill structure rather than treating inputs as flat token sequences.Its design combines software-engineering techniques with progressive disclosure while preserving on-demand access to full content.

APPENDIX A RUNNING EXAMPLE: MARKETING-STRATEG Y-P M M

The running example illustrates how SKILLREDUCER converts a monolithic skill into a compressed, tiered representation. The appendix also connects the pipeline’s taxonomy to five-category clustering and its safeguards to quality preservation.

  • Stage 1: Description: The marketing-strategy-pmm description shrank from 87 to 32 tokens, a 63% reduction.The original listed many features and trigger phrases for product marketing, positioning, and go-to-market strategy.
  • Taxonomy Selection: Taxonomy selection uses a GMM whose first local silhouette peak occurs at k = 5 with silhouette = 0.393, matching the five-category taxonomy.Higher values of k do not consistently improve separation.
  • Quality Safeguards: The pipeline’s quality safeguards verify that key operational concepts remain in the compressed core and promote failed non-core items back into the core.Gate 1 rolls back content types when concepts are missing; Gate 2 recompresses and retests after failures.
  • Quality Boundary: Taxonomy-based separation can regress when examples contain implicit rules, because classification may move those dependencies into reference modules.Across 600 skills, 86.0% achieved scoreC ≥ scoreA; the described limitation concerns deeply interleaved content.

A. Feedback Loop Convergence

The feedback loop progressively promotes items from the reference set into the core after Gate 2 failures. Under a monotonicity assumption it converges, but the paper notes that added content can sometimes distract agents.

  • Convergence Properties: The core set is monotonically non-decreasing because each failed iteration promotes additional items from outside the current core.The promotion operator adds items whenever Gate 2 fails.
  • Convergence Properties: The loop terminates in at most |I| − |I(0)| iterations, with experiments capping the process at K = 2 iterations.The theoretical bound follows from finite item promotion and the implementation cap limits practical feedback rounds.
  • Monotonicity Assumption: If adding content never decreases scoreC, then feedback iterations cannot reduce task performance.This monotonicity condition is explicitly stated as an assumption rather than an unconditional property.
  • Practical Limitation: The monotonicity assumption can fail because promoted items may distract the agent, offsetting their informational benefit.The paper characterizes this as a less-is-more effect and notes that targeted promotions are intended to make distraction less likely.
  • Empirical Behavior: After at most two iterations, 71.1% of triggered skills achieved scoreC ≥ scoreA, while 28.9% violated the monotonicity assumption.The result covers the 38 skills that triggered at least one feedback iteration.

B. Expected Cost Under Progressive Disclosure

Progressive disclosure models expected invocation cost as always-loaded compressed core content plus reference modules loaded probabilistically. Using empirical parameters, the framework predicts substantial body-token savings, with observed reductions spanning a broader range across heterogeneous skills.

  • Cost model: Expected invocation cost always includes the description and compressed core, while reference modules contribute according to their independent loading probabilities.The proof applies linearity of expectation because each reference is loaded only when a read_file call occurs.
  • Empirical parameters: The core fraction ρ, core compression factor α, number of reference modules k, and reference-loading probability p parameterize the expected-cost estimate.The empirical instantiation uses ρ = 0.383, α ≈0.63, k ≈2.1, and p ≈0.30.
  • Empirical parameters: Mean reference size is modeled as 0.617 · |s.b|/k by evenly distributing non-core content across reference modules.The 0.617 fraction corresponds to the non-core portion of the body under the stated empirical partition.
  • Cost estimate: The instantiated expected-cost expression weights compressed core tokens by 0.63 × 0.383 and loaded reference tokens by 0.30 × 0.617.The description term remains separate because descriptions are always loaded.
  • Results: 57.4% expected body cost reduction is obtained in the best case, while a conservative high-usage scenario yields 30.9%.The conservative scenario assumes α = 1.0 and p = 0.5; empirical reductions of 26.8%–43.2% vary with skill length, compression, and reference-loading patterns.
Loading 2603.29919v2…