PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive Multi-Resolution Procedural Knowledge Compression for Large Language Models

Changyue Wang, Weihang Su, Qingyao Ai, Yichen Tang, Runzhong Qiao, Xuancheng Li, Min Zhang, Yiqun Liu

arXiv:2606.12203v1cs.CL

TL;DR

Repeatedly loading full procedural skills increases LLM inference costs, while existing compression methods may not preserve workflow dependencies. SKIM uses adaptive multi-resolution soft tokens and outperforms existing methods in token reduction while maintaining high skill-task accuracy.

  • Problem

    Repeatedly placing full natural-language skills in model contexts raises token costs, while existing compression methods may sever procedural dependencies required for execution.

  • Method

    SKIM progressively trains multi-resolution soft-token representations for procedural skills and selects compression resolutions through offline self-judgment.

  • Results

    SKIM outperforms existing methods in token reduction while maintaining high skill-task accuracy across several skill-based datasets.

  • Takeaways & Limitations

    SKIM better preserves task accuracy than generic hard and soft baselines under smaller token budgets for procedural skill deployment.

  • Takeaways & Limitations

    Experiments cover only Qwen3-8B and Phi-4, leaving SKIM’s accuracy-token tradeoff on substantially larger target models unresolved.

Abstract

from arXiv · show

Large language models (LLMs) are widely used to tackle complex tasks with autonomous workflows. Recently, reusable natural language skills have emerged as a popular paradigm to inject procedural knowledge into LLM applications. Since popular skills are often invoked repeatedly, placing their full text in every context significantly increases prefill cost and latency. While text compression techniques have the potential to solve this problem, most existing methods are designed to compress factual knowledge in documents instead of procedural knowledge, making them insufficient for skill compression. In this paper, we argue that an effective skill compression method should: 1) preserve logical dependencies among workflows and tool protocols, 2) enable lightweight, offline compression for frequently updated community skills, and 3) be adaptable to varying complexities across skills. To address this, we present SKIM (SKIll coMpression), an adaptive multi-resolution soft token compression framework for procedural skills. Depending on the complexity of each skill, SKIM creates different numbers of soft tokens that not only improve the efficiency of LLM inference, but also preserve the effectiveness of skill usage. Experiments indicate that SKIM compresses skills to 30 to 60 percent of their original token length while preserving task performance better than existing compression methods.We have released our code at https://github.com/bebr2/SKIM .

1 Introduction

The paper frames procedural skill compression as distinct from factual document compression because execution depends on preserving workflow logic. It introduces SKIM, an adaptive multi-resolution soft-token framework designed for lightweight offline deployment and effective skill-task performance.

  • Motivation: Skill deployment creates token-consumption costs because reusable natural-language skill files are typically inserted in full at every invocation.The format supports creating, editing, sharing, and reusing agent capabilities, but repeated inclusion increases deployment overhead.
  • Motivation: Procedural skill compression must preserve dependencies among conditions, tool arguments, and workflow information, because breaking a logical link can cause execution failure.Skills encode executable procedures rather than isolated factual keywords.
  • Design Requirements: Updated or shared skills require storage-efficient representations that can be regenerated and transmitted using a lightweight forward pass rather than online gradient optimization.This requirement follows from active user modification, distribution, and deployment of skills.
  • Design Requirements: Skills vary in complexity and information density, so an effective method must support multiple token budgets and select a suitable resolution for each skill-model pair.A fixed compression rate cannot adapt simultaneously to all skills or models.
  • SKIM: SKIM combines a compressor and projector with progressive training stages—skill reconstruction, procedural warm-up, and skill task alignment—to preserve executable dependencies.The framework encodes skills into soft tokens and maps them into the target LLM space.
  • Evaluation: SKIM evaluates skill compression on multiple skill-based datasets, reporting greater token reduction than existing methods while maintaining high skill-task accuracy.Its contributions also include offline self-judgment for selecting compression resolution.

2 Related Work

Related work extends retrieval-augmented generation toward reusable procedural skills, while prompt-compression methods remain limited by latency, storage, optimization cost, or online processing requirements. Existing approaches therefore leave procedural skill distribution insufficiently addressed.

  • Procedural skills: Procedural knowledge is increasingly packaged as reusable skills that external resources select and apply during inference for agentic problem solving.This paradigm extends retrieval-augmented generation beyond declarative knowledge toward procedural capabilities.
  • Prompt compression: Prompt compression methods comprise hard token pruning and soft compression, with examples including task-agnostic LLMLingua-2 and query-aware LongLLMLingua.LongLLMLingua targets retrieval scenarios but introduces additional online latency.
  • Prompt compression: 500xCompressor introduces massive storage pressure, making it unsuitable for skill distribution scenarios.The passage presents storage pressure as a limitation for distributing compressed skills.
  • Procedural compression: TokMem distills an instruction sequence into a single token but requires costly gradient-based optimization for every new procedure.This requirement is impractical for skill ecosystems with rapid community iterations.
  • Procedural compression: DRIFT uses a dual-model soft-token architecture to extract factual information but requires online compression for retrieved long texts.Its design addresses factual information rather than the procedural skill-compression gap described in the passage.

3 Methodology

SKIM replaces a skill’s procedural content with model-specific continuous soft tokens, using a compressor, projector, and adapted target LLM. Its multi-stage training and offline resolution selection support multiple compression budgets while preserving procedural behavior across skill complexities.

  • Compression framework: SKIM compresses a skill’s content field into K continuous tokens in the target LLM’s embedding space, replacing the original text while preserving procedural behavior.The content field contains procedural guidance, tool descriptions, and usage constraints; SKIM produces EK(s) ∈ R^K×dM under a selected token budget.
  • Compression framework: The compressor uses learnable slot tokens to encode skills, while a projector maps slot hidden states into the target LLM space for insertion alongside the user query.The compressor is an autoregressive backbone, and its slot-position hidden states form the latent representation.
  • Inference: A single high-resolution artifact supports lower budgets by prefix truncation, allowing independently compressed multiple skills to be concatenated without recomputation.For K < Kmax, SKIM uses the first K projected tokens from the Kmax representation and preserves selected-skill order during concatenation.
  • Training: SKIM trains in three stages: skill reconstruction, procedural question-answering warmup with WikiHow weak supervision, and alignment on real skill-conditioned tasks.The stages progressively retain skill information, align representations with procedural answer generation, and train on tasks involving decision making, tool calls, and multiple skills.
  • Offline resolution selection: Before deployment, SKIM selects a skill- and model-specific resolution through offline calibration, matching token budgets to skill complexity without adding user-request latency.If no compressed resolution meets the threshold, the system falls back to the original skill text.

4 Experimental Setup

SKIM is evaluated on five skill-annotated benchmarks across two target/compressor model configurations. The experiments compare fixed and adaptive soft-token compression with no-skill, full-text, and hard-compression baselines under matched token budgets.

  • Datasets: Five SRA-Bench skill-annotated datasets cover code generation, mathematical reasoning, logical reasoning, and tool-use tasks.The datasets are BigCodeBench, CHAMP, LogicBench, TheoremQA, and ToolQA; golden annotations fix the relevant skill content per instance.
  • Models: Two target LLM configurations span Qwen3-8B and Phi-4 14B, with Qwen3-8B and Phi-4-mini-instruct serving as compressors, respectively.SKIM performs model-specific offline resolution selection for each skill.
  • Baselines: The baselines include Naive, Full Text, LLMLingua-2, ICAE, and 500xCompressor, with LLMLingua-2 evaluated at compression ratios matched approximately to SKIM’s token budgets.LLMLingua-2 uses ratios 0.3 and 0.55 for its small and medium settings, and 0.7 for BigCodeBench or 0.75 for other tasks in its large setting.
  • SKIM Variants: SKIM compares fixed 256- and 512-token budgets with Adaptive selection among 256 tokens, 512 tokens, and full text.Skills shorter than their compressed form use full text, while multi-skill instances use the highest selected budget.
  • SKIM Variants: Adaptive resolution selection uses an offline exam of 10 diagnostic questions per skill with a 0.9 selection-accuracy threshold.For multi-skill scenarios, skill texts are concatenated before compression to a fixed length.

5 Experimental Results

SKIM delivers a better accuracy–token tradeoff than generic compression methods while preserving skill-task performance. Adaptive resolution selection generally improves fixed-budget variants, though it can slightly reduce accuracy in some cases and depends on Full Text silver references.

  • Overall comparison: SKIM achieves a superior accuracy–token tradeoff, while ICAE and 500xCompressor often underperform Naive.Full Text usually performs best but adds hundreds to thousands of tokens per instance.
  • Adaptive resolution: Adaptive SKIM improves over fixed-budget variants on most datasets, often approaching or exceeding Full Text accuracy with fewer tokens.Its offline exam dynamically selects lower resolutions when diagnostic answers are sufficient.
  • Adaptive resolution: In some cases, such as Phi-4 on LogicBench, a single threshold favors token reduction and slightly lowers accuracy relative to Fix-512.Adaptive also uses Full Text answers as silver references, a dependency described as usually reasonable.
  • Training ablations: Removing reconstruction or procedural warm-up lowers downstream performance, while replacing skill-task data with HotpotQA data also weakens the complete recipe.The ablations indicate that the full training recipe is needed for strongest skill-task performance.
  • Training ablations: Replacing self-generated answers with GPT-5.2 teacher answers does not provide the main benefit; target-distribution supervision is more important.The finding suggests stronger off-policy answers alone are insufficient.
  • Retrieval-noise stress test: Under retrieved distractor skills and extended contexts, SKIM outperforms LLMLingua-2 at comparable or lower token budgets.For Phi-4, higher-resolution SKIM variants surpass Full Text in this BigCodeBench stress setting.

6 Conclusion

SKIM is an adaptive multi-resolution soft token compression framework for procedural skills that preserves task accuracy under smaller token budgets. It trains representations using multiple alignment objectives and selects compression resolutions through offline self-judgment.

  • Contribution: SKIM presents an adaptive multi-resolution soft token compression framework for procedural skills.The framework targets efficient procedural skill deployment.
  • Method: SKIM trains representations through reconstruction, procedural QA, and skill task alignment.
  • Method: SKIM selects compression resolutions through offline self-judgment.
  • Results: SKIM better preserves task accuracy than generic hard and soft baselines under smaller token budgets.
  • Conclusion: SKIM effectively resolves the efficiency-accuracy trade-off in procedural skill deployment.

7 Limitations · A Offline Resolution Selection Algorithm

The paper identifies two limitations: evaluation covers only Qwen3-8B and Phi-4, while SKIM’s model-specific components limit artifact portability. It also provides pseudocode for offline resolution selection in Algorithm 1.

  • 7 Limitations: Two target LLMs, Qwen3-8B and Phi-4, are used in the experiments.The paper does not evaluate SKIM on substantially larger models because of training resource constraints.
  • 7 Limitations: SKIM’s accuracy and token tradeoff on stronger target models remains open.The reported results do not establish how these tradeoffs change when the target model is stronger.
  • 7 Limitations: SKIM trains model-specific projectors and LoRA adapters.This design improves alignment between soft tokens and the target LLM.
  • 7 Limitations: A compressed SKIM artifact is not directly portable across target LLMs.The model-specific projector and LoRA design creates this portability limitation.
  • A Offline Resolution Selection Algorithm: The offline resolution selection procedure is presented as pseudocode.The supplied passage introduces the procedure but does not describe its steps.
  • A Offline Resolution Selection Algorithm: Algorithm 1 contains the offline resolution selection procedure.The supplied text identifies Algorithm 1 as the location of the procedure’s pseudocode.

B Stage 3 Data Construction Algorithm

Stage 3 constructs data by using GPT-5.2 for source-skill evaluation, question generation, and skill splitting, while each target model generates answers for itself.

  • B Stage 3 Data Construction Algorithm: GPT-5.2 handles source skill evaluation, question generation, and skill splitting, whereas each target model performs answer generation.This division is summarized in Algorithm 2.

C Dataset and Hyperparameter Details · D Prompt Templates

The appendix specifies SKIM’s evaluation datasets, training-data safeguards, offline resolution-selection procedure, implementation settings, and baseline configurations. It also documents prompt templates for skill evaluation, decomposition, question generation, answer judgment, ReAct data construction, and tool simulation.

  • C Dataset and Hyperparameter Details: The offline exam generates N questions, obtains full-skill reference answers, evaluates candidates across resolutions, computes accepted-answer fractions α_K(s), and returns the smallest K meeting τ or the full skill text.This procedure is specified in Algorithm 1 and selects resolution per skill.
  • C Dataset and Hyperparameter Details: Five evaluation datasets include multi-skill BigCodeBench and CHAMP, single-skill LogicBench, TheoremQA, and ToolQA, with ToolQA uniquely requiring tool-style interaction.MedCalcBench is excluded because some skills are too short to provide enough room for evaluating compression behavior.
  • C Dataset and Hyperparameter Details: Stage 1 training skills are collected from GitHub, filtered below 500 characters, and separated from test golden skills by a minimum character edit distance greater than 2200.The filtering targets low-quality or underspecified examples, while the edit-distance threshold prevents evaluation skills from appearing in training data.
  • C Dataset and Hyperparameter Details: SKIM implementation settings cover model–compressor pairs, candidate resolution budgets, projector configuration, LoRA settings, offline-exam parameters, and inference decoding.Table 5 summarizes these implementation details, while Table 6 covers ICAE and 500xCompressor baseline hyperparameters using about 60,000 QA-supervised examples.
  • D Prompt Templates: The skill-evaluation prompt filters skills, generates exactly {k} practical user questions, estimates independent subskill decomposability, identifies tools, and determines whether ReAct-style answers are appropriate.It requires JSON-only output with the prescribed schema.
  • D Prompt Templates: The decomposition prompt produces exactly {split_count} self-contained subskills with name, description, and executable content while preserving essential operational details and minimizing overlap.The prompt is applied when evaluation indicates that a skill can be split into independent subskills.
  • D Prompt Templates: Offline question-generation prompts demand concrete, answerable, diverse questions that test practical skill mastery without directly leaking or copying the skill text.Programming and tool-use skills should be tested through code-generation or tool-calling tasks requiring application of the skill’s knowledge.
  • D Prompt Templates: Answer-judgment prompts compare candidate and full-text reference answers for semantic correctness, while tool-simulation prompts return specific, plausible best-effort observations for ReAct trajectories.The judgment rules reject contradictions of key reference facts and require valid JSON; the simulator forbids refusals, placeholders, and claims of ignorance.

E Further Ablation Studies · E.1 Offline Resolution Selection

SKIM’s offline exam controls the tradeoff between skill information, token budget, and accuracy. Stricter thresholds and more exam questions improve selection quality, while the main threshold balances accuracy with fewer tokens.

  • E.1 Offline Resolution Selection: Stricter judgment thresholds load more skill information and improve accuracy.The main threshold of 0.9 approaches the strictest setting in accuracy while using fewer tokens.
  • E.1 Offline Resolution Selection: Generating more exam questions improves selection quality but selects larger token budgets more often.
  • E.1 Offline Resolution Selection: The untrained resolution-budget ablation trains at 256 and 512 tokens but evaluates 64, 128, and 384 tokens by prefix truncation.
  • E.1 Offline Resolution Selection: The untrained resolution-budget evaluation covers BigCodeBench, CHAMP, LogicBench, and TheoremQA, with Naive and Full Text references.
  • E.1 Offline Resolution Selection: The third-stage LoRA ablation evaluates skill-task alignment using Frozen training and rank variants with alpha set to twice the rank.Frozen trains the compressor while keeping target LLM weights fixed.
  • E.1 Offline Resolution Selection: Applying the offline exam without compressed candidates uses more tokens and achieves lower accuracy than the main setting.

E.2 Untrained Resolution Budgets … G Licensing

SKIM transfers across untrained soft-token budgets, benefits from target-model adaptation, and preserves procedural operation sequences in a ToolQA case. The paper also reports offline resolution-selection ablations and licensing information for its models and datasets.

  • E.2 Untrained Resolution Budgets: 64 tokens remain above the Naive reference, while accuracy generally increases with larger budgets, including untrained resolutions obtained by prefix truncation.Only 256 and 512 tokens are training resolutions; budgets include 64, 128, 256, 384, and 512.
  • E.3 Target Model Adaptation: Frozen target-LLM training is weaker than LoRA, especially at larger resolution, while small LoRA ranks recover most of the gain.Larger LoRA ranks are close to each other, so excessively large ranks are unnecessary.
  • F Qualitative Case Study: In the ToolQA coffee lookup case, both compressed methods answer the factual schema question, but only SKIM preserves the ordered database operations needed for the lookup.The case is identified as toolqa_008.
  • G Licensing: Qwen3-8B, BigCodeBench, and ToolQA use the Apache License 2.0, while Phi-4 variants, bge-base-env1.5, LogicBench, TheoremQA, CHAMP, and SRA-Bench use the MIT license.The paper describes its objective as academic exploration consistent with permitted use under these licenses.
  • G Licensing: Figure 9 ablates offline resolution selection by varying the self-judgment accuracy threshold, generated exam questions per skill, and candidate modes.The first two panels report macro accuracy and macro average added tokens; the third plots candidate settings by token count and accuracy.
Loading 2606.12203v1…