PaperlayerView exact revision on arXiv

Source-linked AI summary

Skill is Not One-Size-Fits-All: Model-Aware Skill Alignment for LLM Agents

Jianxiang Yu, Jiapeng Zhu, Bochen Lin, Qier Cui, Zichen Ding, Xiang Li

arXiv:2605.30723v1cs.CL

TL;DR

Skill libraries often reuse the same formulations across backbones, although skill effectiveness varies by model. MASA evolves skills for each target backbone and trains a lightweight rewriter; across three environments and four Qwen3 backbones, it achieves the best success rate in all settings, with gains up to +25.8 points.

  • Problem

    Existing skill-library systems typically reuse a single shared library across backbones, although the same skill can benefit one model and degrade another.

    Source: §1 · pp. 1–2

  • Method

    MASA uses capability-conditioned hierarchical search to evolve general and task-specific skills, then trains a lightweight model-conditioned rewriter to adapt skills in one forward pass.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    Up to +25.8 points, MASA achieved the best success rate in all settings across three environments and four Qwen3 backbones.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    The findings support adapting skill formulations to target backbones rather than sharing them uniformly across models.

    Source: §1 · p. 2, §5 · p. 8

    Empirical evidence is restricted to Qwen3 (4B/8B/14B/32B), and extending to more model families and environments requires substantially more compute.

    Source: Limitations · p. 9

Abstract

from arXiv · show

LLM agents increasingly retrieve externally curated skills-procedural instructions retrieved at decision time-to improve performance on long-horizon interactive tasks. Existing skill libraries are typically treated as model-agnostic, reusing the same skill formulations across backbones with substantially different capacities and behaviors. However, our controlled experiments across multiple model scales show that skill effectiveness is strongly model-dependent: a skill that benefits one backbone can harm another. Motivated by this observation, we propose MASA Model-Aware Skill Alignment, a framework that adapts skills to each target backbone without modifying agent weights. MASA operates in two stages: (1) a hierarchical skill evolution pipeline that iteratively rewrites general and task-specific skills using hill climbing and UCB-driven tree search, guided by environment feedback and model capability profiles; and (2) a lightweight model-conditioned skill rewriter trained on evolution trajectories to reproduce the adaptation in a single forward pass. Experiments across three interactive environments and four backbones show that MASA consistently achieves the best overall performance, with gains of up to 25.8 points over the strongest baseline. The learned rewriter further generalizes to unseen tasks and environments without additional search, consistently outperforming a much larger teacher LLM at a fraction of the inference cost.

1 Introduction

The introduction argues that skill formulations must be aligned to individual backbones because a shared skill can help one model and harm another. It presents MASA as a weight-preserving framework that evolves model-appropriate skills and trains a lightweight rewriter to apply them.

1 Introduction

  • Existing skill libraries typically reuse one formulation across backbones despite deployment across models with substantially different capacities.The introduction frames this as a design problem because real-world systems face latency, inference-cost, and hardware constraints.Source: §1 · p. 1
  • Skill granularity varies by backbone: the same library can boost one model and degrade another, with the pattern also holding across Gemma3 models.Models of the same size from different families can prefer different formulations, suggesting that misaligned skills may distract rather than help.Source: §1 · pp. 1–2, Figure 1 · p. 1
  • MASA aligns skill formulations to each target backbone without modifying agent weights, using environment feedback to guide hierarchical skill evolution.A stronger teacher LLM uses hill climbing for general skills and UCB-driven tree search for task-specific skills, guided by a target-model capability profile.Source: §1 · p. 2
  • MASA trains a lightweight model-conditioned rewriter from evolution trajectories to adapt skills in a single forward pass at deployment.This avoids the costly teacher while retaining the reported ability to outperform it at a fraction of the inference cost.Source: §1 · p. 2
  • Across three environments and four Qwen3 backbones, MASA achieves the highest success rate, with gains up to +25.8 points.The introduction also reports that the rewriter generalizes to unseen tasks and environments in a single forward pass, outperforming a much larger teacher LLM at negligible cost.Source: §1 · p. 2

2 Preliminary Study: One Skill Library Does Not Fit All

The preliminary study holds skill principles fixed while varying textual granularity, finding that skill effectiveness depends on the model, granularity, and task type. These results motivate model-conditioned, search-based, task-specific skill adaptation.

2.1 Setup

  • The ALFWorld validation study compares four Qwen3 backbones with a No Skill control and three skill granularities that preserve the same principles and retrieval pipeline.The Concise, Moderate, and Detailed variants differ in representational depth; the study uses the same retrieval pipeline to isolate granularity effects.Source: §2.1 · p. 2

2.2 Findings

  • Qwen3-4B performs best with Moderate skills, Qwen3-14B and Qwen3-32B with Detailed skills, while Qwen3-8B peaks at No Skill (32.1%).For Qwen3-8B, external guidance can override short, effective action chains and prompt unnecessary exploration.Source: §2.2 · p. 2
  • Under Detailed skills, Qwen3-32B underperforms Qwen3-14B by 4.6 points, while Qwen3-4B performs best at the intermediate Moderate granularity.The results therefore resist simple rules such as scaling detail with model size or always adding or removing detail.Source: §2.2 · p. 3
  • Within a model–granularity pairing, success rates vary by over 60 points across task types; Qwen3-14B with Concise skills scores 74.2 on PICK and 13.7 on COOL.Some task types benefit from detailed skills regardless of model size, while others are insensitive or harmed.Source: §2.2 · p. 3
  • A parallel Gemma3 experiment shows the same scale-dependent trend, suggesting that model-dependent skill effectiveness generalizes across model families.The experiment covered Gemma3 4B, 12B, and 27B models.Source: §2.2 · p. 3
  • The findings call for model-conditioned, search-based, task-type-specific adaptation and open-ended rewriting beyond textual granularity alone.The controlled study varies textual granularity with content fixed, while practical misalignment may also involve decision strategy, framing, or format; MASA is designed to address these requirements.Source: §2.2 · p. 3

3 Method: MASA

MASA adapts skill libraries to target-backbone capability profiles through search-time evolution and a deployment-time rewriter that learns model-conditioned adaptation in one forward pass.

3.1 Problem Formulation and Skill Library

  • With the backbone frozen, MASA optimizes a hierarchical library of cross-task and task-specific skills, conditioned on a structured model card.Its objective balances task success against the nothing-happens rate, a proxy for skill-induced stalling.Source: §3.1 · p. 4

3.2 Hierarchical Model-Conditioned Skill Evolution

  • A teacher analyzes failures and rewrites skills for the target model, using hill climbing for general skills and independent UCB1 tree searches for task-specific skills.General-skill edits are evaluated across the full task suite and accepted only when reward improves; task-specific search evaluates and backpropagates per-type rewards, while holding Stage 1 general skills fixed.Source: §3.2 · pp. 4–5

3.3 Model-Conditioned Skill Rewriter

  • MASA-Rewriter conditions on the model card, task description, and input skills to adapt skills in one forward pass without environment interaction or iterative search.Training maps varied-quality inputs—including search intermediates, cross-model transfers, one-shot rewrites, and perturbed skills—to high-scoring evolved outputs; the rewriter uses Qwen3-4B supervised fine-tuning.Source: §3.3 · p. 5
  • The evolution pipeline supplies per-backbone upper bounds and training signals, while MASA-Rewriter amortizes search for rapid, compute-constrained adaptation.Evolution is preferred when rollout budgets permit thorough optimization; the rewriter avoids further environment interaction at deployment.Source: §3.3 · pp. 5–6

4 Experiments

Experiments show that MASA’s model-conditioned skill evolution outperforms model-agnostic baselines across ALFWorld, WebShop, and search-augmented QA, while MASA-Rewriter transfers learned adaptation to held-out tasks and environments.

4.1 Experimental Setup

  • The evaluation spans ALFWorld, WebShop, and seven search-augmented QA benchmarks, using four Qwen3 backbones and comparing No Skill, shared Base Skill, and DS-Adapter.The Qwen3 models run in non-thinking mode; the teacher LLM is DeepSeek-V4-Pro.Source: §4.1 · p. 6, §4.2 · p. 6

4.2 Skill Evolution Evaluation

  • 12.9 to 61.3: on 4B, MASA improves Bamboogle success rate from the best baseline; on 32B, it ranks first on five of seven datasets.Although evolution uses only NQ and HotpotQA, gains extend to out-of-domain benchmarks, indicating transfer beyond the datasets used for evolution.Source: Table 2 · p. 7, §4.2 · pp. 7–8
  • 31.4, 57.9, 64.3, and 65.7: MASA achieves the highest ALFWorld average success rates on Qwen3 4B, 8B, 14B, and 32B, respectively.Its gains over the strongest baseline are +4.3, +25.8, +20.0, and +20.7, respectively; it also reduces average interaction steps, including 39.1 to 29.2 on 8B and 36.7 to 25.7 on 14B.Source: §4.2 · p. 7
  • 26.4, 28.6, 29.2, and 34.6: MASA achieves these WebShop success rates on Qwen3 4B, 8B, 14B, and 32B, outperforming all baselines.MASA achieves higher success rates in 7–8 steps, compared with 12–13 steps for baselines that succeed, and uses 4.7 steps on 8B.Source: §4.2 · p. 7

4.3 Skill Rewriter Generalization

  • Despite a substantial environment gap (different action spaces and observation formats), Cross-env MASA-Rewriter outperforms DS-Adapter on held-out ALFWorld task types across 4B, 8B, 14B, and 32B backbones by +1.5, +3.0, +2.9, and +3.0, respectively.Source: Figure 3 · p. 8, §4.3 · p. 8
  • +8.8, +13.2, and +7.4: Cross-task MASA-Rewriter exceeds DS-Adapter on 8B, 14B, and 32B, respectively.It uses traces from other ALFWorld task types; the 4B-parameter rewriter also consistently surpasses the DeepSeek-V4-powered DS-Adapter at a fraction of the inference cost.Source: §4.3 · p. 8

5 Conclusion

MASA adapts skills to individual backbones through hierarchical evolution and a lightweight rewriter, achieving strong results across environments and motivating model-aware skill alignment as reusable infrastructure.

5 Conclusion

  • Across three environments and four Qwen3 backbones, MASA achieved the best success rate in all settings, with gains up to +25.8 points.The conclusion frames these results as evidence that skill design cannot assume one formulation works across model scales.Source: §5 · p. 8
  • The lightweight model-conditioned rewriter amortizes hierarchical skill search into one forward pass and generalizes to unseen tasks and environments at negligible deployment cost.MASA combines hierarchical skill evolution with a rewriter that reproduces adaptation without additional search at deployment.Source: §5 · p. 8
  • The authors envision MASA-Rewriter as middleware that automatically adapts existing skill libraries to new backbones, making skill alignment reusable infrastructure rather than per-deployment engineering.The envisioned middleware would require no environment rollouts, retraining, or manual prompt engineering.Source: §5 · p. 8

Limitations

MASA’s current evidence and training are bounded by Qwen3 and three environments, while broader model and domain coverage would require more compute and evaluation support.

Limitations

  • Evidence is limited to Qwen3; extending MASA to more model families and environments would strengthen generality but require substantially more compute.Closed-source evolution requires hundreds of paid-API environment rollouts per backbone; the rewriter partly amortizes this cost once trajectories from a few backbones are available.Source: Limitations · p. 9
  • The rewriter trains on ALFWorld, WebShop, and Search-QA trajectories, while evolution requires automatic success/failure signals that unsignaled tasks lack.Open-ended web tasks or real-world applications would need external evaluators or human annotations, potentially broadening the framework’s applications.Source: Limitations · p. 9

Ethical Considerations · A Related Work

MASA uses public benchmarks and licensed public models, while its agent-capability gains and evolved skills require safety oversight and validation. It differs from prior skill-library and prompt-adaptation work by rewriting retrieved skills for each backbone and amortizing that search with a lightweight rewriter.

Ethical Considerations

  • MASA collects no new human-subject data, uses public benchmarks including ALFWorld and WebShop, and accesses publicly released models under their respective licenses.The experiments also use open-domain QA datasets.Source: Ethical Considerations · p. 9
  • Because skill adaptation may increase agent capabilities, high-risk deployments should include monitoring, policy constraints, and human oversight.Source: Ethical Considerations · p. 9
  • Evolved skills may inherit biases or unsafe heuristics from optimization trajectories and feedback, so libraries should be inspected and validated before deployment.Source: Ethical Considerations · p. 9

A Related Work

B Ablations · C Preliminary Study: Supplementary Details · C.1 Skill Variant Comparison

MASA’s search stages and model-card conditioning each contribute to skill adaptation, with ablation effects varying by environment and backbone. The preliminary skill variants preserve task coverage while varying how much procedural guidance they expose.

B Ablations

  • On ALFWorld, removing task-specific search causes drops of −25.0 for Qwen3-8B and −15.7 for Qwen3-32B.The findings indicate that per-task-type procedural guidance is critical for these backbones.Source: B Ablations · p. 12
  • On ALFWorld, removing general search causes the largest drop for Qwen3-14B (−16.4).This suggests high-level behavioral priors matter when the model can follow them but still benefits from strategic framing.Source: B Ablations · p. 12
  • On WebShop, removing general skills drives SR to single digits for Qwen3-8B, Qwen3-14B, and Qwen3-32B, while task-specific skills have a comparatively modest effect.The passage attributes this asymmetry to WebShop’s demand for consistent high-level decision strategies.Source: B Ablations · p. 12
  • Removing the model card consistently degrades held-out ALFWorld average SR across Cross-env and Cross-task training variants.The Cross-task gap is especially pronounced for Qwen3-14B; Cross-env also shows substantial drops.Source: B Ablations · p. 12
  • Both search stages contribute to MASA, but their relative importance depends on the environment and backbone scale.Removing task-specific search most harms ALFWorld, while removing general skills most harms WebShop.Source: Table 3 · p. 12, B Ablations · p. 12

C.1 Skill Variant Comparison

  • The three nonempty ALFWorld skill variants retain the same skill IDs and task coverage while differing in how much procedural text they expose.The more detailed variants add trigger conditions, executable steps, and failure-prevention cues.Source: §C.1 · p. 12

C.2 Per-Task Breakdown: Qwen3 … E.2 Stage 2: UCB-Driven Tree Search

Per-task and cross-model results show that skill effectiveness depends on task and model characteristics, motivating model-conditioned skills and task-specific search. The appendix details the model cards and search procedures used to evolve those skills.

C.2 Per-Task Breakdown: Qwen3

  • Qwen3 skill success rates swing sharply by task: Qwen3-14B Concise scores 74.2 on PICK vs. 13.7 on COOL, while Qwen3-4B Detailed scores 1.6 on PICK vs. 46.7 on LOOK.These within-condition swings are substantially larger than cross-condition differences and motivate task-specific tree search.Source: §C.2 · p. 13

C.3 Supplementary Validation: Gemma3

  • Gemma3-4B and Gemma3-12B perform best with Concise skills, whereas Gemma3-27B performs best with Detailed skills, and same-scale Qwen3-4B instead peaks with Moderate skills.The different granularity preferences across model families indicate that parameter count alone is not a sufficient proxy for choosing skill form.Source: §C.3 · p. 13, Figure 4 · p. 13

D Model Card Construction

  • Model cards combine architecture metadata, training provenance, and capability profiles; weaknesses come from preliminary rollouts, while evaluation results and oracle skill preferences are excluded.The preliminary rollouts used to summarize weaknesses are disjoint from the evaluation set.Source: D Model Card Construction · p. 13

E Skill Evolution Pipeline Details

E.1 Stage 1: Hill Climbing

  • Stage 1 hill climbing runs for at most 10 iterations, stops after 3 consecutive non-improving iterations, supplies the top 5 historical skill sets, and accepts only strictly better adjusted reward.The patience setting enables early stopping.Source: §E.1 · p. 14

E.2 Stage 2: UCB-Driven Tree Search

  • Stage 2 searches a separate task-specific skill tree per task type, selecting nodes by UCB1 and evaluating candidates with 100 episodes per node for 10 iterations per task type.The UCB1 score uses mean adjusted reward and visit counts, with exploration constant C=1.4.Source: §E.1 · p. 14, §E.2 · p. 14

F Skill Rewriter Training Details · G WebShop Supplementary Results

The supplied passages describe full-parameter SFT for Qwen3-4B and two skill-rewriter training datasets, with augmentation and distinct environment coverage.

F Skill Rewriter Training Details

  • The rewriter is trained with full-parameter SFT on Qwen3-4B in BF16 for five epochs, using AdamW, a 4096-token maximum sequence length, and checkpoint selection by training-loss convergence.Training uses learning rate 1e−5, cosine scheduling, warmup ratio 0.1, gradient checkpointing, and effective batch size 4.Source: F Skill Rewriter Training Details · p. 14
  • Training data pairs cover in-domain tasks and use noisy inputs, partial inputs, and cross-model transfer pairs for augmentation.The noise ratio is 0.3 and the partial-input keep ratio is 0.6.Source: F Skill Rewriter Training Details · p. 14
  • The combined rewriter uses 769 samples across three environments, while the environment-specific rewriter uses 499 WebShop and Search samples.Combined-rewriter data includes ALFWorld Pick/Look/Pick2 only and excludes held-out types, WebShop, and Search.Source: F Skill Rewriter Training Details · pp. 14–15

G.1 Trajectory Analysis: Why Larger Models Fail · G.2 Per-Category Breakdown

The WebShop analysis links larger Qwen3 baselines’ weaker performance to verbose reasoning that uses up the step budget, while reporting that MASA improves success across categories and backbones.

G.1 Trajectory Analysis: Why Larger Models Fail

  • The analysis examines failed WebShop trajectories to explain why larger Qwen3 models (8B/14B/32B) perform worse than 4B under baseline conditions.The investigation focuses on trajectory behavior associated with the performance difference.Source: §G.1 · p. 15
  • Qwen3-14B is the most severe case, with verbose reasoning in 97% of steps.The analysis attributes this pattern to larger Qwen3 models performing worse than 4B under baseline conditions.Source: §G.1 · p. 15
  • Verbose reasoning consumes the fixed step budget on deliberation rather than environment interaction, leaving agents without enough purchase actions to succeed.The agents consider multiple options but do not complete enough purchase actions.Source: §G.1 · p. 15
  • Qwen3-4B has 0% of steps with chain-of-thought and about 73 characters per action, while 8B/14B/32B prepend extensive reasoning preambles.Table 7 reports WebShop trajectory statistics; its caption defines CoT as the fraction of steps containing reasoning preambles.Source: Table 7 · p. 15, §G.1 · p. 15

G.2 Per-Category Breakdown

  • MASA achieves the best success rate in most categories across all backbones, with particularly large gains on OTHER and ELECTRONICS.The reported improvement is broad rather than driven by a single easy category.Source: §G.2 · p. 15
  • Most categories have near-zero success rates for 8B/14B/32B baselines, consistent with the verbose-reasoning bottleneck.The per-category breakdown reports this pattern across the majority of categories.Source: §G.2 · p. 15
  • MASA’s category gains are broad, rather than relying on one easy category to inflate the average.The per-category breakdown describes improvement across the board.Source: §G.2 · p. 15

H Skill Rewriter OOD: Per-Task Breakdown · I Qualitative Analysis: Evolved Skill Examples · J The Use of Large Language Models (LLMs)

OOD skill-rewriter performance depends on the source traces and target task, while MASA evolves distinct color-matching strategies for different backbones. The paper states that LLMs were used only for grammatical and stylistic editing.

I Qualitative Analysis: Evolved Skill Examples

  • Qwen3-4B’s color-guessing leads to a strict exact-match-or-exit rule, whereas Qwen3-8B’s tendency to abandon products leads to buying approximate matches for partial credit.The evolved strategies target each model’s dominant failure mode rather than making minor wording changes.Source: I Qualitative Analysis: Evolved Skill… · p. 15, Table 9 · p. 20
  • Qwen3-14B’s deliberation on bad products prompts a one-step fast-fail exit, while Qwen3-32B’s mishandling of coded colors prompts substring matching with tiebreaking rules.The same color-matching problem therefore receives fundamentally different decision strategies across backbones.Source: I Qualitative Analysis: Evolved Skill… · pp. 15–16, Table 9 · p. 20

J The Use of Large Language Models (LLMs)

H Skill Rewriter OOD: Per-Task Breakdown

  • 29.6→51.9 on 8B CLEAN and 24.0→44.0 on 14B COOL illustrate gains from adding ALFWorld traces; HEAT improves for 4B and 14B, but is comparable for 8B and 32B.These in-environment traces help the rewriter learn ALFWorld-specific action patterns, including navigation sequences and object-interaction protocols.Source: H Skill Rewriter OOD: Per-Task Breakdown · p. 15, Figure 5 · p. 19
  • 56.2% on HEAT for both 14B and 32B shows Cross-env’s strength there; its CLEAN and COOL results are mixed, matching or slightly exceeding DS-Adapter for smaller backbones but falling short in some larger-backbone cells.For 14B CLEAN, Cross-env and DS-Adapter are tied at 40.7.Source: H Skill Rewriter OOD: Per-Task Breakdown · p. 15, Figure 5 · p. 19

Related papers

Loading 2605.30723v1…