PaperlayerView exact revision on arXiv

Source-linked AI summary

SkillFusion: Towards Reliable Multi-Skill Composition for LLM Agents

Enhao Huang, Xinye Zhao, Yiming Li, Bingrun Yang, Nanjun Zhou, Chunshu Xia, Hongwei Yao, Zhan Qin

arXiv:2610.15481v1cs.AI

TL;DR

Co-activated skills can impose incompatible, competing, or redundant directives, while their interactions depend on type and scope. SkillFusion compiles source-linked constraints into reusable merged skills; it achieves the highest non-oracle Success on all three agent benchmarks, beating three-call RCR by 4.8, 4.0, and 3.8 percentage points.

  • Problem

    Co-activated skills can impose incompatible requirements, competing preferences, and redundant instructions, so composition must account for interaction type and scope.

    Source: §1 · pp. 1–2

  • Method

    SkillFusion compiles typed, scoped, source-linked constraints, selectively revisits candidate pairs, and applies interaction-specific resolution to produce reusable merged skills.

    Source: §1 · p. 2, §3.2.2 · p. 5, §3.2.3 · p. 5

  • Results

    SkillFusion achieves the highest non-oracle Success on all three benchmarks, improving over three-call RCR by 4.8, 4.0, and 3.8 percentage points, respectively.

    Source: §4.2 · p. 6

  • Takeaways & Limitations

    Cross-model transfer and instruction scaling further demonstrate the utility of resolving skill interactions before execution.

    Source: §5 · p. 9

    The evaluation concerns task performance and instruction consistency; operational safety requires separate assessment.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Independently authored skills can impose incompatible requirements, competing preferences, and redundant instructions when jointly loaded by an LLM agent. We propose SkillFusion, an offline compiler that reconciles these interactions into a reusable merged skill. Its central idea is to make composition decisions explicit through typed, scoped constraints linked to their source directives. A type-scope gate selects candidate interactions, while selective fallback revisits pairs potentially missed by structural pruning. A classifier identifies interaction types, and an adversarial verifier re-examines uncertain judgments. The resolver applies conditional routing or explicit priorities to hard conflicts, verified abstraction to soft conflicts, and deduplication to redundant instructions. Source-linked checks guide synthesis and compression, and metadata synthesis supports discovery of the merged skill. The resulting artifact retains associated scripts and resources and can be reused across tasks and executor models. SkillFusion achieves the highest mean Success among evaluated non-oracle methods on all three agent benchmarks and on both transfer benchmarks across three executor models. At the largest tested instruction configurations, it achieves over 2.5 times the Success of unresolved multi-skill loading, highlighting the growing value of reconciliation as composition becomes more demanding. The code will be open-sourced after the review process concludes.

1 INTRODUCTION

Jointly loaded skills can conflict, compete, or duplicate requirements, so composition must account for both interaction type and scope. SkillFusion compiles source-linked, typed and scoped directives into reusable merged skills, improving Success over three-call Rewrite–Critique–Revise on three benchmarks.

1 INTRODUCTION

  • Independently authored skills create hard conflicts, competing soft preferences, and redundant instructions; reliable composition must account for interaction type and applicability.For example, spreadsheet editing and auditing directives can appear conflicting if their workflow phases are not distinguished.Source: §1 · pp. 1–2
  • After skills are selected, concatenation leaves reconciliation to the executor, while free-form rewriting makes individual requirements and exceptions difficult to inspect.Prior work addresses skill construction and retrieval, while instruction-following studies report challenges under composed or accumulating constraints.Source: §1 · p. 2
  • SkillFusion represents source-linked directives as typed, scoped constraints, then selectively reconsiders candidates and resolves interactions by type.Its output retains discovery metadata, scripts, and resources for reuse across tasks and executor models.Source: §1 · p. 2
  • 4.8, 4.0, and 3.8 percentage points: SkillFusion improves Success over three-call Rewrite–Critique–Revise on SpreadsheetBench Verified, WebArena-Verified, and SWE-bench Verified, respectively.All three paired 95% intervals exclude zero.Source: §1 · p. 2
  • The paper evaluates task performance and connects it to interaction recovery, controlled mechanisms, component ablations, cross-model transfer, and instruction scaling.The selective-fallback audit recovers 14 of 24 incompatible pairs missed by strict pruning, increasing F1 from .713 to .840.Source: §1 · p. 2

2 RELATED WORK

Related work covers skill construction and routing, skill composition, multi-agent verification, and instruction following under constraint pressure. The paper distinguishes its focus as instruction-level reconciliation of independently authored directives.

Skill construction and routing

  • Skill construction research ranges from iterative refinement and reinforcement-learning discovery to LoRA-based acquisition, while routing work studies retrieval, reranking, and limits of selecting skills from libraries.These lines expand skill supply and selection; instruction-level composition additionally analyzes interactions among directives.Source: §2 · pp. 2–3

Skill composition

  • Prior composition methods select or compose skills at the planning level, check structured preconditions and post-effects, or merge adapter weights; instruction-level directive reconciliation remains distinct.The paper frames its question as merging partially overlapping or conflicting natural-language directives into one coherent specification.Source: §2 · p. 3

Multi-agent verification

  • LLM-as-judge and debate methods use model interaction for evaluation or critique; SkillFusion applies adversarial re-judging to uncertain conflict classifications against source constraints.The paper points to threshold analysis in Appendix A.5.Source: §2 · p. 3

Instruction-following under constraint pressure

  • Instruction-following studies find that multi-constraint composition is challenging, performance degrades as requirements accumulate, and strong models may leave conflicts unclarified or comply only partially.The paper positions explicit offline reconciliation as addressing this constraint-dense setting.Source: §2 · p. 3

3 METHOD

SkillFusion represents skill directives as typed, scoped constraints, then parses, detects, and resolves their interactions into a compressed merged skill while preserving source links.

3.1 PROBLEM SETUP

  • Skills package directives, scripts, and resources; composition represents directives as typed, scoped constraints and evaluates merges by Success, Compliance, and source-constraint Preservation.Source: §3.1 · pp. 3–4

3.2 SKI L LFUS ION PIPELINE

  • SkillFusion uses four LLM roles across parsing, interaction detection with adversarial re-judging, and resolution with compression, producing merged instructions, discovery metadata, and preserved scripts and resources.Source: Figure 2 · p. 4, §3.2 · p. 4

3.2.1 STAGE I: CONSTRAINT PARSING

  • The parser decomposes files into minimal directives, normalizes informal scopes into structured cues, and recovers script-encoded constraints through static inspection and a two-pass semantic filter, retaining provenance.Source: §3.2.1 · pp. 4–5

3.2.2 STAGE II: CONFLICT DETECTION

  • Candidate detection prunes cross-skill constraint pairs with a type–scope gate, then selectively reopens uncertain, global-scope, or high-similarity exclusions before classifying interactions and re-judging low-confidence labels.Source: §3.2.2 · p. 5

3.2.3 STAGE III: CONFLICT RESOLUTION AND COMPRESSION

  • Resolution routes or prioritizes incompatible directives, verifies abstractions for competing preferences with hard-conflict fallback, deduplicates weaker repeats, then source-checks compression and synthesizes discovery metadata.Source: §3.2.3 · pp. 5–6

4 EXPERIMENTS

Across seven experiments, SkillFusion is evaluated on agent benchmarks, component ablations, candidate-generation robustness, executor transfer, instruction scaling, and controlled interaction conditions. The results favor explicit reconciliation, with its advantage persisting across benchmarks and growing as unresolved instruction complexity increases.

4.1 EXPERIMENTAL SETTINGS

  • The experiments cover SpreadsheetBench Verified (400 tasks), WebArena-Verified (812 tasks), SWE-bench Verified (500 tasks), and HumanEval+ (164 tasks), with seven named comparison methods.Main and cross-model evaluations use three full evaluations; HumanEval+ uses four runs for ablations and 20 for scaling and mechanism scenarios.Source: §4.1 · p. 6

4.2 MAIN RESULTS

  • SF-final achieves the highest non-oracle Success on all three benchmarks, improving over three-call RCR by 4.8, 4.0, and 3.8 percentage points.The paired 95% intervals exclude zero on all three benchmarks.Source: §4.2 · p. 6, Figure 3 · p. 7

4.3 COMPONENT ABLATIONS

  • Removing scope normalization lowers HumanEval+ S60 Success from .799 to .768 and also reduces recall and Preservation.The ablations indicate that scope matters in both candidate selection and source-requirement retention.Source: §4.3 · p. 7, Figure 4 · p. 8
  • Removing conflict-type distinction leaves recall at .833 but reduces Preservation from .900 to .850; removing compression raises mean Success to .811 while reducing compactness benefits.Metadata synthesis affects Activation recall without changing Preservation; compression improves Compliance and reduces length.Source: §4.3 · p. 7, Table 2 · p. 8

4.4 HUMAN AUDIT AND CANDIDATE ROBUSTNESS

  • Selective fallback recovers 14 of 24 incompatible pairs missed by strict pruning, raising F1 from .713 to .840 as precision falls from .878 to .847.Exhaustive comparison finds 24 false positives for No gate versus nine for SF-final; verifier-threshold changes Success by at most .006 across two testbeds.Source: §4.4 · p. 8

4.5 CROSS-MODEL GENERALIZATION

  • SF-final leads all non-oracle methods in six benchmark–backbone combinations, with gains over RCR of 1.5–2.9 points on the two transfer backbones.The fixed artifacts transfer without backbone-specific recompilation.Source: §4.5 · p. 8

4.6 SCALING WITH INSTRUCTION COMPLEXITY

  • With four skills, increasing constraints from 20 to 140 reduces Raw Success from .706 to .279, while SF-final falls from .825 to .708.Final–Raw gains reach 42.9 and 42.2 points at the two endpoints; the final–strict gap also widens along both tested axes.Source: §4.6 · pp. 8–9, Figure 5 · p. 9

4.7 CONTROLLED MECHANISM ANALYSIS

  • Hard conflicts have the largest isolated Success loss, while redundancy also lowers Success and Compliance and exceeds the soft-conflict loss in these scenarios.The joint condition has the lowest Success and Compliance, but its mean Success drop is .027 smaller than the sum of isolated drops (.219).Source: Table 3 · p. 9, §4.7 · p. 9

5 CONCLUSION

SkillFusion compiles independently authored skills into reusable specifications by analyzing typed, scoped interactions. Its evaluation connects task Success gains to interaction recovery and further examines transfer and instruction scaling.

5 CONCLUSION

  • SkillFusion compiles independently authored skills into reusable specifications through typed, scoped interaction analysis.Interaction-specific resolution improves task Success across three agent benchmarks, while selective fallback recovers conflicts missed by strict pruning.Source: §5 · p. 9
  • Human auditing, component ablations, and controlled mechanism experiments connect the reported gains to concrete stages of the compiler.Source: §5 · p. 9
  • Cross-model transfer and instruction scaling further demonstrate the utility of resolving skill interactions before execution.Source: §5 · p. 9

ETHICS STATEMENT

The paper evaluates skill composition on public agent benchmarks and reusable skill instructions, while noting that task performance and instruction consistency do not establish operational safety. Its protocol uses fixed task sets and repeated evaluations, with confidence intervals reflecting execution-induced uncertainty rather than task-sampling or compilation uncertainty.

ETHICS STATEMENT

  • More effective composition can facilitate harmful or unauthorized workflows, and LLM-generated resolutions can introduce incorrect instructions.Source: ETHICS STATEMENT · p. 10
  • Consequential deployment calls for independent permission checks, sandboxed execution, human approval, and appropriate protection of private data and upstream licenses.The evaluation concerns task performance and instruction consistency; operational safety requires separate assessment.Source: ETHICS STATEMENT · p. 10

A.1 FIXED EVALUATION PROTOCOL AND METRIC SCOPE

  • The protocol evaluates fixed benchmark sets, including 400 SpreadsheetBench Verified tasks, 812 WebArena-Verified tasks, 500 SWE-bench Verified tasks, and 164 HumanEval+ tasks.Compiled artifacts are fixed for a skill pool and transferred unchanged across backbones; HumanEval+ uses one program per task per run.Source: §A.1 · p. 13
  • Success uses benchmark-native correctness, while Compliance measures satisfied task–instruction judgments and Preservation tracks source constraints retained or covered.Instruction sets can differ between methods; Compression uses instruction-body tokens and excludes scripts, resources, and metadata.Source: §A.1 · p. 13
  • The reported Student-t 95% intervals quantify execution-induced uncertainty in mean scores conditional on the fixed task set and compiled artifact, not task-sampling or compilation uncertainty.Source: §A.1 · p. 13

A.2 ADDITIONAL ABLATION DIAGNOSTICS

The appendix reports diagnostics spanning ablations, compilation cost, interaction auditing, verifier thresholds, cross-model results, scaling, and prior RCR failures. The supplied passages give selected numeric findings and describe the scope of additional tables and figures, but do not provide their full result values.

A.2 ADDITIONAL ABLATION DIAGNOSTICS

  • The additional S60 ablation diagnostics report synthetic recall over 60 incompatible pairs, Preservation over 60 source constraints, and Activation over 164 routing probes.Table 4 covers variants from Figure 4; Success and Compliance are presented in that figure.Source: §A.2 · p. 13, Table 4 · p. 13

A.3 SUCCESS GAINS AND COMPILATION COST

  • RCR uses the same source skills in rewrite, critique, and revise calls; the appendix reports SF-final–RCR Success differences and compilation costs normalized separately to SF-strict = 100,000 in each domain.The table caption specifies paired Student-t 95% intervals for the Success differences, but the supplied passages do not give the table values.Source: §A.3 · p. 13, Table 5 · p. 14, Table 6 · p. 14

A.4 HUMAN AUDIT AND GATE VARIANTS

  • Fallback raised audited recall from .600 to .833 and F1 from .713 to .840, recovering 14 of 24 strict-gate false negatives while leaving 10.The audit contains 300 constraint pairs, including 60 incompatible and 240 compatible pairs; a no-gate variant reaches .917 recall but has 24 false positives versus nine for SF-final and costs 7.610 times SF-strict on the calibration set.Source: §A.4 · p. 14, Table 9 · p. 14

A.5 VERIFIER-THRESHOLD SENSITIVITY

  • Across the verifier-threshold sweep, observed Success varies by .003 on S60 and .006 on SpreadsheetBench.The threshold controls adversarial re-judging of the 1,184 SF-final candidate pairs; Figure 6 shows mean Success over four S60 runs and three SpreadsheetBench runs.Source: §A.5 · p. 15, Figure 6 · p. 15, Table 10 · p. 15

A.6 FULL CROSS-MODEL RESULTS

  • The cross-model figures report all nine methods across three models, with confidence intervals, and transfer fixed compiled artifacts unchanged across backbones.The supplied figure captions identify SpreadsheetBench and WebArena Success and mark retrospective oracle references with hatched bars, but provide no outcome values.Source: §A.6 · p. 15, Figure 7 · p. 16, Figure 8 · p. 16

A.7 ADDITIONAL SCALING RESULTS

  • The scaling results use 164 tasks over 20 runs and report Success, Compliance, and paired SF-final–Raw Success differences.The supplied captions specify confidence intervals but do not provide the plotted outcome values.Source: §A.7 · p. 15, Figure 9 · p. 17, Figure 10 · p. 17

A.8 RCR DIAGNOSTICS AND PRIOR FAILURE AUDIT

  • The RCR diagnostics include source-constraint coverage, a descriptive set of 30 previously annotated failures, and cross-model SpreadsheetBench and WebArena Success figures.The figure captions specify means and confidence intervals, but do not provide plotted values.Source: §A.8 · p. 15, Table 11 · p. 15, Table 12 · p. 16
  • The paired SF-final-minus-Raw Success figure reports percentage-point differences with Student-t 95% intervals, while the compliance figure covers both scaling axes and notes shared records for the four-skill and 80-constraint points.Source: Figure 10 · p. 17, Figure 9 · p. 17

B PIPELINE INTERMEDIATE STATISTICS

The pipeline statistics describe constraint extraction, candidate generation, and the completed artifact, alongside an example of scope disambiguation. They distinguish source accounting and intermediate generation events from final-artifact counts.

B PIPELINE INTERMEDIATE STATISTICS

  • For one fixed SF-final artifact per domain, the parser extracts 160, 78, and 108 constraints, while fallback candidate generation admits 19.0%, 17.6%, and 7.8% of possible cross-skill pairs.The table caption states that each column covers one compilation run and its final artifact.Source: B PIPELINE INTERMEDIATE STATISTICS · p. 17, Table 13 · p. 18
  • The scope-disambiguation example assigns openpyxl to workbook editing and the standard-library OOXML path to structural auditing, making the workflow-phase distinction explicit in the merged instruction.Source: B PIPELINE INTERMEDIATE STATISTICS · p. 17
  • Type counts partition source constraints, script-derived constraints are a subset, and cumulative generation events include intermediate outputs rather than measuring the active set; final constraints count the completed artifact.Retained and replaced partition source constraints; Preservation uses retained or subsumed constraints, while Compression uses instruction-body token counts.Source: B PIPELINE INTERMEDIATE STATISTICS · p. 17, Table 13 · p. 18

C LIMITATIONS AND FUTURE DIRECTIONS

The appendix situates SkillFusion among agent, skill, and instruction-following research, then documents limitations and compiler procedures. Its prompt templates specify how constraints are extracted, classified, resolved, and compressed.

C LIMITATIONS AND FUTURE DIRECTIONS

  • The study is limited by added LLM-call costs, reliance on LLM judgments and pairwise analysis, and evaluation on fixed skill pools and textual instructions with statically inspected scripts.The authors propose caching, batching, incremental recompilation, joint constraint reasoning, program-analysis checks, human validation, broader pools, task-aware compression, and visual or state-dependent constraints for future work.Source: C LIMITATIONS AND FUTURE DIRECTIONS · p. 18

D.1 LLM-BASED AGENTS

D.2 SKILL DESIGN, ACQUISITION, AND REFINEMENT

D.5 INSTRUCTION-FOLLOWING UNDER CONSTRAINT PRESSURE

E.1 STAGE I: EXPLICIT CONSTRAINT EXTRACTION

  • The extraction prompts structure explicit constraints by type, directive, and activation scope; script-derived constraints are filtered to retain task-solving behavior and exclude environment-specific noise.The scope schema uses five cue dimensions and marks a directive global only when it applies unconditionally.Source: §E.1 · p. 20, §E.2 · p. 21, §E.3 · p. 21

E.4 STAGE II: CONFLICT CLASSIFICATION

  • Conflict prompts distinguish hard, soft, redundant, and compatible pairs, while low-confidence judgments receive adversarial re-judging and rule-based adjudication.On disagreement, the protocol preserves a primary hard-conflict label at confidence ≥0.6; otherwise, the second verdict overrides only with a confidence advantage of at least 0.15.Source: §E.4 · pp. 22–23

E.5.1 HARD-CONFLICT RESOLUTION

  • Hard-conflict prompts route only genuinely different scopes and otherwise require explicit priority; soft abstractions are checked for violations, with failed checks sent to the hard-conflict resolver.Redundancy prompts retain the more specific or complete formulation.Source: §E.5.1 · p. 23, §E.5.2 · p. 24, §E.5.3 · p. 24

E.5.4 CLUSTER SUMMARIZATION

  • Compression prompts summarize clusters only when summaries entail their originals, while metadata prompts preserve discovery coverage for every source skill.A failed entailment check leaves a cluster uncompressed; metadata coverage checks whether the merged description still activates for tasks that triggered each source skill.Source: §E.5.4 · pp. 24–25, §E.5.5 · p. 25

F PIPELINE IMPLEMENTATION DETAILS

The implementation normalizes activation scopes to guide candidate selection, extracts script-backed behavioral constraints, adjudicates uncertain classifications, and compresses resolved directives with validation.

F.1 SCOPE NORMALIZATION

  • Scope normalization retains each directive’s original wording and represents activation with five cue dimensions; dimensions are conjunctive, values within a dimension are alternatives, and global scopes overlap all scopes.Selective fallback adds excluded pairs with uncertain flags, global scope, or sufficiently high embedding similarity; SF-strict disables this admission step.Source: §F.1 · p. 25

F.2 SCRIPT CONSTRAINT EXTRACTION

  • Static script inspection infers behavior assumptions supported by code or declared interfaces, then filters out environment-specific noise while retaining task-solving constraints.Accepted constraints retain provenance linking them to their script origin.Source: §F.2 · p. 26

F.3 ADVERSARIAL RE-JUDGING PROTOCOL

  • A second judge re-evaluates pairs when primary confidence is below 0.65, and disagreement is adjudicated using confidence thresholds and a preference for sufficiently confident primary hard-conflict labels.Merge bookkeeping tracks replacements and verdicts and checks that each source identifier maps to one final directive.Source: §F.3 · p. 26

F.4 COMPRESSION OPERATORS

  • Three sequential compression operators factor shared directives, cluster related directives only after entailment verification, and render the result in a canonical skill layout.Failed entailment checks leave clusters uncompressed; scripts and resources retain their source-directory structure in separate bundles.Source: §F.4 · p. 26

G LANGUAGE MODELS AND COMPUTE

DeepSeek-V3.2 handles compilation and serves as the primary executor; GLM-4.7 and MiniMax-M2.5 test transfer using the same compiled artifacts. Real-world benchmarks use publicly available skill pools.

G LANGUAGE MODELS AND COMPUTE

  • DeepSeek-V3.2 performs compiler semantic calls and primary task execution, while GLM-4.7 and MiniMax-M2.5 evaluate transfer with the same artifacts; all use commercial APIs without fine-tuning.DeepSeek-V3.2 also judges instruction compliance from agent outputs.Source: G LANGUAGE MODELS AND COMPUTE · p. 26

H SKILL POOL DETAILS

H.1 HUMANEVAL+ SYNTHETIC SKILL CONSTRUCTION

The HumanEval+ controlled experiments construct synthetic skills from coding-practice directives, varying seed subsets, skill counts, and constraint counts to examine composition under controlled configurations.

H.1 HUMANEVAL+ SYNTHETIC SKILL CONSTRUCTION

  • Synthetic skills are generated from 12 coding-practice seed directives using the official skill-creator, with sampled subsets of one to six seeds controlling skill count and constraint count.The seeds encode contrasting preferences, including functional purity versus object-oriented encapsulation and concise versus defensive coding.Source: §H.1 · p. 27
  • Scaling evaluations vary total constraints from 20 to 140 while fixing a four-seed subset, or vary one to six skills while fixing 20 constraints per skill.Each configuration is evaluated over all 164 HumanEval+ tasks with 20 runs.Source: §H.1 · p. 27
Loading 2610.15481v1…