One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents

1 Introduction

• Nearly one in four installed skills is co-installed with a skill doing the same job, and 37% of judged skills in copied-collection projects have such a counterpart.The prevalence estimates are conservative because public repositories rarely reveal personal skills and plugins.Source: §1 · p. 2
• Task-pass and skill-retrieval benchmarks can miss conflicts because the task may pass while the installed skill’s conventions or safety rules are lost.The paper notes that similar skills do not ask for 73% of the installed skill’s applicable core functions in its experimental pairs.Source: §1 · p. 2
• The study defines checkable core functions from skill text, measures their fulfillment alongside task completion, and uses tool-call records to distinguish substitution from interference.It evaluates 312 experimental pairs selected after mining repository snapshots and judging candidate pairs.Source: §1 · p. 3
• A similar skill takes one in five runs from the installed skill without reducing task completion; fidelity falls by 2.6 points overall and 5.6 points on exclusive core functions.The decline in use of the installed skill is 19.9 percentage points, compared with 5.9 points for an unrelated skill.Source: §1 · p. 3
• A conflict is decided at the first skill read, almost always before file changes; denying the similar skill’s first read made the model switch to the installed skill in 96% of cases.The hook restored exclusive-core-function fidelity to the level of runs that opened the installed skill first, with a significant gain for every model.Source: §1 · p. 4
• The paper contributes a repository-scale study, requirement-level evaluation, a first-read guard, and released cases, harness, and run logs.Its stated implications address platforms, skill authors, and benchmark designers.Source: §1 · p. 4

2.1 Agent Skills in Claude Code

• Claude Code lists skill names and descriptions before the model chooses one, then loads only the chosen skill’s instructions; scope rules can override or list same-named skills differently.Other platforms differ in which same-named skill wins or whether both are listed.Source: §2.1 · p. 4

2.2 Skill Conflicts

• A conflict changes what the user gets from the installed skill through substitution or interference, and both runs in the checkpoint example complete the task despite a lost prohibition.The similar skill is used and the model commits work that the project skill forbids.Source: §2.2 · p. 5

2.3 Skill Types

• Normative skills can lose fidelity without changing task completion, capability skills can affect both, and script-bearing skills risk losing or misapplying their scripts.Normative skills encode conventions; capability skills supply needed knowledge or tools.Source: §2.3 · pp. 5–6

3.1 Overview

• The study fixes its loss measure in advance and checks its main effects on Codex, where similar skills lowered installed-skill use by 18.2 percentage points and exclusive-core-function fidelity by 3.4 points without lowering task completion.The same pairs on Claude Code showed declines of 20.1 and 4.6 points, respectively.Source: §3.1 · p. 6

3.2 Installation Reconstruction

• Repository snapshots yield 5,106 installation lists, including 4,342 with at least two distinct skills, while real projects and dotfiles supply installed skills and collections supply candidates.The corpus snapshots include skills missed by code search.Source: §3.2 · pp. 6–7

3.3 Skill Filtering

• Filtering keeps 207,863 of 219,817 downloaded skill files and groups near-duplicate versions into families, preventing pairs from the same family.Skills must have usable frontmatter and at least ten instruction lines.Source: §3.3 · p. 7

3.4 Candidate Retrieval and Confirmation

• The retrieval-and-confirmation process embeds names and descriptions to shortlist candidates, then uses a pair judge to confirm same-job or near-duplicate pairs.The study pairs skills within projects and across projects to represent existing co-installation and additions from elsewhere.Source: §3.4 · pp. 7–8

3.5 Sampling and Prevalence

• Of 3,754 stratified sampled pairs, the pair judge confirms 1,639, with near-perfect repeat agreement (Cohen’s κ = 0.93).The sample also labels 3,115 skills as normative or capability: 1,730 normative and 1,385 capability.Source: §3.5 · p. 8

3.6 Core Function Extraction

• Core functions are skill-quoted, unique requirements that can be checked in task outputs; the first three applicable functions are primary, and conditional ones count only when their condition occurs.The extractor ranks requirements by wording strength, relevance to purpose, effect, and order, retaining up to ten per skill.Source: §3.6 · p. 8
• 73% of the 2,518 applicable core functions in experimental pairs are exclusive to the installed skill rather than shared with its similar partner.Exclusivity is assessed by comparing each installed-skill function with the partner’s instructions and core functions before any run.Source: §3.6 · p. 8

3.7 Pair Selection and Case Construction

• The experiment includes 312 eligible pairs, requiring a confirmed pair, a runnable installed skill with at least three core functions, and retrievable skill directories.Each installed skill appears once; 81 cross-project pairs have same-named skills.Source: §3.7 · p. 9
• Each pair receives one shared request and starting repository, with checks for task completion and applicable core functions; task completion accepts any competent way of doing the work.The request omits skill names, conventions, and description phrases, and the starting repository contains no agent-directed instructions.Source: §3.7 · p. 9

3.8 Configurations and Setup

• Configurations compare A alone, B alone, A+B, and an unrelated-skill control, while additional tests reverse listing order and vary scope for same-name pairs.Runs start in fresh containers and use three Claude model tiers; repeated-run comparisons show A-use agreement in 88.9% of pairwise comparisons.Source: §3.8 · pp. 9–11

3.9 Measures and Analysis

• Each run records fidelity, task completion, skill use, and disclosure; scripts decide 327 primary core-function checks, while an output judge decides the other 594 and most task-completion checks.Conditional core functions count as not applicable when their condition does not occur.Source: §3.9 · p. 11
• Paired comparisons test co-installation, reversed listing order, and first-read effects; a pre-tool hook blocks B until A is read and measures changes in A’s fidelity.The hook experiment reruns 187 pair–model combinations, each with and without the hook in random order.Source: §3.9 · pp. 11–12

3.10 Use of LLMs and Human Validation

• Two independent human raters validate random samples of every LLM-assisted step, with agreement reported using Krippendorff’s α.The raters label samples independently of the LLM labels and resolve disagreements by discussion.Source: §3.10 · p. 12

4.1 Prevalence

• Nearly one in four installed skills has a same-job skill in the same installation list, while 37% of judged skills in projects copying whole collections do.Most confirmed pairs involve normative skills, followed by capability skills.Source: §4.1 · pp. 12–13

4.2 RQ1: Impact of Conflicts

• With B co-installed, 12.1% of runs use only B and 38.0% use neither skill; those groups carry 36% and 45% of the fidelity loss, respectively.Runs using A carry 20% of the loss.Source: §4.2 · p. 13
• 19.9 pp fewer runs use A and its fidelity falls 2.6 pp, while task completion does not drop; fidelity on A-exclusive functions falls 5.6 pp.The similar skill’s effect on A use exceeds that of an unrelated skill, which lowers use by 5.9 pp.Source: §4.2 · p. 13, Figure 4 · p. 14

4.3 RQ2: Selection and Disclosure

• A personal-directory B lowers A use by 35.0 pp, whereas a plugin B lowers it by 6.2 pp; the models also differ sharply in skill use.Opus 5 uses both skills in 36.8% of A+B runs, while Haiku 4.5 uses neither in 56.4%.Source: §4.3 · pp. 14–15
• Installation scope determines selection for same-name skills, while listing order changes A use by only +0.6 pp; substituted runs name the skill used in 0.9% of replies.A personal-scope B overrides A in every listing, though models still read A’s project files in 29.2% of runs.Source: §4.3 · pp. 14–15, Figure 5 · p. 15

4.4 RQ3: Decision Point

• A pre-tool hook at the first read of B redirects the model to A after 96% of denials and raises fidelity on A-exclusive functions by 9.1 pp.The hook’s gain among runs that reach for B first is significant for every model.Source: Figure 6 · p. 16, §4.4 · p. 16
• Opening B first reduces fulfillment of A’s exclusive core functions by 9.4 pp, and 97% of such runs change no file before that first read.The runs look alike up to the first read, with a median of six tool calls in each condition.Source: §4.4 · pp. 15–16, Figure 6 · p. 16

5.1 Implications for Skill Evaluation

• Benchmarks should measure exclusive core functions alongside task completion, since completion alone misses fidelity losses when a similar skill is co-installed.Core functions can be extracted once per skill and checked by scripts where possible; benchmarks that already check skill requirements could co-install a similar skill.Source: §5.1 · p. 17

5.2 Implications for Agent Platforms

• Platforms can expose which skill and scope ran, warn about same-name overrides, and guard the first skill read or let projects declare which skill must win.A pre-tool hook or a lockfile-like pin can act at the first read; showing the selected skill could also expose runs that use neither skill.Source: §5.2 · p. 17

5.3 Implications for Skill Authors and Users

• Distinctive names and task-specific descriptions can reduce overrides and misrouting, while users and projects can audit personal skills and remove duplicated skills from copied collections.Source: §5.3 · p. 17

• 512 repeated combinations agreed on installed-skill use in 88.9% of comparisons, while intraclass correlations were 0.78 for use and 0.84 for core functions.Three runs would give nearly the same results at three times the cost; the guard experiment randomized the first-read hook.Source: §6 · p. 17
• The full design runs on Claude Code; checks found similar use-of-A drops on Codex and for within-project pairs.For the same pairs, use of A fell 18.2 percentage points on Codex versus 20.1 on Claude Code; within-project pairs showed an 18.3-point drop.Source: §6 · p. 17

• Downloaded skills were stored as data and executed only in isolated containers without credentials, and materials were scanned before human raters received them.The study had two human raters, both authors, and no other human participants.Source: §7 · p. 18

• Earlier skill studies examine security, reusability, and selection, while benchmarks mostly score task passage or retrieval and do not install two skills that do the same job.Some recent benchmarks check requirements written in a skill, but none in this group measures one same-job skill taking over from another.Source: §8 · p. 18
• Tool-selection studies measure which tool is chosen, whereas selecting a similar skill can change how the whole task is done.Source: §8 · p. 18
• Software engineering studies feature interactions and package conflicts; Decca grades dependency conflicts by feature coverage, while skills add model selection based on natural-language descriptions.Source: §8 · p. 18

9 Conclusion

• One in five runs uses a similar skill instead of the installed skill, lowering fidelity on exclusive core functions while task completion stays flat.Source: §9 · p. 18
• Install location decides which skill runs, while final replies rarely identify the skill used.Source: §9 · p. 18
• Conflicts are decided at the first skill read, where a pre-tool hook keeps the installed skill’s core functions.Source: §9 · p. 18
• Skill evaluations should measure unique contributions, and platforms should guard the first read and show which skill ran.Source: §9 · p. 18

Data Availability

• The package includes data rules, prompts, and scripts for rebuilding installation lists from repository snapshots.Source: Data Availability · p. 19
• It provides identifiers for analyzed skills, experimental cases with core functions and checks, and annotations.Source: Data Availability · p. 19
• The package includes the experiment harness, all 6,368 run logs, and judging and analysis scripts.Source: Data Availability · p. 19
• Skill contents are released where licenses permit; otherwise, identifiers and a re-fetch script are provided.Source: Data Availability · p. 19