PaperlayerView exact revision on arXiv

Source-linked AI summary

Skill Constellations: Tracing the Supply Chain of Agent Skills on GitHub

Fahd Seddik

arXiv:2610.11169v1cs.SEcs.CRcs.SI

TL;DR

Agent skills are copied across repositories, but the supply chain lacks provenance, pinned versions, and a fix channel. The authors reconstruct a dated copy network from Git history and find that changes rarely reach every copy.

  • Problem

    Snapshots of copied skills cannot show who copied from whom or when, while the supply chain lacks provenance, pinnable versions, and a fix channel.

    Source: §I · p. 2

  • Method

    The authors reconstruct a dated, directed skill-copy network from Git histories and fit a model of which repositories others copy from to rank audits.

    Source: §I · p. 2, §II · p. 2

  • Results

    11.1% of clone-group changes reach every copy, while transmissions are concentrated in a few repositories.

    Source: C. RQ3: Risk · p. 4, A. RQ1: Structure and Growth · p. 4

  • Takeaways & Limitations

    The work ranks repositories for review, complementing security research that detects dangerous skills.

    Source: §I · p. 2

    The data cover the first ten months of the format on public GitHub, and risk counts are lower bounds because flags mark capability rather than malice.

    Source: §VI · p. 4

Abstract

from arXiv · show

Agent skills are SKILL.md instructions and scripts that AI coding agents such as Claude Code and Codex run with the permissions of their user. Developers share skills by copying them between repositories, which makes them a software supply chain without a registry, versions or provenance. The origin of a copied skill, the reach of a security fix and the repositories that warrant review are therefore unknown. Studies that record which repositories hold a skill at a single point in time cannot reveal who copied it from whom. We contribute the first dated copy network of agent skills, built from the git history of every SKILL.md in GitSkills and covering 2,193,119 skill adoptions across GitHub, together with an interactive viewer. A few repositories are the source of almost all copies, and GitHub stars do not identify them. Skill copies almost never change with their source, and a fix at the source therefore rarely reaches them. We fit a model of which repositories others copy from and use it to rank repositories for audit. Reviewing the 100 repositories it ranks highest prevents 14.9% of later adoptions of high-risk skills, against 0.5% for the 100 most starred, which gives security engineers a short list to check before a skill spreads. Platforms should therefore distribute versioned references rather than copies. Project Website: https://fahdseddik.github.io/Skill-Constellations/

I. INTRODUCTION

Agent skills run with their users’ permissions, and sharing them by copy creates a supply chain without provenance, version pinning, or a fix channel. The introduction motivates reconstructing that supply chain and targeting audits at its sources.

I. INTRODUCTION

  • Skills contain instructions and often scripts that can run shell commands, access networks, and modify files with the agent’s user permissions.That makes their distribution a security concern, not just a reuse pattern.Source: §I · p. 1
  • The supply chain lacks a registry for provenance, pinnable versions, and a channel for delivering fixes.Half of GitHub SKILL.md files are verbatim copies of another file.Source: §I · p. 2
  • Snapshot studies cannot reveal who copied a skill from whom or when, leaving key supply-chain questions unanswered.Prior studies examine snapshots, individual skills, or pairs of skills [3], [4], [5], [6],, [8].Source: §I · p. 2
  • The authors reconstruct a dated, directed copy network from Git history and define each skill’s copy network as its constellation.The network covers GitSkills’ SKILL.md histories and is validated against installer records.Source: §I · p. 2
  • Skill copies do not change with their source, and modified copies gain capabilities by adopting other versions rather than through owner edits [11].Source: §I · p. 2
  • The authors fit a source-choice model to rank repositories for audit, complementing prior work that detects dangerous skills [4], [5],.Source: §I · p. 2

II. DATA AND METHOD

The study reconstructs adoptions and copy events from Git history, models source choice and spread, and evaluates repository audits against installer records and labelled samples.

II. DATA AND METHOD

  • The dataset retains valid SKILL.md files first committed from 1 October 2025 in non-fork repositories and dates adoption by first commit, bounded by repository creation.Each distinct content is treated as one skill.Source: §II · p. 2
  • A copy event requires a commit adding at least ten SKILL.md files, with one earlier adopter holding at least half of existing lineages; larger bundles are bulk copies.A lineage joins versions linked by edits, and its first adoption is its origin.Source: §II · p. 2
  • Figure 3 tracks whether clone-group changes reach every copy and whether modified copies gain command execution through version adoption or owner edits.The caption reports 11.1% consistent changes and 3.32-fold higher odds for a single owner.Source: Figure 3 · p. 3
  • The authors model copy-event source choice with a conditional logit, simulate weekly individual adoption, and evaluate audits on a temporal split.Auditing a repository removes its flagged skills and their later descendants from held-out high-risk adoptions.Source: §II · p. 3
  • Reconstructed dates fall within seven days of installer records in 94.6% of 1,534 folders, while the source rule names the recorded source in 26.7%.The rule credits the earlier adopter of the most lineages, usually a catalog, so source-level measures describe distribution rather than authorship.Source: §II · p. 3
  • Two language models independently label repository types and sampled skill risks, with disagreements and declined items resolved manually before validation.Against the resolved reference, Opus reaches κ = 0.91 on repository types and κ = 0.92 on its weakest skill question; the strict risk flag has 98.3% precision and 80.9% recall.Source: §II · p. 3

A. RQ1: Structure and Growth

Skill spread is organized around repository-level catalog copying rather than isolated individual adoptions. Transmission is concentrated, and prior copying predicts which sources bundle copies prefer.

A. RQ1: Structure and Growth

  • Most repositories share no skills, while the remainder form 50 topic-organized communities of at least 100 repositories; repositories copy catalogs or bundles far more often than individual skills.Source: A. RQ1: Structure and Growth · p. 3
  • A source-choice model estimates a coefficient of 1.64 for the logarithm of past copies, while attachment to individual skills is sublinear across all nine threshold settings.Source: A. RQ1: Structure and Growth · p. 4
  • Transmission is highly overdispersed (negative binomial k = 0.086): most adoptions transmit a skill to no other repository, while a few repositories account for almost all transmissions.Source: A. RQ1: Structure and Growth · p. 4

B. RQ2: Sources

GitHub stars poorly identify skill sources, while dated copy-network measures better identify future transmission sources. Many top static out-degree repositories prove to be late adopters.

B. RQ2: Sources

  • Only 7 of the 20 repositories with the highest static out-degree remain in the top 20 after crediting only skills they held first.The others are late adopters.Source: B. RQ2: Sources · p. 4
  • Current copy out-degree identifies 4.0 times as many later transmissions as stars, while PageRank and the number of skills held perform comparably.Source: B. RQ2: Sources · p. 4
  • Most unstarred top sources are ordinary projects, and stars are nearly uninformative about dated-network out-degree.The passage also notes that fake-star campaigns make stars manipulable.Source: B. RQ2: Sources · p. 4

C. RQ3: Risk

Skill copies rarely track source changes, limiting the reach of source fixes; modifications can also alter command-execution capabilities, and copies cross agent platforms.

C. RQ3: Risk

  • Only 20.9% of skill copies follow a later edit of their source, and just 4.0% of genealogies ever change consistently.Consistent changes become less likely when a clone group spans several owners; a single owner raises their odds 3.32-fold, controlling for group size, platform and content.Source: C. RQ3: Risk · p. 4
  • Modified copies gain command execution more often than they lose it, with gains arising from adopting another skill version rather than owners’ edits.Against the exact version copied, most security-sensitive lines added by adaptations replace content of the same kind.Source: C. RQ3: Risk · p. 4
  • Copies cross agent platforms without adaptation, and all 56 manually checked skills in other agents’ folders named tools only Claude Code provides.Skills in the folders of other agents sometimes name tools available only in Claude Code.Source: C. RQ3: Risk · p. 4

D. RQ4: Intervention

A temporal audit ranking based on the source-choice model prevents more held-out high-risk adoptions than the most-starred or random repository lists. Its advantage persists across alternative splits and simulation, while indirect copies, gradual spread and cross-platform copying shape the review strategy.

D. RQ4: Intervention

  • At the 1 April 2026 split, auditing the 100 repositories ranked highest by the source-choice model prevents far more held-out high-risk adoptions than auditing the 100 most-starred or 100 random repositories.The ordering also holds for splits at 1 March and 1 May, under the lenient flag, and in a calibrated simulation; a quarter of prevented adoptions are indirect copies.Source: D. RQ4: Intervention · p. 4
  • Even a hindsight-chosen audit prevents only about a quarter of high-risk adoptions, because roughly half arrive as new skills.A new skill reaches few of its eventual adopters in its first week, leaving reviewers weeks rather than hours.Source: D. RQ4: Intervention · p. 4
  • A third of copies between agent folders cross platforms, so reviews must cover every agent.The result is reported for the held-out period and the copy-network audits.Source: D. RQ4: Intervention · p. 4

IV. RELATED WORK

Prior work examines skill reuse and agent workflows but does not trace skill spread between repositories. Related studies of copied code, dependencies and vulnerabilities likewise leave much copy-based reuse outside dependency analysis.

IV. RELATED WORK

  • GitSkills copy counts and clone detection identify copied or similar skills but do not show who copied from whom or how changes propagate; prior studies do not measure inter-repository skill spread.Gao et al. find that 53% of reused registry skills are never modified [8], while studies of agent pull requests, context files and injected instructions examine other aspects of agent use,,,, [17].Source: §IV · p. 4
  • Only 2.43% of copy-based reuse is visible to dependency analysis, while dependency studies trace vulnerabilities through declared dependencies that skills lack.Prior work also studies fork commits, clone genealogies within single projects, and vulnerabilities or malware in replicated packages and poisoned repositories,,,,.Source: §IV · p. 4

V. ETHICAL CONSIDERATIONS

The authors limit personal and repository-level disclosures in their reporting.

V. ETHICAL CONSIDERATIONS

  • The study stores no author names, email addresses or commit messages, and reports confirmed malicious skills only in aggregate without naming repositories or owners.These reporting choices limit identification of people and repositories associated with confirmed malicious skills.Source: §V · p. 4

VI. CONCLUSION

The source-choice model prevents more held-out high-risk adoptions than stars, while new skills take weeks to reach most adopters and a third of copies cross platforms. The paper recommends versioned references and timely review, but its data cover only the first ten months of public GitHub.

VI. CONCLUSION

  • Platforms should distribute versioned references, check that skills name only the loading agent’s tools, and users must update the copies they rely on.Copies do not follow their sources.Source: §VI · p. 4
  • Reviewers and registries should rank repositories by current copy out-degree rather than stars, and act within the first weeks of a skill.Source: §VI · p. 4
  • The evaluation covers the format’s first ten months on public GitHub, when distribution practices were forming.Risk flags indicate capability rather than malice; their high precision and low recall make risk counts lower bounds.Source: §VI · p. 4
  • The audit evaluation depends on which repositories are sources, not on who wrote them.Reconstructed dates and order agree with installer records.Source: §VI · p. 4
  • At every split, the source-choice model prevents far more held-out high-risk adoptions than stars; new skills take weeks to reach most adopters, and a third of copies cross platforms.Source: Figure 4 · p. 5
Loading 2610.11169v1…