Source-linked AI summary
SPECMINE: A Large-Scale Corpus of Spec-Driven Development Artifacts
Shyam Agarwal, Bogdan Vasilescu
TL;DR
SDD artifacts increasingly guide AI coding agents, but their ecosystem and relationship to implemented code had not been studied at scale. SPECMINE addresses this gap with large public-GitHub censuses, enriched specification records, pull-request changesets, and typed traceability references, producing a corpus for studying software specification in the age of AI agents.
Problem
SDD artifacts, their producing tools, and their relationships to shipped code lacked a dataset capturing them at scale.
Method
SPECMINE combines broad and Kiro-specific censuses with enriched specification histories and structure, spec-touching pull requests, and a typed traceability index.
Results
470,795 specifications across 73,030 repositories and 17 named tools, plus 98,574 Kiro artifacts across 12,910 repositories, are accompanied by 5,992 pull requests and 2,421,323 typed references.
Takeaways & Limitations
SPECMINE enables empirical study of how software is specified and connected to code across the age of AI agents.
Abstract
from arXiv · showhide
Spec-Driven Development (SDD) is a fast-emerging practice in which a structured natural-language specification, written by a developer, or (more often) drafted by an AI tool and then curated by the developer, drives an AI coding agent's implementation. A wave of tooling (GitHub Spec Kit [3], OpenSpec [4], AWS Kiro [5], and dozens of others) has appeared since 2025, yet the artifacts these tools produce have never been studied at scale. We present SPECMINE, a corpus that captures SDD in public GitHub repositories through two censuses: a broad census of spec.md/specs.md files covering most tools (470,795 files across 73,030 repositories, attributed to 17 named tools), and a Kiro census of its distinct requirements/design/tasks layout (98,574 files across 12,910 repositories). Each spec is enriched with full repository metadata, complete commit history, and parsed document structure. How a spec becomes code is itself an open question, so for 11 tools we sweep every pull request that touches a spec in their repositories with at least ten stars, capturing 5,992 such PRs across 581 repositories with their changesets. That makes the simplest workflow, spec and implementation changing together in one PR, directly observable, and a census-wide index of 2,421,323 typed references (1.28M to code files, 863k to sibling documents, 152k to PRs, 62k refs, 43k branches, 22k issues) gives a second, independent link from spec to code. SPECMINE lets the community study, for the first time, how software is specified in the age of AI agents.
A. Why spec-driven development, and why now
SDD makes structured specifications a central artifact for directing AI-generated code, but these artifacts and their links to shipped code lacked a large-scale dataset. SPECMINE fills this gap by capturing specifications across tools and following them into pull requests.
- SDD tools use Markdown specifications to guide AI agents in generating and modifying code.
- The practice expanded rapidly after 2025 across thousands of repositories and competing tools.
- SPECMINE captures the intent layer across 18 SDD tools and follows specifications into pull requests in mature repositories.
B. What the data set contains
SPECMINE combines a broad filename-based census, a separate Kiro-layout census, pull-request data, and traceability references. The resulting records include repository provenance, structural features, histories, and links from specifications to code.
- The broad census identifies spec.md/specs.md files, attributes them to 17 named tools, and preserves repository and file provenance.Attribution uses directory-layout fingerprints, with a manual check finding all 30 sampled attributions correct.
- The separate Kiro census recovers 98,574 artifacts across 12,910 repositories because Kiro uses requirements, design, and task documents instead of spec.md files.
- The pull-request layer sweeps repositories with at least ten stars for every PR touching specifications across eleven widely adopted tools.
- A 2,421,323-reference index independently links specifications to code through typed references and resolved OpenSpec task-to-code links.The OpenSpec subset contains 435,401 task-to-code references resolved against repository trees at anchoring commits.
C. Summary statistics
SPECMINE v1.0 is a July 2026 snapshot spanning the emergence of SDD, with broad scale, multiple tools, detailed histories and structure, and a curated pull-request layer.
- 470,795 specifications span 73,030 repositories across 17 named tools, alongside 12,910 Kiro repositories.
- 99.7% of specifications were first committed in 2025 or later, and 92% were first committed in 2026.
- Every specification includes full commit history and 39 structural features, while the corpus also contains 266,230 OpenSpec change artifacts.
- 5,992 spec-touching pull requests across 581 repositories form the curated depth layer.
II. INTERNAL STRUCTURE
The dataset uses a relational schema centered on broad-census specification files, with attached histories, structural features, Kiro records, pull requests, OpenSpec references, and traceability data. Construction procedures document filtering, census design, PR sampling, and co-change limits.
- Data organization: The relational schema centers on spec_files, keyed by a file URL digest, with Kiro and pull-request layers attached through repository names.
- Data organization: Satellite tables store per-file commits, 39 content features, Kiro files, tool-specific pull requests, OpenSpec artifacts, code references, and traceability links.
- Documentation: The entity-relationship diagram, condensed data dictionary, appendices, and complete DATA_DICTIONARY.md document the released schema.The complete dictionary covers all 57 tables and 815 columns.
- Construction and quality control: The construction workflow documents adaptive partitioning, determinism checks, filtering from 822,901 rows to 470,795 kept files, and the PR layer’s sampling and co-change heuristic.
III. HOW TO ACCESS
SPECMINE is distributed in citable, analysis-ready formats, with direct loading options and practical guidance for querying or extending the release.
- Each snapshot ships on Zenodo with a citable DOI as a MySQL dump, CSV/Parquet exports, and JSONL spec contents.
- The GitHub mirror provides the schema, loader scripts, an example Jupyter/Colab notebook, and a 500-repository sample.
- A same-PR spec/code query is a single join between tool pull-request tables and their per-file changesets.
- Participants can re-fetch rows or join external issues, CI logs, and registries because every identifier is a live GitHub URL.
- SQL and basic pandas suffice for the Parquet layer, while the full MySQL image requires about 14.7 GB.
A. Licensing, ethics, and privacy
The release uses public GitHub artifacts under repository-specific terms, preserving licensing information and limiting distributed personal data while enabling aggregate authorship analysis.
- All artifacts come from public GitHub repositories and were collected within the API terms of service.
- 28,698 repositories carry a recognized SPDX license, allowing participants to filter for license-compatible subsets.
- Redistribution follows each repository’s terms rather than a single corpus-wide licensing rule.
- Public author and committer logins and numeric IDs are retained for authorship analysis, while commit emails, avatars, and remaining profile fields are dropped.
- Participants are asked to avoid deanonymization and keep personal data out of published results beyond aggregate analysis.
IV. RESEARCH QUESTIONS
SPECMINE frames research questions around adoption, specification quality, spec–code relationships, human–AI collaboration, and specification lifecycle.
- Adoption and diffusion of spec-driven development: SPECMINE enables studies of who adopts SDD and how adoption diffuses across languages, domains, team sizes, and ecosystems.
- Anatomy and quality of specifications: Raw content and 39 structural features across 468,307 specs support research on specification forms, quality, and template reuse.
- The spec–code relationship: The PR layer and traceability index support questions about where implementations live, synchronization, drift, and declared code paths.
- Human–AI collaboration around specs: Authorship, commit cadence, and template signals support questions about distinguishing human-authored from agent-generated specs and shared authorship.
- Lifecycle, evolution, and abandonment: Full per-spec commit histories and lifecycle roles support longitudinal questions about churn, half-life, abandonment, and reconstructable prior states.
V. DATA AVAILABILITY
SPECMINE v1.0 is available in multiple tiers and formats, combining a citable full archive, a direct-use Parquet mirror, and a GitHub mirror with documentation and a sample.
- Version 1.0 is released in three tiers so users can choose a scale suited to their study.
- The full Zenodo dataset provides a citable DOI, MySQL dump, flat CSV/Parquet exports, and JSONL spec contents.
- The Hugging Face mirror publishes every released table as Parquet for direct use with pandas, polars, or DuckDB.
- The GitHub mirror includes schema, loaders, a complete data dictionary, an example notebook, and a curated 500-repository sample.
- The sample contains 28,583 specs, 18,585 Kiro artifacts, 5,992 spec-touching pull requests, and 261,032 traceability references.
- Every identifier is a live GitHub value, repository licenses remain attached, and later snapshots receive separate versioned Zenodo records.
APPENDIX B: CONSTRUCTION AND QUALITY CONTROL
SPECMINE combines adaptive, auditable GitHub censuses with a sampled pull-request layer and independent traceability data. Its release preserves structural, provenance, and quality metadata while documenting scope boundaries and the limits of co-change inference.
- Census construction: Adaptive size partitioning and retry logic address GitHub’s 1,000-result cap and silently short pages, with idempotence and determinism checks.The partition plans ship as audit trails.
- Census construction: 822,901 rows reduce to 575,633 distinct files and 470,795 retained specs after auditable cache, vendor, and pre-2024 filters.Filtered rows retain the firing reason so decisions can be audited or reversed.
- Corpus scope: The filename census intentionally mixes serious projects with tutorials, template clones, and experiments, so studies should report their chosen maturity filters.Only 923 of 73,030 repositories have at least 100 stars; fork, template, archived, content-hash, and scaffold flags support filtering.
- Inference limits: Co-change treats PRs modifying both specs and source files as implementations, but the paper identifies this as an assumption rather than ground truth.81.2% of spec-touching PRs also modify code; later-PR implementations and unrelated spec edits can evade or confound the heuristic.
- Released data: The release parses each spec into 39 structural and quality features and provides typed links among code, sibling documents, PRs, references, branches, and issues.The schema includes repository/file provenance, language and content-family fields, requirement markers, and completeness indicators.
APPENDIX E: FULL TOOL ATTRIBUTION
SPECMINE v1.0 attributes kept specifications across named tools, interpretable path buckets, and a separate Kiro census. The attribution table distinguishes tool categories while identifying caffeine.ai as the repository-count leader and Kiro, Spec Kit, and OpenSpec as developer-adopted leaders.
- Attribution categories: Table III lists all 17 named tools, four interpretable path buckets, and the separate Kiro census with file and repository counts.The path buckets group unattributed files rather than representing tools.
- Attribution results: caffeine.ai leads by repository count, while Kiro, Spec Kit, and OpenSpec lead among developer-adopted tools.The paper qualifies caffeine.ai’s repositories as largely auto-generated single apps.