PaperAtlas: an automatically constructed atlas of computational methods and software from 6.4 million open-access articles

Affiliations

• The authors list Cornell University in Ithaca, NY, and Johns Hopkins University in Baltimore, MD, as affiliations.The passage also lists ORCID identifiers for Neeti Shah, Anagha Vippagunta, and Yash Bhargava.Source: §1 · p. 1

Introduction

• Computational methods are increasingly difficult to collect and organize because they are dispersed across subfields, inconsistently named, and often absent from paper titles that foreground biological results.Existing retrieval systems rank papers by query relevance.Source: Introduction · p. 1
• PaperAtlas uses large language models to identify papers where computation is a contribution, extract artifacts, and organize papers into navigable topics.The introduction distinguishes this approach from literature resources focused on extracting software mentions.Source: Introduction · pp. 1–2
• The project asks what computational methods and software are introduced, how they are distributed across topics, and how often their artifacts match registry records.It also asks how many registered tools an automated LLM pipeline recovers.Source: Introduction · p. 2

Corpus acquisition

• 6,446,741 PMC articles with titles and abstracts were screened; computational scope meant computation was a central scientific result, rather than routine statistical analysis.The corpus excluded methods published only in subscription journals; screening used Qwen2.5-32B-Instruct, and bge-base was selected as the smaller compared encoder for clustering.Source: Corpus acquisition · p. 2, Model and encoder selection · p. 2

Structured extraction

• Computational papers underwent schema-constrained extraction of artifact type and name, biological questions, related tools, databases and a summary from titles, abstracts and limited body text.Qwen2.5-7B-Instruct performed extraction; other released models handled screening, consistency checks and cluster naming, while Qwen3-8B classified biomedical clusters.Source: Structured extraction · pp. 2–3, Language models and AI assistance · p. 5

Compiling the atlas & restriction to biomedical topics

• 267,893 algorithm, software-package and web-server papers were clustered; model papers were excluded, and database and pipeline papers were analyzed separately.Clustering used bge-base embeddings, UMAP and HDBSCAN; unassigned papers joined a nearest cluster only at cosine similarity of at least 0.80, and parent topics were retained when most papers were in biomedical clusters.Source: Compiling the atlas & restriction to… · p. 3

Validation of the categories

• Each atlas cluster’s semantic coherence, separation, label agreement and stability were assessed using word intrusion, clustering metrics, an independent language model and repeated clustering runs.External validation compared cluster annotations with curator-assigned EDAM topics and operations; among clusters with at least five entries, concentration was compared with 1,000 permutations.Source: Validation of the categories · pp. 3–4
• 84.3% agreement on artifact type and 94.9% on artifact name were measured in a 300-paper consistency check, not as accuracy estimates.A separate human audit used stratified samples across screening, artifact type and name, unmatched names, and name matches, with weighted estimates and bootstrap intervals.Source: Validation of the categories · p. 4

Registry comparison

• Registry comparisons applied name filters and strict matching across bio.tools and four package ecosystems, while identity-resolved bio.tools matching also used publications and links.The software-package and web-server subset was selected for specificity, but its denominator inherits artifact-classification error.Source: Registry comparison · p. 4

Recall against bio.tools

• End-to-end recall used all 12,207 software-subset entries, while conditional recall used only entries whose defining publication reached the retained atlas.The former includes losses at screening, extraction, clustering, topic restriction, name recovery and matching; estimates use Wilson 95% confidence intervals and were stratified by publication and tool characteristics.Source: Recall against bio.tools · p. 5

Retrieval layer and structured store

• 82.7% of name-masked bio.tools queries ranked the defining paper first and 93.7% ranked it in the top ten.The retrieval benchmark used curated descriptions of 300 tools; the retrieval layer was not used in the primary analyses.Source: Retrieval layer and structured store · p. 5

Corpus and screening

• 6,446,741 title-and-abstract records were screened: 1,074,191 (16.66%) were classified computational, 5,371,134 (83.32%) experimental, and 1,416 (0.022%) had no valid label.The computational share rose from 13.5% of papers published in 2012 to 20.0% in 2025.Source: Corpus and screening · p. 6

Structured extraction

• In a 1,000-paper comparison, long-context extraction returned names for 48.1% versus 30.2% with the production window, but which window was more accurate was not measured.Artifact type agreed in 73.7% of cases between windows, compared with 97.3% between repeated fixed-window extractions.Source: Structured extraction · p. 6

The atlas and its validation

• The atlas grouped 267,893 tool-type papers into 1,438 clusters under 12 parent topics, with 40,122 papers unassigned; parent-level restriction retained 1,000 clusters and 165,432 papers.The partition had weak geometric separation, while word-intrusion accuracy was 0.875; cluster counts also varied substantially with HDBSCAN parameters.Source: The atlas and its validation · p. 6
• Among 296 retained clusters with at least five linked bio.tools entries, the most frequent EDAM topic averaged 61.7% concentration, versus 26.4% under permutation.This validation covered a subset; the remaining 704 retained clusters were smaller and more recent, and differed in topic composition.Source: The atlas and its validation · p. 7

Artifact names contributed by the literature

• The atlas contained 121,844 distinct artifact name strings across 129,169 naming events, and 95.8% of names were contributed by a single paper.The names included 92,110 new algorithms, 25,241 software packages, and 6,179 web servers.Source: Artifact names contributed by the… · p. 7

Registry matching in the core software subset

• 61.0% of 31,180 distinct software-package and web-server names had no strict match in any of five registries, and 59.6% remained unmatched after identity resolution.Among all 121,844 atlas names, 77.0% were unmatched by strict matching, but this all-name figure is not an estimate of registry coverage because nearly two thirds came from algorithms with no detected software evidence.Source: Registry matching in the core software… · pp. 7–8
• 56.6% of 12,207 bio.tools entries in the software subset were recovered by strict matching; recall rose to 83.6% among the 8,265 entries whose defining paper entered the atlas.The largest end-to-end losses were papers extracted as non-clustered artifact types (16.5%) and papers left unassigned by HDBSCAN (14.2%).Source: Registry matching in the core software… · p. 8

Human audit

• The human audit estimated screening sensitivity at 92.9%; 55.1% of 50 name-based registry matches referred to the same artifact, with small samples and one annotator limiting the audit.The structured resource index recorded 499,531 dataset-accession occurrences and 152,097 code-link occurrences; 385 of 400 sampled repository links resolved, which verifies accessibility but not correspondence to the reported artifact.Source: Human audit · p. 9, Structured resource index · p. 9

• PaperAtlas organizes computational-methods papers for task-oriented browsing and links them to reported dataset accessions and code repositories.Source: Discussion · p. 9
• Name-based registry matching has limits: strict-match recall was 15.4% when a registry name did not appear in the defining paper’s title or abstract.Source: Discussion · p. 9
• Mention-based resources, curated registries, and PaperAtlas capture complementary views of the computational-methods literature.Source: Discussion · p. 9

• 165,432 computational-methods papers were organized into 1,000 biomedical clusters.The atlas provides a reproducible structure for navigating the literature.Source: Conclusion · p. 9
• Where curated bio.tools annotations permitted evaluation, clusters showed substantial concentration of shared EDAM topics.Source: Conclusion · p. 9
• 61.0% of 31,180 distinct software-package and web-server names had no strict match in the five evaluated registry snapshots.Source: Conclusion · p. 9
• 56.6% of bio.tools software entries were recovered end to end, rising to 83.6% when their defining publication entered the atlas.A small manual audit found artifact names accurate in examined cases but registry name matching least reliable.Source: Conclusion · p. 10

• Supplementary material accompanies the article.Source: Supplementary material · p. 10

• The research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.Source: Funding · p. 10

• The authors declare no competing interests.Source: Conflict of interest · p. 10

Data availability

• The pipeline code, release files, analysis materials, atlas, embeddings, registry snapshots and per-paper records are openly available.The passage index and parsed corpus are available on request; verbatim text is distributed only where the source licence permits.Source: Data availability · p. 10
• The September 2026 release is frozen in the archive, while monthly additions of preprints can make current web-resource counts differ.Source: Data availability · p. 10

Tables

• Table 1 covers corpus acquisition, screening and extraction.Source: Table 1 · p. 11
• Table 2 reports row percentages for registry matches across 121,844 distinct artifact names, including strict and identity-resolved matches and names with no registry match.Its registry categories include bio.tools, package registries, and reported code links.Source: Table 2 · p. 11
• Table 3 reports recall for 12,207 bio.tools entries with a qualifying tool type and defining paper in the screened corpus.Its rows use different denominators, with entries whose paper entered the atlas conditional on successful completion of preceding pipeline stages.Source: Table 3 · p. 12, Tables · p. 12

Figures

• Figure 1 presents the workflow, acquisition and screening funnel, publication-year distribution, and screening outcome.The funnel runs from 7,283,594 parsed articles to 165,432 retained under the parent-level biomedical rule.Source: Figure 1 · p. 16
• Figure 2 shows retained-paper embeddings by parent topic, cluster sizes, EDAM-topic concentration against permutations, and validation measures.The validation measures do not use human labels.Source: Figure 2 · p. 17
• Figure 3 compares exact and alias matches across five registries and traces stage-by-stage losses for 12,207 bio.tools entries.Its second panel distinguishes entries recovered by strict matching from additional entries recovered through identity resolution.Source: Figure 3 · p. 18

Supplementary tables

• The screening prompt classified papers as computational or non-computational using only the permitted outputs C and E; extraction used a constrained prompt and enforced JSON schema.The extraction prompt supplied the title, up to 1,600 abstract characters, and the opening 2,296 body tokens; schema-invalid records were unreachable during sampling.Source: Table S1 · p. 18, Supplementary tables · pp. 18–19, Table S2
• Retained parent topics included Biological Data Analysis, whose 430 clusters contained 78,609 papers and were 99.5% biomedical by cluster judgement, while Computational Materials Science was not retained.The retention rule required more than half of a parent’s papers to lie in clusters judged biomedical.Source: Table S3 · p. 19, Supplementary tables · p. 20
• 36 of 36 sampled licence assignments agreed with PMC’s licence field; the corpus contained 831,601 commercial, 203,470 noncommercial, and 39,120 restricted papers.The licence pass assigned every paper to a class, with no undetermined category.Source: Table S7 · p. 21, Supplementary tables · p. 22
• 30,913,457 passages were indexed, including one profile chunk per paper; binary quantization achieved 98.80% Recall@100 using ~4 GB of resident vectors.The comparison used 40 probe queries against a 120,000-vector sample; pass int8 achieved 99.55% Recall@100 using ~31 GB.Source: Supplementary tables · pp. 22–23
• 41,890,329 references were parsed, with 5,767,772 resolving to another corpus paper; the companion store also recorded 597,117 tool relationships and 77,382 distinct code repositories.It contained 285,615 dataset-accession occurrences across 184,701 papers, 596,496 tool mentions, and 592,614 database-use records.Source: Supplementary tables · p. 23
• 56.6% strict recovery across 12,207 bio.tools software entries rose to 83.6% among 8,265 entries whose defining paper entered the atlas.Without clustering, strict recovery was 81.3% across the 12,207 entries.Source: Table S11 · p. 24, Supplementary tables · p. 26
• 72.2% of retained clusters had stability at or above 0.5, while median best-match Jaccard across five other UMAP seeds was 0.675.DBCV was negative for every partition, consistent with overlapping clusters; the reference assignment was reproduced exactly at reassignment threshold 0.80.Source: Supplementary tables · pp. 30–32, Table S13 · p. 30
• 18,251 of 18,302 production failures became valid records after reruns and were included in every analysis.The reruns used the production model, prompt, schema, and input window with a 1,536-token output cap.Source: Table S14 · p. 34, Supplementary tables · p. 35
• DOI-joined citation edges agreed with PMID links in 97.3% of cases, PMID-joined edges in 98.2%, and 294 of 298 sampled edges agreed with Crossref or PubMed references (98.7%).The validation checked all edges for agreement between join identifiers and sampled edges against publisher-deposited Crossref reference lists and PubMed links.Source: Table S15 · p. 35, Supplementary tables · p. 36
• 13.7% of 41,890,329 parsed reference entries resolved to a paper in the corpus, while 86.3% carried a well-formed identifier for a paper outside it.The remaining entries had malformed identifiers (558; 0.0%); none lacked an identifier.Source: Supplementary tables · p. 37
• The duplicate check identified 3,892 groups of records sharing normalized titles under different DOIs, comprising 7,860 records; 1,229 groups contained more than one tool-type record, and 3,316 contained a preprint-server DOI.Source: Supplementary tables · p. 37
• BM25 + dense (RRF) retrieval yielded Recall@1 82.7 (78.3-87.0), Recall@10 92.7 (89.7-95.7), and MRR@10 0.865 (0.829-0.900), versus dense retrieval’s 75.3 (70.7-80.0), 90.3 (87.0-93.3), and 0.808 (0.769-0.846).The known-item test used 300 curated bio.tools descriptions; it tests retrieval of a described tool, not topical search.Source: Supplementary tables · pp. 37–38