Source-linked AI summary
CORD-19: The COVID-19 Open Research Dataset
Lucy Lu Wang, Kyle Lo, Yoganand Chandrasekhar, Russell Reas, Jiangjiang Yang, Doug Burdick, Darrin Eide, Kathryn Funk, Yannis Katsis, Rodney Kinney, Yunyao Li, Ziyang Liu, William Merrill, Paul Mooney, Dewey Murdick, Devvret Rishi, Jerry Sheehan, Zhihong Shen, Brandon Stilson, Alex Wade, Kuansan Wang, Nancy Xin Ru Wang, Chris Wilhelm, Boya Xie, Douglas Raymond, Daniel S. Weld, Oren Etzioni, Sebastian Kohlmeier
TL;DR
Researchers needed computational access to rapidly expanding COVID-19 literature and related coronavirus research. The paper describes CORD-19’s construction, uses, and shared tasks, reporting broad adoption while identifying remaining coverage and research challenges.
Problem
Several hundred new COVID-19 papers were being published daily, creating a need for automated methods to analyze and synthesize information.
Method
CORD-19 ingests papers and preprints from multiple sources through Semantic Scholar, harmonizes and deduplicates metadata, extracts structured full text, and supports research and shared tasks.
Results
CORD-19 was downloaded over 200K times in three months and used for systematic reviews, search and extraction tools, and successful shared tasks.
Takeaways & Limitations
CORD-19 provides computational access to scientific papers and serves as a template for using machine learning to advance scientific research.
Takeaways & Limitations
CORD-19 excludes many relevant document types and remains incomplete within scientific papers, including limited representation of some domains such as the social sciences.
Abstract
from arXiv · showhide
The COVID-19 Open Research Dataset (CORD-19) is a growing resource of scientific papers on COVID-19 and related historical coronavirus research. CORD-19 is designed to facilitate the development of text mining and information retrieval systems over its rich collection of metadata and structured full text papers. Since its release, CORD-19 has been downloaded over 200K times and has served as the basis of many COVID-19 text mining and discovery systems. In this article, we describe the mechanics of dataset construction, highlighting challenges and key design decisions, provide an overview of how CORD-19 has been used, and describe several shared tasks built around the dataset. We hope this resource will continue to bring together the computing community, biomedical experts, and policy makers in the search for effective treatments and management policies for COVID-19.
1 Introduction
CORD-19 is a large, growing dataset of COVID-19 and related coronavirus publications that combines harmonized metadata with extracted full text. It connects computational, biomedical, and policy communities and has already supported substantial research use.
- CORD-19 began with 28K papers and grew to more than 140K papers within weeks, while more than 50% had full text.The dataset is updated regularly with new sources and publications.
- Papers and preprints are collected through Semantic Scholar, then metadata are harmonized and deduplicated and documents are processed to extract full text.
- CORD-19 aims to help machine learning researchers, biomedical experts, and policy makers find information relevant to COVID-19 treatments and management policies more quickly.
- The dataset had been downloaded over 200K times in three months and was used for systematic reviews, search and extraction tools, and shared tasks.
- The article covers CORD-19’s content and creation, design challenges, research and shared tasks, and future roadmap.
2 Dataset
CORD-19 integrates papers and preprints from multiple sources, harmonizes and deduplicates metadata, and produces structured full-text parses. The dataset grew rapidly and covers diverse biomedical research, with substantial but incomplete document and table coverage.
- Data integration: CORD-19 integrates papers and preprints from several sources using bibliographic metadata and persistent identifiers.Sources include PMC, PubMed, WHO, bioRxiv, medRxiv, and arXiv.
- Data integration: Papers are included when COVID-19 or coronavirus-related keywords match their title, abstract, or body text.The retrieval query includes terms such as COVID-19, Coronavirus, SARS-CoV, and MERS-CoV.
- Metadata processing: Metadata are clustered by shared identifiers, assigned persistent CORD UIDs, canonicalized, and filtered to remove unwanted entries.Conflicting identifiers prevent clustering, while canonical entries prioritize document availability and permissive licensing.
- Full-text processing: Structured full text is generated from PDFs and JATS XML in the S2ORC JSON format through separate parsing pipelines.PDFs are processed through TEI XML and S2ORC JSON, while PMC JATS XML uses a custom parser producing the same target format.
- Full-text processing: 48% of CORD-19 papers have PDF parses, 37% have XML parses, and more than 90% of PDFs are successfully parsed.Around 2.6% of papers have multiple PDF SHA associations.
- Dataset contents: The table pipeline extracted 188K tables from 54K documents and matched 33K tables in 19K full-text documents.Matching uses a Jaccard similarity threshold above 0.9 for table captions.
- Dataset contents: CORD-19 contains over 140K papers and over 72K full texts, while 47K papers and 7K preprints released since 2020 comprise nearly 40% of the dataset.The corpus is predominantly Medicine (55%), Biology (31%), and Chemistry (3%), which together constitute almost 90%.
3 Design decision & challenges
CORD-19 was designed for continual updates, multi-source integration, clean canonical metadata, machine-readable full text, and copyright compliance. These requirements support information retrieval and extraction while constraining redistribution and full-text availability.
- Up-to-date: CORD-19 has been updated daily since May 26, requiring reproducible processing, persistent identifiers, and release-compatible changes.The pipeline must produce consistent results as hundreds of COVID-19 publications appear each day.
- Handles data from multiple sources: Integrating multiple sources requires converting differing metadata formats, handling missing or extraneous fields, and supporting new sources.The dataset combines papers from sources with distinct metadata conventions.
- Clean canonical metadata: CORD-19 uses conservative deduplication, combining entries only when identifiers match without conflicts because retaining duplicates is less harmful than removing unique documents.This policy prioritizes recall of potentially useful papers over maximal deduplication.
- Machine readable full text: S2ORC JSON preserves paper structures such as paragraph breaks, section headers, inline references, and citations for NLP processing.The schema also supports character-level indices for annotating entities or spans.
- Observes copyright restrictions: Copyright licenses differ across papers, and some allow reading while restricting republication or commercial redistribution.CORD-19 therefore passes best-to-our-knowledge licensing information to end users.
- Machine readable full text: An example system retrieves relevant papers for a query and extracts supporting snippets from their full-text JSONs.The workflow combines information retrieval with evidence extraction.
4 Research directions
CORD-19 has supported clinical review, clinician-facing tools, biomedical text-mining resources, and shared tasks that organize retrieval and extraction research. These efforts address challenges including finding relevant literature, extracting information, and synthesizing knowledge.
- Direct usage by clinicians and clinical researchers: Clinical researchers use CORD-19 to conduct systematic reviews covering infection and mortality, symptoms, drug repurposing, management policies, and disease interactions.
- Direct usage by clinicians and clinical researchers: Clinicians and researchers face challenges in staying current, identifying useful historical coronavirus papers, extracting information, and synthesizing literature.
- Tools and systems to assist clinicians: Dozens of CORD-19 tools combine text-based information retrieval and extraction to address these clinical and research challenges.
- Research to support further text mining and NLP research: NLP resources built on CORD-19 support entity recognition and linking, sentence extraction, pretrained models, embeddings, and knowledge-graph construction.
- Shared tasks and competitions: Kaggle's CORD-19 Research Challenge used open-ended scientific questions initially, then shifted to completing structured summary tables after medical-expert feedback.
- Shared tasks and competitions: TREC-COVID evaluates ranking systems across multiple rounds with changing topics and CORD-19 versions; around 60 medical experts provide gold rankings, and over 55 teams participated.
5 Discussion
CORD-19 has enabled computational discovery across COVID-19 literature while exposing important gaps in coverage, access, parsing, metadata, and human-centered evaluation. The authors outline ongoing work needed to make the resource more complete and useful.
- 5 Discussion: Several hundred COVID-19 papers appear daily, creating demand for automated analysis, expert vetting, and user-oriented search systems.The authors emphasize that computational methods must incorporate human judgments and serve concrete users.
- 5 Discussion: Future work must determine effective discovery methods, expert-curation strategies, and which extracted findings translate into treatments or management policies.The authors identify these as unresolved questions despite successful engagement with the dataset.
- 5.1 Limitations: CORD-19 excludes many relevant document types and remains incomplete across scientific sources, social sciences, and foreign-language literature.Technical reports, white papers, governmental publications, some domains, and especially early Chinese-language papers are underrepresented.
- 5.1 Limitations: PDF-based scientific text mining requires substantial processing because PDFs mix visual and semantic information and vary in structure.The paper calls for improved scientific PDF parsing and structured formats such as JSON, XML, or HTML.
- 5.2 Call to action: Open-access licensing limits the addition of relevant COVID-19 and related scientific content not already included in CORD-19.Securing release rights for additional papers is identified as a significant part of future work.
- 5 Discussion: CORD-19 supports machine learning, biomedical, and policy communities by making literature computationally accessible for discovery.The dataset is intended to connect complementary expertise and support information retrieval and text analysis.
A Table parsing results
The table-parsing results illustrate both the diversity of scientific table layouts and the resulting extraction challenges. Most structure is preserved, but some common parsing errors remain.
- A Table parsing results: Table 3 presents sample parses spanning diverse scientific table representations.The examples show how varied table structures appear across documents.
- A Table parsing results: The examples expose both the resulting parse structure and common errors for evaluating table extraction quality.This makes the sample useful for inspecting what the pipeline retains and where it fails.
- A Table parsing results: Most table structure is preserved accurately, but representation diversity produces parsing errors.The caption identifies errors as a consequence of differing table formats.