Source-linked AI summary
Building a PubMed knowledge graph
Jian Xu, Sunkyu Kim, Min Song, Minbyul Jeong, Donghyeon Kim, Jaewoo Kang, Justin F. Rousseau, Xin Li, Weijia Xu, Vetle I. Torvik, Yi Bu, Chongyan Chen, Islam Akef Ebeid, Daifeng Li, Ying Ding
TL;DR
Useful biomedical concepts in PubMed are difficult to extract and disambiguate, hindering literature search and collaborator discovery. The authors construct a PubMed knowledge graph by extracting and normalizing bio-entities, disambiguating authors, and integrating funding, affiliation, and educational data from multiple sources. BioBERT outperformed state-of-the-art models by 0.51% in F1 score on average, while author name disambiguation achieved 98.09% F1.
Problem
Useful biomedical concepts in PubMed are difficult to extract and disambiguate, making literature searches and collaborator discovery arduous.
Method
The authors construct a PubMed knowledge graph from 29 million abstracts by extracting and normalizing bio-entities, disambiguating authors, and integrating multi-source funding, affiliation, and educational data.
Results
BioBERT outperformed state-of-the-art models by 0.51% in F1 score on average, while author name disambiguation achieved 98.09% F1.
Takeaways & Limitations
PKG connects bio-entities, authors, articles, affiliations, and funding to provide a comprehensive overview of PubMed data.
Takeaways & Limitations
Updating PKG is complex because it depends on updates from multiple data sources and requires significant computation.
Abstract
from arXiv · showhide
PubMed is an essential resource for the medical domain, but useful concepts are either difficult to extract or are ambiguated, which has significantly hindered knowledge discovery. To address this issue, we constructed a PubMed knowledge graph (PKG) by extracting bio-entities from 29 million PubMed abstracts, disambiguating author names, integrating funding data through the National Institutes of Health (NIH) ExPORTER, collecting affiliation history and educational background of authors from ORCID, and identifying fine-grained affiliation data from MapAffil. Through the integration of the credible multi-source data, we could create connections among the bio-entities, authors, articles, affiliations, and funding. Data validation revealed that the BioBERT deep learning method of bio-entity extraction significantly outperformed the state-of-the-art models based on the F1 score (by 0.51%), with the author name disambiguation (AND) achieving a F1 score of 98.09%. PKG can trigger broader innovations, not only enabling us to measure scholarly impact, knowledge usage, and knowledge transfer, but also assisting us in profiling authors and organizations based on their connections with bio-entities. The PKG is freely available on Figshare (https://figshare.com/s/6327a55355fc2c99f3a2, simplified version that exclude PubMed raw data) and TACC website (http://er.tacc.utexas.edu/datasets/ped, full version).
Background and Summary
The study builds the PubMed Knowledge Graph by integrating biomedical entities, disambiguated authors, funding, and affiliation information from multiple sources. PKG is designed to support searches, collaboration discovery, scholarly-impact analysis, and author profiling.
- Conventional text-mining tools make biomedical literature searches onerous, while identifying researchers studying common entities and potential collaborators remains difficult.
- Previous open-access datasets addressed bio-entity recognition and selected PubMed entities or relationships, but the study aimed to integrate a more comprehensive dataset.
- The integration framework combines bio-entity extraction with connections among authors, ORCID, and funding information.
- BioBERT extracted bio-entities from 29 million PubMed abstracts, and the integrated dataset combined Author-ity and Semantic Scholar author-disambiguation data with NIH, ORCID, and MapAffil information.
- PKG supports applications including finding experts, searching bio-entities, analyzing scholarly impacts, and profiling scientists’ careers.
Methods
The PKG extracts and normalizes biomedical entities, disambiguates authors, and integrates funding, affiliation, and educational data from multiple sources.
- Bio-Entity Extraction: BioBERT-based NER extracts biomedical entities from PubMed abstracts, while multi-type normalization assigns unique IDs to synonymous or polysemous entities.The extraction component combines entity recognition with normalization across multiple entity types.
- Bio-Entity Extraction: WordPiece tokenization and BioBERT fine-tuning support recognition of diverse biomedical entities and seven sequence tags.BioBERT is initialized from case-sensitive BERT, pretrained on PubMed and PubMed Central articles, and fine-tuned for NER.
- Author Name Disambiguation: Automated author name disambiguation is used for PubMed because it can provide high-quality results with complete dataset coverage.The study integrates Author-ity and Semantic Scholar datasets, which offer complementary accuracy and coverage.
- Author Name Disambiguation: Author-ity supplies the primary author identifiers, while Semantic Scholar supplements authors after 2009 because Author-ity covers only earlier PubMed papers.Author-ity has an F1 score of 98.16%, whereas Semantic Scholar has an F1 score of 96.94% and broader author coverage.
- Extended Multi-source Information Integration: PKG integration links PubMed authors and articles with NIH funding, ORCID employment and education histories, and MapAffil fine-grained affiliation data.The integration uses mappings between AND_ID and PMID to connect entities, authors, articles, affiliations, and funding.
Data Records
PKG is distributed as linked data records covering authors, bio-entities, affiliations, researcher histories, education, and NIH projects. The dataset documentation specifies CSV files, field descriptions, and limitations on author disambiguation and future updates.
- PKG is provided on Figshare as seven CSV files covering authors, bio-entities, affiliations, employment, education, and NIH projects.PubMed raw data are excluded from Figshare because of their size; the TACC download includes both raw PubMed data and PKG.
- The seven files are Author_List, Bio_entities_Main, Bio_entities_Mutation, Affiliations, Researcher_Employment, Researcher_Education, and NIH_Projects.
- NIH_Projects contains 12,340,431 project records, 1,790,949 publication records, and 102,070 mapping records.The file maps PI_ID, PMID, and AND_ID using NIH ExPORTER data.
- About 1.3 million author instances, or 1.15%, could not be disambiguated and therefore received AND_ID values of zero.
- Fields with the same names use consistent formats across tables, enabling links across the dataset files.Tables 5–11 provide field names, formats, and short descriptions for the listed data files.
- Updating PKG is complex because it depends on updates from multiple data sources and requires significant computation.The authors hope to refresh PKG quarterly and may develop an integrative ontology in the future.
Technical Validation
Technical validation evaluated biomedical named-entity recognition, multi-type normalization, and author name disambiguation using precision, recall, and F1-based comparisons. BioBERT improved average F1 over state-of-the-art models, while the integrated author-disambiguation result exceeded Semantic Scholar but may be slightly overestimated.
- Biomedical Named-Entity Recognition: BioBERT achieved the highest F1 scores for Genes/Proteins, Diseases, and Drugs/Chemicals, outperforming state-of-the-art models by 0.51% on average.BERT was also highly effective, but state-of-the-art models exceeded general-domain BERT by 2.28% on average.
- Multi-Type Entity Normalization: Several normalization and comparison results are incomplete because some systems reported only selected metrics or omitted independent Drug/Chemical normalization performance.The reported gaps affect comparisons involving GNormPlus, tmVar 2.0, the sieve-based model, SR4GN, and tmChem.
- Multi-Type Entity Normalization: Normalization performance varied across entity types and datasets, with GNormPlus achieving an F1 score of 36.6% on the multispecies BC3GN test set.GNormPlus focused on seven of the 75 species represented in BC3GN, whereas tmVar 2.0 achieved F1 scores close to 90% on two mutation corpora.
- Author Name Disambiguation: NIH ExPORTER supplied high-fidelity PI_IDs and article PMIDs that connected funded researchers with disambiguated author IDs for evaluation.The evaluation used PI_ID records, projects, grant-acknowledging articles, and an author-name crosswalk.
- Author Name Disambiguation: The integrated Author-ity and Semantic Scholar result outperformed Semantic Scholar by 1.15% in F1 and covered authors through 2018, beyond Author-ity’s 2009 coverage.Table 14 reports precision, recall, and F1 for both component datasets and the integrated result.
- Author Name Disambiguation: Author-disambiguation evaluation may be slightly overestimated because NIH principal investigators generally have richer longitudinal publication metadata than newer or less-published authors.Approximately 1.15% of author instances were absent from both integrated source datasets, further theoretically reducing performance.
Usage Notes
The PKG supports researcher-, bio-entity-, and network-centric analyses by linking extracted entities with authors and publications. These views expose research trends, collaboration patterns, and related author or entity neighborhoods.
- Usage Notes: PKG profiling addresses the difficulty of identifying researchers focused on a specific bio-entity across institutions.Unlike keyword searches that return many related articles, bio-entity profiling directly connects entities with disambiguated authors.
- Usage Notes: The use cases examined activity trends for Stephen Silberstein, CGRP, and coronavirus through researcher-centric, bio-entity-centric, and bipartite network analyses.The analyses used PKG-linked publications and extracted bio-entities to study people, entities, and author-entity relationships.
- Researcher-Centric Activity: Silberstein’s profile contained 539 bio-entities from 455 articles, including 342 diseases, 142 drugs, 24 genes, 17 species, and 14 mutations.Headache and migraine each peaked in 2004, while CGRP began appearing in his publications in 2015.
- Bio-Entity-Centric Activity: CGRP research expanded from 13 articles in 1982 to 1,209 in 1991 and 1,517 in 2018, with author counts following a similar trend.The dataset contained 7,877 CGRP articles by 32,392 authors dating back to 1982.
- Bipartite Author-Entity Network: The coronavirus network contained 28,223 disambiguated authors and 5,379 bio-entities, with edges representing author-entity research links.Author-entity edge weights counted papers mentioning an entity, while projection edges counted common neighbors; vertex colors marked communities.
- Bipartite Author-Entity Network: The coronavirus projections reveal both repeated collaborations and similar research interests, while bio-entity connections can expose hidden relationships.Chan Kwok Hung and Yuen Kwok-Yung published 69 papers together, whereas Baric R S and Yuen Kwok-Yung were connected through shared interests without necessarily collaborating.