Source-linked AI summary
ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing
Mark Neumann, Daniel King, Iz Beltagy, Waleed Ammar
TL;DR
Biomedical NLP faces information overload and limited robust, practical public models under domain shift. This paper introduces scispaCy, a spaCy-based toolkit with biomedical models, datasets, and pipelines, and reports robust, near-state-of-the-art performance with competitive speed. Its runtime comparison is not fully apples-to-apples, and it is slower than pipelines designed purely for production use.
Problem
Biomedical NLP has few robust, practical, publicly available models, while domain shift and rapidly growing biomedical literature create processing challenges.
Method
scispaCy builds on spaCy, retrains models for biomedical POS tagging, dependency parsing, and NER, adds tokenization rules, and releases datasets and processing pipelines.
Results
The released scispaCy models achieve close to state-of-the-art performance and competitive speed across biomedical NLP tasks and datasets.
Takeaways & Limitations
scispaCy provides fast, scalable biomedical NLP pipelines intended to support biomedical information extraction and downstream use of syntactic annotation.
Takeaways & Limitations
The runtime comparison is not apples-to-apples, and scispaCy is slower than pipelines designed purely for production use-cases.
Abstract
from arXiv · showhide
Despite recent advances in natural language processing, many statistical models for processing text perform extremely poorly under domain shift. Processing biomedical and clinical text is a critically important application area of natural language processing, for which there are few robust, practical, publicly available models. This paper describes scispaCy, a new tool for practical biomedical/scientific text processing, which heavily leverages the spaCy library. We detail the performance of two packages of models released in scispaCy and demonstrate their robustness on several tasks and datasets. Models and code are available at https://allenai.github.io/scispacy/
1 Introduction
scispaCy addresses biomedical information overload by providing a specialized NLP library built on spaCy. It releases reformatted GENIA data, benchmarks NER models, and evaluates fast biomedical processing pipelines.
- Biomedical publication growth creates information overload involving diverse genes, drugs, and proteins, motivating automated scientific knowledge extraction.
- scispaCy is a specialized biomedical text-processing library built on the spaCy library.
- The paper evaluates scispaCy on POS tagging, dependency parsing, named entity recognition, and sentence segmentation against state-of-the-art models.
- The authors release GENIA 1.0 reformatted into Universal Dependencies v1.0 and aligned with original PubMed abstract text.
- The paper benchmarks 9 named entity recognition models for specific entity extraction applications against strong baselines.
- The released biomedical pipelines include tokenization, part of speech tagging, dependency parsing, and named entity recognition.
2 Overview of (sci)spaCy
scispaCy extends spaCy into practical biomedical processing through retrained models and biomedical tokenization rules. Its released pipelines offer competitive speed and straightforward Python integration, while sacrificing some production-level speed and facing comparison limitations.
- scispaCy builds on spaCy because spaCy provides practical, fast, robust NLP tools with close to state-of-the-art performance.
- The library retrains spaCy models for POS tagging, dependency parsing, and NER on biomedical datasets and adds biomedical tokenization rules.
- The two core packages differ in vocabulary size and word-vector availability: en_core_sci_md has a larger vocabulary and vectors, while en_core_sci_sm does not.
- Both scispaCy models have competitive speed with publicly available biomedical pipelines written in C++ and Java.
- scispaCy is slower than pipelines designed purely for production use, but integrates straightforwardly with Python machine-learning and text-processing libraries.
3 POS Tagging and Dependency Parsing
scispaCy trains joint POS-tagging and dependency-parsing models on biomedical and general-domain data, releasing aligned Universal Dependencies resources. The models are competitive while offering substantially faster dependency parsing and improved robustness to web data.
- Model and data: The joint POS-tagging and dependency-parsing model is an arc-eager transition-based parser trained with a dynamic oracle and shared CNN token features.Both released models use GENIA-derived Universal Dependencies with POS annotations, while OntoNotes data broadens robustness to generic text.
- Model and data: The paper releases GENIA 1.0 converted to Universal Dependencies v1.0, aligned to raw PubMed abstract spans with restored PubMed metadata.The release includes metadata such as linked entities, author information, publication dates, citation statistics, and journal information.
- Evaluation: Both scispaCy models achieve competitive POS-tagging performance with equivalent practical value to state-of-the-art systems.The comparison uses POS-tagging results on the GENIA test set.
- Evaluation: The Biaffine parser outperforms scispaCy by 2-3% on dependency parsing, while scispaCy is approximately 9x faster.This speed advantage is attributed to spaCy’s speed optimizations.
- Robustness: Adding increasing percentages of web data substantially improves OntoNotes performance without reducing biomedical performance.Figure 2 reports unlabeled attachment score for en core sci md, averaged over 3 random seeds.
4 Named Entity Recognition
scispaCy provides broad biomedical NER through a general mention detector and additional entity-type-specific models. Across diverse datasets, its models provide competitive baselines, while the shared detector retains useful recall across domains.
- Models: The main NER model is trained on MedMentions mention spans and recognizes varied entity types and non-standard syntactic phrases without predicting entity types.Additional packages provide finer-grained NER models trained on BC5CDR, CRAFT, JNLPBA, and BioNLP13CG.
- Evaluation: scispaCy evaluates NER across nine datasets spanning diverse biomedical entity types, dataset sizes, and annotation schemes.The datasets cover applications including cancer genetics, disease-drug interactions, pathway analysis, and trial population extraction.
- Results: The scispaCy NER models are competitive baselines for 5 of the 9 specialist datasets against strong baselines.Those baselines do not use multi-task learning across datasets or semi-supervised learning with large pretrained language models.
- Results: The MedMentions-trained mention detector shows a modest average-recall drop on specialist datasets while maintaining recall across their diverse domains.The authors characterize this cross-dataset recall as nontrivial, and Table 6 reports recall on the nine specialist test sets.
5 Candidate Generation for Entity Linking
scispaCy’s entity-linking groundwork uses a reduced UMLS subset and approximate nearest-neighbour candidate generation, with abbreviation handling improving recall. The generator achieves higher recall than Murty et al. (2018) while producing fewer candidates per mention.
- The publicly distributable UMLS subset contains 2.78M unique concepts and covers 99% of mention concepts in MedMentions.
- Candidate generation retrieves nearest UMLS concepts and aliases using approximate nearest-neighbour search over character 3-gram TF-IDF vectors.
- Because aliases can map to multiple concepts, K retrieved strings yield fewer than K candidate entities in practice; K=100 retrieves 54.26±12.45 candidates on average.
- Abbreviation detection substitutes abbreviated mentions with their long forms before nearest-neighbour search to partially remedy missed gold entities.
- 5% absolute improvement over Murty et al. (2018) accompanies 46% fewer candidates per mention on average.
6 Sentence Segmentation and Citation Handling
Biomedical sentence segmentation is difficult because abbreviations, punctuated noun compounds, and citations can resemble sentence boundaries. In-domain dependency parsing replaces brittle rule-based segmentation in the evaluated scispaCy models.
- Abbreviations, punctuated noun compounds, and diverse citation styles create biomedical sentence-boundary errors for standard algorithms.
- The evaluation measures sentence accuracy and full-abstract accuracy on raw, untokenized GENIA development abstracts.
- The segmentation model is also tested on PubMed body-text sentences, which contain more complex citation patterns than abstracts.
- In-domain dependency-parser training completely obviates rule-based sentence segmentation for both scispaCy models.
- The released models include a custom tokenizer but omit a custom sentence segmenter by default.
7 Related Work
Related biomedical NLP systems address clinical processing, entity linking, corpus-based parsing, domain adaptation, and structured knowledge curation. scispaCy is positioned as an openly available, modern alternative focused on robust biomedical text processing.
- cTAKES targets clinical notes, while MetaMap and MetaMapLite focus on UMLS-based entity linking.
- GENIA-based systems include OpenNLP adaptations and the GENIA Tagger, but cited alternatives are less suitable for modern Python workflows or are not openly available.
- Prior domain-adaptation work emphasizes adapting parsers to biomedical text, whereas scispaCy emphasizes robust, high-performance parsing using existing resources.
- Distant supervision and handwritten dependency-path rules have been used to curate structured biomedical resources such as cancer pathway interactions.
8 Conclusion
The paper presents fast, scalable scispaCy pipelines for biomedical NLP that achieve close to state-of-the-art performance. It also releases reformatted and augmented GENIA resources to support downstream biomedical information extraction.
- scispaCy provides robust pipelines for multiple biomedical NLP tasks and is described as fast, easy to use, scalable, and close to state of the art.
- The released models are intended to support biomedical information-extraction applications and downstream use of high-quality syntactic annotation.
- The release includes a reformatted GENIA 1.0 corpus with automatically produced Universal Dependency annotations and recovered, aligned abstract metadata.
- Future work includes a fuller entity linker and additional biomedical pipeline components such as negation detection.