Source-linked AI summary
Cross-type Biomedical Named Entity Recognition with Deep Multi-Task Learning
Xuan Wang, Yu Zhang, Xiang Ren, Yuhao Zhang, Marinka Zitnik, Jingbo Shang, Curtis Langlotz, Jiawei Han
TL;DR
BioNER systems often depend on entity-specific handcrafted features, while neural models face limited training data for each entity type. The paper proposes multi-task learning with shared character- and word-level representations across datasets and reports substantially better benchmark performance. The authors attribute the large gains mainly to this cross-type information sharing, while identifying entity boundary and type conflicts as a future challenge.
Problem
BioNER development relies on costly entity-specific features, and available supervised data is limited for each entity type.
Method
The paper trains BioNER models for different entity types with shared character- and word-level BiLSTM parameters while retaining task-specific components.
Results
The multi-task approach substantially outperforms state-of-the-art systems and strong neural baselines on benchmark BioNER datasets, with marginally added training time.
Takeaways & Limitations
Sharing character- and word-level information across biomedical entity types is the reported main source of the approach’s large performance gains.
Takeaways & Limitations
The authors identify unresolved entity boundary and type conflicts as a future direction for unified multi-type recognition.
Abstract
from arXiv · showhide
Motivation: State-of-the-art biomedical named entity recognition (BioNER) systems often require handcrafted features specific to each entity type, such as genes, chemicals and diseases. Although recent studies explored using neural network models for BioNER to free experts from manual feature engineering, the performance remains limited by the available training data for each entity type. Results: We propose a multi-task learning framework for BioNER to collectively use the training data of different types of entities and improve the performance on each of them. In experiments on 15 benchmark BioNER datasets, our multi-task model achieves substantially better performance compared with state-of-the-art BioNER systems and baseline neural sequence labeling models. Further analysis shows that the large performance gains come from sharing character- and word-level information among relevant biomedical entities across differently labeled corpora.
1 Introduction
BioNER supports biomedical text-mining applications but remains costly to develop because entity-specific handcrafted features are labor-intensive and specialized. The paper addresses limited entity-type training data with multi-task learning that shares representations across datasets.
- BioNER recognizes and classifies biomedical entities and supports applications including relation extraction and knowledge-base completion.
- State-of-the-art BioNER systems often require handcrafted features tailored to each entity type.Examples include capitalization, prefixes, and suffixes.
- Feature generation consumes most development time and produces specialized systems that cannot directly recognize new entity types.
- Neural models reduce manual feature engineering but remain limited by training data and can underperform handcrafted feature-based systems across multiple domains.
- Combining datasets can create false negatives because each corpus usually annotates only one or a few entity types.Multi-task learning is proposed to avoid this issue while using information shared across datasets.
- The proposed multi-task framework shares character- and word-level representations and substantially outperforms state-of-the-art and baseline neural BioNER systems on 15 benchmark datasets.The analysis reports marginal computational overhead and attributes large gains mainly to shared character- and word-level information.
2 Background
The paper frames BioNER as sequence labeling and develops neural architectures that represent characters and words before predicting structured label sequences. BiLSTM-CRF models combine bidirectional sequence representations with CRF-based tag dependencies, motivating shared layers in multi-task models.
- NER problem definition: BioNER assigns each word in a sentence a label from a set indicating whether it belongs to a specific entity type.For example, “O” marks a non-entity and “S-GENE” marks a single-token gene.
- Long Short-Term Memory (LSTM): An LSTM maps an input sequence of word representation vectors to hidden-state vectors while modeling dependencies through recurrent connections.At each step, it uses the current input and previous cell and hidden states to produce updated states.
- Long Short-Term Memory (LSTM): Bidirectional LSTMs process sequences in both directions and concatenate outputs to capture dependencies from preceding and subsequent words.
- BiLSTM-CRF: A BiLSTM-CRF predicts label sequences by adding a CRF layer that models dependencies across output tags.The network maps an input sequence to an output label sequence rather than making independent tagging decisions.
- BiLSTM-CRF: The single-task architecture combines character embeddings, a character-level BiLSTM, word embeddings, a word-level BiLSTM, and a final CRF layer.The CRF outputs entity labels for each word in the input biomedical sentence.
- Multi-task models: The multi-task models vary whether character and word layers are shared or task-specific across entity-type datasets.MTM-C shares character layers, MTM-W shares word layers, and MTM-CW shares both.
3 Deep multi-task learning for BioNER
The paper addresses limited BioNER supervision by training task-specific models that share parameters across datasets. Its multi-task design shares character- and word-level representations while retaining dataset-specific label prediction.
- Single-task baseline: 67.21% of words are out-of-vocabulary in the Table 1 datasets, motivating character-aware modeling alongside word representations.The baseline architecture is designed to handle OOV biomedical words that lack pretrained word embeddings.
- Multi-task formulation: The multi-task approach trains different BioNER models on datasets with different entity types while sharing parameters across them.This is intended to use limited supervised data more efficiently and learn more generalized word and character representations.
- Optimization: The multi-task loss sums dataset-specific log-likelihood terms weighted by positive hyperparameters λ_i.The experiments use λ_i = 1 for all evaluated datasets, although further tuning may improve performance.
- Multi-task variants: Three multi-task models are proposed, differing in which word- and character-level parameters they share across datasets.The paper presents their architectures in Fig. 3.
- Parameter sharing: The model uses a shared word-level and character-level representation with dataset-specific output parameters for label prediction.MTM-CW shares both character- and word-level information, whereas the other proposed multi-task models share only part of the representation.
4 Experimental setup
The experiments evaluate the proposed models across biomedical entity-recognition benchmarks using established dataset splits, exact-match metrics, and comparisons with prior systems and neural baselines. Results are reported for five main datasets and supplemented by all 15 datasets.
- Datasets: 15 biomedical NER datasets are used, with detailed multi-task results reported on five main datasets covering genes, proteins, chemicals, and diseases.Full results on all 15 datasets are provided in supplementary material.
- Datasets: The datasets are divided into training, development, and test sets, with training and development data used to train the final model.All datasets are publicly available, and word labels use the IOBES encoding scheme.
- Baselines: Dataset benchmarks include semi-supervised, conditional-random-field, semi-Markov, and hidden-Markov-model systems tailored to individual benchmarks.Examples include alternating structure optimization for BC2GM, CHEMDNER for BC4CHEMD, TaggerOne for BC5CDR and NCBI-Disease, and an HMM for JNLPBA.
- Evaluation metrics: Predictions are counted correct only when both entity boundaries and types exactly match the gold annotation, followed by precision, recall, and F1 calculation.Macro-averaged F1 is also calculated across entity types.
- Evaluation metrics: BC2GM additionally reports alternative-match scores because its test set provides alternative answers for each entity.A prediction is correct if it matches the ground truth or one of the listed alternatives.
- Comparisons: MTM-CW is compared with state-of-the-art dataset benchmarks and three neural sequence-labeling models using precision, recall, F1, and average training time.Baseline neural models use their published default parameter settings and are repeated three times for the comparisons described.
5 Results
Across benchmark datasets and biomedical entity types, the MTM-CW multi-task model generally outperforms single-task, neural, and state-of-the-art systems. Its character- and word-level sharing also avoids gains from external dictionaries while improving performance with modest training overhead.
- Benchmark datasets: MTM-CW achieves significantly higher F1 scores than Dataset Benchmark on all five evaluated datasets.The comparison uses exact match except for BC2GM, where benchmark performance uses alternative match.
- Multi-task variants: The MTM-CW model is the best-performing multi-task variant, combining character-level morphological information with word-level lexical and contextual information.The comparison covers MTM-C, MTM-W, and MTM-CW.
- Major biomedical entity types: MTM-CW consistently outperforms the Habibi et al. neural model across genes/proteins, chemicals, diseases, and cell lines.It also exceeds state-of-the-art benchmark systems on three of the four entity types, with cell lines as the exception.
- Entity dictionaries: Adding biomedical entity dictionaries produces no significant preprocessing improvement, while post-processing dictionaries hurt performance.The reported explanation is a higher false-positive rate when surface names have different meanings or entity types.
- Training time: MTM-CW improves overall F1 by 8.0% over STM with only 5.1% additional training time.The comparison averages training time across the five main datasets because MTM-CW trains on them jointly.
- Case study: Case studies show that multi-task training improves recognition of long entities and entity boundaries.Examples include preserving a long gene entity and identifying correct protein and disease boundaries.
6 Conclusion
The proposed neural multi-task BioNER approach outperformed state-of-the-art and strong neural models without manual feature engineering, with gains mainly from sharing character- and word-level information across entity types. Future work targets combining single-task and multi-task models and resolving entity boundary and type conflicts.
- The neural multi-task BioNER approach outperformed state-of-the-art systems and several strong neural network models on benchmark datasets without manual feature engineering.
- Large performance gains mainly came from sharing character- and word-level information between biomedical entity types.
- The multi-task model achieved strong performance with only marginally added training time.
- Future work includes combining single-task and multi-task models and resolving entity boundary and type conflicts for unified multi-type recognition.