Source-linked AI summary
NeuroNER: an easy-to-use program for named-entity recognition based on neural networks
Franck Dernoncourt, Ji Young Lee, Peter Szolovits
TL;DR
NER systems benefit from ANN performance, but ANNs are challenging for non-experts and annotation workflows can require cumbersome data conversion. NeuroNER integrates BRAT with ANN-based training and prediction, and achieves state-of-the-art results while supporting accessible use.
Problem
ANNs have strong NER performance but remain challenging for non-expert users, while existing workflows may require time-consuming annotation-data conversion.
Method
NeuroNER integrates BRAT-based annotation with an ANN-based NER engine for training, prediction, monitoring, and deployment labeling.
Results
NeuroNER yields state-of-the-art results and performs on par with state-of-the-art systems on CoNLL 2003 and i2b2 2014 using default hyperparameters.
Takeaways & Limitations
NeuroNER makes ANN-based NER and the annotation-training-prediction workflow accessible to non-expert users.
Abstract
from arXiv · showhide
Named-entity recognition (NER) aims at identifying entities of interest in a text. Artificial neural networks (ANNs) have recently been shown to outperform existing NER systems. However, ANNs remain challenging to use for non-expert users. In this paper, we present NeuroNER, an easy-to-use named-entity recognition tool based on ANNs. Users can annotate entities using a graphical web-based user interface (BRAT): the annotations are then used to train an ANN, which in turn predict entities' locations and categories in new texts. NeuroNER makes this annotation-training-prediction flow smooth and accessible to anyone.
1 Introduction
NER identifies entities of interest for downstream applications, while ANNs offer strong performance but remain difficult for non-experts to use. NeuroNER addresses this usability gap through BRAT-integrated annotation, training, and prediction.
- NER identifies entities such as locations, organizations, and temporal expressions for applications including de-identification and information extraction.Identified entities can also serve as features for other natural language processing tasks.
- Rule-based NER systems are time-consuming to develop and difficult to transfer to new text or entity types.
- ANNs can learn features jointly with model parameters from training data instead of relying on handcrafted, dataset-specific features.The paper states that ANNs have outperformed other supervised NER algorithms.
- NeuroNER integrates the web-based annotation program BRAT with ANN-based NER to make annotation, training, and prediction accessible to non-experts.The tool is described as open source and freely available online.
2 Related Work
Existing NER tools for non-experts commonly rely on CRFs, SVMs, or rules and often require costly data reformatting. NeuroNER streamlines annotation and prediction by incorporating BRAT.
- Publicly available NER systems for non-experts generally use CRFs, SVMs, or rules rather than ANNs.The paper presents NeuroNER as the first ANN-based NER system for non-experts.
- Many NER systems require users to reformat annotated corpora and convert predicted annotations, making workflows time-consuming and error-prone.
- NeuroNER incorporates BRAT to support creating new annotations and improving existing annotations without separate annotation-tool integration.
3 System Description
NeuroNER combines an ANN-based NER engine, BRAT integration, monitoring tools, and pre-trained models. It supports labeled-data training, deployment labeling, and interactive annotation workflows, with state-of-the-art performance on two datasets.
- System components: NeuroNER comprises an NER engine, a BRAT interface, real-time training monitors, and loadable pre-trained models.
- NER engine: The engine accepts training, validation, and test sets with gold labels, plus an optional deployment set containing new unlabeled text.Inputs may use BRAT or CoNLL-2003 formats.
- NER engine: The ANN maps tokens to character-enhanced vectors, predicts label probabilities, and outputs the most likely label sequence through jointly learned layers.
- Configuration: NeuroNER exposes architecture and training hyperparameters through a human-readable configuration file rather than requiring code changes.The dataset folder is the only parameter that must be changed; other parameters have reasonable defaults.
- Monitoring: Training progress is monitored through learning plots and TensorBoard, including F1-score evolution, confusion matrices, and class-level reports.TensorBoard provides real-time, shareable, interactive views of TensorFlow runs.
- Annotation workflow: BRAT integration supports creating annotations from scratch or iteratively improving existing annotations with repeated manual correction and engine runs.NeuroNER both consumes and produces BRAT-formatted data, and BRAT can be deployed as a web application for crowdsourcing.
- Performances: The evaluation uses the English CoNLL 2003 and i2b2 2014 NER datasets, including de-identification of protected health information in i2b2.
- Performances: NeuroNER performs on par with state-of-the-art systems on CoNLL 2003 and i2b2 2014 using default, unoptimized hyperparameters.Table 1 compares test-set F1-scores with the best published methods.
4 Conclusions
NeuroNER is an ANN-based NER tool designed for non-experts that integrates with BRAT and achieves state-of-the-art results.
- NeuroNER combines accessibility for non-expert users, ANN-based NER, state-of-the-art results, and smooth integration with BRAT.