PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Recent Advances in Named Entity Recognition from Deep Learning models

Vikas Yadav, Steven Bethard

arXiv:1910.11470v1cs.CLcs.LG

TL;DR

Named entity recognition is important for NLP applications, but recent neural architectures require synthesis against longstanding feature-engineered and knowledge-based approaches. This survey compares these systems and finds that neural models generally perform better, while combining neural architectures with past feature-engineering insights yields further gains.

  • Problem

    Named entity recognition supports applications including question answering and information retrieval, motivating evaluation of recent neural architectures against earlier approaches.

  • Method

    The paper surveys NER systems across knowledge-based, bootstrapped, feature-engineered, and neural approaches, comparing neural models using different word and sub-word representations.

  • Results

    Neural network models generally outperform feature-engineered systems, and an affix-based extension achieves state-of-the-art performance in Spanish, Dutch, and German while coming within 1% of the best English model.

  • Takeaways & Limitations

    Character+word hybrid neural networks generally outperform other representational choices, and incorporating lessons from feature-engineered models can further improve neural NER.

  • Takeaways & Limitations

    Knowledge-based NER depends on complete domain dictionaries and rules; incomplete resources can produce low recall, especially for new or unapproved drugs.

Abstract

from arXiv · show

Named Entity Recognition (NER) is a key component in NLP systems for question answering, information retrieval, relation extraction, etc. NER systems have been studied and developed widely for decades, but accurate systems using deep neural networks (NN) have only been introduced in the last few years. We present a comprehensive survey of deep neural network architectures for NER, and contrast them with previous approaches to NER based on feature engineering and other supervised or semi-supervised learning algorithms. Our results highlight the improvements achieved by neural networks, and show how incorporating some of the lessons learned from past work on feature-based NER systems can yield further improvements.

1 Introduction

NER identifies named entities and supports applications including question answering and information retrieval. This survey reviews neural architectures alongside earlier feature-engineered approaches and reports performance gains with minimal feature engineering.

  • NER identifies entities such as people, locations, organizations, drugs, times, procedures, and proteins in text.
  • NER commonly serves as an initial component in question answering, information retrieval, coreference resolution, and topic modeling.
  • The survey contrasts knowledge-based and feature-engineered systems with neural architectures using minimal feature engineering.
  • Neural models on news corpora improved previous state-of-the-art results by 1.59% in Spanish, 2.34% in German, 0.36% in English, and 0.14% in Dutch.
  • The survey organizes reviewed systems into knowledge-based, bootstrapped, feature-engineered, and neural-network categories.

2 Previous surveys

Earlier NER surveys covered broad supervised, semi-supervised, and unsupervised approaches, but existing work largely emphasized feature engineering or individual domains and languages. The paper addresses the absence of a comprehensive neural-NER survey comparing systems across multilingual and multidomain settings.

  • Nadeau and Sekine’s survey covered supervised, semi-supervised, and unsupervised NER systems, common features, and evaluation metrics.
  • Sharnagat’s later survey added introductory neural-network NER systems while retaining supervised, semi-supervised, and unsupervised coverage.
  • Existing surveys primarily focus on feature-engineered models and often address only one language or domain.
  • The paper identifies a gap: no comprehensive survey of modern neural NER systems compares feature-engineered and neural systems across multilingual and multidomain settings.

3 Methodology

The survey identifies relevant NER papers through searches of Google, Google Scholar, and Semantic Scholar, then prioritizes highly cited studies for review. Included papers either introduce neural NER architectures or report top-performing results on NER datasets.

  • The authors searched Google, Google Scholar, and Semantic Scholar using queries about neural architectures and deep-learning NER models.
  • They sorted retrieved papers by citation count and read at least the top three for each query.
  • A paper was considered if it introduced a neural NER architecture or represented a top-performing model on an NER dataset.

4 NER datasets

NER datasets span newswire, multilingual, biomedical, social-media, and structurally complex settings, with entity types varying by domain and language. Some datasets contain nested or discontinuous entities rather than only flat annotations.

  • CoNLL 2002 and 2003 use newswire from Spanish, Dutch, English, and German, annotating PER, LOC, ORG, and MISC entities.
  • NER shared tasks cover languages including Indian languages, Arabic, German, and seven Slavic languages, with entity types varying across datasets.
  • Biomedical tasks target entities such as proteins, DNA, RNA, cells, clinical personal data, drugs, brands, chemicals, and biological locations.
  • Twitter NER is challenged by orthographic variability and incomplete sentences, and its entity categories reflect user behavior.
  • Some datasets annotate nested entities, entity heads, or discontinuous entities, creating structures beyond flat named-entity annotations.

5 NER evaluation metrics

NER evaluation distinguishes label correctness from boundary correctness, with CoNLL adopting exact entity matching and micro-F-score. Other shared tasks use relaxed or strict F1 to accommodate differing segmentation and boundary requirements.

  • Type scoring checks label correctness regardless of entity boundaries, whereas text scoring checks boundary correctness regardless of label.
  • CoNLL exact match counts a prediction only when the complete entity label and word span exactly match the gold annotation.
  • CoNLL micro-F-score is the harmonic mean of exact-match precision and recall.
  • Relaxed F1 accepts partially correct entity identification, while strict F1 requires exact character-offset agreement with human annotation.
  • Relaxed F1 supports comparison when datasets lack word offsets and systems use different segmentation techniques.

6 NER systems

Knowledge-based NER relies on lexicons and domain knowledge rather than annotated training data. Its precision is generally high with lexicons, but incomplete resources constrain recall and require expert maintenance.

  • Knowledge-based NER systems rely on lexicons and domain-specific knowledge, so they do not require annotated training data.
  • These systems work well when their lexicon is exhaustive but can fail on entity classes absent from the underlying dictionaries.
  • New or unapproved drugs are absent from DrugBank dictionaries by definition, creating a concrete failure case for DrugNER.
  • Knowledge-based systems generally achieve high precision but often low recall because of domain- and language-specific rules and incomplete dictionaries.
  • Domain experts are needed to construct and maintain the knowledge resources used by these systems.

6.2 Unsupervised and bootstrapped systems

Unsupervised and bootstrapped NER systems reduce reliance on annotated data by starting from seeds, patterns, gazetteers, or corpus-derived signals. Their approaches range from web pattern extraction to noun-phrase filtering and gazetteer expansion.

  • Early unsupervised systems used minimal training data, including labeled seeds and a small set of orthographic and contextual features.
  • Etzioni et al. used eight generic pattern extractors on open-web text to improve NER recall without supervised annotation.
  • Gazetteer construction and entity ambiguity resolution combined extracted and existing gazetteers, yielding 88%, 61%, and 59% F-scores for MUC-7 location, person, and organization entities.
  • Zhang and Elhadad combined seed-based candidate discovery, noun-phrase detection, low-IDF filtering, and classification for biology and medical NER.
  • 53.8% and 69.5% accuracy were achieved on biology and medical data, respectively, by the seed-and-IDF unsupervised model.

6.3 Feature-engineered supervised systems

Feature-engineered supervised NER learned from annotated examples using models such as HMMs, SVMs, CRFs, AdaBoost, and structural learning. These systems combined orthographic, contextual, lexical, gazetteer, syntactic, and embedding-based features to achieve strong results across datasets.

  • 6.3 Feature-engineered supervised systems: Supervised NER models replace human-curated rules by learning predictions from example inputs and expected outputs.
  • 6.3 Feature-engineered supervised systems: HMM, SVM, CRF, and decision-tree approaches were common supervised machine-learning systems for NER.
  • 6.3 Feature-engineered supervised systems: 96.6% and 94.1% F scores were achieved on MUC-6 and MUC-7 using HMMs with orthographic features, trigger words, and class-specific word lists.
  • 6.3 Feature-engineered supervised systems: 73.66% and 68.08% Fscore were obtained on Spanish and Dutch CoNLL 2002 using capitalization, sentence position, known-name context, and dictionary names.
  • 6.3 Feature-engineered supervised systems: 81.39% and 77.05% F scores were achieved on Spanish and Dutch CoNLL 2002 by AdaBoost models using capitalization, trigger words, previous tags, bag-of-words, and gazetteers.
  • 6.3 Feature-engineered supervised systems: Structural learning divided NER into auxiliary tasks, selecting each classifier according to confidence and achieving 89.31% and 75.27% F score on English and German.
  • 6.3 Feature-engineered supervised systems: Semi-supervised feature systems combined orthography, character n-grams, lexicons, affixes, n-grams, and unsupervised clusters to approach state-of-the-art CoNLL performance.
  • 6.3 Feature-engineered supervised systems: DrugNER CRF systems used domain resources such as FDA, DrugBank, Jochem, ChEBI, chemical affixes, and MedLine-trained word embeddings.

6.4 Feature-inferring neural network systems

Feature-inferring neural NER systems represent words using words, characters, or both, while incorporating selected features such as gazetteers, capitalization, lexicons, and affixes. Across languages and domains, these architectures combine neural sequence models with word and sub-word representations, and often use CRF or softmax prediction layers.

  • Representation-based architectures: Modern neural NER architectures classify sentence representations as word-level, character-level, or combinations of word and sub-word units.Word-level models use word embeddings, while character and combined models derive representations from character sequences.
  • Word level architectures: 89.59% F1 was achieved on English CoNLL 2003 by a convolutional word-level model using gazetteers and SENNA embeddings.The model replaced a Bi-LSTM with a convolution layer and used a CRF layer for final prediction.
  • Character level architectures: Character-level models predict character labels with recurrent networks and convert them into word labels through post-processing or Viterbi decoding.Reported results included 80.23% F-score for Vietnamese and 82.18%, 79.36%, 84.52%, and 70.12% on Spanish, Dutch, English, and German CoNLL datasets, respectively.
  • Character+Word level architectures: Word-plus-character systems combine word embeddings with CNNs or character LSTMs, sentence-level Bi-LSTMs, and softmax or CRF prediction.These systems achieved 83.71 F-score on CHEMDNER and 85.75%, 81.74%, 90.94%, and 78.76% F-scores on Spanish, Dutch, English, and German CoNLL data, respectively.
  • Character+Word level architectures: 91.62% F1 was achieved on English CoNLL 2003 by adding lexicons and capitalization features to a word-plus-character model.Lexicon features encoded whether dictionary matches marked the beginning, inside, or end of entity types.
  • Character + Word + affix model: 87.26%, 87.54%, 90.86%, and 79.01% were achieved on Spanish, Dutch, English, and German CoNLL datasets by adding learned affix embeddings to character-plus-word models.The study found complementary information in affixes, importance of selecting high-frequency affixes, and benefits from embedding affixes rather than expanding other embeddings.

7 Discussion

The survey finds that neural NER systems generally outperform feature-engineered systems, with word+character hybrids performing especially well. It also finds that adding affix features from earlier approaches can further improve neural architectures.

  • Neural systems outperform feature-engineered systems across the evaluated languages and DrugNER settings, despite the latter using domain-specific resources.The gaps are 1.59% for Spanish, 2.34% for German, 8.90% on DrugNER MedLine, and 3.50% on the overall DrugNER dataset.
  • Word+character hybrid models generally outperform both word-based and character-based neural models.Reported advantages include 0.52% over word-based and 5.12% over character-based models for English, plus 14.25% over word models on DrugNER MedLine.
  • Table 1 compares NER systems across Spanish, Dutch, English, and German and records whether each approach uses dictionary lookups.Best performance in each system category is highlighted.
  • Table 2 reports DrugNER results separately for MedLine and DrugBank test data.The test split contains 80.10% MedLine data and 19.90% DrugBank data; Yadav et al.'s results have no decimal places.
  • Incorporating affix features into a character+word neural model produces a strong model with state-of-the-art results in Spanish, Dutch, and German.Its English performance is within 1% of the best model.

8 Conclusion

The survey identifies consistent advantages for neural NER models, especially character+word hybrids, while showing that earlier feature-engineering insights can still improve modern architectures.

  • Neural networks generally outperform feature-engineered NER models, character+word hybrids generally outperform other representations, and affix-based extensions achieve state-of-the-art performance.These findings summarize the survey's comparisons of classic and neural NER systems.
Loading 1910.11470v1…