PaperlayerView exact revision on arXiv

Source-linked AI summary

A Unified MRC Framework for Named Entity Recognition

Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Guoyin Wang, Fei Wu, Jiwei Li

arXiv:1910.11476v9cs.CL

TL;DR

Flat NER sequence labeling cannot represent nested entities when one token requires multiple labels, motivating a unified alternative. The paper reformulates both tasks as MRC with category-specific natural-language queries and reports state-of-the-art results across nested and flat datasets, while noting that flat-NER gains cannot be clearly attributed to MRC rather than BERT.

  • Problem

    Sequence labeling assigns a single label to each token, limiting its suitability for nested NER where tokens may belong to overlapping entities.

  • Method

    The framework converts NER into MRC by representing each entity type with a natural-language query and extracting answer spans, using binary start and end classifiers to allow multiple spans.

  • Results

    The method achieves state-of-the-art results on both nested and flat NER datasets, with reported gains across ACE04, ACE05, GENIA, KBP17, CoNLL 2003, OntoNotes, and Chinese datasets.

  • Takeaways & Limitations

    MRC provides a unified formulation that addresses overlapping entities and supplies prior information about the entity category to extract.

  • Takeaways & Limitations

    For flat NER, the paper cannot determine how much of the improvement comes from the MRC formulation versus BERT pre-training.

Abstract

from arXiv · show

The task of named entity recognition (NER) is normally divided into nested NER and flat NER depending on whether named entities are nested or not. Models are usually separately developed for the two tasks, since sequence labeling models, the most widely used backbone for flat NER, are only able to assign a single label to a particular token, which is unsuitable for nested NER where a token may be assigned several labels. In this paper, we propose a unified framework that is capable of handling both flat and nested NER tasks. Instead of treating the task of NER as a sequence labeling problem, we propose to formulate it as a machine reading comprehension (MRC) task. For example, extracting entities with the \textsc{per} label is formalized as extracting answer spans to the question "{\it which person is mentioned in the text?}". This formulation naturally tackles the entity overlapping issue in nested NER: the extraction of two overlapping entities for different categories requires answering two independent questions. Additionally, since the query encodes informative prior knowledge, this strategy facilitates the process of entity extraction, leading to better performances for not only nested NER, but flat NER. We conduct experiments on both {\em nested} and {\em flat} NER datasets. Experimental results demonstrate the effectiveness of the proposed formulation. We are able to achieve vast amount of performance boost over current SOTA models on nested NER datasets, i.e., +1.28, +2.55, +5.44, +6.37, respectively on ACE04, ACE05, GENIA and KBP17, along with SOTA results on flat NER datasets, i.e.,+0.24, +1.95, +0.21, +1.49 respectively on English CoNLL 2003, English OntoNotes 5.0, Chinese MSRA, Chinese OntoNotes 4.0.

1 Introduction

The paper unifies flat and nested NER by reformulating entity extraction as machine reading comprehension rather than sequence labeling. Natural-language queries address overlapping entities and encode category information, while experiments report gains across nested and flat benchmarks.

  • Sequence labeling assigns one class per token, making it unsuitable when nested entities require multiple categories for one token.
  • The proposed framework formulates both flat and nested NER as MRC, using a natural-language query for each entity type and extracting answer spans.For example, the PERSON category becomes a question asking which person is mentioned in the text.
  • Independent questions allow overlapping entities from different categories to be extracted separately, naturally addressing nested-NER overlap.
  • Query descriptions encode prior information about entity categories and can help disambiguate similar tagging classes.
  • +1.28, +2.55, +5.44, and +6.37% are reported gains on ACE04, ACE05, GENIA, and KBP17, respectively.The experiments cover both nested and flat NER datasets.
  • The authors aim for the framework to inspire new paradigms for entity recognition.

2 Related Work

Related work covers sequence-labeling and structured approaches to nested NER, MRC span extraction, and the broader trend of converting NLP tasks into question answering. The paper distinguishes its NER formulation by incorporating more factual query knowledge and analyzing query-construction strategies.

  • Earlier nested-NER systems used layered CRFs, parse trees, mention hyper-graphs, span classifiers, and neural hyper-graph representations.
  • Later work incorporated contextual embeddings, iterative extraction, and sequence-to-sequence generation for nested NER.
  • MRC models traditionally extract answer spans by predicting their starting and ending positions from a passage and question.
  • Question-answering formulations have also been applied to relation extraction, summarization, and sentiment analysis.
  • Unlike prior relation-extraction work using less diverse template queries, this paper incorporates synonyms and examples and analyzes query-building strategies.

3 NER as MRC

The framework reformulates NER as MRC: each entity type becomes a natural-language query, and the model extracts matching spans from the context. It predicts multiple start and end positions, then learns to align them so overlapping and same-category entities can be recovered.

  • Task Formalization: NER is reformulated as MRC by pairing each entity type with a natural-language question and extracting its answer span from the input sequence.Annotated data are converted into QUESTION, ANSWER, CONTEXT triples.
  • Query Generation: Query generation uses annotation guidelines as references because queries encode label prior knowledge and influence final results.The paper also considers query strategies such as keywords, templates, Wikipedia definitions, and synonyms.
  • Model Backbone: BERT concatenates the query and passage, then supplies context representations for downstream span prediction.The query representations are dropped after BERT produces the context representation matrix E.
  • Span Selection: The adopted span-selection strategy uses separate binary classifiers to predict multiple start and end indexes for each query.Unlike two n-class classifiers, binary token-level predictions can return multiple spans for one query.
  • Span Selection: Because nearest-end matching fails for overlapping entities, a binary matching model predicts which candidate start and end indexes form spans.Candidate indexes are obtained by applying argmax to each row of the start and end probability matrices.
  • Train and Test: Training jointly minimizes weighted start-index, end-index, and span-matching losses, while testing selects indexes separately before alignment.The weights α, β, and γ control the contributions of the three losses, and parameters are shared at the BERT layer.

4.1 Experiments on Nested NER

The nested NER experiments evaluate the model on four established datasets using span-level micro-averaged precision, recall, and F1, with comparisons against prior baselines. The model substantially improves over previous state-of-the-art results across all four datasets.

  • Datasets and setup: The evaluation covers ACE 2004, ACE 2005, GENIA, and KBP2017, whose nested mentions comprise 24%, 22%, 10%, and 19%, respectively.Hyperparameters are tuned on corresponding development sets, and evaluation uses span-level micro-averaged precision, recall, and F1.
  • Baselines: The study compares the proposed model with hypergraph, segmental hypergraph, anchor-region, prior-best, and Seq2Seq-BERT baselines.These baselines represent prior approaches for modeling overlapping or nested entity mentions.
  • Results: 85.98%, 86.88%, 83.75%, and 80.97% F1 are achieved on ACE04, ACE05, GENIA, and KBP-2017, respectively.The reported gains over previous state-of-the-art performances are +1.28%, +2.55%, +5.44%, and +6.37%, respectively.
  • Results: +1.28%, +2.55%, +5.44%, and +6.37% improve over previous state-of-the-art performances on ACE04, ACE05, GENIA, and KBP17, respectively.The largest reported gain is on KBP17, while the smallest is on ACE04.

4.2 Experiments on Flat NER

The flat NER experiments evaluate English and Chinese datasets with span-level micro-averaged metrics and compare the proposed model with established tagging and contextual-embedding baselines. The model reports gains across the listed English and Chinese benchmarks.

  • Datasets and setup: The evaluation covers English CoNLL2003 and OntoNotes 5.0 plus Chinese OntoNotes 4.0 and MSRA datasets.Hyperparameters are tuned on corresponding development sets, with span-level micro-averaged precision, recall, and F1 used for evaluation.
  • Datasets and setup: CoNLL2003 contains four entity types, English OntoNotes 5.0 contains 18, Chinese MSRA contains three, and Chinese OntoNotes 4.0 annotates 18.The datasets include both English and Chinese news or mixed-source text, depending on the benchmark.
  • Baselines: The model is compared with BiLSTM-CRF, ELMo, CVT, and BERT-Tagger for English, plus Lattice-LSTM, BERT-Tagger, and Glyce-BERT for Chinese.The baselines include tagging models and Chinese models incorporating lattice or glyph information.
  • English results: +0.24% F1 improvement over fine-tuned BERT tagging is reported on English CoNLL 2003, while English OntoNotes 5.0 gains +1.95%.The reported explanation links the larger OntoNotes gain to its greater number of entity types and category-level data sparsity, alongside query-encoded prior information.
  • Chinese results: +0.95% F1 improvement over fine-tuned BERT tagging is reported on Chinese MSRA, while Chinese OntoNotes 4.0 rises from 79.16% to 82.11%.The paper describes these results as new state-of-the-art results on the Chinese datasets.

5 Ablation studies

The ablations examine whether gains come from MRC or BERT, how query construction affects performance, and whether MRC generalizes with limited or unseen labels.

  • 5.1 Improvement from MRC or from BERT: +1.95%: BERT-MRC outperforms BERT-Tagger on English OntoNotes 5.0.Comparisons with QAnet and BiDAF, which do not rely on large-scale pretraining, also support the importance of the MRC formulation.
  • 5.1 Improvement from MRC or from BERT: Attention matrices capture semantic similarity between tagging classes and context sentences.In the example, “Flevland” matches geographical, cities and state.
  • 5.2 How to Construct Queries: BERT-MRC outperforms BERT-Tagger for every query type except Position Index of Labels.Annotation Guideline Notes achieve the highest F1 score; index-based queries lack meaningful information, while Wikipedia definitions are relatively general.
  • 5.2 How to Construct Queries: BERT-MRC achieves acceptable zero-shot results when trained on CoNLL 2003 and tested on OntoNotes5.0, unlike BERT-Tagger’s 31.87% accuracy.OntoNotes5.0 has 18 entity types, including 15 unseen in CoNLL03; BERT-Tagger cannot predict labels unseen during training.
  • 5.2 How to Construct Queries: BERT-MRC achieves comparable performance to BERT-Tagger on Chinese OntoNotes 4.0 with half the training data.The result is presented as evidence that query-based prior knowledge can help when fewer training samples are available.

6 Conclusion

The paper reformulates NER as an MRC question-answering task. This formulation handles overlapping entities, encodes category prior knowledge, and achieves SOTA results on nested and flat NER datasets.

  • The proposed MRC formalization handles overlapping or nested entities and encodes prior knowledge about the entity category to extract.
  • The method obtains SOTA results on both nested and flat NER datasets.
Loading 1910.11476v9…