Source-linked AI summary
An Effective Transition-based Model for Discontinuous NER
Xiang Dai, Sarvnaz Karimi, Ben Hachey, Cecile Paris
TL;DR
Biomedical NER often contains discontinuous mentions that flat sequence tagging cannot recover, motivating a model beyond conventional tagging assumptions. The paper introduces a transition-based neural model and reports strong discontinuous-NER performance across three biomedical data sets without sacrificing continuous-mention accuracy.
Problem
Biomedical NER includes discontinuous and overlapping mentions that conventional sequence tagging assumptions cannot represent reliably.
Method
The paper proposes an end-to-end transition-based model with generic neural encoding, specialized actions, and attention for discontinuous NER.
Results
The model achieves the highest F1 and recall across all three data sets under both evaluation settings, while improving recall without sacrificing precision.
Takeaways & Limitations
Transition-based decoding provides an effective way to recognize discontinuous mentions while preserving accuracy on continuous mentions.
Takeaways & Limitations
Prior intermediate representations can remain ambiguous during decoding, including whether shared tokens form mentions independently.
Abstract
from arXiv · showhide
Unlike widely used Named Entity Recognition (NER) data sets in generic domains, biomedical NER data sets often contain mentions consisting of discontinuous spans. Conventional sequence tagging techniques encode Markov assumptions that are efficient but preclude recovery of these mentions. We propose a simple, effective transition-based model with generic neural encoding for discontinuous NER. Through extensive experiments on three biomedical data sets, we show that our model can effectively recognize discontinuous mentions without sacrificing the accuracy on continuous mentions.
1 Introduction
Biomedical NER must handle discontinuous and overlapping mentions that conventional flat sequence tagging cannot represent. The paper motivates a transition-based model with neural encoding to recover these mentions without reducing continuous-mention accuracy.
- Motivation: Biomedical NER supports pharmacovigilance and clinical-information extraction, where complex mentions occur frequently.Applications include identifying adverse drug events and extracting conditions from unstructured medical records.
- Problem: Flat sequence tagging assumes mentions neither overlap nor contain separated token spans, leaving discontinuous mentions comparatively understudied.Discontinuous mentions consist of components separated by intervals, as illustrated in Figure 1.
- Problem: Exhaustively enumerating discontinuous and overlapping spans is exponential in sentence length, while prior methods face high complexity, ambiguous decoding, or weak feature generalization.Existing approaches also rely on manually designed features tailored to particular entity types.
- Motivation: Discontinuous mentions can denote compositional clinical concepts distinct from their individual components, such as ‘left atrium dilated’ having its own UMLS CUI.Recognizing the combined disorder or symptom can therefore be more useful than recognizing separate body-location or general-feeling components.
- Motivation: Overlapping discontinuous mentions must remain separate because shared components can participate in distinct entities, which matters for downstream entity linking.For example, ‘muscle’ can be shared by ‘muscle pain’ and ‘muscle fatigue’.
- Contribution: The paper proposes an end-to-end transition-based model with generic neural encoding, specialized actions, and attention to identify discontinuous components.Experiments on three biomedical data sets evaluate whether discontinuous recognition improves without sacrificing continuous-mention accuracy.
2 Prior Work
Prior discontinuous-NER systems predict intermediate token or sentence representations that can be difficult to decode unambiguously. The proposed transition-based representation instead maps action sequences to mention outputs without this ambiguity, while differing from nested-NER formulations.
- Existing Approaches: Existing discontinuous-NER methods are mainly token-level sequence taggers or sentence-level models that jointly predict mention combinations.Token-level systems assign tags to tokens, whereas sentence-level systems represent combinations through structures such as hypergraphs.
- Token-Level Approaches: Expanded BIO-style tag sets add indicators for shared heads and discontinuous bodies, including BH, IH, BD, and ID.These indicators represent components shared by multiple mentions and components exclusive to discontinuous mentions.
- Limitations: Intermediate representations can be ambiguous: the same BIO sequence may encode whether ‘muscle’ is itself a mention alongside ‘muscle pain’ and ‘muscle fatigue’.This ambiguity arises during decoding from tags or sub-hypergraphs into final mentions.
- Proposed Approach: The proposed transition-based model also uses an action sequence, but its outputs can always be decoded unambiguously into mention outputs.This directly addresses the ambiguity identified in prior token-level and sentence-level approaches.
- Related Distinctions: Structured multi-label and hypergraph methods flexibly represent discontinuous spans, but discontinuous overlap differs from nested NER because neither overlapping mention necessarily contains the other.Existing nested-NER models target complete containment and cannot be directly used for the overlapping structure studied here.
- Related Distinctions: Discontinuous NER can be converted into nested NER by adding entity types and annotating shared and component mentions separately.The conversion uses labels such as Body Location and General Feeling to represent the nested structure.
3 Model
The model frames discontinuous NER as shift-reduce parsing over a stack and buffer, using specialized transitions, neural span representations, and attention to select valid next actions.
- 3 Model: The parser repeatedly predicts an action from its current stack, buffer, and action history until both stack and buffer are empty.Hard constraints restrict selection to valid actions for the current parser state.
- 3 Model: Six specialized transitions support discontinuous and overlapping mentions: SHIFT, OUT, COMPLETE, REDUCE, LEFT-REDUCE, and RIGHT-REDUCE.LEFT-REDUCE and RIGHT-REDUCE retain one component so spans can participate in multiple mentions.
- 3 Model: Token representations combine bidirectional LSTM context, token and character embeddings, and frozen ELMo features; a fine-tuned BERT variant performs worse than ELMo feature extraction.The resulting contextual representations are used for tokens in the buffer.
- 3 Model: Stack-LSTM encodes individual stack tokens, while a multilayer perceptron composes spans created by REDUCE actions.The composed span replaces the two original spans on the stack.
- 3 Model: Multiplicative attention lets each stack span attend to buffer tokens, capturing dependencies such as combining an existing span with a future token.The parser concatenates top-stack spans, attended representations, and previous-action features before predicting the next action.
4 Data sets
The evaluation uses three biomedical data sets containing substantial discontinuous-mention coverage, with different source domains and discourse styles.
- 4 Data sets: The study evaluates CADEC, ShARe 13, and ShARe 14, where around 10% of mentions are discontinuous.Table 1 reports descriptive statistics for the three data sets.
- 4 Data sets: CADEC uses adverse drug event annotations from a patient forum, while ShARe 13 and 14 identify disorder mentions in clinical text.Only CADEC adverse drug event annotations are used because they contain the discontinuous annotations.
- 4 Data sets: CADEC differs from ShARe in discourse style: patients commonly use idioms, whereas practitioners tend to use compact terms.The data sets share a biomedical field but differ in tenor and communication purpose.
- 4 Data sets: CADEC is split into 70% training, 15% development, and 15% test posts, while ShARe uses shared-task splits with 10% of training documents held out for development.Models are selected by development-set micro-average strict-match F1 and then evaluated on the test set.
5 Baseline Models
The baselines include a strong continuous-span flat model and two discontinuous NER models, evaluated in settings that include all mentions rather than only discontinuous cases.
- 5 Baseline Models: The comparison uses Flair as a flat NER baseline, a BIO Extension model, and a graph-based model.The flat model requires transforming discontinuous and overlapping mentions into covering spans before training.
- 5 Baseline Models: The evaluation includes all mentions because real data sets do not provide a fixed discontinuous-mention ratio in advance.This tests performance beyond sentences selected for containing discontinuous mentions.
- 5 Baseline Models: The Wang and Lu (2019) model is excluded because its effectiveness depends strongly on training and testing only sentences containing at least one discontinuous mention.The proposed evaluation instead reflects a setting where continuous mentions greatly outnumber discontinuous mentions.
6 Experimental Results
On whole-test evaluation, the proposed model outperforms three baselines and prior reported results in recall and F1 while preserving precision. It also achieves the highest F1 and recall on discontinuous-mention-focused evaluations across all three data sets.
- Whole-test evaluation: The model outperforms three baselines and prior reported results in recall and F1 on the whole test set without sacrificing precision.The graph-based model has the highest precision but substantially lower recall, yielding the lowest F1.
- Implications: Improved recall is especially relevant to pharmacovigilance and medical-record summarization, where recall is at least as important as precision.
- Discontinuous-mention evaluation: The model achieves the highest F1 and recall across all three data sets both when evaluating relevant sentences and when evaluating discontinuous mentions alone.Table 3 separates sentences containing at least one discontinuous mention from discontinuous mentions only.
- Discontinuous-mention evaluation: On CADEC, 88% of discontinuous mentions involve overlapping structure, which helps explain why BIO Extension performs poorly when discontinuous mentions alone are evaluated.BIO Extension reaches only 1.8 F1 in the discontinuous-mentions-only setting.
7 Analysis
The analysis examines how overlap, mention length, and interval length affect discontinuous NER, finding that the proposed model generally has the strongest recall while multi-overlap and some annotation patterns remain difficult.
- Impact of Overlapping Structure: The model performs best on No overlap in ShARe 13 and 14 and Left overlap in CADEC and ShARe 14.No overlap comprises half of discontinuous mentions in ShARe, while Left overlap comprises half in CADEC.
- Impact of Overlapping Structure: No model is effective on Multi-overlap mentions, although this category represents a very small portion of discontinuous mentions in all three data sets.These mentions involve crossing compositions and remain especially challenging.
- Impact of Overlapping Structure: On CADEC No overlap, our model predicts no discontinuous mentions correctly because the category is small and its annotations are linguistically inconsistent.Severity indicators are included as intervals inconsistently, complicating learning and evaluation.
- Impact of Mention and Interval Length: Recall generally decreases as mention length increases, while interval length has no straightforward effect on recognition difficulty.Very short and very long intervals are both difficult; the easiest interval length is 2 on CADEC and 3 on ShARe 13 and 14.
- Impact of Mention and Interval Length: Our model achieves the highest recall in most length settings, recognizing continuous and discontinuous mentions across varied mention lengths.BIO E. is strong on continuous mentions but fails when interval length exceeds zero.
- Example Predictions: Long, overlapping mentions expose decoding weaknesses in previous models, whereas our model correctly identifies both mentions in the severe joint-pain example.The BIO extension predicts most tags but decodes an incorrect mention, while the graph-based model identifies none.
8 Summary
The paper proposes a transition-based model for discontinuous NER that preserves continuous-mention accuracy. Across three biomedical data sets, it outperforms two existing discontinuous NER models, particularly in recall.
- 8 Summary: The paper proposes a simple, effective transition-based model that recognizes discontinuous mentions without sacrificing continuous-mention accuracy.
- 8 Summary: Across three biomedical data sets, the model is more effective than two existing discontinuous NER models, especially in recall.