Source-linked AI summary
An Analysis of Simple Data Augmentation for Named Entity Recognition
Xiang Dai, Heike Adel
TL;DR
Named entity recognition has received less augmentation study because token transformations can alter entity labels. The paper adapts simple augmentation methods for NER and evaluates them on two domain-specific datasets with recurrent and transformer encoders. Simple augmentation improves performance over strong baselines, with the largest benefits for small training sets.
Problem
NER augmentation is less studied because token transformations may change the labels in this token-level sequence-labeling task.
Source: §1 · p. 1Method
The paper surveys existing augmentation methods, adapts selected techniques to NER, and evaluates them on MaSciP and i2b2-2010 using LSTM and BERT encoders.
Source: §1 · p. 1, §4.2 · p. 4Results
Simple data augmentation improves NER performance over strong baselines, including pretrained transformers, especially when training sets are small.
Source: §1 · p. 1, §5 · p. 5, §4.4 · p. 5Takeaways & Limitations
Data augmentation is effective for NER across recurrent and transformer-based models, with stronger benefits in low-data settings.
Source: §4.4 · p. 5
Abstract
from arXiv · showhide
Simple yet effective data augmentation techniques have been proposed for sentence-level and sentence-pair natural language processing tasks. Inspired by these efforts, we design and compare data augmentation for named entity recognition, which is usually modeled as a token-level sequence labeling problem. Through experiments on two data sets from the biomedical and materials science domains (i2b2-2010 and MaSciP), we show that simple augmentation can boost performance for both recurrent and transformer-based models, especially for small training sets.
1 Introduction
The paper addresses data scarcity in expert-intensive biomedical and materials-science domains by adapting simple augmentation methods to token-level NER. Experiments compare these methods across domain-specific datasets and show improvements over strong transformer baselines.
- Expert annotation is time-consuming in biomedical and materials-science domains, limiting the availability of labeled data.Source: §1 · p. 1
- Sentence-level augmentation methods manipulate words or generate artificial instances, but NER requires token-level labels that transformations may alter.Source: §1 · p. 1
- The paper surveys sentence-level augmentation techniques and adapts selected methods for NER.Source: §1 · p. 1
- Experiments compare augmentations on MaSciP and i2b2-2010, showing that simple augmentation can improve over strong baselines with large-scale pretrained transformers.Source: §1 · p. 1
2 Related Work
Prior NLP augmentation work uses lexical transformations, structural swaps, mention substitutions, and generative models. These approaches span tasks including classification, translation, question answering, dialogue understanding, and part-of-speech tagging.
Word replacement
Mention replacement
- Mention replacement substitutes entities with other entities of the same type or swaps male and female entities to address coreference bias.Source: §2 · p. 2
Swap words
- Swap-based augmentation changes word positions, applies syntactic transformations, or rotates dependency-tree fragments.Source: §2 · p. 2
Generative models
- Generative approaches create augmented data through back-translation, sequence-to-sequence generation, or bilingual dictionary and unsupervised translation methods.Source: §2 · p. 2
3 Data Augmentation for NER
The paper designs simple NER augmentations that modify tokens, mentions, or within-segment order while addressing token-label alignment. Some methods preserve labels, whereas others update BIO labels when replacements change mention structure.
- The proposed NER augmentations avoid externally trained machine-translation or syntactic-parsing models, which are difficult to train in low-resource domain-specific settings.Source: §3 · p. 2
Label-wise token replacement
- Label-wise token replacement samples replacement tokens with the same label, preserving the original label sequence.Source: §3 · p. 2
Synonym replacement
- Synonym replacement uses WordNet synonyms and derives BIO labels for multi-token synonyms based on whether the replaced token begins or continues a mention.Source: §3 · pp. 2–3
Examples
- Table 1 contrasts original and augmented instances, distinguishing methods that change only token sequences from those that may also change label sequences.Source: Table 1 · p. 3
Mention replacement
- Mention replacement swaps each mention with another training-set mention of the same entity type and changes the corresponding BIO sequence.Source: §3 · p. 3
Shuffle within segments
- Shuffle within segments divides tokens into same-label segments, optionally shuffles tokens within each segment, and keeps label order unchanged.Source: §3 · p. 3
All
- The All setting creates multiple augmented instances for each training example, with one instance produced by each augmentation method.Source: §3 · p. 3
4 Experiments and Results
The experiments evaluate simple NER augmentation across two domain-specific datasets, LSTM and BERT encoders, and progressively larger training sets. Augmentation generally improves baseline performance, with method effectiveness depending on model type and training-set size.
Experimental setup
- Experiments compare augmentation on MaSciP and i2b2-2010 using LSTM and BERT-based NER models.MaSciP covers materials-science synthesis procedures, while i2b2-2010 identifies Problems, Treatments, and Tests in patient reports.Source: §4.1 · p. 4, §4.2 · p. 4
- Evaluation uses span-level F1, with development-set model selection and five random-seed repetitions reported as means and standard deviations.Table 3’s Δ column reports averaged improvement over augmentation baselines, and underlining marks significant gains.Source: §4.2 · p. 4, Table 3 · p. 5
Results
- All augmentation techniques improve over the no-augmentation baseline, but no single method wins consistently across recurrent and transformer models.Synonym replacement performs best on average with transformers, whereas mention replacement appears most effective for recurrent models.Source: §4.4 · p. 5
- Applying all augmentation methods together outperforms individual methods on average, while single methods can perform better with complete training sets.The paper links this pattern to a trade-off between augmented-instance diversity and validity.Source: §4.4 · p. 5
- Augmentation is more effective for small training sets: all methods significantly improve results with 50 instances, whereas some reduce performance on complete sets.The small, medium, and large training sets contain 50, 150, and 500 mention-containing sentences, respectively.Source: §4.1 · p. 4, §4.4 · p. 5, Table 3 · p. 5
- The study finds significant improvements with pretrained transformer models, motivating evaluation of augmentation beyond recurrent models trained largely from scratch.The authors emphasize BERT because self-supervised pretraining is intended to capture broad knowledge.Source: §4.2 · p. 4, §4.4 · p. 5
5 Conclusion
The paper adapts simple augmentation methods from sentence-level NLP to token-level NER and evaluates them on two domain-specific datasets. The results show that simple augmentation can improve performance over strong baselines.
Conclusion
- The study adapts previously used sentence-level and sentence-pair augmentation methods to token-level NER.The adaptation addresses NER’s sequence-labeling setting, where token transformations can affect labels.Source: §5 · p. 5
- Experiments on two domain-specific datasets show that simple augmentation can improve performance over strong baselines.The conclusion covers the paper’s empirical findings without selecting a single augmentation method as universally best.Source: §5 · p. 5