Source-linked AI summary
Adversarial training for multi-context joint entity and relation extraction
Giannis Bekoulis, Johannes Deleu, Thomas Demeester, Chris Develder
TL;DR
Joint entity and relation extraction needs robust models that avoid costly feature engineering and external NLP tools. This paper adds adversarial training to a neural joint-extraction baseline and evaluates it across contexts and languages, finding consistent improvements over the baseline. Overall F1 gains range from approximately 0.4% to 0.9%, although the NER component benefits only modestly.
Problem
Joint extraction requires models that can perform NER and relation extraction together without relying on external parsers or manually extracted features.
Method
The paper applies adversarial training to a neural joint model that combines entity recognition with multi-label relation extraction.
Results
AT consistently improves the baseline’s joint extraction performance across datasets, contexts, and languages, with overall F1 gains ranging from approximately 0.4% to 0.9%.
Takeaways & Limitations
AT improves each task separately and overall joint-model performance while reaching high performance early in training.
Takeaways & Limitations
The improvement is mainly limited by modest gains for the NER component, while relation extraction increases by approximately 1% F1 except on ACE04.
Abstract
from arXiv · showhide
Adversarial training (AT) is a regularization method that can be used to improve the robustness of neural network methods by adding small perturbations in the training data. We show how to use AT for the tasks of entity recognition and relation extraction. In particular, we demonstrate that applying AT to a general purpose baseline model for jointly extracting entities and relations, allows improving the state-of-the-art effectiveness on several datasets in different contexts (i.e., news, biomedical, and real estate data) and for different languages (English and Dutch).
1 Introduction
The paper introduces adversarial training for jointly extracting entities and relations, addressing robustness in a related-task setting. It evaluates the approach across multiple contexts and languages, finding consistent additional gains over a strong baseline.
- Motivation: Adversarial training regularizes neural models with small perturbations and had not previously been investigated for joint entity and relation extraction.The paper frames AT as using clean and adversarial examples to improve robustness and identifies its joint-task study as the first of its kind to the authors’ knowledge.
- Baseline and contribution: The baseline jointly performs named entity recognition and relation extraction while using automatically extracted features without external parsers or manually designed features.It extracts all entities and corresponding relations at once and supports multiple relations per entity through multi-label modeling.
- Evaluation: The study evaluates AT on datasets spanning news, biomedical, and real-estate contexts in English and Dutch.The experiments use a strong baseline that already outperforms previous models relying on automatically extracted features.
- Evaluation: Applying AT during training consistently increases joint extraction effectiveness beyond the baseline model.The reported gains are observed across the study’s diverse datasets and settings.
2 Related work
Prior joint extraction systems used manual features, external NLP tools, or restricted entity-pair formulations. Neural methods reduced feature-engineering demands, while adversarial training extended robustness-oriented regularization to several NLP tasks and motivated its use here.
- Joint extraction: Earlier joint models performed NER and relation extraction together but depended on manually extracted features or external NLP tools.These dependencies introduced additional complexity.
- Neural approaches: Neural approaches addressed feature-design issues using architectures such as recurrent and convolutional neural networks.Some systems still used external dependency parsers, manually extracted features, or entity-pair replication.
- Adversarial training: Adversarial training was introduced for robustness to input perturbations and later applied to text classification, relation extraction, and POS tagging.Unlike dropout-style methods that add random noise, AT generates perturbations that are connected to the model’s behavior.
3 Model
The model jointly represents tokens with word and character embeddings, processes them with a BiLSTM, predicts entity tags with a CRF, and extracts relations through multi-label head selection. Adversarial training perturbs concatenated word representations with worst-case noise and trains on both original and perturbed examples.
- Joint model: The baseline detects entity types and boundaries together with relations between entities from a token sequence.Character embeddings capture morphological information, while pre-trained word embeddings provide additional token representations.
- Joint model: Word and character representations feed a BiLSTM, followed by CRF-based entity prediction and a relation extraction layer.The figure presents these as the model’s main processing components.
- Relation extraction: Relation extraction is formulated as multi-label head selection, allowing each word to participate in multiple relations with other words.The model predicts head and relation vectors using sigmoid probabilities and threshold-based decoding.
- Adversarial training: Adversarial training adds worst-case perturbations to concatenated word representations to make the model robust to input perturbations.The perturbation is chosen to maximize the joint loss under a bounded norm and is approximated using the loss gradient.
- Adversarial training: Training minimizes the combined loss on the original and adversarial examples.The final objective sums LJOINT(w; θ̂) and LJOINT(w + ηadv; θ̂).
4 Experimental setup
The experiments evaluate the models across four datasets using cross-validation, multiple entity-scoring settings, and F1 comparisons with prior work.
- Datasets and splits: Experiments cover ACE04, CoNLL04, and DREC datasets using cross-validation procedures defined or aligned with prior studies.The supplied setup specifies 5-fold cross-validation for ACE04, shared splits for CoNLL04 entity classification, 10-fold cross-validation for CoNLL04 NER, and evaluation on DREC.
- Evaluation: Entity evaluation includes strict, boundary-only, and relaxed settings, while relations require correct relation types and argument entities.Strict scoring requires correct entity boundaries and types; boundary scoring ignores entity type; relaxed scoring applies when boundaries are known.
- Comparisons: Table 1 compares baseline, baseline EC, and baseline (EC) + AT models using F1 scores for the two subtasks and their average performance.The table also distinguishes models using external NLP tools and includes S, R, and B evaluation types.
5 Results
The baseline performs strongly across contexts, while adversarial training consistently improves joint extraction, with overall F1 gains ranging from approximately 0.4% to 0.9%.
- Baseline results: The baseline exceeds or closely matches prior systems across ACE04, ADE, and CoNLL04, including a ∼2% ACE04 gain over Katiyar and Cardie (2017).The baseline also reports a 2.5% overall improvement on ADE and more than 4% improvement over models without manually extracted features on CoNLL04.
- Adversarial-training results: AT improves joint performance in every experiment, with overall F1 gains of 0.4% on ACE04, 0.4% and 0.8% on CoNLL04, ∼1% on DREC, and 0.7% on ADE.The improvements apply across the reported entity-classification and NER settings, with DREC showing approximately 1% gains in both settings.
- Training dynamics: AT models approach maximum validation performance earlier than baseline models, with DREC reaching maximum validation performance from the first epochs.Figure 2 presents smoothed validation F1 trends and 95% confidence intervals from 10–30 epochs onward, depending on the dataset.
- Magnitude and scope of gains: AT improves overall F1 over the baseline by approximately 0.4% to 0.9%, while relation extraction gains are approximately 1% except on ACE04.The authors attribute the smaller overall gains partly to limited benefits for the NER component and note especially small CoNLL04 evaluation-set improvement.
6 Conclusion
The study applies adversarial training to joint entity recognition and relation extraction and evaluates its effectiveness across multiple contexts. Experiments show improvements for each task and for overall joint performance, with high performance reached early in training.
- Contribution: The study proposes adversarial training as a regularization method for joint entity recognition and relation extraction across multiple contexts.Its contribution combines investigating AT over a multi-context baseline with a large-scale experimental evaluation.
- Conclusion: AT improves each task separately and the overall baseline joint model while reaching high performance during the first training epochs.This conclusion summarizes the reported experimental findings across the evaluated settings.