Source-linked AI summary
TPLinker: Single-stage Joint Extraction of Entities and Relations Through Token Pair Linking
Yucheng Wang, Bowen Yu, Yueyang Zhang, Tingwen Liu, Hongsong Zhu, Limin Sun
TL;DR
Existing joint extraction models can struggle with overlapping relations and exposure bias because sequential steps use ground-truth conditions during training but must extract independently at inference. TPLinker converts the task into one-stage token pair linking with a handshaking tagging scheme. On two public datasets, it achieves state-of-the-art performance and improves results on overlapping and multiple-relation extraction.
Problem
Sequential joint extraction models suffer exposure bias and face difficulty identifying overlapping relations that share one or both entities.
Method
TPLinker is a one-stage joint extraction model that uses token pair linking and relation-specific handshaking tags to align entity-boundary tokens.
Results
TPLinker outperforms all baselines and achieves state-of-the-art results on two public datasets, with improvements on overlapping and multiple-relation cases.
Takeaways & Limitations
The results support TPLinker’s capability to handle overlapping and multiple relations while narrowing the gap between training and inference.
Takeaways & Limitations
On WebNLG⋆, similar meanings among relations and missing correct annotations hurt TPLinker’s precision.
Abstract
from arXiv · showhide
Extracting entities and relations from unstructured text has attracted increasing attention in recent years but remains challenging, due to the intrinsic difficulty in identifying overlapping relations with shared entities. Prior works show that joint learning can result in a noticeable performance gain. However, they usually involve sequential interrelated steps and suffer from the problem of exposure bias. At training time, they predict with the ground truth conditions while at inference it has to make extraction from scratch. This discrepancy leads to error accumulation. To mitigate the issue, we propose in this paper a one-stage joint extraction model, namely, TPLinker, which is capable of discovering overlapping relations sharing one or both entities while immune from the exposure bias. TPLinker formulates joint extraction as a token pair linking problem and introduces a novel handshaking tagging scheme that aligns the boundary tokens of entity pairs under each relation type. Experiment results show that TPLinker performs significantly better on overlapping and multiple relation extraction, and achieves state-of-the-art performance on two public datasets.
1 Introduction
Joint entity-relation extraction addresses the limitations of pipelined and sequential models, which can overlook subtask interactions, struggle with overlap, and suffer exposure bias. TPLinker uses one-stage token pair linking and reports stronger performance, especially for overlapping and multiple relations.
- Pipelined approaches separate entity detection from relation classification, ignoring their interaction and becoming susceptible to cascading errors.
- Joint models integrate entity and relation information, but existing approaches struggle to identify relations in which one entity participates in multiple relations.
- Decoder-based and decomposition-based models create a training-inference mismatch by relying on ground-truth context during training but model-generated inputs during inference.
- TPLinker formulates joint extraction as token pair linking and uses three link matrices per relation to identify entity boundaries and relation-specific entity pairs.
- TPLinker avoids interdependent extraction steps, removing dependence on ground-truth conditions during training and making training and testing more consistent.
- TPLinker outperforms prior work on two public datasets and significantly improves extraction for Normal, SEO, EPO, and multiple-relation cases.
2 Related Work
Prior joint extraction methods include feature-based, neural, tagging, decoder-based, and cascade approaches, but overlapping relations remain difficult. Existing methods have specific limitations in handling SEO, EPO, or multi-word entities.
- Feature-based joint extraction methods require complicated feature engineering and depend heavily on NLP tools for feature extraction.
- Some existing methods recognize SingleEntityOverlap relations but fail to extract EntityPairOverlap triplets.
- Sequence-to-sequence approaches address overlapping relations but fail to generate multi-word entities.
- Cascade binary tagging first identifies possible subject entities, then predicts relations and object entities for each subject, decomposing overlapping extraction into stages.
3 Methodology
TPLinker converts joint entity–relation extraction into token-pair linking using handshaking tag sequences, enabling one-stage decoding of overlapping relations.
- 3.1.1 Tagging: TPLinker aligns entity-boundary token pairs under each relation type to formulate joint extraction as token pair linking.The method uses entity head-to-tail, subject-head-to-object-head, and subject-tail-to-object-tail links.
- 3.1.1 Tagging: The handshaking scheme enumerates token pairs, tags their links in matrices, and flattens the retained upper-triangular entries into sequences.Lower-triangular reversal links are mapped to tag 2 before the remaining entries are flattened.
- 3.1.1 Tagging: The tagging design naturally handles SingleEntiyOverlap and nested entities, while separate matrices for relation types address EntityPairOverlap.Entity extraction links are shared across relations because they do not depend on relation type.
- 3.1.2 Decoding: Decoding first recovers entity spans and tail links, then combines head and tail links for each relation to produce the predicted triplet set.The decoder stores entities by head position and checks compatible subject–object boundary pairs.
- 3.2 Token Pair Representation: Token-pair representations are generated from contextual token vectors by a learned Handshaking Kernel, then classified into link labels.The token vectors come from a basic encoder, and the kernel uses a parameter matrix and bias vector.
4.1 Datasets
Experiments use NYT and WebNLG datasets, with variants distinguished by entity-span annotation standards and test-set overlap patterns.
- 4.1 Datasets: The experiments use the public NYT and WebNLG datasets, each represented with partial-span and whole-span annotation versions.The first version annotates the last word of entities, whereas the second annotates the whole entity span.
- 4.1 Datasets: The test sets are split into Normal, SPO, and EPO categories to analyze overlapping relations and triplet multiplicity.Normal contains sentences without overlapping triples; SPO means Single Pair Overlapping and EPO means Entity Pair Overlapping.
4.2 Evaluation
Evaluation follows established matching conventions and reports micro-averaged precision, recall, and F1-score against baseline systems.
- 4.2 Evaluation: NYT⋆ and WebNLG⋆ use Partial Match, while NYT and WebNLG use Exact Match for extracted triplets.Partial Match requires correct relation and entity heads; Exact Match requires the complete subject and object spans.
- 4.2 Evaluation: The evaluation reports standard micro Precision, Recall, and F1-score in line with the baselines.These metrics are reported for comparison with previous work.
4.3 Implementation Details
TPLinker is implemented in PyTorch, optimized with Adam, and evaluated with both GloVe–BiLSTM and BERT encoders.
- 4.3 Implementation Details: TPLinker uses PyTorch with Adam optimization and compares a GloVe-plus-2-layer-BiLSTM encoder against base cased English BERT.The BiLSTM hidden dimensions are 300 and 600, with dropout rate 0.1 on embeddings and hidden states.
4.4 Comparison Models
The comparison includes tagging, encoder-decoder, multi-head, graph-based, reinforcement-learning, and span-based baselines for joint entity-relation extraction.
- NovelTagging converts joint extraction into sequence labeling by incorporating entity and relation roles.
- CopyRE generates all triples with an encoder-decoder model but can copy only the last word of an entity.
- MultiHead formulates extraction as multi-head selection after identifying candidate entities.
- GraphRel uses graph convolutional networks and splits entity mention pairs into word pairs for overlapping-relation prediction.
- OrderCopyRE extends CopyRE with reinforcement learning, while ETL-Span uses span tagging and hierarchical triplet decoding.
4.5 Experimental Results and Analysis
TPLinker outperforms the evaluated baselines, particularly on overlapping or multiple relations, while also offering faster inference; dataset annotation and relation-label issues qualify some comparisons.
- Main Results: 2.3 percentage points on NYT⋆, 14.0 on NYT, and 3.6 on WebNLG separate TPLinkerBERT from state-of-the-art models in F1-score.
- Main Results: TPLinker performs well under both last-word and whole-span entity annotation, but WebNLG and WebNLG⋆ differ substantially.
- Main Results: The WebNLG comparison retains 127 meaningless empty-entity triplets and differing relation inventories, because the experiments do not correct these issues.
- Different Sentence Types: 6.3 percentage points is TPLinker’s advantage over CasRel on sentences containing at least five relations.
- Different Sentence Types: TPLinker improves most on harder sentences with overlapping relations or more than two relations, while CasRel remains competitive on simpler subsets.
- Computational Efficiency: 3.6 times is TPLinkerBERT’s approximate inference-speed advantage over CasRelBERT, with batch processing enabled by its one-stage design.
5 Conclusion
TPLinker is presented as a one-stage token-pair-linking model for overlapping entity-relation extraction, achieving state-of-the-art results on two public datasets.
- TPLinker converts joint entity-relation extraction into a token pair linking game using a novel handshaking tagging strategy.
- The authors describe TPLinker as the first one-stage model capable of extracting all kinds of overlapping relations without exposure bias.
- TPLinker outperforms all baselines and achieves state-of-the-art performance on two public datasets.
- Further analysis demonstrates TPLinker’s capabilities on sentences containing overlapping or multiple relations.