Source-linked AI summary
A Novel Cascade Binary Tagging Framework for Relational Triple Extraction
Zhepei Wei, Jianlin Su, Yue Wang, Yuan Tian, Yi Chang
TL;DR
Relational triple extraction is important for knowledge-graph construction but existing methods struggle with overlapping triples that share entities. CasRel formulates relations as subject-to-object functions and implements this idea with cascade binary tagging. It outperforms state-of-the-art methods, including by 17.5% and 30.2% F1-score on NYT and WebNLG, respectively.
Problem
Existing relational triple extraction methods struggle with overlapping triples because multiple relations can share entities in one sentence.
Method
CasRel uses a principled cascade binary tagging framework that models each relation as a function mapping subjects to objects.
Results
17.5% and 30.2% F1-score improvements over the best state-of-the-art method are achieved on NYT and WebNLG, respectively.
Takeaways & Limitations
The framework handles overlapping triples by design and improves consistently across different overlapping scenarios.
Abstract
from arXiv · showhide
Extracting relational triples from unstructured text is crucial for large-scale knowledge graph construction. However, few existing works excel in solving the overlapping triple problem where multiple relational triples in the same sentence share the same entities. In this work, we introduce a fresh perspective to revisit the relational triple extraction task and propose a novel cascade binary tagging framework (CasRel) derived from a principled problem formulation. Instead of treating relations as discrete labels as in previous works, our new framework models relations as functions that map subjects to objects in a sentence, which naturally handles the overlapping problem. Experiments show that the CasRel framework already outperforms state-of-the-art methods even when its encoder module uses a randomly initialized BERT encoder, showing the power of the new tagging framework. It enjoys further performance boost when employing a pre-trained BERT encoder, outperforming the strongest baseline by 17.5 and 30.2 absolute gain in F1-score on two public datasets NYT and WebNLG, respectively. In-depth analysis on different scenarios of overlapping triples shows that the method delivers consistent performance gain across all these scenarios. The source code and data are released online.
1 Introduction
Relational triple extraction supports knowledge-graph construction but remains difficult when triples overlap in shared entities. CasRel addresses this through a principled subject-to-object formulation and cascade binary tagging, achieving large gains over prior methods.
- Motivation: Relational triples are central knowledge-graph facts, represented as subject–relation–object tuples extracted from natural-language text.The paper motivates extraction as a crucial step toward constructing large-scale knowledge graphs.
- Motivation: Overlapping triples challenge sequence tagging and relation classification because shared entities violate their usual single-tag or single-relation assumptions.These patterns include triples sharing one or two entities, while relation classification also faces severe class imbalance and confusion among multiple valid relations.
- Core formulation: CasRel models each relation as a function mapping subjects to objects rather than as a discrete label assigned to an entity pair.Extraction first identifies possible subjects, then applies relation-specific taggers to identify all corresponding objects.
- Model: The proposed framework is instantiated as a cascade binary tagging model built on a Transformer encoder, combining its tagging framework with pre-trained language-model knowledge.Its modules include a BERT-based encoder, subject tagging, and relation-specific object tagging.
2 Related Work
Prior relational triple extraction methods commonly use pipelines or joint models, but overlapping triples remain insufficiently addressed. CasRel instead models relations as subject-to-object functions within a jointly decoded framework.
- Task background: Relational triple extraction is an important information-extraction task for constructing large-scale knowledge graphs such as DBpedia, Freebase, and Knowledge Vault.The task is also described as well studied in information extraction.
- Existing approaches: Earlier pipelines recognize entities before classifying entity-pair relations, but errors can propagate and the stages neglect their relevance.Feature-based and neural joint models reduce some limitations, although some still pipeline detected entity pairs into relation classification.
- Overlapping triples: Most previous methods ignore overlapping triples, while early dedicated approaches use sequence-to-sequence copying or graph convolutional networks but still treat relations as discrete labels.Overlapping triples include patterns where multiple triples share entities.
- CasRel formulation: CasRel jointly decodes entities and relations from a triple-level objective while representing relations as functions mapping subjects to objects.This differs fundamentally from classifying relations for subject–object pairs.
3 The CASREL Framework
CASREL formulates relational triple extraction directly at the triple level and decomposes it into subject detection followed by relation-specific object tagging. This cascade tagging design handles overlapping triples without assumptions about shared entities and uses BERT representations to condition extraction on sentence context and each detected subject.
- 3 The CASREL Framework: CASREL models the training likelihood directly over relational triples, avoiding assumptions about how multiple triples share entities and motivating its cascade tagging scheme.The factorization learns a subject tagger and, for each relation, an object tagger conditioned on the subject and sentence.
- 3.1 BERT Encoder: BERT encodes each input sentence with bidirectional Transformer representations, using subword and positional embeddings while omitting segmentation embeddings for single-sentence inputs.The resulting contextual token representations feed the tagging modules.
- 3.2 Cascade Decoder: The cascade decoder first detects all candidate subjects, then applies relation-specific object taggers to each subject to identify associated objects and relations.All relation-specific object taggers operate for each detected subject, enabling multiple triples to be extracted in one pass over the candidates.
- 3.2 Cascade Decoder: The subject tagger assigns separate binary start and end tags to tokens, using thresholded probabilities from BERT-encoded token representations to identify subject spans.Multiple subject spans are formed by matching each start token with the nearest subsequent end token.
- 3.2 Cascade Decoder: Figure 2 illustrates that high-level binary tags change across subject iterations, so different candidate subjects yield different relational triples.For example, the second iteration represents the triple (Washington, Capital of, United States Of America).
- 3.2 Cascade Decoder: For each detected subject, the object tagger incorporates subject features and predicts object start and end positions separately for every relation.The same decoding process is iteratively applied to each detected subject, with subject representations combined with token representations.
4 Experiments
Experiments on NYT and WebNLG evaluate CASREL against strong baselines for relational triple extraction, including overlapping and increasingly complex sentences. CASREL consistently outperforms prior methods, with the largest gains in difficult overlapping settings.
- Main Results: 17.5% and 30.2% F1-score improvements over the best state-of-the-art method are achieved on NYT and WebNLG, respectively.CASREL outperforms all baselines on Precision, Recall, and F1-score; CASRELrandom and CASRELLSTM remain competitive without pre-trained BERT.
- Dataset Analysis: The NYT–WebNLG performance gap for existing models is attributed to their difficulty handling overlap, because WebNLG contains more EPO and SEO sentences.NYT is mainly Normal, whereas WebNLG is dominated by the harder overlapping classes.
- Different Overlapping Patterns: CASREL remains consistently strong across Normal, EPO, and SEO overlap patterns, whereas most baselines degrade as overlap difficulty increases.Normal is easiest for baselines, while EPO and SEO are harder; CASREL performs especially strongly on the hard patterns.
- Different Numbers of Triples: CASREL achieves excellent F1-score across all five classes defined by the number of triples in a sentence.Its greatest improvements over CopyRRL occur in the most difficult class, N≥5, on both datasets.
- Different Numbers of Triples: CASREL suffers the least from increasing sentence complexity and is therefore better suited to complicated multi-triple scenarios than the baselines.The results support the framework’s ability to extract multiple, possibly overlapping, relational triples from complex sentences.
5 Conclusion
The paper introduces CASREL, a cascade binary tagging framework based on modeling relations as functions mapping subjects to objects. This formulation enables simultaneous extraction of multiple relational triples from a sentence.
- CASREL models relations as functions mapping subjects to objects rather than as discrete labels for entity pairs.The framework is derived from a principled problem formulation for relational triple extraction.
- CASREL can simultaneously extract multiple relational triples from sentences without suffering from the overlapping-triple problem.
A Implementation Details
The implementation uses mini-batch training, validation-based hyperparameter selection, and early stopping. It also fixes the input length and tagger threshold settings, while noting that threshold tuning is outside the paper’s scope.
- Training uses batch size 6, learning rate 1e−5, validation-selected hyperparameters, and early stopping after 7 consecutive epochs without validation improvement.
- The model uses 12 stacked bidirectional Transformer blocks with hidden-state size 768.
- The input length is capped at 100 words for comparison with previous work, and start/end tag thresholds are fixed at 0.5 without tuning.The paper states that careful threshold tuning might improve performance but is outside its research scope.
B Error Analysis
The error analysis compares CASREL’s performance across relational-triple elements and finds different difficulty patterns on NYT and WebNLG. NYT errors are more associated with entity identification, whereas WebNLG errors are more associated with relation identification.
- Error Analysis: Table 4 evaluates performance on individual and combined triple elements, including E1, R, E2, (E1, R), (R, E2), and the full triple.E1 denotes the subject, E2 the object, and R the relation; a combined element is correct only when all included components are correct.
- Error Analysis: On NYT, the small F1 gap between (E1, E2) and the full triple contrasts with larger gaps involving relation-entity combinations, indicating relation identification is easier than entity identification.
- Error Analysis: On WebNLG, the larger gap between (E1, E2) and the full triple indicates that relation misidentification causes more degradation than entity misidentification.
- Error Analysis: The WebNLG pattern indicates that CASREL has greater difficulty identifying relations than entities, opposite to the NYT observation.
C Supplemental Experiments
Supplemental experiments compare CASREL with prior methods across datasets, evaluation metrics, and dataset conditions. They show that annotation quality, preprocessing choices, and metric selection materially affect how performance should be interpreted.
- Evaluation settings: Partial Match is used for ACE04, NYT10-HRL, and NYT11-HRL, while Exact Match is used for Wiki-KBP because prior methods do not support a unified metric.Some methods cannot handle multi-word entities or are not open-source, making unified comparison difficult.
- Dataset study: The supplemental datasets are poorly suited to testing overlapping triples because their test sentences contain few such cases.Table 6 reports dataset statistics for these conditions.
- Dataset study: Imperfect annotations can omit valid triples, causing precision and recall to underestimate CASREL’s actual extraction capability.In one NYT11-HRL example, the annotations contain one triple although the model extracts three, including two omitted valid triples.
- Dataset study: NYT11-HRL results only partially reflect CASREL’s potential because imperfect annotations and preprocessing choices can distort evaluation.The paper nevertheless reports competitive performance on this dataset.
- Dataset study: NYT11-HRL performance differs sharply across preprocessing versions, with F1-score ranging from 53.9 to 89.6.The versions differ in test-set construction, overlap prevalence, and treatment of multi-word entities, so their results are not directly equivalent.