Source-linked AI summary
Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction
Rujun Han, Qiang Ning, Nanyun Peng
TL;DR
Pipeline systems separate event extraction from temporal relation classification, and errors in the first stage can propagate. This paper jointly predicts events and relations with shared representations and structured inference; it reports improved end-to-end relation F1 on two benchmarks.
Problem
Existing systems treat event extraction and temporal relation classification as separate pipeline stages, and event-extraction errors can propagate to relation classification.
Source: §1 · p. 1Method
The model jointly predicts events and temporal relations using shared contextualized representations and structured prediction.
Source: §1 · p. 2Results
The model improves end-to-end temporal relation extraction F1 by 10% over CAEVO on TB-Dense and 6.8% over CogCompTime on MATRES.
Source: §1 · p. 2Takeaways & Limitations
Joint prediction can use a high-confidence NONE relation involving a non-event as a signal for the event classifier that at least one candidate is not an event.
Source: §1 · p. 2The authors identify more robust event-relation constraints, especially those considering event types, as a future research direction.
Source: §7 · p. 9
Abstract
from arXiv · showhide
We propose a joint event and temporal relation extraction model with shared representation learning and structured prediction. The proposed method has two advantages over existing work. First, it improves event representation by allowing the event and relation modules to share the same contextualized embeddings and neural representation learner. Second, it avoids error propagation in the conventional pipeline systems by leveraging structured inference and learning methods to assign both the event labels and the temporal relation labels jointly. Experiments show that the proposed method can improve both event extraction and temporal relation extraction over state-of-the-art systems, with the end-to-end F1 improved by 10% and 6.8% on two benchmark datasets respectively.
1 Introduction
Temporal relation extraction represents events as graph nodes and their temporal relations as labeled edges. The proposed joint model shares event and relation representations and predicts both together, addressing pipeline error propagation while enforcing structural constraints.
1 Introduction
- The task builds a graph whose nodes are events and whose edges encode temporal relations; non-events should have NONE relations to every other node.The example graph includes BEFORE, INCLUDES, VAGUE, and NONE relations.Source: §1 · p. 1
- Existing systems use separate event-extraction and temporal-relation-classification stages, often training the relation classifier with gold events.In that pipeline, event-extraction errors propagate to relation classification and cannot be corrected afterward.Source: §1 · p. 1
- Joint prediction can use high-confidence NONE relations involving a non-event to signal to the event classifier that at least one candidate is not an event.The paper illustrates this with the relation between Hutu and war.Source: §1 · p. 2
- The event and relation modules share contextualized embeddings and a neural representation learner, producing a graph-structured output for events and relations.The paper presents this shared representation as an improvement to event representations.Source: §1 · p. 2
- Valid graph predictions require NONE for pairs involving non-events and prohibit temporal-relation loops; ILP enforces these constraints, and SSVM trains the joint model end to end.The loop constraint follows from temporal transitivity: if A is before B and B before C, A must be before C.Source: §1 · p. 2
- 10% over CAEVO on TB-Dense and 6.8% over CogCompTime on MATRES in end-to-end temporal relation extraction F1 are reported for the proposed method.The introduction also reports ablation studies of shared representations and structured learning.Source: §1 · p. 2
2 Related Work
Prior work addressed event extraction and temporal relation extraction largely as separate tasks, while joint entity–relation extraction offered related modeling approaches. The paper proposes a neural structured prediction model to jointly extract events and relations while enforcing global structure.
2 Related Work
- The paper reports no prior joint event–temporal-relation extraction work and turns to joint entity–relation extraction as a related line of research.Examples include table-filling or global-structure approaches, beam search, and LSTM-based joint prediction; the cited discussion notes that one LSTM approach did not ensure prediction consistency.Source: §2 · p. 2
- For temporal-domain event extraction, prior systems used conventional machine-learning models with hand-engineered features, while more recent neural progress was concentrated on settings with gold events.The paper introduces a neural event extractor for this task, while noting that it is not the main contribution.Source: §2 · p. 2
- Prior temporal relation systems used local pairwise classification, linguistic and syntactic rules, structured learning, or neural methods.Examples named include CAEVO, structured-learning work by Yoshikawa et al. (2009), Ning et al. (2017), and Leeuwenberg and Moens (2017), and neural methods by Tourille et al. (2017), Cheng and Miyao (2017), Meng et al. (2017), and Meng and Rumshisky (2018).Source: §2 · p. 2
- The Figure 2 caption describes a joint structured-learning architecture in which event predictions and relation edges are represented together.The caption says grey bars denote predicted event tokens and that edges between predicted events are forced to agree.Source: Figure 2 · p. 3
- The authors describe their approach as the first end-to-end trainable neural SSVM for jointly extracting events and temporal relations with global structure enforced by ILP constraints.They motivate the model by prior joint entity–relation work.Source: §2 · p. 3
3 Joint Event-Relation Extraction Model
The model jointly assigns event and temporal-relation labels using shared neural scoring and structured prediction. Global inference and learning incorporate constraints on coherent event-relation graphs.
3.1 Neural SSVM
- The neural SSVM trains structured predictions by requiring the gold assignment to outscore the best competing assignment by a Hamming-distance margin.It uses ILP for MAP inference and balances event, relation, and regularization losses with hyperparameters.Source: §3.1 · p. 3
3.2 Multi-Tasking Neural Scoring Function
- Both tasks share contextualized BERT representations and a BiLSTM, while event and relation scorers use token-level and pair-level inputs.Relation inputs also include token distance, tense, and event polarity.Source: §3.2 · p. 4
3.3 MAP Inference
- MAP inference uses ILP during training and testing to find globally coherent event and relation assignments under one-label, consistency, symmetry, and transitivity constraints.The global objective combines scores from local event and relation scorers.Source: §3.3 · p. 4
3.3.1 Objective Function
- The global objective selects the highest-scoring assignment across event and relation candidates, with a hyperparameter controlling their relative weights.Each candidate receives one label assignment.Source: §3.3.1 · p. 4
3.3.2 Constraints
- Constraints require positive temporal relations exactly when both tokens are events, and enforce inverse labels for reversed pairs and transitivity across event triples.The relation inventory distinguishes positive labels from NONE.Source: §3.3.2 · pp. 4–5
3.4 Learning
- Training first optimizes local scoring functions with cross-entropy, then switches to global SSVM loss to adjust the network for structural properties.Direct SSVM training from scratch degraded performance.Source: §3.4 · p. 5
4 Implementation Details
The implementation compares pipeline baselines with four end-to-end model variants. The structured joint model uses global inference after a staged training procedure.
4.1 Baselines
- CAEVO and CogCompTime serve as pipeline baselines, using conventional learning algorithms and manually designed features for separate event and relation models.The authors calculate CAEVO’s event and end-to-end relation scores because Chambers et al. (2014) did not report them.Source: §4.1 · p. 5
4.2 End-to-End Event Temporal Relation Extraction
- The single-task model trains separate event and relation predictors; the multi-task variant shares the BiLSTM but still relies on event predictions to identify NONE pairs.Both are evaluated by constructing relation candidates from event-model outputs.Source: §4.2 · p. 5
- The pipeline joint model trains its relation component on event-model predictions, producing NONE pairs that help distinguish negative from positive relations.Training begins with gold events and relation candidates before switching to predicted candidates.Source: §4.2 · p. 5
- The structured joint model re-optimizes the best pipeline joint model with SSVM loss; ILP assigns event and relation labels simultaneously after filtering unlikely event candidates.The event-probability threshold restricts inference search space rather than fixing event predictions before relation construction.Source: §4.2 · pp. 5–6
- Models use cross-entropy except for SSVM training, and hyperparameters are selected on a development set for TB-Dense or an 80/20 training split for MATRES.The SSVM objective fixes C = 1 and tunes CE; Gurobi solves ILP inference.Source: §4.2 · p. 6
5 Experimental Setup
The experiments use TB-Dense and MATRES, datasets intended to improve temporal-relation annotation coverage or quality. Evaluation follows two relation-scoring conventions that differ in which labels are excluded.
5.1 Temporal Relation Data
- Table 1 summarizes dataset statistics, with its MATRES figures excluding the AQUAINT section.Source: Table 1 · p. 6, §5.1 · p. 6
- Temporal-relation corpora often have missing annotations, making both modeling and evaluation difficult; TB-Dense and MATRES are used as the benchmarks.TB-Dense examines event pairs within the same or neighboring sentences, while MATRES uses multi-axis annotation and event startpoints to improve inter-annotator agreement.Source: §5.1 · p. 6
5.2 Evaluation Metrics
- The study reports standard micro-average scores while excluding NONE, and a second metric that excludes both NONE and VAGUE.The two metrics follow conventions used in prior work.Source: §5.2 · p. 6
6 Results and Analysis
The structured joint model achieves the strongest reported end-to-end relation results on both benchmarks, and improves event extraction over baselines. Further analyses examine gold-event relation extraction, label imbalance, global constraints, and common error types.
6 Results and Analysis
- 10.0% on TB-Dense and 6.8% on MATRES are the structured joint model’s end-to-end relation F1 gains over baselines; it achieves 49.4% and 59.6% F1, respectively.The model also achieves all best-recall and F1 scores reported in Table 2.Source: §6 · p. 6, Table 2 · p. 7, §6.1 · p. 7
6.2 Event Extraction
- 3.5% on TB-Dense and 2.6% on MATRES are the structured model’s event-extraction F1 gains over baselines; its gains over single-task models are 1.3% and 0.9%, respectively.TB-Dense event improvements are difficult because the single-task model is already near 89% F1, close to the 87% inter-annotator agreement reported for TimeBank events.Source: §6.2 · p. 7, §6 · p. 7
6.3 Relation Extraction with Gold Events
- On gold events, sharing the LSTM layer improves TB-Dense relation classification by 2.6%; on MATRES, multi-task improvement over single-task is marginal, with both nearly 10% above baseline.The joint models are not evaluated on gold events because they are not trained in that setting.Source: §6.3 · p. 8
6.4 Discussion
- Neural models concentrate predictions across relation labels under training-sample imbalance, whereas CAEVO’s predictions are more evenly spread.VAGUE is difficult to distinguish from other temporal relations, and the authors say NONE compounds this challenge in end-to-end extraction.Source: §6.3 · p. 8, §6.4 · p. 9
- Up to 4× class weights significantly improve INCLUDES and IS INCLUDED F1 with less than a 2% decrease in overall F1, but excessive weights degrade those classes.Increasing SIMULTANEOUS weights up to 10× yields no correct predictions; the authors suggest more labels may be needed to improve minority-class and overall performance.Source: §6.4 · p. 8
- 0.9% and 1% are the TB-Dense and MATRES F1 gains from event-relation consistency; temporal transitivity provides no or only marginal gain.The authors hypothesize that BERT contextual embeddings may capture transitivity and that NONE pairs reduce the constraint’s usefulness.Source: Table 6 · p. 9, §6.4 · p. 9
- Some true events receive low event-module scores and are filtered out, automatically assigning NONE to their relation pairs; high-scoring non-events can instead be jointly assigned positive relations.The authors identify these as two major error types and suggest stronger event detection may help address them.Source: §6.4 · p. 9
- The authors suggest improving VAGUE performance with stronger contextual and commonsense modeling or datasets whose annotations better separate VAGUE from other positive relations.This is presented as a direction for improvement, not a demonstrated result.Source: §6.4 · p. 9
7 Conclusion
The paper presents a neural structured prediction model with joint representation learning for end-to-end event-temporal relation extraction, reporting improvements over previously published systems on two benchmarks. It identifies stronger event-relation constraints and multi-dataset event extraction as future directions.
7 Conclusion
- 10% on TB-Dense and 6.8% on MATRES are the reported improvements over previously published systems.The paper reports these results for end-to-end event-temporal relation extraction.Source: §7 · p. 9
- The proposed model jointly predicts events and temporal relations using neural structured prediction and joint representation learning to avoid pipeline error propagation.Source: §7 · p. 9
- Future work could strengthen ILP global assignments with event-aware constraints and improve event extraction by incorporating multiple datasets.Source: §7 · p. 9