Source-linked AI summary
Document-level Event Extraction via Heterogeneous Graph-based Interaction Model with a Tracker
Runxin Xu, Tianyu Liu, Lei Li, Baobao Chang
TL;DR
Document-level event extraction must recover arguments scattered across sentences and account for interdependencies among events. GIT combines a heterogeneous graph interaction network with a Tracker, and reports a 2.8 F1 improvement over previous state of the art. Further analysis finds it effective for cross-sentence extraction and multi-event scenarios.
Problem
Document-level event extraction must handle arguments scattered across sentences and the non-trivial interdependency among correlated events in a document.
Source: §1 · pp. 1–2Method
GIT models global interactions among sentences and entity mentions with a heterogeneous graph and uses a Tracker to incorporate interdependency among event records during prediction.
Source: §1 · p. 2, §3.4 · p. 4Results
Takeaways & Limitations
Abstract
from arXiv · showhide
Document-level event extraction aims to recognize event information from a whole piece of article. Existing methods are not effective due to two challenges of this task: a) the target event arguments are scattered across sentences; b) the correlation among events in a document is non-trivial to model. In this paper, we propose Heterogeneous Graph-based Interaction Model with a Tracker (GIT) to solve the aforementioned two challenges. For the first challenge, GIT constructs a heterogeneous graph interaction network to capture global interactions among different sentences and entity mentions. For the second, GIT introduces a Tracker module to track the extracted events and hence capture the interdependency among the events. Experiments on a large-scale dataset (Zheng et al., 2019) show GIT outperforms the previous methods by 2.8 F1. Further analysis reveals GIT is effective in extracting multiple correlated events and event arguments that scatter across the document. Our code is available at https://github.com/RunxinXu/GIT.
1 Introduction
Document-level event extraction must handle arguments scattered across sentences and interdependencies among correlated events. GIT addresses these challenges with a heterogeneous graph interaction network and a Tracker, and the reported experiments show improved performance.
1 Introduction
- Document-level extraction is needed because sentence-level methods miss events whose arguments occur in different sentences, while documents may contain correlated events.The arguments of one event may be scattered across sentences, and correlated event records may share arguments or provide useful reasoning information.Source: §1 · pp. 1–2
- Prior approaches either neglect cross-sentence correspondence, event interdependency, or the extraction of multiple events across long documents.The cited approaches variously query neighboring sentences, fuse sentence and entity information, or focus on a single event.Source: §1 · p. 2
- GIT uses a heterogeneous graph interaction network to model global interactions among sentences and entity mentions, capturing context for arguments scattered across sentences.The graph uses sentence-sentence, sentence-mention, intra-mention-mention, and inter-mention-mention edges.Source: §1 · p. 2
- GIT's Tracker follows extracted event records so their interdependency can be considered during prediction.The Tracker is introduced to ease extraction of correlated events.Source: §1 · p. 2
- 2.8 F1 improvement over the previous state-of-the-art is reported for GIT on the large-scale dataset.The reported gains are especially strong for cross-sentence events and multiple-event scenarios.Source: §1 · p. 2
2 Preliminaries
The paper defines document-level event extraction through entities, event types, and event records. Its focus is extracting event records, without requiring event-trigger identification.
2 Preliminaries
- An entity mention is a text span referring to an entity, an event argument is an entity assigned a role, and an event record contains arguments for an event type.Event roles are predefined for each event type; the paper uses “record” as shorthand for event record.Source: §2 · p. 2
- The task comprises entity extraction, event-type detection, and event-record extraction, which finds arguments for expressed events among document entities.An entity may have multiple mentions across the document.Source: §2 · p. 2
- Event-trigger identification is not required, reducing annotation effort and broadening the application scenarios described by the paper.Source: §2 · p. 2
3 Methodology
GIT processes document sentences into entities, models interactions between sentence and mention nodes, detects event types, and extracts records with a Tracker that uses global memory.
3.1 Entity Extraction
- Sentence-level Transformer representations and a CRF layer identify candidate entities using BIO sequence tagging.Inference decodes the label sequence with the Viterbi algorithm.Source: §3.1 · p. 3
3.2 Heterogeneous Graph Interaction Network
- A heterogeneous graph connects sentence and entity-mention nodes through four edge types, and graph convolutions model their global interactions.The edge types represent sentence-to-sentence, sentence-to-mention, within-sentence mention-to-mention, and same-entity mention-to-mention relations; removing any edge type reduces event-detection performance.Source: §3.2 · pp. 3–4
3.3 Event Types Detection
- Event types are detected as a multi-label classification task using sentence features and multi-head attention.The number of possible event types determines the attention output dimension.Source: §3.3 · p. 4
3.4 Event Records Extraction
- GIT expands an ordered argument tree to extract records and uses a Tracker’s shared global memory to incorporate information from previously extracted records.Each root-to-leaf path represents a unique record; the Tracker encodes record paths and conditions entity-candidate features for predicting the next argument role.Source: §3.4 · pp. 4–5
3.5 Training
- Training combines the losses from the three subtasks with different weights.Further training details are given in Appendix A.Source: §3.5 · p. 5
4 Experiments
GIT is evaluated on a large Chinese financial document-level event extraction dataset, with results reported for overall performance, cross-sentence records, and multi-record documents. Ablations and a case study examine the heterogeneous graph and Tracker modules.
4.1 Dataset
- The Chinese financial dataset has up to 32,040 documents, five event types, 35 argument roles, and averages 20 sentences per document.About 29% of documents express multiple events, and an event record involves roughly six sentences on average.Source: §4.1 · p. 5
4.4 Main Results
- GIT improves 2.8 micro F1 over Doc2EDAG overall, including a 4.5 improvement for Equity Underweight events.Table 1 reports test-set F1 scores by event type and overall micro F1.Source: Table 1 · p. 6, §4.4 · p. 6
4.2 Experiments Setting
- GIT uses 8- and 4-layer Transformer encoding and decoding modules, a 3-layer GCN, and Adam training for 100 epochs at a 1e−4 learning rate.The hidden and feed-forward dimensions are 768 and 1,024, respectively.Source: §4.2 · p. 6
4.3 Baselines and Metrics
- Baselines include DCFEE variants and Doc2EDAG/Greedy-Dec; all three document-level EE subtasks use F1, with event-record extraction analyzed here.DCFEE-S produces one record at a time, whereas DCFEE-M produces multiple possible argument combinations.Source: §4.3 · p. 6
4.4 Main Results
- Set IV, containing records involving the most sentences, shows GIT's largest improvement: 3.7 F1 over Doc2EDAG.The four sets are ordered by increasing average number of sentences involved in records.Source: Table 2 · p. 6, §4.4 · p. 6
- On multi-record documents, GIT surpasses strong baselines by 4.9–35.3, while multi-record extraction remains harder than single-record extraction.The Tracker is designed to use interdependency among records in this setting.Source: §4.4 · pp. 6–7, Table 3 · p. 7
4.5 Analysis
- Removing the full graph drops F1 by 2.0 (2.5 on Set IV); Tracker ablations show same- and cross-type record links matter, with its advantage growing as record counts rise.The reported graph ablation also finds a 1.0–1.4 micro F1 decrease when an edge type is removed.Source: §4.5 · pp. 7–8, Figure 4 · p. 7
4.6 Case Study
- In the Equity Pledge case, GIT recovers Record 2 arguments split between Sentences 5 and 8, using interactions across sentences and information from Record 1.Doc2EDAG fails to extract those arguments in the described example.Source: §4.6 · p. 8, Figure 5 · p. 8
5 Related Work
Prior event extraction work includes sentence-level models and document-level approaches that incorporate context in different ways. GIT distinguishes itself by combining a heterogeneous graph over sentences and mentions with Tracker-based interdependency modeling for multiple event records.
Sentence-level Event Extraction
- Sentence-level approaches include pipeline and joint trigger-argument models, dependency-tree methods, and approaches using document context, pretrained language models, external knowledge, or question answering.These studies commonly evaluate on ACE05; the passage notes difficulty extracting multiple qualified events spanning sentences.Source: §5 · p. 8
Document-level Event Extraction
- Document-level methods include feature-based cross-sentence relations, central-sentence extraction, Transformer fusion, and multi-granularity or seq2seq models.Some described approaches focus on a single event record or event type.Source: §5 · p. 8
- GIT differs by using a heterogeneous graph for global sentence-mention interactions and Tracker for global interdependency across multiple event records.The passage contrasts this with Zheng et al. (2019), which uses Transformer fusion but neglects event interdependency.Source: §5 · pp. 8–9
6 Conclusion
GIT addresses scattered event arguments and correlated events in document-level extraction with a graph interaction network and a Tracker. The paper reports a 2.8 F1 improvement over previous state-of-the-art methods.
6 Conclusion
- 2.8 F1 improvement over previous state-of-the-art methods is reported on a large-scale public dataset.Further analysis finds GIT effective for cross-sentence event extraction and multi-event scenarios.Source: §6 · p. 9
- GIT uses a heterogeneous graph interaction network to model global interactions among sentences and entity mentions.A Tracker follows extracted records to consider their global interdependency during extraction.Source: §6 · p. 9
- Document-level event extraction must handle arguments scattered across sentences and multiple correlated events expressed in one document.GIT is proposed to address both challenges.Source: §6 · p. 9
A Training Details
Training uses scheduled sampling to gradually replace gold entities with model predictions, with experiments run on the same hardware and dependency versions. The appendix also reports limited hyperparameter search and entity-extraction results.
A Training Details
- From epochs 10 to 20, scheduled sampling linearly raises predicted-entity use from 0% to 100%.This is adopted to mitigate error propagation from the training–inference gap.Source: A Training Details · p. 10
- All experiments, including baselines, use the same eight Tesla-V100 GPUs and Python dependency versions.Source: A Training Details · p. 10
- The authors report little effort to tune GIT hyperparameters; Table 6 lists the trials and marks the adopted values in bold.Source: A Training Details · p. 10, Table 6 · p. 10
- Test checkpoints are selected by dev-set Micro F1 performance.Table 9 identifies each model’s best dev-set epoch and corresponding score.Source: A Training Details · p. 11
- Entity-extraction performance is similar across models, which use the same structure and methods for entity extraction.Source: Table 7 · p. 11
B Additional Evaluation Results
The appendix reports additional results for entity extraction, event-type detection, and event-record extraction, including comparisons with results reported by Zheng et al. (2019).
B Additional Evaluation Results
- Event-record extraction results are summarized comprehensively in Table 10, including precision, recall, and F1 scores.Source: B Additional Evaluation Results · p. 11
- The appendix also presents entity-extraction results in Table 7.Source: B Additional Evaluation Results · p. 11
- Event-type detection results are presented in Table 8.Source: B Additional Evaluation Results · p. 11
C Complete Document for the Examples
The appendix provides complete document versions of the Figure 1 running example and the Figure 5 case study. It also identifies additional evaluation tables and their reported comparisons.
C Complete Document for the Examples
- All models exceed 90.0 Micro F1 for event-type detection, and GIT slightly outperforms Doc2EDAG.Source: Table 8 · p. 12
- Table 9 identifies the best dev-set epoch for each model and its corresponding performance.Source: Table 9 · p. 12
- Figure 6 presents the original complete document corresponding to the Figure 5 case study.Source: C Complete Document for the Examples · p. 11, Figure 6 · p. 12
- GIT consistently outperforms other baselines in the comprehensive event-record extraction results summarized in Table 10.Source: Table 10 · p. 13
- Figure 7 presents the original complete document for Figure 1’s running example; the paper says its Figure 1 rendering is translated from Chinese and simplified.For argument-role meanings, the paper directs readers to Zheng et al. (2019).Source: C Complete Document for the Examples · p. 11, Figure 7 · p. 14