PaperlayerView exact revision on arXiv

Source-linked AI summary

Document-Level Event Role Filler Extraction using Multi-Granularity Contextualized Encoding

Xinya Du, Claire Cardie

arXiv:2005.06579v1cs.CL

TL;DR

Document-level event extraction needs broader context to identify role fillers across sentences, yet neural sequence models face long-sequence challenges. This paper investigates end-to-end models and context length, and proposes a reader that combines sentence- and paragraph-level representations. On MUC-4, the reader substantially improves over prior work.

  • Problem

    Few event-extraction studies make decisions beyond individual sentences, although role-filler evidence can be spread across sentences.

    Source: §1 · pp. 1–2

  • Method

    The paper studies end-to-end neural sequence tagging and context length, and proposes a reader that dynamically combines sentence- and paragraph-level representations.

    Source: §1 · p. 2, §6 · p. 9

  • Results

    The multi-granularity reader achieves substantial improvements over baseline models and prior work on MUC-4.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    Document-level role filler extraction can be successfully tackled with end-to-end neural sequence models, while very long context may lower performance.

    Source: §6 · p. 9

    The reader overlooks some role fillers, including implicit roles for which external knowledge could help confirm the event role.

    Source: §5 · p. 9

Abstract

from arXiv · show

Few works in the literature of event extraction have gone beyond individual sentences to make extraction decisions. This is problematic when the information needed to recognize an event argument is spread across multiple sentences. We argue that document-level event extraction is a difficult task since it requires a view of a larger context to determine which spans of text correspond to event role fillers. We first investigate how end-to-end neural sequence models (with pre-trained language model representations) perform on document-level role filler extraction, as well as how the length of context captured affects the models' performance. To dynamically aggregate information captured by neural representations learned at different levels of granularity (e.g., the sentence- and paragraph-level), we propose a novel multi-granularity reader. We evaluate our models on the MUC-4 event extraction dataset, and show that our best system performs substantially better than prior work. We also report findings on the relationship between context length and neural model performance on the task.

1 Introduction

The introduction frames document-level role filler extraction as span identification requiring both sentence-level understanding and broader context. It motivates end-to-end neural sequence tagging, investigates context length, and proposes a multi-granularity reader that substantially improves over baselines and prior work.

1 Introduction

  • Document-level extraction identifies spans denoting event-specific role fillers, but the complete task can also involve coreference resolution and event tracking.This work focuses only on role filler extraction.Source: §1 · p. 1
  • Earlier pipeline methods use separate role and context classifiers, risking error propagation and requiring manually engineered, domain-specific features.Examples include lexico-syntactic patterns and lexical or discourse bridge features.Source: §1 · p. 2
  • Long documents challenge neural models because RNNs struggle to scale to very long sequences and BERT limits input length below that of many event articles.The paper studies context length and proposes dynamically aggregating sentence- and paragraph-level representations.Source: §1 · p. 2
  • The multi-granularity reader dynamically combines local and broader contextual representations and substantially improves over baseline models and prior work on MUC-4.The paper also investigates how input context length affects performance.Source: §1 · p. 2

2 Related Work

Prior work spans sentence-level extraction and document-context methods, but much of it either stops at sentence-level decisions or uses pipeline approaches with hand-designed features. This paper instead treats role filler extraction as supervised, end-to-end sequence tagging with learned local and cross-sentence context.

2 Related Work

  • Sentence-level event extraction methods target triggers and arguments within individual sentences and rarely generalize to document-event extraction.Examples include hand-designed features, neural models, and pretrained contextualized representations.Source: §2 · p. 2
  • Some document-context approaches make cross-sentence decisions, but related event extraction work still performs sentence-level extraction.Other approaches use document embeddings to aid event detection, or jointly extract events and entities.Source: §2 · pp. 2–3
  • The full MUC-style document task includes role filler extraction, mention coreference resolution, and event tracking, while this work addresses role filler extraction.The task is also described as template filling.Source: §2 · pp. 2–3
  • The paper learns intra-sentence and paragraph-level representations in an end-to-end sequence-tagging approach, rather than relying on manually designed features or pipeline stages.Its supervised models use predefined event schemas.Source: §2 · p. 3
  • Cross-sentence relation extraction assumes mentions are provided, whereas this work extracts role-filler spans and determines their types during sequence tagging.The paper also considers long-context modeling with LSTM layers and pretrained BERT representations.Source: §2 · p. 3

3 Methodology

Document-level role-filler extraction is framed as BIO sequence tagging over contiguous sentences. The proposed multi-granularity reader combines sentence- and paragraph-level token representations for labeling.

3.1 Constructing Paired Token-tag Sequences

  • Documents and gold filler spans are converted to BIO-tagged token sequences, and readers are trained on sequences of varied context lengths.Source: §3.1 · pp. 3–4
  • Training uses overlapping contiguous-sentence sequences with balanced positive and negative examples, while test sequences are grouped in order without overlap.For paragraph readers, training uses average paragraph length and testing uses each paragraph's actual length.Source: §3.1 · p. 4
  • Each token combines a fixed GloVe word embedding with frozen BERT-base representations averaged across all 12 layers, then passes through a three-layer bidirectional LSTM.Source: §3.2 · pp. 4–5
  • A conditional random field jointly scores tag sequences, maximizes the gold sequence's log probability during training, and selects the highest-scoring sequence at decoding.Source: §3.2 · p. 5
  • Using BERT's last-layer representations or summing its 12 layers gave consistently worse results than the selected representation.Source: §3.2 · p. 5

3.3 Multi-Granularity Reader

  • The multi-granularity reader builds sentence- and paragraph-level representations for each token, then combines them through sum or gated fusion.The gate controls how much information is incorporated from each granularity.Source: §3.3 · pp. 5–6

4 Experiments and Analysis

Experiments on MUC-4 compare document-level neural readers with prior systems and examine how context length affects performance. The multi-granularity reader performs significantly better than non-end-to-end systems and base k-sentence readers on macro-average F-1.

4.1 Dataset and Evaluation Metrics

  • MUC-4 contains 1,700 documents; the comparable split uses 1,300 for training, 200 for development, and 200 for testing.Evaluation reports macro-averaged precision, recall, and F-1 using head-noun matching, plus exact-match accuracy; duplicate fillers are conflated before scoring.Source: §4.1 · p. 6

4.2 Baseline Systems and Our Systems

  • The context-length comparison varies k across single-sentence, double-sentence, paragraph, and longest-feasible chunk readers; the multi-granularity reader combines intra- and inter-sentence representations.The multi-granularity reader processes documents paragraph by paragraph.Source: §4.2 · p. 7

4.3 Results and Findings

  • Contextualized k-sentence readers outperform their non-contextualized counterparts, especially when k > 1; transformer parameters are frozen during training.Fine-tuning the transformers yields worse results.Source: §4.3 · p. 7
  • Paragraph-level context raises F-1 over two-sentence context, but longer input makes readers more conservative and significantly lowers F-1.Moving from one to two sentences improves precision but lowers recall, yielding no F-1 gain; paragraph context raises precision while recall stays similar.Source: §4.3 · p. 7
  • The multi-granularity reader significantly outperforms non-end-to-end systems and base k-sentence readers on macro-average F-1.The reported comparison is summarized in Table 1.Source: §4.3 · p. 7, Table 1 · p. 8

5 Further Analysis

The analysis examines how model components affect extraction and illustrates where readers succeed or miss role fillers. The examples also show that some implicit roles remain difficult to recover.

5 Further Analysis

  • Replacing gated fusion with a simple sum lowers precision and F-1; removing BERT lowers recall and F-1, while independent token labeling lowers precision and recall.The ablation study also reports that the model becomes more conservative without BERT’s contextualized representations.Source: §5 · p. 8, Table 3 · p. 9
  • The multi-granularity and single-sentence readers extract targets that the paragraph reader misses when the attack and targets appear only in the last sentence.The multi-granularity reader uses both paragraph-level and intra-sentence context.Source: §5 · p. 8
  • The multi-granularity and paragraph readers reject “members of the civil group” as a perpetrator and extract “the bomb” as the weapon.The passage says paragraph-level context or reasoning about the expression can help distinguish the role.Source: §5 · p. 9
  • The reader still overlooks role fillers, including “La Tandona,” whose perpetrator-organization role is implicit and whose phrase appears nowhere else in the corpus.The passage notes that external knowledge, such as Wikipedia, could help confirm the role.Source: §5 · p. 9
  • A role can require narrative reasoning even without explicit words such as “kill” or “kidnap” near the target.The example concerns identifying Jorge Serrano Gonzalez as involved in a terrorism event.Source: §5 · p. 9

6 Conclusion and Future Work

The paper concludes that end-to-end neural sequence models can tackle document-level role filler extraction, and that very long input contexts can lower performance. Its multi-granularity reader combines sentence- and paragraph-level representations and substantially improves over prior work on the benchmark dataset.

6 Conclusion and Future Work

  • The multi-granularity reader dynamically incorporates paragraph- and sentence-level contextualized representations and achieves substantial improvement over prior work on the benchmark dataset.The conclusion also reports qualitative analysis supporting the improvement.Source: §6 · p. 9
  • End-to-end neural sequence models can tackle document-level role filler extraction, but very long input context might be hard to capture and can lower performance.Source: §6 · p. 9
  • Future work could explore jointly extracting role fillers, handling coreferential mentions, and constructing event templates.Source: §6 · p. 9

Related papers

Loading 2005.06579v1…