PaperlayerView exact revision on arXiv

Source-linked AI summary

One for All: Neural Joint Modeling of Entities and Events

Trung Minh Nguyen, Thien Huu Nguyen

arXiv:1812.00195v1cs.CL

TL;DR

Prior event extraction work often assumes entity mentions are provided, while joint models have relied on discrete features. This paper jointly predicts entity mentions, triggers, and arguments using shared deep representations, achieving state-of-the-art event extraction performance with predicted mentions.

Abstract

from arXiv · show

The previous work for event extraction has mainly focused on the predictions for event triggers and argument roles, treating entity mentions as being provided by human annotators. This is unrealistic as entity mentions are usually predicted by some existing toolkits whose errors might be propagated to the event trigger and argument role recognition. Few of the recent work has addressed this problem by jointly predicting entity mentions, event triggers and arguments. However, such work is limited to using discrete engineering features to represent contextual information for the individual tasks and their interactions. In this work, we propose a novel model to jointly perform predictions for entity mentions, event triggers and arguments based on the shared hidden representations from deep learning. The experiments demonstrate the benefits of the proposed method, leading to the state-of-the-art performance for event extraction.

Sentence Encoding

Each input word is represented by concatenating its pre-trained word embedding with binary linguistic-feature vectors. The embeddings are updated during training.

Sentence Encoding

  • Each word vector concatenates a pre-trained word embedding with binary vectors for POS, chunk, and dependency information.The dependency vector indicates which relations surround the word in the parsed sentence.Source: Sentence Encoding · p. 3
  • The model updates the pre-trained word embeddings during training.Source: Sentence Encoding · p. 3
  • POS tags and chunking tags are represented with one-hot vectors, while surrounding dependency relations use a binary vector.Source: Sentence Encoding · p. 3

Sentence Representation

The sentence representation uses shared contextual states and a staged probability decomposition to predict entity types, event triggers, and argument roles. Predictions proceed from entity labeling to left-to-right trigger and argument prediction.

Sentence Representation

  • A bidirectional GRU produces contextual hidden vectors shared by EMD, ED, and ARP, facilitating communication and knowledge transfer among the subtasks.Source: Sentence Representation · p. 3
  • The model first predicts entity types for every token, then scans left-to-right to predict trigger types and argument-role rows conditioned on earlier predictions.The decomposition estimates P(A, T, E|W) by predicting E before the trigger and argument predictions.Source: Sentence Representation · p. 3
  • The entity detector uses a feed-forward softmax to assign each word an entity-type distribution from its hidden representation and local context.Local context concatenates embeddings in a window around the word, with zero padding beyond sentence boundaries.Source: Entity Mention Detector · p. 4, Sentence Representation · p. 4
  • Entity labels are decoded with Viterbi using transition scores that penalize an I label unless preceded by its matching B label.The model does not condition a word's entity representation on the previous word's predicted entity type.Source: Entity Mention Detector · p. 4
  • At each word, argument-role probabilities condition on the current trigger prediction and earlier argument choices, while prior triggers and roles capture inter-event dependencies.Source: Trigger and Argument Prediction · p. 4, Sentence Representation · p. 4
  • During training, gold entity mentions identify entity-start tokens for argument prediction; evaluation instead uses predicted entity labels to find those starts.Source: Trigger and Argument Prediction · p. 4

Training

The model is trained by optimizing a negative log-likelihood with differently weighted task losses. Experiments use the ACE 2005 split and development-tuned settings.

Training

  • Training minimizes the negative log-likelihood of the joint probability, with EMD, ED, and ARP loss terms weighted differently to encourage their convergence together.Source: Training · p. 4
  • Experiments use ACE 2005 with 529 training documents, 30 development documents, and 40 test documents, following the prior-work split.Stanford CoreNLP supplies POS tagging, chunking, and dependency parsing.Source: Dataset, Parameters, and Resources · p. 5
  • The selected settings include 300-dimensional embeddings and RNN hidden units, a context window of 2, and dropout of 0.5, tuned on development data.The feed-forward networks use 600 hidden units; the loss coefficients are α = 0.5, β = 1.0, and γ = 0.5.Source: Dataset, Parameters, and Resources · p. 5

Comparing to the State of the Art for Trigger and Argument Predictions

On ACE 2005, Joint3EE outperforms the compared pipelined and joint feature-based systems, including the prior best system without gold entity mentions. Its gains over that system are significant for event argument identification and argument role classification.

Comparing to the State of the Art for Trigger and Argument Predictions

  • Joint3EE is compared with feature-based pipelines, a deep-learning pipeline, and sentence- and document-level joint feature systems.The comparison set also includes NP-Candidate-Deep-Learning, which uses noun-phrase candidates and does not model EMD.Source: Comparing to the State of the Art for… · p. 5
  • With predicted entity mentions, Pipelined-Deep-Learning reports 67.0% trigger and 37.4% argument-role classification, versus 69.3% and 55.4% with perfect mentions.The authors cite this drop, alongside comparisons with fully joint systems, as evidence for jointly modeling entity mentions with triggers and roles.Source: Comparing to the State of the Art for… · p. 5
  • On the ACE 2005 test set, Joint3EE significantly outperforms NP-Candidate-Deep-Learning by 2.7% in event argument identification and 2.0% in argument role classification, measured as absolute F1 improvements.Source: Table 1 · p. 6, Comparing to the State of the Art for… · p. 6
  • Joint3EE is significantly better than Joint-Feature-Document on all F1 scores, while deep learning also improves over feature-based models in pipelined and joint settings.Source: Comparing to the State of the Art for… · p. 6

Performance of EMD

Joint3EE improves entity mention detection over both a separately trained deep-learning detector and the prior joint feature model. The authors relate its larger argument-role advantage to shared representations that can encode information not available in predicted entity mentions.

Performance of EMD

  • EMD-Joint3EE improves absolute F1 by 0.8% (p < 0.05) and outperforms EMD-Joint-Feature by 0.5% on F1, supporting gains from jointly modeling EMD with ED and ARP via deep learning.Source: Performance of EMD · p. 6
  • The EMD gap is 0.8%, while the ARP gap is 13.9%; Pipelined-Deep-Learning lacks entity subtype cues available with manually annotated mentions.The authors note subtype information is helpful for ARP, but unavailable when only predicted entity boundaries and types are used.Source: Performance of EMD · p. 6
  • Joint3EE’s shared hidden vectors can implicitly encode fine-grained information, compensating for its absence as explicit entity subtype features and improving ARP.The compared EMD baselines include a separately trained deep-learning model and the state-of-the-art joint feature model.Source: Performance of EMD · p. 6

The Effect of External Features

Removing external hand-designed features lowers Joint3EE’s performance across entity mention, event trigger, and argument prediction. Even without those features, End-to-end-DL remains comparable to the referenced state-of-the-art systems on several task comparisons.

The Effect of External Features

  • Removing external features lowers F1 across EMD, ED, and ARP, although End-to-end-DL remains comparable to the state-of-the-art systems on several comparisons.Table 4 reports classification F1 for all three subtasks.Source: The Effect of External Features · p. 7, Table 4 · p. 7
  • End-to-end-DL is 1.2% worse than Joint-Feature-Document on EMD and 0.9% worse than NP-Candidate-Deep-Learning on ED.It uses no external hand-designed toolkit features, unlike the referenced systems.Source: The Effect of External Features · p. 7
  • On ARP, End-to-end-DL significantly outperforms Joint-Feature-Document by 1.9% despite using no external hand-designed features.The paper presents this as a promising step toward a state-of-the-art end-to-end event-extraction system.Source: The Effect of External Features · p. 7

Error Analysis

The error analysis finds that trigger identification and argument-role classification remain important sources of error for Joint3EE. Missed triggers often involve words absent from training, while incorrect triggers reflect confusable contexts; argument errors are mainly role confusions.

Error Analysis

  • Trigger classification F1 is 69.8%, close to trigger identification F1 of 72.5%, suggesting trigger-word identification is the main error source.The top three event types account for 41.1% of missed-trigger errors and 53.2% of incorrect-trigger errors; Attack and Transport appear frequently in both.Source: Error Analysis · p. 7, Table 5 · p. 7
  • Missed triggers mostly involve words absent from training, whereas incorrect triggers arise in confusable contexts that need better contextual modeling.The authors illustrate these patterns with an unseen Attack trigger and a contextually misleading use of “fire.”Source: Error Analysis · p. 7
  • Of 209 correctly identified arguments that were misclassified, 159 (76.1%) had incorrect roles, and only 24 of those (15.1%) also had incorrect entity types.Common confusions include Place versus Destination, Origin versus Destination, and Seller versus Buyer.Source: Error Analysis · p. 7
  • Distinguishing the confused argument roles would require better mechanisms or network architectures for modeling input context.The paper concludes that Joint3EE achieves state-of-the-art event-extraction performance with predicted entity mentions.Source: Error Analysis · p. 7, Conclusion · p. 7
Loading 1812.00195v1…