Source-linked AI summary
Joint Extraction of Events and Entities within a Document Context
Bishan Yang, Tom Mitchell
TL;DR
Existing extraction systems often separate events from entities and make sentence-level predictions despite their contextual interdependence. This paper decomposes the problem into structured submodels and combines them for document-wide joint inference, substantially outperforming established event and entity extraction baselines. The authors identify lexical sparsity, shallow long-range context understanding, and complex language as remaining error sources.
Problem
Prior systems commonly extract events and entities separately and infer events sentence by sentence, despite context-dependent dependencies between them.
Method
The paper learns within-event structures, event-event relations, and entity extraction models, then combines them in a joint document-level optimization framework.
Results
The approach substantially outperforms state-of-the-art event extractors and significantly improves a strong entity-extraction baseline on the ACE corpus.
Takeaways & Limitations
Joint document-level inference provides a unified way to extract event triggers, semantic roles, and entities using their dependencies and contextual relations.
Takeaways & Limitations
Remaining errors involve lexical sparsity, shallow understanding of especially long-range context, and complex language such as metaphor, idioms, and sarcasm.
Abstract
from arXiv · showhide
Events and entities are closely related; entities are often actors or participants in events and events without entities are uncommon. The interpretation of events and entities is highly contextually dependent. Existing work in information extraction typically models events separately from entities, and performs inference at the sentence level, ignoring the rest of the document. In this paper, we propose a novel approach that models the dependencies among variables of events, entities, and their relations, and performs joint inference of these variables across a document. The goal is to enable access to document-level contextual information and facilitate context-aware predictions. We demonstrate that our approach substantially outperforms the state-of-the-art methods for event extraction as well as a strong baseline for entity extraction.
1 Introduction
Events and entities require document-level contextual interpretation, but prior systems commonly extract them separately or sentence by sentence. The paper addresses these limitations with joint structured inference and reports stronger extraction performance.
- Motivation: Event and entity interpretation depends on surrounding discourse, as shown by repeated attacks whose shared participants and context clarify the second event.The second sentence alone contains little event information but, with the preceding sentence, indicates another U.S. attack on Baghdad.
- Motivation: Separate entity-then-event pipelines can propagate entity errors into event arguments, reducing performance relative to gold entity information.Misclassifying “Baghdad” as a person prevents identifying it as the attack location.
- Motivation: Sentence-level event extraction commonly ignores document context, and prior attempts lack a unified model jointly extracting events across an entire document.The paper positions document-wide joint extraction as an unresolved gap in earlier work.
- Approach: The approach decomposes learning into within-event dependencies, cross-document event co-occurrence, and entity extraction before combining them in joint optimization.The framework jointly extracts events, semantic roles, and entities.
- Approach: The within-event model captures dependencies between events and arguments and between semantic roles and argument entity types.This structured representation explicitly links event structure to entity typing.
- Results: On the ACE corpus, the approach significantly outperforms state-of-the-art event extraction methods and a strong entity-extraction baseline.The reported evaluation covers both event and entity extraction.
2 Task Definition
The task follows ACE annotations for entity mentions, event triggers, and event arguments. Events are represented through triggers, typed entities, and semantic roles linking arguments to events.
- Entities: An entity mention is a noun-phrase or pronoun reference to an object or set of objects in the world.The paper adopts the ACE definition of entities and entity mentions.
- Events: An event trigger is a word or phrase that clearly expresses an event’s occurrence and may be a verb, noun, or occasionally an adjective.Examples of adjective triggers include “dead” and “bankrupt.”
- Events: Event arguments are entity mentions filling event-specific semantic roles, including participants, places, times, and attributes such as JOB-TITLE or CRIME.The paper treats event arguments as entities associated with roles in an event.
- Task scope: The extraction task covers ACE entity types, 33 event subtypes, and 35 semantic roles, with eight time-related roles collapsed into one.Values and TIME expressions are included as entities, while rare time roles are grouped to address limited training examples.
- Annotation structure: ACE annotations distinguish event triggers and entity mentions by color, label their types above them, and mark semantic roles on trigger-entity links.The example depicts “conviction” evoking a CONVICT event with CRIME and DEFENDANT roles.
3 Approach
The approach decomposes joint event–entity extraction into tractable within-event, event-relation, and entity models, then combines them through globally constrained document-level inference.
- 3.1 Learning Within-event Structures: The model represents each trigger candidate with an event-type variable, each potential argument with a role variable, and each argument entity with an entity-type variable.Trigger candidates use 33 ACE event types plus NONE; role variables use 28 semantic roles plus NONE, while entity variables use 9 entity types plus NONE.
- 3.1 Learning Within-event Structures: Within-event modeling captures dependencies among event types, argument roles, and entity types using unary and pairwise feature functions.Invalid event–role and role–entity configurations receive zero probability, enforcing compatibility constraints such as PERSON roles requiring PER entities.
- 3.1 Learning Within-event Structures: Training uses L-BFGS and exact sum-product marginals, with valid-configuration structure reducing joint-distribution complexity to O(M ×(k1T +k2N)).The paper notes k1 = 6 and k2 = 3 in ACE, and describes these averages as small in practice.
- 3.2 Learning Event-Event Relations: The event-event model propagates information between trigger pairs using contextual, syntactic, coreference, lexical, and semantic-frame features.Training pairs co-occur in a sentence or are connected by a coreferent subject or object across sentences.
- 3.3 Entity Extraction: Entity mentions are extracted with a BIO linear-chain CRF using lexical, contextual, orthographic, gazetteer, and word-embedding features.The entity model provides marginal probabilities that contribute to joint inference.
- 3.4 Joint Inference: Joint inference combines within-event, event-relation, and entity confidence scores while enforcing agreement among overlapping variables.The resulting optimization seeks globally optimal assignments for trigger, argument-role, and entity variables and is formulated as an integer linear program.
4 Experiments
Experiments on ACE2005 show that joint event-entity modeling improves event and entity extraction, while error analysis identifies remaining challenges in trigger and argument prediction.
- Event extraction results: JOINTEVENTENTITY achieves the best performance across all event-extraction evaluation categories and improves both precision and recall over WITHINEVENT.The authors attribute this to information propagation across event and entity mentions, producing more context-aware and semantically coherent predictions.
- Event extraction results: JOINTEVENTENTITY outperforms CROSS-DOC and CNN, achieving new state-of-the-art results for end-to-end event trigger and argument extraction.CROSS-DOC performs cross-document inference, whereas CNN extracts event triggers at the sentence level.
- Entity extraction results: Joint modeling significantly improves entity-extraction recall and F1 compared with the strong CRFENTITY baseline.The improvements extend to more accurate predictions for several major entity types, including PER, GPE, ORG, and TIME.
- Error analysis: Most event-trigger errors are missing triggers, while only 3.7% involve misclassified event types; most argument errors are missing arguments, with 4.1% involving misclassified roles.Among missing triggers, 5% have correctly identified event types, and 10% of missing arguments have correctly identified entity types.
- Error analysis: Remaining event-extraction errors arise from lexical sparsity, shallow long-range context understanding, and complex language such as metaphor, idioms, and sarcasm.Entity gains are limited in sentences without relevant event triggers or arguments, while coreference information could avoid some entity misclassifications.
5 Related Work
Prior event-extraction research uses pipelines and increasingly incorporates document-level event relations, including temporal, causal, inhibition, and narrative-schema relations.
- Pipeline approaches: Pipeline classifiers extract event triggers before their arguments, and convolutional neural networks have been used to improve these systems.Pipeline approaches suffer from error propagation between stages.
- Document context: Document-level methods have modeled event-event relations such as causality and inhibition, especially in biological texts.These relations are used to exploit dependencies among events beyond individual sentences.
- Document context: For general texts, much document-context work focuses on temporal event relations.The cited approaches include Chambers and Jurafsky, Do et al., and McClosky and Manning.
- Event relations and argument typing: Narrative schemas link event verbs with coreferring syntactic arguments, while entity types constrain which roles event arguments can fill.The model adopts this intuition to relate triggers across sentences and grounds arguments by entity type.
6 Conclusion
The paper introduces joint document-level inference for event triggers, semantic roles, and entities. It reports large-margin gains over state-of-the-art event extractors and substantial improvement over a strong entity-extraction baseline, while identifying coreference and richer event relations as future extensions.
- Conclusion: The model jointly infers event triggers, semantic roles, and entities across an entire document.It integrates models for within-event structures, event-event relations, and entity extraction.
- Conclusion: The approach outperforms state-of-the-art event extractors by a large margin and substantially improves a strong entity-extraction baseline.The reported improvements cover both event extraction and entity extraction.
- Future work: Future work includes integrating entity and event coreference and more sophisticated causal and temporal event-relation models.These components are intended as additional elements of the joint inference framework.