Source-linked AI summary
Constructing Narrative Event Evolutionary Graph for Script Event Prediction
Zhongyang Li, Xiao Ding, Ting Liu
TL;DR
Event-pair and event-chain models do not fully use dense event connections for script event prediction. The paper constructs a narrative event evolutionary graph and applies a scaled graph neural network to infer subsequent events, achieving the best reported performance on the New York Times corpus against state-of-the-art baselines.
Problem
Event-pair and event-chain models do not fully exploit rich connections among events for script event prediction.
Method
The paper extracts narrative event chains, constructs NEEG as an event graph, and uses SGNN to model interactions and learn event representations for subsequent-event selection.
Results
SGNN achieves 52.45% script event prediction performance, a 3.2% improvement over the best baseline model, PairLSTM.
Takeaways & Limitations
Event graph structure is more effective than event pairs and chains and can significantly boost prediction performance and robustness.
Abstract
from arXiv · showhide
Script event prediction requires a model to predict the subsequent event given an existing event context. Previous models based on event pairs or event chains cannot make full use of dense event connections, which may limit their capability of event prediction. To remedy this, we propose constructing an event graph to better utilize the event network information for script event prediction. In particular, we first extract narrative event chains from large quantities of news corpus, and then construct a narrative event evolutionary graph (NEEG) based on the extracted chains. NEEG can be seen as a knowledge base that describes event evolutionary principles and patterns. To solve the inference problem on NEEG, we present a scaled graph neural network (SGNN) to model event interactions and learn better event representations. Instead of computing the representations on the whole graph, SGNN processes only the concerned nodes each time, which makes our model feasible to large-scale graphs. By comparing the similarity between input context event representations and candidate event representations, we can choose the most reasonable subsequent event. Experimental results on widely used New York Times corpus demonstrate that our model significantly outperforms state-of-the-art baseline methods, by using standard multiple choice narrative cloze evaluation.
1 Introduction
Script event prediction selects the most reasonable subsequent event from a candidate list, but event-pair and event-chain models do not fully exploit dense event connections. The paper constructs NEEG and introduces SGNN to model these interactions for large-scale prediction.
- Motivation: Script event prediction chooses the most reasonable subsequent event given an existing event context and candidate list.The task supports applications including discourse understanding, intention recognition, and dialog generation.
- Motivation: Existing event-pair and event-chain models do not fully explore rich connections among events.The paper therefore proposes using event graph structure and network embedding for prediction.
- Motivation: Graph structure can distinguish a correct candidate through dense connections that pair- and chain-based models may miss.In the example, D forms a strongly connected component with context events B and C, whereas C–E is stronger than C–D in the training chains.
- Approach: NEEG stores event evolutionary principles and patterns as a directed cyclic graph whose nodes are events and edges represent relations such as temporal and causal relations.The graph is constructed from narrative event chains and supports subsequent-event inference.
- Approach: The paper presents SGNN to model event interactions on large-scale dense directed graphs and learn event representations for prediction.This is one of the paper’s stated contributions, alongside constructing an event graph instead of relying on event pairs or chains.
- Results: On the New York Times corpus, the model achieves the best performance among state-of-the-art baselines under standard multiple choice narrative cloze evaluation.The paper reports its empirical comparison using the MCNC evaluation setting.
2 Model
The model constructs a narrative event evolutionary graph from extracted event chains, then applies a scaled gated graph neural network to infer subsequent events from local subgraphs. It represents events, models their interactions, and ranks candidates by relatedness to the context.
- Model overview: The framework first constructs an event evolutionary graph from narrative event chains, then uses a scaled graph neural network for graph inference.The two stages are graph construction and SGNN-based inference.
- Narrative event evolutionary graph construction: Narrative event chains are extracted from newswire text, with each event represented by a predicate verb and its grammatical relation to the protagonist entity.Events contain predicate, subject, object, and indirect-object components before abstraction into predicate-GR representations.
- Narrative event evolutionary graph construction: NEEG nodes represent predicate-GR events, while directed weighted edges encode predicate-GR bigram transitions counted in training event chains.An edge vi → vj receives a weight based on the frequency of the corresponding bigram.
- Narrative event evolutionary graph construction: The constructed NEEG contains 104,940 predicate-GR nodes and 6,187,046 directed weighted edges, providing dense event interactions.The graph is intended to capture richer connections than isolated event pairs or chains.
- Scaled graph neural network: SGNN avoids processing the entire graph by feeding GGNN a subgraph containing the context and candidate events, enabling training on the large NEEG.The subgraph uses initial event representations and its adjacency matrix as GGNN inputs.
- Choosing the correct subsequent event: SGNN learns initial event representations, updates them through gated graph propagation, and computes context-candidate relatedness scores to select the subsequent event.Attention can assign different weights to context events, and several similarity metrics can serve as the score function.
3 Evaluation
The evaluation uses accuracy on a New York Times event-chain dataset and compares SGNN with established pair-, embedding-, network-, and neural-based baselines. SGNN achieves the best reported performance, with differences significant against all baselines except PairLSTM.
- Evaluation setup: Accuracy of choosing the correct subsequent event is the evaluation metric, and SGNN is compared with several state-of-the-art baseline methods.The baselines include PMI, Bigram, Word2vec, DeepWalk, EventComp, and PairLSTM.
- Results: SGNN achieves the best performance against state-of-the-art baselines on the standard multiple-choice narrative cloze evaluation.The reported results are measured on the test set using script event prediction accuracy.
- Results: Differences between SGNN and all baseline methods are significant at p < 0.01, except for the comparison between SGNN and PairLSTM, which has p = 0.246.The significance tests use a t-test.
- Evaluation setup: The dataset is built from the New York Times portion of Gigaword using extracted event chains, with five candidate subsequent events and one correct answer per context.The same corpus and extraction approach as prior work are used for comparison.
4 Results and Analysis
Experiments show SGNN benefits from graph structure, attention, and event-composition choices, achieving strong prediction accuracy and fast convergence on the New York Times corpus.
- Overall Results: Graph-based models outperform pair-based and chain-based models, indicating that event graphs provide richer interaction information.
- Overall Results: Attention improves SGNN performance, indicating that context events contribute unequally to selecting the subsequent event.
- Overall Results: 52.45%: SGNN achieves the best script event prediction performance, improving 3.2% over PairLSTM.
- Overall Results: 54.93%: combining SGNN, EventComp, and PairLSTM achieves the highest reported performance because pair, chain, and graph structures complement one another.
- Overall Results: SGNN quickly reaches stable high accuracy and outperforms PairLSTM throughout training under the reported learning-curve comparison.
- Event Semantic Composition: Concatenating verb and argument embeddings performs best, whereas averaging is worst because many events lack an indirect object.
- Similarity Metrics: Euclidean similarity achieves the best result, although the compared score metrics have relatively small performance gaps.
5 Related Work
Related work progresses from event pairs and chains toward graph-based representations, while prior graph neural networks motivate SGNN’s scalable design for large graphs.
- Script Learning: Neural sequence and composition models improved script learning through LSTMs, Seq2Seq prediction, and low-dimensional event vectors.
- Script Learning: Earlier script-learning models used event pairs or event chains, whereas this work uses event graph structure for prediction.
- Graph Representation Learning: Network embedding methods such as DeepWalk, LINE, and node2vec learn graph-node representations from graph structure or random walks.
- Graph Neural Networks: Earlier graph neural networks could process directed, undirected, labeled, and cyclic graphs but could require propagation to convergence.
- Graph Neural Networks: SGNN extends GGNN by training on subgraphs containing concerned nodes instead of feeding the whole graph, making it feasible for large-scale graphs.
- Graph-based Organization of Events: Prior event-graph work organized narrative, temporal, or causal relations, while this paper proposes an event evolutionary graph for abstract evolutionary patterns.
6 Conclusion
The paper constructs NEEG from narrative event chains and applies SGNN to model event interactions for subsequent-event prediction.
- NEEG captures dense event connections, while SGNN learns event representations for selecting subsequent events.
- Experiments report that event graph structure is more effective than event pairs and chains and improves prediction performance and robustness.