PaperlayerView exact revision on arXiv

Source-linked AI summary

Investigating Forecasting Capabilities of Narrative Networks on Emerging Stories

Ravi Teja Karumuri, Steven Corman, Huan Liu, Joshua Garland

arXiv:2610.35628v1cs.SI

TL;DR

The paper asks whether narrative-network structure can forecast how real, evolving stories develop, beyond prior work focused on retrospective analysis or a single fictional story. It builds news-derived networks and evaluates next-day emergent-link prediction across three events. Preferential Attachment ranks above other standard predictors and exceeds EdgeBank on every event, with the signal concentrated among highly connected entities.

  • Problem

    Whether narrative-network structure can predict emergent relations in real, evolving narratives remains untested beyond a prior demonstration on one fictional story.

    Source: §1 · p. 1

  • Method

    The authors use an LLM-based Semantic Edge Parser to build rolling-window news graphs and predict next-day emergent links between existing narrative elements.

    Source: §1 · p. 1, §3 · p. 3

  • Results

    Preferential Attachment is the strongest and most consistent predictor, exceeding EdgeBank by 11 to 15 ROC-AUC percentage points on every event, with the signal concentrated in high-degree entities.

    Source: §5 · p. 8

  • Takeaways & Limitations

    For emergent narrative relations, local endpoint structure is a stronger signal than global proximity, and the advantage is specific to the emergent setting.

    Source: §5 · p. 8

    The task predicts relations only between elements already present in the window and does not model arrivals of entirely new elements.

    Source: §1 · p. 2, §5 · p. 8

Abstract

from arXiv · show

Networks have been used to model the interactions among the elements that underlie a narrative. In this paper, we investigate whether the structure of these networks can predict which links will emerge as a storyline evolves. We construct these networks from news articles using a Semantic Edge Parser (SEP) built on a Large Language Model (LLM), and evaluate the networks on three disparate real-world storylines, the 2022 Canadian Freedom Convoy, Hurricane Melissa (2025), and the 2025-26 Iranian protests. To test the narrative network's predictive capability, we construct a rolling window of the narrative and predict the emergent links between existing narrative elements on the day following the rolling window period. We identify candidate links with seven standard link predictors, including the Adamic-Adar index, Common Neighbors, and Preferential Attachment. Forecasting next-day links across the three storylines, we find Preferential Attachment to be the strongest standard link predictor, achieving a mean ROC-AUC of 0.76-0.79. It exceeds EdgeBank, a memorization baseline, by 11.3-14.5 points on every event. To summarize, narrative-network structure ranks next-day emergent relations above chance, though this advantage is concentrated among the most frequently mentioned, highly connected entities.

1 Introduction

Narrative networks make story structure explicit, but whether that structure can forecast new relations in real, evolving narratives remains untested. This paper frames the question as next-day prediction of emergent links in news-derived networks across three real events.

1 Introduction

  • Prior computational narratology mainly analyzes completed narratives, while earlier narrative-graph forecasting was demonstrated on only one fictional story.The paper asks whether network structure can predict links between nodes such as actors and locations as real narratives evolve.Source: §1 · p. 1
  • The study builds daily narrative graphs from rolling windows of news coverage and predicts next-day directed links between nodes already present in the window graph.Links absent from the window graph but present the following day are treated as emergent; forecasting entirely new elements is excluded because a predictor needs both endpoints in the graph.Source: §1 · pp. 1–2
  • Candidate links are scored with standard structural predictors and compared with EdgeBank and a Temporal Graph Network, while recurring links are separated from emergent ones.Separating them addresses the concern that recurrence-dominated evaluation can obscure whether a method captures narrative development.Source: §1 · p. 2
  • Across three events, structural predictors outperform a memorization baseline, and the study examines how much this advantage reflects endpoint degree and hub structure.The events are the 2022 Freedom Convoy, the 2025–26 Iranian protests, and Hurricane Melissa (2025).Source: §1 · p. 2

2 Related Work

Prior work represents narratives through similarity, discourse, or explicit relations, and has studied narrative change mainly retrospectively or through generalized event sequences. This paper builds on relation-based narrative graphs and link prediction to examine emergent relations in an unfolding story.

2 Related Work

  • Similarity and discourse approaches describe corpus relationships or documents, while relation-based approaches represent narrative elements as nodes and actions as edges.Corman et al.’s formalism instead promotes verbs to nodes, connecting narrative networks to hypergraph and n-ary knowledge-graph representations [14].Source: §2 · p. 2
  • Earlier forecasting work predicts generalized event sequences or characterizes change after a story is complete, rather than forecasting emergent relations within one unfolding narrative.Story timelines and chains describe coverage development, and a survey places next-event prediction between extraction and generation [35].Source: §2 · p. 2
  • Classical link predictors rank unobserved node pairs by structural proximity, but recurrence-heavy test sets can conflate new-link prediction with prediction of repeated edges.Prior work argues that dynamic link prediction should separate new from previously observed links [23].Source: §2 · pp. 2–3
  • This study adopts established predictors and protocols for distinguishing new and recurring edges, and evaluates its LLM-based parser under three independent models.Corman et al. already forecast new narrative relations; this work examines forecasting ability on new versus recurring edges.Source: §2 · p. 3

3 Methods

The methods build directed narrative graphs from news and test whether their structure predicts next-day links between existing story elements across three events.

3.1 Semantic Edge Parsing

  • Semantic Edge Parsing uses an LLM to turn article sentences into labeled directed edges, with actions represented as nodes linking the entities they relate.The parser resolves coreference and treats multi-word expressions as single nodes.Source: §3.1 · p. 3

3.2 The Narrative Network

  • Daily article graphs are merged into a directed network, and consecutive daily snapshots are aggregated into a window graph with duplicate edges counted as weights.Edge weights are not used by the structural scorers, and weighting the graph does not improve forecasting performance.Source: §3.2 · pp. 3–4

3.3 Emergent Link Prediction

  • Emergent links are directed edges appearing the next day between nodes already in the window graph, absent from that graph.This restriction separates new relations from the arrival of new narrative elements and excludes recurring links.Source: §3.3 · p. 4
  • The structural scorers are adapted to directed graphs so candidate-link scores retain the direction of who did what to whom.For an ordered pair, common neighbors are nodes on a directed two-path from source to target.Source: §3.3 · p. 4

3.4 Data

  • The evaluation covers the 2022 Freedom Convoy, Hurricane Melissa, and the 2025–26 Iranian protests, spanning civil protest and natural-disaster narratives.The event mix is intended to check whether findings are specific to one kind of story.Source: §3.4 · p. 4

3.5 Evaluation Protocol

  • The primary emergent protocol scores unconnected ordered pairs among the 1,000 highest-degree nodes using a 14-day window and five sampled negatives per positive.TGN is evaluated only under the standard chronological train-validation-test protocol; the authors make no claim about TGN on the emergent protocol.Source: §3.5 · p. 5
  • Table 3 identifies Preferential Attachment as the strongest predictor, beating EdgeBank on every event.The caption reports a Wilcoxon p < 10^-6 and wins on every day-split.Source: Table 3 · p. 5

4 Results

Across three events, Preferential Attachment leads emergent-link prediction; its advantage is associated with hub structure and persists across candidate-set sizes and SEP extractors.

4.1 Emergent Link Prediction

  • PA achieved mean ROC-AUCs of 0.785, 0.755, and 0.793 across the Convoy, Iran, and Melissa, beating EdgeBank by 14.5, 11.3, and 11.4 points.Adamic–Adar, Common Neighbors, and Resource Allocation tied on Convoy and Iran; normalized indices were weakest overall.Source: §4.1 · p. 5, Figure 4 · p. 6

4.2 Emergent Versus Standard Evaluation

  • With recurrence included, EdgeBank led on Convoy and Iran and TGN on Melissa; on Convoy, EdgeBank scored 0.781 versus PA’s 0.654.For emergent links, EdgeBank fell to 0.640 while PA scored 0.785; the figure contrasts these protocols.Source: §4.2 · p. 6, Figure 6 · p. 7, Table 4 · p. 7

4.3 Why Structure Wins: A Case Study

  • Emergent-link rates rise from roughly 1–2% in the lowest degree decile to 34–37% in the highest, while PA’s illustrated false positives attach to Trudeau.The case study also shows PA missing a link between peripheral nodes; its margin over EdgeBank shrinks as the highest-degree nodes are removed.Source: Figure 8 · p. 7, §4.3 · p. 7, Figure 7 · p. 7

4.4 Robustness to the Candidate-Set Size

  • PA remained the top scorer at every candidate-set cutoff from 500 to 2,000 nodes across all three events.Scores rose with larger candidate pools, which add low-activity pairs that endpoint degree separates easily.Source: Table 5 · p. 8, §4.4 · p. 8

4.5 Robustness Across LLM Extractors

  • PA ranked first for every event under Haiku, GPT-4.1, and Gemini extraction; GPT-4.1 and Gemini generally scored slightly higher than Haiku.All three extractors scored well above EdgeBank.Source: §4.5 · p. 8

5 Conclusion and Discussion

The paper finds that narrative-network structure can rank next-day emergent relations above chance, with Preferential Attachment strongest across three events. The signal is concentrated in highly connected, frequently mentioned entities, and the authors identify limits in scope and evaluation.

Conclusion and Discussion

  • The study forecasts emergent links introduced by later coverage as a way to test whether narrative-network structure anticipates how a story continues.The question is whether the structure of narrative networks carries information about future story states.Source: §5 · p. 8
  • Preferential Attachment exceeds EdgeBank by 11 to 15 ROC-AUC percentage points on every event, while structural predictors rank next-day emergent relations above chance under random negatives.The degree-based result indicates that the signal is concentrated in high-degree, frequently mentioned entities rather than finer relational structure.Source: §5 · p. 8
  • The forecasting signal is local rather than path-based, with local scorers outperforming Random Walk with Restart for emergent relations.Under the recurrence-inclusive protocol, memorization and a temporal graph network lead because the test set is dominated by recurring edges.Source: §5 · p. 8
  • The study covers three English-language events across two domains; narratives that split into competing sub-stories may not be well represented by one accumulating graph.The evaluation uses uniformly random negatives, which favor degree-based scorers; endpoint degree and news salience may account for much of Preferential Attachment’s advantage.Source: §5 · p. 8

Ethics and Privacy Statement

The work uses public news indexed by GDELT and no human subjects or private data. Its Semantic Edge Parsing prompt specifies a single-pass process for extracting and normalizing narrative edges.

Ethics and Privacy Statement

  • The study uses public news indexed by GDELT and involves no human subjects or private data.It reports results under three independent extractors to mitigate bias from the language-model extractor, treating the parser as a measurement instrument rather than ground truth.Source: Ethics and Privacy Statement · pp. 8–9

A Semantic Edge Parsing Prompt

  • The Semantic Edge Parsing prompt directs a single pass over each article to emit a CSV edge list while resolving coreference and normalizing entities.The output has source, target, label, and weight columns.Source: A Semantic Edge Parsing Prompt · p. 10
  • The prompt represents subject edges from nouns to verbs, object edges from verbs to nouns, adjective edges from modifiers to nouns, and prepositional edges labeled by the preposition.For prepositional edges, the preposition is not a source or target node.Source: A Semantic Edge Parsing Prompt · p. 10
  • The prompt lowercases words, reduces verbs to infinitives and nouns to singular forms, and keeps multi-word proper nouns as single nodes.It also encodes modal and negated verbs with prefixes and strips tense or aspect auxiliaries.Source: A Semantic Edge Parsing Prompt · p. 10
  • The prompt replaces pronouns with their referents, resolves shortened names to their longest form, and deduplicates identical source-target-label triples while counting their occurrences.Conjunctions produce separate edges, and embedded clauses resolve the subject and verb of each clause.Source: A Semantic Edge Parsing Prompt · p. 10

B Extractor Configuration

The extractor configuration holds the parsing procedure constant while changing the language-model backend, allowing performance differences to reflect the model rather than parsing code.

B Extractor Configuration

  • SEP is run through a common gateway with Haiku, GPT-4.1, and Gemini as alternative model backends.The configuration uses temperature 0.0 and one call per article.Source: B Extractor Configuration · p. 10
  • Coreference resolution and normalization occur within the same pass that emits the edge list, rather than in separate stages.The prompt replaces pronouns with referents, resolves shortened names, and normalizes tokens.Source: B Extractor Configuration · p. 10
  • For the example sentence, the parser emits edges from police to clear, clear to protester, clear to bridge, and clear to demonstration.The latter two edges use the labels “from” and “after,” respectively.Source: B Extractor Configuration · p. 10

C Per-Extractor Results

Across extractors, Preferential Attachment is the top scorer for every event in the emergent protocol. It also leads on Precision@k, with tied candidate scores handled through uniformly random ranking.

C Per-Extractor Results

D Precision@k

  • Equal-scoring candidates are ranked uniformly at random with a fixed seed; this matters especially for EdgeBank, which gives most candidates the same score.The passage says this tie handling is immaterial for structural indices, whose scores are effectively continuous.Source: D Precision@𝑘 · p. 11
  • Preferential Attachment is top on every event for Precision@k, consistent with the ROC-AUC and PR-AUC ranking.Table 9 reports Precision@k with k equal to the number of positives, averaged over the three extractors in the emergent protocol at W=14.Source: Table 9 · p. 11

E Weighted vs. Unweighted Indices

Recurrence weighting does not improve the indices and slightly hurts performance, especially for Preferential Attachment.

E Weighted vs. Unweighted Indices

  • Recurrence weighting never helps and slightly hurts weighted scorers, most for Preferential Attachment.Table 10 reports ROC-AUC averaged over three extractors under the emergent protocol.Source: Table 10 · p. 11

F Data Collection

The study builds three news corpora from GDELT records, retrieves and deduplicates article text, and applies event-specific collection filters.

F Data Collection

  • The corpora use GDELT 2.0 records, with full article text retrieved and near-duplicates removed by normalized-text deduplication.Queries imposed no per-day row cap, producing the full census over each collection window.Source: F Data Collection · p. 11
  • Convoy and Melissa queries filter by URL slug and English-origin records, while the Iran query uses GKG themes and locations without that language filter.The retrieved Iran corpus is nevertheless over 99% English, as are all three event corpora.Source: F Data Collection · p. 11
  • The collection windows cover the 2022 Freedom Convoy, Hurricane Melissa from 2025-10-25 to 2025-12-08, and Iranian protests beginning 2025-12-25.The Iran window ends on the export date.Source: F Data Collection · pp. 11–12

G Baselines and Hyperparameters

The structural-scoring protocol fixes the candidate set, negative sampling, and reproducibility settings; EdgeBank and TGN use specified standard-protocol baselines.

G Baselines and Hyperparameters

  • Structural scorers use a 14-day window, a 5:1 negative ratio, and unconnected ordered pairs among the top 1,000 nodes as candidates.Days with fewer than 50 nodes or 10 positive edges are skipped; runs use fixed seed 0 and candidate ordering.Source: G Baselines and Hyperparameters · p. 12
  • EdgeBank scores a directed pair 1 if it appeared anywhere in the training window, using unlimited memory.TGN uses a chronological 70/15/15 split, 1:1 destination-perturbed negatives, and DyGLib defaults with num_runs=1.Source: G Baselines and Hyperparameters · p. 12
  • TGN is evaluated only on the standard protocol because it requires a single large train/test split.Structural scorers instead use day-by-day forward chaining.Source: G Baselines and Hyperparameters · p. 12

H Rolling-Window Sensitivity: All Events

As the rolling window lengthens, structural scorers maintain or improve their performance while EdgeBank declines toward chance, widening the structure–memorization margin across all three events.

H Rolling-Window Sensitivity: All Events

Loading 2610.35628v1…