PaperlayerView exact revision on arXiv

Source-linked AI summary

An automated pipeline for the discovery of conspiracy and conspiracy theory narrative frameworks: Bridgegate, Pizzagate and storytelling on the web

Timothy R. Tangherlini, Shadi Shahsavari, Behnam Shahbazi, Ehsan Ebrahimzadeh, Vwani Roychowdhury

arXiv:2008.09961v1cs.CLcs.SI

TL;DR

Computational descriptions of conspiracy narrative structure remain limited, and fragmented discussions make these frameworks difficult to reconstruct. The paper develops a graph-based pipeline to extract and aggregate context-specific actants and relationships from posts and news reports. Its comparison finds Pizzagate spanning domains through interpretations of hidden knowledge, while Bridgegate remains rooted in New Jersey politics and develops over a longer period.

  • Problem

    Computational work describing conspiracy narrative structures is limited, and fragmented, coded, or disappearing discussions make their underlying frameworks difficult to reconstruct.

    Source: Introduction · pp. 2–3

  • Method

    The pipeline aggregates actants and relationships from broad collections into context-dependent graph frameworks, using a computational actant-relationship model.

    Source: Introduction · pp. 3–4

  • Results

    Pizzagate’s framework links multiple domains through interpretations of hidden knowledge, whereas Bridgegate remains in New Jersey politics and was still in flux nearly seven years after discovery.

    Source: Introduction · p. 4

  • Takeaways & Limitations

    The comparison identifies different narrative-framework patterns: cross-domain alignment in Pizzagate and a single political domain with prolonged development in Bridgegate.

    Source: Introduction · p. 4

    Noisy social-media text can disrupt relationship extraction by altering dependency trees and producing erroneous arguments or relationships.

    Source: Limitations · p. 18

Abstract

from arXiv · show

Although a great deal of attention has been paid to how conspiracy theories circulate on social media and their factual counterpart conspiracies, there has been little computational work done on describing their narrative structures. We present an automated pipeline for the discovery and description of the generative narrative frameworks of conspiracy theories on social media, and actual conspiracies reported in the news media. We base this work on two separate repositories of posts and news articles describing the well-known conspiracy theory Pizzagate from 2016, and the New Jersey conspiracy Bridgegate from 2013. We formulate a graphical generative machine learning model where nodes represent actors/actants, and multi-edges and self-loops among nodes capture context-specific relationships. Posts and news items are viewed as samples of subgraphs of the hidden narrative network. The problem of reconstructing the underlying structure is posed as a latent model estimation problem. We automatically extract and aggregate the actants and their relationships from the posts and articles. We capture context specific actants and interactant relationships by developing a system of supernodes and subnodes. We use these to construct a network, which constitutes the underlying narrative framework. We show how the Pizzagate framework relies on the conspiracy theorists' interpretation of "hidden knowledge" to link otherwise unlinked domains of human interaction, and hypothesize that this multi-domain focus is an important feature of conspiracy theories. While Pizzagate relies on the alignment of multiple domains, Bridgegate remains firmly rooted in the single domain of New Jersey politics. We hypothesize that the narrative framework of a conspiracy theory might stabilize quickly in contrast to the narrative framework of an actual one, which may develop more slowly as revelations come to light.

Introduction

The paper addresses limited computational attention to conspiracy narrative structure and the difficulty of reconstructing stories told in fragments. It develops a graph-based pipeline and compares Pizzagate with Bridgegate, finding contrasting domain scope and narrative development.

Introduction

  • Narrative structure has received little attention, and fragmented, coded, or disappearing online discussions make characters, relationships, contexts, and hidden events difficult to identify.Understanding these structures can reveal imagined threats, alleged special knowledge, and proposed countermeasures.Source: Introduction · p. 2
  • The paper asks whether automated methods can reconstruct narrative frameworks as network graphs and whether conspiracy theories and actual conspiracies differ structurally.The authors distinguish conspiracy theories as narrative accounts from actual conspiracies as covert factual events.Source: Introduction · pp. 2–3
  • The approach refines an actant-relationship graph to represent narrative domains computationally, supporting a macroscopic view of many actants and their relationships.The model draws on narrative graph representations and the actantial model.Source: Introduction · p. 3
  • The pipeline aggregates actants and relationships from broad collections, groups context-dependent actants into supernodes and subnodes, and ranks relationships to construct a narrative framework.It does not presuppose one framework or a conspiracy classification for the corpus.Source: Introduction · p. 4
  • Pizzagate links multiple domains through theorists’ interpretations of hidden knowledge, while Bridgegate remains within New Jersey politics and was still changing nearly seven years after discovery.Bridgegate’s framework contained more actants and relationships than Pizzagate’s.Source: Introduction · p. 4
  • The authors hypothesize that Pizzagate’s framework stabilized relatively quickly, with possible rapid expansion when new domains aligned with existing ones.They relate this cross-domain alignment to a monological belief system.Source: Introduction · p. 4

Data

The study analyzes archived online discussions for Pizzagate and archived New Jersey news reports for Bridgegate. The collection period captures different patterns of narrative development, while the authors note that broader corpus selection need not start from pre-established datasets.

Data

  • Pizzagate materials came from archived r/pizzagate and v/pizzagate discussions, while Bridgegate materials came from archived news reports seeded by articles about George Washington Bridge lane closures.Both corpora were cleaned to remove images, URLs, videos, advertisements, and non-English text.Source: Data · p. 5
  • 17,498 Pizzagate posts and 385 Bridgegate news reports form the corpora; the Pizzagate actant count remained stable after its first month, while Bridgegate revelations continued over time.The study’s end date missed April–May 2019 Bridgegate revelations that added an actant and relationship edges.Source: Data · p. 6
  • The Pizzagate dataset comprised 17,498 posts (42,979 sentences) through February 2018; Bridgegate used a similar end date and an archive of 385 news reports (20,433 sentences).Source: Data · p. 6
  • The pipeline can work without pre-established datasets by selecting target discussions from larger internet collections, including through topic modeling.The authors note that selecting a target domain could overlook related posts, while topic modeling can identify broader discussion domains.Source: Data · p. 6

A graphical narrative model for generation of posts and story fragments

The model represents narrative events as context-specific networks of actants and relationships. It treats observed stories as partial expressions of an underlying structure, while allowing relationships to be grouped and weighted by context.

A graphical narrative model for generation of posts and story fragments

  • Actants are graph nodes, and relationships between pairs or groups are labeled with their nature, context, and likelihood.Although multi-actant relationships can be represented as hyper-edges, the paper decomposes them into coupled pairwise edges.Source: A graphical narrative model for… · pp. 6–7
  • The graphs capture relationships that emerge under specific conditions, rather than predefined facts in a conventional semantic or entity-relationship network.The narrative framework supplies the overarching structure while specific actants, roles, and relationships vary with circumstances.Source: A graphical narrative model for… · p. 7
  • Stories and partial stories are modeled as outputs of an underlying domain-dependent structure whose parameters are informed by observed data.This is the paper’s generative account of how posts and story fragments express the narrative model.Source: A graphical narrative model for… · p. 7
  • Each context defines a weighted network in which actants have context-specific participation probabilities and edges carry distributions over relationship categories.The contexts are hidden parameters that express the environments in which actants operate.Source: A graphical narrative model for… · p. 7
  • Relationship categories group words, often verbs, that imply similar relationships, while relationships may also be implied by other syntactic structures.The “Abuse” category, for example, includes domain-specific verbs connecting named people to “Children” in Pizzagate.Source: A graphical narrative model for… · pp. 7–8, Figure 1 · p. 8

Learning narrative structure from large scale and unstructured text data

The pipeline treats posts and reports as partial samples of a hidden narrative network, then estimates its actants and context-specific relationships from corpus text. It builds a hierarchical network that can be examined at multiple scales.

Learning narrative structure from large scale and unstructured text data

  • Stories are modeled as samples of a hidden network: users select a context, sample actants and relationships, then compose a post or story fragment.The model assumes each post activates only a subgraph of the narrative framework.Source: Learning narrative structure from… · pp. 8–9
  • Syntax extraction combines dependency patterns and semantic role labeling, then cleans and deduplicates relationship tuples; pronouns are resolved using sentences grouped by post.The extracted tuples have two noun-phrase arguments and a verb or other phrase as the relationship.Source: Learning narrative structure from… · pp. 10, 12
  • Corpus-wide aggregation groups semantically similar phrases into actants and preserves contextual roles as supernodes and subnodes.For example, one sentence is split into three pairwise relationships, which are then aggregated across the corpus.Source: Learning narrative structure from… · pp. 9, 11
  • The two-stage actant method groups frequent entities and concepts into contextual supernodes, then clusters phrase embeddings within each supernode into subnodes.The method uses BERT embeddings and k-means for subnode creation, with small clusters deleted and similar labeled clusters merged.Source: Learning narrative structure from… · pp. 12–13
  • Context-specific relationships are ranked by verb significance, using conditional probabilities to downweight common verbs and a KL-divergence score to select salient ones.The comparison uses verb frequency in sentences containing a pair of actants against verb frequency across the corpus.Source: Learning narrative structure from… · p. 14
  • The final network supports multiscale projections, while centrality analysis identifies structural breaks such as the Wikileaks node separating Pizzagate domains.Bridgegate has no small set of central nodes whose removal disconnects the network, because main-actant connections predated the lane-closing conspiracy.Source: Learning narrative structure from… · p. 15

Community detection

The method identifies stable community cores from repeated community-detection runs, then adds sufficiently connected shared nodes to form potentially overlapping communities. It uses these communities to isolate frequently activated parts of the narrative framework.

Community detection

  • The method treats network communities as interaction subdomains and notes that Louvain communities can vary across runs, with loosely connected nodes shifting assignments.Repeated co-occurrence helps distinguish stable core nodes from nodes shared across communities.Source: Community detection · p. 15
  • Repeated Louvain runs produce a normalized node co-occurrence matrix, whose thresholded connected components define disjoint community cores.The number of nontrivial connected components determines the number of cores.Source: Community detection · pp. 15–16
  • Each core is extended with nodes whose co-occurrence with a core member exceeds a lower threshold, allowing the resulting communities to overlap.The method retains co-occurrence counts and disregards nodes at or below the extension threshold.Source: Community detection · p. 16
  • After community detection, frequently activated subnodes are grouped by supernode to identify the central narrative framework and less frequent components.Less frequently activated communities may represent meta-narratives, unrelated conversations, or emerging narrative frameworks.Source: Community detection · pp. 16–17

Visualization

The pipeline first generates graph visualizations automatically, then uses supervised visualization tools to make layouts and relationship labels easier to read. Different tools support inter-domain views and substantively labeled narrative graphs.

Visualization

  • NetworkX automatically renders actants as nodes and relationships as directed edges, labeling edges with the highest-significance phrases.Source: Visualization · p. 17
  • User-supervised visualization allows layout and labeling choices, with MuxViz used for inter-domain connections and Oligrapher for substantively labeled edges.Gimp and Photoshop support the edge-labeled visualizations, and other outputs use Cytoscape or Gephi.Source: Visualization · p. 17
  • The final visualization parameters are set by the user in the individual applications.Source: Visualization · p. 17

Evaluation

Because neither Pizzagate nor Bridgegate has a canonical ground-truth graph, the authors evaluate against expert-labeled New York Times illustrations and articles. They assess actant coverage and relationship matching, while revising inconsistent or incomplete relationship labels.

Evaluation

  • The evaluation uses expert-labeled graphs rather than ground truth because canonical accounts are unavailable or contested for both cases.The authors note that Bridgegate’s canonical summary had not yet been written while appeals and further reporting continued.Source: Evaluation · p. 17
  • New York Times illustrations and accompanying articles provide the expert reference, but relationship labels are poorly specified for Pizzagate and inconsistent for Bridgegate.The Pizzagate reference omits frequently mentioned actants including Bill Clinton and the Clinton Foundation.Source: Evaluation · pp. 17–18
  • For Pizzagate, two annotators labeled remaining edges from the accompanying article, with an arbitrator resolving disagreements; Bridgegate labels were likewise revised by a three-person team.Bridgegate speculative questions were excluded from the revised graph because they were not inter-actant relationships.Source: Evaluation · pp. 17–18
  • The evaluation separately measures actant extraction and relationship extraction, matching relationship phrases to expert labels by embedding cosine similarity with a 0.85 threshold.The actant evaluation checks illustrated nodes with and without a threshold and counts additional extracted actants.Source: Evaluation · p. 18

Limitations

The authors identify constraints in extraction, temporal representation, presentation, and generalization; several pipeline capabilities remain out of scope.

Limitations

  • Noisy extraction—especially from social-media posts—can distort relationships; cautious filtering trades coverage for confidence, while entity and domain aggregation remain ambiguous.The authors also identify ambiguity in aggregating relationships and assigning supernodes to domains.Source: Limitations · pp. 18–19
  • Online sources make consistent timestamps difficult, while newspaper dates are less problematic but news-service republication can create duplicates.Inconsistent timestamping and reposting or quoting earlier posts exacerbate the problem for conspiracy-theory sources.Source: Limitations · p. 19
  • Some supernodes and relationships still need hand labeling; relationship labels are less semantically rich and grammatical, and high-level narrative abstractions are not addressed.Source: Limitations · p. 19
  • The comparison illustrations show only minimal sets, while the frameworks omit event order and their visualizations can vary across software and be difficult to navigate.The authors describe their frameworks as snapshot views that do not include the order in which inter-actant relationships are established.Source: Limitations · pp. 19–20
  • Because the analysis covers only Pizzagate and Bridgegate, the authors say its observations cannot be generalized to either genre’s structural features.The cases were selected because external narrative-framework graphs were available for validation.Source: Limitations · pp. 19–20
  • The pipeline currently handles only English-language materials and cannot analyze rapidly changing data streams in real time.Source: Limitations · p. 20

Results

The results build supernodes and subnodes from extracted relationships and clustered arguments for both corpora. Pizzagate’s evaluation actants remain fully covered above a 50-mention threshold, while a higher threshold misses some Bridgegate actants.

Results

  • For both corpora, the pipeline tokenizes and stems extracted headwords, clusters argument embeddings with k = 20, then prunes and merges subnodes into supernodes.The resulting supernodes and subnodes are summarized in Table 2, with a fuller table in S1 File.Source: Results · p. 20, Table 2 · p. 22
  • All Pizzagate evaluation actants are covered when the actant list is limited to those mentioned more than 50 times.The comparison uses gold-standard evaluation data; Table 3 summarizes pipeline actant discovery against that data.Source: Results · p. 20, Table 3 · p. 22
  • Raising Bridgegate’s actant threshold from 50 to 150 mentions misses five actants and their relationships, while other central reporting and legal figures replace these misses.The named misses are Lori Grifa, Evan Ridley, Phillip Kwon, Paul Nunziato, and Nicole Crifo; replacements include Randy Mastro, Michael Critchlet, and Shawn Boburg.Source: Results · pp. 22–23
  • Fig 6 contrasts missed low-frequency cannibalism with frequently mentioned Bill Clinton and the Clinton Foundation, which the NY Times illustration omits.The figure uses grey for undiscovered top-ranked actants and green for highly ranked discoveries absent from the illustration.Source: Results · p. 22, Figure 6 · p. 23
  • Collapsing subnode subgraphs yields relationships between supernodes, labeled with the highest-relevance score among their subgraph edges.Table 4 compares extracted edges with NY Times and expert-labeled edges, including recall and cosine-similarity statistics.Source: Results · p. 23

Discussion

The pipeline builds multi-scale narrative networks that expose Pizzagate’s linked domains and Bridgegate’s single-domain structure. Its finer-grained graphs add relationships beyond overview illustrations, while the authors suggest structural differences may help distinguish conspiracy theories from actual conspiracies.

Discussion

  • Pizzagate’s central framework spans four domains, organized around seven cores, while nucleations and unrelated cores remain less connected.The domains are Democratic politics, the Podestas, casual dining, and Child Sex Trafficking and Satanism.Source: Discussion · p. 25
  • Removing Wikileaks relationships disconnects Pizzagate’s other domains into smaller components, whereas Wikileaks links them in the aggregate graph.The comparison indicates that the framework’s cross-domain connection depends on relationships generated by Wikileaks subnodes.Source: Discussion · p. 25, Figure 9 · p. 26
  • The full Pizzagate graph also contains meta-narratives, background communities, possible new-framework nucleations, and unrelated discussions.The detected Edgar Welch meta-narrative matches expert annotator annotations; some other communities may connect Pizzagate to broader conspiracy narratives.Source: Discussion · pp. 27–28
  • Bridgegate forms one connected component of 386 nodes, consistent with its New Jersey politics domain and unlike Pizzagate’s dependence on cross-domain links.Removing conspiracy-related relationships would still leave New Jersey politics connected.Source: Discussion · p. 28
  • Bridgegate’s cast of actants took nearly six years to be fully described, with new actants spiking around newly discovered aspects and court cases.By comparison, Pizzagate’s actants emerged over approximately one month.Source: Discussion · p. 28
  • The pipeline combines macro community overviews with meso- and micro-scale subnode networks and ego-networks to expose context-dependent relationships.Separate power and contextual-interaction networks represent professional or personal ties and relationships grounded in particular contexts.Source: Discussion · pp. 28–29
  • The finer-grained graphs reveal Pizzagate actants and relationships absent from the NY Times illustration, including Wikileaks’ role linking otherwise disconnected domains.The authors also describe richer relationships in the Bridgegate Chris Christie ego-network and suggest structural differences could help analysts distinguish conspiracy theories from actual conspiracies.Source: Discussion · p. 30
  • Established narrative frameworks can generate admissible story fragments, a capacity the authors say might also be used to interrupt harmful narratives.They also describe the pipeline as a way to study story generation and the factors shaping participation in narrative circulation.Source: Discussion · p. 31

Conclusion

The paper addresses the need to understand generative narrative frameworks alongside story circulation, given the difficulty of discerning fact from fiction across media. It proposes automated framework discovery and argues that structural patterns may help analysts distinguish conspiracy theories from actual conspiracies.

Conclusion

  • The authors identify a need to understand the narrative frameworks behind stories as well as how stories circulate across recognized and informal media.They note that distinguishing factual from fictional accounts is difficult given their speed and intensity of spread.Source: Conclusion · pp. 32–33
  • The automated Christie graph’s relationship descriptions closely match the roles portrayed in the NY Times Bridgegate graph, though its edge labels use only lead verbs.The accompanying phrases provide context for interpreting those verbs.Source: Figure 16 · p. 33
  • The authors hypothesize that actual conspiracies tend toward a single domain, robustness to deletions, and peripheral growth, while conspiracy theories may form rapidly and be fragile across linked domains.They propose that reporting adds evidence and actants to actual-conspiracy networks, while conspiracy-theory frameworks may stabilize quickly and shift when new domains align.Source: Conclusion · p. 33
  • The paper concludes that its methods derive narrative frameworks from a corpus and support macroscopic analysis of complex narratives.Source: Conclusion · p. 34
  • The authors suggest that identifying structural differences between conspiracy theories and actual conspiracies could help analysts assess emerging narratives and inform public safety and democratic-institution protection.They also describe the pipeline as offering insight into domains linked through interpretations of “hidden knowledge.”Source: Conclusion · p. 34

Supporting information

The supporting information file provides a detailed description of the processing pipeline and additional results for both datasets.

Supporting information

  • The supporting information describes the Fig 3 processing pipeline and provides additional results for both datasets.Source: Supporting information · p. 34

Related papers

Loading 2008.09961v1…