Source-linked AI summary
The emotional arcs of stories are dominated by six basic shapes
Andrew J. Reagan, Lewis Mitchell, Dilan Kiley, Christopher M. Danforth, Peter Sheridan Dodds
TL;DR
Written stories evoke emotional experiences whose trajectories can be studied computationally, but emotional arcs remain distinct from plot and intended meaning. The paper analyzes 1,327 Project Gutenberg books with SVD, hierarchical clustering, and a Self Organizing Map, finding broad support for six core emotional arcs across methods. It also reports that some arcs are associated with higher download-based success, while downloads are only a rough proxy for success.
Problem
The paper asks whether written stories contain recurring emotional trajectories that can be identified computationally and used to test theories about common core stories.
Method
The study extracts sentiment time series from 1,327 Project Gutenberg books and analyzes them with SVD, Ward hierarchical clustering, and a Self Organizing Map.
Results
Six core emotional arcs receive broad support because they appear as SVD modes, Ward clusters, and unsupervised-learning clusters.
Takeaways & Limitations
Emotional arcs may support analysis of large literary collections and could aid story generation, argument construction, and teaching common sense to artificial intelligence systems.
Takeaways & Limitations
Downloads are only a rough proxy for story success, leaving sales and cultural influence as more meaningful measures for further analysis.
Abstract
from arXiv · showhide
Advances in computing power, natural language processing, and digitization of text now make it possible to study a culture's evolution through its texts using a "big data" lens. Our ability to communicate relies in part upon a shared emotional experience, with stories often following distinct emotional trajectories and forming patterns that are meaningful to us. Here, by classifying the emotional arcs for a filtered subset of 1,327 stories from Project Gutenberg's fiction collection, we find a set of six core emotional arcs which form the essential building blocks of complex emotional trajectories. We strengthen our findings by separately applying Matrix decomposition, supervised learning, and unsupervised learning. For each of these six core emotional arcs, we examine the closest characteristic stories in publication today and find that particular emotional arcs enjoy greater success, as measured by downloads.
I. INTRODUCTION
Written stories convey emotional experiences through words, and emotional arcs provide a way to study those trajectories computationally. The paper tests whether recurring emotional patterns can be identified using complementary data-science methods.
- Written stories must capture the transfer of experience without tone, facial expression, or body language, including the emotional experience evoked in readers.
- The study examines emotional arcs as trajectories invoked by words, distinguishing them from plot and intended meaning.Different plots and structures can produce similar emotional arcs.
- Automated emotional-arc extraction is used to test folkloristic theories about recurring core stories and quantify trends as literary analysis becomes data-rich.
- The analysis combines SVD, hierarchical clustering with Ward’s method, and a Self Organizing Map, with each method contributing a different analytical function.SVD finds a basis, clustering classifies arcs, and SOM generates similar arcs from noise.
- The paper organizes its analysis into methods, combined results, and conclusions, with a graphical outline provided in supplementary material.
A. Emotional arc construction
Emotional arcs are constructed by sliding fixed word windows through filtered Project Gutenberg fiction and converting each window into a sentiment score. The resulting time series can then be decomposed into orthogonal emotional-arc modes.
- Emotional arc construction: Sentiment is measured in sliding 10,000-word windows using the Hedonometer and labMT dictionary, selected for lexical coverage and meaningful word-shift analysis.The paper notes that dictionary methods generally perform worse than random on individual sentences, motivating longer windows.
- Project Gutenberg Corpus: The corpus contains 1,327 mostly fictional books selected from Project Gutenberg using English-language, length, download, and Library of Congress classification filters.Books have between 20,000 and 100,000 words and more than 40 downloads.
- Emotional arc construction: Each book’s emotional arc is generated by sliding a fixed Nw = 10,000-word window through the text and recording n Hedonometer sentiment scores.The segment spacing is Ns = (N − (Nw + 1))/n.
- Project Gutenberg Corpus: Front and back matter are removed using regular-expression matches that cover 98.9% of included books, after excluding titles containing specified keywords.
- Interpretation: The method captures major highs and lows but does not detect emotional moments confined to a paragraph or sentence.The Harry Potter example illustrates that an emotional arc alone cannot recover all subplots or plot details.
- Principal Component Analysis (SVD): SVD decomposes the matrix of book sentiment time series into an orthogonal basis of emotional-arc modes and associated mode coefficients.The right singular vectors form the orthonormal basis, while W = UΣ represents mode coefficients.
D. Hierarchical Clustering
The paper compares emotional arcs using hierarchical clustering and an unsupervised Self Organizing Map. Ward’s method minimizes within-cluster variance using distances between mean-centered book trajectories.
- Hierarchical Clustering: Ward’s hierarchical clustering generates story groups by minimizing variance between clusters of books.
- Hierarchical Clustering: The clustering uses mean-centered emotional arcs and a distance matrix computed across corresponding sentiment windows in pairs of books.
- Self Organizing Map (SOM): The Self Organizing Map clusters emotional arcs by matching each arc to the most similar arc among randomly collected candidates.
- Self Organizing Map (SOM): The SOM uses an 8x8 square grid containing 64 nodes, roughly 5% of the number of books.
- Self Organizing Map (SOM): The neighborhood influence function uses α = −0.15, while the learning adaptation function uses β = −0.15.These are described as standard training hyper-parameters for the reported results.
III. RESULTS
Across a filtered corpus of 1,327 books, three independent analyses consistently identify six core emotional arcs: rise, fall, fall-rise, rise-fall-rise, and fall-rise-fall patterns, with SVD also identifying Icarus as the negation of Man in a hole.
- Core emotional arcs: 1,327 books yield broad support for six emotional arcs: Rags to riches, Tragedy, Man in a hole, Cinderella, and Oedipus, with Tragedy also called Riches to rags.The supplied list names five positive labels because Tragedy/Riches to rags is one arc; the sixth is Icarus, identified as the negation of Man in a hole in the SVD analysis.
- Cross-method support: The same six arcs emerge as SVD modes, hierarchical-clustering groups, and unsupervised-learning clusters.The paper treats agreement across these three methods as support for the arcs.
- SVD results: The first four SVD modes recognize the core arcs, while modes 5 and 6 combine them into compositions.Modes 7–12 explain comparatively little variance and do not align with core archetypes from the other methods.
- SVD results: The first 12 SVD modes explain 80% of variance from mean-centered time series and 94% from raw time series.The modes are derived from mean-centered emotional arcs, so the first mode need not represent the language positivity bias or average labMT score.
- SVD results: Mode 1 captures 30% of the variance and spans both Rags to riches and Tragedy emotional arcs.The analysis examines both positive and negative versions of modes because SVD coefficients can have either sign.
- Characteristic stories: Closest SVD examples include The Winter’s Tale for Rags to riches, Lady Susan for Tragedy, and The Magic of Oz for Man in a hole.The paper also identifies Children of the Frost as closely following Man in a hole and Romeo and Juliet as a famous tragedy ranked eighth among sorted tragedies.
B. Hierarchical Clustering
Hierarchical clustering separates the corpus into groups corresponding mainly to Man in a hole, Tragedy, Rags to riches, and Oedipus arcs, with the first four arcs appearing among the eight most different clusters.
- Cluster structure: The clustering analysis displays a dendrogram of the 60 clusters with highest linkage cost and reports average silhouette coefficients.Within-cluster silhouette distributions are used to assess the appropriate number of clusters.
- Related SVD visualization: Figure 3 shows the top 12 SVD modes, their singular-value weighting, and normalized per-book coefficients in the range -1 to 1.The figure scales the first singular-value entry to 1 and uses Tukey box plots for the coefficient distributions.
- Cluster structure: Characteristic books label clusters by selecting the most central book based on total distance to other books in each cluster.The clusters are treated as fully connected weighted networks, and the most central node provides the representative book.
- Cluster results: The first four of the six emotional arcs appear among the eight most different clusters.This result is reported with reference to Fig. S15.
- Cluster results: Man in a hole clusters account for 30% of the Gutenberg corpus, while Tragedy, Rags to riches, and Oedipus account for 32%, 5%, and 31%, respectively.Man in a hole clusters span different variances and remain separate from the other arcs.
- Related SVD visualization: Figure 4 compares the first three SVD modes and their negations with the closest stories, reporting each story’s Project Gutenberg ID and download count.Emotional arcs are rescaled using the inverse mode coefficient so they can be viewed on the same scale as the SVD modes.
C. Self Organizing Map (SOM)
Unsupervised Self-Organizing Maps reveal spatially coherent groups of stories whose emotional arcs correspond to recurring core patterns. These groups reinforce the identification of recognizable story types across the Gutenberg corpus.
- SOM results: Seven spatially coherent groups emerge on the two-dimensional SOM, containing five core emotional arcs with differing variance.The B-Matrix demonstrates clustering strength, while the heat map shows where stories select winning nodes.
- SOM results: The SOM’s seven groups are organized as A and G, B and I, C, D, E, H, and F.H and F are spatially close but remain separated by the B-Matrix because their arcs are distinct.
- Characteristic arcs: The groups contain recognizable “Man in a hole,” “Rags to riches,” “Oedipus,” “Icarus,” and “Tragedy” arcs.Panels A and G show varying “Man in a hole” strength; D contains “Icarus,” while E and H contain “Tragedy.”
- Related clustering view: The dendrogram displays Ward-cluster structure using average emotional arcs, central books, cluster sizes, and silhouette values.Its 60 leaf nodes report cluster sizes and the most central book in each cluster; higher average silhouette values indicate a more appropriate cluster count.
E. The Success of Stories
The study examines whether emotional trajectories relate to story success by comparing downloads across stories associated with SVD modes. Modes corresponding to particular arcs show higher download levels, although the results are influenced by individual books and downloads are only a rough success proxy.
- Download results: “Icarus” (-SV 2), “Oedipus” (-SV 3), and two sequential “Man in a hole” arcs (SV 4) are the three most successful emotional arcs.Both polarities of modes 3 and 4 have markedly higher median downloads, while high variance makes the mean less informative.
- Download results: The first four modes contain the greatest total number of books but are not the most popular by downloads.The comparison examines books most similar to each SVD mode rather than all books equally.
- Implications: The authors conclude that emotional arcs are important for story success, while treating downloads as only a rough proxy for success.The paper proposes that emotional-arc analysis could support story generation, argument construction, and teaching common sense to artificial intelligence systems.
- Limitations and extensions: The analysis is limited by the use of a less curated fiction selection and cannot yet answer detailed popularity questions across time and regions.The authors identify curated full-text fiction as an extension for more detailed comparisons.
- Download results: Modes SV 3 through -SV 4 exhibit higher average downloads and more variance than the other modes.The download distributions span 20 to 30,000 downloads and are shown in log10 space.
Appendix A: Plot theories
The appendix distinguishes emotional arcs from plot while reviewing established theories that classify stories by plot structure. It also describes sentiment time-series construction, including sliding-window choices and normalization across books.
- Plot theories: The study treats emotional arcs as trajectories of reader-perceived sentiment rather than direct measures of plot or intended meaning.Different plots can produce similar emotional arcs, and a falling sentiment trajectory can arise from varied plot and structural combinations.
- Plot theories: Existing plot theories propose systems ranging from three patterns to twenty or thirty-six specific plots.Examples include happy, unhappy, and tragic endings; conflict categories; Booker’s seven structures; Tobias’s twenty plots; and Polti’s thirty-six plots.
- Plot theories: Booker’s seven narrative structures include overcoming the monster, rags to riches, the quest, and voyage and return.The appendix also lists comedy, tragedy, and rebirth among Booker’s examples.
- Emotional arc construction: Emotional arcs are generated by sentiment analysis over sliding windows whose length, breakpoint, and overlap determine temporal resolution and comparability.The method balances meaningful sentiment scores, temporal resolution, and consistency across texts, then normalizes arc length for books of different lengths.
- Emotional arc construction: The analysis uses the LabMT dictionary to compute average happiness for text samples and subtracts each arc’s mean before distance measurement or clustering.The normalized frequency of each dictionary word is combined with its sentiment score to obtain the sample’s average happiness.
1. Null emotional arc construction
The null emotional arcs are created by disrupting word order, while corpus preprocessing filters English fiction and removes front and back matter. The filtering and cleanup procedures are documented with their coverage and average removal rates.
- Null construction: The null set is generated by randomly shuffling the words within each book, producing a word-salad version that removes the original sequence.Alternative null constructions include phrase-level Markov sampling, continuous-space random walks, and sentence shuffling.
- Null construction: A second null construction uses a 2-gram Markov chain to produce locally coherent but narratively disrupted text.The appendix labels this output the “nonsense” version and provides an excerpt from Alice’s Adventures in Wonderland.
- Corpus preprocessing: The corpus is filtered to English books of 20,000–100,000 words, sufficient Project Gutenberg downloads, English-fiction Library of Congress classes, and titles without selected keywords.The manuscript version uses more than 40 downloads, while the broader processing description also documents a more-than-20-download threshold.
- Corpus preprocessing: Front- and back-matter filters remove text from 98.9% of ebooks, averaging less than 1% of beginning lines and 3–4% of ending lines.The first detection pass succeeds in 78.9% of cases.
Appendix E: Principal Component Analysis (SVD)
The SVD appendix explains how emotional-arc matrices are decomposed into modes and coefficients, then reconstructed and inspected through representative stories. It also documents mode coverage and related story lists.
- SVD decomposition: SVD decomposes the emotional-arc matrix into an orthogonal basis of emotional modes.The appendix illustrates the decomposition with randomly selected arcs, singular values, and representative sinusoidal modes.
- SVD decomposition: The matrix is written as W · V T, with each book’s mode coefficients in W and each mode represented by a row of V T.Book i reconstructed with mode j is W[i, j] · V T[j, :].
- Characteristic stories: The appendix presents closest stories for modes 4–6 and lists books associated with each mode by variance support and downloads.Supplementary tables organize stories represented by core emotional arcs, including top-ten lists sorted by downloads.
Appendix F: Additional Hierarchical Clustering Figures
The supplementary clustering figures show solutions with two through nine clusters and provide silhouette plots for evaluating the corresponding groupings. All clustering uses agglomerative clustering with Ward’s method.
- Cluster evaluation: Figures S17–S18 provide silhouette plots for the 2–5 and 6–9 cluster solutions, respectively.These plots accompany the cluster visualizations for comparing candidate cluster counts.
Appendix G: Additional SOM Figures
The appendix visualizes the emotional arcs nearest to the nine most frequently winning SOM nodes. These characteristic stories make the previously identified core arcs visible.
- SOM visualizations: The Self Organizing Map displays emotional arcs closest to each of its nine most frequently winning nodes.The visualization uses a winner-take-all implementation of the SOM.
- SOM visualizations: The top nine SOM-node vectors are accompanied by their closest sentiment time series.These pairings show representative emotional-arc patterns associated with the nodes.
- SOM visualizations: The core stories identified by other methods are readily visible in the SOM visualization.
Appendix H: Null comparison details
The null-comparison appendices test whether emotional-arc structure persists in nonsense and word-salad texts. Across decomposition, clustering, and SOM analyses, these controls show weaker or absent coherent structure than the real books.
- Null-analysis setup: The null analyses apply SVD, agglomerative clustering, and SOM to nonsense English fiction books with more than 40 downloads.The appendix first presents example nonsense and word-salad texts, then examines the resulting time series and full analyses.
- Null-analysis results: The nonsense-book SVD, clustering, and SOM figures provide full null-control results beyond the example book comparisons.The appendix organizes these analyses as additional checks using transformed texts.
- Example null comparisons: The true Romeo And Juliet emotional arc has more structure than both 2-gram Markov nonsense versions and randomly shuffled word-salad versions.Figures S20 and S21 compare the original book with 11 generated or shuffled versions in each control.
- SVD comparison: Word-salad SVD results show higher-frequency modes appearing sooner and a more even spread of mode coefficients.
- SVD comparison: Nonsense-book SVD spectra are muted, indicating lower total variance explained and less important ordering of singular vectors.The comparison is between emotional arcs of nonsense books and individual Project Gutenberg books.
- Clustering comparison: Ward clustering of word-salad emotional arcs has comparatively low linkage cost, indicating an absence of distinct clusters.The hierarchical clustering therefore lacks the separated structure seen in meaningful grouped patterns.
- Clustering comparison: Word-salad cluster means and central emotional arcs have high variance without a visible signal.This result is shown for four clusters at linkage threshold 850.
- SOM comparison: Nonsense-book SOM results form clear clusters in the imposed two-dimensional space, but their winning-node arcs show little coherent structure overall.The B-Matrix indicates clustering, while the top-node emotional arcs remain weakly structured relative to real books.