Source-linked AI summary
Humor in Collective Discourse: Unsupervised Funniness Detection in the New Yorker Cartoon Caption Contest
Dragomir Radev, Amanda Stent, Joel Tetreault, Aasish Pappu, Aikaterini Iliakopoulou, Agustin Chanfreau, Paloma de Juan, Jordi Vallmitjana, Alejandro Jaimes, Rahul Jha, Bob Mankoff
TL;DR
The paper asks what differentiates funny captions in the New Yorker contest and compares unsupervised methods for ranking them, using side-by-side reader judgments. Negative sentiment, human-centeredness, and lexical centrality most consistently select funnier captions, and the authors make the corpus public for research and a shared task.
Problem
The paper asks what distinguishes funny captions from the other submissions in the New Yorker cartoon caption contest.
Source: §1 · pp. 1–2Method
Results
Negative sentiment, human-centeredness, and lexical centrality consistently select funnier captions, and captions reflecting participants’ collective wisdom outperform semantic outliers.
Source: §6 · p. 8Takeaways & Limitations
The public corpus includes selected cartoons, captions, manual entity annotations, extracted topics, and funniness scores for research and a shared task.
Source: §6 · p. 8
Abstract
from arXiv · showhide
The New Yorker publishes a weekly captionless cartoon. More than 5,000 readers submit captions for it. The editors select three of them and ask the readers to pick the funniest one. We describe an experiment that compares a dozen automatic methods for selecting the funniest caption. We show that negative sentiment, human-centeredness, and lexical centrality most strongly match the funniest captions, followed by positive sentiment. These results are useful for understanding humor and also in the design of more engaging conversational agents in text and multimodal (vision+text) systems. As part of this work, a large set of cartoons and captions is being made available to the community.
1 Introduction
The New Yorker’s weekly contest asks readers to caption a cartoon and choose a winner from the editors’ three selections. The paper uses computational ranking and reader judgments to study what distinguishes funny captions.
1 Introduction
- Readers caption each weekly cartoon, then choose the winner from the editors’ three selected submissions.The contest had been running for more than 10 years and had become a cultural phenomenon.Source: §1 · p. 1
- The contest has prompted questions about what makes cartoons funny to New Yorker readers.Source: §1 · p. 1
- The paper ranks captions using unsupervised features and asks Mechanical Turk workers to judge which selected captions is funnier.Source: §1 · p. 2
2 Related Work
Prior humor research includes binary humor detection, sentiment and human-centeredness features, and tweet classification. This work instead ranks humorous texts and examines cartoon captions, including features based on depicted objects.
2 Related Work
- Mihalcea and Strapparava tested humorous-versus-nonhumorous one-line text classification using features such as alliteration, antonymy, and adult slang.Source: §2 · pp. 2–3
- Mihalcea and Pullman found negative sentiment and frequent social relationships and personal pronouns in humor; Reyes et al. used these and other features in a humor taxonomy.Source: §2 · p. 3
- Tweet studies classified types or topics, including irony, education, humor, and politics, while Zhang et al used crafted features based on humor theories and affect.Source: §2 · p. 3
- Unlike most prior binary humor classification, this work ranks humorous texts and examines cartoon captions, including features based on depicted objects.Source: §2 · p. 3
3 Data
The authors draw on a large corpus of contest captions and evaluate a subset of 50 cartoons and 298,224 captions, with contest-specific caption, cartoon, and score materials.
3 Data
- The source corpus contains more than 2 million captions from over 400 contests, while the experiments use 50 cartoons and 298,224 captions.Source: §3 · p. 3
- Each contest includes 5,000+ tokenized captions, the three editor-selected captions including the winner, frequent n-grams, and tfidf scores.Source: §3 · p. 3
- The dataset also includes manually labeled visible cartoon objects and antijokes collected from two sites devoted to unfunny captions.Source: §3 · p. 3
4 Experimental Setup
The authors compare unsupervised ranking methods and controls built from contest-caption properties, content, and cartoon references. They describe the methods and the evaluation measures used to compare them.
4 Experimental Setup
- More than a dozen unsupervised methods are compared with editor-selected captions and antijokes as controls; some methods rank toward either sentiment extreme.The methods are grouped as originality-based, generic, content-based, New Yorker, and control methods; ties are broken randomly.Source: §4 · p. 4
4.1 Originality-based methods
- Originality methods use a contest-caption lexical network for LexRank centrality, lexrank scores, thematic cluster sizes, and tfidf scores.The thematic clusters are produced with a graph clustering method.Source: §4.1 · p. 4
4.2 Content-based methods
- Content methods score Freebase entities, sentiment polarity, and human-centeredness using people and relatives word sets, proper names, and pronoun exclusions.Only 13.20% of captions had positive polarity, 51.09% negative polarity, and 25.33% mentioned at least one person; 3.60% contained a relatives-set word.Source: §4.2 · pp. 5, 7
4.3 Generic methods
- Generic methods measure syntactic complexity, references to cartoon objects, and unusual formatting such as punctuation or very long words.The authors manually labeled objects in each of the 50 evaluation cartoons to calculate concreteness.Source: §4.3 · p. 7
- Table 1 defines s4 and s3 using judge vote differences of 4+ and 3+ votes, respectively, and marks CU2R, CU3, OR2, and CU2 as best.Source: Table 1 · p. 7
5 Evaluation
The evaluation compared method-selected caption pairs through blinded Amazon Mechanical Turk judgments, with seven Turkers assessing each pair. It summarized outcomes using vote-margin thresholds and a score that included all pairwise comparisons, including ties.
5 Evaluation
- Seven Turkers judged each blinded caption pair for a cartoon, choosing which was funnier, whether both were funny, or neither was funny.All pairs from the methods were compared for each cartoon.Source: §5 · p. 7
- The evaluation reported three vote-based comparisons: thresholded vote margins and all pairwise comparisons, including ties.For each evaluation, ni counts cases meeting its constraint and si averages votes in favor minus votes against.Source: §5 · pp. 7–8
- 5,594/15,154 comparisons (36.9%) had a vote difference of at least four, while 8,131/15,154 (53.6%) had a difference of at least three.A random process would produce a difference of at least four votes, excluding ties, with probability 12.5%.Source: §5 · p. 8
6 Conclusion
The study found that negative sentiment, human-centeredness, and lexical centrality most consistently selected captions judged funnier, with positive sentiment and proper formatting also ranking strongly. The authors are making the corpus available for research and a shared task on funniness detection.
6 Conclusion
- Negative sentiment, human-centeredness, and lexical centrality most consistently selected captions judged funnier than alternatives.The authors also report that negative captions were funnier than positive captions, and that captions reflecting participants’ collective wisdom outperformed semantic outliers.Source: §6 · p. 8
- Positive sentiment and proper formatting were the next strongest features after negative sentiment, human-centeredness, and lexical centrality.The first two strongest features align with prior findings reported by Mihalcea and Pullman.Source: §6 · p. 8
- The public corpus includes 50 selected cartoons, more than 5,000 captions per cartoon, manually annotated entities, automatically extracted topics, and funniness scores.It is intended for research and a shared task on funniness detection.Source: §6 · p. 8
7 Future Work
The authors plan to extend their unsupervised approach with supervised and ensemble methods, while exploring additional language phenomena and semantic features. They note that ensembles may be ill-suited when captions are funny in different ways.
7 Future Work
- The authors plan to explore supervised methods and ensembles after this paper’s unsupervised methods.They caution that ensembles may not work well because equally funny captions can use different forms of humor, such as absurdity or irony.Source: §7 · p. 8
- Planned future work includes recognizing puns and other creative uses of language.The passage gives “Tell my wife I’ll be home in a minotaur” as an example pun.Source: §7 · p. 8
- The authors also plan to explore more semantic features.Source: §7 · p. 8