Source-linked AI summary
Generative artificial intelligence enhances creativity but reduces the diversity of novel content
Anil R. Doshi, Oliver P. Hauser
TL;DR
The paper examines whether access to generative-AI ideas makes writers more creative or anchors them toward less diverse outputs. In an online experiment, AI access increased stories’ novelty and usefulness, while AI-assisted stories were more similar to one another.
Problem
The paper addresses whether access to generative-AI ideas causally changes the creativity of human-written stories.
Method
An online experiment exogenously assigned writers to human-only or human-with-GenAI-ideas conditions and evaluated their stories’ creativity.
Results
GenAI access increased average story novelty by 6.7% and usefulness by 6.4% relative to human-only writing.
Takeaways & Limitations
GenAI can enhance creative-story evaluations, particularly for less creative writers, but AI-assisted stories become more similar to one another.
Takeaways & Limitations
The design disallowed customized or repeated GenAI interactions, so its estimated effects may be lower bounds for more interactive use.
Abstract
from arXiv · showhide
Creativity is core to being human. Generative artificial intelligence (GenAI) holds promise for humans to be more creative by offering new ideas, or less creative by anchoring on GenAI ideas. We study the causal impact of GenAI on the production of a creative output in an online experimental study where some writers are could obtain ideas for a story from a GenAI platform. Access to GenAI ideas causes an increase in the writer's creativity with stories being evaluated as better written and more enjoyable, especially among less creative writers. However, GenAI-enabled stories are more similar to each other than stories by humans alone. Our results have implications for researchers, policy-makers and practitioners interested in bolstering creativity, but point to potential downstream consequences from over-reliance.
Results
Access to GenAI ideas improved evaluator-assessed creativity and story characteristics, particularly for less creative writers, while reducing creators’ perceived ownership and increasing similarity among stories. Writers’ self-assessments showed no significant differences across conditions.
- Overall effects: 6.7% higher novelty and 6.4% higher usefulness were found for stories with GenAI ideas versus Human only stories.Novelty increased by 6.7% (b=0.259, p=0.001; Human only mean 3.85), and usefulness by 6.4% (b=0.319, p<0.001; Human only mean 5.02).
- Dose of GenAI access: 9.0% higher usefulness resulted from access to up to five AI ideas versus no GenAI access, exceeding the one-idea condition by 5.1%.Access to one idea increased usefulness by 3.7% (b=0.185, p=0.039), while access to five increased it by 9.0% (b=0.453, p<0.001).
- Self-assessment: Writers’ self-assessments found no significant differences in novelty, usefulness, or story characteristics between GenAI and non-GenAI conditions.This contrasts with evaluators’ positive assessments of creativity and other story characteristics.
- Story characteristics: GenAI-assisted stories were more enjoyable, more likely to contain plot twists, and better written, with effects generally larger when writers could access five ideas.For five ideas, enjoyment was b=0.375 (p<0.001), plot twists b=0.468 (p<0.001), and better writing b=0.372 (p<0.001).
- Heterogeneity by inherent creativity: Among low-DAT writers, five GenAI ideas improved novelty by 10.7% and usefulness by 11.5%, while high-DAT writers showed little creativity effect.For low-DAT writers, one idea improved novelty by 6.3% and usefulness by 5.5%; five ideas increased assessed writing quality by up to 26.6%.
- Ownership and similarity: Evaluators attributed less ownership to writers whose stories had GenAI access and could identify AI assistance, while stories across GenAI conditions were more similar to one another.Evaluators’ AI-influence assessments were significant for one idea (b=6.21, p<0.001) and five ideas (b=4.96, p<0.001); ownership was 25.4% lower with one idea.
Discussion
Experimentally providing writers access to GenAI increased average story novelty and usefulness, with especially large gains for less creative writers. The authors also identify design limitations and caution that wider adoption could have consequences for variation in creative outputs.
- Main findings: Access to GenAI causally increased average story novelty and usefulness relative to writers working alone.The effect was measured experimentally in an online short-story-writing study.
- Main findings: Requesting multiple GenAI ideas—up to five different starting points—particularly drove the creativity gains.The ideas formed a branching “tree” of potential storylines.
- Heterogeneous effects: Less creative writers gained 10% to 11% in creativity and 22% to 26% in enjoyability from GenAI access.GenAI-enabled stories were also described as more novel, more useful, well written, and enjoyable overall.
- Heterogeneous effects: GenAI access equalized story evaluations by removing advantages or disadvantages associated with writers’ inherent creativity.This finding is consistent with evidence that GenAI can help less productive workers in other domains.
- Limitations: The design minimized endogenous writer decisions but did not allow customized prompts or repeated writer–GenAI interactions, which may understate the effect.Writers could still opt into receiving ideas, preserving investment and receptiveness to GenAI output.
- Caution and future research: Despite enhancing average creativity, wider adoption of GenAI for creative tasks raises concerns about sufficient variation in the resulting outputs.The study used GPT-4, whose rapid technological evolution may limit how long the findings remain current.
Methods
The study randomly assigned writers to produce stories with no GenAI idea, one GenAI idea, or five GenAI ideas, after measuring trait creativity. Separate evaluators assessed the stories’ creativity and stylistic characteristics using preregistered outcomes and regression analyses.
- Writer study: Writers first completed a divergent association task measuring trait creativity, then wrote an eight-sentence story about one of three randomized adventure topics.The topics were the open seas, the jungle, and a different planet.
- Experimental conditions: Writers were randomized to Human only, Human with 1 GenAI idea, or Human with 5 GenAI ideas conditions.The Human only condition provided only a text box, while GenAI conditions offered optional generated ideas.
- Evaluator study: Separate evaluators assessed 293 stories on stylistic characteristics, novelty, usefulness, and perceived human-versus-AI authorship.The evaluator study produced 3,519 evaluations by 600 evaluators, with each evaluator assessing up to six stories.
- Outcome variables: Creativity was operationalized through average novelty and usefulness indices, while additional outcomes captured enjoyment, writing quality, boredom, humor, surprise, and changed expectations.Outcome variables were generally measured on 9-point scales from 1 (not at all) to 9 (extremely).
- Statistical analysis: Analyses used OLS regressions with robust standard errors, clustering evaluator-study errors at the participant level, and treated Human Only as the baseline.The study was preregistered at AsPredicted.org and received ethics approval from UCL and the University of Exeter.
Supplemental Information for: Generative artificial intelligence enhances creativity
The supplemental information documents the study’s evaluation measures, supporting analyses, and evidence that human-only stories more closely resemble stories from participants who declined AI ideas than their assigned AI ideas. This comparison suggests the human-only group did not generally use AI covertly.
- Supplemental materials: The supplemental materials organize the study’s questions, supporting tables and figures, similarity analysis, preregistered analysis, and study screenshots.The contents list identifies six sections covering these materials.
- Similarity analysis: The similarity analysis randomly assigned simulated AI ideas to human-only participants and GenAI-idea participants who did not access AI ideas.This design tested whether human-only participants may have used AI despite reporting that they did not.
- Similarity analysis: Stories from human-only participants more closely matched stories by participants who chose not to generate a GenAI story than their assigned GenAI ideas.The distributions’ mode and range were more similar for the first two groups, and summary statistics reflected this pattern.