Source-linked AI summary
How AI Ideas Affect the Creativity, Diversity, and Evolution of Human Ideas: Evidence From a Large, Dynamic Experiment
Joshua Ashkinaze, Julia Mendelsohn, Li Qiwei, Ceren Budak, Eric Gilbert
TL;DR
The paper examines how exposure to LLM-generated ideas affects human creativity and collective idea diversity, a question made salient by growing exposure and prior research often involving active AI engagement. In a dynamic Alternate Uses Task experiment, participant ideas fed forward as examples for later participants. High AI exposure increased collective diversity and its rate of change, but did not increase individual creativity.
Problem
As exposure to LLM output grows, evidence is needed on how seeing AI-generated ideas affects the creativity, diversity, and evolution of human ideas.
Source: §1 · p. 1Method
Results
High AI exposure increased collective diversity and its rate of change, but did not affect individual creativity.
Source: §1.1 · p. 2, §6.5 · p. 11Takeaways & Limitations
The findings suggest passive exposure to AI ideas can affect collective thought, with effects varying across individuals and tasks.
Source: §6.5 · p. 11The study measured AI-idea effects using a single task, so other, especially more complex, tasks remain to be explored.
Source: §6.4 · p. 10
Abstract
from arXiv · showhide
Exposure to large language model output is rapidly increasing. How will seeing AI-generated ideas affect human ideas? We conducted an experiment (800+ participants, 40+ countries) where participants viewed creative ideas that were from ChatGPT or prior experimental participants and then brainstormed their own idea. We varied the number of AI-generated examples (none, low, or high exposure) and if the examples were labeled as 'AI' (disclosure). Our dynamic experiment design -- ideas from prior participants in an experimental condition are used as stimuli for future participants in the same experimental condition -- speaks to the interdependent process of cultural creation: creative ideas are built upon prior ideas. Hence, we capture the compounding effects of having LLMs 'in the culture loop'. We find that high AI exposure (but not low AI exposure) did not affect the creativity of individual ideas but did increase the average amount and rate of change of collective idea diversity. AI made ideas different, not better. There were no main effects of disclosure. We also found that self-reported creative people were less influenced by knowing an idea was from AI and that participants may knowingly adopt AI ideas when the task is difficult. Our findings suggest that introducing AI ideas may increase collective diversity but not individual creativity.
1 Introduction
The study asks how passive exposure to AI-generated ideas and disclosure of their AI origin affect human creativity and collective diversity. In a dynamic brainstorming experiment, high AI exposure increased collective diversity and its rate of change, but not individual creativity.
1 Introduction
- The study asks how seeing LLM-generated ideas affects human creativity, diversity, and evolution, including whether labeling ideas as AI moderates their influence.The authors distinguish passive exposure from active engagement, which is common in prior human-AI creativity studies.Source: §1 · p. 1
1.1 Findings
- High AI exposure increased collective idea diversity and its rate of change, but did not affect individual creativity.The ideas became more different from one another, not necessarily better; disclosure effects varied with self-reported creativity and task difficulty.Source: §1.1 · pp. 2–3
1.2 Defining Concepts and Variables.
- Creativity is often framed as divergent thinking, and the Alternate Uses Task asks people to produce original uses for everyday objects.The study elicits one idea per participant and focuses on originality.Source: §1.2.1 · p. 3
1.2.1 Creativity.
- Individual-level creativity is measured using the originality score from a fine-tuned GPT-3 classifier with r=0.81 overall correlation with human judgments.The task uses items on which the classifier had its highest accuracy and matches its training task and items.Source: §1.2.1 · p. 3
1.2.2 Idea Diversity & AI Adoption.
- The study treats creativity as an idea-level outcome and diversity as a property of an idea set, measuring semantic divergence and convergence toward AI examples.Ideas are embedded with SBERT, a model designed for sentences and used to capture semantic similarity.Source: §1.2.2 · p. 3
Idea diversity
- Idea diversity is the median pairwise cosine distance between idea embeddings in a set.Mean pairwise distance and average distance to the set centroid are used as robustness checks.Source: Idea diversity · p. 3
AI adoption
- AI adoption is measured as the maximum cosine similarity between a participant’s idea and the AI examples they saw.The maximum is used because inspiration may come from a single idea; mean and median similarities are noisier robustness checks.Source: AI adoption · p. 3
2 Related Work
Prior work on AI and creativity has focused largely on active co-creation, with mixed findings about creativity and diversity. This study uses a large-scale, multiple-worlds design to examine passive exposure and how ideas evolve as they feed forward between participants.
2.1 Human-AI Co-Creation
- Earlier human-AI co-creation research mostly studies active interaction, whereas this study examines passive exposure to off-the-shelf models.Prior findings suggest effects may vary with task difficulty and user attitudes toward AI output.Source: §2.1 · p. 3
- Prior studies report that generative AI can decrease idea diversity, increase individual creativity, or produce mixed creativity effects.This study adds a dynamic test in which ideas feed forward, systematically varies disclosure, and uses a large sample to estimate effects and heterogeneity.Source: §2.1 · pp. 3–4
2.1.1 Crowdsourced Creativity.
- Crowdsourcing studies show that the creativity and diversity of ideas people see can shape the ideas they produce.This study extends that approach by examining how creativity and diversity unfold across subsequent generations.Source: §2.1.1 · p. 4
2.1.2 Collective Dynamics & Multiple-Worlds Experiments.
- The multiple-worlds paradigm creates parallel, independently evolving groups with and without AI ideas, enabling comparisons of collective effects over time.The design models many worlds under controlled conditions rather than studying individuals in isolation.Source: §2.1.2 · p. 4
2.1.3 Our Contributions.
- The experiment separates exposure from disclosure and uses a large public sample to study attitude-related variation in human-AI outcomes.Because responses feed forward, the design can compare rates of change and examine cumulative effects over time.Source: §2.1.3 · p. 4
3 Experiment
The experiment varied AI-example exposure and disclosure while participants brainstormed creative uses for everyday items. Human ideas circulated through condition-specific response chains, allowing the study to examine how ideas developed across iterations.
3.2 Stimuli Creation
- The experiment used tire, pants, shoe, table, and bottle, selected because the creativity classifier had its highest human-judgment correlations on those items.AI examples were generated with zero-shot ChatGPT-3.5 prompts adapted from Stevenson et al. [74] to match human response lengths.Source: §3.2 · p. 4
3.3 Recruitment Strategy
- The study recruited volunteers through Facebook ads, Reddit, and the Creative Mornings newsletter, targeting technology- and creativity-oriented participants as well as neutral-source users.Recruitment aimed for intrinsically motivated participants and offered personalized information about their results.Source: §3.3 · p. 4, sample characteristics · p. 5
3.4.1 Pre-Treatment Questions.
- Participants answered pre-treatment questions about self-perceived creativity, perceived AI creativity, and attitudes toward AI.The attitude question was taken from Pew to limit response burden.Source: §3.4.1 · p. 5
3.1 Summary
- Participants completed five trials, each pairing an item with an experimental condition, with no item or condition repeated in the sequence.After viewing example ideas, participants generated a creative use for the assigned item.Source: §3.1 · p. 4, §3.4.2 · p. 5, Figure 2 · p. 5
3.4.3 Task Instructions.
- Participants ranked example ideas before submitting their own idea; exposure varied by whether AI ideas were included, and disclosure by whether those ideas were labeled.The ranking task was included to encourage engagement with the examples, and participants received feedback after submitting an idea.Source: §3.4.3 · p. 5
3.4.4 Response Chains.
- Each condition–item response chain reset every 20 trials; seven chains for each of 25 combinations yielded 3,500 targeted responses.Chains began with six seed ideas from the Organisciak Dataset, then incorporated ideas from earlier participants in the chain.Source: §3.4.4 · p. 6
4 Outcome Measures
The study assessed idea diversity, creativity, and AI adoption at local, global, and evolutionary levels. The measures distinguish individual-trial outcomes from change across response-chain iterations and comparisons across all responses in a condition.
4 Outcome Measures
- Local analysis measured outcomes per trial, evolution measured change with response-chain trial number, and global analysis compared all responses within a condition.Pairwise comparisons used a Holm-Bonferroni adjustment.Source: §4 · pp. 6–7
- Three outcomes—idea diversity, creativity, and AI adoption—were examined at local, global, and evolution levels; all ideas were embedded using SBERT.Table 1 summarizes the outcome measures and analysis levels.Source: §4 · p. 6, Table 1 · p. 7
- For diversity and AI adoption, cosine distance and cosine similarity, respectively, were scaled by 100 for easier interpretation.Source: §4 · p. 7
5 Results
High AI exposure increased collective idea diversity and its rate of change, but conditions did not differ in individual creativity. AI adoption varied with self-perceived creativity and prompt difficulty.
5.1 Recruited Participants
- The experiment received over 3,000 responses from 48 countries; participants completed four of five trials on average.The sample was 50% women, 43% men, 4% non-binary, 3% not disclosed, and 1% self-described; mean age was 34.92 years (SD=10.86).Source: §5.1 · p. 7
5.2 High AI exposure increases collective diversity
- Both high-exposure conditions had greater global idea diversity than control: disclosed (δ=0.31, p=0.001) and undisclosed (δ=0.26, p=0.001).The low-exposure undisclosed condition also exceeded control, but with a smaller effect (δ=0.11, p=0.001).Source: §5.2 · pp. 7–8, Figure 7 · p. 7
5.3 High AI exposure increases the speed at which idea diversity develops
- Relative to control, diversity slopes were higher for High Exposure, Undisclosed (β=0.53, p=0.03) and High Exposure, Disclosed (β=0.57, p=0.02).Low-exposure conditions did not differ from control in rate of change; control diversity decreased over trials (β=−0.39, p=0.03).Source: §5.3 · p. 8, Figure 8 · p. 8
5.4 Seeing AI ideas does not affect creativity
- Creativity did not differ by condition (F(4, 19.86)=0.12, p=0.97), and its temporal evolution also did not differ significantly.The trial-number-by-condition interaction did not improve model fit (χ2(4)=6.52, p=0.16).Source: §5.4 · p. 8
5.5 How AI Adoption differs depending on task and self-perceived creativity
- AI adoption was measured as the maximum similarity between a submitted idea and the AI examples a participant saw, and examined in relation to task and person characteristics.Attitudinal variables were included as moderators only when they improved the baseline model fit.Source: §5.5 · pp. 8–9
5.5.1 People who identify as creative are less influenced by AI dis-
- For lower-creativity participants, disclosure reduced the exposure-related increase in adoption; for higher-creativity participants, adoption effects did not differ by disclosure.Among higher-creativity participants, the contrast was Δ=−1.69, d=−0.14, p=0.59; among lower-creativity participants, it was Δ=7.77, d=0.65, p=0.03.Source: §5.5.1 · p. 9, Figure 9 · p. 9
5.5.2 Participants may adopt AI ideas more for harder prompts.
- For high exposure, prompt difficulty correlated with adoption more strongly when AI ideas were disclosed (ρ=0.8) than undisclosed (ρ=0.3).The authors call this finding speculative because the experiment used only five prompts.Source: §5.5.2 · p. 9, Figure 10 · p. 9
6 Discussion
The discussion interprets passive exposure to AI ideas as a way to study how ideas build on one another, finding greater collective diversity without greater individual creativity. It also qualifies the findings by task, participant, measurement, and design limits.
6.1 AI makes ideas different but not better
- In the control condition, responses converged over time; adding AI ideas partially reset that convergence, potentially providing additional voices in group brainstorming.The authors suggest this mechanism may counter groupthink, while noting that its effectiveness and boundary conditions need further study.Source: §6.1 · p. 10
6.2 High creativity people are less influenced by the source label of ideas
- Self-reported highly creative participants adopted ideas based on exposure rather than disclosure, whereas source labels affected adoption among lower-creativity participants.The authors suggest highly creative participants may rely more on an idea’s content than on its source.Source: §6.2 · p. 10
6.3 Participants may adopt AI ideas when the task is difficult
- When AI ideas were labeled, participants were more likely to adopt them for difficult prompts, but this finding is speculative because it rests on few items.The discussion connects this pattern to prior work on difficult tasks and automation reliance, without treating it as conclusive.Source: §6.3 · p. 10
6.4 Limitations And Future Work
- The study used one task, one operationalization of AI ideas, and a non-representative sample; its creativity measure captures originality, and its cultural-evolution design is stylized.The authors also flag the task-difficulty adoption result as speculative because it uses only five AUT items.Source: §6.4 · pp. 10–11
6.1 AI makes ideas different but not better
- AI exposure increased collective idea diversity, including its rate of change, but did not improve average individual creativity.The authors suggest the diversity shift may accumulate over time, even without higher individual creativity.Source: §6.1 · pp. 9–10, §6.5 · p. 11
6.5 Conclusion: Passive exposure to AI ideas affects collective thought
- The authors conclude that passive exposure affected collective thought, with effects varying across individuals and tasks, while individual creativity did not increase.They call for further research using multiple-worlds experiments.Source: §6.5 · p. 11
A.1 Choosing AUT Items
- Individual-level creativity was operationalized as classifier-scored originality, using five AUT items selected for high classifier accuracy.The classifier was used on the same task and items it was trained on.Source: §A.1 · p. 13
A.2 Generating GPT Ideas
The appendix describes how GPT ideas were generated and how local, condition-level, and over-time outcomes were analyzed. Its methods compare modified-prompt word counts with human responses and model diversity over response-chain trial numbers.
A.2 Generating GPT Ideas
- The modified GPT prompt produced ideas averaging 4.44 words (SD = 1.34), closer to human responses at 4.56 (SD = 4.97) than the original prompt at 25.38 (SD = 8.55).The authors added an instruction to match the mean human response length; the difference was significant at p < 0.001.Source: §A.2 · p. 13
B.1 Local Level
- Local-outcome models used crossed participant and response-chain random intercepts to account for repeated measures and response-chain clustering.Potential moderators were retained only when likelihood-ratio tests showed improved model fit at p < 0.05.Source: §B.1 · p. 14
B.2 Global
- Global diversity compared sets of ideas by condition and item, using 50 Monte Carlo samples of 50 ideas and paired permutation tests.The procedure also calculated Cliff’s Delta as a non-parametric effect size.Source: §B.2 · pp. 14–15
B.3 Evolution
- For diversity over time, ideas were pooled by condition, item, and trial; mixed models tested whether diversity slopes differed by condition.Median pairwise distance was the main measure, with mean pairwise distance and mean distance from the centroid used as robustness checks.Source: §B.3 · p. 15
C.1 Factor 1: LLM Exposure
- The exposure analysis frames its expectation around how creative ChatGPT answers are relative to human ideas.The passage introduces this as an intuition, not a reported test result.Source: §C.1.1 · p. 15
C.1.1 Effects on Creativity.
The section reviews why exposure to AI ideas might affect creativity and diversity, and why disclosure effects could vary by task difficulty or participant attitudes. It distinguishes passive exposure from prior work on active AI co-creation.
C.1.1 Effects on Creativity.
- ChatGPT ideas in preliminary testing scored as more creative than human ideas, while prior work leaves unclear whether merely seeing AI ideas increases human creativity.The passage also notes that GPT-4 scored among the top human percentiles on a similar verbal creativity task, based on human judgments.Source: §C.1.1 · p. 15
- Prior AI co-creation studies report mixed diversity effects, but mostly test active engagement rather than passive exposure to off-the-shelf LLM ideas.The effect of passive exposure on human idea diversity therefore remains unclear in the cited discussion.Source: §C.1.2 · p. 15
C.2 Factor 2: LLM Disclosure
- Disclosure may affect creativity differently in passive exposure than in chatbot collaboration, and the authors propose that self-perceived creativity could shape responses to AI’s presence.The prior chatbot study attributed higher creativity after disclosure to the novelty of brainstorming with a chatbot; the authors note their study measures exposure, not direct engagement.Source: §C.2.1 · pp. 15–16
C.2.2 Effects on Diversity.
- Disclosure could increase AI-idea adoption through automation bias or reduce it through aversion, with prompt difficulty and participant attitudes proposed as moderators.Prompt difficulty is operationalized as the inverse rank of average creativity in the control condition: lower-scoring items are treated as harder.Source: §C.2.2 · p. 16
D Experiment Screenshots
The supplementary material shows an example disclosed, low-exposure trial and the feedback participants received. It also describes the AUT items and pre-treatment question choices.
D Experiment Screenshots
- The example trial is the low-exposure, disclosed condition, with two AI ideas labeled as AI; participants received feedback after each trial.Source: Figure 12 · p. 16, Figure 13 · p. 16
SI 1 AUT Items
- Table 4 lists AUT items by dataset frequency and classifier accuracy, while Table 5 summarizes the prompt experiment and notes that ChatGPT sometimes returned fewer ideas than requested.Classifier accuracy is defined as correlation with human creativity ratings; overall accuracy was r = 0.81 and the five-item subset reached r = 0.90.Source: Table 4 · p. 17, Table 5 · p. 17
SI 2 Pre-Treatment Questions
- Pre-treatment questions used Pew’s AI-attitude wording and ordering, and gender options followed Spiel et al. guidance with single selection; age, gender, and country were optional.Source: SI 2 Pre-Treatment Questions · p. 17
SI 3 Exclusion Criteria for Analysis
The supplementary sections describe exclusions, compare classifier-scored originality for AI and human ideas, and explain participant feedback. They also note that unavailable feedback metrics were sometimes replaced with random values.
SI 3 Exclusion Criteria for Analysis
- The analysis removed 36 confirmed profane ideas, leaving 3,414 of 3,452 responses; exclusions were not significantly associated with condition on the reported tests.Two responses shown in error were also removed, and two implausible ages were treated as missing for participant summaries.Source: SI 3 Exclusion Criteria for Analysis · pp. 17–18
SI 4 Human vs AI Ideas
- Across five items, 1,500 ideas from the modified Stevenson-prompt experiment were compared with 1,500 randomly sampled Organisciak Dataset ideas; ChatGPT ideas had higher originality (β = 0.62, t(2994) = 22.49, 95% CI = [0.56, 0.67]).Originality scores were model-predicted and ranged from 1 to 5.Source: SI 4 Human vs AI Ideas · p. 19
SI 5 Participant Feedback
- Participants received feedback on their classifier-scored creativity relative to human and AI samples, response uniqueness, and rankings; a figure summarizes self-perceived creativity by interest and AI sentiment.Uniqueness used cosine distance between word2vec embeddings, while ranking accuracy used rank-order correlation with the classifier.Source: SI 5 Participant Feedback · p. 19, Figure 14 · p. 20
- When either feedback metric could not be calculated, the study returned an arbitrary random number.Source: SI 5 Participant Feedback · p. 19
SI 6 Sample Characteristics
The sample profile notes that English proficiency was not assessed, although response sources and short response length suggest it was unlikely to confound results. Table 9 presents the distribution of AI Feeling.
SI 6 Sample Characteristics
- Table 9 presents the distribution of AI Feeling.Source: Table 9 · p. 20
- 77.85% of responses came from the United States, Canada, Germany, the United Kingdom, and Australia; English proficiency was not assessed, but median response length was six words.The authors considered English proficiency an unlikely confounder given the countries represented and the short responses.Source: SI 6 Sample Characteristics · p. 20
SI 7 Model Selection
The supplement describes model-selection checks, diversity and creativity analyses, and AI-adoption measures. It reports no condition main effect for local idea diversity, while the tested moderation evidence was mixed.
SI 7 Model Selection
- Interactions entered selected models only when likelihood-ratio tests improved fit at p<0.05; adding IsSocialMedia and its treatment interaction did not improve fit (p>0.39 for all models).Selected models already controlled for participant source through Interest Group.Source: Table 10 · p. 21, SI 7 Model Selection · p. 21
SI 8 Idea Diversity
- Global diversity used 50 Monte Carlo runs of 50-idea sets, measuring pairwise SBERT distances and centroid distance after trial six to avoid seed confounding.The measures included mean and median pairwise distances and mean distance from the set centroid.Source: SI 8 Idea Diversity · p. 21
Local.
- Local diversity had no condition main effect (F(4, 19.95)=0.09, p=0.98); moderation in High Exposure, Undisclosed was not significant after adjustment (p=0.06).The top-versus-bottom decile contrast was Δ=2.62, d=0.34 before adjustment; its unadjusted p-value was 0.01.Source: Local · p. 23, Table 16 · p. 24
- Evolution models pooled ideas by item, condition, and trial, measured three distance metrics, controlled set size, and excluded trials one through six to remove seed influence.The model’s reference level was the control condition.Source: Local · p. 25, Table 17 · p. 25
SI 9 Creativity
- Table 18 reports creativity-predictor coefficients from a model with participant and response-chain random intercepts nested within item-condition combinations.Source: Table 18 · p. 26
SI 10 AI Adoption
- AI adoption was maximum cosine similarity between a response and AI examples; authors did not interpret condition main effects because high exposure supplied more AI examples.They focused instead on subgroup differences, since more examples could raise maximum similarity by chance.Source: SI 10 AI Adoption · p. 27
SI 11 Implementation Complications
The online experiment encountered high traffic and moderation challenges. Capacity interruptions did not affect collected responses, while moderation sometimes allowed profane responses to circulate and produced false positives.
SI 11 Implementation Complications
- High concurrent traffic required temporarily pausing the experiment, adding resources, or implementing changes, but the capacity issues did not affect collected responses.Source: Server Capacity · p. 29
Server Capacity.
- Moderation began after a Reddit influx; some profane ideas had already circulated, 10 of 46 flagged ideas were false positives, and condition was unrelated to flag or exclusion counts.Source: Content Moderation · p. 29
Content Moderation.
- The transparent bag-of-words filter reflected Heroku CPU limits that ruled out pretrained classifiers and API rate limits that risked slowing the experiment.Source: Content Moderation · p. 29
- Response chains averaged 19.73 trials (SD=1.45; median 20; range 14–24), with no detected length differences by item, condition, or their interaction.Source: Small Deviations From 20 Trials Per… · p. 29