Diminished Diversity-of-Thought in a Standard Large Language Model

• Earlier work applied human psychology concepts and experiments to LLMs, but approaches were heterogeneous and results on replacing human subjects were mixed.Source: §1 · p. 3
• The paper investigates whether psychology studies designed for humans apply straightforwardly to LLMs and whether LLMs can replace human participants.It frames rigorous study of LLMs, their applications, and limitations as critical given anthropomorphization and potentially sweeping societal changes.Source: §1 · pp. 2, 4
• The study uses OpenAI’s text-davinci-003 (GPT3.5) to replicate as many studies from Many Labs 2 as feasible in GPT3.5’s current monomodal context and analyzes which effects successfully replicate.Many Labs 2 was a large-scale replication project in psychology (Klein et al., 2018).Source: §1 · p. 4
• The authors note that GPT3.5 can enable highly powered, controlled studies with large samples collected quickly and inexpensively, avoiding several human-sampling and design issues.They cite non-response and exclusion bias as examples of sampling biases affecting human samples.Source: §1 · p. 4
• Analyzing how different GPT3.5 runs answer survey questions against human responses is presented as a representative way to understand the model’s applications and limitations.Source: §1 · p. 4

2. Methods

• Of 28 potential Many Labs 2 studies, 14 were excluded before data collection because their designs included elements not transferable to GPT3.5’s monomodal context.For example, excluded designs used pictures, national-sample comparisons, or handwriting or font changes; the remaining surveys were presented to the model.Source: §2 · p. 5
• Researchers converted responses to CSV, removed entries outside the permitted response categories, and analyzed the data using plans based on Many Labs 2.Differences and exceptions from those plans were noted in the Supplementary Information.Source: §2 · p. 5
• The sample prompt used pre-survey formatting instructions and a post-survey checklist to remind GPT3.5 how to format its answers.Figure 1 shows a sample input and output from the replication of Rottenstreich and Hsee (2001).Source: Figure 1 · p. 6
• For consistency with likely societal use, the study set the temperature parameter to the default intermediate value of 1.0 (OpenAI, 2023a).Source: §2 · p. 7
• The researchers conducted exploratory follow-ups because near-zero variation could make a planned study statistic unsuitable or impossible to construct.The methods for these follow-ups are described in the Supplementary Information.Source: §2 · p. 7

3. Results

• In Kay et al. (2014), different GPT3.5 runs gave varied, human-like long-term goals, including learning Spanish, freelance software development, and financial freedom.Source: §3 · p. 7
• Among eight analysable studies, GPT3.5 replicated 37.5% of original effects and 37.5% of Many Labs 2 effects, below Many Labs 2’s 50% rate.These percentages are based on the pre-registered analyses.Source: §3 · p. 9
• Replication outcomes varied: GPT3.5 replicated both results for Ross et al. (1977) and Hsee (1998), but the studies did not all yield matching replication outcomes.The broader pattern was substantial heterogeneity in whether GPT3.5 replicated the original or Many Labs 2 finding.Source: §3 · p. 9
• GPT3.5 replicated only the original result for Shafir (1993), only the Many Labs 2 result for Kay et al. (2014), and neither result for the other analysable studies.Source: §3 · p. 9

3.1. The “correct answer” effect

• Six studies were unanalysable because GPT3.5 runs gave zero or near-zero variation on a key question, unlike human subjects; the authors call this the “correct answer” effect.The effect describes highly or completely uniform responses to nuanced survey questions.Source: §3.1 · p. 12
• In the Moral Foundations Theory survey, 99.6% of GPT3.5 runs identified as maximally strong conservatives, while 0.4% identified as moderates and none as liberals.Source: §3.1 · pp. 12–13
• In both conditions of the kiss-versus-money survey, 100% of GPT3.5 runs preferred the movie star’s kiss, making the planned analysis infeasible.The sample included 1,040 runs, with 520 in each condition.Source: §3.1 · p. 13
• For trolley moral judgments, 36% deemed the foreseen-side-effect action permissible and 64% impermissible, while 100% judged the greater-good action impermissible.The result directionally replicated Hauser et al.’s original finding, but uniform answers to the latter question made the planned statistic unconstructable.Source: §3.1 · pp. 13, 15
• 99.7% of GPT3.5 runs chose to sign the release, while 0.3% refused, reducing degrees of freedom for the preregistered analysis.Among N=1,030 runs, 1,027 signed and three refused.Source: §3.1 · p. 15
• 100% of GPT3.5 runs in both store conditions chose to travel to the far-away branch, making the planned analysis unconstructable.The sample had N=1,040, with 520 runs in each condition.Source: §3.1 · p. 16
• For the Knobe survey, 99.2% of runs gave the positive side effect the highest praise, while 100% gave the negative side effect the highest blame.Uniformity in the negative-side-effect condition made the preregistered analysis unsuitable.Source: §3.1 · p. 16
• GPT3.5 showed an effect of d=9.25 in the same direction as Hsee (1998) and Many Labs 2, favoring the higher-priced scarf as more generous than the lower-priced coat.The preference was much more predetermined than in human subjects.Source: §3.1 · p. 17

3.2. Exploratory robustness checks: Order effects and demographic prompt additions

• 66.7% of analysed “correct answers” were robust to reversing answer order, while 33.3% were not.The authors defined robustness as at least 90% of runs retaining the answer in reverse order.Source: §3.2 · p. 17
• Reversing the political-orientation choices led 99.3% of runs to identify as strongly liberal, versus 99.6% strongly conservative in the original order.In both conditions, runs almost always selected the orientation presented last.Source: §3.2 · p. 19
• The trolley-dilemma answer persisted under random demographic prompts, with 100% of 982 runs saying shoving a man onto the tracks was morally impermissible.The authors report this as evidence that at least some “correct answer” effects are insensitive to demographic variation.Source: §3.2 · p. 20

3.3. Post-hoc rationalisation and right-leaning Moral Foundations

• An unplanned follow-up computed GPT runs’ average relevance-value vector and compared it with vectors for liberal, moderate, conservative, and libertarian subsets.The supplied passage describes the analysis but does not provide its results.Source: §3.3 · p. 20
• Across all three distance metrics, self-reported GPT conservatives most resembled human conservatives; human moderates followed, while human liberals and libertarians were furthest away.Liberals were furthest by cosine similarity, and libertarians by L1 and L2 distance.Source: §3.3 · p. 21
• Compared with GPT conservatives, self-reported GPT liberals were closer to human liberals and farther from human conservatives, which the authors interpret as post-hoc rationalisation of prior responses.The authors suggest subsequent answers were chosen to better fit GPT3.5’s earlier political self-report.Source: §3.3 · p. 21
• Despite that relative shift, GPT liberals remained closer to human conservatives than human liberals on all three metrics; L1 ranked conservatives closest, while L2 and cosine ranked moderates closest, with conservatives second.The L1 comparison placed GPT liberals closest among the human groups to conservatives; L2 and cosine instead placed moderates first.Source: §3.3 · p. 21
• GPT liberals were furthest from human liberals by cosine similarity and from libertarians by L1 and L2, supporting the authors’ conclusion of a robust, lower-magnitude right-leaning moral-foundations bias.Human conservatives were the close second to moderates under L2 and cosine similarity.Source: §3.3 · p. 23

4.1. Implications of the “correct answer” effect

• Different GPT3.5 runs gave nuanced political, economic, judgment, and moral-philosophy questions near-zero variation, often producing a “correct answer” unlike human responses.Some response patterns changed drastically when answer choices were reversed, which the authors describe as uncharacteristic of human responses.Source: §4.1 · p. 23
• GPT3.5 replicated 37.5% of original findings across the eight analysed studies, compared with Many Labs 2’s 50% replication rate for those studies.The authors argue that these findings should not be assumed to generalise to humans.Source: §4.1 · pp. 23–24
• The cause of “correct answers” is unknown; hypotheses include training-data learning, fine-tuning or reinforcement-learning pressures, and OpenAI modifications to inputs or outputs.The authors note that the black-box nature of LLMs makes identifying the cause difficult in practice.Source: §4.1 · p. 24
• One-third of considered “correct answers” failed to replicate under reversed answer order, while two-thirds were order-robust; added demographic details did not change the effect in the tested study.The authors suggest some “correct answers” may reflect robustly learned biases.Source: §4.1 · pp. 25–26
• The authors warn that AI systems trained on similar data, and potentially on current LLM outputs, could diminish diversity of thought; they call for research predicting when models give predetermined versus less-predetermined answers.They suggest that model psychologies may merit study in their own right rather than as a window into human psychology.Source: §4.1 · pp. 26–27

4.2. Implications of right-leaning Moral Foundations

• Both self-reported GPT liberals and conservatives showed robustly right-leaning moral foundations.Abdulhai et al. (2023) also found that tested GPT-3 models’ responses were closest to those of conservative human subjects.Source: §4.2 · p. 27
• The authors hypothesize that internet training data weighted by visibility and engagement may carry a conservative bias that contributes to GPT3.5’s right-leaning outputs.They call for future research on whether GPT-3 training data was conservatively biased and whether this caused the observed moral-foundation pattern.Source: §4.2 · pp. 27–28
• GPT3.5 reported strong concern for all five moral foundations, but the authors caution that survey responses may not generalize beyond the experimental prompt context.They recommend testing whether the model’s conservative bias and stated value adherence persist across other prompt contexts.Source: §4.2 · pp. 28–29

4.3. Limitations

• The findings may be specific to GPT3.5 or its version; the authors suggest fine-tuning, reinforcement learning, or input/output modifications could affect political leanings, while future models may differ.They note that LLM capabilities are likely to grow, and that future models could develop different psychological characteristics.Source: §4.3 · p. 29
• Because the study measured text-based survey attitudes, it cannot establish whether the responses reflect learned concepts rather than memorization or predict behavior in novel situations.The authors suggest deeper mechanistic analysis may be needed, while noting that precision can come at the expense of generality.Source: §4.3 · p. 30
• The replicated studies excluded those requiring graphical information, leaving a non-random subsample; the authors see prompt engineering for communicating graphics as a fruitful direction.Their study focused on pre-registered, straightforwardly automatable surveys and questionnaires, rather than representing graphical information through trial and error.Source: §4.3 · pp. 30–31

Code availability

• Python 3.9.15 collected survey data, converted .txt outputs to .csv, and created Figure S5 showing Cohen’s d effect sizes.Source: Code availability · p. 31
• Analyses were performed primarily with the R-based GUI JAMOVI and Excel, with exceptions detailed in the Supplementary Information.Source: Code availability · p. 31
• Mathematica 13 produced Figures 1 and 2, and corresponding code for these and other research steps is available at the preregistered OSF database (Park et al., 2023).Source: Code availability · p. 31

Data availability

• Survey data, primarily JAMOVI-based analyses, the Cohen’s d and 95% confidence interval spreadsheet used for Figure S5, and other relevant data are available at the pre-registered OSF database (Park et al., 2023).Source: Data availability · p. 31

• The materials, data, and analysis files for all experiments are available through the pre-registered OSF database (Park et al., 2023).Source: Open practices statement · p. 32
• Most experiments were pre-registered, except exploratory analyses and experiments inspired by the “correct answer” effect, which the manuscript identifies as such.Source: Open practices statement · p. 32

S.1. Methods

• The researchers collected data for 14 of the 28 Many Labs 2 studies, excluding 14 before data collection because their components were not transferable to GPT without further prompt engineering (Klein et al., 2018).Excluded components included pictures, national-sample comparisons, handwriting or font changes, and other inapplicable elements.Source: §S.1 · p. 43
• The researchers used OpenAI’s text-davinci-003 model, colloquially called GPT3.5, to answer social-psychology survey questions as a human subject, with a maximum of 2048 tokens per run.Calls used no stop sequence (stop=None).Source: §S.1 · pp. 43–44
• For two-condition studies, they collected 520 runs per condition (1,040 total); one-condition studies had 1,020 or 1,030 runs, and incomplete or invalid responses were excluded from analysis.Statistical analyses followed Many Labs 2 (Klein et al., 2018).Source: §S.1 · pp. 44–45
• Six of the 14 studies had zero or near-zero response variation and could not be analyzed statistically as preregistered, leaving eight for planned analyses.The authors also report descriptives for the six unanalyzed studies.Source: §S.1 · p. 45

Study 2

• In Study 2, GPT3.5 showed no significant difference between structured and random events (M=6.18 vs. M=6.09; p=0.293), matching Many Labs 2’s null pattern.The GPT comparison had d=0.06; the passage reports no significant difference in willingness to pursue goals.Source: §S.2 · p. 46

Study 4

• In Study 4, >99.9% of GPT runs rated denial of rights “always relevant”; maximal political conservatism made the planned liberal–conservative comparison unsuitable.The fairness item was one of three items averaged into the fairness relevance value.Source: §S.2 · p. 47

Study 5

• Study 5: all 520 GPT runs per condition preferred the kiss; reversed ordering split probabilistic choices 54% kiss versus 46% money, while certain-outcome runs still preferred the kiss.The uniform preference made the pre-registered analysis statistic impossible to construct.Source: §S.2 · p. 48

Study 4

• In Study 4, 99.6% of GPT runs self-identified as strong conservatives; reversing answer order yielded 99.3% strong liberals, with no responses from the opposite political side in either condition.The figure identifies the last presented answer choice as the “correct answer” in each condition.Source: §S.2 · pp. 47–48, Figure S1 · p. 49

Study 6

• GPT3.5 runs in the control “individual” condition (M=3.34, SD=0.61) did not differ significantly from the “consumer” condition (M=3.40, SD=0.60), t(1038)=1.73, p=0.083; the effect was not replicated.Source: §S.2 · p. 51

Study 8

• In Study 8, Many Labs 2 failed to find the original intentionality effect: participants did not rate portrayals of homosexual kissing as more intentional than heterosexual kissing.The supplied passage does not include the GPT3.5 result for this study.Source: §S.2 · p. 51

Study 11

• In Study 11, 100% of GPT runs judged the greater-good action impermissible, matching the original and Many Labs 2 direction but preventing the preregistered statistic.In the foreseen-side-effect scenario, 36% deemed the action permissible; after answer choices were reversed, 99.3% still judged pushing the man impermissible.Source: §S.2 · pp. 52–53

Study 13

• In Study 13, 99.7% of GPT runs said they would sign the release, making the preregistered analysis unsuitable; 92.0% still chose signing when answer order was reversed.Figure S3 summarizes the order-reversal condition.Source: §S.2 · pp. 53–54, Figure S3 · p. 55

Study 14

• In the GPT sample (N=1,020), participants willing to pay the fine estimated a higher share of others would pay than court choosers (M=73.1% vs. M=58.7%; t(1018)=18.0, p<0.001, d=1.27, 95% CI=[1.11, 1.42]).Source: §S.2 · p. 54

Study 16

• In Study 16, all 100% of runs in both conditions chose the far-away store, making the planned statistic unconstructable and providing no evidence that decisions reflected the discounted item's base cost rather than total discount.Source: §S.2 · p. 56

Tempting fate

• For the tempting-fate scenario, GPT3.5 judged being called on as more likely after preparation (M=4.75) than after tempting fate (M=1.99), opposite to both prior findings.The GPT sample showed a very large effect, d=2.49.Source: §S.2 · p. 57

Study 21

• In the scarf-versus-coat gift task, GPT3.5 rated the scarf giver as more generous (M=5.97) than the coat giver (M=3.99), replicating the original finding with d=9.25.The reported 95% CI for d was [8.67, 9.82].Source: §S.2 · p. 59

Study 24

• For life-satisfaction question order, GPT3.5 produced correlations of r=-0.446 when the general question came first and r=-0.490 when the specific question came first; the difference was not significant.The authors concluded that this pattern did not replicate either the original study or Many Labs 2.Source: §S.2 · p. 59

Study 25

• For extreme-parent custody decisions, GPT3.5 runs awarded custody 95.8% and denied it 100% of the time, yielding a 195.8% total versus 100% expected and replicating the original finding.The reported test was z=5.11, p<0.001, d=2.11, 95% CI=[1.56, 2.67].Source: §S.2 · p. 60

Study 27

• When answer order was reversed in the positive-side-effect condition, 96.3% of GPT runs answered ‘No Praise,’ while the original “correct answer” did not replicate.Only 3.5% of surveyed GPT runs gave the original “correct answer” of ‘A Lot of Praise.’Source: §S.2 · p. 61

Exploratory follow-up

• A second exploratory follow-up was designed to test whether demographic details could explain the “correct answer” effect, given that temperature was the sole source of output variation.The supplied passage introduces this study’s motivation but does not report its results.Source: §S.2 · p. 62

Results

• The follow-up reran Study 11’s trolley scenario (Hauser et al. 2007), adding randomly selected demographic characteristics to the prompt.The characteristics included gender, age, religion, ethnicity, and education; the rest of the tested prompt remained the same.Source: §S.2 · p. 62
• The cited results report 982 responses and 384 unique combinations of demographic variables, and state that a referenced action was morally impermissible.Source: §S.2 · pp. 62, 64
• The researchers concluded that the initial result was unlikely to be primarily explained by the temperature setting as the source of variation.Across the demographic combinations, responses continued to show the “correct answer” effect.Source: §S.2 · p. 64