Thousands of AI Authors on the Future of AI

• Decision-makers across the private sector, academia, and government are addressing AI’s opportunities and threats.Source: §1 · p. 2
• Forecasting AI’s progress and impact is difficult, and there is no apparent consensus among AI experts or established method for making these judgments well.The introduction describes combining noisy approaches, from trend extrapolation and agent analysis to economic models and expert judgments.Source: §1 · p. 2
• AI researchers’ familiarity with the technology and its past progress can inform forecasts, but they may lack forecasting skills and expertise in non-technical factors.The paper cautions that their predictions are one piece of evidence, not a reliable guide to objective truth.Source: §1 · p. 2
• The fall 2023 survey collected views from 2,778 researchers published in the prior year at six top AI venues on AI progress and its social consequences.The authors describe it as the largest survey of AI researchers to date.Source: §1 · p. 2

• The 2023 ESPAI survey is the third in a series and included around four times as many participants as the 2022 survey by expanding from two publication venues to six.It added questions about future AI systems and potential risks.Source: §2 · p. 2
• Questions generally elicited Likert-scale responses, probability estimates, or estimates of a future year.Source: §2 · p. 2
• To assess framing effects, randomized respondent subsets received either fixed-years or fixed-probabilities questions, with each participant using one framing throughout.Fixed-years questions ask the probability of a milestone by a given year; fixed-probabilities questions ask when it will be feasible at a given probability.Source: §2 · p. 2
• Some participants received questions on only one of several topics, with subset sizes varying by question importance and the value of a larger sample.Most questions were not assigned to all 2,778 participants.Source: §2 · p. 2
• For milestone forecasts, respondents supplied three year-probability pairs that were used to fit a gamma cumulative distribution function.Source: §2 · p. 2
• Figure 1 displays 50%-probability milestone years and 25%-to-75% intervals for 39 tasks, four occupations, and two measures of human-level performance.The intervals represent participants’ aggregate uncertainty, not estimation uncertainty.Source: Figure 1 · p. 3

3.1 How soon will 39 tasks be feasible for AI?

• All but four of 39 tasks were forecast to have at least a 50% chance of feasibility within ten years, while six named tasks were expected to take 12–27 years.Examples of near-term milestones included coding a payment-processing site and creating songs indistinguishable from hit artists’ work.Source: §3.1 · p. 4

3.1.1 Comparison with 2022

• 21 of 32 shared task forecasts moved earlier and 11 moved later from 2022 to 2023; the median expected-feasibility year shifted 1.0 years earlier on average.The shift had SD = 2.0 and SE = 0.18.Source: §3.1.1 · p. 4

3.2 How soon will human-level performance on all tasks or occupations be feasible?

• The aggregate 2023 forecast gave HLMI a 50% chance by 2047 and a 10% chance by 2027, while FAOL reached a 50% chance only by 2116.The FAOL 50% forecast was down 48 years from 2164 in 2022; HLMI and FAOL 50% forecasts differed by more than 60 years across the 2016, 2022, and 2023 surveys, and the authors did not know why.Source: §3.2.1 · p. 4, §3.2.2 · p. 7, §3.2.3 · p. 7

3.2.3 Differences between HLMI and FAOL

• Respondents educated in Asia anticipated HLMI 11 years earlier than respondents from other combined regions; 44% said they disagreed with the typical researcher’s timing view “Not much,” 46% “A moderate amount,” and 10% “A lot.”For HLMI versus FAOL, the authors note that question wording and framing may help explain the forecast gap, but its cause is unknown.Source: §3.2.3 · p. 7, §3.2.4 · p. 8, §3.2.5 · p. 8

3.3 Framing effect of fixed-years vs fixed-probabilities

• The fixed-years framing yielded a 34-year HLMI 50% horizon, twice the 17 years in the fixed-probability framing, though both were shorter than the 2022 combined forecast of 37 years.The survey combined the two framings with equal weight because neither is known to be more accurate.Source: §3.3 · p. 8

3.5 What causes AI progress?

• Respondents’ estimates varied widely across five AI-progress inputs, and the relatively even distribution of predictions cut against a narrative that cheap computing was the dominant driver.The authors also noted unusually many 0% and 100% answers, suggesting possible misunderstanding of the question.Source: §3.5 · p. 9

3.6 Will there be an intelligence explosion?

• Across three questions, the median respondent did not overall expect rapid acceleration from an intelligence explosion, but gave it substantial credence.The questions concerned acceleration after HLMI and AI outperforming humans across professions.Source: §3.6 · p. 10

4.1 How concerning are 11 future AI-related scenarios?

• The scenarios most often rated substantial or extreme concerns were false-information spread (86%), public-opinion manipulation (79%), dangerous-tool access and authoritarian control (73% each), and AI-worsened inequality (71%).The ratings do not distinguish whether concern reflected a scenario’s disastrousness, likelihood, or both.Source: §4.1 · p. 12

4.2 How good or bad for humans will High-Level Machine Intelligence be?

• 68.3% thought good outcomes from HLMI more likely than bad, yet 64% assigned non-zero probabilities to both extremely good and extremely bad outcomes.Among net optimists, 48.4% gave at least 5% credence to extremely bad outcomes; 58.6% of net pessimists gave at least 5% to extremely good outcomes.Source: §4.2 · p. 12, Figure 10 · p. 13

4.3 How likely is AI to cause human extinction?

• 41.2%–51.4% estimated a greater than 10% chance of human extinction or severe disempowerment, while the median estimates across the questions were 5% or 10%.The estimates were mostly consistent with responses to the general question about the long-run impact of HLMI.Source: §4.3 · p. 14, Figure 13 · p. 15

4.4 Views on others’ concerns about AI

• Among 671 respondents asked why people are concerned about future AI risks, 54.7% said concerns were due to misunderstandings of AI research to a large extent or almost entirely.The responses were 44% “to a large extent” and 10.7% “almost entirely.”Source: §4.4 · p. 16

4.5 What rate of AI progress would produce the most optimism?

• Respondents disagreed about whether faster or slower global AI progress over the next five years would best support humanity’s future.Large departures from the current rate were less popular.Source: §4.5 · p. 16, Table 3 · p. 16

4.6 How much should AI safety research be prioritized?

• 70% of respondents thought AI safety research should be prioritized more than it currently is.The survey defined AI safety research as research primarily aimed at minimizing potential AI-system risks rather than improving capabilities.Source: §4.6 · p. 16, Figure 14 · p. 17

4.7 How worthy and hard is the alignment problem?

• Most respondents called the alignment problem very important (41%) or among the field’s most important problems (13%), and harder (36%) or much harder (21%) than other AI problems.They did not generally consider it more valuable to work on today than other AI problems.Source: §4.7 · p. 17, Figure 15 · p. 18

5.1 Summary of results

• 21 of 32 shorter-term milestones were expected earlier than in 2022, while human-level performance was expected 13 to 48 years sooner on average.The difference depended on question phrasing.Source: §5.1 · p. 19
• 38% to 51% of respondents assigned at least a 10% chance to advanced AI causing extinction-level outcomes, across four questions.A majority assigned at least a 5% chance to these outcomes.Source: §5.1 · p. 19
• More than half of respondents considered over half of eleven potential AI scenarios substantially or extremely concerning.The scenarios included misinformation, authoritarian control, and inequality.Source: §5.1 · p. 19

5.2.1 Forecasting is hard, even for experts

• Wide disagreement and substantial framing effects limit confidence that aggregate expert forecasts accurately reflect future outcomes.The authors note that AI researchers are not known to have unusual general forecasting skill, and recommend treating their forecasts as one part of broader evidence.Source: §5.2.1 · p. 19

5.2.2 Participation

• The 15% response rate leaves possible participation bias, although the authors found no evidence of bias that would affect results to a large extent.The analysis included checks at survey and question level.Source: §5.2.2 · pp. 19–20

5.2.3 Change in sample from 2022

• The expanded 2023 sample may make year-to-year comparisons less meaningful, though NeurIPS/ICML-only respondents had similar views to the full sample.The 2023 survey also included researchers from ICLR, AAAI, IJCAI, and JMLR.Source: §5.2.3 · p. 20

6 Methods

• In October 2023, the team surveyed recent authors at six top-tier AI venues about AI progress, impacts, and safety, with ethics approval and preregistered analysis.Source: §6 · p. 20

6.1 Survey questions

• Most questions were drawn from earlier surveys, while new questions were tested in interviews and questions were assigned to participant subsets to keep the survey brief.The survey flow is summarized in Figure 16.Source: §6.1 · p. 20

6.2 Randomization

• For milestone forecasts, respondents were randomly assigned one of two framings: years until 10%, 50%, and 90% feasibility, or probabilities at specified future-year intervals.Participants received the same framing across the relevant questions.Source: §6.2 · p. 20

6.3 Recruitment

• Recruitment covered 2022 authors at six AI venues, expanding beyond the two venues used in 2022 and contacting more than three times as many recipients.The team collected about 21,800 names and found email addresses for 20,066.Source: §6.3 · pp. 20, 22

6.4 Fielding

• The online survey received 2,778 responses, a 15% response rate; after a paid pilot drew more replies, rewards were offered to all participants.The survey was delivered by private link and reminders were sent during fielding.Source: §6.4 · p. 22

6.5 Data preparation

• Data preparation removed invalid or uninterpretable numerical responses and corrected some year entries; answers expressing “never” were assigned 100,000,000 years.The analysis was reported as insensitive to the upper-bound value.Source: §6.5 · p. 22

• Open Philanthropy funded the research project.Source: §8 · p. 24

• Undergraduate study in Asia was associated with shorter HLMI forecasts, while time in field and citation count had no significant effect.Source: Figure 17 · p. 28

A.2.1 Amount of thinking about the issue

• The survey asked all respondents how much they had thought about the social impacts of smarter-than-human machines.Source: §A.2.1 · p. 28
• Respondents who had thought more about social impacts were less likely to assign over 10% credence to extremely optimistic outcomes and more likely to assign over 10% to extremely negative outcomes.The authors caution that this association may reflect either improved prediction or selection: concern may lead people to think more about the topic.Source: §A.2.1 · p. 28, Table 4 · p. 29

A.2.2 Academia vs industry, undergraduate region, and amount of thought

• Table 5 reports median outcome probabilities by demographic group, while the accompanying text identifies academia or industry, undergraduate region, and amount of thought as the compared factors.Source: §A.2.2 · p. 29, Table 5 · p. 29

A.3 How likely is AI to cause human extinction?

• 5% was the median response for both academia and industry; the median was 10% among respondents who graduated in Asia and 5% among those who graduated in North America or Europe.Those who had thought a lot or a great deal about the question had a 9% median, versus 5% among those who had thought little or very little.Source: §A.3 · p. 29
• Similar answers across question variants, earlier surveys, and demographic groups provide robust evidence that most AI researchers see a nontrivial risk of AI-related extinction or similar catastrophes.Source: §A.3 · p. 29

B.1 How soon will human-level performance on all tasks or occupations be feasible?

• Participants given fixed-years questions forecast later milestone dates than those given fixed-probabilities questions.Source: Figure 18 · p. 30

B.2 How soon will Full Automation of Labor be feasible?

• The aggregate-forecast table reports that 2023 estimates for 50% feasibility were earlier than in 2016, while 10% estimates were later; both HLMI and FAOL forecasts were earlier in 2023 than in 2022.Source: Table 6 · p. 30

B.3 Will there be an intelligence explosion?

• For explosive global technological improvement, median predictions were 20% two years post-HLMI and 80% after 30 years; for AI vastly better than humans in all tasks, they were 10% and 60%, respectively.Source: Figure 19 · p. 31

B.4 How much should AI safety research be prioritized?

• The survey asked how much AI safety research should be prioritized using two framings, one of which included biased AI as an example.Source: Figure 20 · p. 32

C Full Descriptions of Milestones

• Milestones were operationalized with specific task conditions, including building a payment-processing website from specifications and downloading and fine-tuning an open-source language model without further human input.Other descriptions include one-shot recognition of a new object and translation of a difficult, newly discovered language using a single bilingual document.Source: C Full Descriptions of Milestones · pp. 33–34

Participation bias

• The authors found no evidence suggesting strong participation bias in the survey.Source: D Additional Information on… · p. 37
• To encourage broad participation and reduce opinion-based entry, the survey offered a reward, sent reminders, and described its topic broadly as the future of AI progress.The invitations also linked to earlier surveys and identified authors and affiliations, so some topic information remained available to recipients.Source: D Additional Information on… · p. 37

Question-level response bias

• Opinion questions were answered by an average of 96% of those who saw them, which the authors say leaves very little scope for question-level participation bias.Source: D Additional Information on… · p. 37

Participation bias

• Industry researchers were 61% as likely to respond as the base rate, while women were 66%, highly cited researchers 69%, and people in Asia 84%.The authors also report minimal influence of most demographic factors on beliefs, with female respondents generally expecting less extreme outcomes and Asian-educated respondents anticipating HLMI 11 years earlier than the comparison group.Source: D Additional Information on… · p. 37
• Only 7.6% of respondents said they had given “a great deal” of thought to when HLMI would be developed, and 10.3% said this about social impacts.Source: D Additional Information on… · p. 37

F Yuen’s Test (Bootstrap Version) Results

• The reported fields include the variable, P-value, and Ty.Source: F Yuen’s Test (Bootstrap Version)… · p. 38
• The table headings include a trimmed mean difference.Source: F Yuen’s Test (Bootstrap Version)… · p. 38
• The reported confidence interval has lower and upper bounds.Source: F Yuen’s Test (Bootstrap Version)… · p. 38