Source-linked AI summary
Long-Term Trends in the Public Perception of Artificial Intelligence
Ethan Fast, Eric Horvitz
TL;DR
The paper addresses how public perceptions of AI in news have changed over time and develops indicators to measure engagement, sentiment, hopes, and concerns. Using crowdsourced annotations and natural language processing on 30 years of New York Times articles, it finds sharply increased discussion since 2009, generally more optimistic coverage, and rising concerns about loss of control, ethics, and work alongside growing healthcare and education hopes.
Problem
The paper asks which optimistic and pessimistic visions of AI are common in public discourse and how those visions have changed over time.
Method
The study combines crowdsourcing and natural language processing to characterize AI impressions in New York Times articles across 30 years.
Results
Discussion of AI increased sharply since 2009 and remained consistently more optimistic than pessimistic, while specific concerns and hopes changed over time.
Takeaways & Limitations
The indicators provide a long-term way to track engagement, sentiment, and specific public hopes and concerns about AI in news.
Takeaways & Limitations
Earlier than 1986, the available data are too sparse to extend the main analyses to 1956.
Abstract
from arXiv · showhide
Analyses of text corpora over time can reveal trends in beliefs, interest, and sentiment about a topic. We focus on views expressed about artificial intelligence (AI) in the New York Times over a 30-year period. General interest, awareness, and discussion about AI has waxed and waned since the field was founded in 1956. We present a set of measures that captures levels of engagement, measures of pessimism and optimism, the prevalence of specific hopes and concerns, and topics that are linked to discussions about AI over decades. We find that discussion of AI has increased sharply since 2009, and that these discussions have been consistently more optimistic than pessimistic. However, when we examine specific concerns, we find that worries of loss of control of AI, ethical concerns for AI, and the negative impact of AI on work have grown in recent years. We also find that hopes for AI in healthcare and education have increased over time.
Introduction
The paper asks how public visions of AI in the New York Times have changed over time and develops indicators to characterize those impressions across 30 years. It finds that AI discussion grew sharply, remained generally more optimistic than pessimistic, and shifted in the prevalence of specific hopes and concerns.
- Research questions: The paper compares competing optimistic and pessimistic visions of AI and asks how common each vision is and how they evolved over time.The examples include AI creating opportunities and assisting surgeons, alongside displaced workers and dystopian surveillance.
- Approach: The study characterizes impressions expressed about AI in New York Times news over 30 years.Its indicators capture engagement, general sentiment, and specific hopes and concerns.
- Scope: The paper treats New York Times coverage as a proxy for public opinion and engagement because few corpora extend comparably far into the past.The indicators are intended for ongoing tracking of present-day articles.
- Approach: The analysis combines crowdsourcing and natural language processing to annotate AI mentions for relevance, optimism or pessimism, and specific hopes and concerns.The annotations support indicators and a classifier for automatically extracting impressions from new articles.
- Findings: AI discussion increased sharply since 2009, while coverage remained generally more optimistic than pessimistic.The study also reports changing prevalence for ideas such as healthcare, loss of control, lack of progress, and positive effects on work.
Indicators of Impressions about AI
The paper defines indicators for engagement, attitude, and recurring hopes and concerns about AI. These categories cover social, economic, institutional, and existential consequences associated with AI discussions.
- General measures: The indicators measure AI engagement, general sentiment, and hopes and concerns about AI’s future.Engagement serves as a proxy for how much AI is discussed in the news.
- General measures: The optimism-versus-pessimism measure captures whether discussion implies a hopeful or negative attitude toward AI’s future.Examples include self-driving benefits and surveillance dangers.
- Hopes: Positive-impact indicators include AI improving work, education, transportation, healthcare, decision making, entertainment, singularity outcomes, and human-AI merging.Examples include tutoring, diagnosis, self-driving cars, personalized medicine, and robotic limbs.
- Hopes and concerns: The categories distinguish positive and negative forms of both singularity and human-AI merging.Positive merging includes robotic limbs, whereas negative merging includes cyborg soldiers.
- Concerns: Concern indicators include loss of control, job displacement, military applications, absent ethics, lack of progress, harmful singularity outcomes, and negative human-AI merging.These categories span safety, employment, warfare, ethics, technological progress, and human transformation.
Data: Thirty Years of News Articles
The dataset covers New York Times articles from 1986 through 2016 and extracts AI-related paragraphs for annotation. Paragraph segmentation is used to preserve local context while making annotation more manageable.
- Corpus: The analysis covers more than 3 million New York Times articles published between January 1986 and May 2016.The full corpus spans the study’s 30-year news period.
- Corpus construction: The dataset combines New York Times API metadata with full-text pages scraped from article URLs.Metadata includes article titles, sections, and current URLs.
- Annotation units: The researchers segment articles into paragraphs because paragraphs are generally self-contained enough for accurate annotation while retaining context.This avoids sentence-level failures to connect references and document-level failures to locate relevant passages.
- Filtering: Filtering for paragraphs containing “artificial intelligence,” “AI,” or “robot” produced more than 8000 AI-related paragraphs.“Robot” was included to increase coverage, with relevance filtered later.
Crowdsourcing to Annotate Indicators
The study uses crowdsourcing to label AI-related paragraphs for attitude and specific hopes or concerns, then evaluates voting thresholds against ground truth. It chooses a one-vote threshold to preserve recall for sparse themes.
- Annotation process: More than 8000 AI-related paragraphs were annotated through crowdsourcing with attitude ratings, binary hope-or-concern labels, and relevance checks.The attitude task used a 5-point scale from pessimistic to optimistic.
- Annotation quality: Three independent workers annotated each task, with 97% agreement on AI relevance and 70% agreement when distinguishing optimism from pessimism.Relevance and attitude ratings were averaged across workers.
- Voting scheme: The voting-threshold decision balances false positives against false negatives in hope and concern labels.More votes reduce false positives but can increase false negatives.
- Validation: The voting schemes were evaluated using a ground-truth dataset for paragraphs associated with military applications of AI.This enabled comparison of precision and a proxy for recall.
- Voting scheme: Requiring two or more votes yielded 100% precision and 59% recall, whereas one vote yielded 80% precision and 100% recall.The authors selected one vote because many themes were sparsely represented, including 231 mentions of loss of control.
Trends in the Public Perception of AI
Across 30 years of New York Times coverage, AI discussion rose sharply after 2009 and was generally more optimistic than pessimistic, while associated hopes and concerns shifted over time.
- Engagement: AI discussion rose dramatically beginning in late 2009, after reaching its lowest level in 1995 during the AI winter.The cause of the post-2009 rise is unclear, though it followed renewed use of deep learning and coverage of the Asilomar meeting.
- Sentiment: Optimistic AI coverage was consistently more common than pessimistic coverage, at roughly 2–3 times the level over 30 years.Both optimistic and pessimistic coverage increased sharply after 2009.
- Associated ideas: AI-associated keywords shifted across periods from space weapons and chess to search engines and driverless vehicles.Some associations persisted, including robots across periods, while science fiction peaked in the early 1990s.
- Hopes and concerns: Concern about losing control of AI became more than three times as common as a percentage of AI articles as in the 1980s.Ethical concerns also increased, while concern about insufficient progress peaked in 1988 and then generally declined.
- Hopes and concerns: Positive views of AI’s impact on work declined while negative views increased sharply, whereas hopes for AI in education and healthcare grew over time.Positive views of merging with AI and the role of AI in fiction also increased.
News Articles from 1956 to 1986
Earlier New York Times abstracts show changing AI-related themes from military and space applications toward concerns about jobs and hopes for healthcare, but the data are too sparse for the later analyses.
- Data scope: Only 40 abstracts referenced AI and 247 mentioned robots between 1956 and 1986, limiting analysis of this earlier period.The first AI reference appeared in 1977.
- Data scope: Because the earlier data were too sparse, the study used manual topic annotations rather than extending its later analyses to 1956.The New York Times provides abstracts over a longer period than full-text articles.
- Historical themes: Robots were associated with military applications and missiles in the 1950s, while space dominated associations in the 1960s and 1970s.The examples include guided missiles and a robot launched into orbit.
- Historical themes: Interest in AI increased considerably in the early 1980s, alongside the first mentions of job displacement, a robot-caused death, and AI in healthcare.The healthcare example involved a robot preparing meals and performing chores for quadriplegics.
External validity
The study tested whether New York Times trends generalized beyond news by applying a classifier to five years of Reddit posts, finding a similar shift in concern about losing control of AI.
- Replication: The study replicated a primary New York Times finding using five years of Reddit posts from a diverse online community.The replication addressed whether newspaper trends generalized to the broader public.
- Method: A classifier trained on annotated New York Times paragraphs predicted loss-of-control mentions in Reddit AI-related posts.It used logistic regression with TF-IDF features and a positive-class threshold of 0.9.
- Method: Validation on 100 annotated Reddit posts yielded precision of 0.8 for the classifier.The classifier was then applied to every Reddit post mentioning artificial intelligence.
- Findings: The Reddit trend mirrored the New York Times trend, providing some evidence for external validity.The evidence supports similarity in shifts among Reddit users and newspaper coverage over the same period.
Related Work
Prior work uses long-term text corpora, news, social media, and crowdsourcing to study cultural change and public attitudes. This paper extends those approaches to quantify public concerns about AI and frame future discussions.
- This work is presented as the first to quantify AI concerns through direct analysis of public-facing text.
- The resulting indicators are intended to support future discussions, including those associated with the One Hundred Year Study of Artificial Intelligence.
- Public opinion polls support greater optimism than pessimism about AI alongside increasing existential fear and worry about jobs.
- Earlier studies mined books, music, news, and social media to measure cultural, artistic, and public-attitude changes over time.
- Crowdsourcing can identify themes that are difficult to analyze with fully automated approaches and bootstrap classifiers for larger corpora.
Conclusion
The paper introduces indicators for tracking AI engagement, sentiment, hopes, and concerns across 30 years of New York Times articles. Discussion increased sharply after 2009 and remained more optimistic than pessimistic, while several specific concerns grew recently.
- Discussion of AI increased sharply since 2009 and has consistently been more optimistic than pessimistic.
- Fear of losing control of AI has increased in recent years despite the overall optimistic balance of discussion.