Source-linked AI summary
A brief history of AI: how to prevent another winter (a critical review)
Amirhosein Toosi, Andrea Bottino, Babak Saboury, Eliot Siegel, Arman Rahmim
TL;DR
AI’s rapid growth and broad applications make its history of repeated summers and winters consequential, especially given the economic stakes of another downturn. This article reviews AI’s development and considers how to avoid another winter, highlighting the risks of exaggerated promises and the challenges of trustworthy systems.
Problem
The prospect of another AI winter matters because prior winters brought funding cuts, and a future downturn could cost jobs, close startups, and widen the economic gap by tens of billions of dollars by 2025.
Source: §5 · p. 12Method
The article reviews AI’s history and identifies considerations for avoiding another winter, including philosophical perspectives and the risks of exaggerated claims.
Source: §1 · p. 3, §5 · p. 12Results
The article recounts how exaggerated promises and reports were followed by funding cuts and winters, while expert systems’ unmet promises and hardware constraints preceded another industry collapse.
Source: §3.3 · p. 8, §3.4 · p. 9, §3.5 · p. 9Takeaways & Limitations
The article concludes that AI’s prospects are promising but trustworthy use must account for fragility, injustice, opacity, and interpretability-performance trade-offs.
Source: §5 · pp. 13–14AI models can trade interpretability against performance, while medical diagnostic systems must remain intelligible and explainable to stakeholders.
Source: §5 · p. 14
Abstract
from arXiv · showhide
The field of artificial intelligence (AI), regarded as one of the most enigmatic areas of science, has witnessed exponential growth in the past decade including a remarkably wide array of applications, having already impacted our everyday lives. Advances in computing power and the design of sophisticated AI algorithms have enabled computers to outperform humans in a variety of tasks, especially in the areas of computer vision and speech recognition. Yet, AI's path has never been smooth, having essentially fallen apart twice in its lifetime ('winters' of AI), both after periods of popular success ('summers' of AI). We provide a brief rundown of AI's evolution over the course of decades, highlighting its crucial moments and major turning points from inception to the present. In doing so, we attempt to learn, anticipate the future, and discuss what steps may be taken to prevent another 'winter'.
1 Introduction
AI has become prominent across everyday life, with recent advances and applications drawing widespread attention. This article sets out to explain what AI is and trace its history.
1 Introduction
- AI is attracting global attention through reported breakthroughs, including increasingly convincing deepfakes and AlphaGo’s victory over the Go world champion.The introduction also quotes prominent figures predicting broad societal and industrial impact.Source: §1 · p. 2
- The article aims to explain what AI is and recount the history behind this rapidly evolving technology.It situates AI’s developments within a longer history built on earlier work.Source: §1 · p. 3
2 What is AI?
AI is described as machines performing tasks associated with human intelligence, but the field also distinguishes approaches by whether they emulate human thought or behavior, or pursue rationality.
2 What is AI?
- AI is described as computer systems performing tasks that normally require human intelligence, including perception, speech recognition, decision-making, and translation.Other definitions emphasize mimicking human perception, reasoning, learning, and problem-solving.Source: §2 · pp. 3–4
- Russell and Norvig distinguish four approaches: acting humanly, thinking humanly, acting rationally, and thinking rationally.Source: §2 · p. 4
- The Turing test asks whether a human interrogator can distinguish a machine’s answers from a person’s; its extended version adds vision, speech recognition, and robotics.The basic test requires language processing, knowledge representation, automated reasoning, and machine learning.Source: §2 · p. 4
3 History of AI
The article frames AI history as recurring cycles of promise and disappointment, then traces early conceptual, mathematical, and technological precursors to the field.
3 History of AI
- AI’s history is divided into three cycles of public excitement and disappointment, described as recurring ridges of promise and valleys of disappointment.Source: §3 · p. 4
3.1 Prehistoric Events
- Asimov’s I, Robot is described as an early source of intelligent-machine ideas, while McCulloch and Pitts’ 1943 article marked a first step toward AI implementation.The article presents the mathematical neuron model as a foundation for artificial neural networks.Source: §3.1 · pp. 4–5
- Work on the basic physiology and function of brain neurons showed that essentially any computable function could be modeled as a connected network; six years later, Hebb proposed a rule to tune connection strengths.Source: §3.1 · p. 5
- Minsky and Edmonds built SNARC, a 40-neuron analog machine that successfully found its way out of a maze.The article also recounts Turing’s universal computing machine and imitation game, and early checkers programs developed in 1952.Source: §3.1 · p. 6
3.2 The First Summer of AI
The first AI summer began with the Dartmouth meeting and early demonstrations of symbolic reasoning, learning, neural networks, and other AI applications.
3.2 The First Summer of AI
- The 1956 Dartmouth workshop coined “artificial intelligence,” coordinated the emerging field, and is presented as the beginning of its first summer.Source: §3.2 · p. 6
- The Logic Theorist proved 38 of 52 theorems in a chapter of Principia Mathematica, and Newell and Simon later developed the human-problem-solving-oriented General Problem Solver.Source: §3.2 · p. 6
- Samuel’s 1956 checker program learned through rewards and reached intermediate play above its developer; Rosenblatt’s single-layer perceptron learned to classify two categories.The perceptron identified left-versus-right cards in 50 attempts.Source: §3.2 · p. 7
- Early summer-era developments also included LISP, geometry theorem proving, evolutionary algorithms, industrial robotics, microworld programs, ELIZA, and SHAKEY.LISP became a prevailing AI programming language for three decades.Source: §3.2 · pp. 7–8
3.3 The First Winter of AI
The first AI winter followed funding cuts after public hype and expert overpromising, reinforced by government reports that questioned AI research prospects. The account identifies three contributing weaknesses: human-imitation approaches, underestimated problem complexity, and negative perceptions of neural networks.
3.3 The First Winter of AI
- Public hype and expert overpromising preceded major AI research funding cuts in the late 1960s; government reports then helped usher in the first winter, which lasted until the 1980s.The ALPAC report (1966) and Lighthill report (1973) targeted AI research, and US and UK governments reduced university support.Source: §3.3 · p. 8
- Early systems often copied human problem-solving instead of analyzing tasks bottom-up, while oversimplified frameworks left many real-world problems intractable despite assumptions that more hardware would suffice.The passage attributes this scaling misconception to a failure to recognize problem complexity and developments in computational complexity theory.Source: §3.3 · p. 8
- Minsky’s critique that a single-layer perceptron could not implement XOR contributed to negative perceptions of neural networks and global funding cuts in neural-network research.The passage says this was not a general critique of neural networks.Source: §3.3 · pp. 8–9
3.4 Expert Systems; the Revival of AI, and the Second Summer
As general-purpose search approaches struggled to scale, AI researchers turned to narrower, domain-specific expert systems. Their commercial uptake and broader investment helped drive an AI industry boom during the 1980s.
3.4 Expert Systems; the Revival of AI, and the Second Summer
- Expert systems addressed weak AI’s poor scalability by using domain-specific information for stronger reasoning in narrower areas, and quickly reached corporations.Earlier weak AI approaches offered general solutions using search across possible states but struggled with larger or more complex domains.Source: §3.4 · p. 9
- DENDRAL inferred molecular structure from mass-spectrometry data using special-purpose laws, and was an early effective knowledge-intensive system.The passage describes it as providing expertise rather than basic knowledge.Source: §3.4 · p. 9
- MYCIN diagnosed sepsis and recommended antibiotic dosages using about 600 rules and uncertainty calculations, while R1 configured computer orders and added $40 million in revenue in nearly four years.MYCIN performed on par with human experts and better than medical interns; R1 was the first successful commercial expert system, according to the passage.Source: §3.4 · p. 9
- By the late 1980s, AI had become a billion-dollar industry, with companies developing expert systems, vision systems, robots, and specialized software and hardware.The passage traces the industry’s growth from a few million dollars in 1980 to billions by the late 1980s.Source: §3.4 · p. 9
3.5 The Second Winter of AI
The expert-systems boom collapsed when companies fell short of promises and hardware makers did not meet specialized requirements, ushering in another AI winter. During and after this period, research revisited backpropagation, adopted statistical methods and public benchmarks, and regained public attention with Deep Blue’s 1997 chess victory.
3.5 The Second Winter of AI
- During the second winter, researchers revisited backpropagation and shifted toward established statistical methods, including hidden Markov models for handwriting recognition and speech processing.The passage also links the conservative shift to public benchmark datasets and competitions used to measure research advances.Source: §3.5 · pp. 9–10
3.6 Man versus Machine
- In 1997, IBM’s Deep Blue defeated chess champion Garry Kasparov in a widely broadcast match that renewed public interest in AI systems.The passage says the news captured public imagination and IBM’s share values rose to all-time highs.Source: §3.6 · p. 10
3.7 Information Age: Enter Big Data
Big data, powerful computers, and labeled image datasets helped create conditions for renewed AI interest. In 2012, a deep convolutional network surpassed ImageNet competitors, marking a new boom in AI and drawing broad public attention to deep learning.
3.7 Information Age: Enter Big Data
- Powerful computers and Internet growth enabled large-scale data, while ImageNet gathered millions of labeled images and contributed significantly to computer vision.The passage places these developments in the late 1990s onward and identifies 2009 as the year ImageNet was created.Source: §3.7 · p. 10
3.8 Return of Neural Networks
- In 2012, AlexNet significantly enhanced ImageNet classification benchmarks, surpassing competitors with a deeper convolutional network trained using GPUs and millions of labeled images.The passage notes that GPUs shortened training time and labeled data helped address overfitting; AlexNet used rectified linear unit activations and dropout.Source: §3.8 · p. 10, §3.7 · p. 10
- Deep learning methods later dominated AI-related research, with systems performing on par with or outperforming human experts in some tasks, including computer vision and medical image diagnosis.AlphaGo’s 2016 win over the world Go champion brought renewed attention from media, governments, industry, scholars, and investors.Source: §3.8 · p. 10
- Figure 8 presents a timeline of AI developments from the field’s beginnings to the present.The passage identifies the figure as summarizing AI’s timeline.Source: Figure 8 · p. 11, §3.8 · p. 11
4 Where Do We Stand Now?
AI’s recent ascent reflects converging computing, data, and algorithmic advances, alongside expanding investment, research activity, and applications. The authors warn that high-stakes uses make trustworthy algorithms especially important.
4 Where Do We Stand Now?
- Parallel processing, greater memory, massive data collection, and improved algorithms accompany AI’s steady upward climb since the early 2020s.The passage also reports computing power doubling nearly every two years.Source: §4 · p. 11
- From 2010 to 2020, global investment in AI startups rose from $1.3 billion to more than $40 billion, averaging nearly 50% annual growth.Corporate AI investment in 2020 was reported at nearly $70 billion globally.Source: §4 · p. 11
- AI research and applications have expanded, including protein-folding advances and pandemic drug-development work, while life-or-death uses make trustworthy algorithms critical.Peer-reviewed AI articles grew roughly 12-fold worldwide from 2000 to 2020.Source: §4 · pp. 11–12
5 The Future of AI
The authors argue that avoiding another AI winter requires tempered claims, attention to technical and methodological limits, and responsible development. They discuss explainability and neuro-symbolic approaches as ways to address trust and combine complementary strengths.
5 The Future of AI
- Exaggerated promises alongside technical limitations have recurred in AI winters, which brought research budget cuts; the authors urge attention to these risks.They cite the perceptron’s XOR limitation and expert-system limitations as examples.Source: §5 · p. 12
- Avoiding another winter calls for more modest predictions, clearer AI terminology, stronger scientific methodology, and attention to computational complexity and philosophical critique.The authors report that 40% of European startups claiming to use AI did not actually do so, citing an ambiguous public definition.Source: §5 · p. 12
- Deep-learning decisions may be opaque, creating trustworthiness challenges in critical settings such as health care, transportation, and financial systems.The concern does not arise in the same way in tasks such as winning a board game.Source: §5 · p. 13
- Language models can acquire biases from human-generated data, and their use in sensitive contexts makes avoiding biased decisions against communities or cultures important.The passage notes gender associations involving BERT and peers, and says GPT-3 has 175 billion parameters [100].Source: §5 · p. 13
- AI systems may be fragile or unjust, and the authors point to ethics guidelines and an XAI roadmap among efforts to encourage responsible implementation.Examples include adversarially altered stop signs or skin-lesion images, discriminatory risk tools, and sexist recruitment tools.Source: §5 · p. 13
- A reported interpretability-performance trade-off complicates explainable AI: medical systems need sufficient complexity while remaining intelligible and retraceable to professionals.The passage presents symbolic-connectionist convergence and hybrid models as promising directions, while identifying the paradigms’ historical division as an obstacle [103].Source: Figure 9 · p. 13, §5 · p. 14
- The co-citation network depicts symbolic and connectionist communities and their proximities, but is not comprehensive and omits some contributors.The figure is presented as illustrating the paradigms’ dichotomies.Source: Figure 10 · p. 15
- Neuro-symbolic AI seeks to combine neural trainability from raw data and robustness with symbolic explainability, provable correctness, and expert-knowledge-based design [105].Its hoped-for benefits include out-of-vocabulary handling, small-data generalization, error recovery, and explainability.Source: §5 · p. 15
6 Summary
AI has advanced rapidly and can outperform humans in more tasks, but its history includes two winters following periods of high expectations. The authors argue that the current wave of hopes should not be taken for granted.
6 Summary
- Despite rapid advances and decades of praise, AI has experienced two winters after periods of growth and high expectations, making current hopes uncertain.The authors stress that researchers, corporations, startups, and governments have borne costs during past downturns.Source: §6 · p. 16
Glossary
The glossary defines core AI concepts, learning methods, models, data resources, institutions, and technical terms used in the paper.
Core AI and learning terms
- AI is described as computer systems imitating intelligent human behavior; machine learning uses data for task learning, while deep-learning and neural-network entries concern brain-like systems.The glossary also identifies artificial neural networks as a term without a separate explanatory entry.Source: Glossary · p. 20
Models and learning methods
- Reinforcement learning optimizes cumulative reward; hidden Markov models represent Markov processes, and general problem solver names 1959 software intended as a universal solver.The glossary also defines Bayesian networks as directed acyclic graphs describing variables and conditional dependencies.Source: Glossary · p. 20
- Perceptron is a supervised binary-classification approach; backpropagation trains feedforward networks, and vanishing gradients can prevent weights from changing.The glossary describes convolutional neural networks as stacked convolutional layers primarily used for computer vision, and ReLU as an activation function.Source: Glossary · pp. 20–21
Areas and capabilities
- Natural language processing covers methods such as translation; computer vision extracts information from images or video, while speech recognition enables computers to comprehend spoken language.Cognitive science is defined as the interdisciplinary study of the mind and its processes.Source: Glossary · p. 20
Concepts and tests
- The glossary defines a Turing test by whether an interrogator can distinguish machine from human answers, and logical notation as symbols used for logical representations.The XOR entry gives the truth conditions for the exclusive-OR operator.Source: Glossary · pp. 20–21
Organizations, tools, and systems
- Institutional entries include CVPR, AAAI, NeurIPS, WiML, AI4ALL, and the AI Index; technical entries include GPUs, Moore’s law, DeepMind, AlphaFold, BERT, and GPT-3.The glossary also defines MCC as a US computer-industry research and development consortium.Source: Glossary · pp. 21–22
AI system concepts
- Expert systems simulate human-expert decisions using bodies of knowledge, often represented as if–then rules; strong AI contrasts with weak AI’s task-specific scope.The glossary describes XAI as AI whose outcomes humans can understand, in contrast to an opaque black box.Source: Glossary · pp. 21–22
Resources and data
- The glossary names research resources including MNIST, ImageNet, and ILSVRC, and defines big data and data science in terms of data scale and analysis.It also defines Markov decision processes as stochastic control frameworks for partly random, partly controlled outcomes.Source: Glossary · p. 21