Gathering Strength, Gathering Storms: The One Hundred Year Study on Artificial Intelligence (AI100) 2021 Study Panel Report

ABOUT AI100

• AI100’s Standing Committee forms a Study Panel every five years to review AI’s progress and assess the technical and societal challenges and opportunities ahead.The study’s history first appeared in the 2016 report’s preface.Source: ABOUT AI100 · p. 4

INTRODUCTION

• The 2021 report is organized around 12 standing questions and two workshop questions, with an overview of the panel’s responses before the full discussions.It also includes a brief conclusion and an appendix annotating the 2016 report.Source: INTRODUCTION · p. 5

HOW TO CITE THIS REPORT

• The report’s formal citation identifies its 17 authors and gives Stanford University, Stanford, CA, and September 2021 as publication details.Michael L. Littman is listed as chair of the Study Panel.Source: HOW TO CITE THIS REPORT · p. 5, Study Panel · p. 5

Study Panel

• The Study Panel roster lists members’ university and research-institution affiliations, including appointments in the United States, Canada, the United Kingdom, Japan, and Australia.The roster identifies Littman as chair.Source: Study Panel · p. 5

SQ2. What are the most important advances in AI?; SQ5. What are the prospects for more general artificial intelligence?

• AI now supports language, image, medical, financial, and recommendation applications, while fully autonomous driving progress fell short of predictions and image uses face backlash.The report also describes self-supervised learning, continual learning, and task generalization as capabilities relevant to more general AI, which remains far from achieved.Source: STANDING QUESTIONS AND SECTION SUMMARIES · pp. 7–8

SQ3. What are the most inspiring open grand challenge problems?

• Open challenges include generalizability, causality, and normativity; a broader goal is AI that collaborates with people and aligns decisions with fluid human values.The report describes collective intelligence, cognitive neuroscience, and computational modeling as prominent approaches to understanding intelligence, while consciousness and integrating multiple modalities remain mysterious.Source: STANDING QUESTIONS AND SECTION SUMMARIES · p. 7

SQ4. How much have we progressed in understanding the key mysteries of human intelligence?

• Recent views emphasize collective intelligence, brain implementation of psychological and social processes, and computational models, while consciousness and cross-modal integration remain poorly understood.The report describes collective intelligence as a property of groups as well as individuals.Source: STANDING QUESTIONS AND SECTION SUMMARIES · pp. 7–8

SQ6. How has public sentiment towards AI evolved, and how should we inform/educate the public?; SQ7. How should governments act to ensure AI is developed and used responsibly?

• Public attention to AI has grown, while media coverage can distort its potential; governments, universities, and nonprofits are expanding education, and countries are pushing back against indiscriminate facial recognition.The report says researchers should move beyond educating the public toward participatory engagement; it also notes that few countries had enacted AI-specific regulation outside data-related rules.Source: STANDING QUESTIONS AND SECTION SUMMARIES · pp. 8–9

SQ10. What are the most pressing dangers of AI?

• AI can reproduce or amplify discrimination through biased decisions and feedback loops; in high-stakes applications, poor integration can also lead people to mistrust or over-rely on systems.The report names techno-solutionism—the view of AI as a panacea—as a pressing danger.Source: STANDING QUESTIONS AND SECTION SUMMARIES · pp. 9–10

SQ11. How has AI impacted socioeconomic relationships?; SQ12. Does it appear “building in how we think” works as an engineering strategy in the long run?

• AI’s aggregate economic effects have been small to date, while general-purpose methods face limits in computation, data, brittleness, and semantics that may renew interest in specialized components.The report says AI’s economic significance has been comparatively small relative to expectations, and that inequality predates substantial commercial AI use.Source: STANDING QUESTIONS AND SECTION SUMMARIES · p. 10

SQ8. What should the roles of academia and industry be?; SQ9. What are the most promising opportunities for AI?; WQ1. Lessons from “Prediction in Practice” workshop

• Industry leads AI investment as researchers move from academia, while human-AI teams and stakeholder involvement offer opportunities; high-stakes prediction also depends on problem definition and integration into institutions.Workshop participants stress multidisciplinary understanding, practitioner and community relationships, and attention to technical limitations when integrating predictive systems.Source: STANDING QUESTIONS AND SECTION SUMMARIES · p. 9, WORKSHOP QUESTIONS AND SECTION SUMMARIES · p. 11

WQ2. Lessons from “Coding Caring” workshop

• AI caregiving can support autonomy and well-being, but companion robots should augment rather than replace caring relationships, since good care requires respect and dignity that procedural algorithms cannot encode.Some patients may prefer robotic care in privacy-sensitive situations.Source: WORKSHOP QUESTIONS AND SECTION SUMMARIES · p. 11

SQ2. WHAT ARE THE MOST IMPORTANT ADVANCES IN AI?

• AI made major progress across standard sub-areas and produced breakthrough applications in games, medicine, logistics, autonomous driving, translation, and personal assistance.The report says these developments emerged over the last five years.Source: SQ2. WHAT ARE THE MOST IMPORTANT… · p. 12
• Language models can generate human-like text and perform above human levels on some question-answering benchmarks, but exploit spurious correlations and lack deep understanding.Their capabilities support translation, classification, speech recognition, writing aids, and chatbots; smaller-language data quality and bias remain challenges.Source: SQ2. WHAT ARE THE MOST IMPORTANT… · p. 13
• ImageNet programs complete visual-identification work 100 times faster than three years earlier, while AI systems can generate novel images and photorealistic videos.Deepfakes have lowered barriers to illicit sexual content and identity theft, prompting research into automatic detection.Source: SQ2. WHAT ARE THE MOST IMPORTANT… · pp. 13–14
• AI agents now outperform human opponents in several games, and AlphaZero learned expert-level Go, Shogi, and Chess through self-play.The report cites StarCraft II, Quake III, Alpha Dogfight, and poker as other examples of progress.Source: SQ2. WHAT ARE THE MOST IMPORTANT… · pp. 14–15
• Robots have become more agile and useful in industrial settings, but most robotics systems remain lab-bound and autonomous delivery robots still make mistakes in novel environments.Fully autonomous driving has progressed more slowly than predicted; safety in complex environments makes it harder and more expensive than anticipated, though vehicles operate in some locales.Source: SQ2. WHAT ARE THE MOST IMPORTANT… · pp. 15–16
• AI is increasingly used in healthcare for diagnosis, drug discovery, risk scoring, and operational support, with some diagnostic systems rivaling expert pathologists and radiologists.The report notes bias in some risk-scoring systems and ethical concerns in other automated healthcare uses.Source: SQ2. WHAT ARE THE MOST IMPORTANT… · pp. 16–17
• AI adoption in finance and recommender systems is expanding, while raising concerns about social effects, fairness, diversity, and polarization.Financial applications include lending, payments, robo-advising, and risk monitoring; recommender systems increasingly use deep learning and content analysis, with research exploring diversity and fairness tradeoffs.Source: SQ2. WHAT ARE THE MOST IMPORTANT… · pp. 17–18

SQ3. WHAT ARE THE MOST INSPIRING OPEN GRAND CHALLENGE PROBLEMS?

• A central open challenge is building AI that cooperates with people and aligns with complex human values; benchmark-focused research can move the field away from that goal.Addressing it requires collaboration between computer scientists, social scientists, and humanists.Source: SQ3. WHAT ARE THE MOST INSPIRING OPEN… · p. 19

Turing Test

• Language models can produce human-like text but remain unreliable, may violate social norms, and raise concerns about systems deceiving people and destabilizing human relationships and institutions.The report proposes a contemporary Turing challenge: fluent communication without being mistaken for a human, with the system’s nonhumanness apparent without baffling its teammates.Source: SQ3. WHAT ARE THE MOST INSPIRING OPEN… · p. 20

GENERALIZABILITY

• Robust generalization remains an open challenge: AI must transfer learning to novel tasks, learn from small samples and analogy, and adapt to shifts in the examples it encounters.The report points to integrating symbolic and probabilistic reasoning as a possible requirement for greater generality.Source: SQ3. WHAT ARE THE MOST INSPIRING OPEN… · p. 22

CAUSALITY

• Current machine-learning systems can discover patterns and perform varied tasks, but have limited causal knowledge and cannot reliably predict the effects of novel interventions or alternative conditions.It remains open whether sufficiently complex causal models can be learned from text alone, without interaction.Source: SQ3. WHAT ARE THE MOST INSPIRING OPEN… · p. 23

NORMATIVITY

• AI systems must adapt to culturally and locally variable norms and fit into human institutions; fairness cannot be achieved simply by imposing statistical output constraints.Accountability requires reviewable normative justifications consistent with human values, not only explanations of how a decision was reached.Source: SQ3. WHAT ARE THE MOST INSPIRING OPEN… · pp. 23–24

SQ4. HOW MUCH HAVE WE PROGRESSED IN UNDERSTANDING THE KEY MYSTERIES OF HUMAN INTELLIGENCE?

• Groups can outperform individuals when they encourage challenge and constructive criticism, while social networks can spread beliefs and foster polarization even among rational decision-makers.Group performance is better predicted by qualities such as turn-taking and empathy than by members’ IQs; information bubbles can split people into incompatible subgroups.Source: SQ4. HOW MUCH HAVE WE PROGRESSED IN… · pp. 25–26
• Cognitive neuroscience links intelligence to efficient processing and connectivity, and also to flexible, plastic brain networks; computational reinforcement learning has helped explain dopamine’s role in learning and decision-making.Social neuroscience studies how social environments shape cognition, emotion, and behavior.Source: SQ4. HOW MUCH HAVE WE PROGRESSED IN… · pp. 27–28
• Deep-learning models now dominate computational modeling of cognition, while Bayesian approaches remain active and the debate weighs accurate prediction against interpretable logic.Deep-learning models also serve as classification tools for building stimulus sets and as resources for studying language and discourse.Source: SQ4. HOW MUCH HAVE WE PROGRESSED IN… · p. 28
• How people integrate information across senses, bodies, tools, and other people remains largely mysterious, and rigorous models of intentional action, shared intentionality, free will, and agency remain limited.The nature of consciousness also remains open, and it is unclear whether understanding it is necessary to explain human intelligence.Source: SQ4. HOW MUCH HAVE WE PROGRESSED IN… · p. 29

SQ5: WHAT ARE THE PROSPECTS FOR MORE GENERAL ARTIFICIAL INTELLIGENCE?

• Progress toward more general AI focuses on self-supervised or self-motivated learning, continual learning across domains without extensive retraining, and generalizing acquired skills to new tasks.Most current state-of-the-art AI applications remain narrow AI, designed to excel at specific tasks.Source: SQ5: WHAT ARE THE PROSPECTS FOR MORE… · pp. 29–30
• Self-supervised transformer models learn from incomplete examples without human-created labels and can handle varied data, then achieve strong task performance directly or with limited fine-tuning.Transformers use attention to process sequences with long-range dependencies and have become a go-to approach in natural language processing.Source: SQ5: WHAT ARE THE PROSPECTS FOR MORE… · pp. 30–31
• Meta-learning supports transferable representations and program modules that can be adapted to new tasks, while continual-learning methods aim to prevent new training from erasing previously learned skills.Some approaches use brain-inspired neuromodulatory processes to gate network activity and reduce forgetting.Source: SQ5: WHAT ARE THE PROSPECTS FOR MORE… · p. 31
• Deep reinforcement learning remains task-specific in many cases, but intrinsic motivation and learned generative world models are being used to support multitask or continual learning and improve learning speed.Intrinsic motivation can reward exploration or learning from prediction errors; world models let agents test policies in imagined scenarios and build increasingly challenging curricula.Source: SQ5: WHAT ARE THE PROSPECTS FOR MORE… · p. 32
• These approaches remain early steps toward general AI: despite promise in restricted domains, they need to scale to the diverse, complex problems of the real world.Approaches include learning from few examples, continual learning without inter-task interference, and self-supervised or intrinsically motivated learning.Source: SQ5: WHAT ARE THE PROSPECTS FOR MORE… · p. 32
• Common sense includes largely unconscious world knowledge, causal understanding, and the ability to perceive abstract similarities and make analogies.Recent research explores causal models, intuitive physics, abstraction, and analogy.Source: SQ5: WHAT ARE THE PROSPECTS FOR MORE… · pp. 32–33
• AI remains far from human abilities in these areas, and may never attain common sense or general intelligence without tighter coupling to the physical world.Addressing these issues may advance AI and improve understanding of human mechanisms of general intelligence.Source: SQ5: WHAT ARE THE PROSPECTS FOR MORE… · p. 33
• Public understanding of AI has broadened and become more nuanced, moving beyond fears of Terminators and robot overlords.Source: SQ5: WHAT ARE THE PROSPECTS FOR MORE… · p. 33

• AI-related words appeared more than ten times as often in the 2017–18 US Congress session as in previous sessions, while searches for “machine learning” roughly doubled since 2016.Source: SQ6. HOW HAS PUBLIC SENTIMENT TOWARDS… · p. 33
• Public concerns have shifted beyond robot-overlord fears toward discrimination, inequality, labor impacts, surveillance, manipulation, and disinformation.These concerns contribute to efforts to promote trustworthy AI.Source: SQ6. HOW HAS PUBLIC SENTIMENT TOWARDS… · p. 33
• Public sentiment differs by region and gender: surveyed Asian countries were largely positive, Western countries more divided and skeptical, and men more positive than women.A separate 2019 survey found more positive attitudes among people who were wealthy, educated, male, or had technology experience.Source: SQ6. HOW HAS PUBLIC SENTIMENT TOWARDS… · p. 34
• Public communication can blur AI-related harms with effects of broader platforms and business models, while scientific communication has not reached a sufficiently broad range of publics.The report says education should distinguish AI from other information technologies without artificially separating it from the platforms that shape its development and deployment.Source: SQ6. HOW HAS PUBLIC SENTIMENT TOWARDS… · pp. 34–36
• AI-race and human-versus-AI competition narratives obscure human agency and responsibility, as well as collaborative and participatory alternatives.The report describes technological determinism as the false claim that society is shaped by technology independently of human choices and values.Source: SQ6. HOW HAS PUBLIC SENTIMENT TOWARDS… · p. 35
• Industry messaging promotes “AI for good” and “responsible/ethical AI,” but raises ethicswashing concerns; opposition to indiscriminate facial recognition also grew.The passage notes that IBM, Amazon, and Microsoft announced pauses or moratoria on law-enforcement use of facial-recognition technology.Source: SQ6. HOW HAS PUBLIC SENTIMENT TOWARDS… · p. 36
• Public outreach should clarify AI’s varied uses and risks, avoid breakthrough hype, and shift from one-way education toward participatory engagement.The report emphasizes that different groups face different risks and priorities, and that participatory models can support democratic involvement.Source: SQ6. HOW HAS PUBLIC SENTIMENT TOWARDS… · pp. 36–37

Law, Policy, and Regulation

• Few countries have enacted AI-specific regulation; the EU has proposed concrete frameworks, 24 countries permit autonomous vehicles in limited settings, and only Belgium has enacted a law on lethal autonomous weapons.The report describes the EU as the most active government body in proposing regulatory frameworks, including its 2021 Coordinated Plan on AI [145].Source: SQ7. WHAT ARE GOVERNMENTS DOING TO… · pp. 37–38
• 48.3% of the global population used social media in 2020, while debate continues over whether and how governments should regulate platforms’ content moderation and other risks.Companies make most content-moderation decisions under their own legal guidance and policies; some, such as Facebook, have created independent oversight boards, and the report notes few easy policy answers.Source: SQ7. WHAT ARE GOVERNMENTS DOING TO… · p. 39

AI Research & Development as a Policy Priority

• $1.5 billion was the US government’s 2020 investment in unclassified AI-related R&D; China’s estimated government AI R&D investment was $9.4 billion in 2018.The report also notes significant recent increases in European public investment and urges governments to invest more in AI research, regulation, cross-disciplinary research, K–12 education, and public-facing applications such as healthcare.Source: SQ7. WHAT ARE GOVERNMENTS DOING TO… · pp. 39–40

Cooperation and Coordination on International Policy

• Governments have established AI cooperation initiatives, including the OECD AI Policy Observatory and the G7’s Global Partnership on AI, while US-EU coordination focuses on standards and threats to security and human rights.The report also describes earlier regional and bilateral efforts, and notes that international coordination could help ease tensions as nations compete for AI dominance.Source: SQ7. WHAT ARE GOVERNMENTS DOING TO… · pp. 40–41

Case Study: Lethal Autonomous Weapons

• In the lethal autonomous weapons debate, smaller countries have supported a ban while major militaries argue existing humanitarian law is sufficient, amid concerns about accountability, proliferation, and legality.The UN Convention on Certain Conventional Weapons has debated the issue since 2013; the report says growing military uses of AI will require government expertise to implement safety and reliability standards.Source: SQ7. WHAT ARE GOVERNMENTS DOING TO… · p. 41

From Principles to Practice

• At least 117 AI-principles documents appeared between 2015 and 2020, most published by companies, but principles have limited utility without government oversight, enforceability, or accountability.The report describes efforts to turn principles into practice through implementation and testing, including rights-based approaches, due process and recourse, and the multistakeholder Partnership on AI.Source: SQ7. WHAT ARE GOVERNMENTS DOING TO… · pp. 41–42

Dynamic Regulation, Experimentation, and Testing

• AI’s pace strains slow, inflexible regulatory systems, so the report recommends more adaptive governance, including regulatory science, risk-based impact assessments, and experimentation and testing.Examples include agency research to inform regulatory decisions, AI or algorithm impact assessments, FAA UAS test sites that feed data and incident reports into rulemaking, and simulations before deployment.Source: SQ7. WHAT ARE GOVERNMENTS DOING TO… · pp. 42–43

SQ7. WHAT ARE GOVERNMENTS DOING TO ENSURE THAT AI IS DEVELOPED AND USED RESPONSIBLY?

• The National Security Commission on Artificial Intelligence recommended federal data repositories and access to large-scale computing resources, while optimal resource allocation remains an open problem..Source: SQ7. WHAT ARE GOVERNMENTS DOING TO… · p. 45
• The report says tighter academic–industry coupling may reduce attention to long-term or commercially inconvenient problems, while company incentives to address privacy and fairness do not ensure those issues are handled correctly.It also notes student intellectual-property concerns and cites the Partnership on AI as a corporate investment in responsible-use collaboration.Source: SQ7. WHAT ARE GOVERNMENTS DOING TO… · p. 46
• Academic researchers are increasingly deploying AI beyond the lab, and nonprofit and commercial collaborations have supported public-facing applications.Examples include Duolingo and university–industry pandemic initiatives.Source: SQ7. WHAT ARE GOVERNMENTS DOING TO… · p. 46
• A study counted 131 AI faculty departures to industry and 90 faculty taking reduced academic roles from 2004 to 2018, and concluded that departures negatively affected Ph.D. training.The report notes no sustained decline in computer science Ph.D. graduates yet, but warns one could develop; internships and company-led courses can complement university education.Source: SQ7. WHAT ARE GOVERNMENTS DOING TO… · pp. 46–47
• Government oversight may be needed because companies can withdraw support from ethics initiatives that conflict with business interests, even as academic–industry collaboration enables research access and policy guidance.The report calls for regulation to be developed in consultation with researchers who understand the issues and how to address them.Source: SQ7. WHAT ARE GOVERNMENTS DOING TO… · p. 47

SQ9. WHAT ARE THE MOST PROMISING OPPORTUNITIES FOR AI?

• Studies find that human–AI teams often do not outperform AI-only teams, motivating continued work on more effective collaboration methods.The report says stakeholders should help design AI assistants, and users need to understand system limitations while designers understand the use context.Source: SQ9. WHAT ARE THE MOST PROMISING… · pp. 48–49
• AI can support decisions by summarizing complex information and forecasting outcomes, but forecasts have data-based limits and biases and may cause harm if overtrusted or used in decisions affecting people.Examples include clinical summaries, early-warning systems, public-sector triage, and education support.Source: SQ9. WHAT ARE THE MOST PROMISING… · pp. 49–50
• AI assistance may broaden access to health, accessibility, and safety support, including home care, disability tools, lane-keeping, surgical precision, and fraud or phishing detection.The report cautions that such systems can cause harm when designed or integrated without adequate consideration of their sociotechnical context.Source: SQ9. WHAT ARE THE MOST PROMISING… · pp. 50–51
• AI could automate tasks across biology, health, public services, communications, education, and training, but realizing benefits depends on suitable data, integration, stakeholder-aligned objectives, and responsible regulation.Many current barriers lie in data collection and organization, sociotechnical integration, and legal or economic incentives rather than algorithms alone.Source: SQ9. WHAT ARE THE MOST PROMISING… · p. 52

SQ10. WHAT ARE THE MOST PRESSING DANGERS OF AI?

• AI can be treated as a panacea rather than a tool, while biased systems may reproduce or amplify discrimination and create feedback loops that deepen it.Automated social-service systems can rigidly deny access, and recommendation algorithms can reinforce biases in the information users encounter.Source: SQ10. WHAT ARE THE MOST PRESSING… · p. 53
• Accepting AI predictions as facts can entrench unequal outcomes, especially when crime or sentencing systems use skewed data or users mistake probabilities for certainty.The report notes that predictive policing disproportionately projected crimes in areas with more non-white and low-income residents, and that algorithmic risk estimates may be misread by courts.Source: SQ10. WHAT ARE THE MOST PRESSING… · pp. 53–54
• Opaque algorithms and biased proxies can conceal discriminatory decisions, leaving affected people without adequate information, due process, or redress.The report also warns that data-driven systems can use seemingly innocuous attributes correlated with protected classes.Source: SQ10. WHAT ARE THE MOST PRESSING… · pp. 54–55
• AI-enabled deepfakes and bots can spread disinformation, undermine social trust, and manipulate public discourse for economic or political advantage.The report describes these systems as potential threats to democracy and tools for manipulating evidence and power structures.Source: SQ10. WHAT ARE THE MOST PRESSING… · p. 55
• In healthcare, models that fail to generalize or reflect biased training data can worsen disparities, while clinicians may either distrust systems or rely on them when they are wrong.These risks affect high-stakes decisions including pain management, cancer prediction, and healthcare resource allocation.Source: SQ10. WHAT ARE THE MOST PRESSING… · pp. 55–56

SQ11. HOW HAS AI AFFECTED SOCIOECONOMIC RELATIONSHIPS?

• AI has not yet caused large aggregate labor-market effects or played a major role in rising inequality, though its impacts may be more meaningful in sectors where it is prevalent.The report describes AI’s labor-market effects as localized and notes that data ownership is currently only one factor in growing market concentration.Source: SQ11. HOW HAS AI AFFECTED… · pp. 56–59
• AI’s economic significance may grow as it spreads, but adoption could be gradual, offset by aging workforces, or intensify inequality if policymakers fail to respond.The report notes that evidence of aggregate automation acceleration is limited, while future policy could help distribute technology-driven prosperity more equitably.Source: SQ11. HOW HAS AI AFFECTED… · pp. 59–60

AI methods and limitations

• AI has swung between general-purpose and specialized approaches, and limits in computation, data, robustness, and semantics are prompting renewed interest in combining learned methods with domain-specific components.The report suggests that progress may come from combining both approaches, including neurosymbolic AI, alongside faster processors and larger datasets.Source: SQ11. HOW HAS AI AFFECTED… · pp. 61–63

Problem formalization

• Problem formalization shapes algorithm design and policy outcomes, so practitioners should ask what is being predicted, whose goals define it, and whether AI is warranted at all.Different definitions of need or policy goals can change who receives scarce housing assistance or family subsidies; technical solutions may also obscure root causes and nontechnical alternatives.Source: SQ11. HOW HAS AI AFFECTED… · p. 64

Integration, not deployment

• Public-sector systems should be integrated into local organizational settings rather than simply deployed, with frontline workers’ knowledge, resources, maintenance, and affected communities considered.The COVID-19 clinical AI efforts were deemed unfit for clinical use amid poor data sharing, insufficient cross-disciplinary expertise, and overoptimism; stakeholder participation can help, but should not be tokenistic.Source: SQ11. HOW HAS AI AFFECTED… · p. 65

Responsible AI in public decisions

• High-stakes public AI requires multidisciplinary understanding of context, meaningful relationships with practitioners and affected communities, and attention to technical limits and political values.Predictive systems are political as well as technical: their effects depend on how they fit into existing decisions, policies, and institutions.Source: SQ11. HOW HAS AI AFFECTED… · pp. 63, 64, 67

Governance practices

• Governance can range from restrictions on accuracy or agency to process rules, moratoria, and bans, while policies may address data use, procurement, impact assessments, and jurisdiction.Governments also face decisions about who audits systems, when audits occur, and whether models should expire as conditions change; the pandemic illustrated how rapidly changing patterns can undermine prior data.Source: SQ11. HOW HAS AI AFFECTED… · pp. 66–67

AI and caregiving

• Intimate AI caregiving raises questions about privacy, compassion, trust, encoded ideas of well-being, automation of care labor, stereotypes, and whether technology displaces human responsibilities.Examples include companion robots that speak only in female voices and digital or robotic assistance during the COVID-19 demand for physical separation.Source: SQ11. HOW HAS AI AFFECTED… · pp. 67–68
• AI can support caregiving through diagnosis, wound care, home monitoring, mobility, and interventions for children with autism, while also offering assistance and companionship in aging and institutional care.Some patients may prefer robotic help for intimate activities when it protects privacy or dignity, though these tools can reshape caring relationships and may affect care infrastructure over time.Source: SQ11. HOW HAS AI AFFECTED… · p. 68

SQ11. HOW HAS AI AFFECTED SOCIOECONOMIC RELATIONSHIPS?

• Care technologies should supplement, not replace, human relationships; robots lack the contextual judgment and moral agency needed to meet many standards of genuine care.Care systems should respect patients’ independence and self-determination; artificial sentiment may be deceptive or manipulative, though some patients may prefer AI in particular situations.Source: SQ11. HOW HAS AI AFFECTED… · pp. 68–69
• Care AI can shift responsibility away from people and institutions, erode caregiving knowledge, and enable invasive monitoring that raises privacy and consent concerns.ElliQ lets relatives access a senior’s camera and activity sensors, potentially placing data control with family members rather than the older person.Source: SQ11. HOW HAS AI AFFECTED… · p. 69
• Care technologies need pre-market oversight and designs informed by caregivers and care receivers, with social values prioritized over market-led automation and inequities in care work addressed.The report urges regulators, developers, and funders to support better human care, and calls for designs sensitive to cross-cultural differences and potential biases.Source: SQ11. HOW HAS AI AFFECTED… · p. 70

CONCLUSIONS

• Governments should regulate influential AI applications, invest sustained resources, and build AI literacy in K–12 education, while researchers communicate risks and benefits without hype.The report also calls for clear communication between human and automated decision-makers and says AI’s success should be measured by how it empowers people.Source: CONCLUSIONS · p. 71

ANNOTATIONS ON THE 2016 REPORT

• The 2021 report broadens attention to geopolitical and security dimensions of AI governance and treats many social and ethical harms as present realities, not hypothetical risks.Compared with 2016, it also records national AI strategies and legislation, and recommends investment in federal data and computational resources.Source: ANNOTATIONS ON THE 2016 REPORT · p. 75
• The report highlights the scale and scope of national and international efforts addressing AI safety, ethical design, and deployment.It notes fewer anticipated public-safety benefits than the 2016 report, while emphasizing governance challenges.Source: ANNOTATIONS ON THE 2016 REPORT · p. 77
• The 2021 report cites worldwide government and corporate intrusions into citizens’ privacy and references emerging governance approaches.It validates the 2016 report’s concerns about privacy and other ethical and social issues associated with emerging technologies.Source: ANNOTATIONS ON THE 2016 REPORT · p. 77
• The 2021 panel gives less focus to removing impediments and more support to oversight and restrictions that limit societal harm.It also endorses spreading best practices and adapting regulatory regimes, as the 2016 report had acknowledged.Source: ANNOTATIONS ON THE 2016 REPORT · p. 78
• The report treats AI bias as a sociotechnical challenge shaped by human choices throughout system development, not just by data.It says technical solutions can address bias only partially.Source: ANNOTATIONS ON THE 2016 REPORT · p. 78
• The 2021 report more urgently emphasizes harms from invasive AI surveillance and its growing importance to international security.This contrasts with the 2016 report’s more balanced framing of public-safety applications and their potential pitfalls.Source: ANNOTATIONS ON THE 2016 REPORT · pp. 81–82
• The two reports broadly agree that smart regulation is appropriate, while uninformed regulation is harmful.The annotations also say discussion has shifted from preventing AI-related harms to mitigating harms that have emerged.Source: ANNOTATIONS ON THE 2016 REPORT · p. 82
• Engaging people affected by an AI system early in its design builds trust by making the system trustworthy rather than seeking trust after deployment.The annotations note that trust is less prominent in the 2021 report than in the 2016 report.Source: ANNOTATIONS ON THE 2016 REPORT · p. 82
• Concerns that strict regulation could exacerbate privacy harms have been missing from recent public conversations about technology and privacy.Source: ANNOTATIONS ON THE 2016 REPORT · p. 82