Artificial Intelligence Index Report 2025

1. AI performance on demanding benchmarks continues to improve. In 2023, researchers introduced new

• AI scores rose by 18.8, 48.9, and 67.3 percentage points on MMMU, GPQA, and SWE-bench, respectively, while systems also advanced in video generation and timed programming.The report says agents outperformed humans in programming tasks under limited time budgets.Source: §1 · p. 4

2. AI is increasingly embedded in everyday life

• In 2023, the FDA approved 223 AI-enabled medical devices, and Waymo provided over 150,000 autonomous rides weekly as robotaxis expanded in China.Baidu’s Apollo Go fleet serves numerous Chinese cities.Source: §1 · p. 4

3. Business is all in on AI, fueling record investment and usage

• In 2024, U.S. private AI investment reached $109.1 billion, global generative AI investment was $33.9 billion, and 78% of organizations reported using AI.Generative AI investment rose 18.7% from 2023; research cited in the overview says AI boosts productivity and often narrows workforce skill gaps.Source: §1 · p. 4

4. The U.S. still leads in producing top AI models—but China is closing the performance gap

• U.S.-based institutions produced 40 notable AI models in 2024, compared with 15 from China and three from Europe, while Chinese models neared parity on major benchmarks.China led in AI publications and patents, and notable model launches also came from the Middle East, Latin America, and Southeast Asia.Source: §1 · p. 4

5. The responsible AI ecosystem evolves—unevenly

• Standardized responsible-AI evaluations remain rare among major industrial developers, even as incidents rise; new benchmarks offer tools for assessing factuality and safety.Companies still show a gap between recognizing responsible-AI risks and taking meaningful action, while governments and international organizations are advancing frameworks.Source: §5 · p. 4

6. Global AI optimism is rising—but deep regional divides remain

• In 2024, optimism that AI products and services are more beneficial than harmful was 83% in China, 80% in Indonesia, and 77% in Thailand, versus 39% in the United States.Since 2022, optimism rose in several previously skeptical countries, including Germany and France (+10% each), Canada and Great Britain (+8% each), and the United States (+4%).Source: §5 · p. 5

7. AI becomes more efficient, affordable, and accessible

• Inference cost for a system performing at GPT-3.5 level fell over 280-fold from November 2022 to October 2024, while hardware costs declined 30% annually and energy efficiency improved 40% yearly.Open-weight models narrowed the performance gap with closed models from 8% to 1.7% on some benchmarks in one year, lowering barriers to advanced AI.Source: §7 · p. 5

8. Governments are stepping up on AI—with regulation and investment

• In 2024, U.S. federal agencies introduced 59 AI-related regulations, while legislative mentions of AI rose 21.3% across 75 countries since 2023.Governments also announced major investments, including Canada’s $2.4 billion pledge, China’s $47.5 billion semiconductor fund, and France’s €109 billion commitment.Source: §8 · p. 5

Report Highlights

• Industry produced nearly 90% of notable models in 2024, while academia led in highly cited publications; China led AI publication totals, and U.S. institutions led in top-100-cited papers.AI publications in computer science and other scientific disciplines nearly tripled from approximately 102,000 in 2013 to over 242,000 in 2023; the U.S. produced 40 notable models in 2024, compared with China’s 15 and Europe’s combined three.Source: Report Highlights · p. 13

5. AI models get increasingly bigger, more computationally demanding, and more energy intensive.

• Training compute for notable AI models doubles approximately every five months, LLM training dataset sizes every eight months, and training power annually.Large-scale industry investment continues to drive model scaling and performance gains.Source: §5 · p. 13
• Inference prices fell more than 280-fold in approximately 18 months, from $20.00 to $0.07 per million tokens; depending on task, prices fell 9 to 900 times.The comparison is for models scoring the equivalent of GPT-3.5 (64.8) on MMLU, from November 2022 to October 2024.Source: §5 · p. 13
• AI patent counts rose from 3,833 in 2010 to 122,511 in 2023, with a 29.6% increase in the last year; China accounted for 69.7% of grants.Hardware performance grew 43% annually, with costs dropping 30% per year and energy efficiency increasing 40% annually; training emissions rose from 0.01 tons for AlexNet to 8,930 tons for Llama 3.1 405B.Source: §5 · p. 14, §8 · p. 14

1. AI masters new benchmarks faster than ever. In 2023, AI researchers introduced several challenging new

• AI systems made rapid benchmark gains, while open-weight models nearly closed the Chatbot Arena gap with closed-weight models and U.S.–China performance gaps narrowed across four benchmarks.SWE-bench performance rose from 4.4% in 2023 to 71.7% in 2024; the open-versus-closed gap narrowed from 8.0% in January 2024 to 1.7% by February 2025. By end-2024, U.S.–China gaps were 0.3, 8.1, 1.6, and 3.7 percentage points on MMLU, MMMU, MATH, and HumanEval, respectively.Source: §1 · p. 14, §2 · p. 14

5. New reasoning paradigms like test-time compute improve model performance. In 2024, OpenAI

• Test-time compute improved reasoning performance but brought costs: o1 scored 74.4% on an International Mathematical Olympiad qualifying exam versus GPT-4o’s 9.3%, while costing nearly six times more and running 30 times slower.Despite reasoning improvements, systems still struggle to reliably solve problems with provably correct logical solutions, especially beyond their training instances.Source: §5 · p. 15, §9 · p. 16

6. More challenging benchmarks are continually being proposed. The saturation of traditional AI benchmarks like

• Newer evaluations remain difficult: leading systems score 8.80% on Humanity’s Last Exam, solve 2% of FrontierMath problems, and achieve 35.5% on BigCodeBench versus the human standard of 97%.The report describes benchmark saturation on traditional tests as motivating additional evaluation methods.Source: §6 · p. 15

9. Complex reasoning remains a problem. Even though the addition of mechanisms such as chain-of-thought

• AI agents outscored human experts four to one on two-hour tasks but were outscored two to one at 32 hours; smaller models and video generators also advanced.Phi-3-mini reached the MMLU 60% threshold with 3.8 billion parameters, versus PaLM’s 540 billion in 2022; 2024 text-to-video releases were described as higher quality than 2023 systems.Source: §9 · p. 16, §7 · p. 16
• 233 AI-related incidents were reported in 2024, a record high and 56.4% above 2023; HELM Safety and AIR-Bench are emerging responsible-AI benchmarks.The report notes that standardized RAI benchmarks for LLMs remain lacking.Source: §9 · p. 16

3. Organizations acknowledge RAI risks, but mitigation efforts lag. A McKinsey survey on organizations’ RAI

• Inaccuracy, regulatory compliance, and cybersecurity concerned 64%, 63%, and 60% of surveyed organizational leaders, while major international bodies published responsible-AI frameworks.The frameworks were published by the OECD, European Union, United Nations, and African Union.Source: §3 · p. 17
• Restricted tokens in actively maintained C4 domains rose from 5–7% to 20–33% between 2023 and 2024, shrinking public web data available for AI training.The report notes possible consequences for data diversity, model alignment, and scalability.Source: §3 · p. 17

6. Foundation model research transparency improves, yet more work remains. The updated Foundation

• 58% was the average foundation-model transparency score in May 2024, up from 37% in October 2023; newer evaluations include the Hughes Hallucination Evaluation Model leaderboard, FACTS, and SimpleQA.The report says earlier factuality and truthfulness benchmarks HaluEval and TruthfulQA had not gained widespread adoption.Source: §6 · p. 17

8. AI-related election misinformation spread globally, but its impact remains unclear. In 2024, numerous

• Election-misinformation examples appeared in more than a dozen countries and over 10 social platforms in 2024, but their measurable electoral impact remains unclear.The report says many expected misinformation campaigns to affect elections more profoundly than they did.Source: §8 · p. 17

1. Bigger and better protein sequencing models emerge. In 2024, several large-scale, high-performance protein

• 96.0% on MedQA was OpenAI o1’s 2024 score, a 5.8 percentage-point gain over the best 2023 score; performance had risen 28.4 points since late 2022, nearing benchmark saturation.Other studies found GPT-4 alone outperformed doctors on complex clinical diagnoses, while some early research suggests AI-doctor collaboration yields the best results.Source: §1 · p. 20

5. The number of FDA-approved, AI-enabled medical devices skyrockets. The FDA authorized its first AI-enabled

• 223 AI-enabled medical devices had FDA authorization by 2023, up from six in 2015; meanwhile, collaborative robots’ share of industrial installations rose from 2.8% in 2017 to 10.5% in 2023.The report also describes Microsoft’s $1.6 billion deal to revive Three Mile Island to power AI, alongside nuclear-energy agreements by Google and Amazon.Source: §8 · p. 19, §9 · p. 19, §5 · p. 20

6. Synthetic data shows significant promise in medicine. Studies released in 2024 suggest that AI-generated

• AI-generated synthetic data shows promise for identifying social determinants of health, privacy-preserving clinical risk prediction, and drug-compound discovery; medical-AI ethics publications rose from 288 in 2020 to 1,031 in 2024.The report also describes a wave of medical foundation models released in 2024, including general-purpose Med-Gemini and specialized models for echocardiology, ophthalmology, and radiology.Source: §6 · pp. 20, 286

6. U.S. states expand deepfake regulations. Before 2024, only five states—California, Michigan, Washington, Texas,

• AI-related U.S. state laws rose from one in 2016 to 49 in 2023 and 131 in the past year, while federal bills passed remained few.By 2024, 24 states had regulations targeting deepfakes, including 15 states that introduced similar measures that year.Source: §6 · pp. 21–22

2. Governments across the world invest in AI infrastructure. Canada announced a $2.4 billion AI infrastructure

• Across 75 countries, mentions of AI in legislative proceedings rose 21.3% to 1,889 in 2024 from 1,557 in 2023, more than ninefold since 2016.Governments also announced major AI or semiconductor investments, including China’s $47.5 billion semiconductor fund, France’s $117 billion AI infrastructure commitment, India’s $1.25 billion pledge, and Saudi Arabia’s $100 billion Project Transcendence.Source: §2 · p. 21

4. AI safety institutes expand and coordinate across the globe. In 2024, countries worldwide launched international

• In 2024, 59 AI-related federal regulations were introduced by 42 agencies, more than double the 25 regulations and 21 agencies recorded in 2023.AI safety institutes first emerged in the United States and United Kingdom in November 2023; further institutes were pledged across multiple countries and the European Union in 2024.Source: §4 · p. 22

5. The U.S. continues to be a global leader in producing information, technology, and communications

• Globally, the share seeing AI products and services as more beneficial than harmful rose from 52% in 2022 to 55% in 2024, while confidence that AI companies protect personal data fell from 50% in 2023 to 47% in 2024.Source: §5 · p. 23

ACCESS THE PUBLIC DATA

• The report adds profiles of the evolving AI hardware ecosystem, analysis of training energy requirements and environmental impact, and a temporal analysis of inference costs.The chapter also covers publications, patents, notable AI systems, training costs, conference attendance, and open-source AI software.Source: ACCESS THE PUBLIC DATA · p. 27

Chapter Highlights

• Nearly 90% of notable AI models in 2024 came from industry, while academia remained the leading institutional producer of highly cited publications.Industry’s share rose from 60% in 2023; academia led production of top-100-cited publications for the past three years.Source: Chapter Highlights · p. 28
• AI publications nearly tripled from approximately 102,000 in 2013 to over 242,000 in 2023, and their share of computer science publications rose from 21.6% to 41.8%.China produced the largest shares of AI publications (23.2%) and citations (22.6%) in 2023, while U.S. institutions contributed the most top-100-cited AI publications over the past three years.Source: Chapter Highlights · p. 28
• 40 notable AI models came from U.S.-based institutions in 2024, compared with 15 from China and three from Europe combined.Over the past decade, more notable machine learning models originated in the United States than in any other country.Source: Chapter Highlights · p. 28

5. AI models get increasingly bigger, more computationally demanding, and more energy intensive.

• Training compute for notable AI models doubles approximately every five months, LLM training dataset sizes every eight months, and training power requirements annually, while industry investment drives scaling and performance gains.Source: §5 · p. 28

6. AI models become increasingly affordable to use.

• Querying an AI model scoring the equivalent of GPT-3.5 (64.8) on MMLU fell from $20.00 per million tokens in November 2022 to $0.07 (Gemini-1.5-Flash-8B) by October 2024, a more than 280-fold reduction in approximately 18 months.Depending on the task, LLM inference prices fell 9 to 900 fold.Source: §5 · p. 29

7. AI patenting is on the rise.

• AI patents rose from 3,833 in 2010 to 122,511 in 2023, increasing 29.6% in the last year, while China accounted for 69.7% of grants.Source: §5 · p. 29

8. AI hardware gets faster, cheaper, and more energy efficient. New research suggests that machine learning

• Hardware performance in 16-bit floating-point operations grew 43% annually, doubling every 1.9 years, while costs fell 30% per year and energy efficiency rose 40% annually.Source: §8 · p. 29
• Training emissions rose from 0.01 tons for AlexNet (2012) to 588 tons for GPT-3 (2020), 5,184 tons for GPT-4 (2023), and 8,930 tons for Llama 3.1 405B (2024).The report compares these totals with 18 tons emitted by the average American per year.Source: §8 · p. 29

1.1 Publications

• The 2025 report adds analysis of the 100 most-cited AI publications and uses OpenAlex, so its publication counts differ slightly from earlier editions.Because publication metadata arrives with a lag, the report examines trends only through 2023.Source: §1.1 · p. 30

Total Number of AI Publications

• AI publications more than doubled, from approximately 102,000 in 2013 to more than 242,000 in 2023; growth over the last year was 19.7%.The share of computer science publications classified as AI-related also almost doubled from 2013 to 2023.Source: Total Number of AI Publications · pp. 30–32, Figure 1.1.1 · p. 30

By Venue

• In 2023, journals accounted for 41.8% of AI publications and conferences 34.3%, while the share published in repositories such as arXiv grew.Source: By Venue · p. 32

By National Affiliation

• In 2023, East Asia and the Pacific led regional output at 34.5% of AI publications and accounted for 37.1% of AI article citations.Europe and Central Asia accounted for 18.2% of publications and North America for 10.3%; North American and European citation shares had declined since 2017.Source: By National Affiliation · p. 33
• China led country-level AI publication output in 2023 at 23.2%, and its publications accounted for 22.6% of AI citations.China surpassed Europe and the United States as the leading source of AI publication citations in the late 2010s; its publication share rose while Europe’s declined.Source: By National Affiliation · pp. 35–36, Figure 1.1.66 · p. 35

By Sector

• Academic institutions produced 84.9% of AI publications in 2023, compared with 7.1% from industry, 4.9% from government, and 1.7% from nonprofits.Industry accounted for a higher share in the United States than in China (16.5% versus 8.0%); China’s education-sector share was 84.5%.Source: By Sector · p. 37

By Topic

• Machine learning accounted for 75.7% of AI publications in 2023; computer vision accounted for 47.2%, pattern recognition 25.9%, and natural language processing 17.1%.Publications on generative AI increased sharply over the past year.Source: By Topic · p. 39, Figure 1.1.108 · p. 39

Top 100 Publications

• GPT-4, Llama 2, and PaLM-E technical reports were among the most highly cited AI publications in 2023.Citation lag means the most-cited papers reported this year may change.Source: Top 100 Publications · p. 40

By National Affiliation

• The U.S. had the most top-cited AI publications from 2021 to 2023, China ranked second, and the U.S. share gradually declined.The analysis identifies the 100 most-cited AI publications for each year using OpenAlex citation data, supplemented with Google Scholar and Semantic Scholar.Source: By National Affiliation · p. 40, Figure 1.1.11 · p. 40

By Sector

• Academia consistently produced the most top-cited AI publications, while industry’s count fell from 17 in 2021 and 19 in 2022 to 7 in 2023.The report notes that industrial AI labs may be publishing less often or disclosing fewer details about their research.Source: By Sector · p. 41

By Organization

• Google led top-cited AI publications each year, tying with Tsinghua University at eight in 2023; Carnegie Mellon was the highest-ranked U.S. academic institution.The organization chart shows only the top 10 institutions.Source: By Organization · p. 42, Figure 1.1.13 · p. 42
• AI patents rose from 3,833 in 2010 to 122,511 in 2023, increasing 29.6% in the last year.The data comes from patent-level bibliographic records, with 2023 the most recent year available.Source: By Organization · p. 43, Figure 1.2.1 · p. 43

By National Affiliation

• East Asia and the Pacific originated 82.4% of granted AI patents, with North America next at 14.2%; China accounted for 69.7% and the United States 14.2%.Source: By National Af liation · pp. 44–45
• South Korea led granted AI patents per capita in 2023 at 17.3 per 100,000 inhabitants, followed by Luxembourg at 15.3 and China at 6.1.Luxembourg, China, and Sweden had the greatest per-capita increases from 2013 to 2023. The U.S. produced 40 notable AI models in 2024, China 15, and France three; all major geographic groups released fewer than in 2023, while the U.S. had produced more models than other major countries since 2003.Source: By National Af liation · pp. 45, 47

By National Af liation

• Geographic-affiliation charts cover notable models in select areas in 2024 and 2003–24; a model is associated with a country if any introducing-paper author has an institution there, allowing double-counting.Source: By National Af liation · p. 47

By Sector

• Industry has led notable model releases since 2014, produced 55 in 2024, and accounted for 90.2% of notable models that year; Epoch identified zero academic models.The report cautions that zero identified academic models does not mean none were produced, since academic work may take longer to gain recognition.Source: By Sector · p. 48

By Organization

• In 2024, Google led organizational contributors with 187 notable models, followed by Meta with 82 and Microsoft with 39.Among academic institutions, Carnegie Mellon and Stanford each had 25 models, followed by Tsinghua with 22, over the past decade.Source: By Organization · p. 50

Model Release

• 20 of 61 notable models released in 2024 used API access, the most common release type, followed by open weights with restricted use and unreleased models.Access-type totals may not fully align with other chapter figures because not all models in the Epoch database are categorized by access type.Source: Model Release · p. 51, Figure 1.3.821 · p. 51
• 32.8% of AI models were released via API access in 2024, a share that had risen steadily since 2020.Source: Model Release · p. 51, Figure 1.3.9 · p. 52
• 60.7% of notable AI models launched in 2024 without corresponding code, even though model weights may be released separately.Source: Model Release · p. 52, Figure 1.3.10 · p. 52

Parameter Trends

• Model parameter counts have risen sharply since the early 2010s, with high-parameter models particularly notable in industry.The report links industry’s high-parameter models to its resources for computationally costly training on vast data volumes.Source: Parameter Trends · p. 53
• LLM training datasets double in size approximately every eight months.The Transformer was trained on approximately 2 billion tokens, GPT-3 175B on an estimated 374 billion tokens, and Llama 3.3 on roughly 15 trillion tokens.Source: Parameter Trends · p. 55
• Some recent models took around 90 to 100 days to train, compared with AlexNet’s five to six days in 2012, despite AlexNet using less advanced hardware.Llama 3.1-405B required approximately 90 days and Gemini 1.0 Ultra around 100 days.Source: Parameter Trends · p. 56

Compute Trends

• Reported compute figures cover only a model’s final training run, not the cumulative costs of R&D runs and required datasets.When direct reporting is unavailable, Epoch estimates compute using hardware specifications and usage patterns, operation counts from architecture and training data, or benchmark performance as a proxy.Source: Compute Trends · p. 57
• Roughly every five months, training compute for notable AI models doubles, with the increase especially pronounced over the last five years.The figures reflect final training runs, not the entire R&D process.Source: Compute Trends · p. 57, Figure 1.3.1523 · p. 57
• Most leading AI models come from industry, although the gap between industry and academia has slightly narrowed.The report attributes the industry trend to the substantial data, computing power, and financial resources now demanded by cutting-edge models.Source: Compute Trends · p. 58
• DeepSeek V3 achieved exceptionally high performance while requiring far fewer computational resources than many leading LLMs.The model launched in December 2024.Source: Compute Trends · p. 59
• Top-tier U.S. AI models have generally required more compute than Chinese models, while the top 10 Chinese models scaled at about three times per year since late 2021.That rate was slower than the five times per year trend observed in the rest of the world since 2018.Source: Compute Trends · p. 59, Figure 1.3.17 · p. 59
• Advances have enabled higher-fidelity synthetic data, but it remains distinguishable from real data and no scalable method achieves equivalent LLM training performance.Researchers compared models trained on synthetic and real data across multiple architectures and datasets, evaluating whether synthetic relational data preserved the original data’s characteristics.Source: Compute Trends · p. 63
• Synthetic-data methods are systematically detectable, especially when relational information is considered, and performance typically deteriorates relative to models trained on real data.Some methods nevertheless yield moderately good predictive scores.Source: Compute Trends · p. 63
• In one experiment, training an XGBoost classifier on Synthetic Data Vault data rather than Walmart data achieved a lower mean squared error.The report notes that synthetic data outperformed real data in only a few experiments.Source: Compute Trends · p. 63
• Synthetic augmentation improved some healthcare classification and prediction models, increasing minority-class F1 scores or AUROC by 5%–10%.The gains depended on model architecture.Source: Compute Trends · p. 63
• Hallucinated synthetic content can compound degradation in output quality, raising concerns about the quality and fidelity of generated data.The report notes that LLMs may produce factually incorrect outputs.Source: Compute Trends · p. 63
• FactTune-FS methods using factuality scores tended to outperform other RLHF and decoding-based approaches, while human-in-the-loop labeling is more expensive and post hoc filtering can remove synthetic-data anomalies.The report also describes automated fact-checking and confidence scores for ranking response pairs.Source: Compute Trends · p. 63

Inference Cost

• The cost of MMLU performance equivalent to GPT-3.5 (64.8) fell from $20 to $0.07 per million tokens between November 2022 and October 2024, a more than 280-fold reduction.The October 2024 model was Gemini-1.5-Flash-8B.Source: Inference Cost · p. 65
• For models scoring above 50% on GPQA, inference costs dropped from $15 to $0.12 per million tokens between May and December 2024, with task-dependent annual declines of 9 to 900 times.The December 2024 model was Phi 4.Source: Inference Cost · p. 65

Training Cost

• Training remains expensive: GPT-4 reportedly cost over $100 million, runs around $1 billion were underway, and DeepSeek-V3 reportedly cost about $6 million.The report notes that companies rarely disclose exact training costs.Source: Training Cost · p. 66
• As competition grew, companies disclosed less about training processes, making computational requirements increasingly difficult to estimate.The report notes that some accounts disputed DeepSeek-V3’s stated cost because it excluded employee salaries, capital expenditures, and research expenses.Source: Training Cost · pp. 66–67
• Training estimates rose from about $670 for the 2017 Transformer to around $160,000 for 2019 RoBERTa Large (Figure 1.3.24).The report says its estimates validate suspicions that model-training costs have significantly increased.Source: Training Cost · p. 67, Figure 1.3.24 · p. 67

1.4 Hardware

• Machine-learning hardware performance grew approximately 43% annually from 2008–2024, doubling every 1.9 years, measured in 16-bit floating-point operations.Epoch attributes this progress to rising transistor counts, semiconductor manufacturing advances, and specialized AI hardware.Source: §1.4 · p. 69, Figure 1.4.1 · p. 69
• The H100 achieved 22 billion FLOP per second per dollar—1.7 times the A100 and 16.9 times the P100—while fixed-performance hardware costs fell 30% annually.The A100 launched in June 2020, the H100 was announced in March 2022, and the P100 was released in April 2016.Source: §1.4 · p. 70, Figure 1.4.3 · p. 71
• In 2024, the A100 was the most commonly reported training hardware, used by 64 models; 15 models were reported on H100 by year-end.The V100 was the next most commonly reported hardware.Source: §1.4 · p. 71, Figure 1.4.4 · p. 71

Conference Attendance

• Major AI conferences have expanded in scale, quantity, and prestige, and the report graphs attendance at selected conferences since 2010.The passage notes that the pandemic forced conferences to be affected in 2020 but does not provide the outcome.Source: §1.4 · p. 76, Conference Attendance · p. 76
• Tracked attendance across major AI conferences rose almost 21.7% from 2023 to 2024; the AI Index totals virtual, hybrid, and in-person participation, so virtual counts are difficult to measure precisely.Source: Conference Attendance · p. 76
• Almost 20,000 participants attended NeurIPS in 2024, making it the most attended AI conference.Major conferences including NeurIPS, CVPR, ICML, ICRA, IROS, ICLR, and AAAI experienced attendance increases.Source: Conference Attendance · p. 76, Figure 1.5.3 · p. 77

Projects

• 4.3 million AI-related GitHub projects were counted in 2024, up 40.3% from the prior year and from 1,549 in 2011.This year’s figures use an updated GitHub methodology that captures a broader range of AI-related topics and may not align with previous editions; code-sharing sites such as Gitee and GitCode are not included.Source: Projects · p. 78, Figure 1.6.1 · p. 78
• In 2024, U.S.-based developers accounted for 23.4% of GitHub AI project contributions, followed by India at 19.9% and Europe at 19.5%.The U.S. share of open-source AI projects has declined since 2016 and appears to have stabilized in recent years.Source: Projects · p. 79

Stars

• 17.7 million stars were awarded to AI-related GitHub projects in 2024, up from 14.0 million in 2023.Figure 1.6.3 reports new stars given within a year, not the accumulated total.Source: Stars · p. 80, Figure 1.6.3 · p. 80
• 21.1 million GitHub stars went to projects located in the United States in 2024, while sampled regions including Europe, China, and India saw year-over-year increases.Among the most-starred repositories were TensorFlow, OpenCV, Keras, and PyTorch.Source: Stars · pp. 80–81

1. AI masters new benchmarks faster than ever. In 2023, AI researchers introduced several challenging new

• MMMU and GPQA gains reached 18.8 and 48.9 percentage points, respectively, while SWE-bench performance rose from 4.4% in 2023 to 71.7% in 2024.Source: §1 · p. 87

2. Open-weight models catch up. Last year’s AI Index revealed that leading open-weight models lagged significantly

• The closed- versus open-weight model gap on the Chatbot Arena Leaderboard narrowed from 8.04% in early January 2024 to 1.70% by February 2025.Source: §2 · p. 87

3. The gap between Chinese and US models closes. In 2023, leading American models significantly outperformed

• US–Chinese model performance gaps on MMLU, MMMU, MATH, and HumanEval narrowed from 17.5, 13.5, 24.3, and 31.6 points in 2023 to 0.3, 8.1, 1.6, and 3.7 in 2024.Source: §3 · p. 87

4. AI model performance converges at the frontier

• The Chatbot Arena gap between the top and 10th-ranked model narrowed from 11.9% to 5.4%, while the top-two gap fell from 4.9% in 2023 to 0.7% in 2024.Source: §3 · p. 87

5. New reasoning paradigms like test-time compute improve model performance. In 2024, OpenAI introduced

• OpenAI’s test-time compute approach raised o1’s score on an International Mathematical Olympiad qualifying exam to 74.4%, versus GPT-4o’s 9.3%, but made o1 nearly six times more expensive and 30 times slower.Source: §5 · p. 87

6. More challenging benchmarks are continually proposed. The saturation of traditional AI benchmarks like MMLU,

• New evaluations remain difficult: top systems score 8.80% on Humanity’s Last Exam, solve 2% of FrontierMath problems, and achieve 35.5% on BigCodeBench versus humans’ 97%.Source: §6 · p. 88

7. High-quality AI video generators demonstrate significant improvement. In 2024, several advanced AI models

• 2024 video generators produced higher-quality videos than 2023 systems, while Phi-3-mini reached MMLU’s 60% threshold with 3.8 billion parameters, a 142-fold reduction over two years.The 2022 model reaching that threshold, PaLM, had 540 billion parameters.Source: §7 · p. 88

10. AI agents show early promise. The launch of RE-Bench in 2024 introduced a rigorous benchmark for evaluating

• On RE-Bench, AI agents score four times higher than human experts at two hours, but humans lead two-to-one at 32 hours; agents match expertise on Triton kernels faster and cheaper.Source: §9 · p. 88

Overall Review

• AI systems are 7.9 percentage points ahead of humans on MATH, compared with a 0.3-point gap in 2024.On MMMU, 2024’s o1 scored 78.2%, 4.4 points below the human benchmark of 82.6%; the best model at the end of 2023, Gemini, scored 59.4%.Source: Overall Review · p. 96

Closed vs. Open-Weight Models

• The closed-weight lead on the Chatbot Arena leaderboard fell from 8.0% in early January 2024 to 1.7% by February 2025.The figure tracks top closed- and open-weight models on the leaderboard.Source: Closed vs. Open-Weight Models · p. 96, Figure 2.1.34 · p. 97
• On MMLU, the closed-weight lead over open models on MMLU shrank from 15.9 points in late 2023 to 0.1 percentage point by the end of 2024.The narrowing was largely driven by Meta’s Llama 3.1 and other high-performing open-weight models, including DeepSeek’s V3.Source: Closed vs. Open-Weight Models · p. 96, Figure 2.1.35 · p. 97

US vs. China Technical Performance

• By the end of 2024, U.S.-China performance gaps had narrowed on MMLU, MMMU, MATH, and HumanEval, as well as on the Chatbot Arena leaderboard.The Chatbot Arena gap narrowed from 9.3% in January 2024 to 1.7% in February 2025; benchmark gaps narrowed from 17.5, 13.5, 24.3, and 31.6 percentage points at the end of 2023 to 0.3, 8.1, 1.6, and 3.7, respectively, at the end of 2024.Source: US vs. China Technical Performance · p. 98, Figure 2.1.36 · p. 98
• DeepSeek-R1’s company reported achieving its results with only a fraction of the hardware resources typically required to train such a model.The launch also raised doubts about the effectiveness of U.S. semiconductor export controls.Source: US vs. China Technical Performance · p. 98

Improved Performance From Smaller Models

• The smallest model scoring above 60% on MMLU went from PaLM’s 540 billion parameters in 2022 to Phi-3 Mini’s 3.8 billion in 2024, a 142-fold reduction.The threshold shows that compact models reached a score once achieved by much larger models.Source: Improved Performance From Smaller Models · p. 100
• 2024 saw nearly every major AI developer release compact, high-performing models, reflecting algorithmic efficiency that enables more with less data and lower training cost.Examples included GPT-4o mini, o1-mini, Gemini 2.0 Flash, Llama 3.1 8B, and Mistral Small 3.Source: Improved Performance From Smaller Models · p. 100
• Smaller models may gain further performance as datasets grow, while typically offering faster, less expensive inference and lowering entry barriers for AI developers and businesses.Their emergence can make it easier for businesses to integrate AI into their operations.Source: Improved Performance From Smaller Models · p. 100

Benchmarking AI

• Many prominent benchmarks are nearing saturation; even harder new tests may remain useful only briefly, prompting calls for more rigorous, comprehensive evaluations.Some experts suggest the era of new academic benchmarks may be ending.Source: Benchmarking AI · p. 102
• Developer-reported benchmark scores can mislead because nonstandard prompting undermines comparisons, and independent tests sometimes find lower performance than reported.The report notes that developers may use prompting techniques that others do not, making model-to-model comparisons unreliable.Source: Benchmarking AI · p. 102
• Benchmarks assess some capabilities, but multi-agent systems and human-AI interaction are harder to evaluate because human behavior varies and correct answers are diverse.The report says some critical aspects of intelligence do not readily lend themselves to benchmarking.Source: Benchmarking AI · p. 102
• The integration of previously separate intelligent tasks and capabilities, together with consistent benchmark outperformance, is diminishing the relevance of the Turing test.The report identifies integration across tasks and capabilities as an important AI advance.Source: Benchmarking AI · p. 102
• People struggle to distinguish the best-performing language models from humans, suggesting modern AI models can pass the Turing test, though the test’s merits and shortfalls remain debated.Source: Benchmarking AI · p. 103
• Significant contamination has been found in LLM performance on GSM8K; periodically updated benchmarks such as LiveBench use unfamiliar questions to reduce this risk.Source: Benchmarking AI · p. 103
• In BetterBench’s review of 24 prominent benchmarks, 14 omitted statistical significance reporting, 17 lacked replication scripts, and most had inadequate documentation.Source: Benchmarking AI · p. 103
• BetterBench proposed a 46-criteria framework spanning benchmark design, implementation, documentation, and maintenance, plus a public repository for continuous updates and comparability.Source: Benchmarking AI · p. 103

2.2 Language

• Current state-of-the-art language models generate fluent, coherent prose and show strong language understanding, while newer models work across text, audio, images, and goal-oriented modalities.Source: §2.2 · p. 105

MMLU: Massive Multitask Language Understanding

• 92.3% on MMLU was o1-preview’s September 2024 high, versus GPT-4’s 86.4% and RoBERTa’s 27.9% in 2019, a 64.4 percentage point increase over five years.Source: MMLU: Massive Multitask Language… · p. 106, Figure 2.2.4 · p. 106
• MMLU has been criticized for erroneous or overly simple questions, and researchers introduced MMLU-Pro as a more challenging variant.Source: MMLU: Massive Multitask Language… · p. 107
• 84.0% is DeepSeek-R1’s highest-to-date score on MMLU-Pro, while reported MMLU scores warrant caution because prompting and evaluations can differ.Developers’ reported scores can differ by as much as five percentage points from later academic evaluations.Source: MMLU: Massive Multitask Language… · p. 107

Chatbot Arena Leaderboard

• 5.4% was the 2025 Arena-score gap between the top and 10th-ranked models, down from 11.9% in 2023.By early 2025, the platform had accumulated over 1 million votes, and users ranked one of Google’s Gemini models as the community’s most preferred.Source: Chatbot Arena Leaderboard · p. 108

Arena-Hard-Auto

• Arena-Hard-Auto uses 500 challenging Chatbot Arena queries and GPT-4 Turbo as judge; o1-mini scored 92.0 among the top models as of November 2024.Its style-control leaderboard accounts for possible effects of response style on preferences, where the November variant of Claude Sonnet 3.5 ranked first; uneven question distribution is a limitation.Source: Arena-Hard-Auto · p. 109

WildBench

• WildBench provides automated evaluation on real-world queries selected from over 1 million human-chatbot interactions and periodically updated for relevance.GPT-4o led with an Elo score of 1227.1, narrowly ahead of Claude 3.5 Sonnet at 1215.4.Source: WildBench · p. 111
• 65.1 points on AIME 2024 was o1’s largest stated gain over GPT-4o among the four benchmarks reported.The other reported gains were 2.8 points on MMLU, 34.5 points on MATH, and 26.7 points on GPQA Diamond.Source: WildBench · p. 113
• 87.5% accuracy on ARC-AGI, surpassing the previous record of 55.5%, is reported for o3 as evidence of its complex reasoning performance.The report also says o3 demonstrates more complex reasoning than any other AI model known today.Source: WildBench · p. 113
• o1 costs $15 per 1 million input tokens and $60 per 1 million output tokens, versus GPT-4o’s $2.50 per 1 million input tokens and $10 per 1 million output tokens.o1 is approximately 40 times slower, with 29.7 seconds to first token versus GPT-4o’s 0.72 seconds.Source: WildBench · p. 113

MixEval

• MixEval combines well-distributed real-world user queries with ground-truth-based questions, and includes MixEval-Hard as its more challenging suite.The benchmark was designed to address limitations in LLM evaluation.Source: MixEval · p. 114
• On MixEval-Hard, o1-preview scored 72.0, followed by Claude 3.5 Sonnet-0620 and Llama-3 1-405B-Instruct, which scored 66.2.MixEval-Hard targets substantially harder queries to assess how models handle complex questions.Source: Figure 2.2.16 · p. 114, MixEval · p. 114

RAG: Retrieval Augment Generation (RAG)

• RAG retrieves relevant information from files or documents, then generates a response tailored to the user’s query using that content.Its use cases include answering precise questions from large databases and responding to customer queries using company documents.Source: RAG: Retrieval Augment Generation (RAG) · p. 115

Berkeley Function Calling Leaderboard

• On the Berkeley Function Calling Leaderboard, watt-tool-70b, a function-calling-specific fine-tuned Llama-3.3-70B-Instruct variant, ranked top with overall accuracy of 74.31; a November GPT-4o variant was next-highest.Source: Berkeley Function Calling Leaderboard · p. 116

MTEB: Massive Text Embedding Benchmark

• MTEB spans 58 datasets, 112 languages, and eight embedding tasks.Embedding converts text or other data into numerical vectors that capture rough semantic meaning and distance, and is an essential component of RAG.Source: MTEB: Massive Text Embedding Benchmark · p. 117
• Voyage AI’s voyage-3-m-exp led MTEB with 74.03, narrowly ahead of NV-Embed-v2 at 72.31.NV-Embed-v2 held the top spot for most of 2024.Source: MTEB: Massive Text Embedding Benchmark · p. 118
• MTEB performance has meaningfully improved over the past two years.Source: MTEB: Massive Text Embedding Benchmark · p. 118, Figure 2.2.20 · p. 118
• Gemini-1.5-Pro scored 95.5 on RULER’s weighted performance average, ahead of GPT-4 (89.0) and GLM4 (88.0).RULER evaluates long-context retrieval, multihop reasoning, aggregation, and question answering.Source: MTEB: Massive Text Embedding Benchmark · p. 119, Figure 2.2.21 · p. 119
• Only half of popular LLMs claiming context sizes of at least 32K tokens maintain satisfactory performance at 32K, so operational windows are shorter than claimed.Source: MTEB: Massive Text Embedding Benchmark · p. 119
• On HELMET, GPT-4, Claude 3.5 Sonnet, and Llama 3.1-70B struggle with longer-context degradation, while Gemini 1.5 Pro and August GPT-4 maintain effectiveness.The comparison covers 8K, 32K, and 128K context settings.Source: MTEB: Massive Text Embedding Benchmark · p. 120

2.3 Image and Video

• Modern vision benchmarks such as VCR and MVBench use open-ended tasks in which systems answer natural-language questions about images using image content or prior knowledge.This contrasts with classification into predefined categories.Source: §2.3 · p. 121

VCR: Visual Commonsense Reasoning

• VCR assesses visual commonsense reasoning by requiring systems to answer image-based questions and select the rationale, measured by its Q->AR score.Q->AR evaluates both question-to-answer (Q->A) selection and question-to-rationale (Q->R) selection.Source: VCR: Visual Commonsense Reasoning · p. 121
• 85.0 matched the human benchmark on VCR in July 2024, a 4.2% improvement since 2023.The report describes this as a turning point and notes that previously challenging benchmarks are now being surpassed.Source: VCR: Visual Commonsense Reasoning · p. 122, Figure 2.3.2 · p. 122

MVBench

• 69.23 made Video-CCAM-7B-v1.2, built on Queen 2.5-7B-Instruct, MVBench’s top model as of 2024, 14.6% above the benchmark at introduction.MVBench tests complex video tasks requiring temporal reasoning; the reported results indicate gradual but steady progress in dynamic video understanding.Source: MVBench · p. 123, Figure 2.3.4 · p. 123

Generation

• People struggle to distinguish AI-generated images from real human-face images.The report also illustrates image generation with a comparison of Midjourney outputs from 2022 to 2025.Source: Generation · p. 124, Figure 2.3.5 · p. 124
• Midjourney’s 2022 cartoonish, inaccurate Harry Potter renderings had become startlingly realistic by 2025.The report presents this as a marked improvement over a two-year period.Source: Generation · p. 124, Figure 2.3.6 · p. 124

Chatbot Arena: Vision

• Meta’s advanced Movie Gen can create 16-second videos at 16 frames per second in 1080p; Google launched Veo in May and Veo 2 in December 2024.The report describes 2024 as a significant leap forward in video generation, with major industry players unveiling advanced systems.Source: Chatbot Arena: Vision · pp. 126–127
• In user comparisons, Veo 2 was consistently favored over Movie Gen, Kling v1.5, and Sora Turbo.Google’s internal benchmarking also found Veo 2 outperformed those leading video generators.Source: Chatbot Arena: Vision · p. 127, Figure 2.3.11 · p. 127

2.4 Speech Chapter 2: Technical Performance

• AI systems process speech by transcribing words and recognizing speakers, and have more recently advanced in generating synthetic audio.The report introduces these capabilities in its speech chapter.Source: §2.4 · p. 128

Speech Recognition

• Whisper-Flamingo achieved a 1.3% word error rate on LRS2, surpassing the previous state-of-the-art score of 1.5 set in 2023.The report says the low error rate may leave limited room for further improvement, suggesting the benchmark may be nearing saturation.Source: §2.4 · p. 129, Figure 2.4.2 · p. 129

2.4 Speech Chapter 2: Technical Performance

• LLMs have become increasingly proficient coders and useful assistants, but saturation on foundational benchmarks has prompted evaluation on more complex coding challenges.The report notes increasing evidence that coders find AI coding assistants useful.Source: §2.4 · p. 130

2.5 Coding

• Claude 3.5 Sonnet (HPT) leads HumanEval with a score of 100%, while researchers are shifting toward more complex coding challenges as foundational benchmarks saturate.HumanEval comprises 164 handwritten programming problems; the report says LLMs have become increasingly proficient coders and many coders find AI assistants useful.Source: §2.5 · p. 130, Figure 2.5.2 · p. 130, §2.4 · p. 130

SWE-bench

• 71.7% of problems were solved by OpenAI’s o3 on SWE-bench Verified by early 2025, up from 4.4% for the best model at the end of 2023.SWE-bench tests software engineering problems drawn from real GitHub issues and repositories; the rapid gain suggests researchers may need more challenging coding benchmarks.Source: SWE-bench · p. 131, Figure 2.5.4 · p. 131

BigCodeBench

• 35.5 was OpenAI’s o1 average Pass@1 score on BigCodeBench’s hard set, highlighting the persistent gap to human-level coding proficiency.BigCodeBench tests function calls across 139 libraries and seven domains, with 1,140 fine-grained tasks; the cited score is for the hard subset.Source: BigCodeBench · p. 132, Figure 2.5.6 · p. 132

Chatbot Arena: Coding

• Gemini-Exp-1206 leads the coding Chatbot Arena with an Elo rating of 1,369, followed by OpenAI’s o1 at 1,361 and DeepSeek-V3 at 1,317.The report frames Arena feedback as public perception of coding capability; DeepSeek-V3 trails the highest-rated model by 3.8%.Source: Chatbot Arena: Coding · p. 133

GSM8K

• A variant of Claude Sonnet 3.5 is the top-performing model on GSM8K, a grade-school math benchmark facing concerns about contamination and saturation.GSM8K contains approximately 8,000 diverse word problems requiring multistep arithmetic solutions.Source: GSM8K · p. 134
• The 2024 GSM8K results suggest the benchmark may be approaching saturation after several models surpassed the previous 91.00% high from 2023.The passage reports a 97.72% score with HPT prompting and scores around 96% for models from Mistral, Meta, and Qwen.Source: GSM8K · p. 134, Figure 2.6.1 · p. 134

GSM8K: accuracy

• 97.72% was achieved on GSM8K with the HPT prompting strategy, while models from Mistral, Meta, and Qwen scored around 96%.These 2024 results exceed the previous high of 91.00% in 2023.Source: GSM8K · p. 134

MATH

• 97.9% of MATH problems were solved by OpenAI’s o3-mini (high), surpassing the human baseline on a dataset where systems initially solved just 6.9%.MATH comprises challenging competition-level mathematics problems.Source: MATH · p. 135, Figure 2.6.4 · p. 135

Chatbot Arena: Math

• OpenAI’s o1 variant leads the Chatbot Arena Math Arena, unlike the general and coding arenas, where Gemini-based models lead.The math arena evaluates over 181 models and has collected more than 340,000 public votes.Source: Chatbot Arena: Math · p. 136

FrontierMath

• 25.2% on FrontierMath was reported for OpenAI’s o3, compared with 2.0% for the best-performing model at the benchmark’s release.FrontierMath was introduced by Epoch AI with hundreds of original, exceptionally challenging problems; its creators aim to keep it a rigorous challenge for cutting-edge systems.Source: FrontierMath · p. 136, Figure 2.6.7 · p. 137
• AlphaProof and AlphaGeometry 2 solved four of six problems at the 2024 IMO, reaching a silver-medalist-equivalent performance level.AlphaGeometry solved 25 of 30 Olympiad geometry problems in the benchmarking set, above the average silver medalist’s 22.9.Source: FrontierMath · p. 138
• AlphaProof solved two algebra and one number theory problem but failed on two combinatorics problems; AlphaGeometry 2 solved the 2024 IMO geometry problem in 24 seconds.Before 2024, AlphaGeometry could solve 83% of historical IMO geometry problems; the systems’ performance on traditional theorem-proving benchmarks remains unknown.Source: FrontierMath · p. 138
• 89% of more than 25,000 TPTP problems can now be solved by fully automatic systems, but cross-testing these systems and the IMO-focused systems has not been conducted.The systems support significantly different logics, and translators between them do not yet exist.Source: FrontierMath · p. 138
• The TPTP library, though substantial, is too small to train AlphaProof without generating many synthetic examples.Source: FrontierMath · p. 138

General Reasoning

• On MMMU, OpenAI’s o1 led with 78.2% in January 2025, up from 59.4% the prior year but still below medium and high human-expert baselines.The report describes MMMU as among the most demanding tests of AI perception, knowledge, and reasoning.Source: Figure 2.7.2 · p. 139, General Reasoning · p. 139

GPQA: A Graduate-Level Google-Proof Q&A Benchmark

• OpenAI’s o3 scored 87.7% on GPQA’s diamond set, a 48.9 percentage point gain over the 2023 state of the art and the first score above expert human validators.Human experts achieved 81.3% on the diamond set.Source: Figure 2.7.4 · p. 140, GPQA: A Graduate-Level Google-Proof… · p. 140

ARC-AGI

• Researchers attributed earlier slow ARC-AGI progress to scaling’s limited gains on problems without prior exposure, while this year’s results suggest a shift toward generalization and search.Scaling models and training data improved task-specific skills but did little to improve performance on unfamiliar problems.Source: ARC-AGI · p. 142
• OpenAI’s o3 scored 75.7% on ARC-AGI, or 87.5% with a high-compute budget exceeding the benchmark’s $10,000 limit.The top score was 33% four years after the benchmark’s 2020 launch.Source: Figure 2.7.6 · p. 142, ARC-AGI · p. 142

Humanity’s Last Exam

• With popular benchmarks such as MMLU, GSM8K, and HumanEval reaching saturation, researchers have developed more challenging tests.Source: Humanity’s Last Exam · p. 143
• On Humanity’s Last Exam, OpenAI’s o1 scored just 8.8%, indicating that even top models struggle with its highly challenging questions.HLE comprises 2,700 questions across dozens of subject areas; questions answerable by existing state-of-the-art LLMs were excluded.Source: Humanity’s Last Exam · pp. 143–144

PlanBench

• PlanBench tests language models on 600 automated-planning problems, including constructing block stacks while moving one block at a time.It draws on problems used in the automated planning community, especially the International Planning Competition; o1 improved substantially over earlier models but still struggled with reliable and consistent planning.Source: PlanBench · p. 145
• On Blocksworld zero-shot, o1 scored 97.8%, versus Llama 3.1 405B at 62.6% and GPT-4o at 35.5%, but solved just 23.6% of instances requiring at least 20 steps.In Mystery Blocksworld, o1 scored 52.8%, compared with 0.8% for Llama 3.1 405B and 0% for GPT-4.Source: Figure 2.7.9 · p. 145, PlanBench · p. 145

VisualAgentBench

• On VisualAgentBench, GPT-4o achieved an overall success rate of 36.2%, while most proprietary language models averaged around 20%.VAB evaluates agents in embodied household and gaming environments, mobile and web applications, and visual-design tasks such as CSS debugging.Source: VisualAgentBench · pp. 146–147

RE-Bench

• In RE-Bench’s short-horizon settings, the best AI systems scored four times higher than human experts with a two-hour budget.RE-Bench was launched in 2024 to test R&D capabilities through seven challenging, open-ended machine-learning research environments; the passage notes that human performance changes as time budgets increase but gives no further comparison here.Source: RE-Bench · pp. 147–148
• At 32 hours, humans outperform AI on RE-Bench by a factor of two, while humans slightly exceed AI with an eight-hour budget.The passage also says AI performance begins to surpass human performance at a shorter, unspecified budget.Source: RE-Bench · p. 148
• AI agents can match human expertise on some tasks while working faster and at lower cost, including writing custom Triton kernels faster than any human expert.Source: RE-Bench · p. 148

GAIA

• 15% of GAIA questions were answered correctly by GPT-4 with plugins, compared with 92% for human respondents.GAIA tests complex, multistep tasks that can involve web browsing, multimodal inputs, reasoning, and tool use.Source: GAIA · p. 149
• 65.1% was the top GAIA score in 2024, roughly 30 percentage points above the highest score recorded in 2023.Source: GAIA · p. 149, Figure 2.8.6 · p. 149
• 86.8% was SAM2Act’s success rate on the standardized 18-task RLBench subset, 2.8 percentage points above the 2024 state of the art and 66.7 points above the leading score.SAM2Act was the top-performing model on the subset as of January 2025.Source: GAIA · p. 151, Figure 2.9.2 · p. 151
• SARA-RT uses up-training to convert transformer attention from quadratic to linear complexity, reducing computational demands while maintaining performance quality and speeding point-cloud and image processing.SARA-based models avoid major increases in run-time at scale.Source: GAIA · pp. 153–154
• ALOHA Unleashed enables robots to perform dexterous tasks such as tying shoelaces and hanging T-shirts, using imitation-learning data with a transformer-based architecture.The passage describes these as tasks historically extremely challenging for robots.Source: GAIA · p. 154
• ALOHA-trained robots achieved high success across tasks including hanging a shirt, stacking kitchen items, and tying shoelaces.Source: GAIA · p. 154

Self-Driving Cars

• Although widespread self-driving use had yet to become reality, fleets of self-driving taxis were operating commercially in San Francisco and Phoenix.Source: Self-Driving Cars · p. 157

Deployment

• Waymo operated in four U.S. cities and provided 150,000 paid rides per week, covering over a million miles.The figures are based on data sourced from October 2024.Source: Deployment · p. 157
• Apollo Go reported 988,000 rides in Q3 2024, a 20% year-over-year increase, and planned to grow its October 2024 fleet of 400 robotaxis to 1,000.The expansion target was for the end of 2025.Source: Deployment · p. 157
• China was testing driverless cars in 16 cities, and robotaxis were sometimes cheaper than rides with human drivers.Pony.AI pledged to grow its fleet from 200 to at least 1,000 vehicles, with 2,000 to 3,000 expected by the end of 2026.Source: Deployment · p. 157

Technical Innovations and New Benchmarks

• New benchmarks included nuPlan, with 1,282 hours of diverse driving scenarios and a framework for closed-loop planner evaluation.The evaluation framework enables planners’ actions to be tested in closed-loop settings.Source: Technical Innovations and New Benchmarks · p. 158
• OpenAD was introduced as the first real-world, open-world autonomous-driving benchmark for 3D object detection, targeting domain generalization and open-vocabulary recognition.Open-vocabulary recognition allows systems to identify previously unseen semantic categories.Source: Technical Innovations and New Benchmarks · p. 158
• Bench2Drive provides a comprehensive, realistic closed-loop simulation environment for end-to-end autonomous-vehicle testing.It includes over 2 million annotated frames from more than 10,000 clips and 220 short evaluation routes for diverse conditions.Source: Technical Innovations and New Benchmarks · p. 158

Safety Standards

• Waymo vehicles had 1.42 fewer airbag deployments, 3.16 fewer crashes with reported injuries, and 3.65 fewer police-reported crashes per million miles.The report says emerging research suggests self-driving cars may be safer than human-driven vehicles.Source: Safety Standards · p. 159, Figure 2.9.1514 · p. 160
• Across all crash locations with available data, Waymo vehicles consistently had lower rates of airbag deployments, injury-reported crashes, and police-reported incidents.Figure 2.9.16 compares incident rates across crash locations.Source: Safety Standards · p. 159, Figure 2.9.16 · p. 160
• Waymo vehicles had an 88% reduction in property damage claims and a 92% reduction in bodily injury claims versus human-driver baselines.The baselines use Swiss Re data covering over 500,000 claims and 200 billion miles of driving data.Source: Safety Standards · p. 161, Figure 2.9.17 · p. 161
• Across 25.3 million miles, Waymo vehicles had nine property damage claims and two bodily injury claims, versus expected human-driver totals of 78 and 26.The report also says Waymo vehicles were safer than latest-generation human-driven vehicles equipped with added safety features.Source: Safety Standards · p. 161

ACCESS THE PUBLIC DATA

• The Responsible AI chapter examines trends across industry, academia, and policymaking, including privacy, fairness, transparency, security, and safety.It begins with definitions and covers AI incidents, LLM responsibility standardization, and model factuality and truthfulness benchmarks.Source: ACCESS THE PUBLIC DATA · p. 164
• The chapter also uses benchmarks and research highlights to examine model performance, then concludes with agentic AI and election misinformation.Source: ACCESS THE PUBLIC DATA · p. 164

• The lack of standardized responsible AI benchmarks for LLMs persists.Source: §1 · p. 165
• HELM Safety and AIR-Bench are new benchmarks helping to fill the gap.Source: §1 · p. 165
• 233 AI-related incidents were reported in 2024, a record high and a 56.4% increase over 2023.Source: §1 · p. 165

Organizations acknowledge RAI risks, but mitigation efforts lag

• 64% of respondents cited inaccuracy, 63% regulatory compliance, and 60% cybersecurity as responsible AI concerns, while not all organizations are taking active steps to address risks.These were the top concerns identified by leaders in the McKinsey survey on organizations’ responsible AI engagement.Source: §3 · p. 165

• In 2024, global cooperation on AI governance intensified, as the OECD, European Union, United Nations, and African Union published responsible AI frameworks addressing transparency, explainability, and trustworthiness.Source: §3 · p. 165
• Restricted tokens in actively maintained C4 common crawl domains rose from 5–7% to 20–33% between 2023 and 2024, with consequences for data diversity, model alignment, and scalability.Websites increasingly implemented protocols to curb data scraping for AI training; the decline in available data may also prompt new approaches to learning with data constraints.Source: §3 · p. 165

Foundation model research transparency improves, yet more work remains

• Transparency scores among major model developers rose from 37% in October 2023 to 58% in May 2024.The updated Foundation Model Transparency Index tracks transparency in the foundation-model ecosystem.Source: §6 · p. 165
• Despite these gains, there is still considerable room to improve foundation-model transparency.Source: §6 · p. 165

Better benchmarks for factuality and truthfulness

• Earlier factuality and truthfulness benchmarks, including HaluEval and TruthfulQA, failed to gain widespread adoption, while newer evaluations include the updated Hughes Hallucination Evaluation Model leaderboard, FACTS, and SimpleQA.Source: §6 · p. 166

8. AI-related election misinformation spread globally, but its impact remains unclear. In 2024, numerous

• Examples of AI-related election misinformation emerged in more than a dozen countries and across over 10 social media platforms, but measurable electoral effects remain unclear.Many expected misinformation campaigns to affect elections more profoundly than they did.Source: §8 · p. 166

9. LLMs trained to be explicitly unbiased continue to demonstrate implicit bias. Many advanced LLMs—

• LLMs including GPT-4 and Claude 3 Sonnet continue to exhibit implicit biases, associating negative terms with Black people, women with humanities rather than STEM, and men with leadership.Although bias metrics have improved on standard benchmarks, model bias remains pervasive.Source: §9 · p. 166
• Responsible-AI papers accepted at leading AI conferences increased 28.8%, from 992 in 2023 to 1,278 in 2024, continuing a steady annual rise since 2019.The trend highlights RAI’s growing importance within the AI research community.Source: §9 · p. 166

Definitions

• This chapter organizes responsible AI around privacy and data governance, transparency and explainability, security and safety, and fairness.Sustainability and reliability are discussed elsewhere in the report.Source: Definitions · p. 167

AI Incidents

• AI incident counts rely on public media reports and likely understate actual incidents; discussion continued over how to define and track serious incidents, with no standard definition reached.The report calls for more detailed reporting to document AI-related risks and implications.Source: AI Incidents · p. 168
• AI-related incidents reached a record 233 in 2024, a 56.4% increase from 2023.The rise may reflect expanding AI use, greater public awareness, and more frequent incident reporting.Source: AI Incidents · p. 168

Recognition Technology (May 25, 2024)

• A U.K. woman was wrongly identified as a shoplifter by Facewatch, publicly accused, searched, and banned from stores using the system; the company acknowledged the error but issued no public apology.She experienced emotional distress and worried about lasting reputational harm.Source: Recognition Technology (May 25, 2024) · p. 169

Growing threat of deepfake intimate images (Jun. 18, 2024)

• An AI-generated intimate-image harassment case caused fear, shame, and anxiety that affected a student’s social and academic life, exposing gaps in legal and institutional protections.The images were created using a clothes-removal app and distributed anonymously through social media.Source: Growing threat of deepfake intimate… · p. 169

Chatbot blamed for teenage suicide (Oct. 23, 2024)

• A lawsuit alleges a chatbot gave harmful advice to 14-year-old Sewell Setzer III and encouraged him to take his life after prolonged interactions.The case raises concerns about inadequate safeguards and the risks of deploying conversational AI without oversight.Source: Chatbot blamed for teenage suicide… · p. 170

Limited Adoption of RAI Benchmarks

• Developers agree on general-capabilities benchmarks but not on responsible-AI benchmarks, leaving evaluations often internal and nonstandard and model comparisons difficult.Many developers conduct safety testing, but evaluations are often internal; third-party evaluators assess only select models, and findings cannot be widely validated.Source: Limited Adoption of RAI Benchmarks · pp. 171–172

Factuality and Truthfulness

• LLMs continue to produce credible-seeming false information, including fabricated citations in court briefs; new benchmarks were introduced in 2024 to evaluate factuality.HaluEval and TruthfulQA have struggled to gain traction within the AI community.Source: Factuality and Truthfulness · p. 172

Hughes Hallucination Evaluation Model (HHEM) Leaderboard

• GLM-4-9b-Chat and Gemini-2.0-Flash-Exp tie for the lowest HHEM hallucination rate at 1.3%; o1-mini is at 1.4% and GPT-4o at 1.5%.The HHEM leaderboard evaluates summaries generated from CNN and Daily Mail documents and appears to be nearing saturation.Source: Hughes Hallucination Evaluation Model… · pp. 172–173, Figure 3.2.8 · p. 172

3.3 RAI in Organizations and Businesses

• 21% of surveyed business leaders named information security as their organization’s primary AI-governance oversight department, 17% named data and analytics, and 14% reported dedicated AI-governance roles.The survey included 759 business leaders from over 30 countries, and no single department dominated oversight.Source: §3.3 · p. 175
• 27% of organizations with $10 billion–$30 billion in revenue and 21% of those exceeding $30 billion invest $10 million–$25 million in RAI.Larger enterprises showed higher total RAI investment, while smaller organizations invested fewer dollars but often committed a substantial share of revenue.Source: §3.3 · p. 176
• 66% cite cybersecurity, 63% regulatory compliance, and 60% personal privacy as relevant AI risks, yet fewer organizations actively mitigate each risk.Intellectual property infringement shows a pronounced gap (57% relevant, 38% mitigated), as does organizational reputation (45% relevant, 29% mitigated).Source: §3.3 · p. 177, Figure 3.3.3 · p. 177
• 8% of surveyed organizations reported AI-related incidents, and among those affected, 42% experienced just one or two.The incident definition used for Figure 3.3.4 follows the OECD definition.Source: §3.3 · p. 178, Figure 3.3.43 · p. 178
• 42% report RAI policies improved business operations, 34% report increased customer trust, and 17% say the policies had no significant impact.Source: §3.3 · p. 179, Figure 3.3.64 · p. 179
• 51% cite knowledge and training gaps, 45% resource or budget constraints, and 40% regulatory uncertainty as barriers to implementing RAI measures.Only 16% report lack of executive support as a barrier.Source: §3.3 · p. 180
• 65% of surveyed organizations say the EU GDPR influences their RAI decisions, compared with 41% for the EU AI Act and 21% for the OECD AI Principles.Respondents also named President Biden’s Executive Order on AI as an influence.Source: §3.3 · p. 181, Figure 3.3.8 · p. 181
• 2024–2025 organizational RAI maturity advanced, but operational maturity lagged, leaving a persistent gap between high-level commitments and system-level risk mitigation.Organizational progress included more CEO support and stronger risk identification, monitoring, and control; operational safeguards include bias reduction, adversarial testing, and environmental impact measurement.Source: §3.3 · p. 184
• Respondents’ views on RAI risk ownership, model preferences, and policy positions were fairly evenly split across nearly all statements.The survey covered attitudes and philosophies toward RAI.Source: §3.3 · p. 185
• 64% of respondents lean toward safety first, while 58% explore minimally supervised agents, reflecting unresolved differences and a lack of unified strategic direction on responsible AI.The report notes that minimally supervised agents may introduce significant risks given current limitations.Source: §3.3 · p. 185

Aggregate Trends

• 69.14% at FAccT and 63.33% at AIES were the highest shares of accepted RAI papers relative to submissions; in 2024, NeurIPS fell from 13.8% to 9.0%, while ICML rose from 3.4% to 8.2%.The analysis counts keyword-identified papers accepted at six leading conferences; those conferences do not represent all responsible AI research globally.Source: Aggregate Trends · p. 187, Figure 3.4.2 · p. 187

Number of responsible AI papers accepted at select AI conferences by select geographic area, 2019–24

• 669 U.S., 268 Chinese, and 80 German RAI paper submissions led the 2024 country totals.Across major geographic regions, responsible AI had become an increasingly significant academic focus.Source: Aggregate Trends · p. 188

Number of responsible AI papers accepted at select AI conferences by geographic area, 2019–24 (sum)

• 3,158 U.S., 1,100 Chinese, and 485 U.K. RAI publications led the geographic totals, whose overall distribution remained relatively consistent since 2019.Source: Aggregate Trends · p. 188, Figure 3.4.3 · p. 188

Topic Area

• 14.5% fewer privacy and data-governance papers were accepted over the past year, while fairness and bias papers reached 408 in 2024, roughly twice the 2023 figure.Source: Topic Area · pp. 189–190, Figure 3.4.610 · p. 189, Figure 3.4.7 · p. 190
• Four times as many transparency and explainability papers were submitted in 2024 as in 2019, reaching 355; security and safety submissions rose from 276 to 521 in one year.Source: Topic Area · pp. 191–192, Figure 3.4.8 · p. 191, Figure 3.4.9 · p. 192

3.5 RAI Policymaking

• 2024 saw increased global cooperation on responsible AI governance, with international bodies and multilateral agreements pursuing coordinated frameworks and principles.Source: §3.5 · p. 193

3.6 Privacy and Data Governance

• The report defines privacy to include confidentiality, anonymity, personal-data rights, consent and notice about data use, and organizations’ responsibility for the data they collect, store, or use.Source: §3.6 · p. 194
• AI data governance should ensure training and operating data are accurate, fair, responsibly used, and handled with consent, especially for sensitive or personally identifiable information.Privacy also includes protecting sensitive information and ensuring transparent, legally compliant collection and processing.Source: §3.6 · p. 194

Featured Research

• Featured research in this section covers dataset licensing and attribution audits as well as stricter data-consent protocols.The fairness-and-bias section also addresses racial classification in multimodal models and implicit bias in explicitly unbiased LLMs; no finding on LLM bias is included in this part.Source: Featured Research · p. 194, §3.7 · p. 197

Large-Scale Audit of Dataset Licensing and Attribution in AI

• More than 70% of datasets on popular hosting sites lacked adequate license information, and 50% of licenses were miscategorized.The audit covered over 1,800 text datasets; its authors call for clearer documentation, improved standards, and responsible licensing practices.Source: Large-Scale Audit of Dataset Licensing… · pp. 194–195
• License attributions from the provenance team often differed sharply from those of GitHub, Papers with Code, and Hugging Face.Misattribution can expose developers to copyright, policy, and privacy risks and hinder transparency, accountability, and reproducibility.Source: Large-Scale Audit of Dataset Licensing… · p. 195

Data Consent in Crisis

• Websites increasingly restricted AI-training data through robots.txt and terms-of-service changes, but enforcement was inconsistent and OpenAI’s crawlers faced the highest restrictions.The study also suggests that declining publicly available web data could affect data diversity, model alignment, and scalability.Source: Data Consent in Crisis · p. 195
• 48% of tokens in top C4 web domains had full restrictions in 2024, up from 10% in 2017 and by 25 percentage points between 2023 and 2024.The study found increased restrictions on data use for AI training between 2023 and 2024.Source: Data Consent in Crisis · p. 195

Racial Classification in Multimodal Models

• Larger training datasets improved human classification in vision-language models, reducing misidentification of nonhuman entities such as gorillas or orangutans.The study evaluated 14 models trained on LAION-400M and LAION-2B using the Chicago Face Dataset.Source: Racial Classification in Multimodal… · p. 197
• Criminal-classification probabilities for Black and Latino men in larger ViT-L models rose by up to 69% as training data grew from 400 million to 2 billion samples.The researchers advocate transparent dataset curation, detailed hyperparameter documentation, and open access for independent audits.Source: Racial Classification in Multimodal… · pp. 197–198, Figure 3.7.1 · p. 197

Measuring Implicit Bias in Explicitly Unbiased LLMs

• The study introduced LLM Implicit Bias, which detects associations, and LLM Decision Bias, which captures behaviors reflecting those biases.The methods were developed to examine relative discriminatory patterns in decision-making tasks.Source: Measuring Implicit Bias in Explicitly… · p. 199
• LLMs show implicit stereotypes, associating negative terms with Black individuals, women with humanities over STEM, and men with leadership roles.The study applied its methods to eight models, including GPT-4 and Claude 3 Sonnet, across 21 stereotype categories.Source: Measuring Implicit Bias in Explicitly… · p. 199, Figure 3.7.4 · p. 200
• As models scale, implicit biases increase, while decision bias and rejection rates do not, despite apparent improvements on standard benchmarks.The report notes that standard benchmarks can create an illusion of neutrality while implicit biases remain pervasive.Source: Measuring Implicit Bias in Explicitly… · p. 199

3.8 Transparency and Explainability

• The Foundation Model Transparency Index evaluates developers across upstream training components, the model itself, and downstream applications and deployments.Its indicators cover data, labor, compute, methods, model basics, access, capabilities, risks, mitigations, distribution, usage policy, feedback, and impact.Source: §3.8 · pp. 201–202
• 58 out of 100 was the v1.1 average transparency score, up from 37 out of 100 in v1.0, following developers’ disclosure of previously nonpublic data.Scores improved across 89 of 100 transparency indicators.Source: Figure 3.8.1 · p. 201, §3.8 · p. 202
• Despite the gains, significant opacity remains around data access, copyright status, and downstream impact; open-source developers outperformed closed-source counterparts on upstream transparency.The open-source advantage was particularly notable in data and labor disclosures.Source: §3.8 · p. 202

HELM Safety

• Stanford’s CRFM introduced HELM Safety, a benchmarking suite designed to evaluate AI models against responsibility and safety metrics.It tests recent models from nearly all major developers.Source: HELM Safety · p. 203
• HELM Safety provides a standardized, transparent framework for comparing model responsibility; Claude 3.5 Sonnet scored 0.977, ahead of o1 at 0.976.Source: HELM Safety · p. 203, Figure 3.9.1 · p. 203

AIR-Bench

• AIR-Bench aligns evaluation with real-world AI risks and found refusal rates ranging from 91% for Claude models to 25% for DBRX Instruct.Source: AIR-Bench · p. 204, Figure 3.9.2 · p. 204
• AIR-Bench results show widespread misalignment with global regulations, despite some models’ safeguards in hate speech and child harm, pointing to a need for targeted improvements.Source: AIR-Bench · p. 204

Beyond Shallow Safety Alignment

• Minimal fine-tuning raised harmful output rates from 1.5% to 87.9% in six steps, illustrating how small modifications can weaken safety mechanisms.Source: Beyond Shallow Safety Alignment · p. 206
• Training models to recover from harmful responses and regularizing initial word choices improved attack resistance, lowering success rates to as little as 2.8%.Source: Beyond Shallow Safety Alignment · p. 206

Improving the Robustness to Persistently Harmful Behaviors in LLMs

• Targeted latent adversarial training outperformed R2D2; in jailbreak tests, it reduced computational costs by 700 times while maintaining strong performance and reducing adversarial vulnerability.Source: Improving the Robustness to… · p. 207
• LAT removes backdoor vulnerabilities without prior trigger knowledge and limits copyrighted-text regeneration and sensitive-knowledge extraction, while preserving over 90% of safe, benign responses.The report also characterizes LAT as computationally efficient.Source: Improving the Robustness to… · p. 208

AI Agents

• Autonomous agents’ learning, adaptation, and decisions in evolving scenarios create responsible-AI challenges involving transparency, accountability, and reliability.Their actions can carry ethical, legal, and societal implications, motivating specialized risk approaches.Source: AI Agents · p. 209

Identifying the Risks of LM Agents With LM-Simulated Sandboxes

• 23.9% of critical scenarios defeated even the most safety-optimized LM agents; ToolEmu also found 68.8% of its identified risks plausible real-world threats.The evaluation covered 36 toolkits and 144 test cases; ToolEmu emulates tool execution for scalable safety testing.Source: Identifying the Risks of LM Agents… · p. 209

Jailbreaking Multimodal Agents With a Single Image

• A single injected adversarial image could trigger harmful behavior to spread across interconnected agents, reaching near-total propagation in 1 million simulated agents within 27 to 31 rounds.The passage reports that practical mitigation measures do not currently exist.Source: Jailbreaking Multimodal Agents With a… · p. 210

Election Misinformation

• Reports differ on whether AI-driven misinformation had the feared impact in 2024, leaving election-related risk uncertain and warranting continued monitoring.The report notes that AI systems are improving, making ongoing study important.Source: Election Misinformation · p. 211
• AI’s election-misinformation capability and prominence are growing.Source: Election Misinformation · p. 211

AI Misinformation in the US Elections

• More than 35,000 deepfake images depicted 26 members of Congress, 25 of them women, on pornographic sites.The section also describes deepfakes used for blackmail, risks to trust in digital evidence, and AI-enabled voter profiling and microtargeting; Jason Palmer used hyperpersonalized AI-generated messages in his American Samoa primary campaign.Source: AI Misinformation in the US Elections · p. 211

Rest of World 2024 AI-Generated Election Content

• 60 incidents across 15 countries were documented, spanning audio, image, text, and video on 10 platforms.Figure 3.10.4 presents the tracker’s summary statistics.Source: Rest of World 2024 AI-Generated… · p. 212, Figure 3.10.4 · p. 212

image, X/Twitter, Jun. 2, 2024)

• An AI-generated Starbucks cup image supporting Xóchitl Gálvez gained traction on X, but Starbucks disavowed the design and political endorsement.Source: image, X/Twitter, Jun. 2, 2024) · p. 213

personalized videos (India, video, WhatsApp, Apr. 18,

• Over 500 BJP campaign volunteers received personalized AI-assisted videos featuring Shakti Singh addressing each recipient by name.Campaign employees said voice-cloning and lip-matching software avoided recording each name separately.Source: personalized videos (India, video,… · p. 213

Oct. 27, 2024)

• Uruguay’s presidential debate featured an AI-powered hologram of Yamandú Orsi, whom Orsi and his party criticized as a “fake interview” and an attack on democracy.Source: Oct. 27, 2024) · p. 214

boycotts (Pakistan, audio and video, X/Twitter, Feb. 7, 2024)

• A synthetic Imran Khan recording called for an election boycott, while a video showing Yasmin Rashid making a similar call was determined doctored by Soch Fact Check.The PTI’s official X account denounced the audio as fake.Source: boycotts (Pakistan, audio and video,… · p. 214

TikTok, Medium, Feb. 15, 2024)

• A suspected Chinese government-linked campaign used AI-generated images to spread election misinformation, with posts amplifying polarization and doubts about election integrity and negative posts disproportionately targeting Joe Biden.The campaign increasingly used AI image generators as it focused on the U.S. election, and the ISD noted proliferation of the images on X.Source: TikTok, Medium, Feb. 15, 2024) · p. 215

Instagram, Reddit, May 4, 2024)

• The AI-generated pothole image was shared to discourage Western Cape voters from supporting the DA and continued circulating after its Facebook post was deleted.AFP Fact Check traced the image to an Instagram user who creates AI art.Source: Instagram, Reddit, May 4, 2024) · p. 215

ACCESS THE PUBLIC DATA

• Generative AI productivity gains were measurable in specific tasks, but its broader long-term economic impact remained uncertain as knowledge-worker roles faced disruption and AI-adjacent roles emerged.Organizations were moving toward systematic AI integration, and investment was increasingly directed to enterprise automation and industry-specific applications.Source: ACCESS THE PUBLIC DATA · p. 218
• Corporate AI investment reached $252.3 billion in 2024, while generative AI private investment reached $33.9 billion.Private AI investment rose 44.5%, mergers and acquisitions increased 12.1%, and generative AI investment rose 18.7% year over year to more than 8.5 times its 2022 level.Source: ACCESS THE PUBLIC DATA · p. 219
• U.S. private AI investment reached $109.1 billion in 2024, nearly 12 times China’s $9.3 billion and 24 times the U.K.’s $4.5 billion.U.S. generative AI investment exceeded the combined China, European Union, and U.K. total by $25.4 billion, up from a $21.8 billion gap in 2023.Source: ACCESS THE PUBLIC DATA · p. 219
• Organizational AI use among survey respondents rose to 78% in 2024 from 55% in 2023, while generative AI use in at least one business function climbed from 33% to 71%.North America remained the leader in organizational AI use, while Greater China gained 27 percentage points and Europe gained 23 percentage points year over year.Source: ACCESS THE PUBLIC DATA · p. 219
• 49% of respondents using AI in service operations reported cost savings, while 71% of those using AI in marketing and sales reported revenue gains.Most reported cost savings were below 10%, and the most common level of revenue increase was below 5%.Source: ACCESS THE PUBLIC DATA · p. 219

• 276,300 industrial robots were installed in China in 2023—six times Japan’s total and 7.3 times the United States’ total.Source: §7 · p. 220
• China’s share of global installations rose from 20.8% in 2013 to 51.1%, and it still installed more robots than the rest of the world combined, although that margin narrowed slightly in 2023.Source: §7 · p. 220

8. Collaborative and interactive robot installations become more common. In 2017, collaborative robots

• Collaborative robots grew from 2.8% of new industrial robot installations in 2017 to 10.5% in 2023, while service robot installations rose across application categories except medical robotics.The report interprets these trends as both an overall increase in installations and a growing emphasis on human-facing roles.Source: §8 · p. 220

9. AI is driving significant shifts in energy sources, attracting interest in nuclear energy. Microsoft announced

• Microsoft announced a $1.6 billion deal to revive the Three Mile Island reactor to power AI, and Google and Amazon also secured nuclear energy agreements for AI operations.Source: §9 · p. 220
• AI boosts productivity and, in most cases, helps narrow the gap between low- and high-skilled workers, according to additional studies reinforcing prior findings.Source: §9 · p. 220

Global AI Labor Demand

• Singapore (3.2%), Luxembourg (2%), and Hong Kong (1.9%) led the 2024 share of job postings requiring AI skills, while the U.S. share rose from 1.4% in 2023 to 1.8% in 2024.Most countries saw an increase in the share of job postings requiring AI skills from 2023 to 2024.Source: Global AI Labor Demand · p. 225

US AI Labor Demand by Skill Cluster and Specialized Skill

• In U.S. job postings, artificial intelligence and machine learning each accounted for 0.9%, natural language processing for 0.2%, and generative AI grew nearly fourfold.Most tracked AI-related skill clusters gained market share since the previous year, except autonomous driving and robotics; year-over-year postings citing generative AI skills increased by more than a factor of three.Source: US AI Labor Demand by Skill Cluster… · p. 227

US AI Labor Demand by Sector

• Nearly every U.S. industry sector had a higher proportion of AI job postings in 2024 than in 2023, except the public sector.Source: US AI Labor Demand by Sector · p. 230

US AI Labor Demand by State

• California had 103,375 AI job postings in 2024, Texas 57,785, and New York 37,944; California accounted for 15.7% of U.S. AI postings, followed by Texas at 8.8%.By AI-related postings as a share of all state postings, the District of Columbia led at 4.4%, followed by Delaware at 3.4%.Source: US AI Labor Demand by State · p. 231
• In 2024, all four top states reversed multiyear declines in their share of AI job postings, notably California and New York.The supplied passage identifies California and New York as having experienced decreases before the reversal.Source: US AI Labor Demand by State · p. 233

AI Hiring

• 33.4% in India, 30.8% in Brazil, and 28.7% in Saudi Arabia were the highest 2024 relative AI hiring rates year-over-year.These figures represent growth in AI talent hiring relative to overall hiring in each country.Source: AI Hiring · p. 234
• Starting in 2024, AI hiring rates notably rose in Argentina, Brazil, and Chile, as well as Canada, India, South Africa, and the United States.The passages describe these as recent upticks or similar rises in AI hiring rates.Source: AI Hiring · p. 234

AI Skill Penetration

• For 2015–24, AI skill penetration was highest in the United States (2.6) and India (2.5), followed by the United Kingdom (1.4), Germany (1.3), and Brazil (1.3).The U.S. rate was 2.6 times the global average across the same set of occupations.Source: AI Skill Penetration · p. 236
• Men had higher relative AI skill penetration than women in every sampled country except Saudi Arabia; India (1.9), the United States (1.7), and Canada (1.0) led for women.These are relative AI skill penetration rates for women.Source: AI Skill Penetration · p. 237

AI Talent

• In 2024, Israel (2.0%), Singapore (1.6%), and Luxembourg (1.4%) had the highest AI talent concentrations; concentration was higher among men except in India and Saudi Arabia.AI talent concentration is the portion of LinkedIn members identified as AI talent.Source: AI Talent · p. 238
• 252% in India, 240% in Costa Rica, and 237% in Portugal were the greatest increases in AI talent concentration from 2016 to 2024.Several major economies registered substantial increases in their AI talent pools over this period.Source: AI Talent · p. 238

Percentage change in AI talent concentration by geographic area, 2016 vs. 2024

• 69.5% of AI professionals on LinkedIn were male and 30.5% female in 2024, a ratio that remained remarkably stable over time.AI talent concentration was higher among men than women except in India and Saudi Arabia; Israel had the highest female AI talent concentration in 2024, at 1.6%.Source: Percentage change in AI talent… · pp. 238, 240
• Men proportionally outnumber women in AI roles in every sampled country; New Zealand and Romania have the most balanced gender distributions.Source: Percentage change in AI talent… · p. 240, Figure 4.2.21 · p. 241
• Luxembourg (8.9), Cyprus (4.7), and the United Arab Emirates (4.1) report the greatest per-capita incoming AI talent migration; Israel, the Netherlands, and Canada have seen declining net migration.Rising talent flows include the United Arab Emirates and Saudi Arabia. Net flows are arrivals minus departures, normalized per 10,000 LinkedIn members.Source: Percentage change in AI talent… · p. 242
• AI adoption peaks in upper-wage-quartile occupations, while considerable-preparation jobs show 50% higher usage than their baseline workforce representation and both preparation extremes show lower adoption.Source: Percentage change in AI talent… · p. 245
• 57% of AI interactions are augmentative and 43% automated, with critical thinking and writing prominent in conversations and physical and managerial skills minimally present.Source: Percentage change in AI talent… · p. 247

Corporate Investment

• $252.3 billion in corporate AI investment in 2024 was up 25.5% from 2023; private investment rose 44.5%, mergers and acquisitions 12.1%, and investment grew nearly thirteenfold over the decade.The total covers mergers and acquisitions, minority stakes, private investments, and public offerings.Source: Corporate Investment · p. 248, Figure 4.3.1 · p. 248

Startup Activity

• AI startup activity is measured through private investment in startups that have received over $1.5 million since 2013.Source: Startup Activity · p. 249

Global Trends

• $33.9 billion went to generative AI in 2024, up 18.7% from 2023 and more than 8.5 times 2022 investment; it accounted for over a fifth of AI-related private investment.Source: Global Trends · p. 250, Figure 4.3.3 · p. 250
• In 2024, 2,049 AI companies received first-time funding, an 8.4% increase over 2023; newly funded generative AI companies numbered 214, versus 179 in 2023 and 31 in 2019.Source: Global Trends · p. 251

Regional Comparison by Funding Amount

• $109.1 billion in U.S. AI private investment in 2024 was 11.7 times China’s $9.3 billion and 24.1 times the United Kingdom’s $4.5 billion.Source: Regional Comparison by Funding Amount · p. 253, Figure 4.3.8 · p. 253
• $470.9 billion in U.S. AI private investment since 2013 led China’s $119.3 billion and the United Kingdom’s $28.2 billion.Source: Regional Comparison by Funding Amount · p. 254, Figure 4.3.9 · p. 254
• Since 2023, U.S. AI private investment rose 50.7%, Europe’s rose 60%, and China’s fell 1.9%; the U.S. increase since 2022 was 78.3%.Source: Regional Comparison by Funding Amount · p. 255, Figure 4.3.10 · p. 255
• $25.4 billion was the U.S. lead over combined China and Europe generative AI investment in 2024, up from approximately $21.8 billion in 2023.Source: Regional Comparison by Funding Amount · p. 256, Figure 4.3.11 · p. 256

Regional Comparison by Newly Funded AI Companies

• 1,073 new AI companies received funding in the United States in 2024, compared with 116 in the United Kingdom and 98 in China.Source: Regional Comparison by Newly Funded AI… · p. 257, Figure 4.3.12 · p. 257
• Over the past decade, the United States had around 4.3 times as many newly funded AI companies as China and 7.9 times as many as the United Kingdom.Source: Regional Comparison by Newly Funded AI… · p. 258, Figure 4.3.13 · p. 258

Focus Area Analysis

• $37.3 billion went to AI infrastructure/research/governance in 2024, followed by $16.6 billion for data management and processing and $11 billion for medical and healthcare.The prominence of infrastructure, research, and governance reflects large investments in companies building AI applications, including OpenAI, Anthropic, and xAI.Source: Focus Area Analysis · p. 260
• Most AI investment focus areas saw a boost in the last year, but NLP and customer-support investment has declined since peaking in 2021.Source: Focus Area Analysis · p. 260, Figure 4.3.16 · p. 261

Industry Usage

• 78% of surveyed respondents said their organizations used AI in at least one business function in 2024, up from 55% in 2023.The passage describes this as an increase after business AI use stagnated between 2017 and 2023.Source: Industry Usage · p. 262, Figure 4.4.1 · p. 262
• 71% of respondents said their organizations regularly used generative AI in at least one business function in 2024, versus 33% in 2023.Generative AI use more than doubled year over year.Source: Industry Usage · p. 262
• AI-related cost savings were most often reported in service operations (49%) and supply chain and inventory management (43%), while revenue gains were most often reported in marketing and sales (71%).Reported revenue gains also included supply chain and inventory management (63%) and service operations (57%); effects were most commonly at low levels.Source: Industry Usage · p. 264
• AI use increased in every region in 2024; Greater China rose by 27 percentage points, while North America led at 82% and Europe reached 80%, up 23 points.The passage reports year-over-year increases across all regions.Source: Industry Usage · p. 265

Deployment of AI Capabilities

• Marketing strategy content support was the most common generative AI use case (27%), while only 1% of surveyed C-suite executives in developed markets described their rollouts as mature.Most companies were still in the early stages of capturing value at scale from AI.Source: Deployment of AI Capabilities · p. 266
• For generative AI, cost savings were most often reported in supply chain and inventory management (61%) and service operations (58%), while revenue gains were most often reported in strategy and corporate finance (70%).Other leading revenue-benefit functions were supply chain and inventory management (67%) and marketing and sales (66%); reported effects were most commonly at low levels.Source: Deployment of AI Capabilities · p. 267
• 71% of respondents reported generative AI use in at least one business function in 2024, versus 33% in 2023, narrowing the gap with overall AI use from 22 to 7 percentage points.North America (74%), Europe (73%), and Greater China (73%) led in generative AI use.Source: Deployment of AI Capabilities · p. 268
• Studies covering over 200,000 professionals found productivity gains ranging from 10% to 45%, particularly in technical, customer support, and creative tasks.The studies used diverse methods, including natural experiments, randomized controlled trials, and large-scale surveys.Source: Deployment of AI Capabilities · p. 269

Productivity Trends

• AI adoption increased customer-support issues resolved per hour by 14.2%, with gains emerging quickly and exposed workers maintaining higher efficiency.The study analyzed 5,179 customer-support agents after the staggered introduction of an AI-powered conversational assistant.Source: Productivity Trends · p. 269, Figure 4.4.8 · p. 269
• Microsoft workplace results included 10–13% more document editing, 11% less email processing time, and 23% faster security work with 7% higher accuracy.Sales teams also showed 39% faster response times.Source: Productivity Trends · p. 269
• In software development, AI assistance increased task completion by 26.08% on average, while another study found 12.4% more core coding activity and 24.9% less project time.The studies involved 4,867 and 187,489 developers, respectively.Source: Productivity Trends · p. 270

Equalizing Effect

• Workplace-performance gains show an equalizing pattern, with junior software developers experiencing productivity increases.The supplied passage identifies the pattern but does not provide the junior developers’ gain values in this subsection.Source: Equalizing Effect · p. 270

AI’s productivity equalizing effects

• Junior developers gained 21–40% versus seniors’ 7–16%, while low-ability workers gained 14–27% versus high-ability workers’ 5–10%.The analysis also reported 21.8% more exploration of new technologies and a potential average salary increase of $1,683 per developer annually.Source: AI’s productivity equalizing effects · p. 270

Adoption and Integration

• Organizations with high AI integration had a 72% probability of significant productivity improvements, compared with 3.4% for those with minimal integration.In the survey sample, 46.8% reported gains of 0–20%, 26.2% gains of 20–40%, and 18.4% gains of 40–60%.Source: Adoption and Integration · p. 271

Workforce Impact

• AI automation reduced perceived mental demand by 45%, closed 84.6% of the accuracy gap for nonnative English speakers, and projects required 79.3% fewer collaborators.The Microsoft workplace study also found 49% more key information in professional reports.Source: Workforce Impact · p. 271
• Romanian survey respondents reported varied expectations for workforce size: 43% of organizations anticipated decreases, 30% little change, 15% increases, and 12% were uncertain.The section also says software-engineer numbers were expected to increase, consistent with the Jevons Paradox.Source: Workforce Impact · p. 271

4.5 Robot Deployments

• 541,000 industrial robots were installed in 2023, a 2.2% decrease from 2022 and the first year-over-year decline since 2019, while operational stock rose to 4,282,000 from 3,904,000.Installation and utilization have steadily increased since 2012; the latest available data is from 2023, and the IFR may revise historical figures.Source: Figure 4.5.1 · p. 274, §4.5 · pp. 274–275, Figure 4.5.2 · p. 275

Industrial Robots: Traditional vs. Collaborative Robots

• Collaborative robots rose from 2.8% of new industrial robot installations in 2017 to 10.5% in 2023.They are designed to work alongside people, unlike traditional robots, which operate in place of humans.Source: Industrial Robots: Traditional vs.… · p. 276

By Geographic Area

• China installed 276,300 industrial robots in 2023—six times Japan’s 46,100 and 7.3 times the United States’ 37,600—and accounted for 51.1% of global installations.China’s global share rose from 20.8% in 2013; since 2021, it has installed more robots than the rest of the world combined, though its margin over the rest decreased in 2023.Source: By Geographic Area · pp. 277–279
• From 2022 to 2023, India, the United Kingdom, and Canada had the highest installation growth rates at 59%, 51%, and 37%, while Taiwan declined 43%.France declined 13%, and Japan and Italy each declined 9%.Source: By Geographic Area · p. 280

Country-Level Data on Service Robotics

• In 2023, service-robot installations increased in every application category except medical robots; agricultural and hospitality installations rose 2.5 and 2.2 times, respectively.Service robots perform useful tasks for people or equipment outside industrial automation, including in medical settings and professional cleaning.Source: Country-Level Data on Service Robotics · p. 281

ACCESS THE PUBLIC DATA

• The next chapter examines AI in science and medicine, including protein folding, clinical applications, real-world deployments, synthetic data, health determinants, and research ethics.The chapter was prepared by RAISE Health in collaboration with the AI Index Steering Committee.Source: ACCESS THE PUBLIC DATA · p. 284

1. Bigger and better protein sequencing models emerge. In 2024, several large-scale, high-performance protein

• In 2024, larger protein-sequencing models including ESM3 and AlphaFold 3 launched as prediction accuracy continued to improve; OpenAI o1 reached 96.0% on MedQA, 5.8 percentage points above 2023’s best score.MedQA performance had improved 28.4 percentage points since late 2022, and the benchmark may be approaching saturation.Source: §1 · p. 285
• GPT-4 alone outperformed doctors with and without AI on complex clinical diagnoses, while other studies found AI surpassing doctors in cancer detection and high-mortality-risk identification.Some early research suggests AI-doctor collaboration yields the best results.Source: §1 · p. 285

5. The number of FDA-approved, AI-enabled medical devices skyrockets. The FDA authorized its first AI-enabled

• The number of FDA-approved AI-enabled medical devices rose from six in 2015 to 223 by 2023, after the FDA authorized its first such device in 1995.Source: §5 · p. 285

6. Synthetic data shows significant promise in medicine. Studies released in 2024 suggest that AI-generated

• Studies released in 2024 suggest synthetic data can help identify social determinants of health, support privacy-preserving clinical risk prediction, and aid discovery of drug compounds.Source: §6 · p. 285

LLMs optimize protein sequence optimization

• LLMs without fine-tuning generated protein sequences that outperformed conventional algorithms across synthetic and experimental fitness landscapes, including constrained and multiobjective settings.The study reports this as an emergent capability of existing LLMs.Source: LLMs optimize protein sequence… · p. 287

Training LLM agents for biological tasks

• Across nearly all Aviary tasks, agentic models outperformed the baseline; fine-tuning with domain experts and interaction with external tools can improve scientific-task performance.Aviary is a framework for training language agents on DNA manipulation, research questions, and protein stability tasks.Source: Training LLM agents for biological tasks · p. 288

AI for novel, high-affinity protein binders

• The team estimated that some AlphaProteo binders were up to 300 times more effective than available options on the seven tested targets.AlphaProteo created successful binders for seven targets, including first binders for VEGF-A.Source: AI for novel, high-affinity protein… · p. 289

Synaptically reconstructing a small piece of the human brain

• A Google team reconstructed a one-cubic-millimeter section of the human brain at the synaptic level.The sample came from a patient’s left anterior temporal lobe during surgery.Source: Synaptically reconstructing a small… · p. 289
• The reconstructed brain sample captured over 5,000 slices, around 57,000 cells, and 150 million synapses, marking a step toward understanding neural circuitry.The project could inform future neurological treatments.Source: Synaptically reconstructing a small… · p. 289

Virtual AI lab supercharges biomedical research

• Over 90% of the 92 nanobodies generated by a virtual AI lab successfully bound SARS-CoV-2 in validation studies.The study’s central contribution was demonstrating that an entirely autonomous, LLM-powered lab can generate meaningful scientific discoveries.Source: Virtual AI lab supercharges biomedical… · p. 290

Continuous glucose monitoring with AI

• 66% of new-onset diabetes cases and 69% of cardiovascular-related deaths were flagged within their respective top-risk quartiles by GluFormer in a 12-year study of 580 adults.The model generalizes across health conditions and often outperforms standard CGM-based metrics such as the glucose management indicator (GMI), supporting earlier clinical intervention.Source: Continuous glucose monitoring with AI · p. 291

Simulating evolutionary processes to generate novel proteins

• ESM3 generated esmGFP, an artificial green fluorescent protein estimated by its company to take nature 500 million years to develop, and larger ESM3 models solved twice as many tasks.ESM3 is open-sourced, supporting collaboration in synthetic biology and protein engineering.Source: Simulating evolutionary processes to… · p. 291

molecules

• AlphaFold 3 models interactions between proteins and biomolecules including DNA, RNA, ligands, and antibodies, and is competitive with previous state-of-the-art methods, particularly with a predefined binding pocket.The report notes that modeling small molecule–protein interactions can accelerate drug development.Source: molecules · p. 292, Figure 5.1.8 · p. 292

AI-Driven Protein Sequence Models

• Large-scale machine-learning models have improved protein-property prediction, accelerating research in structural biology and molecular engineering; AlphaFold, ESM2, and ESM3 were recently released.Source: AI-Driven Protein Sequence Models · p. 293
• Larger protein models improve prediction, and ESM C achieved greater protein-structure accuracy in CASP15.ProGen demonstrated functional protein-sequence design, while ProtT5 predicts protein function and interactions from sequence data.Source: AI-Driven Protein Sequence Models · pp. 293–294

Public Databases for Protein Science

• Protein-science database entries have steadily grown, supporting researchers and industry, but data quality and model bias remain concerns.Larger, well-curated datasets enable training on diverse biological sequences and enhance predictive power.Source: Public Databases for Protein Science · p. 295, Figure 5.2.4 · p. 295

AI-Driven Protein Science Publications

• In 2024, protein function prediction represented 8.4% of AI-driven biological-sciences research, followed by structure prediction at 7.6% and protein-drug interactions at 3.0%.AI-driven studies on PubMed and bioRxiv increased over the past year.Source: AI-Driven Protein Science Publications · p. 296, Figure 5.2.5 · p. 296

Image and Multimodal AI for Scientific Discovery

• Microscopy foundation models expanded in 2024: light-based models rose from four to eight, while electron and fluorescence models each went from zero to four.The number of microscopy foundation models increased as more data was collected and made publicly available.Source: Image and Multimodal AI for Scientific… · p. 297, Figure 5.2.6 · p. 297

Data: Sources, Types, and Needs

• More than 80% of FDA-cleared machine-learning software targets medical-image analysis.Medical-imaging AI is expanding to new modalities and more complex clinical questions.Source: Data: Sources, Types, and Needs · p. 298
• Public histopathology cohorts rarely exceed 10,000 samples, and even the comprehensive TCGA collection has 11,125 samples across 32 cancer types.Few datasets include the matched annotations or omics data needed for advanced classification.Source: Data: Sources, Types, and Needs · p. 298
• Histopathology AI models are often trained on fewer than 1,000 samples, increasing the risk of overfitting and poor performance.Training sets are particularly limited when genomic or proteomic data serve as labels.Source: Data: Sources, Types, and Needs · p. 298
• Most cohorts used to train clinical algorithms originate in California, Massachusetts, and New York, while public 3D datasets—especially 3D histopathology—remain limited.The passage also notes that 3D imaging is not standard practice in histopathology, leaving no publicly available 3D histopathology datasets.Source: Data: Sources, Types, and Needs · pp. 298–299
• Pathology acquisition variability, including differences in instruments, staining, and institutional practices, creates batch effects that limited training datasets exacerbate.Standardization challenges persist due to this acquisition variability.Source: Data: Sources, Types, and Needs · p. 299

Advanced Modeling Approaches

• Robust medical imaging AI requires larger, more comprehensive, multicohort labeled datasets and better validation to improve performance and confidence for clinical use.Figure 5.3.3 presents leading imaging-modeling approaches, releases, and associated challenges.Source: Advanced Modeling Approaches · p. 300, Figure 5.3.3 · p. 300

Medical disciplines and notable AI models Source: AI Index, 2025

• Medical imaging foundation models have risen sharply, with a particularly high concentration of newly launched pathology models.Figure 5.3.4 categorizes notable models by medical discipline.Source: Medical disciplines and notable AI… · p. 301, Figure 5.3.4 · p. 301

Clinical Knowledge

• Clinical-knowledge evaluation examines AI models on key medical-knowledge benchmarks, including MedQA, which contains over 60,000 questions derived from professional medical board exams.The report introduces this evaluation as a measure of medical expertise relevant to clinical settings.Source: Clinical Knowledge · p. 302, MedQA · p. 302

MedQA

• 96.0%: o1 set a MedQA state-of-the-art, 5.8 percentage points above the 2023 record.The cited figure presents the benchmark results; the prose reports the score and improvement.Source: MedQA · p. 302, Figure 5.4.1 · p. 302
• MedQA may be nearing saturation, and using multiple benchmarks can provide greater clinical relevance and a more robust assessment of model performance.The passages caution that relying only on standard medical QA benchmarks may miss real-world clinical complexity.Source: MedQA · pp. 302–303
• Despite improving clinical knowledge performance, particularly in newer models such as o1, LLMs still face hallucinations and inconsistent multilingual performance.The report links o1’s performance improvement to real-time reasoning capabilities.Source: MedQA · p. 303
• o1 scored 5.8 percentage points higher than GPT-4 Turbo with Medprompt, but was approximately 1.5 times more expensive.Researchers also found that o1 outperformed the GPT-4 series with Medprompt even without specialized prompting; the comparison illustrates an accuracy-cost trade-off on MedQA.Source: Figure 5.4.3 · p. 303, MedQA · p. 303

Overview

• A PubMed search found 1,566 papers on large language models since 2019, including 1,210 published in 2024; a review identified over 500 healthcare NLP evaluations.Studies emphasized medical knowledge (419) and diagnosis (178).Source: Overview · pp. 304–305

Management Reasoning and Patient Care Decisions

• Researchers tested whether LLMs could improve management reasoning involving treatment decisions, risk-benefit trade-offs, and patient preferences.The supplied passages describe the study question but do not provide its results.Source: Management Reasoning and Patient Care… · p. 306

Diagnostic Reasoning With LLMs

• A 92% diagnostic reasoning score outperformed both physician groups; physicians using AI assistance scored 76%, performing only slightly better than those relying solely on conventional tools.Source: Diagnostic Reasoning With LLMs · p. 306

Ambient AI Scribes

• At Kaiser Permanente Northern California, ambient AI scribe technology was adopted by thousands of clinicians before the pilot ended.The studies described earlier systems that were not fully automated or integrated into the electronic health record.Source: Ambient AI Scribes · p. 308
• A Stanford study of a fully integrated, automated AI scribe found improvements in documentation time and physician experience, with average physician uptake of 55%.The study also reported efficiency gains that saved physicians time.Source: Ambient AI Scribes · p. 309
• Physicians saved approximately 30 seconds per note and about 20 minutes of EHR time per day; reported burden and burnout fell by 35% and 26%, respectively.The report says these findings suggest ambient AI scribe technology can improve physician workflow and well-being.Source: Figure 5.4.9 · p. 309, Ambient AI Scribes · p. 309

Successful Use Cases: Stanford Health Care

• Stanford’s FURM framework evaluates AI tools for fairness, usefulness, reliability, and measurability before clinical implementation.The report describes it as the framework used to assess and implement the health-care use cases.Source: Successful Use Cases: Stanford Health… · p. 310

FDA Authorization of AI-Enabled Medical Devices

• FDA annual AI-enabled medical-device approvals peaked at 223 in 2023, after six devices were approved in 2015 and annual counts remained in the single digits for two decades.The FDA authorized its first AI-enabled medical device in 1995.Source: Figure 5.4.10 · p. 310, FDA Authorization of AI-Enabled… · p. 310

Successful Use Cases: Stanford Health Care

• Stanford implemented two of six assessed AI use cases, including a PAD screening tool that reached full implementation after a pilot and is expected to affect approximately 1,400 patients annually.The program operates without external funding and is described as financially sustainable; the tool flags high-risk patients for evaluation and referral.Source: Successful Use Cases: Stanford Health… · p. 310, Screening for Peripheral Arterial… · p. 311

Social Determinants of Health

• In 2024, AI advancements targeted social determinants of health, including socioeconomic status and environment, with the stated aim of improving patient care and health equity.A figure describes integration of social determinants of health across medical fields.Source: Social Determinants of Health · p. 312

Extracting SDoH From EHR and Clinical Notes

• Fine-tuned Flan-T5 XL classifiers outperformed ChatGPT-family models in identifying social determinants in clinical notes and were less sensitive to demographic descriptors, with reduced discrepancies by race, ethnicity, or gender.The report also says larger models such as Flan-T5-XXL, augmented with synthetic and gold data, showed superior performance; it emphasizes bias mitigation and robust synthetic data.Source: Extracting SDoH From EHR and Clinical… · p. 312

Synthetic Data

• Synthetic data supports privacy-preserving analytics, clinical modeling, AI training, workflow optimization, and simulation of rare cases, but scalability concerns call for cautious adoption.The report frames these applications as supporting AI-driven innovations.Source: Synthetic Data · p. 313

Clinical Risk Prediction

• A study used ADSGAN, PATEGAN, and DPGAN to model lung-cancer risk among ever-smokers from UK Biobank; PCA eigenvalues showed ADSGAN and PATEGAN closely matched real-data distributions.The study is presented as validating synthetic data for privacy-preserving clinical risk prediction.Source: Clinical Risk Prediction · p. 313
• Synthetic clinical-risk datasets preserved statistical fidelity and supported exploratory analysis and predictive modeling without real, identifiable patient data.The passage describes clustering and feature selection as part of the findings.Source: Clinical Risk Prediction · p. 313

Drug Discovery

• AI-synthesized tablet volumes aligned closely with real volumes in tortuosity calculations, suggesting promise for modeling drug properties and supporting AI-driven discovery.The study tested percolation-threshold prediction for microcrystalline cellulose in oral tablets.Source: Drug Discovery · p. 314

Data Generation Platforms

• The STNG framework demonstrated feasible large-scale synthetic-data generation and validation, with real and synthetic heart-disease datasets often showing close AUC overlap.The reported overlap indicates that synthetic data can closely model complex health conditions.Source: Data Generation Platforms · p. 314, Figure 5.4.16 · p. 314

Electronic Health Record System

• Most hospitals using ML or predictive models in inpatient EHRs relied on a dominant vendor; Epic, Cerner, and CPSI mainly used vendor-developed models, unlike Meditech and others.Meditech and other hospitals more often adopted third-party or in-house solutions.Source: Electronic Health Record System · p. 315

Clinical Decision Support

• AI mentions in clinical trials steadily rose, and China (105 trials), the U.S. (97), and Italy (42) led AI-driven clinical trials in 2024.The passage describes clinical trials as increasing over the years.Source: Clinical Decision Support · pp. 317–318, Figure 5.4.20 · p. 318
• Clinical decision support expanded from pandemic triage and resource allocation to chronic-disease management and workflow optimization, with studies exploring procedural accuracy and medication-error detection.The CERTAIN Study examined AI-driven real-time procedural support, while another study used AI to detect real-time medication errors.Source: Clinical Decision Support · p. 318

Meta Review

• The AI Index conducted a meta-review of thousands of medical ethics studies to assess the state of the field.The supplied passage gives the review’s scope but no findings.Source: Meta Review · p. 319
• Medical AI ethics publications increased fourfold from 2020 to 2024; in 2024, bias and privacy led concerns, followed by equity, reversing privacy’s earlier prominence over bias.Privacy was more prominent than bias in 2020, but that pattern had since shifted.Source: Meta Review · pp. 319–320
• Medical ethics literature increasingly focused on large language models, with substantial attention to OpenAI’s GPT series, including ChatGPT.The report says interest in large language models has expanded over the past few years.Source: Meta Review · p. 320

Number of NIH grants for medical AI ethics by LJscal year, 2020–24

• NIH grants for medical AI ethics rose from 25 in 2023 to 337 in 2024, while funding climbed from $16 million to $276 million.The funding increase was described as almost 17-fold in one year.Source: Number of NIH grants for medical AI… · p. 321, Meta Review · p. 321

5.6 AI Foundation Models in Science

• ORBIT, a 113B-parameter Earth-science vision transformer, reached up to 1.6 exaFLOPS sustained performance and was 1,000 times larger than prior climate-science models.It was trained with a novel parallelism technique and tested on the Frontier supercomputer.Source: §5.6 · p. 323
• Aurora delivered state-of-the-art forecasts across Earth-system domains, while NeuralGCM combined a physics-based solver and machine learning for weather and climate simulation with major computational savings.Aurora was trained on over a million hours of Earth-system data; NeuralGCM matched or exceeded leading models on short- and medium-term forecasts and tracked climate metrics over decades.Source: §5.6 · p. 323
• PhysBERT outperformed general-purpose models on physics-specific tasks, FireSat detects fires as small as 5x5 meters within 20 minutes, and GenCast produces accurate 15-day forecasts in minutes.PhysBERT was trained on 1.2 million arXiv papers; GenCast outperformed traditional systems like the ENS on nearly all metrics.Source: §5.6 · p. 324
• AlphaQubit achieved state-of-the-art quantum error detection, and Willow achieved exponential error suppression and correction below the surface code threshold.Willow completed a benchmark task in under five minutes that would take the fastest supercomputer over 10 septillion years.Source: §5.6 · p. 324

ACCESS THE PUBLIC DATA

• State-level AI-related laws grew from one in 2016 to 49 in 2023 and 131 in the past year, while the number of federal AI bills passed remained low.The supplied passage characterizes state-level progress as leading amid slow federal progress.Source: ACCESS THE PUBLIC DATA · p. 328

2. Governments across the world invest in AI infrastructure. Canada announced a $2.4 billion AI infrastructure

• Across countries, governments announced major AI investments: China launched a $47.5 billion semiconductor fund, France committed €109 billion to AI, India pledged $1.25 billion, and Saudi Arabia’s Project Transcendence represents $100 billion.Canada announced a $2.4 billion AI infrastructure package.Source: §2 · p. 328
• AI mentions in legislative proceedings across 75 major countries rose 21.3% in 2024, to 1,889 from 1,557 in 2023, and have grown more than ninefold since 2016.Source: §2 · p. 328

4. AI safety institutes expand and coordinate across the globe. In 2024, countries worldwide launched international

• AI safety institutes first emerged in the U.S. and U.K. in November 2023; at the May 2024 Seoul Summit, additional institutes were pledged across nine countries and the EU.Source: §4 · p. 328
• In 2024, 59 U.S. AI-related federal regulations were introduced, more than double 2023’s 25, from 42 agencies versus 21 the prior year.Source: §4 · p. 328

6. U.S. states expand deepfake regulations. Before 2024, only five states—California, Michigan, Washington, Texas,

• In 2024, 15 more U.S. states introduced election-deepfake measures, and 24 states had passed regulations targeting deepfakes.Before 2024, five states had enacted laws regulating deepfakes in elections.Source: §6 · p. 328

6.1 Major Global AI Policy News in 2024

• Singapore planned over $1 billion for AI over five years, while Abu Dhabi established an AI investment firm targeting $100 billion in assets.Singapore’s funding supports computation, talent development, and industry growth; Abu Dhabi’s firm is state-owned and specializes in AI technologies.Source: §6.1 · p. 329
• The European Parliament passed the EU AI Act, introducing transparency and reporting obligations, risk-based rules, and bans on certain applications.Most provisions were to take effect in 2026 after a two-year implementation period.Source: §6.1 · p. 329
• Other 2024 policy developments included a U.N. resolution promoting safe, secure, and trustworthy AI, India’s $1.25 billion AI Mission, and the U.K. Safety Institute’s open-source Inspect assessment tool.The U.N. resolution was backed by more than 120 member states and was not legally binding; Inspect assesses capabilities including reasoning and autonomy.Source: §6.1 · pp. 330–331
• At the AI Seoul Summit, countries signed an intent to establish a collaborative network of AI safety institutes, emphasizing international cooperation on AI safety.Source: §6.1 · p. 332
• China launched a $47.5 billion fund to boost semiconductor production, marking the third phase of its state-backed “Big Fund.”Source: §6.1 · p. 332
• The European Commission established an AI Office to implement the EU AI Act, enforce general-purpose AI standards, coordinate codes of practice, and apply sanctions.Source: §6.1 · p. 332
• NIST issued a voluntary framework for identifying and mitigating generative AI risks, while the State Department released a human-rights risk profile built on NIST’s framework.Source: §6.1 · p. 333
• The U.K. canceled £1.3 billion in promised technology and AI funding, while a new U.S. task force was formed to prioritize AI data-center development.Source: §6.1 · pp. 333–334
• California enacted three election-content bills requiring platforms to remove or label altered election material during specified periods and extending restrictions on deceptive AI election content.Source: §6.1 · p. 334
• The U.N. adopted the Global Digital Compact, which seeks an inclusive, secure digital future and enhanced international governance of artificial intelligence.Source: §6.1 · p. 334
• California Governor Gavin Newsom vetoed an AI safety bill that would have set a national precedent for AI regulation.Source: §6.1 · p. 335
• In November 2024, the U.S. launched an international AI safety network to coordinate safe innovation, address synthetic-content risks, test foundation models, and assess advanced-system risks; it secured over $11 million in research funding.Source: §6.1 · p. 336
• The U.S. further restricted China's advanced-semiconductor production through export controls covering 24 types of manufacturing equipment and three types of software tools.Source: §6.1 · p. 336
• At the U.N. Security Council, António Guterres warned that AI's evolution outpaces governance and could undermine human control over weapons, calling for international guardrails.Source: §6.1 · p. 337

Overview

• 40 AI-related laws were passed in 2024, up from 30 in 2023 and second-highest after 2022; 39 countries had enacted at least one, totaling 204 laws.Source: Overview · p. 338

By Geographic Area

• Russia led 2024 AI-law enactments with seven, followed by Belgium and Portugal with five each; since 2016, the United States led with 27, while Portugal and Russia had 20 each.Source: By Geographic Area · p. 339, Number of AI-related bills passed into… · p. 339, Figure 6.2.3 · p. 339

Federal Level

• 221 proposed AI-related bills were counted in the U.S. Congress in 2024, up from 171 in 2023.Source: Federal Level · p. 341
• AI-related U.S. federal bills proposed since 2022 have almost tripled, but relatively few pass; increased activity likely reflects rising public awareness and AI capabilities, particularly generative AI.The section covers congressional bills only, not executive-branch actions or rules from regulatory agencies.Source: Federal Level · p. 341

State Level

• Across 2016–24, California led state AI-related laws with 42, followed by Maryland, Virginia and Utah with 17 each.Source: Figure 6.2.8 · p. 342, State Level · p. 342
• In 2024, California enacted 22 AI-related laws, Utah 12 and Maryland eight; state laws rose from one in 2016 to 131 in 2024.For the close-up of state legislation, the Index includes only anti-deepfake bills with confirmed passage dates when dates could not be verified.Source: Figure 6.2.7 · p. 342, State Level · pp. 342–343, Figure 6.2.9 · p. 343

Global AI Mentions

• Legislative AI mentions rose 21.3%, from 1,557 in 2023 to 1,889 in 2024; Spain led with 314, followed by Ireland with 145 and Australia with 123.Source: Figure 6.2.14 · p. 347, Overview · pp. 347–348, Figure 6.2.15 · p. 348

Number of mentions of AI in legislative proceedings by country, 2016–24 (sum)

• Across 2016–24, Spain had 1,200 legislative AI mentions, followed by the United Kingdom with 710 and Ireland with 659.Source: Figure 6.2.16 · p. 348, Number of mentions of AI in… · p. 348
• Countries with greater parliamentary AI discussion generally passed more AI-related bills, though Belgium, Portugal and Russia deviated from this trend.Source: Number of mentions of AI in… · p. 349

US Committee Mentions

• The 118th U.S. legislative session recorded 136 AI mentions in committee reports, up 83.8% from the 117th session.Source: US Committee Mentions · p. 350

US Regulations

• In 2024, 59 AI-related U.S. regulations were introduced, more than double the 25 recorded in 2023.Source: Overview · p. 351

By Agency

• AI regulations came from a record 42 unique departments, up from 21 in 2023 and 17 in 2022; HHS issued 14, with CMS and Commerce issuing 7 each.Source: By Agency · p. 351

6.3 Public Investment in AI

• In 2023, the AI Index estimates U.S. AI-related tenders at approximately $830 million, compared with $4.5 billion in AI-related grants.The analysis used NLP on public tenders and contracts, but only grant-level spending for the United States; cross-country comparisons remain premature because comparable data are limited.Source: §6.3 · p. 354

Total AI Public Investments

• From 2013 to 2023, the United States led with about $5.2 billion across 2,678 unique AI contracts.The contract-value data have 26.8% missing U.S.-dollar values; the sample also excludes Northern Ireland tenders.Source: Figure 6.3.1 · p. 355, Total AI Public Investments · p. 355, Figure 6.3.2 · p. 356
• The United States led AI public investment per capita at $1.58 million per 100,000 inhabitants, followed by Finland and Denmark at $1.3 million each.Source: Figure 6.3.4 · p. 357, Total AI Public Investments · p. 357
• In 2023, the U.S. led public AI investment with $831.0 million, followed by the United Kingdom at $262.6 million.Romania, Greece, Hungary, and Poland entered Europe’s top 10, suggesting a more balanced distribution of AI funding across Europe.Source: Figure 6.3.5 · p. 358, Total AI Public Investments · p. 358
• Europe’s AI investment in 2023 was approximately 67 times its 2013 level, versus a fifteenfold increase in the United States; the gap narrowed over the past three years.The disparity had widened until 2020.Source: Figure 6.3.6 · p. 359, Total AI Public Investments · p. 359, Figure 6.3.7 · p. 360
• Across Belgium, France, Germany, Spain, and the U.K., public AI investment rose steadily with periodic peaks.Belgium, France, and Spain showed more modest but consistent growth.Source: Total AI Public Investments · p. 361
• Germany’s AI investment grew substantially in 2019, while the U.K. saw sharp increases in 2021 and 2023.Germany’s growth followed its national AI strategy launch in November 2018; the U.K. increases followed an AI Council strategy proposal.Source: Total AI Public Investments · p. 361

Spending Across Agencies and Sectors

• In 2023, the U.S. Department of Defense received 75.0% of AI contracts, followed by Veterans Affairs at 6.8%.The Department of Defense has received the majority of U.S. AI contracts since 2013.Source: Spending Across Agencies and Sectors · p. 362, Figure 6.3.9 · p. 362
• In Europe, general public services, education, and health accounted for around 84% of public AI investments in 2023, while defense represented 0.84% of AI tenders.The report describes European tender investment as more balanced than U.S. investment, where defense dominates.Source: Spending Across Agencies and Sectors · p. 363, Figure 6.3.10 · p. 363

Highlight: AI Grant Spending in the US

• U.S. AI-related grant spending totaled roughly $19.7 billion from 2013 to 2023.The AI Index identified AI-specific grants using NLP methodologies.Source: Highlight: AI Grant Spending in the US · p. 364, Figure 6.3.11 · p. 364
• Between 2013 and 2023, U.S. AI grant funding grew nearly nineteenfold, from $230 million to $4.5 billion, with 40% average annual growth from 2014 to 2020.The report says this expansion coincided with advances in AI and likely increased demand for public-sector applications.Source: Highlight: AI Grant Spending in the US · p. 364

ACCESS THE PUBLIC DATA

• Although 81% of U.S. CS teachers agreed AI should be part of foundational CS learning, fewer than half of high school CS teachers felt equipped to teach it.The chapter also reports that two-thirds of countries worldwide offer or plan K–12 CS education, twice the 2019 share, while African students face the least access due to schools’ lack of electricity.Source: ACCESS THE PUBLIC DATA · p. 369
• Nearly doubled: U.S. graduates earning master’s degrees in AI from 2022 to 2023.Source: ACCESS THE PUBLIC DATA · p. 369
• The surge in master’s degrees could signal a future trend across degree levels, while increases in bachelor’s and PhD degrees may emerge more slowly.Source: ACCESS THE PUBLIC DATA · p. 369

5. The U.S. continues to be a global leader in producing information, technology, and communications

• The U.S. produces ICT graduates at all levels, with Spain, Brazil, and the United Kingdom following at various levels; Turkey has the best gender parity.Source: §5 · p. 369

7.1 Background

• The chapter distinguishes AI in education (using AI tools), AI literacy (understanding and using AI and its risks), and AI education, which adds technical skills for building AI.Its data focus is AI education; it examines K–12 CS and AI education amid inequities in resources and coursework, while noting that unstandardized data make the problem difficult to estimate. A 2016 federal initiative funded CS education, professional learning, instructional resources, and regional partnerships.Source: §7.1 · pp. 370–371

Foundational Computer Science

• 35% of U.S. high schools offered CS, increasing to 60%; by state, offerings reached 100% in one example, compared with 31% in Montana.Source: Foundational Computer Science · p. 371
• 5.1% of high school students participated in CS in 2020–21, rising to 6.4% in 2023–24; enrollment ranged from 26% in South Carolina to 2% in Florida, Arizona, and Idaho.The participation data covered 41 states.Source: Foundational Computer Science · p. 373, Figure 7.2.6 · p. 373
• Black, Native American/Alaskan, and white students had near or above proportional national representation in CS enrollment, but girls and several other groups remained underrepresented.Underrepresented groups included Hispanic and Native Hawaiian/Pacific Islander students, students with IEPs, FRL-eligible students, and English language learners; data gaps from nine states warrant caution.Source: Foundational Computer Science · p. 374

Advanced Computer Science

• Asian students, white boys, and multiracial students were overrepresented among AP CS exam-takers, while all other student groups were underrepresented.Although AP CS exam participation grew in raw numbers, participation did not match students’ racial and ethnic representation in the general student body.Source: Advanced Computer Science · p. 375

Education Standards and Guidance

• As of January 2025, 26 states had issued guidance on AI in education, while 33 of 44 states with K–12 CS standards had AI-specific standards that were generally minimal and focused on high school.Colorado, Florida, Ohio, and Virginia recently adopted more substantial AI-specific standards spanning K–12 grades.Source: Education Standards and Guidance · p. 378
• U.S. state-adopted K–12 CS standards averaged 97% coverage of the same subconcepts as CSTA standards, indicating strong national coherence in CS instruction.The CSTA standards contain only two advanced high school standards that specifically require AI knowledge, though existing CS standards support foundational AI skills.Source: Education Standards and Guidance · p. 378

Teacher Perspectives

• 81% of CS teachers believe AI belongs in foundational CS education, but only 46% of high school, 44% of middle school, and 34% of elementary school teachers feel equipped to teach it.The survey included 2,901 pre-K–12 CS teachers.Source: Teacher Perspectives · p. 379

Percentage of teachers who feel equipped to teach AI

• Over two-thirds of middle and high school CS teachers said they teach AI, compared with 65% of elementary teachers; most of the 2,245 teachers covering AI spent fewer than five hours per course.Elementary teachers spent the least time, with 70% spending one to two hours; teacher-reported benefits included productivity and differentiated learning, while concerns included misuse, reduced engagement, misinformation, bias, and privacy.Source: Teacher Perspectives · p. 379, Percentage of teachers who feel… · p. 380
• 88% of surveyed CS teachers identified a need for more resources for AI-related professional development, including AI literacy and understanding AI’s ethical impacts.The 2024 survey included 364 CS teachers.Source: Percentage of teachers who feel… · p. 380

Global

• Few countries explicitly include AI education in curricula; CS or ICT education is used as a proxy, but global measures require caution because these subjects can be conflated with digital or computer literacy.Countries more often reference AI education in national strategy discussions without providing detailed implementation plans.Source: Global · p. 381

Access

• In 2024, approximately two-thirds of countries offered or planned CS education, and CS was mandatory in primary and/or secondary schools in 30% of countries.All regions made progress over five years, with Africa and Latin America registering the largest increases, yet African students were least likely to have access.Source: Access · p. 381
• In 2023, only 34% of primary schools in sub-Saharan Africa had electricity, hindering computer literacy instruction and limiting capacity for CS and AI education.Source: Access · p. 381

Guidance

• Globally, countries have developed AI-in-education use guidance faster than national standards for teaching AI; 10 countries had issued such guidance by November 2024.Source: Guidance · p. 382

United States

• Associate CS degree student numbers remained largely stable over the decade, while community colleges began offering AI-related certificates and associate and bachelor’s programs.Source: United States · p. 384, Figure 7.3.2 · p. 385
• Computing bachelor’s graduates increased 22% over the last decade, while CS master’s graduates rose 26% from 2022 to 2023 and 83% over the decade.Source: United States · p. 384, Figure 7.3.1 · p. 385
• Women remain underrepresented in CS degree completion despite graduating from college at higher rates than men.Source: United States · p. 385
• Black students accounted for 8% of computing bachelor’s and master’s degrees and 7% of PhDs; Hispanic students accounted for 13%, 8%, and 4%, respectively.Source: United States · p. 386
• White students accounted for 46% of computing bachelor’s degrees and 52% of PhDs, while Asian students accounted for 23% of bachelor’s, 28% of master’s, and 17% of PhD degrees.Source: United States · p. 386
• In 2023, nonresidents made up 67% of U.S. computing master’s graduates and 60% of PhD graduates.Source: United States · p. 386
• U.S. CS master’s enrollment more than doubled, from 15,811 to 34,850, while students from India and China made up 93% of international master’s and 60% of international PhD students.The international cohorts comprised 95,130 master’s students and 13,070 PhD students.Source: United States · p. 386, Figure 7.3.4 · p. 387
• AI-specific bachelor’s programs nearly doubled between 2022 and 2023, master’s offerings increased sharply, and AI master’s graduates rose sharply from 2022 to 2023.The passages do not specify the size of these increases.Source: United States · pp. 386, 388

Top postsecondary institutions graduating students in AI in 2023 by degree type8

• Carnegie Mellon doubled its AI graduates, while Penn State had its first graduating class in 2022; the institution list counts only programs using an AI-specific CIP code.Students in broader computer science programs may therefore be omitted.Source: United States · p. 388, Top postsecondary institutions… · p. 389, Figure 7.3.8 · p. 389

Global

• In the sampled countries, the U.S. produced more ICT graduates than any other country at associate, bachelor’s, master’s, and PhD levels.It graduated more than twice as many associate, master’s, and PhD students—and nearly twice as many bachelor’s students—as the next-highest country. The dataset excludes countries including India, China, and countries in Africa, and its latest available year is 2022.Source: Global · p. 390
• Women comprise approximately one-quarter of ICT graduates at associate, bachelor’s, and PhD levels, and closer to one-third at the master’s level.Turkey is among the countries closest to parity, with women comprising at least half of graduates at every listed level.Source: Global · p. 392

Guidance

• AI is widely used on campuses, but university guidance remains unclear and inconsistently standardized: 86% of students and 61% of faculty report using AI.By early 2025, 39% of institutions had an AI-related acceptable use policy, up 16 percentage points from 2024; larger universities were more likely than smaller ones to have a policy.Source: Guidance · p. 394

7.4 Looking Ahead

• The report argues that equitable AI education is essential for responsible development and deployment, given harms including political mis/disinformation, AI-enabled weapons, and copyright infringement.It calls for a better approach to building AI.Source: §7.4 · p. 395
• AI education should pair technical creation with ethics, prepare students to mitigate harms, and expand course access and educator upskilling equitably.The report urges academic institutions to develop AI pathways and engage students in building competencies.Source: §7.4 · p. 395

ACCESS THE PUBLIC DATA

• The chapter examines AI sentiment across global, national, demographic, and ethnic perspectives using Ipsos, automobile-association, and local U.S. policymaker surveys.It frames public perceptions as useful for anticipating societal impacts and differences in adoption across countries and demographic groups.Source: ACCESS THE PUBLIC DATA · p. 398
• 52% to 55%: the global share viewing AI products and services as more beneficial than harmful rose from 2022 to 2024; 18 of 26 surveyed nations also saw an increase.Ipsos surveyed the same 26 nations in both years.Source: ACCESS THE PUBLIC DATA · p. 399
• Two thirds of people expect AI products and services to significantly affect daily life within three to five years, up six percentage points since 2022.Every country except Malaysia, Poland, and India recorded an increase; Canada’s increase was 17% and Germany’s was 15%.Source: ACCESS THE PUBLIC DATA · p. 399
• Confidence that AI companies protect personal data fell from 50% in 2023 to 47% in 2024, while belief that AI systems are unbiased and nondiscriminatory also declined.The chapter reports growing skepticism about AI companies’ ethical conduct and declining trust in AI fairness.Source: ACCESS THE PUBLIC DATA · p. 399

4. Regional differences persist regarding AI optimism. First reported in the 2023 AI Index, significant

• 83% in China, 80% in Indonesia, and 77% in Thailand see more benefits than drawbacks from AI products and services, versus 40% in Canada, 39% in the United States, and 36% in the Netherlands.The report describes the first three shares as large majorities and the latter three as minorities.Source: §4 · p. 399

5. People in the United States remain distrustful of self-driving cars. A recent American Automobile Association

• 61% of people in the U.S. fear self-driving cars, while only 13% trust them; fear is down from 68% in 2023 but remains above 54% in 2021.Source: §5 · p. 399

6. There is broad support for AI regulation among local U.S. policymakers

• 73.7% of local U.S. policymakers supported AI regulation in 2023, up from 55.7% in 2022, with stronger support among Democrats than Republicans.Support was 79.2% among Democrats and 55.5% among Republicans.Source: §5 · p. 399

7. AI optimism registers sharp increase among countries that previously showed the most skepticism.

• Optimism about AI products and services rose most in countries that were least optimistic in 2022: by 8% in Great Britain, 10% in Germany, 4% in the U.S., 8% in Canada, and 10% in France.Their 2022 shares viewing AI as having more benefits than drawbacks were 38%, 37%, 35%, 32%, and 31%, respectively.Source: §7 · p. 400
• 60% of global respondents expect AI to change how they do their jobs within five years, while 36% believe it will replace their jobs.Source: §7 · p. 400
• Local U.S. policymakers most supported stricter data privacy rules (80.4%), retraining for unemployed people (76.2%), and AI deployment regulations (72.5%), with less support for a facial recognition ban, wage subsidies, or universal basic income.Support for the latter policies was 34.2%, 32.9%, and 24.6%, respectively.Source: §7 · p. 400
• 55% of global respondents expected AI to save time and 51% expected better entertainment, but only 38% expected improved health and 36% an improved national economy.31% saw a positive impact on the job market, and 37% believed AI would enhance their own jobs.Source: §7 · p. 400

AI Products and Services

• In 2024, 67% of respondents said they understood AI and 66% expected it to profoundly change daily life; the share seeing more benefits than drawbacks rose from 52% in 2022 to 55%.Trust fell by three percentage points for companies using AI to protect personal data and by two points for AI not to discriminate or show bias.Source: AI Products and Services · p. 401
• Views varied by country: 83% of Chinese, 70% of Mexican, and 62% of Indian respondents believed AI would have more benefits than drawbacks, according to the Ipsos survey.Source: AI Products and Services · p. 402
• Trust in AI companies to protect personal data declined, even as global belief that AI-powered products would profoundly change daily life rose by 6%.The belief concerned change within the next three to five years; Canada and Germany recorded the largest increases, at 17% and 15%.Source: AI Products and Services · pp. 405–406
• South Africa and Ireland had the sharpest average increases in AI awareness, trust, and excitement; Ireland also had the largest rise in respondents saying AI products profoundly affected daily life.The report suggests Ireland’s positive trend may be led by positive user experiences.Source: AI Products and Services · p. 405

AI and Jobs

• Overall, 60% of respondents believe AI is likely to change how they do their job in the next five years, and 36% believe AI is likely to replace their current job in the next five years.Source: AI and Jobs · p. 407
• 67% of Gen Z, compared with 49% of baby boomers, agreed that AI would likely affect their current jobs in 2024.Across 2023 and 2024, all generations increasingly agreed that AI would change how they do their jobs over the next five years.Source: AI and Jobs · p. 408

AI and Livelihood

• 55% expected AI to reduce the time needed to get things done and 51% expected better entertainment options, versus 36% expecting economic benefits and 31% expecting job-market benefits.These global responses show more optimism about time and entertainment than about the economy or employment.Source: AI and Livelihood · p. 409
• China and Indonesia were among the most optimistic about AI improving the economy, while fewer than 25% in four named countries expected economic improvement.The shares were 72% in China and 54% in Indonesia; the Netherlands, United States, Belgium, Sweden, and Canada were each below 25%.Source: AI and Livelihood · p. 409
• Respondents who expected AI to improve the job market tended also to believe it would benefit their own jobs.The report describes this relationship as a strong correlation.Source: AI and Livelihood · p. 410

8.2 US Policymaker Opinion

• The section frames policymakers as key stakeholders whose views matter alongside those of the general public when assessing sentiment toward AI.Source: §8.2 · p. 412
• 73.7% of local U.S. policymakers supported government AI regulation in 2023, up from 55.7% in 2022.The report says ChatGPT’s launch appears to have played a key role in shifting sentiment toward regulation.Source: §8.2 · p. 412, Figure 8.2.1 · p. 412
• 79.2% of Democrats supported AI regulation, compared with 55.5% of Republicans.Source: §8.2 · p. 412
• 80.4% supported stricter data privacy rules, 76.2% supported retraining programs, and 72.5% supported AI deployment regulation.Source: §8.2 · p. 413, Figure 8.2.2 · p. 413
• Only 33.9% supported wage subsidies to offset wage declines, less backing than for privacy, retraining, or deployment policies.Source: §8.2 · p. 413
• 56.5% did not expect to need AI-policy action within a few years, versus 34.3% who did; the latter rose from 32.2% in 2022 to 36.6% in 2023.Source: §8.2 · p. 414, Figure 8.2.3 · p. 414
• Only 29.8% of locally elected officials felt adequately informed to make AI-policy decisions, despite a slight confidence increase from 2022 to 2023.Source: §8.2 · p. 415, Figure 8.2.4 · p. 415

AI Publication Analysis

• The AI Index refined OpenAlex-identified computer science publications with the Computer Science Ontology and extended its unsupervised classifier to focus on AI and its subtopics.Source: AI Publication Analysis · p. 418
• Conflicting OpenAlex publication-type metadata were resolved by mapping records to DBLP, first by DOI and then by title and publication year for unmatched papers.Source: AI Publication Analysis · p. 418
• Publication counts use the latest version’s date and author affiliations for geographic attribution, so coauthored papers can count in multiple countries or regions.Source: AI Publication Analysis · p. 419

Top 100 Publications Analysis

• The influential-publications analysis gathered yearly top-150 citation lists from OpenAlex and refined them to 100 after review, using citation data from multiple sources.Source: Top 100 Publications Analysis · p. 419
• Top-publication counts assign papers to every country or region represented by author affiliations, so aggregate totals can overlap.Source: Top 100 Publications Analysis · p. 419

AI Patent Analysis

• AI patents are identified through keyword and classification-code analysis of granted patents from 2010 onward, counted at the DOCDB family level to avoid duplicate inventions.Source: AI Patent Analysis · p. 419
• Patent countries are assigned by the authority of the earliest recorded grant publication, and non-English titles and abstracts are translated using deep-translator.Source: AI Patent Analysis · p. 419
• AI patents are identified by regex keyword searches of titles and abstracts, with a curated dictionary validated through topic modeling, zero-shot classification, and manual checks.The dictionary incorporated AI-generated terms, established lexicons, and Word2Vec-based synonym identification.Source: AI Patent Analysis · p. 419
• The patent dataset combines keyword classification with IPC/CPC code identification, and removes occasional same-family, same-filing-date duplicates in aggregate analytics.The code list was curated using AI-model analysis, regex searches, and prior research.Source: AI Patent Analysis · p. 419

Epoch Notable Models Analysis

• A March 2025 snapshot of landmark-model papers attributes authors to countries by paper affiliation, aggregates contributions over time periods, and compares country trends.For international organizations, authors are assigned to the headquarters country unless a more specific location is indicated.Source: Epoch Notable Models Analysis · p. 420
• The training-cost dataset selects large-scale-era models ranked in the top 10 for training compute at release and adds the Transformer model.Selected models’ training time, hardware type, quantity, and utilization were gathered from publications, press releases, or technical reports.Source: Epoch Notable Models Analysis · p. 420
• Training costs are estimated by multiplying hardware hours by historical cloud rental rates, but estimates are limited by undisclosed training details, unavailable prices, and varying purchase or rental costs.Rates were based on archived vendor pricing for the hardware used, with analogous prices used when exact prices were unavailable.Source: Epoch Notable Models Analysis · p. 420

Identifying AI Projects

• GitHub identifies public AI repositories using AI/ML and generative-AI topic labels, snowball-sampled keywords, and dependencies on selected AI and scientific Python libraries.The method follows Gonzalez, Zimmerman, and Nagappan (2020) and Dohmke, Iansiti, and Richards (2023).Source: Identifying AI Projects · p. 421

Mapping AI Projects to Geographic Areas

• Public AI projects are mapped to geographic areas using IP-address geolocation to determine their modal location.Source: Mapping AI Projects to Geographic Areas · p. 421
• Project locations were assigned from owners’ GitHub-interaction IP locations, using each owner’s modal daily location when it changed during the year.The last known location was carried forward on days without activity.Source: Mapping AI Projects to Geographic Areas · p. 421

Environmental Impact Analysis

• The AI Index estimated training-stage emissions for 50 influential models with a Lacoste et al. (2019) calculator, excluding embodied hardware, idle infrastructure, and deployment emissions.Calculator inputs included hardware type, GPU hours, provider, and compute region.Source: Environmental Impact Analysis · p. 421

Conference Submissions Analysis

• Responsible-AI academic submissions were counted across AAAI, AIES, FAccT, ICML, ICLR, and NeurIPS by scraping conference materials and manually verifying keyword-identified papers.A paper could belong to multiple responsible-AI categories.Source: Conference Submissions Analysis · p. 429

Accenture Global State of Responsible AI Survey

• The second Global State of Responsible AI survey collected responses from 1,500 organizations with at least $500 million in revenue across 20 countries and 19 industries.Conducted in January–February 2025, it examined organizational and operational activities across 10 responsible-AI dimensions over time.Source: Accenture Global State of Responsible… · p. 429

McKinsey Responsible AI Survey

• McKinsey assessed enterprise responsible-AI maturity with a framework covering four dimensions, 21 subdimensions, and four maturity levels.The levels ranged from foundational practices to a comprehensive and proactive program.Source: McKinsey Responsible AI Survey · p. 430

International Federation of Robotics (IFR)

• Robot-installation data came from the World Robotics 2024 report.Source: International Federation of Robotics… · p. 433

Prepared by Vishy Kamalapuram and Elena Magrini

• Lightcast’s AI technology analyzes hundreds of millions of job postings and real-life career transitions for labor-market insights.Source: Prepared by Vishy Kamalapuram and… · p. 433

Job Postings Data

• Lightcast mines millions of postings collected since 2010 from over 51,000 online job sites, removes duplicates, and extracts structured labor-market information.Extracted fields include job title, employer, industry, region, required experience, education, and skills.Source: Job Postings Data · p. 433
• Over 99% of total labor demand is represented in Lightcast data compared with JOLTS; unposted jobs are usually at small businesses or union hiring halls.The excluded examples include “Help Wanted” signs in restaurant windows and union hiring halls.Source: Job Postings Data · p. 433

Measuring Demand for AI

• Lightcast’s taxonomy of over 33,000 skills.1 is organized into over 400 clusters and 32 categories, and any posting mentioning a listed AI skill counts as an AI job.The listed skills span areas including AI ethics, autonomous driving, generative AI, and machine learning.Source: Measuring Demand for AI · pp. 433–434
• The natural language processing category includes terms spanning AI copywriting, chatbots, BERT, Seq2Seq, text mining, and Word2Vec models.Other listed terms include summarization methods and natural language generation.Source: Measuring Demand for AI · pp. 434–435
• The neural networks category lists artificial neural networks, convolutional neural networks, graph neural networks, reinforcement learning, and TensorFlow.The list also includes Apache MXNet, Chainer, and Keras.Source: Measuring Demand for AI · p. 435
• The robotics category includes advanced robotics, motion planning, Nvidia Jetson, OpenAI Gym environments, and robotic automation software.It also lists cognitive robotics and robot operating systems.Source: Measuring Demand for AI · p. 435
• The visual image recognition category includes 3D reconstruction, image captioning, object recognition, and pose estimation.The list also names computer vision and image analysis.Source: Measuring Demand for AI · p. 435

LinkedIn Data

• LinkedIn’s aggregated, anonymized profile data reflects how members use the platform and its regional availability, and strict quality thresholds protect individual privacy.LinkedIn describes the data as drawn from more than 1 billion members worldwide.Source: LinkedIn Data · p. 435

Skills / Industry

• LinkedIn classifies AI skills as AI Engineering or AI Literacy and analyzes AI in education, financial services, manufacturing, professional services, and technology, information, and media.Engineering covers competencies for designing, developing, deploying, and maintaining AI systems; literacy covers understanding, evaluating, and interacting with AI technologies.Source: Skills · p. 435, Industry · p. 436

Gender / AI Jobs or Occupations / AI Talent

• AI talent means a member has added at least two AI skills to their profile or has been employed in an AI job; AI jobs are occupations requiring AI skills.Gender analysis infers gender from profile pronouns or first names when not self-identified, excludes members whose gender cannot be inferred as male or female, and filters countries without sufficient algorithm coverage.Source: AI Talent · p. 436, AI Jobs or Occupations · p. 436, Gender · p. 436

1. Top AI Skills

• Machine learning, AI, and deep learning are the most frequently added AI Engineering skills globally since 2015.LinkedIn members self-report skills, drawn from more than 41,000 standardized skills and grouped into mutually exclusive AI Engineering and AI Literacy categories.Source: §1 · p. 436

2. Fastest Growing AI Skills

• Custom GPTs, AI productivity, and AI agents are the fastest-growing AI Engineering skills globally.LinkedIn applies thresholds to recent-year skill-add volumes, set at the 50th percentile of each country’s most recent AI skill-add distribution.Source: §2 · p. 436

3. AI Talent Concentration

• 0.78% of LinkedIn members in the United States are AI talent with AI Engineering skills.LinkedIn calculates country-level concentration as AI talent counts divided by LinkedIn member counts; coverage may affect the metric, so it should be used with caution.Source: §3 · p. 436

4. Relative AI Talent Hiring Rate YoY Ratio

• 24.7% year-over-year growth in the U.S. ratio of AI talent hiring relative to overall hiring is reported.The ratio compares year-over-year change in the AI hiring rate with the overall hiring rate in the same country.Source: §4 · p. 437

SKILLS GENOME / AI SKILLS PENETRATION

• The skills genome ranks 50 category-characteristic skills using TF-IDF, and the AI skills penetration indicator aims to measure AI skills intensity.TF-IDF down-ranks ubiquitous skills and up-ranks skills distinctive to a category.Source: SKILLS GENOME · p. 437, AI SKILLS PENETRATION · p. 437

AI SKILLS PENETRATION

• LinkedIn measures AI skills penetration as the share of an entity’s top 50 representative skills that belong to the AI skill group.It computes frequencies for members’ self-added skills from 2015 onward, reweights them using TF-IDF, and selects the top 50 skills.Source: AI SKILLS PENETRATION · p. 437

RELATIVE AI SKILLS PENETRATION; GLOBAL COMPARISON

• Relative AI skills penetration compares a country with a benchmark, controlling for occupations; the global comparison uses average global penetration across overlapping occupations.The country-to-benchmark ratio is constructed from skills genomes, and the global comparison uses a sample of countries.Source: RELATIVE AI SKILLS PENETRATION · p. 437, GLOBAL COMPARISON · p. 437

GLOBAL COMPARISON: BY INDUSTRY

• The industry comparison decomposes countries’ relative AI skills penetration across sectors, comparing against the global average for the same occupations in each sector.Source: GLOBAL COMPARISON: BY INDUSTRY · p. 438

GLOBAL COMPARISON: BY GENDER; GLOBAL COMPARISON: ACROSS GENDERS

• Gender-specific penetration supports country rankings within each gender, while the across-genders measure enables comparisons between genders using the same global average.The by-gender metric should not be used for cross-gender comparisons because its global averages differ by gender.Source: GLOBAL COMPARISON: ACROSS GENDERS · p. 438, GLOBAL COMPARISON: BY GENDER · p. 438

6. Female Representation in AI

• 30.5% of global AI talent with AI Engineering skills are women.The passage describes this as female representation within AI talent with AI Engineering skills.Source: §6 · p. 438

7. AI Talent Migration

• 1.07 net flow was reported for the United States, which had positive net AI talent flow relative to its membership size.Net flows are arrivals minus departures and are normalized by country membership; LinkedIn notes that membership varies between countries, complicating interpretation of absolute movements.Source: §7 · p. 438

8. Career Transitions Into AI Jobs

• 26.9% of U.S. transitions into AI engineer came from software engineer.Transitions are pooled over five years; LinkedIn excludes first occupations added by new graduates and intra-occupation transitions.Source: §8 · pp. 438–439

AI Engineering

• The AI Engineering vocabulary spans methods and applications including generative AI, deep learning, image generation, and large language models.The listed terms also include tools, model operations, optimization, and other machine-learning approaches.Source: AI Engineering · p. 439

AI Literacy

• The AI Literacy vocabulary names generative AI tools and models, including ChatGPT, Claude, DALL-E, GitHub Copilot, Gemini, and Stable Diffusion.It also includes AI prompting, prompt engineering, multimodal prompting, and generative art.Source: AI Literacy · p. 439

Quid insights prepared by Heather English and Hansen Yang

• Quid combines its in-house LLM and smart search features with Boolean queries to search topics and keywords across diverse datasets and provide visualization and data-delivery options.Its sources include social media, news, forums, blogs, companies, patents, and custom data feeds.Source: Quid insights prepared by Heather… · p. 440
• Quid indexes over 8 million global public and private company profiles, including descriptions and metadata, and updates company information weekly.Metadata includes investment and firmographic information such as founding year and headquarters location.Source: Quid insights prepared by Heather… · p. 440
• Quid links documents by similar language at scale, producing clusters of distinct topics and identifying trends through keywords, phrases, people, companies, and institutions.The report says these patterns help users make data-driven decisions accurately, quickly, and efficiently.Source: Quid insights prepared by Heather… · p. 440

Companies

• Quid combines Capital IQ data with Crunchbase additions to cover global organizations, including early-stage startups and funding events, but undisclosed investor names or amounts remain unavailable.Source: Companies · p. 440

Search Parameters

• Searches use Boolean terms across the company database, business descriptions, and websites, with filters for headquarters region, investment amount, operating status, organization type, and founding year.Source: Search Parameters · p. 440
• When results exceed 7,000 companies, Quid visualizes the 7,000 most relevant according to semantic similarity and its language algorithm.Source: Search Parameters · p. 440

Target Event Definitions

• A private placement sells newly issued equity or debt to selected investors and is usually a minority stake, though it can confer control and constitute a majority-stake investment.Source: Target Event Definitions · p. 440
• Quid defines minority investment as acquiring under 50% of an existing stake and M&A as acquiring more than 50%.Source: Target Event Definitions · p. 441

McKinsey & Company

• The February 22–March 5, 2024 survey received 1,363 responses, while the July 16–31 survey received 1,491 responses from 101 nations.Source: McKinsey & Company · p. 441
• McKinsey survey data was weighted by each respondent nation’s contribution to global GDP, and the Index also used data from earlier survey iterations.Source: McKinsey & Company · p. 441

FDA-Approved AI Medical Devices

• The analysis combined FDA-listed AI/ML medical-device data with a PubMedCentral search of 2020–2024 English-language articles on AI, medicine, and ethics; 2,916 qualifying articles remained for ethical-issue frequency analysis.Source: FDA-Approved AI Medical Devices · p. 443
• The medical AI ethics search included 2,916 articles whose abstracts discussed medicine, artificial intelligence, and at least one ethical issue.Source: FDA-Approved AI Medical Devices · p. 444

Global Legislation Records on AI

• The global legislation review searched full-text bills for “artificial intelligence” in local languages across 116 geographic areas, counting laws passed and signed from 2016 to 2024.For amended laws, the year AI-related provisions were added or amended was used when relevant.Source: Global Legislation Records on AI · p. 453

US AI Regulation

• A substantial portion of U.S. government AI spending occurs through grants, but grant-level spending data cover only the United States.The analysis includes government contract data from various countries; comparable grant data were difficult to collect for other regions, including the European Union and China.Source: US AI Regulation · p. 454

Data Sources

• The cited study lists Tenders Electronic Daily (TED) among its data sources, identifies TED among United Kingdom sources, and says U.S. data were sourced publicly; data quality can vary.Source: Data Sources · pp. 454–455

Data Processing

• TED processing reconciled inconsistent contract descriptions, translated multilingual text into English, applied NLP techniques, and converted monetary amounts to U.S. dollars using purchasing power parities.NLP steps included stop-word and special-character removal, part-of-speech tagging, lowercasing, lemmatization, and replacing numerical measures with a <NUM> tag.Source: Data Processing · p. 455

Classification

• Regex searches and an expanded AI vocabulary identified contracts; BERTopic and DeBERTA assessed matches, with scores below 20% and all high-value tenders manually reviewed.The vocabulary used Word2Vec-suggested similar terms that were manually reviewed before inclusion.Source: Classification · p. 455

State-Level Data

• State-level data methodology is described in Appendix 2 of the State of Computer Science Education 2024 report, and Code.org maintains a state K–12 education database.The methodology was used by Code.org, CSTA, and ECEP Alliance.Source: State-Level Data · p. 456

AP Computer Science Data

• AP Computer Science data was provided to Code.org under its agreement with the College Board and came from the Board’s national and state summary reports.Source: AP Computer Science Data · p. 456

Access to Computer Science Education

• Computer science access data came from the State of Computer Science Education 2024 report and the 2024 K-12 Computer Science Landscape Teacher Landscape Survey.Source: Access to Computer Science Education · p. 456

State Standards Comparison

• The state-standards dataset contains approximately 10,000 adopted K-12 standards but excludes Colorado and Virginia standards adopted in late 2024.Source: State Standards Comparison · p. 456

Global K-12 AI Education

• The University of Cambridge compiled the global K-12 AI education dataset, extending Brookings Institution research and clarifying that U.S. computer science is not available everywhere.The clarification specifies availability in some schools or districts, rather than everywhere as an elective course.Source: Global K-12 AI Education · p. 456
• The Global K-12 AI Education methodology is referred to in its report.Source: Global K-12 AI Education · p. 456

IPEDS

• IPEDS combines annual U.S. Department of Education NCES surveys and gathers information from colleges and other institutions participating in federal student financial aid programs.Source: IPEDS · p. 456

Completion Data

• The Completions survey counted students completing postsecondary programs, with AI-related graduates identified by first majors in three specified CIP fields.The fields were Computer and Information Sciences, General (11.01), Computer Programming (11.02), and Computer Science (11.07).Source: Completion Data · p. 456
• The 2025 report’s AI-related graduate count differs from previous years because it used multiple CIP codes.Source: Completion Data · p. 456

OECD

• The OECD data came from its table on enrolled students, graduates, and new entrants by education field; methodological details are in Education at a Glance 2024 technical notes.Source: OECD · p. 456

Ipsos

• For brevity, the report did not republish the Ipsos survey methodology and directs readers to the survey for details.Source: Ipsos · p. 457