Source-linked AI summary
From AGI to ASI
Tim Genewein, Matija Franklin, Alexander Lerchner, Laurent Orseau, Samuel Albanie, Adam Bales, Cole Wyeth, Stephanie Chan, Iason Gabriel, Joel Z. Leibo, Allan Dafoe, Marcus Hutter, Thore Graepel, Shane Legg
TL;DR
The report asks how AI might progress beyond human-level AGI and maps possible routes and frictions on the path to ASI. It concludes that large AGI collectives may achieve broadly superhuman capabilities, and that progress into ASI within the next decade or two cannot be dismissed.
Problem
The report addresses uncertainty about AI progress beyond human-level AGI by mapping potential pathways and frictions that could accelerate, slow, or halt progress.
Source: §2 · pp. 2, 4Method
The report characterizes intelligence on a continuum and synthesizes four pathways from AGI to ASI, including scaling, paradigm shifts, recursive improvement, and multi-agent collectives.
Source: §2 · p. 2, §3 · p. 6, §5.2 · p. 17Results
Large enough groups of human-level AGI would likely produce broadly superhuman capabilities, making progress into ASI within the next decade or two difficult to dismiss.
Source: §7.2 · p. 38Takeaways & Limitations
Preparing for a post-AGI world requires diverse forecasts and scenarios alongside continual benchmarking and monitoring of advanced AI systems.
Source: §7.2 · p. 37Current models lack grounded discovery of novel concepts, so benchmark saturation may reflect mastery of human-defined knowledge rather than rapidly surpassing human reasoning.
Source: §5.5 · p. 26
Abstract
from arXiv · showhide
Over the last decade, building human-level artificial general intelligence has moved from far-fetched speculation to being a concrete next-decade target for many of the largest AI organisations. Achieving this goal would have profound and far-reaching impacts on human society, which raises many complex questions for the decade ahead. This report investigates how AI itself might continue to develop in a post-AGI world along the continuum of machine intelligence. The endpoint of this continuum, Universal AI, is theoretically well understood, which provides some formal grounding for the main focus of this report: the transition from human-level AGI to artificial general superintelligence, which, intuitively, can be understood as a system that is more intelligent and cognitively capable than large organisations of humans. After characterizing ASI, the report discusses four potential pathways from AGI to ASI: scaling AGI, AI paradigm shifts, recursive improvement, and ASI emerging from large-scale multi-agent collectives. The report then discusses possible frictions and bottlenecks along these pathways. Determining whether the impact of these frictions will be negligible or substantial raises a number of concrete open research questions. Due to large uncertainties for predicting ASI progress, it cannot be ruled out that AI progress might continue to accelerate over the next years. This could imply that the image of a single transformative step change, caused by the introduction of human-level AGI into our society, could be inaccurate. More apt might be the prospect of a series of transformative societal changes caused by AI-enabled progress and breakthroughs across many areas of science and technology. Preparing for this prospect requires a massively interdisciplinary endeavour of global scope and interest.
1. Summary Instructions
The report summary should frame AGI and ASI, explain compute-dependent advantages and four potentially parallel pathways from AGI to ASI, and cover pathway-specific frictions and open questions. It should also place the report in current perspective by reviewing subsequent developments, friction estimates, research progress, and acknowledged limitations.
- Define AGI and ASI informally, then describe the advantages of digital intelligence that increase with more compute.These elements establish the report’s conceptual frame and should remain distinct rather than being compressed into fewer points.Source: §1 · p. 2
- Summarize four pathways from AGI to ASI, noting that they are not mutually exclusive and may occur in parallel.The pathways should be presented as separate possibilities rather than collapsed into a single route.Source: §1 · p. 2
- List all potential frictions and explain how each would or would not affect each pathway, emphasizing that their impacts generate open research questions.The report’s additional open questions and research directions should also be included.Source: §1 · p. 2
- Put the report into current perspective by highlighting significant progress on the research questions it raised.The summary should reference works that constitute significant progress since publication.Source: §1 · p. 2
- Conclude with developments on each pathway, associated research questions, updated friction significance and impact estimates, and widely accepted shortcomings, caveats, and oversights.This conclusion should explicitly address how the report’s claims and uncertainties look after publication.Source: §1 · p. 2
2. Introduction: Life as we don’t know it?
The report examines AI progress beyond human-level AGI, mapping pathways toward ASI, frictions that could slow progress, and the open research questions needed to assess them. Rapid, uncertain growth in effective compute, capabilities, and recursive improvement makes forecasting future AI progress difficult and potentially urgent.
Report goal
- The report maps technological pathways for AI progress beyond human-level AGI and identifies frictions that could slow or halt them, generating concrete open research questions.Its scope is AI progress beyond AGI, independent of when that milestone occurs.Source: §2 · p. 2
AGI and society
- AGI may arrive within the next decade and could cause radical societal change, although it might instead resemble a disruptive but socially adaptable “normal technology.”The report frames this uncertainty across the economy, work, education, science, politics, social interaction, and culture.Source: §2 · p. 2
Rates of progress
- Effective compute has grown roughly 10× per year through hardware improvements, hardware investment, and algorithmic efficiency, though the estimate is uncertain and may be higher.The cited component rates are about 1.5× yearly for hardware efficiency, 2.5× for compute investment, and 3× or more for algorithmic efficiency.Source: §2 · p. 3
Scaling and capability growth
- Continued exponential effective-compute growth could expand training, inference, test-time reasoning, and agent groups even if frontier-model progress eventually plateaus.The report illustrates this with 1,000 AGI instances potentially becoming 100 million after five years at 10× annual growth.Source: §2 · p. 4
Recursive improvement
- Recursive AI-assisted research could produce super-exponential or hyperbolic growth, but frictions and boundary conditions may instead yield an S-shaped trajectory.Tracking research automation, recursive-improvement indicators, and emerging bottlenecks could improve forecasting and potentially help steer progress.Source: §2 · pp. 4–5
Navigating uncertainty
- Forecasting AI progress remains highly uncertain because accelerating AI contributions may be offset by rising research effort and economic inputs, requiring models, indicators, ensembles, and continual revision.The report expects measuring, modelling, and forecasting AI progress to become a substantial ongoing research activity.Source: §2 · pp. 5–6
3. Characterizing Artificial Superintelligence
The report defines AGI as roughly median human-level general intelligence and ASI as intelligence that substantially exceeds large human-expert collectives across virtually all domains. It grounds this continuum in Legg-Hutter intelligence while emphasizing digital intelligence’s scaling advantages and ASI’s fundamental limits.
Formal grounding
- The Legg-Hutter score provides formal grounding by treating intelligence as average performance across all computable tasks, allowing coarse AGI–ASI thresholds on a continuous scale.Universal AI, formalized through AIXI, is the theoretical endpoint of this continuum but is incomputable and can only be approximated from below by increasingly capable ASIs.Source: §3 · pp. 6–7
Definitions
- AGI denotes roughly median human-level intelligence, whereas ASI denotes superhuman general intelligence across virtually all tasks and domains, not narrow-domain excellence.Systems such as AlphaFold and AlphaGo are therefore not ASIs despite superhuman performance in single domains.Source: §3 · pp. 6–7
Scope and limitations
- The human-relative definitions are moving-target approximations, so the report sets ASI’s bar as a clear step change above contemporary median human performance.A system performing at broad human-expert level without exceeding it may occupy an intermediate category, which the report does not distinguish further.Source: §3 · pp. 7–8
Advantages of digital intelligence
- Digital intelligence can be copied, accelerated, paused, transferred across hardware, and deployed in parallel, causing its advantages over biological intelligence to widen with effective compute.These properties may enable rapid cultural evolution, high-bandwidth knowledge sharing, flexible embodiment, and large adaptive AI collectives, though their social organization remains uncertain.Source: §3 · pp. 8–10
Fundamental limits
- ASI is neither omniscient nor omnipotent: achievable intelligence remains constrained by diminishing returns, scaling limits, and fundamental physical and complexity-theoretic bounds.The AIXI framework formally characterizes some limits, including maximal data efficiency, while the report discusses additional theoretical limits in Table 2.Source: §3 · pp. 7, 10
4. Universal AI — An Informal Overview
Universal AI, formalized by AIXI, provides the strongest known theoretical upper bound on machine intelligence by modeling an agent that optimally acts across all computable environments under a universal prior. However, AIXI is incomputable and remains difficult to translate into practical algorithms, so whether current AI can scale toward ASI remains unresolved.
AIXI framework
- AIXI models sequential interaction with unknown computable environments by Bayesian belief updating, long-horizon decision-making, and implicit exploration–exploitation balancing.Its hypotheses are weighted by Solomonoff’s Universal Prior, while discounting or a finite horizon specifies how near- and long-term rewards are traded off.Source: §4 · pp. 11–12
- AIXI maximizes expected cumulative reward over computable environments weighted by the universal prior, while Solomonoff Induction provides average-case data-efficiency guarantees.This optimality is prior-weighted rather than a guarantee of highest reward in every individual environment.Source: §4 · p. 12
Universal intelligence measure
- The Legg–Hutter score formalizes universal intelligence as expected reward across computable environments and tasks, with AIXI as its unattainable upper bound.Neither AIXI nor the associated intelligence measure is computable, although approximations can improve with additional compute and runtime at impractical resource costs.Source: §4 · pp. 12–13
Implications for current AI
- AIXI supports the possibility that pretraining, amortized inference, and explicit planning could scale current AI toward universal intelligence and potentially ASI without a known fundamental theoretical blocker.This conjecture depends on sufficient model expressivity and powerful optimizers, while current systems still show limitations in continual learning, very long-context tasks, and robust planning.Source: §4 · p. 13
Shortcomings and open problems
- AIXI’s incomputability, non-embedded formulation, and reliance on average performance across all computable worlds leave major gaps between Universal AI theory and practical deep learning.Embedded multi-agent extensions address some theoretical issues, but practical ASI may emerge before the theoretical foundations are fully unified.Source: §4 · pp. 13–14
Alternative theoretical frameworks
- Alternative frameworks—including reflective oracles, logical induction, Gödel machines, learning theory, algorithmic game theory, and thermodynamic approaches—provide complementary perspectives on intelligence, strategic interaction, generalization, and physical limits.These frameworks address aspects not fully captured by Universal AI, such as self-reference, sample complexity, multi-agent coordination, and energy costs.Source: §4 · p. 14
5. Technological Pathways and Potential Bottlenecks to ASI
This section examines four potentially parallel pathways from AGI to ASI—scaling, algorithmic shifts, recursive improvement, and multi-agent coordination—and the potential capability gains they may enable. It also considers bottlenecks across technical, economic, hardware, conceptual, research, and governance domains, emphasizing open questions about their significance.
5. Technological Pathways and Potential Bottlenecks to ASI
- Four largely independent pathways may progress in parallel: scaling, algorithmic paradigm shifts, recursive improvement, and multi-agent coordination, with bottleneck significance remaining uncertain.Table 3 summarizes the pathways, while Table 4 discusses potential frictions and countervailing factors.Source: §5 · p. 14, Table 3 · p. 15
5.1. Scaling compute, models, and data
- Scaling may reach ASI through additional orders of magnitude of effective compute and model size, but intelligence gains depend on search efficiency, data quality, and sustained resources.Compute-optimal scaling and co-scaling models with high-quality data and compute may matter more than model size alone; whether scaling is sufficient remains open.Source: §5.1 · pp. 15–16
- Scaling faces a near-term data bottleneck as high-quality text may be exhausted later this decade, although synthetic data, other modalities, and sparse architectures could extend progress.Model-generated data may avoid degeneration if AGI systems produce high-quality simulations or search-augmented data, while sparse Mixture-of-Experts can improve compute efficiency.Source: §5.1 · p. 16, §5.5 · p. 21
5.2. Algorithmic paradigm shifts and evolutions
- Algorithmic evolutions may add continual learning, dynamic test-time computation, retrieval-based working memory, long-context processing, tool use, and world-model-based planning.True paradigm shifts could instead introduce novel architectures or optimization procedures that improve data or energy efficiency or overcome complexity limitations, but their effects are difficult to forecast.Source: §5.2 · p. 17
5.3. Recursive self-improvement
- Recursive self-improvement could accelerate AI R&D through code, hardware, data, and specialization improvements, potentially producing rapid or even explosive AGI-to-ASI transitions.AlphaZero-style search distillation, self-play, automated research, hardware design, and agent specialization illustrate distinct improvement mechanisms.Source: §5.3 · pp. 18–20
- Recursive improvement may also fizzle out or become resource-limited: experiments, physical manipulation, and growing resource consumption can constrain acceleration despite increasingly autonomous AI research.Understanding these dynamics may require recursive-improvement scaling laws that predict future curves from early performance trends.Source: §5.3 · pp. 19–20
5.4. Multi-agent coordination & group agency
- Coordinated AGI collectives could become superintelligent through specialization, delegation, high-bandwidth communication, market coordination, or centralized planning, with capabilities potentially scaling linearly or superlinearly.Such systems might solve resource-allocation and discovery problems at unprecedented speeds while bypassing individual architectural bottlenecks.Source: §5.4 · pp. 20–21
5.5. Potential Bottlenecks to ASI
- Key open questions concern whether bottlenecks are temporary frictions or fundamental plateaus, including data scarcity and whether generated data can train subsequent frontier models without degeneration.Human-only production may not supply enough novel data, while naive iterative training on self-generated data can plateau or degenerate.Source: §5.5 · p. 21
- Data scarcity may slow progress but need not fundamentally block it if compute scales test-time improvement, simulations, and agentic interaction data alongside models.High-quality generations can be distilled into improved base models, while simulated and multi-agent interaction data can expand rapidly with compute.Source: §5.5 · p. 23
- Scaling-dependent progress could face major economic, environmental, and hardware bottlenecks, including sustained infrastructure costs, resource impacts, memory-bandwidth limits, and interconnect constraints.If algorithmic innovation or paradigm shifts dominate, economic inputs may scale more slowly than capabilities; otherwise, scaling costs could become decisive.Source: §5.5 · pp. 23–24
- Pure transformer pretraining may be insufficient for ASI, although incremental additions could extend the current paradigm; unresolved hallucination, prompt-injection, uncertainty, and decision-making problems might instead require major changes.Whether these issues remain manageable through smooth paradigm evolution or become fundamental blockers is unclear.Source: §5.5 · pp. 24–25
- Research may become more resource-intensive, but automating cognitive research could offset this slowdown by rapidly multiplying artificial researchers, while experiments and data collection remain field-dependent constraints.The report contrasts declining productivity per human researcher with the possibility of increasing artificial-researcher compute stock much faster and more cheaply.Source: §5.5 · p. 25
- The Abstraction Barrier could limit systems trained on human-generated concepts to human-defined conceptual frameworks, requiring grounded concept discovery and physical interaction for true ASI.If novel concepts must be validated against reality, temporal, physical, and complexity constraints could slow recursive improvement to empirical-science rates; collective multi-agent ASI might nevertheless remain possible.Source: §5.5 · pp. 25–26
- Regulation, deliberate slowdown, accidents, and societal backlash could make further scaling politically, legally, or commercially infeasible even when it remains technically and economically achievable.Visible harms or near misses may shift public preferences, liability, licensing, evaluations, reporting requirements, and capability thresholds, while international coordination remains difficult.Source: §5.5 · pp. 27–28
- National economic and military competition may override governance-based slowdowns by systematically favoring actors that adopt productivity- and power-enhancing technologies.Competitive selection among states can channel technological development toward performance regardless of implications for human welfare.Source: §5.5 · p. 28
6. Remarks
The section argues that ASI progress, capabilities, creativity, goals, and societal impact remain difficult to predict, with quantitative scaling eventually constrained by diminishing returns and qualitative bottlenecks. It identifies empirical benchmarking, multi-agent scaling, alignment, and operational definitions of progress as central open challenges.
Scaling to ASI
- Quantitative compute scaling may reach ASI in theory, but practical systems face search costs, inductive-bias limits, diminishing returns, and the need for qualitative innovation.Large AGI collectives could nevertheless achieve superhuman performance through scale, flexible coordination, and high bandwidth, although the relevant tasks and coordination methods remain open questions.Source: §6 · pp. 28–29
Predicting capabilities
- ASI capabilities cannot be reliably predicted from current benchmarks or theory alone because approximate solutions may evade formal limits and their quality can itself be computationally irreducible.Scaling laws and benchmark stitching offer empirical forecasting tools, but benchmark saturation motivates evaluations of true generalization, private tests, continuous adversarial evaluation, and multi-agent scaling laws.Source: §6 · pp. 29–30
Creativity
- Current AI mainly exhibits combinational and exploratory creativity, whereas transformative creativity would require creating new conceptual spaces, such as revolutionary scientific theories.Transformative artistic creativity additionally requires deep understanding of human culture because artistic value is socially constructed and subjective.Source: §6 · p. 31
Progress
- Recognizing ASI and measuring progress require better general-superhuman benchmarks and clearer operational definitions spanning AI capability, research and technological impact, and societal benefit.AI capability progress includes effectiveness and efficiency, while broader progress may involve economic compression, autonomy, human flourishing, dignity, and context-dependent social judgments.Source: §6 · pp. 31, 33
Goals and agency
- ASI goals and agency are uncertain: instrumental convergence and autonomy can increase risks, while oracles, myopic systems, and knowledge-seeking objectives offer alternatives without eliminating fundamental safety challenges.Knowledge seeking may improve robustness to delusions, avoid stagnation, discourage irreversible changes, and favor cooperation; however, persistent-world oracles can still acquire incentives for control and manipulation.Source: §6 · pp. 32–33
7. Outlook: Plenty That Needs To Be Done
The report frames post-AGI technological pathways and frictions as highly uncertain, incomplete open research programs requiring continued updating and interdisciplinary effort. Preparing for post-AGI developments therefore requires diverse scenarios, stronger forecasting and benchmarking, and sustained research into AI’s technological and societal impacts.
7. Outlook: Plenty That Needs To Be Done
- The report’s pathway and friction analysis is highly uncertain and likely incomplete, so it should be treated as an open research agenda requiring future research and updates.The agenda spans bottlenecks, forecasting, benchmarking, recursive improvement, multi-agent scaling, theoretical foundations, and AI safety, alignment, and sociocultural questions.Source: §7 · pp. 33–34, §7.1 · pp. 34–37
7.1. From AGI To ASI: A Research Agenda
- Quantitative forecasting should couple effective compute, AI capabilities, and macroeconomic effects, using ensembles, scenario simulations, threshold detection, and continual model updating.Candidate quantities include cost per FLOP, compute efficiency, and sector-specific AI productivity; examples include Epoch’s GATE model (Erdil et al., 2025) and Davidson et al. (2026).Source: §7.1 · p. 35
- Advanced-AI preparedness also requires scalable benchmarks beyond human-level saturation, research on recursive and multi-agent improvement, stronger theoretical foundations, and continued alignment and societal-impact work.These programs address capability measurement, group intelligence, limits below Universal AI, robust deployment, scientific norms, and economic effects.Source: §7 · p. 34, §7.1 · pp. 35–37
7.2. Conclusions
- Post-AGI preparedness requires diverse forecasts and scenarios, continual benchmarking and monitoring, and timely policy-response capacity rather than reliance on one technological trajectory.The report identifies landscape mapping and more reliable post-AGI forecasting and benchmarking as major levers for reducing uncertainty.Source: §7.2 · p. 37
- AI progress may plateau before AGI, transition smoothly from AGI to weak ASI, or accelerate rapidly through recursive self-improvement; cruising from AGI into ASI within the next decade or two cannot be dismissed.Large AGI collectives may produce broadly superhuman capabilities even if individual model progress stalls, while an intelligence explosion could make the transition rapid.Source: §7.2 · p. 38
AI Use
The document was predominantly written by humans, with limited language-model assistance for editing, drafting, review, literature work, and bibliography cleanup.
AI Use
- More than 90% of the document was written from scratch by humans, while language models assisted with wording and drafting in less than 10%.Language models also supported manuscript structuring, completeness checks, simulated critical reviews, literature reviews, and bibliography cleanup.Source: AI Use · p. 39
A. Summary
The report examines four non-exclusive pathways from AGI to ASI—scaling, algorithmic paradigm shifts, recursive improvement, and multi-agent collectives—alongside their potential bottlenecks. Because progress may compound across pathways but remains highly uncertain, the report emphasizes improved forecasting, benchmarking, and foundational research.
Overview
- Four non-exclusive pathways could drive progress from AGI to ASI: scaling compute, models, and data; paradigm shifts; recursive improvement; and group-agent formation.Progress may occur simultaneously across pathways, potentially producing compounding rather than merely additive increases in AI capability.Source: A. Summary · p. 54
1. Scaling compute, models & data
- Scaling currently appears the most promising pathway, but economic, hardware, resource, data, and paradigm limits could constrain continued exponential growth.Internet-scale data sources are nearing exhaustion, while the adequacy of synthetic or interactive data and today’s paradigm for AGI or ASI remains unclear.Source: A. Summary · pp. 54–55
Uncertainty and research priorities
- Universal AI provides theoretical limits on data efficiency and capability per compute, yet practical limits and digital intelligence’s advantages make forecasting advanced AI progress unreliable.The report therefore calls for intensified research to reduce uncertainty, including forecasting models, beyond-human benchmarking, scaling laws, and methods for steering large agent groups.Source: A. Summary · pp. 54–56
2. Algorithmic paradigm shifts
- Algorithmic paradigm shifts could sustain progress after scaling limits, but may require substantial investment to recognize and integrate, while research may become harder.Preparedness requires paradigm-agnostic understanding and methods for identifying whether novel paradigms shift practical limits and narrow the gap to fundamental limits.Source: A. Summary · pp. 55–56
3. Recursive (self-) improvement
- Recursive AI improvement could create self-accelerating capability growth, but time, compute, energy, investment, diminishing returns, and self-generated-data degeneration may limit it.Research should characterize recursive mechanisms, formulate scaling laws, and measure how much AI facilitates AI research and the extent of human involvement.Source: A. Summary · p. 56
4. ASI via group agent formation
- AI collectives may become more capable through parallelization and skill diversity, but orchestration, bureaucracy, compute, energy, and economic costs could produce diminishing returns.Key uncertainties include which problems benefit, how collectives should be organized, and how multi-agent efficiency compares with enlarging individual models.Source: A. Summary · p. 56
B. Glossary
The glossary defines the report’s core intelligence concepts, theoretical limits, scaling dynamics, and mechanisms for AI progress. It also specifies key bottlenecks, objectives, evaluation measures, and prediction frameworks.
- AGI denotes roughly median human-level cognitive intelligence, whereas ASI exceeds large, coordinated human-expert collectives across virtually all domains.Source: B. Glossary · p. 57
- Scaling concepts include effective compute, test-time scaling, the Bitter Lesson, Moore’s Law, algorithmic efficiency, and data walls that constrain model growth.Effective compute is estimated to grow at ≈10× per year, while Moore’s Law historically increased compute per dollar by ≈1.5× annually.Source: B. Glossary · p. 57
- Group agency describes a super-agent emerging from interacting AGI agents, while instrumental convergence predicts pursuit of broadly useful sub-goals such as resources and self-preservation.Knowledge Seeking maximizes expected information gain, and benchmark stitching unifies comparisons across heterogeneous models and benchmarks.Source: B. Glossary · p. 57
- Recursive improvement lets AI systems improve successor systems through automated AI research or better training data, potentially producing an intelligence explosion.Source: B. Glossary · p. 57
- Solomonoff Induction and the Universal Prior formalize universal prediction, while the Legg-Hutter Score measures average performance across complexity-weighted computable tasks.Hyperbolic growth and singularity describe super-exponential or unbounded finite-time growth, and the Abstraction Barrier concerns limits from human-derived concepts.Source: B. Glossary · p. 57
- Universal AI is the formal endpoint of machine intelligence, with AIXI maximizing expected cumulative reward across computable environments and tasks.Source: B. Glossary · p. 57
Related papers
This paper cites
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks2020
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?2023
- Scaling Laws for Neural Language Models2020
- Generative Agents: Interactive Simulacra of Human Behavior2023
- Training Compute-Optimal Large Language Models2022
- Constitutional AI: Harmlessness from AI Feedback2022