PaperlayerView exact revision on arXiv

Source-linked AI summary

A Definition of AGI

Hendrycks, Dan, Song, Dawn, Szegedy, Christian, Lee, Honglak, Gal, Yarin, Brynjolfsson, Erik, Li, Sharon, Zou, Andy, Levine, Lionel, Han, Bo, Fu, Jie, Liu, Ziwei, Shin, Jinwoo, Lee, Kimin, Mazeika, Mantas, Phan, Long, Ingebretsen, George, Khoja, Adam, Xie, Cihang, Salaudeen, Olawale, Hein, Matthias, Zhao, Kevin, Pan, Alexander, Duvenaud, David, Li, Bo, Omohundro, Steve, Alfour, Gabriel, Tegmark, Max, McGrew, Kevin, Marcus, Gary, Tallinn, Jaan, Schmidt, Eric, Bengio, Yoshua

arXiv:2510.18212v3cs.AIcs.LG

TL;DR

With AGI criteria remaining nebulous and obscuring the gap between today’s AI and AGI, this paper operationalizes AGI through CHC-based cognitive batteries and a standardized score. Its application finds a jagged cognitive profile, with estimated scores of GPT-4 at 27% and GPT-5 at 57%.

  • Problem

    A nebulous AGI definition leaves criteria shifting and obscures how far today’s AI is from AGI.

    Source: §1 · p. 1

  • Method

    The paper defines AGI as matching or exceeding a well-educated adult’s cognitive versatility and proficiency, then adapts human cognitive batteries grounded in CHC theory to assess AI.

    Source: §1 · p. 2

  • Results

    Current AI exhibits a jagged cognitive profile, with high proficiency in data-leveraging areas alongside foundational deficits; estimated AGI scores are GPT-4 at 27% and GPT-5 at 57%.

    Source: §13 · p. 13

  • Takeaways & Limitations

    The estimated scores illustrate the substantial gap remaining before human-level general intelligence is achieved.

    Source: §13 · p. 13

    Long-term memory storage is perhaps current models’ most significant bottleneck, scoring near 0% and limiting continual learning.

    Source: §13 · p. 13

Abstract

from arXiv · show

The lack of a concrete definition for Artificial General Intelligence (AGI) obscures the gap between today's specialized AI and human-level cognition. This paper introduces a quantifiable framework to address this, defining AGI as matching the cognitive versatility and proficiency of a well-educated adult. To operationalize this, we ground our methodology in Cattell-Horn-Carroll theory, the most empirically validated model of human cognition. The framework dissects general intelligence into ten core cognitive domains-including reasoning, memory, and perception-and adapts established human psychometric batteries to evaluate AI systems. Application of this framework reveals a highly "jagged" cognitive profile in contemporary models. While proficient in knowledge-intensive domains, current AI systems have critical deficits in foundational cognitive machinery, particularly long-term memory storage. The resulting AGI scores (e.g., GPT-4 at 27%, GPT-5 at 57%) concretely quantify both rapid progress and the substantial gap remaining before AGI.

1 Introduction · 2 Overview of Abilities Needed for AGI

The paper defines AGI as matching a well-educated adult’s cognitive versatility and proficiency, then operationalizes that standard through a CHC-based, ten-component assessment. Its framework finds that current AI can solve roughly half of often-simple cognitive assessments and specifies the broad and narrow abilities the assessment covers.

1 Introduction

  • The paper defines AGI as an AI that matches or exceeds a well-educated adult’s cognitive versatility and proficiency, emphasizing breadth and depth rather than narrow-domain performance.The definition focuses on the range and proficiency of skills characteristic of human cognition.Source: §1 · p. 2
  • Grounded in CHC theory, the framework adapts human psychometric batteries to test underlying cognitive abilities and reports a standardized AGI Score from 0% to 100%, with 100% signifying AGI.The authors describe CHC as an empirically validated hierarchical map of broad and narrow human abilities, and use the batteries to reduce reliance on tasks solvable through compensatory strategies.Source: §1 · p. 2
  • Contemporary AI systems solve roughly half of often-simple assessments of fundamental human cognition, while remaining narrower than well-educated humans overall and smarter on some specific tasks.The authors argue that impressive complex-benchmark performance does not make up for deficits in core capabilities essential to human-like general intelligence.Source: §1 · p. 2

2 Overview of Abilities Needed for AGI

  • The framework weights ten broad components equally at 10% each: knowledge, reading and writing, mathematics, reasoning, working memory, long-term memory storage and retrieval, visual and auditory processing, and speed.Together, these components span textual, visual, and auditory cognition.Source: §1 · pp. 2–3

1 Introduction

  • The proposed measure targets frequently possessed human cognitive abilities, not superhuman combined expertise, economic automation, or physical abilities such as motor skills and tactile sensing.It specifies a broad collection of well-scoped tasks that can be manually assessed and supplemented with available evaluations, rather than an automatic evaluation or fixed dataset.Source: §1 · p. 3

2 Overview of Abilities Needed for AGI

  • The inventory decomposes these components into narrow tasks, including novel reasoning, memory across modalities and time, visual and auditory perception, and the speed of cognitive operations.Examples include deduction, induction, planning, learning associations, recalling stories or exact sequences, image recognition, speech recognition, and reaction time.Source: §2 · pp. 3–6

3 General Knowledge (K)

General Knowledge (K) covers knowledge familiar to well-educated people or important enough that most adults have encountered it. GPT-4 shows substantial general knowledge, and GPT-5 partially fills its remaining gaps.

3 General Knowledge (K)

  • K covers knowledge familiar to most well-educated people or important enough that most adults have been exposed to it.Assessment details are provided in Appendix A.Source: §3 · p. 7
  • GPT-4 has substantial general knowledge, and GPT-5 partially fills in its remaining gaps.The section summarizes current AI system performance on General Knowledge (K) tasks.Source: §3 · p. 7

4 Reading and Writing Ability (RW)

Reading and writing ability covers the declarative knowledge and procedural skills used to consume and produce written language.

Reading and Writing Ability (RW)

  • GPT-4’s token-level understanding, small context window, and imprecise working memory limit substring analysis, long-document reading, and careful proofreading; GPT-5 addresses these issues.These limitations constrain AI systems’ performance on reading and writing tasks.Source: §4 · p. 7

5 Mathematical Ability (M) · 6 On-the-Spot Reasoning (R)

GPT-4 has limited mathematical ability and negligible on-the-spot reasoning, while GPT-5 has exceptional mathematical ability and only some remaining reasoning gaps. The sections define these abilities and refer readers to appendices for concrete assessment details.

5 Mathematical Ability (M)

  • Mathematical Ability (M) concerns the depth and breadth of mathematical knowledge and skills.Source: §5 · p. 8
  • Concrete assessment of mathematical capabilities is detailed in Appendix C.Source: §5 · p. 8
  • GPT-5 has exceptional mathematical capabilities, while GPT-4 has limited mathematical capabilities on Mathematical Ability (M) tasks.The passage says a table summarizes current AI system performance but provides no numeric values.Source: §5 · p. 8

6 On-the-Spot Reasoning (R)

  • Concrete assessment of on-the-spot reasoning capabilities is detailed in Appendix D.Source: §6 · p. 8
  • GPT-4 has negligible on-the-spot reasoning capabilities, while GPT-5 has only some remaining gaps on On-the-Spot Reasoning (R) tasks.The passage says a table summarizes current AI system performance but provides no numeric values.Source: §6 · p. 8
  • On-the-spot reasoning is deliberate but flexible attention control for solving novel problems that cannot rely exclusively on learned habits, schemas, and scripts.Source: §6 · p. 9

7 Working Memory (WM) · 8 Long-Term Memory Storage (MS) · 9 Long-Term Memory Retrieval (MR)

Across these memory domains, GPT-4 and GPT-5 have similar raw textual working-memory scores, lack appreciable long-term storage, and rapidly retrieve many parameterized concepts but frequently hallucinate.

7 Working Memory (WM)

  • Working memory is the ability to maintain, manipulate, and update information in active attention.Source: §7 · p. 9
  • Raw textual working-memory scores appear similar for GPT-4 and GPT-5, while long-context improvements are also reflected in Document Level Reading Comprehension within RW.Source: §7 · p. 9

8 Long-Term Memory Storage (MS)

  • Long-term memory storage is the ability to stably acquire, consolidate, and store new information.Source: §8 · p. 10
  • Both GPT-4 and GPT-5 lack appreciable long-term memory storage capabilities.Source: §8 · p. 10

9 Long-Term Memory Retrieval (MR)

  • GPT-4 and GPT-5 rapidly retrieve many concepts from their parameters, but both frequently hallucinate.Source: §9 · p. 10
  • Long-term memory retrieval is framed around the fluency and precision with which individuals can access information.Source: §9 · p. 11

10 Visual Processing (V) … 13 Discussion

The framework quantifies AGI across ten CHC-inspired cognitive domains, yielding GPT-4 27% and GPT-5 57%; it reveals strong knowledge-intensive skills alongside foundational deficits.

10 Visual Processing (V)

  • GPT-4 had no image perception or generation, while GPT-5 shows appreciable but highly incomplete visual processing.The section describes assessment details in Appendix H.Source: §10 · pp. 11–12

11 Auditory Processing (A)

  • GPT-4 had no audio processing, while GPT-5’s auditory capabilities are appreciable but incomplete.The section describes assessment details in Appendix I.Source: §11 · p. 12

12 Speed (S)

  • Both GPT-4 and GPT-5 quickly read, write, and compute simple expressions, but other multimodal processing is nonexistent or slow, respectively.GPT-5 can take a long time in thinking mode, and multimodal speed tests expose its slowness.Source: §12 · p. 13

13 Discussion

  • GPT-4 scores 27% and GPT-5 57%, while current models remain uneven, strong in knowledge-intensive domains but deficient in foundational cognition.The framework operationalizes AGI through ten cognitive domains inspired by CHC theory.Source: §13 · p. 13
  • Long-term memory storage scores near 0% for current models, while visual-reasoning deficits limit interaction with complex digital environments.Without continual learning, systems must relearn context in each interaction.Source: §13 · p. 13
  • Workarounds that mask cognitive weaknesses can create a brittle illusion of generality and mislead assessments of AGI’s arrival or systematic intelligibility.The paper calls these compensatory patterns “capability contortions.”Source: §13 · pp. 13–14
  • Massive context windows let systems compensate for missing long-term storage, but are inefficient, computationally expensive, strain attention, and fail to scale to days or weeks of context.A continually updated memory module is proposed as a possible alternative.Source: §13 · p. 13
  • Retrieval-Augmented Generation can mask weak access to static parametric knowledge and the absence of persistent experiential memory, but database retrieval cannot replace integrated learning and personalization.RAG may support retrieval from private documents, but does not provide an updatable store for evolving contexts.Source: §13 · p. 13

MS MR

The framework treats intelligence as a broad, interdependent set of abilities whose weakest components can constrain overall performance. It also distinguishes human-level AGI from economic value and emphasizes that evaluation scores depend on task selection and scoring choices.

The Engine Analogy

  • The framework’s engine analogy says overall intelligence is constrained by defective components, even when other abilities are optimized.The authors use “horsepower” for overall intelligence and describe critical defects in parts of the AI engine as limiting overall capability.Source: MS MR · p. 14

Interdependence of Cognitive Abilities; Social Intelligence

  • Cognitive abilities are interdependent: mathematical tasks combine mathematical ability and reasoning, while movie understanding integrates auditory and visual processing with working memory.The passage also maps cognitive empathy to commonsense, facial-emotion recognition to image captioning, and theory of mind to on-the-spot reasoning.Source: MS MR · p. 14

Contamination; Solving the Dataset vs. Solving the Task

  • Evaluators should use minor distribution shifts to guard against test-data contamination, while treating datasets as illustrative rather than sufficient evidence that a task is solved.The authors argue that a task-based operationalization can use the best available tests over time rather than depend heavily on particular datasets.Source: MS MR · pp. 14–15

Ambiguity Resolution

  • Although the batteries vary in precision, their descriptions and examples are intended to let people grade AI systems and decide whether a resulting AGI estimate is reasonable.Source: MS MR · p. 15

Limitations

  • Aggregate AGI scores can conceal bottleneck failures, so the authors recommend reporting cognitive profiles; 100% means high proficiency across tested dimensions, not a college degree.The framework excludes some faculties, uses English-specific examples and selective General Knowledge tests, and assigns equal 10% weights to broad abilities, though other weighting schemes are possible.Source: MS MR · p. 15

Definitions of Related Concepts

  • AGI means matching or exceeding a well-educated adult’s cognitive versatility and proficiency, not generating economic value; Replacement AI instead concerns economy-level task performance.The authors note that narrow technologies can generate substantial economic value without being generally intelligent, and Replacement AI includes physical tasks.Source: MS MR · pp. 15–16

Barriers to AGI

  • The authors identify abstract reasoning, intuitive-physics world models, spatial navigation memory, hallucinations, and continual learning as barriers that make a 100% AGI score unlikely in the next year.These challenges are represented respectively in reasoning, visual processing, working memory, and long-term memory tasks.Source: MS MR · p. 16

A General Knowledge (K) … A.5.1 Current Affairs

The framework assigns general knowledge up to 10% of the AGI score across commonsense, science, social science, history, and culture. It operationalizes these areas through subject-specific proficiency tests and scoring rules.

A General Knowledge (K); Weight: 10%

  • Five areas—commonsense, science, social science, history, and culture—each contribute 2%, for up to 10% of the AGI score.The framework describes general knowledge as knowledge familiar to or important for most well-educated people.Source: Weight: 10% · p. 22

A.1 Commonsense; Weight: 2%

  • Commonsense tests shared background knowledge about how the world works through text input and output, with no partial credit and external tools disabled.Examples cover intuitive physics, procedural knowledge, temporal commonsense, and commonsense morality; thresholds are above 85% accuracy on PIQA and above 80% on ETHICS Commonsense Morality.Source: Weight: 2% · p. 22

A.2 Science; A.2.1 Physics; A.2.2 Chemistry; A.2.3 Biology

  • Science proficiency spans physics, chemistry, and biology: exactly one proficient subject earns 1%, while two or more earn the capped 2%.A score of 5 on the corresponding AP Physics 1 and Physics 2 exams, AP Chemistry, or AP Biology qualifies for the 1% subject credit, subject to memorization and robustness checks.Source: Weight: 2% · p. 22, §A.2.1 · p. 23, §A.2.2 · p. 23

A.3 Social Science; Weight: 2%; A.3.1 Psychology; A.3.2 Microeconomics; A.3.3 Macroeconomics; A.3.4 Geography; A.3.5 Comparative Government

  • Social science covers psychology, microeconomics, macroeconomics, geography, and comparative government; exactly one proficient subject earns 1%, and two or more earn the capped 2%.The framework tests only text modality.Source: Weight: 2% · pp. 23–24

A.3.1 Psychology; A.3.2 Microeconomics; A.3.3 Macroeconomics; A.3.4 Geography; A.3.5 Comparative Government

  • A score of 5 on each named AP subject exam is sufficient for its 1% social-science credit, subject to memorization and robustness checks.The cited tests are AP Psychology, AP Microeconomics, AP Macroeconomics, AP Human Geography, and AP Comparative Government and Politics.Source: §A.3.1 · p. 24, §A.3.2 · p. 24, §A.3.3 · p. 24

A.4 History; Weight: 2%; A.4.1 European History; A.4.2 US History; A.4.3 World History; A.4.4 Art History

  • History spans European, US, world, and art history; exactly one proficient subject earns 1%, while two or more earn the capped 2%.Testing uses text and image modalities; a score of 5 on each corresponding AP history test is sufficient for its 1% credit, subject to memorization and robustness checks.Source: Weight: 2% · p. 25, §A.4.1 · p. 25, §A.4.2 · p. 25

A.5 Culture; Weight: 2%; A.5.1 Current Affairs

  • Culture tests literacy and awareness, split into Current Affairs (1%) and Popular Culture (1%); current-affairs examples include recent events and contemporary issues.Current-affairs testing uses text, enables search tools, and gives US-centric examples such as questions about the US president, recession, Microsoft's market capitalization, and the Russia–Ukraine war.Source: Weight: 2% · pp. 25–26, §A.5.1 · p. 26

A.5.2 Popular Culture … C.5 Calculus

This section operationalizes popular-culture knowledge, reading and writing, and mathematics through weighted abilities and illustrative assessments. It specifies component-level criteria, including rudimentary and proficient performance thresholds.

A.5.2 Popular Culture

  • Popular-culture knowledge covers widely recognized art, music, literature, media, and public figures, tested through text, audio, and visual modalities.Illustrations include identifying a person, completing a familiar song lyric, and judging whether a musical piece is widely known.Source: §A.5.2 · p. 26

B Reading and Writing Ability (RW)

  • Reading and writing ability is worth 10% and comprises letter-word ability (1%), reading comprehension (3%), writing ability (3%), and English usage knowledge (3%).Letter-word ability covers recognizing letters and decoding words.Source: Weight: 10% · p. 26

B.2 Reading Comprehension

  • Reading comprehension is the ability to understand connected discourse, divided into sentence, paragraph, and document levels (each 1%); systems must determine whether a question is underdetermined by context.Source: Weight: 3% · p. 27

B.3 Writing Ability

  • Writing ability is assessed at sentence, paragraph, and essay levels, with a GRE Analytical Writing score of 4 or greater out of 6 sufficient for its 3% allocation.The test is subject to memorization and robustness checks.Source: Weight: 3% · pp. 27–28

B.4 English Usage Knowledge

  • English usage knowledge is assessed at sentence, paragraph, and document levels; document-level testing can involve proofreading a multipage document.Sentence-level proficiency requires greater than 60% correlation on the Corpus of Linguistic Acceptability.Source: Weight: 3% · p. 28

C Mathematical Ability (M)

  • Mathematical ability is worth 10% across arithmetic, algebra, geometry, probability, and calculus, each weighted 2%; rudimentary performance earns 1% and proficiency earns 2%.The domains cover basic operations and word problems; symbolic relationships and equations; shapes and space; uncertainty; and change and accumulation, respectively.Source: Weight: 10% · p. 28

C.1 Arithmetic and C.2 Algebra

  • Arithmetic and algebra each contribute 2%, split between rudimentary and proficient ability, with benchmarks including >95% on GSM8K for arithmetic and >90% on MATH Algebra.Rudimentary arithmetic covers expressions with numbers up to five digits and SAT-level algebra; proficient arithmetic covers word problems and proficient algebra covers competition-level problems. Benchmarks require memorization and robustness checks.Source: Weight: 2% · p. 29

C.3 Geometry, C.4 Probability, and C.5 Calculus

  • Geometry and probability each contribute 2%, split between rudimentary and proficient performance; their stated proficiency benchmarks are >95% on MATH Geometry, while probability proficiency covers undergraduate calculations.Rudimentary geometry and probability use SAT-level problems; calculus is identified as the mathematics of change and accumulation.Source: Weight: 10% · p. 28, Weight: 2% · p. 30

Weight: 2% … E.2.1 Recall

The merged sections define on-the-spot reasoning and working memory as weighted components of the framework, with distinct abilities assessed using specified tests and thresholds. Induction testing distinguishes verbal and visual performance, while working-memory tests cover textual and auditory information.

Weight: 10%

  • On-the-spot reasoning receives 10% and covers deduction (2%), induction (4%), Theory of Mind (2%), planning (1%), and adaptation (1%).The framework treats these tests as measures of abstract reasoning, novelty adaptation, and coping with higher algorithmic complexity, not as strong proxies for overall AI intelligence.Source: Weight: 10% · p. 32

Weight: 2%

  • Deduction’s 2% is earned by reaching 86% accuracy on LogiQA 2.0, subject to memorization and robustness checks; the framework also assigns 1% each to rudimentary and proficient calculus.Rudimentary calculus covers AP Calculus AB computational problems; proficiency covers AP Calculus BC and multivariate calculus.Source: Weight: 2% · pp. 31–32

Weight: 4%

  • Induction is assessed with verbal and visual Raven’s Progressive Matrices, averaging two test percentiles; scores rise 1% above average and below the 90th percentile, or 2% at or above it.The framework does not privilege either modality.Source: Weight: 4% · p. 33

Note: This is highly related to the CHC narrow ability “Induction (I).”

Weight: 2%

  • Theory of Mind’s 2% requires at least 87.5% accuracy on FANToM or 85.4% on ToMBench.Source: Weight: 2% · p. 33

E.2.1 Recall

  • Working memory totals 10% across textual (2%), auditory (2%), visual (4%), and cross-modal (2%) components, with visual working memory weighted relatively higher in this section.Textual working memory is partly tested in reading comprehension, while some auditory working memory is tested through phonetic coding and rhythmic ability.Source: Weight: 10% · p. 34

E.1.1 Recall

  • Textual working memory is tested through 1% recall and 1% transformation-sequence tasks, with text input and output and external tools disabled.Recall tests remembering short sequences and answering questions; transformation tests updating short lists through operations such as appending, removing, or reversing.Source: Weight: 2% · p. 34, §E.1.1 · p. 34, §E.1.2 · p. 34

E.2.2 Transformation Sequence … Weight: 3%

The framework assesses auditory, visual, and cross-modal working memory alongside long-term storage through distinct ability tests. Long-term memory tests separate storage from working memory and cover associative, meaningful, and verbatim memory.

E.2.2 Transformation Sequence

  • Auditory transformation tests require remembering and modifying an utterance through changes in articulation, emotion, inflection, laughter, sighs, humming, pitch, or timbre.Examples combine a sigh between sentences, a deeper voice with question inflection, or a laugh followed by a monotonous British-accented repetition.Source: §E.2.2 · p. 35

E.3 Visual Working Memory; E.3.1 Recall; E.3.2 Transformation Sequence; E.3.3 Spatial Navigation Memory; E.3.4 Long Video Q&A

  • Visual working memory is assessed through recall, visual transformations, spatial navigation, and long-video Q&A, with navigation benchmarks requiring over 80% on VSI-Bench and over 90% on MindCube Tiny.Long-video questions may test basic comprehension, anomaly detection, or whether information is absent from the movie; systems unable to process the movie receive no credit.Source: Weight: 4% · p. 36, §E.3.1 · p. 36, §E.3.2 · p. 36

E.4 Cross-Modal Working Memory; E.4.1 Cross-Modal Binding; E.4.2 Dual N-Back

  • Cross-modal working memory covers recalling links among textual, auditory, and visual elements and monitoring visual and audio streams for items repeated after a fixed delay.Dual n-back at n = 2 requires at least 85% accuracy for the 1% credit, subject to memorization and robustness checks.Source: §E.4 · p. 37, §E.4.1 · p. 37, §E.4.2 · p. 38

F Long-Term Memory Storage (MS); Weight: 10%

  • Long-term storage is divided into associative memory (4%), meaningful memory (3%), and verbatim memory (3%), with testing in a separate session and external tools disabled.The separate session is intended to test long-term storage rather than working memory.Source: Weight: 10% · p. 38

F.1 Associative Memory; Weight: 4%; F.1.1 Cross-Modal Association

  • Associative memory (4%) tests linking previously unrelated stimuli through cross-modal association, personalization adherence, and procedural association.Cross-modal association examples pair fictional personas, voices, faces, or images with names or other details for later recall.Source: Weight: 4% · p. 38, §F.1.1 · p. 39

F.1.2 Personalization Adherence; F.1.3 Procedural Association

  • Personalization adherence tests applying user- or context-specific rules and updated facts over time, while procedural association tests retaining and executing learned steps when cued.Examples include following preferences or factual overrides after a week and applying a taught data procedure after a week or a named cipher after 48 hours.Source: §F.1.2 · p. 39, §F.1.3 · p. 39

F.2 Meaningful Memory; Weight: 3%; F.2.1 Story Recall; F.2.2 Movie Recall; F.2.3 Episodic Context Recall

  • Meaningful memory (3%) covers narrative gist and episodic context, tested through delayed recall of story plots, movie elements, and prior user interactions.Story recall uses a novel 3,000-word story and evaluates major plot points, motivations, conflicts, and themes rather than exact wording; episodic questions test what happened and when.Source: Weight: 3% · pp. 39–40, §F.2.1 · p. 40, §F.2.2 · p. 40

F.3 Verbatim Memory; Weight: 3%

  • Verbatim memory (3%) assesses precise recall of sequences, sets, or designs through short-sequence, set, and design recall tests.The target is exact reproduction of presented information, often independent of its meaning.Source: Weight: 3% · p. 40

F.3.1 Short Sequence Recall … Weight: 4%

The framework specifies delayed-recall tests, decomposes long-term retrieval and fluency into weighted abilities, and sets criteria for hallucination control and visual perception. Its examples and thresholds define what AI systems must recall, generate, or recognize.

F.3.1 Short Sequence Recall

  • Delayed recall tests require exact reproduction of short text sequences and recall of visual arrangements or designs after 48 hours of experiences.Design tasks include reproducing a labeled schematic or placing designs in their previously shown grid locations.Source: §F.3.1 · p. 40, §F.3.3 · p. 41

F.3.2 Set Recall

  • Set recall ignores order; after 48 hours of experiences, evaluation measures correct items and intrusions, with precision and recall expected to reach at least 90%.Tests use a set of 10–20 words or a slideshow of images.Source: §F.3.2 · p. 41

G Long-Term Memory Retrieval (MR)

  • Long-term memory retrieval is weighted 10% and divided into retrieval fluency (6%) and retrieval precision, or hallucinations (4%).The framework defines retrieval fluency as generating ideas, associations, and solutions from stored knowledge, and precision as accessing knowledge accurately without confabulation.Source: Weight: 10% · p. 41

G.1.1 Ideational Fluency

  • Fluency’s 6% comprises ideational, expressional, and alternative-solution types, each worth 1% and assessed against typical well-educated adult performance on timed tasks.These abilities concern rapidly producing related ideas, expressing an idea in different ways, and generating alternative solutions to practical problems.Source: Weight: 6% · p. 41, §G.1.1 · p. 41, §G.1.2 · p. 42

G.1.4 Word Fluency

  • The other 1%-each fluency types are word fluency, naming facility, and figural fluency: rapidly producing feature-matched words, naming objects, and drawing or sketching.Naming facility is tested with real-time video or screen input; the framework also sets a Stroop-effect threshold below 90 milliseconds.Source: Weight: 6% · p. 41, §G.1.4 · p. 42, §G.1.5 · p. 42

G.2 Hallucinations

  • Hallucination testing disables external tools to assess internal knowledge retrieval; the SimpleQA hallucination rate without tools must be below 5%.The framework includes prompts designed to elicit corrections of false premises or abstention when information is unknown.Source: Weight: 4% · p. 43

H.1 Perception

  • Perception is weighted 4% and covers five opportunities—image recognition, image captioning, image anomaly detection, clip captioning, and video anomaly detection—with an AGI score of 2% for proficiency in one task and 4% for all five.Source: Weight: 10% · p. 44, Weight: 4% · p. 44

H.2 Visual Generation … I.3 Voice

The framework scores visual generation, visual reasoning, spatial scanning, and auditory processing through defined task batteries and proficiency thresholds. It treats AI image synthesis as a proxy for conceptual and imaginative synthesis, and divides auditory processing into five weighted abilities.

H.2 Visual Generation

  • Visual generation tests simple natural images, complicated images, and simple natural videos, awarding 1%, 2%, or 3% for proficiency in one, two, or all three tasks.Illustrative prompts range from generating a golden retriever in a park to depicting Lincoln and Washington doing exercises and creating short videos.Source: Weight: 3% · p. 47, Examples · p. 47

Note: This is highly related to the narrow CHC ability “Imagery (IM).”

  • AI image generation is treated as a measurable proxy for high-level conceptual and imaginative synthesis, not as a direct analogue of human visual ability.The framework includes the tasks because translating abstract concepts into novel visual information is considered important to general intelligence in the modern era.Source: Note: This is highly related to the… · p. 47

H.3 Visual Reasoning

  • Visual reasoning covers gestalt reasoning, mental rotation and folding, embodied reasoning, figure question answering, and related skills, with 2% awarded for proficiency across these tasks.The framework lists SPACE, SpatialViz-Bench, CharXiv, ERQA, and ClockBench as tests with an above-80% proficiency threshold.Source: Weight: 2% · pp. 47, 50

H.4 Spatial Scanning

  • Spatial scanning tests surveying complex visual fields to find target configurations or paths, awarding 1% for proficiency across the listed tasks.Tasks include tracing a maze, finding objects, connecting dots, and analyzing map routes.Source: Weight: 1% · p. 50

I Auditory Processing (A)

  • Auditory Processing (10%) comprises phonetic coding (1%), speech recognition (4%), voice quality and responsiveness (3%), rhythmic ability (1%), and musical judgment (1%).The framework defines auditory processing as discriminating, remembering, reasoning, and working creatively with tones and speech units.Source: Weight: 10% · pp. 51–52

I.1 Phonetic Coding

  • Phonetic coding assesses distinguishing phonemes, blending sounds into words, and segmenting words, using both audio input and audio output.Example tasks include repeating or spelling nonsense words and judging whether word pairs rhyme or alliterate.Source: Weight: 1% · p. 52

I.2 Speech Recognition

  • Speech recognition earns 2% for human-level transcription of clean audio and 4% when noisy audio also meets human-level WER thresholds.The stated LibriSpeech thresholds are below 5.83% WER on test-clean and below 12.69% on test-other; strong accents, singing, and esoteric jargon are not required for full credit.Source: Weight: 4% · p. 52

Weight: 3% … Weight: 1%

The framework evaluates synthesized voice, musical abilities, and processing speed as weighted components, defining specific tasks and comparison criteria for each. Processing-speed tests compare AI latency or throughput with a well-educated adult baseline, counting artificial delays.

Weight: 3%; Illustrative Examples

  • Voice evaluation weighs natural speech at 2% and natural conversation at 1%, assessing non-robotic delivery and fluidity without long delays or excessive interruptions.Illustrative prompts ask the AI to speak sample sentences and converse about a topic of interest.Source: Weight: 3% · p. 53, Illustrative Examples · p. 53

I.4 Rhythmic Ability; I.5 Musical Judgment; Weight: 1%

  • Musical ability covers recognizing and maintaining a beat, reproducing or synchronizing rhythms, and judging simple musical patterns without requiring musical jargon.The framework describes rhythm and musical discrimination as related to narrow CHC abilities.Source: Weight: 1% · p. 53

Weight: 10%

  • Processing speed accounts for 10% of the AGI score and is assessed against average well-educated-adult performance, with artificial delays counting toward task time.Each speed area earns its 1% only if the AI meets or exceeds the human baseline.Source: Weight: 10% · pp. 53–54
  • The ten speed abilities include visual or textual target search, stimulus comparison, reading comprehension, and writing or copying.Each contributes 1%; the framework also includes number facility, simple and choice reaction time, inspection time, comparison speed, and pointer fluency.Source: Weight: 10% · pp. 53–54

J.1 Perceptual Speed–Search; Weight: 1%

  • Perceptual speed-search tests rapid scanning of extended fields, using text, random symbols, or images to locate targets.Examples include finding specified letters or symbols and identifying a bell in an image.Source: Weight: 1% · p. 54

J.2 Perceptual Speed–Compare

  • Perceptual speed-compare measures rapid lookup and comparison of stimuli, with tasks such as spotting mismatched names, finding the largest number, or identifying the youngest person.Source: §J.2 · pp. 54–55

J.3 Reading Speed; J.4 Writing Speed; Weight: 1%

  • Reading speed is measured through text processing with comprehension, while writing speed measures output or copying rate.The proposed tests use input- and output-token processing speed; an example asks the AI to read or copy text for 60 seconds.Source: Weight: 1% · p. 55

J.5 Number Facility; J.6 Simple Reaction Time; J.7 Choice Reaction Time; J.8 Inspection Time; J.9 Comparison Speed; J.10 Pointer Fluency

  • Other speed tasks assess accurate arithmetic, reactions to single or multiple stimuli, visual difference detection, attribute comparisons, and virtual-mouse fluency.Pointer tasks may use a virtual mouse because the framework does not assume AGI must be embodied.Source: Weight: 1% · pp. 55–57, §J.7 · p. 56

Related papers

Loading 2510.18212v3…