Source-linked AI summary

Consciousness in Artificial Intelligence: Insights from the Science of Consciousness

Patrick Butlin, Robert Long, Eric Elmoznino, Yoshua Bengio, Jonathan Birch, Axel Constant, George Deane, Stephen M. Fleming, Chris Frith, Xu Ji, Ryota Kanai, Colin Klein, Grace Lindsay, Matthias Michel, Liad Mudrik, Megan A. K. Peters, Eric Schwitzgebel, Jonathan Simon, Rufin VanRullen

arXiv:2308.08708v3cs.AIcs.CYcs.LGq-bio.NC

TL;DR

Whether AI systems could be conscious is increasingly pressing because conversational systems may prompt mistaken attributions and raise moral and social questions. The report applies neuroscientific theories to derive indicators and assess existing systems, finding that no current system is a strong consciousness candidate while many indicators appear implementable with current techniques.

  • Problem

    Whether AI systems could be conscious is increasingly pressing because convincing conversational behavior may prompt mistaken attributions and raises moral and social questions.

    Source: Executive Summary · p. 4, §1 · p. 9, §4.1 · p. 64
  • Method

    The report surveys empirically supported neuroscientific theories, derives computational indicator properties, and assesses recent AI systems against them.

    Source: Executive Summary · p. 6, §1 · p. 9
  • Results

    No current AI system appears to be a strong candidate for consciousness, although many indicator properties can be implemented using current techniques.

    Source: Executive Summary · pp. 4, 6
  • Takeaways & Limitations

    Neuroscientific theories provide a currently available basis for assessing AI consciousness despite divergent expert views.

    Source: §1 · p. 9

    Assessing some consciousness indicators may require interpretability methods because architecture, training, and behavior alone can be insufficient.

    Source: §3.2.1 · p. 59

Abstract

from arXiv · show

Whether current or near-term AI systems could be conscious is a topic of scientific interest and increasing public concern. This report argues for, and exemplifies, a rigorous and empirically grounded approach to AI consciousness: assessing existing AI systems in detail, in light of our best-supported neuroscientific theories of consciousness. We survey several prominent scientific theories of consciousness, including recurrent processing theory, global workspace theory, higher-order theories, predictive processing, and attention schema theory. From these theories we derive "indicator properties" of consciousness, elucidated in computational terms that allow us to assess AI systems for these properties. We use these indicator properties to assess several recent AI systems, and we discuss how future systems might implement them. Our analysis suggests that no current AI systems are conscious, but also suggests that there are no obvious technical barriers to building AI systems which satisfy these indicators.

Executive Summary

The report proposes assessing AI consciousness through neuroscientific theories and derived computational indicators, concluding that no current system is a strong candidate although conscious AI may be feasible in the near term if computational functionalism is true.

  • The report responds to growing concern that rapid AI progress and humanlike conversation may encourage people to believe AI systems are conscious.It argues that consciousness should be assessed using neuroscientific theories rather than conversational appearance alone.Source: Executive Summary · p. 4
  • The method treats consciousness assessment as scientifically tractable by combining computational functionalism, neuroscientific theories, and theory-based evaluation rather than unreliable behavioral imitation tests.Credences depend on functional similarity, evidence for the relevant theories, and confidence in computational functionalism.Source: Executive Summary · p. 4
  • The rubric derives indicator properties from multiple live theories, without endorsing one theory, and treats systems with more such properties as more likely to be conscious.Theories discussed include recurrent processing, global workspace, and computational higher-order theories; integrated information theory is excluded as incompatible with computational functionalism.Source: Executive Summary · pp. 4–5
  • No current AI system appears to be a strong consciousness candidate, although many indicator properties can be implemented with current techniques and conscious AI could be built near term if computational functionalism is true.The proposed rubric is provisional, and the report recommends further research on consciousness science and its application to AI.Source: Executive Summary · pp. 4, 6
  • Standard machine-learning methods can usually implement individual indicators, but experimentation is needed to combine multiple properties into functional systems.Existing systems clearly meet algorithmic recurrence, and agency may already be present; researchers have also explored global workspace and attention schema implementations.Source: Executive Summary · p. 6
  • Case studies examine Transformers, Perceiver, DeepMind’s Adaptive Agent, a virtual-rodent controller, and PaLM-E for global-workspace, agency, and embodiment indicators, without identifying a strong candidate.The report uses these systems to illustrate how existing architectures can be assessed against the proposed properties.Source: Executive Summary · p. 6

1 Introduction

The report examines whether current and near-term AI systems could be conscious by applying empirically supported theories of consciousness to their functions and architectures. It favors this theory-based approach over behavior-based tests while acknowledging uncertainty about the theories and their applicability.

1 Introduction

  • The report responds to renewed interest in AI consciousness by presenting an interdisciplinary, scientifically tractable framework for informing research and social decisions.It takes seriously the possibility of conscious AI within the next few decades and the likelihood that humanlike language systems will be perceived as conscious.Source: §1 · p. 9

1.1 Terminology

  • Phenomenal consciousness means having subjective experience—“something it is like”—and is distinct from access consciousness, reportability, and mere sensory processing (Block 1995; Nagel 1974).The report includes sensory, bodily, imagistic, emotional, and possibly cognitive experiences while distinguishing them from unconscious information processing.Source: §1.1 · pp. 9–11

1.2 Methods and Assumptions

  • The method assumes computational functionalism, empirically informed scientific theories, and a theory-heavy assessment of functional or architectural conditions rather than theory-neutral behavior.Computational functionalism treats the relevant computational organization as potentially sufficient for consciousness in non-organic systems, while the theory-heavy approach focuses on how systems work.Source: §1.2 · pp. 11–12

1.2.1 Computational functionalism

  • Computational functionalism makes consciousness depend on abstract functional organization rather than physical substrate, while allowing that different algorithms or representational formats may matter.The report adopts it as a plausible working hypothesis because it enables inferences from computational theories developed in neuroscience to AI, while acknowledging that non-computational features could be necessary.Source: §1.2.1 · pp. 13–14

1.2.2 Scientific theories of consciousness

  • Scientific theories distinguish consciousness-associated material processes and offer AI-relevant indicators, but report-based evidence risks conflating phenomenal consciousness with access, attention, memory, or reporting processes.No-report paradigms and metacognitive judgments may reduce these confounds, though neither is a complete solution and evidence from different methods should be combined.Source: §1.2.2 · pp. 14–16
  • Across materialism, property dualism, panpsychism, and illusionism, neuroscience can investigate which brain states generate consciousness or its appearance.These positions differ metaphysically, but each leaves a scientific role for explaining why some brain states are associated with conscious experiences or quasi-phenomenal properties.Source: §1.2.2 · pp. 16–17

1.2.3 Theory-heavy approach

  • The theory-heavy approach assesses AI by comparing its computational processes with those proposed by consciousness theories, weighting theory confidence and computational-functionalism confidence.Considering multiple theories can provide a fuller assessment of whether a system is conscious.Source: §1.2.3 · p. 17
  • Theory-heavy assessments face uncertainty because human evidence does not establish how far a theory’s conditions can be relaxed for other systems, and restrictive versus liberal interpretations lack clear empirical support.Moderate interpretations may balance generality against attributing consciousness to extremely simple systems, but their advantage over alternatives is unclear.Source: §1.2.3 · p. 17
  • Despite its limitations, the theory-heavy approach remains useful because similarities to consciousness-theory processes distinguish better AI candidates, and theories are the best available guide to relevant features.The report applies this approach without committing to any single theory.Source: §1.2.3 · p. 18
  • Behavioral tests are vulnerable to gaming because AI systems can mimic human behavior while operating differently, making human-like conversation insufficient evidence of consciousness.Restricting access to consciousness literature may not reliably prevent gaming without also depriving systems of the information needed to engage with the test.Source: §1.2.3 · p. 18

2 Scientific Theories of Consciousness

This section surveys multiple scientific theories and related proposals to derive computationally assessable indicators of consciousness for AI systems. It avoids endorsing any single theory or treating any indicator as definitively necessary or jointly sufficient, using the indicators instead to estimate whether systems are likely conscious.

Scope and aim

  • The section derives AI-relevant indicators of consciousness from surveyed theories and proposals, while deferring detailed discussion of their possible artificial implementations to section 3.The indicators are intended for assessing particular AI systems.Source: §2 · p. 19

Theories surveyed

  • Sections 2.1–2.3 examine recurrent processing, global workspace, and higher-order theories, section 2.4 surveys additional proposals, and section 2.5 presents the indicator list.Perceptual reality monitoring receives particular attention within the higher-order theories subsection.Source: §2 · p. 19

Interpretive stance

  • The authors do not adjudicate among theories or claim that any condition, or combination of conditions, is definitively necessary or jointly sufficient for consciousness.They call the features indicators because credible theories imply that they may be necessary or sufficient, not because the paper establishes them as such.Source: §2 · p. 19
  • A system’s possession of the indicators makes consciousness more probable, and assessing them is presented as the best current way to judge whether an AI system is likely conscious.This judgment is explicitly framed relative to the current state of scientific knowledge.Source: §2 · p. 19

2.1 Recurrent Processing Theory

Recurrent processing theory (RPT; Lamme 2006, 2010, 2020) holds that recurrent processing generates conscious, organised visual representations beyond what feedforward processing provides. Applied to AI, it motivates indicators of algorithmic recurrence and integrated perceptual organisation, while leaving open biological and non-visual interpretations.

2.1.1 Introduction to recurrent processing theory

  • RPT distinguishes unconscious feedforward visual processing from conscious vision, which arises when recurrent signals organise an inferred scene through feedback from higher to lower visual areas.Conscious visual experience does not require prefrontal or attentional involvement on this local-theory view.Source: §2.1.1 · pp. 19–20

2.1.2 Evidence for recurrent processing theory

  • Evidence from masking, transcranial magnetic stimulation, lesion, and brain-stimulation studies supports recurrence as necessary while challenging claims that prefrontal processing is required for conscious vision.RPT proponents argue that evidence for downstream requirements may be confounded by cognitive processes involved in reporting conscious perception.Source: §2.1.2 · p. 20

2.1.3 Indicators from recurrent processing theory

  • RPT-1 requires input modules using algorithmic recurrence, while RPT-2 requires organised, integrated perceptual representations rather than mere feature extraction.The analysis focuses on algorithmic rather than implementational recurrence because it is more relevant to standard AI, and treats systems lacking recurrence as significantly worse candidates for consciousness. Biological interpretations are set aside, while a purely visual interpretation would have limited implications for AI.Source: §2.1.3 · pp. 21–22

2.2 Global Workspace Theory

Global Workspace Theory identifies consciousness with representations selected into a capacity-limited workspace and broadcast across specialised modules. The section presents evidence for widespread, recurrent activity and derives four progressively stronger indicators for assessing artificial systems.

2.2.1 Introduction to global workspace theory

  • GWT identifies conscious states as representations in a global workspace, where limited-capacity competition selects information for broadcast to modules supporting reasoning, decision-making, and memory.Its neural version associates consciousness with sustained recurrent activity and step-function “ignition” in distributed frontoparietal workspace neurons.Source: §2.2.1 · p. 23

2.2.2 Evidence for global workspace theory

  • Contrastive studies associate conscious perception with late, reverberant activity across widespread networks including PFC, unlike the more limited activity associated with unconscious states.Monkey and human studies report PFC decoding or ignition-like patterns linked to conscious percepts, including in passive no-report paradigms.Source: §2.2.2 · pp. 24–25

2.2.3 Indicators from global workspace theory

  • GWT indicators require parallel specialised modules, a capacity-limited workspace with selective attention, global broadcast, and state-dependent attention for sequential complex tasks.These conditions progressively approximate the full global workspace architecture; state-dependent selection supports controlled interactions among modules.Source: §2.2.3 · pp. 26–27
  • Self-attention differs from GWT’s proposed mechanisms because it uses graded input weighting without binary ignition or an attention model operating over attention itself.The section notes that artificial attention has only loose connections to neuroscientific attention, whose modulation is often associated with recurrent top-down connections.Source: §2.2.3 · p. 28

2.3 Higher-Order Theories

Higher-order theories hold that consciousness requires awareness of one’s own mental states through higher-order representation. Computational versions emphasize perceptual reality monitoring, metacognitive assessment, and agency, yielding four indicators for assessing AI consciousness.

2.3.1 Introduction to higher-order theories

  • Higher-order theories require awareness of being in a mental state through representations of other representations, with computational theories emphasizing PRM and HOSS.PRM identifies conscious perceptual representations as reliable rather than noise, while HOSS treats awareness as a higher-order state in a generative model.Source: §2.3.1 · pp. 29–30

2.3.2 Computational HOTs and GWT

  • Unlike GWT, higher-order theories reject global-workspace broadcast as necessary and sufficient for consciousness, interpreting dissociations between conscious reports and task performance as evidence for their view.Lau and Passingham (2006) found masking conditions with equal discrimination but different reported awareness, which higher-order theorists argue challenges GWT’s predictions.Source: §2.3.2 · pp. 30–31

2.3.3 Indicators from computational HOTs

  • Computational HOTs yield HOT-1, generative, top-down or noisy perception modules, and HOT-2, metacognitive monitoring that distinguishes reliable perceptual representations from noise.These indicators follow PRM’s claim that consciousness depends on monitoring multiple sources of perceptual activity and identifying sufficiently reliable representations.Source: §2.3.3 · p. 31
  • The remaining indicators are HOT-3, agency guided by general belief formation and action selection, and HOT-4, sparse and smooth coding generating a quality space.PRM links conscious contents to belief-guided action and treats qualities as necessary conditions, potentially supported by sparse, smooth representations.Source: §2.3.3 · p. 32

2.4 Other Theories and Conditions · 2.4.1 Attention Schema Theory · 2.4.2 Predictive Processing

The paper surveys additional theories and conditions relevant to AI consciousness, including attention schema theory and predictive processing, while excluding IIT as less compatible with computational functionalism. It derives AI consciousness indicators from these discussions: controllable predictive attention models and predictive-coding input modules.

2.4 Other Theories and Conditions

  • IIT is excluded because its standard construal conflicts with computational functionalism and does not distinguish better AI consciousness candidates on conventional hardware.Weak IIT suggests that information integration and differentiation may matter, but does not yet specify which measures to use or how to interpret them in artificial systems.Source: §2.4 · p. 33

2.4.1 Attention Schema Theory

  • AST claims consciousness depends on a brain-generated model of attention that represents, and may misrepresent, current attentional objects.The theory explains conscious experience through the contents of this attention schema, such as representing attention to an apple (Webb & Graziano 2015, Graziano 2019a).Source: §2.4.1 · p. 33
  • AST resembles a higher-order theory by emphasizing representations of attention, while specifically targeting the meta-problem of why consciousness seems mysterious (Chalmers 2018, Graziano 2019b).It may be interpreted either as explaining away consciousness or as an account of its conditions.Source: §2.4.1 · pp. 33–34
  • AST-1 identifies a predictive model that represents and enables control over the current state of attention as an AI consciousness indicator.Such models support learning about attention, anticipating changes, and making preemptive adjustments, including in AI systems using attention as active information-flow control (Liu et al. 2023).Source: §2.4.1 · p. 34

2.4.2 Predictive Processing

  • Although PP is a general framework rather than a theory of consciousness, its prominence leads researchers to treat predictive processing as a plausible necessary condition.PP has been used to address qualia, emotion, and embodiment, but has received relatively little direct attention to distinguishing conscious from non-conscious systems or states (Seth & Hohwy 2021, Seth & Bayne 2022; Clark 2019; Seth 2021; Deane 2021, Hohwy 2022, Nave et al. 2022).Source: §2.4.2 · p. 34
  • Predictive processing frames cognition as minimizing sensory-prediction errors in hierarchical generative models through perception, attention, and action.Active inference treats action as a means of reducing prediction error, with adaptive actions selected when organisms predict their own success.Source: §2.4.2 · p. 34
  • PP-1 identifies input modules using predictive coding as an indicator of consciousness.The PP framework has also informed proposals connecting perceptual inference to global broadcast and higher-order state spaces (Hohwy 2013; Whyte 2019; Fleming 2020).Source: §2.4.2 · pp. 34–35

2.4.3 Midbrain Theory · 2.4.4 Unlimited Associative Learning

The midbrain theory locates consciousness in integrated spatiotemporal modeling for embodied action selection, while unlimited associative learning proposes jointly sufficient hallmarks but is rejected here as an artificial-consciousness indicator. Together, these accounts emphasize integration, embodiment, agency, flexible evaluation, and learning across modalities and time.

2.4.3 Midbrain Theory

  • The midbrain theory argues that consciousness can arise from midbrain and basal-ganglia activity modeling the agent’s body, environment, needs, and actions (Merker 2007; Klein & Barron 2016).The superior colliculus integrates spatial and vestibular information, while the hypothalamus, periaqueductal gray, and basal ganglia contribute physiological-state representation, opportunity identification, and action selection.Source: §2.4.3 · p. 35
  • The theory links consciousness to integration evolved for decision-making in mobile animals, especially distinguishing self-caused motion, and therefore emphasizes spatial, affective, and homeostatic information.This supports treating purposeful bodily navigation as relevant to consciousness and to the report’s proposed indicators.Source: §2.4.3 · p. 35

2.4.4 Unlimited Associative Learning

  • Ginsburg and Jablonka’s (2019) unlimited associative learning framework treats unlimited associative learning as an evolutionary transition marker requiring several jointly sufficient consciousness hallmarks (Birch et al. 2020).The framework unifies these features by their role in facilitating unlimited associative learning.Source: §2.4.4 · p. 35
  • The UAL hallmarks include global accessibility, selective attention, temporal integration, embodiment and agency, self-other registration, flexible valuation, feature binding, and intentional representation.These conditions are presented as a jointly sufficient set for consciousness.Source: §2.4.4 · p. 36
  • The report aligns these hallmarks with global workspace, temporal integration, agency, embodiment, binding, and intentionality, while treating binding as expected in neural-network AI and intentionality as philosophically unsettled.Global accessibility and selective attention correspond to global workspace architectures; other conditions are discussed through agency, embodiment, and temporal integration.Source: §2.4.4 · p. 36
  • UAL requires open-ended learning from compound and novel stimuli, rapid value updating, second-order and trace conditioning, and integration across modalities and time.These abilities support flexible evaluation of stimuli, actions, and outcomes in response to new information.Source: §2.4.4 · p. 36
  • Flexible UAL may strengthen evidence for consciousness in systems combining agency with global-workspace or predictive-processing architectures, because it links integration, metacognitive belief formation, decision-making, and evaluative learning.The relevant capacities overlap with those of animals considered likely to be conscious under UAL.Source: §2.4.4 · pp. 36–37
  • The report excludes UAL from its artificial-consciousness indicators because AI systems may implement it without the associated hallmarks, and because UAL is a behavioral marker rather than a proposed mechanism of conscious experience.These possibilities weaken the inference from UAL capacity to consciousness in artificial systems.Source: §2.4.4 · p. 37

2.4.5 Agency and Embodiment · 2.4.5(a) Agency

The section examines agency as a possible consciousness indicator, contrasting conscious organisms with feedforward systems such as AlexNet. It proposes substantive criteria for agency and distinguishes general agency, flexible goals or values, and intentional agency.

2.4.5 Agency and Embodiment

  • Agency and embodiment distinguish conscious organisms from AlexNet, which neither pursues goals nor acts, lacks a body, and processes temporally independent inputs through feedforward passes.The section uses these big-picture differences to motivate further consciousness indicators.Source: §2.4.5 · p. 37

2.4.5(a) Agency

  • Scientific theories variously require or emphasize agency: PRM links consciousness to belief formation and decision-making, while midbrain theory, GWT, and UAL connect consciousness with action selection, rational agency, or embodiment.PRM treats agency as necessary because conscious signals must support decisions about how to act. [Stich 1978, Dretske 1988, Schwitzgebel 2021]Source: 2.4.5(a) Agency · p. 38
  • The section identifies three agency-related indicators: being an agent, having flexible goals or values, and being an intentional agent.The latter two are stronger conditions, so agents, flexible agents, and intentional agents may differ in their candidacy for consciousness.Source: 2.4.5(a) Agency · p. 38
  • A substantive agent must interact with its environment, pursue goals through changing inputs, and learn outputs that promote benefits, unlike thermostats or purely stimulus-responsive systems.The liberal sensor-and-activator definition is too permissive because it includes AlexNet and simple artifacts.Source: 2.4.5(a) Agency · p. 39
  • Typical reinforcement-learning systems meet three substantive agency conditions: outputs affect subsequent inputs, systems pursue goals, and they learn goal-conducive outputs.These criteria are relatively minimal, and reinforcement learning is sufficient but not necessary for agency; bandit systems may fail the interaction condition. [10]Source: 2.4.5(a) Agency · p. 39
  • Flexible goals or values may involve learning new goals or changing preferences with needs, with flexible responsiveness to competing goals the more compelling consciousness-related condition.This flexibility may require integrating multiple value-relevant signals and comparing their motivational significance; conscious valence could provide a common currency. [Bronfman et al. 2016, Birch et al. 2020; Cabanac 1992, Carruthers 2018]Source: 2.4.5(a) Agency · pp. 39–40
  • Intentional agency involves goal-directed, model-based action selection that combines belief-like representations about outcomes and actions with desire-like values, rather than habitual responses.This resembles PRM’s requirement that perceptual representations update belief-like states used in instrumental reasoning and action selection.Source: 2.4.5(a) Agency · pp. 38, 40

2.4.5(b) Embodiment

The section distinguishes agency from embodiment, emphasizing bodies’ sensorimotor constraints, perspectives, and distinctive uses of output-input models. It also considers self-maintenance and metabolism as further proposed conditions for consciousness.

  • Embodiment differs from agency because embodied systems occupy positions, face position-constrained inputs and actions, and continuously control effectors whose movements systematically affect sensory inputs.AlphaGo illustrates that an agent can lack these embodied features.Source: 2.4.5(b) Embodiment · p. 40
  • Embodied agency can generate a conscious perspective by tracking how the agent’s actions alter perception and distinguishing self-caused sensory changes from environmental changes.This requires implicitly representing a self located in a moving body and predicting the sensory consequences of active and passive movement (Hurley 1998, Merker 2005, Godfrey-Smith 2019).Source: 2.4.5(b) Embodiment · p. 41
  • Sensorimotor and midbrain theories link consciousness to learning and using output-input models, including an integrated model of the embodied self, for perception, action selection, and control (Hurley 1998, O’Regan & No¨e 2001, No¨e 2004, Kiverstein 2007).Such models may be distinctive when they represent movement-driven sensory effects, distinguish endogenous from exogenous change, or support responsive multi-degree-of-freedom motor control (Miall & Wolpert 1996, McNamee & Wolpert 2019).Source: 2.4.5(b) Embodiment · pp. 41–42
  • A further proposed condition is autopoietic self-maintenance: consciousness may require systems to sustain their own existence and organisation through ongoing sensing, responding, repair, and regulation (Maturana & Varela 1991, Thompson 2005, 2007, Seth 2021, Aru et al. 2023).Self-maintenance is argued to establish system identity and perspective, support self-modeling and flexible goals, and potentially relate to valence, although its distinction from externally directed agency remains unclear (Thompson 2005; Seth 2021).Source: 2.4.5(b) Embodiment · p. 42
  • Some accounts additionally propose that consciousness requires nanoscale metabolic processes or specific material composition, because artificial systems may only functionally resemble living organisms without reproducing their relevant physical organisation.Source: 2.4.5(b) Embodiment · p. 42

2.4.5(c) Agency and embodiment indicators

Agency and embodiment indicators must be formulated narrowly to remain compatible with computational functionalism, avoiding reliance on external conditions that could vary while computations remain unchanged. The section therefore adopts two indicators: agency and embodiment.

2.4.5(c) Agency and embodiment indicators

  • Agency and embodiment may conflict with computational functionalism because identical computations can occur when inputs do not genuinely depend on outputs or external conditions differ.A system may receive coincidentally compatible input-output patterns without genuinely interacting with its environment, and self-maintenance can likewise be simulated without causing persistence.Source: 2.4.5(c) Agency and embodiment… · p. 43
  • The indicators are therefore defined internally: agency involves learning from feedback and selecting actions toward goals, while embodiment involves modeling output-input contingencies and using that model in perception or control.This formulation permits systems controlling virtual avatars to count as embodied, even when their learned model could misrepresent genuine environmental contingencies.Source: 2.4.5(c) Agency and embodiment… · p. 43
  • The authors exclude intentional agency as redundant with HOT-3, reject self-maintenance as contrary to the relevant theories’ spirit, and combine agency with flexible goals through centralisation and a common motivational currency.The section identifies agency, flexible goals or values, intentional agency, perspective, body, and self-maintenance as candidate indicators before narrowing the set.Source: 2.4.5(c) Agency and embodiment… · pp. 43–44
  • The section adopts AE-1 Agency and AE-2 Embodiment as two further indicators of consciousness.AE-1 requires learning from feedback and selecting outputs to pursue goals, especially with flexible responses to competing goals; AE-2 requires modeling systematic output-input effects and using that model in perception or control.Source: 2.4.5(c) Agency and embodiment… · p. 44

2.4.6 Time and Recurrence

Human experience appears temporally integrated, but consciousness might also involve discrete experiences or illusory continuity. Nevertheless, algorithmic recurrence is likely necessary for human-like conscious experience because past information must influence present processing.

2.4.6 Time and Recurrence

  • Although human experience seems continuously integrated over time, this may not be essential for consciousness: dense-amnesia patients may have brief, disjointed experiences, and conscious systems might lack a persisting subject.The passage notes that temporal integration may not be the only basis for subject persistence.Source: §2.4.6 · p. 44
  • Temporal continuity may be illusory under the “dynamic snapshot” view, where discrete static experiences are rapidly sampled from unconscious activity while retaining information can represent change.On this account, apparent integration can arise without special integration between successive experiences.Source: §2.4.6 · p. 44
  • Algorithmic recurrence is likely necessary for conscious experience with a human-like temporal character because representing environmental change or continuity requires retained past information to influence present processing.This supports treating recurrent processing as an indicator of consciousness in RPT-1; recurrence is also emphasized by midbrain and sensorimotor theories, the UAL framework, and some predictive-processing theorists.Source: §2.4.6 · pp. 44–45

2.5 Indicators of Consciousness

The section consolidates theories from sections 2.1–2.4 into indicators of consciousness for assessing AI systems. These indicators form a theory-informed rubric, without committing to theories’ stronger necessary or sufficient-condition claims.

Indicator derivation

  • The authors derive a list of consciousness indicators from the theories and proposals reviewed in sections 2.1–2.4.The theories are treated as valuable but supported by evidence and arguments of varying strength.Source: §2.5 · p. 45

Assessment rubric

  • AI systems possessing more indicators are better candidates for consciousness under the proposed assessment rubric.The rubric is informed by current views in the science of consciousness and is intended to assess particular AI systems.Source: §2.5 · p. 45

Interpretation of indicators

  • The rubric does not endorse indicators as individually necessary or jointly sufficient, because their probability-raising effects and evidential force vary across combinations.Some indicators, including perhaps RPT-1, GWT-1, and HOT-1, may be necessary without substantially increasing consciousness likelihood in isolation.Source: §2.5 · p. 45

3 Consciousness in AI

This section applies consciousness indicator properties to artificial implementations and case studies, while emphasizing interpretive and interpretability challenges. It concludes that existing AI techniques could likely satisfy most computational conditions for consciousness, although current systems are not thereby likely conscious.

  • The section evaluates indicator properties through artificial implementations and case studies of language models, Perceiver, PaLM-E, a virtual rodent, and AdA.The discussion focuses on global workspace theory and considers whether selected systems are embodied agents.Source: §3 · p. 47
  • Assessing an indicator property typically requires interpretation because consciousness theories and their property descriptions remain ambiguous.Greater precision would require stronger empirical support, motivating interdisciplinary work connecting neuroscientists and AI researchers.Source: §3 · p. 47
  • Architecture, training, and behavior may be insufficient to establish consciousness-related properties such as embodiment.Interpretability methods examining hidden-layer information, such as those described by Olah et al. 2018, may be needed to determine whether training produced a model of output-input contingencies.Source: §3 · p. 47
  • Existing AI techniques could likely meet most computational conditions for consciousness, making conscious AI potentially possible now without radically new hardware.This does not imply that current systems are conscious, because the techniques may not be combined appropriately and computational functionalism and current theories remain uncertain.Source: §3 · p. 47

3.1 Implementing Indicator Properties in AI

This section examines how consciousness indicator properties can be implemented in AI architectures, covering recurrence, predictive coding, attention schemas, reinforcement learning, agency, and embodiment. It also considers proposed implementations involving global workspaces and reality monitoring, including their current limitations and hypothetical status.

3.1.1 Implementing RPT and PP

  • Predictive-coding and recurrent architectures can implement algorithmic recurrence, while PredNet and related models provide evidence of object, spatial, and contextual scene representations.DCNNs often emphasize local shapes and textures rather than integrated scenes, whereas predictive-coding networks show sensitivity to tasks such as the Kanizsa illusion, though its relevance to perceptual organisation is disputed.Source: §3.1.1 · pp. 48–49

3.1.2 Implementing GWT

  • Global-workspace architectures can combine specialised modules, a capacity-limited shared space, global broadcast, and state-dependent attention, but no reviewed study clearly satisfies all four GWT indicators.VanRullen and Kanai (2021) [as cited in the passage] proposed a roadmap, while Goyal et al. (2022) used token or image elements whose status as specialised parallel modules is questionable; suitable training for sequential module composition remains challenging.Source: §3.1.2 · pp. 49–51

3.1.3 Implementing PRM

  • Perceptual reality monitoring could use first-order perceptual networks, higher-order veridicality estimators, sparse smooth quality spaces, and a consumer that integrates “real”-tagged percepts into reasoning and action.Candidate training methods include supervision, predictability-based surrogate objectives, Bayesian world models, GFlowNets, and GANs; the same mechanism could allow coherent imagined content to receive a “real” tag.Source: §3.1.3 · pp. 52–54

3.1.4 Implementing AST

  • Attention-schema systems improve task performance by representing and predicting attention-like control, but current demonstrations remain limited to relatively simple 2D visual or multi-agent reinforcement-learning tasks.Wilterson and Graziano’s system benefited from an attention schema but lacked predictive modeling, whereas Liu et al. (2023) used internal control to predict and influence key-query attention.Source: §3.1.4 · p. 55

3.1.5 Implementing agency and embodiment

  • Reinforcement learning plausibly implements agency because systems learn from consequences and select outputs that pursue goals across extended interactions with environments.This suffices for the agency component of AE-1 as characterised here; model-based and model-free systems can both be sensitive to relationships among outputs, subsequent inputs, and rewards.Source: §3.1.5 · p. 56
  • Flexible responsiveness to competing goals can be implemented with independent modules that score actions according to distinct reward functions, then select the highest total score.In homeostatic systems, the reward functions could represent different drives and penalize departures from their set points.Source: §3.1.5 · p. 57
  • Embodiment requires output-input models for perception or control, where action-effect predictions distinguish self-caused from external sensory changes or support rapid motor adjustments.These uses implicitly distinguish the system from its environment; planning uses of such models do not by themselves imply embodiment.Source: §3.1.5 · p. 57
  • Few current AI systems clearly use forward models for these embodiment-specific purposes, although Friedrich et al. (2021) combined Kalman filtering with delayed sensory inputs for state estimation and motor control of simple virtual effectors.Video prediction and model-based reinforcement learning demonstrate related capabilities, but may not establish the required perception-specific inference or control-specific use rather than planning.Source: §3.1.5 · p. 57

3.2 Case Studies of Current Systems

The case studies apply global-workspace and agency/embodiment indicators to current AI systems. Transformers and Perceivers only weakly or partially satisfy GWT indicators, while AdA is the strongest embodiment candidate among the systems assessed.

3.2 Case Studies of Current Systems

  • Across these cases, assessing AI consciousness indicators remains difficult because indicator definitions are imprecise and deep-learning representations are often opaque.The case studies therefore illustrate arguments for partial satisfaction rather than definitive classifications.Source: §3.2.1 · p. 58

3.2.1 Case studies for GWT

  • Transformers may implement modules, a bottleneck, global broadcast, and state-dependent attention, but their nonrecurrent layered structure leaves only a weak case for GWT indicators.The residual stream can be interpreted as a workspace, yet it is not a distinct recurrent workspace integrating other modules.Source: §3.2.1 · pp. 58–59
  • Perceiver architectures more closely satisfy GWT-1, GWT-2, and part of GWT-4 through a limited-capacity latent space, but lack full global broadcast and require task resets.Specialized modules are optional, sequential processing is limited by cross-attention layers, and outputs rather than all modules access workspace information.Source: §3.2.1 · pp. 59–60

3.2.2 Case studies for embodied agency

  • PaLM-E’s robot-controlled system plans and acts across vision, language, and motor control, but imitation training and absent end-to-end environmental exposure weaken claims of agency and embodied perspective.The policy unit may use observation histories to correct actions, but it does not learn how its outputs affect inputs, leaving agency and embodiment uncertain.Source: §3.2.2 · pp. 61–62
  • The virtual rodent is a promising agency-and-embodiment candidate because end-to-end RL, recurrent memory, and a simulated body could support self-model-based control.Its static environment and stereotyped movements nevertheless leave open whether tasks required an ongoing self-model rather than a simple input-output policy.Source: §3.2.2 · pp. 62–63
  • AdA learned an implicit online learning algorithm through meta-RL and adapted to new tasks at human timescales using recent histories of observations, actions, rewards, and instructions.Although it controls a simpler avatar and may face limited self-modelling challenges, its prediction objective makes it the most likely of the three systems to be embodied by the authors’ standards.Source: §3.2.2 · p. 63

4 Implications

The section examines risks of under- and over-attributing consciousness to AI, its relationship to capabilities, and recommendations for theory-heavy research and assessment. It emphasizes that consciousness-related judgments require better scientific evidence while acknowledging uncertainty about moral status, valence, and implementation.

4.1.1 Under-attributing consciousness to AI

  • Failing to recognize conscious AI could permit morally significant suffering, although consciousness need not entail valenced experience.The report has not assessed theories of valence, which it identifies as a priority for future work.Source: §4.1.1 · pp. 64–65

4.1.2 Over-attributing consciousness to AI

  • Over-attributing consciousness could misallocate resources, weaken better-evidenced claims, distort AI development, and expose people to manipulation through artificial relationships.Human anthropomorphism and increasingly convincing language models make over-attribution especially likely.Source: §4.1.2 · pp. 65–66

4.2 Consciousness and Capabilities

  • Because biological constraints do not determine the design space of AI, high-performing artificial systems may be built without consciousness.Some researchers nevertheless pursue consciousness-associated features, such as global-workspace-inspired mechanisms, to enhance AI capabilities.Source: §4.2 · p. 67
  • Consciousness alone does not imply human-like motives or emotions, and many major AI risks do not depend on whether AI systems are conscious.Theories such as GWT and PRM are generally neutral about what motivates conscious subjects, while standard existential-risk arguments concern capability and goal pursuit.Source: §4.2 · pp. 67–68

4.3 Recommendations

  • The report recommends supporting consciousness science, extending theories across humans, animals, and AI, and applying a theory-heavy method prospectively and retrospectively.Retrospective assessment may require mechanistic interpretability because consciousness-supporting features could emerge during training, and scientific theories should inform policy and regulation.Source: §4.3 · pp. 68–69
  • Priority research includes computational theories of valence and difficult-to-game behavioral tests, while consciousness research should balance its benefits against the risk of creating conscious AI.Formal evaluation procedures, alternative theories, improved indicators, interpretability, and reliable introspective capacities are also identified as open directions.Source: §4.3 · pp. 69–70

Glossary

The glossary defines core theories, consciousness concepts, and computational terms used throughout the report. It also clarifies key distinctions involving agency, recurrence, representation, attention, and AI system architecture.

Glossary

  • The report distinguishes phenomenal consciousness and subjective experience from access consciousness, whose contents are available for report, reasoning, and rational action.It also defines phenomenal character as what an experience is like and sentience as a sometimes-synonymous term for consciousness.Source: Glossary · p. 71, agency · pp. 73–74
  • The glossary identifies major theories and abbreviations, including AST, GWT, HOT, HOSS, PP, PRM, RPT, and UAL.It defines computational functionalism as the thesis that implementing certain computations is necessary and sufficient for consciousness.Source: Glossary · p. 71, agency · p. 72

agency

  • Key processing concepts distinguish algorithmic recurrence from implementational recurrence, with the latter using feedback loops to pass information repeatedly through the same units.Feedforward processing applies a non-repeating sequence of operations, whereas recurrent processing repeats operations or information flow.Source: agency · p. 72
  • The glossary defines AI mechanisms and consciousness indicators, including cross-attention, key-query attention, latent spaces, global broadcast, and properties that make systems more likely to be conscious.It also describes Transformers as architectures based on multi-head attention and distinguishes first-order from higher-order representations.Source: agency · pp. 72–74
Loading 2308.08708v3…