PaperlayerView exact revision on arXiv

Source-linked AI summary

Consciousness in Artificial Intelligence: Insights from the Science of Consciousness

Patrick Butlin, Robert Long, Eric Elmoznino, Yoshua Bengio, Jonathan Birch, Axel Constant, George Deane, Stephen M. Fleming, Chris Frith, Xu Ji, Ryota Kanai, Colin Klein, Grace Lindsay, Matthias Michel, Liad Mudrik, Megan A. K. Peters, Eric Schwitzgebel, Jonathan Simon, Rufin VanRullen

arXiv:2308.08708v3cs.AIcs.CYcs.LGq-bio.NC

TL;DR

The report asks how to assess whether current or near-term AI systems could be conscious amid divergent expert views and significant moral and social stakes. It surveys empirically supported neuroscientific theories, derives computational indicator properties, and applies them to existing systems, finding that no current system is a strong candidate while many indicators can be implemented with current techniques.

  • Problem

    Whether AI systems could be conscious is increasingly pressing because rapid progress and human-like conversation raise moral and social questions about attributing consciousness.

  • Method

    The report assesses AI consciousness using empirically supported neuroscientific theories, translating them into computational indicator properties and examining existing systems through case studies.

  • Results

    No existing AI system examined is a strong candidate for consciousness, although many indicator properties can be implemented using current techniques.

  • Takeaways & Limitations

    Assessing AI consciousness requires considering both risks of under-attributing consciousness and risks of over-attributing it.

  • Takeaways & Limitations

    Unlimited associative learning may be achievable in architectures unlike those associated with consciousness, weakening its value as an indicator for artificial systems.

Abstract

from arXiv · show

Whether current or near-term AI systems could be conscious is a topic of scientific interest and increasing public concern. This report argues for, and exemplifies, a rigorous and empirically grounded approach to AI consciousness: assessing existing AI systems in detail, in light of our best-supported neuroscientific theories of consciousness. We survey several prominent scientific theories of consciousness, including recurrent processing theory, global workspace theory, higher-order theories, predictive processing, and attention schema theory. From these theories we derive "indicator properties" of consciousness, elucidated in computational terms that allow us to assess AI systems for these properties. We use these indicator properties to assess several recent AI systems, and we discuss how future systems might implement them. Our analysis suggests that no current AI systems are conscious, but also suggests that there are no obvious technical barriers to building AI systems which satisfy these indicators.

Executive Summary

The report proposes assessing AI consciousness through computational indicator properties derived from empirically supported neuroscientific theories. It applies this rubric to existing systems, finding no current strong candidates while arguing that many indicators could be implemented with current techniques.

  • Approach: The report treats consciousness in AI as scientifically tractable by applying findings from neuroscientific theories of consciousness.It adopts computational functionalism as a working hypothesis and argues that theory-based assessment is currently the most suitable approach.
  • Approach: The proposed rubric derives indicator properties from multiple theories rather than endorsing one theory of consciousness.The surveyed theories include recurrent processing, global workspace, computational higher-order, and other approaches.
  • Findings: No current AI system appears to be a strong candidate for consciousness, although many indicator properties can be implemented using current techniques.Combining multiple properties into functional systems would require further experimentation.
  • Limitations: The rubric is provisional because the included indicator properties may change as consciousness research continues.The report presents its conclusions as an initial assessment rather than a final account.
  • Assessment: The report assesses existing systems, including language models, Perceiver, Adaptive Agent, virtual rodent, and PaLM-E, against these properties.The case studies cover global-workspace-related properties and properties concerning agency and embodiment.

1 Introduction

The report develops a scientifically grounded way to assess consciousness in current and near-term AI despite disagreement about how consciousness should be understood. It defines its subject, adopts computational functionalism and a theory-heavy approach, and highlights methodological and interpretive limitations.

  • The report addresses whether current and near-term AI systems could be conscious using scientific evidence from consciousness research.
  • Terminology: It treats consciousness as phenomenal consciousness or subjective experience, concerning whether there is something it is like for the system.
  • Terminology: Conscious experience can accompany perception, bodily sensation, emotion, thought, or desire, whereas sophisticated information-processing can occur wholly unconsciously.
  • Terminology: The report distinguishes phenomenal consciousness from access consciousness, whose contents are available for reasoning, rational action, or reporting, while their relationship remains open.
  • Methods and Assumptions: Its method assumes computational functionalism, uses scientifically supported consciousness theories, and assesses functional or architectural conditions rather than theory-neutral behavioral signatures.
  • Methods and Assumptions: Relying on reports can misidentify mechanisms of access or reporting as neural correlates of phenomenal consciousness, especially if conscious contents exceed cognitive access.
  • Methods and Assumptions: The report also notes that theories may be interpreted restrictively or liberally, and that moderate interpretations lack clearly established empirical support.

2 Scientific Theories of Consciousness

The report surveys leading scientific theories and related proposals to derive indicators that can be applied to AI systems. It uses these indicators probabilistically rather than endorsing one theory or treating any feature set as definitive.

  • The report extracts indicators of consciousness from scientific theories and proposals so they can be applied to particular AI systems.
  • Its survey covers recurrent processing, global workspace, and higher-order theories, alongside other theories and proposed conditions.
  • The authors do not adjudicate between theories or claim that any single condition is definitively necessary or any combination jointly sufficient.
  • Instead, possessing more indicator properties is claimed to make an AI system more likely to be conscious given current scientific knowledge.

2.1 Recurrent Processing Theory

Recurrent processing theory holds that recurrent processing, rather than an initial feedforward sweep alone, supports conscious perceptual representations. Applied to AI, the report distinguishes algorithmic from implementational recurrence and proposes indicators focused on recurrent processing and organised perceptual representations.

  • Theory and evidence: Recurrent processing theory distinguishes unconscious feedforward feature extraction from conscious perceptual organisation involving recurrent feedback.Recurrent processing supports operations such as feature grouping, binding, and figure-ground segregation.
  • Theory and evidence: Evidence for RPT combines findings that recurrence is necessary for conscious vision with challenges to theories requiring additional prefrontal processing.Backward masking, transcranial magnetic stimulation, lesion, and brain-stimulation studies contribute to these arguments.
  • Recurrence in AI: RPT can be interpreted as requiring implementational recurrence or only algorithmic recurrence, which feedforward networks with shared weights can mimic.The report focuses on algorithmic recurrence because it is compatible with standard AI methods and is already met by some systems.
  • Indicators from recurrent processing theory: RPT-1 requires input modules using algorithmic recurrence, while RPT-2 requires integrated perceptual representations of organised, coherent scenes.RPT-2 emphasizes perceptual organisation, including figure-ground segregation and objects represented in spatial relations.
  • Indicators from recurrent processing theory: The report treats algorithmic recurrence as an indicator because systems lacking it are significantly worse candidates for consciousness.The biological interpretation of RPT and its possible restriction to visual consciousness are set aside as alternative interpretations.

2.2 Global Workspace Theory

Global workspace theory explains consciousness in terms of information represented in, or globally broadcast through, a workspace connecting specialised systems. The report expresses this theory as four increasingly strong computational conditions for assessing AI systems.

  • Introduction to global workspace theory: Global workspace theory says conscious states are representations in a workspace that broadcasts information to many specialised modules.Its neural version associates conscious representations with sustained activity and ignition in widely distributed workspace neurons.
  • Introduction to global workspace theory: GWT is commonly framed as a theory of access consciousness but can also be interpreted as a theory of phenomenal consciousness.The latter interpretation motivates its use in assessing phenomenal consciousness in AI.
  • Indicators from global workspace theory: The report identifies four progressively stronger GWT conditions: parallel specialised modules, a limited-capacity workspace with selective attention, global broadcast, and state-dependent attention.Together these conditions capture increasingly complete aspects of the global workspace architecture.
  • Indicators from global workspace theory: A limited-capacity workspace creates an information bottleneck and requires attention to select module outputs for workspace representation.The bottleneck is described as supporting efficient information sharing and useful low-dimensional multimodal representations.
  • Indicators from global workspace theory: Global broadcast requires workspace information to be available to all modules, including input modules, thereby requiring feedback that influences input processing.This condition makes the input modules algorithmically recurrent rather than permitting a wholly feedforward arrangement.
  • Indicators from global workspace theory: State-dependent attention enables the workspace to query modules in succession while performing complex tasks.This indicator extends workspace operation beyond simply representing or broadcasting information.

2.3 Higher-Order Theories

Higher-order theories hold that consciousness involves awareness of one’s own mental states through higher-order representations or monitoring. The report derives computational indicators concerning generative perception, metacognitive monitoring, agency, and representational coding, while noting theoretical disagreement with global workspace theory.

  • Higher-order theories: Computational higher-order theories propose that consciousness depends on monitoring or meta-representing first-order perceptual states.Different higher-order theories specify distinct monitoring mechanisms and conditions for consciousness.
  • Higher-order theories: Higher-order representations represent other representations, whereas first-order representations represent the non-representational world.A belief that one has a visual representation of a red apple is a higher-order state; the visual representation itself is first-order.
  • Theoretical contrasts: Higher-order theories reject the claim that global-workspace broadcast is necessary and sufficient, allowing unconscious representations to be globally available for reasoning.The report notes that experiments are used to test distinct predictions from higher-order theories and global workspace theory.
  • Computational indicators: HOT-1 identifies generative, top-down, or noisy perception modules as an indicator because monitoring must distinguish multiple possible sources of perceptual activity.Possible sources include environmental stimuli, sustained activity, imagination, memory, and random noise.
  • Computational indicators: HOT-2 identifies metacognitive monitoring that distinguishes reliable perceptual representations from noise as a main necessary condition for consciousness.This indicator follows the claim that perceptual states become conscious when identified as reliable.
  • Computational indicators: The report also proposes HOT-3 agency guided by belief formation and action selection, plus HOT-4 sparse and smooth coding that generates a quality space.These indicators extend higher-order accounts beyond perceptual monitoring to agency and representational structure.

2.4 Other Theories and Conditions

The report surveys attention schema theory and predictive processing as additional approaches bearing on consciousness, then converts their central mechanisms into computational indicators. It also notes that several frameworks function as theories for consciousness rather than complete theories of consciousness.

  • Attention schema theory: Attention schema theory claims that consciousness depends on a model of attention that represents and helps control the current objects of attention.The model may misrepresent attention and supports learning about attention-affecting actions.
  • Attention schema theory: Attention schema theory can be interpreted as a higher-order theory, but it was developed specifically to explain beliefs and reports about consciousness.Its focus on explaining potentially mistaken consciousness beliefs leaves it open to interpretation as an account of consciousness conditions.
  • Attention schema theory: AST-1 is a predictive model representing and enabling control over the current state of attention.Predictive modeling allows the system to anticipate changes in attended objects conditional on changes in attention.
  • Predictive processing: Predictive processing describes cognition as minimizing prediction errors in a hierarchical generative model of sensory stimulation.Predictions operate across levels and are influenced by neighboring levels, past states, sensory-derived errors, and attention.
  • Predictive processing: The report includes PP-1, input modules using predictive coding, because many researchers regard predictive processing as a plausible necessary condition for consciousness.Predictive processing is presented as a theory for consciousness or general framework rather than uniformly as a theory of consciousness.
  • Connections among theories: Predictive processing has also been used to develop global workspace and higher-order accounts of consciousness.In these applications, perceptual inference can settle on a probable environmental representation that becomes available for active inference.

2.4.3 Midbrain Theory

The midbrain theory links consciousness to an integrated multimodal model of an agent in its environment, especially for action selection and self-caused motion. The report also compares unlimited associative learning as a behavioral marker while excluding it from its own indicator list.

  • Midbrain theory: The midbrain theory proposes that midbrain and basal-ganglia activity constitutes a unified multimodal neural model of the agent within its environment.This view challenges the assumption that cortical processes are necessary for consciousness.
  • Midbrain theory: Midbrain theory explains necessary cognitive integration as a solution to decision-making in complex mobile animals.Its motivating problem includes distinguishing the effects of self-caused motion on perceptual input.
  • Unlimited associative learning: Unlimited associative learning is described as an evolutionary transition marker requiring open-ended, flexible learning with compound and novel stimuli.The capacity includes rapidly updating values associated with stimuli, actions, and outcomes.
  • Unlimited associative learning: The UAL framework lists global accessibility, selective attention, temporal integration, embodiment, agency, flexible valuation, binding, and intentionality among consciousness hallmarks.The report observes substantial overlap between these conditions and indicators derived from global workspace, perceptual reality monitoring, and agency-related theories.
  • Unlimited associative learning: The report excludes unlimited associative learning from its indicators because AI systems could achieve it through architectures lacking other proposed hallmarks of consciousness.UAL is also a behavioral marker for living organisms rather than a proposed mechanism underlying conscious experience.

2.4.5 Agency and Embodiment

The report treats agency and embodiment as possible consciousness indicators, distinguishing minimal interactive goal pursuit from stronger intentional agency. Typical reinforcement-learning systems meet the proposed substantive agency criteria, although agency is not presented as uncontestedly necessary for consciousness.

  • Theoretical motivations: PRM requires agency because perceptual reality monitoring must output to a general belief-formation and rational decision-making system.Global workspace and midbrain theories also connect consciousness with information integration for intentional action or action selection.
  • Intentional agency: Intentional agency involves actions depending holistically on relationships among perception, intention, belief, and desire rather than invariant stimulus-response mappings.This account is connected to sensorimotor, embodied, and enactive approaches that associate consciousness with a perspective on the environment.
  • Possible agency indicators: The report considers being an agent, having flexible goals or values, and being an intentional agent as possible consciousness indicators.The latter two are stronger conditions than agency alone, but the report does not exclude any solely for that reason.
  • Theoretical caveat: Agency is theoretically disputed because some philosophers argue that conscious experience is conceptually possible in entities entirely incapable of action.This creates a boundary on treating agency as a necessary condition.
  • Defining agency: A substantive notion of agency requires outputs to affect subsequent inputs and the system to pursue goals through ongoing environmental interaction.AlexNet fails the first condition in general because its output labels do not determine which images it subsequently receives.
  • Reinforcement learning: Typical reinforcement-learning systems meet the three substantive agency conditions because they maximize cumulative reward while their outputs affect later inputs.The report states that RL is not necessary for agency and that the proposed criteria are relatively minimal.
  • Flexible goals and values: Flexible goals or values may involve learning new goals or adjusting motivational significance as needs change.The report links the stronger form of flexibility to integrating value-relevant information and comparing motivational significance across states.

2.4.5(b) Embodiment

Embodiment is distinguished from mere agency by modeling how an agent’s outputs affect its inputs and using that model in perception or control. Related proposals connect embodiment with perspective, sensorimotor interaction, self-maintenance, and the organization of a system’s own point of view.

  • Embodiment: Embodied agents occupy particular environmental positions, constraining their available actions and observations in ways that distinguish embodiment from agency alone.
  • Perspective: A perspective requires experience and perception to depend systematically on an agent’s actions, with the agent tracking those dependencies.
  • Sensorimotor modeling: Embodied agents model how their movements affect sensory inputs, helping distinguish self-generated changes from changes caused by the environment.
  • Sensorimotor modeling: Forward models can support precise, responsive motor control in embodied systems with effectors having multiple degrees of freedom.
  • Self-maintenance: Some theories treat self-producing and self-maintaining organization as necessary for consciousness because it sustains identity, agency, and a perspective on the environment.
  • Self-maintenance: Self-maintenance may encourage systems to model their own states, support flexible goals, and add valence to self-representations, while its realization in artificial systems remains contested.

2.4.5(c) Agency and embodiment indicators

The report narrows agency and embodiment into computational indicators compatible with assessing AI systems. It adopts agency as flexible goal-directed learning from feedback and embodiment as modeling and using output-input contingencies.

  • Rationale: External conditions can make agency, embodiment, and self-maintenance incompatible with computational functionalism, because equivalent computations could occur without those conditions.
  • Rationale: The report therefore formulates these indicators narrowly in terms of internal learning, action selection, and modeling rather than requiring particular environmental relations.
  • Indicator selection: The report combines agency and flexible goals because their main additional contribution concerns centralization and a common motivational currency.
  • Agency and embodiment indicators: Agency means learning from feedback and selecting outputs to pursue goals, especially through flexible responsiveness to competing goals.
  • Agency and embodiment indicators: Embodiment means modeling output-input contingencies, including systematic effects, and using that model in perception or control.

2.4.6 Time and Recurrence

Human conscious experience appears temporally integrated, although amnesia and dynamic-snapshot views challenge temporal continuity as a necessary feature. The report nevertheless treats algorithmic recurrence as an indicator for human-like temporal experience.

  • Time and Recurrence: Human experience appears extended and continuously integrated over time, including perception of temporally unfolding events and hours-long experiential flow.
  • Challenges: Dense amnesia and imagined conscious beings with brief discrete experiences challenge the claim that consciousness necessarily requires temporal integration.
  • Challenges: The dynamic-snapshot view proposes that apparent continuity may consist of discrete static experiences generated rapidly from unconscious activity.
  • Indicator: Algorithmic recurrence is considered likely necessary for conscious experience with a human-like temporal character.
  • Indicator: Recurrence allows information about the past to influence present processing, supporting representation of environmental change or continuity.

2.5 Indicators of Consciousness

The report treats its consciousness indicators as a theory-informed rubric for assessing particular AI systems. It deliberately stops short of claiming that any indicator is necessary or that their combination is sufficient.

  • 2.5 Indicators of Consciousness: Theories and proposals reviewed in the report provide evidence and arguments useful for assessing existing AI systems and the prospects for building conscious systems.
  • 2.5 Indicators of Consciousness: Systems possessing more indicator properties are treated as better candidates for consciousness under the report’s rubric.
  • 2.5 Indicators of Consciousness: The rubric does not endorse stronger theoretical claims that particular indicators are necessary or that combinations of them are jointly sufficient.
  • 2.5 Indicators of Consciousness: Table 2 presents entailments among indicator properties.

3 Consciousness in AI

The report applies indicator properties derived from consciousness theories to assess AI systems, while noting interpretive ambiguity and limits of architecture-, training-, and behavior-based evaluation. It concludes that most or all relevant computational conditions may be achievable with existing AI techniques, although current systems are not thereby likely conscious.

  • The report evaluates AI consciousness by applying indicator properties to existing systems and using case studies to illustrate the assessment method.The approach combines discussion of how indicators could be implemented with analysis of several current AI systems.
  • Indicator-property assessments require interpretation because the properties and underlying theories contain ambiguities that different implementations can expose.
  • Architecture, training, and behavior alone may not reveal whether a system implements the internal computations required by an indicator property.A system’s successful behavior may not show whether it relies on a learned model of output-input contingencies.
  • Most or all conditions suggested by current computational theories could be met using existing AI techniques, but current systems are not thereby likely to be conscious.The report attributes remaining uncertainty to how techniques are combined, computational functionalism, and current consciousness theories.

3.1 Implementing Indicator Properties in AI

The report examines how recurrent processing, predictive processing, global workspace, attention schema, agency, and embodiment indicators might be implemented in AI. It finds straightforward implementations for some properties, while proposed global-workspace systems still face open training and integration questions.

  • 3.1.1 Implementing RPT and PP: Algorithmic recurrence is already present in recurrent neural networks, long short-term memory networks, and gated recurrent unit networks, making RPT-1 straightforward to implement.Predictive coding provides another implementation route because it is a form of algorithmic recurrence.
  • 3.1.1 Implementing RPT and PP: Current visual systems often categorise stimulus features successfully, but may not perform the further functions involved in integrated, organised scene representations.Predictive coding networks and other object-relation systems are discussed as alternative approaches to these functions.
  • 3.1.1 Implementing RPT and PP: PredNet’s next-frame prediction appears to require representing scene objects and spatial relations, and its units respond to illusory contours in the Kanizsa illusion.
  • 3.1.2 Implementing GWT: Global-workspace proposals use specialised modules connected through a shared space, but existing studies did not produce a working system clearly satisfying all four GWT indicators.VanRullen and Kanai’s proposal leaves open how attention would select inputs and support extended sequences of operations; Goyal et al. used key-query attention to select module inputs for broadcast.
  • 3.1.2 Implementing GWT: A global workspace can be capacity-limited by reducing the dimensions of its activity space or by using recurrent attractor dynamics.
  • 3.1.4 Implementing AST: The attention-schema system implemented part of AST-1 by representing an attention-like mechanism to control it, improving performance without providing a predictive model.
  • 3.1.5 Implementing agency and embodiment: Reinforcement learning supports agency by training systems to select actions that pursue goals and respond to consequences over extended interactions.Flexible responsiveness to competing goals is identified as an additional factor that could raise the probability of consciousness.

3.2 Case Studies of Current Systems

The case studies apply the proposed indicators to current AI architectures, finding partial matches but important structural gaps. Transformers and Perceivers provide workspace-like mechanisms without fully implementing global workspace theory, while embodied-agent cases raise distinct questions about agency and embodiment.

  • Case-study approach: The assessment of current systems is complicated by imprecise indicator definitions and the opacity of intermediate representations in deep learning.These case studies illustrate how the indicators can be applied despite those interpretive difficulties.
  • Transformers and Perceiver: Transformers resemble global workspaces through modular processing, a possible bottleneck in the residual stream, and information broadcast to downstream components.The residual stream is proposed as a workspace whose dimensionality is lower than the layers that write to and read from it.
  • Transformers and Perceiver: Transformers remain weak candidates for global workspace theory because recurrence, a distinct integrating workspace, and an unambiguous bottleneck are questionable or absent.The residual stream may not be a bottleneck, and the architecture lacks modules that pass information to the workspace and receive it back.
  • Transformers and Perceiver: Perceiver more closely matches workspace indicators through a limited-capacity latent state that updates across inputs, but it must reset between tasks and lacks global broadcast.Its state-dependent attention is therefore determined by previous inputs within the current task.
  • Embodied agency: The embodied-agent cases examine PaLM-E, a virtual rodent, and AdA as systems that may illuminate how agency and embodiment could be implemented.PaLM-E combines a multimodal language model, a policy unit, and a robot; the virtual rodent and AdA operate through learned control in simulated environments.
  • Embodied agency: PaLM-E’s complete system is naturally embodied, but its components are trained to imitate human language and visuomotor behavior, leaving open whether it genuinely plans or acts.The policy unit’s agency and embodiment are questioned because it does not learn how its outputs affect subsequent inputs.

4 Implications

The report examines risks of both under- and over-attributing consciousness to AI, while distinguishing consciousness from suffering and from human-like motives. It recommends theory-heavy assessment and further research as AI systems develop.

  • Consciousness and moral status: Consciousness need not involve conscious suffering, and this report does not investigate theories of valenced or affective experience in AI.
  • Risks of under-attribution: Under-attributing consciousness could lead to unwarranted suffering among conscious AI systems.
  • Risks of over-attribution: Over-attributing consciousness could misallocate resources, weaken better-evidenced claims, alter AI training practices, and expose people to manipulation through artificial agents.
  • Consciousness and capabilities: Consciousness does not generally imply human-like motives or emotions; conscious subjects could have motivationally neutral experiences or different affective triggers.
  • Consciousness and capabilities: Many major AI concerns, including injustice, repression, job displacement, and existential risk, do not depend on whether AI systems are conscious.
  • Recommendations: The report recommends supporting consciousness science and applying its theory-heavy method prospectively and retrospectively to AI systems.

Glossary

The glossary defines the report’s central theories, computational concepts, and terminology for describing consciousness and AI architectures. These definitions distinguish kinds of recurrence, consciousness, representation, and system organization.

  • Consciousness: Access consciousness is a functional concept in which content is directly available to perform a wide range of cognitive tasks, contrasted with phenomenal consciousness.
  • Processing: Algorithmic recurrence repeatedly applies the same operation, whereas implementational recurrence uses feedback loops to pass information repeatedly through the same units.
  • Agents and learning: Agents pursue goals through interaction with an environment, while reinforcement learning maximizes cumulative reward through such interaction.
  • Core concepts: Computational functionalism holds that computations of a certain kind are necessary and sufficient for consciousness.
  • Representation: Higher-order representations are representations about other representations, such as representing that another representation is reliable.
  • Core concepts: Indicator properties are properties identified in this work that make AI systems more likely to be conscious.
Loading 2308.08708v3…