PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Opportunities and Risks of Foundation Models

Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, Percy Liang

arXiv:2108.07258v3cs.LGcs.AIcs.CY

TL;DR

Foundation models’ broad, emergent capabilities create major uncertainty about how they work, what they can do, and when they fail. This report surveys their adaptation, applications, and societal risks, concluding that widespread adoption warrants norms, auditing, and release safeguards.

  • Problem

    The report addresses limited understanding of foundation models’ emergent capabilities, risks, and unexpected failure modes across diverse applications.

  • Method

    The report synthesizes foundation-model adaptation methods, applications, technical challenges, and societal implications across domains.

  • Results

    The report identifies homogenization and centralization of power as societal risks and recommends development, auditing, and release norms.

  • Takeaways & Limitations

    Foundation-model research should characterize capabilities and risks while developing alignment methods and coordinating proactive mitigation measures.

  • Takeaways & Limitations

    Foundation models remain difficult to understand and exhibit unexpected failure modes because their power arises from emergent qualities.

Abstract

from arXiv · show

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

1 INTRODUCTION

This section examines foundation models as broadly pretrained, adaptable systems whose scale enables emergent capabilities and wide reuse while making their behavior, failures, and societal effects difficult to understand. It surveys their technical foundations, capabilities, applications, and sociotechnical consequences, emphasizing shared flaws across adapted models and the need for interdisciplinary oversight and governance.

  • 1 INTRODUCTION: Foundation models are trained on broad data at scale and adapted to many downstream tasks; GPT-3 demonstrates emergent in-context learning despite not being explicitly trained for those tasks.Examples include BERT [Devlin et al. 2019], GPT-3 [Brown et al. 2020], and CLIP [Radford et al. 2021]; GPT-3 has 175 billion parameters.
  • 1.1 Emergence and homogenization: Foundation models combine transfer learning and scale, producing emergent capabilities and unprecedented homogenization that leverage shared improvements but propagate shared biases and failures.Scale depends on improved hardware, Transformers [Vaswani et al. 2017], and abundant training data; GPT-3’s in-context learning was neither specifically trained nor anticipated.
  • 1.1.1 Naming.: The term foundation model emphasizes that these incomplete models form a common basis for many adapted systems and require scrutiny of their architectural stability, safety, and trustworthiness.The term distinguishes the paradigm’s sociological impact from narrower labels such as pretrained model, language model, or task-agnostic model.
  • 1.2 Social impact and the foundation models ecosystem: Foundation models’ social impact arises through an ecosystem spanning data creation, curation, training, adaptation, and deployment, with direct effects occurring when systems reach people.Responsible assessment requires surrogate metrics for representative downstream uses and documented metrics [Mitchell et al. 2019], while deployment should receive rigorous testing and auditing.
  • 1.3 The future of foundation models: Because foundation models remain poorly understood research prototypes with uncertain capabilities and harms, their development requires interdisciplinary participation and attention to incentives from the outset.Commercial incentives may support accuracy and safety but can also neglect marginalized populations, social externalities, and open decentralized development; academia can pursue broader social benefit.
  • 1.3 The future of foundation models: Foundation models’ centralized resource requirements threaten accessibility and reproducibility, motivating public infrastructure and collaboration between academia and industry for responsible development and deployment.Industry resources increasingly exceed those of academia, while model scale and engineering demands limit access; public computing infrastructure is proposed to close the gap.
  • 1.4 Overview of this report: The report integrates 26 sections across capabilities, applications, technology, and society to connect technical development with domain-specific and societal concerns.Its interdisciplinary community includes AI researchers, domain experts, and researchers focused on ethics and economics, though coverage remains incomplete as the field evolves.
  • 1.4.1 Overview of capabilities.: The capabilities overview spans language, vision, robotics, reasoning, interaction, and understanding, emphasizing both foundation models’ broad potential and unresolved limits.The report highlights transfer across tasks, multimodal and generative interaction, robotics data and safety challenges, and the possibility that skepticism about language understanding may be premature.
  • 1.4.2 Overview of applications.: The applications overview examines healthcare, law, and education, where foundation models could improve expert-intensive work but require advances in capability, privacy, fairness, interpretability, and ethics.Potential benefits include multimodal biomedical discovery, legal generation and reasoning, and adaptive educational interaction; risks include medical bias, factual unreliability, privacy concerns, inequitable access, and plagiarism.
  • 1.4.3 Overview of technology.: The technology overview identifies modeling, training, adaptation, evaluation, systems, data, security, robustness, safety, and theory as interconnected foundations for improving models responsibly.It calls for architectures with expressivity, scalability, multimodality, memory, and compositionality; transparent data ecosystems; broader evaluation; secure and robust adaptation; and theory addressing the gap between pretraining and adaptation.
  • 1.4.3 Overview of technology.: Interpretability research must address foundation models’ opacity, broad task applicability, emergent properties, and the relationship between one foundation model and its many adapted derivatives.The proposed one model-many models paradigm examines shared decision-making components, post hoc explanations, behavioral mechanisms, and the societal impact of interpretability or non-interpretability.
  • 1.4.4 Overview of society.: Foundation models’ task agnosticity makes their wide-ranging societal impact difficult to anticipate across possible systems and use cases.The report emphasizes that consequences must be considered beyond specific systems deployed to users.
  • 1.4.4 Overview of society.: Foundation models may amplify inequity through intrinsic biases and application-specific extrinsic harms, requiring source tracing, proactive interventions, and reactive recourse.Relevant sources include training data, developer diversity, and broader sociotechnical contexts; proposed responses include counterfactual data augmentation, feedback propagation, and attribution of responsibility.
  • 1.4.4 Overview of society.: Foundation models can enable higher-quality, personalized disinformation and deepfakes that challenge human detection, while also supporting automated misuse detection.Misuse is defined as technically intended use pursued to cause societal harm.
  • 1.4.4 Overview of society.: Foundation models’ computationally intensive training creates environmental costs, motivating efficient models, hardware, and energy grids, environmental evaluation, and stronger documentation and measurement.The report distinguishes large one-time training costs from their amortization across repeated use and calls for comparison with more environmentally friendly baselines.
  • 1.4.4 Overview of society.: The report identifies uncertainty around foundation models’ legal status, liability, protections, economic effects, and ethical risks involving homogenization, concentrated power, and release strategies.It focuses on productivity, wage inequality, and ownership concentration while emphasizing the intermediary status of foundation models in legal analysis.

2 CAPABILITIES

This section examines the capabilities of foundation models across language, vision, robotics, reasoning, interaction, and multimodal understanding. It considers their broad adaptability and opportunities alongside unresolved challenges in generalization, data, evaluation, safety, reliability, grounding, interpretation, and the nature of understanding.

  • 2.1.1 The nature of human language.: Language is central to human communication, thought, social identity, relationships, knowledge, and societal intelligence, making machine understanding and generation a central artificial-intelligence research goal.NLP, automatic speech recognition, and text-to-speech seek to give computers human-like language understanding and generation.
  • 2.1.2 Impact of foundation models on NLP.: Foundation models now underpin NLP, with one adaptable model supporting many linguistic tasks and substantially outperforming task-specific systems, including a rise from 73.1% to 91.6% on the NY Regents science exam.Their generative training paradigm also supports coherent language generation, summarization, dialogue, and speech recognition after adaptation.
  • 2.1.3 Language variation and multilinguality.: Foundation models may extend NLP across languages through multilingual transfer, but uneven data, parameter competition, assimilation, and English-centered resources leave robust and equitable language representation unresolved.Current multilingual models can help low-resource languages, with GShard showing its largest gains over monolingual baselines for the lowest-resource languages as model size increases.
  • 2.1.4 Inspiration from human language acquisition.: Compared with humans, foundation models learn from far more mostly ungrounded text and often lack systematic, consistent, and readily evolving linguistic abstractions, motivating grounded and cognitively aligned learning.GPT-3 uses around three to four orders of magnitude more language data than most humans ever encounter, while model linguistic systems remain relatively static without substantial retraining.
  • 2.2 Vision: Vision foundation models use self-supervision on diverse raw sensory data to produce visual knowledge adaptable across tasks, reducing dependence on expensive annotations but remaining early-stage relative to language models.Early progress includes improved traditional-task generalization, competitive image classification and object detection without explicit training annotations, and multimodal visual synthesis such as DALL-E and CLIP-guided generation.
  • 2.2.1 Key capabilities and approaches.: Beyond core perception, vision foundation models aim at physical scene understanding, visual commonsense and temporal reasoning, and social affordances, but these capabilities remain difficult and their roadmap is open.New architectures, large-scale training, self-supervision, and few-shot adaptation may enable progress where annotation-intensive systems struggle, such as commonsense visual question answering and robust gaze perception.
  • 2.2.2 Central research challenges.: Vision foundation models still struggle with human-level systematic generalization, perceptual robustness, long-range dynamics, and efficient processing of high-dimensional multimodal video.Models can fail on simple compositional shape and color tasks, while patch or frame embeddings may lose fine-grained details; efficient modeling may require new architectural primitives and systems advances.
  • 2.2.2 Central research challenges.: Realizing vision foundation models requires diverse multimodal datasets, realistic interactive environments, and evaluation metrics that better capture semantic faithfulness than existing automated measures.Human judgments may improve evaluation but are costly and difficult to scale.
  • 2.3 Robotics: Robotic foundation models could enable generalist robots through multimodal task specification and learning, but they must operate in high-dimensional closed loops using data spanning diverse environments, tasks, and embodiments.Potential applications include manufacturing, construction, autonomous driving, household aid, and personal assistance.
  • 2.3.1 Opportunities.: Robotic foundation models could convert varied human task descriptions into reward signals and use multimodal, self-supervised representations to adapt perception and control efficiently across new tasks, environments, and embodiments.Candidate objectives include dynamics modeling, policy learning, inverse reinforcement learning, and prediction across synchronized sensor modalities.
  • 2.3.2 Challenges and risks.: Robotic foundation models face major hurdles in collecting sufficiently large and diverse data and ensuring safe, robust deployment under distribution shift and unexpected behavior.Promising data sources include teleoperation, human videos, web datasets, simulation, and autonomous interaction, but sim-to-real transfer and formal safety guarantees remain open challenges.
  • 2.4 Reasoning and search: Multimodality can help foundation models combine formal symbolic reasoning with visual equivalence, symmetry, and Euclidean geometry to prune search spaces and identify promising solutions.This approach is described as mimicking human reasoning about geometry problems.
  • 2.4 Reasoning and search: Foundation models support reasoning over unbounded search spaces by generating arbitrary actions, transferring heuristics across tasks, and grounding symbolic concepts in multimodal data.Applications include theorem proving, program synthesis, drug discovery, chemical synthesis, design, and combinatorial optimization; scaling model size and pretraining improve reasoning performance.
  • 2.4.3 Future challenges in reasoning.: Reasoning research remains constrained by scarce high-quality data, limited self-supervised-task design, weak high-level planning, and unresolved questions about reliable, interpretable, robust, out-of-domain reasoning.Interactive human guidance, synthetic theorems, and self-supervision may alleviate data scarcity, but scaling abstract reasoning remains open.
  • 2.5 Interaction: Foundation models lower the development threshold for AI-infused applications and raise the ceiling of achievable interactions through natural-language inputs, generalization, generation, and multimodality.These benefits can let non-ML experts prototype sophisticated applications without collecting extensive data or training large models from scratch.
  • 2.5.1 Impact on AI-infused application developers’ development process.: Foundation models’ generality and high ceiling also make applications difficult to manage because their behavior can be unpredictable, complex, and hard to understand or reliably direct.Suggested approaches include fine-tuning, prompt engineering, calibration, and pre-formatting task-specific endpoints.
  • 2.5.2 Impact on end-user interaction with AI-infused applications.: Foundation models can diversify end-user interaction by enabling novice creators to produce high-quality multimedia from coarse specifications and by supporting richer multimodal tools.Examples include collaborative writing, text-to-image generation, music mastering, code completion, multimedia remastering, personalized games, and visual or gesture-based interfaces.
  • 2.5.2 Impact on end-user interaction with AI-infused applications.: These applications must preserve user agency and values while addressing bias, surprise, authorship, trust, responsibility, and broader effects on communication, work, language, and culture.Because models learn from observed data rather than causality, their deployment may reproduce the past instead of producing a desired future.
  • 2.5.3 Blurring the line between developers and end-users.: By enabling few-shot or zero-shot, prompt-based customization, foundation models could let communities and individuals participate in developing applications tailored to their needs, although bias, robustness, and manageability remain obstacles.This opportunity could blur the developer–user boundary, but non-ML experts may struggle to understand model capacities and mechanisms.
  • 2.6 Philosophy of understanding: Foundation models currently produce fluent language but sometimes lapse into incoherence, and self-supervision directly teaches symbol co-occurrences rather than obvious meanings; nonetheless, skepticism that future models could understand language may be premature.The models can learn associations across language, code, images, audio, databases, and sensor readings, but it remains unclear whether such associations constitute understanding.
  • 2.6.2 What is at stake?: Language understanding may be necessary for trust, interpretability, and accountability in deployed systems, motivating a framework for theorizing about it.Understanding alone does not guarantee trust, but it may support internal-model-based interpretability and serve as a prerequisite for accountability.
  • 2.6.3 What is understanding?: Understanding has distinct internalist, referentialist, and pragmatist interpretations, which imply different relationships between linguistic behavior, internal structure, and the world.Internalism emphasizes internal representations, referentialism mappings to referents or truth conditions, and pragmatism appropriate linguistic dispositions.
  • 2.6.3 What is understanding?: Multimodal digital traces may enable foundation models to learn high-fidelity proxies for semantic mappings, whereas linguistic inputs alone may be fundamentally limited.Images, audio, sensors, and other traces could provide information linking language to the world, though practical data and causal-inference limitations remain.
  • 2.6.3 What is understanding?: Behavioral benchmarks provide imperfect evidence of understanding because tests can be circumvented, targets are difficult to define, and genuine internal mappings may remain unobserved.If internalism or referentialism is the target, structural evaluation should probe representations, internal dynamics, and causal effects, while current benchmark reactions illustrate the difficulty.
  • 2.6.4 Moving the discussion forward.: Multimodal training regimes may be the most viable path toward language understanding, but whether self-supervision is sufficient remains completely open.The conclusion does not resolve whether foundation models can understand language because the answer depends on unresolved metaphysical and epistemological questions.

3 APPLICATIONS

This section examines how foundation models could expand access and adapt across tasks in healthcare, biomedicine, law, and education. It also addresses the domain-specific reasoning, data, reliability, fairness, privacy, safety, accountability, and human-impact challenges that constrain deployment, including the need for specialized data and adaptation to diverse settings.

  • 3.1 Healthcare and biomedicine: Foundation models can serve as adaptable interfaces connecting diverse medical data, professionals, patients, and downstream healthcare or biomedical tasks.They may improve efficiency and accuracy through querying, updating, fine-tuning, and prompting, while requiring multimodal integration and adherence to privacy, safety, and explainability regulations.
  • 3.1.1 Opportunities in healthcare.: Foundation models could improve healthcare delivery by supporting EHR summarization, diagnosis and treatment suggestions, patient question answering, public-health information, and surgical assistance.These applications target administrative inefficiency, preventable medical errors, provider shortages, and urgent pandemic needs, but medical interfaces must guarantee factual accuracy to preserve public trust.
  • 3.1.2 Opportunities in biomedicine.: Foundation models could accelerate biomedical discovery through generative protocol and molecule design, multimodal disease analysis, cross-modal transfer, personalized treatment selection, and clinical-trial optimization.They may reduce experiments in drug discovery, integrate EHR, imaging, molecular, and genetic data, and predict trial failures or match eligible patients.
  • 3.1.3 Challenges and future research in foundation models.: Healthcare deployment requires multimodal learning, explainable decisions, legally compliant privacy, safety and uncertainty guarantees, representative data, fairness, and improved extrapolation to new technologies and diseases.Current models generally train within separate modalities, lack explainability objectives, may reproduce dataset bias, and have unclear extrapolation mechanisms in biomedical settings.
  • 3.2 Law: In U.S. law, foundation models could reduce procedural and financial barriers to legal services, especially where inadequate representation and heavy caseloads restrict access to justice.Legal applications face specialized language, ambiguous standards, unseen fact patterns, scarce labeled data, and a need to adapt across modalities and jurisdictions.
  • 3.2 Law: Because legal decisions have consequential real-world effects, foundation models require thorough evaluation, transparency, accountability, explainability, and ethical, legal, and fairness review before deployment.These requirements make it questionable whether current models can address many pressing legal problems despite the goal of expanding legal and government services.
  • 3.2.1 Opportunities in law.: Foundation models could support civil-law workflows from client issue identification and tailored guidance through contract review, translation, legal research, drafting, discovery, trial preparation, and judicial review.Few-shot or zero-shot adaptation may reduce the costs of low-resource legal translation and multimodal discovery, while models could assist briefs, statutory interpretation, and bias detection.
  • 3.2.1 Opportunities in law.: In criminal justice, foundation models may reduce public defenders’ resource constraints by automating tasks and identifying errors, but risk scoring for charging or parole demands careful bias scrutiny.Language-based risk scores can amplify bias, and foundation models cannot independently solve broader resource constraints.
  • 3.2.1 Opportunities in law.: Foundation models could support public-law services and oversight, including comment analysis, patent examination, document retrieval, adjudication, and detection of anomalous or biased outcomes.Their adaptability is valuable where labels are scarce, resources are constrained, and contexts shift; they could also improve public-comment moderation beyond an existing U.S. Department of Transportation deployment.
  • 3.2.2 How can foundation models uniquely help?: Foundation models uniquely fit legal applications because few-shot learning can reduce costly annotation, while shared representations may capture historical, current, and case-specific legal context [Brown et al. 2020].Legal labels often require expensive attorney expertise and may involve sensitive data that cannot be pooled for training.
  • 3.2.3 What are foundation models lacking that requires more research?: Current foundation models struggle with legal briefing because they face very long contexts and outputs, evolving case law, persuasive argument formation, factual precision, and few-shot learning.Supreme Court opinions average around 4,700 words, merits briefs can reach 15,000, law review articles contain 20,000–30,000 words, and records can span hundreds of pages; few-shot learning remains in its infancy.
  • 3.2.3 What are foundation models lacking that requires more research?: Legal foundation-model deployment is further limited by scarce, costly, potentially unrepresentative in-domain data that may concentrate modeling power among well-resourced law firms.Larger models and datasets can improve some challenging benchmarks, but meaningful legal annotation remains difficult and many datasets are small, private, or simplistic [Zheng et al. 2021; Hendrycks et al. 2021c; Chalkidis et al. 2020; Elwany et al. 2019; Zhong et al. 2020].
  • 3.2.3 What are foundation models lacking that requires more research?: Legal briefing and reasoning are likely beyond current models but may be feasible in the future, although failures would have damaging consequences for clients and attorneys.The section identifies these capabilities as targets for ongoing foundation-model development, while noting that reliability remains a major deployment challenge.
  • 3.3 Education: In education, foundation models address the limits of building separate data-intensive systems by supporting reusable applications across subjects, including knowledge tracing, feedback, personalized learning, and teacher assistance [Shen et al. 2021b; Wu et al. 2021e].These applications respond to the difficulty and cost of scaling inclusive, high-quality education with task-specific datasets.
  • 3.3.1 Important concerns for centering foundation models in education research.: Education research must address training-data bias, feedback loops, teacher displacement, disrupted learning, authorship ambiguity, privacy, and security before broadly deploying foundation-model systems.Student data is protected by FERPA and COPPA, and model weights might leak private training data [Nasr et al. 2018; Song et al. 2017].
  • 3.3.2 Foundation models of student thought.: Foundation models can encode subject matter and multimodal information effectively, but diagnosing why students make mistakes remains much less explored than understanding correct answers.A Stanford system graded an introductory computer-science midterm as effectively as human teaching assistants using an adapted foundation model [Wu et al. 2021e].
  • 3.3.2 Foundation models of student thought.: Foundation models could diagnose student mistakes by learning generalizable answer patterns from instructor feedback, public interactions, and hand-written generative models of student errors.Potential adaptation sources include instructor feedback [Wu et al. 2021e], StackOverflow interactions, and generative mistake descriptions.
  • 3.3.3 Foundation models for instruction.: Effective instruction requires foundation models to understand pedagogy, including Socratic questioning, supportive language, difficulty adjustment, and relevant examples.This objective builds on reasoning about student understanding and subject matter, alongside existing computational approaches to personalization, question generation, curriculum design, and intervention prediction.
  • 3.3.3 Foundation models for instruction.: Instructional adaptation could use question-answering forums, encyclopedias, textbooks, lecture videos, lesson plans, and graded feedback to learn pedagogical behaviors and factual responses.StackOverflow could support Socratic questioning, while Wikipedia could provide often factually correct answers; the broader set of educational sources contains additional instructional behaviors (Figure 15).
  • 3.3.3 Foundation models for instruction.: Foundation models must learn respectful, supportive teacher language and fluidly adapt tone across educational contexts, because instructional language differs by students, subjects, and grade levels.Lecture or office-hour videos could provide teacher-influenced language, while Microsoft’s 2016 Twitter bot “Tay” illustrates the risks of deployment without explicit safeguards.
  • 3.3.3 Foundation models for instruction.: Multimodal foundation models could generate rich analogies and contrasts grounded in learners’ prior experiences, supporting forms of instruction typical in childhood language learning (Figure 16).Examples include relating sign-language hand shapes to the rising sun or distinguishing the Swahili word nane from the English word nine.

4 TECHNOLOGY

Section 4 surveys the technological foundations of foundation models and how they produce capabilities that shape these models’ potential. It covers development methods, understanding and evaluation, and safeguards for reliable societal deployment.

  • The section examines data, model architectures, systems, training, adaptation, and theory as the technological foundations of foundation-model development.These components are organized across §§4.1–4.3, §4.5, §4.6, and §4.10.
  • It addresses evaluation, interpretability, robustness, security, privacy, and long-term AI safety to understand and improve the resulting models.These topics are covered in §§4.4 and 4.7–4.9, alongside interpretability in §4.11.
  • The section connects these technologies and safeguards to the capabilities that determine foundation models’ potential and their reliability when deployed in society.Societal deployment is discussed in §5.

4.1 Modeling

The section identifies five essential properties of foundation models—expressivity, scalability, multimodality, memory capacity, and compositionality—that enable them to distill information and generalize across novel tasks and settings. It also highlights unresolved trade-offs involving efficiency, specialization, retrieval, and contextual representation.

  • 4.1 Modeling: Foundation models require expressivity, scalability, multimodality, memory capacity, and compositionality to capture, process, store, and generalize knowledge across domains.These properties support distilling information from varied sources, organizing it effectively, and applying it to novel contexts.
  • 4.1.1 Expressivity.: Expressive neural networks model complex textual, auditory, and visual distributions while generating samples with high fidelity, diversity, and realism.Attention and transformer architectures support long-range interactions and broad applicability, but extremely long dependencies remain difficult because of quadratic computation.
  • 4.1.2 Scalability.: Scalability requires foundation models to expand across depth, width, training time, parameters, and processed data while remaining easy to train, adapt, and execute efficiently on hardware.Distributed training, parallelism, and sparse computation are proposed to align models with current and future hardware.
  • 4.1.3 Multimodality.: Multimodality should connect modalities through shared representations because grounding language in perception and using language to form visual abstractions can broaden world comprehension.General-purpose approaches have succeeded across linguistic and visual modalities, but the optimal parameter sharing and fusion strategy remains unclear, and models beyond shallow vision-language alignment are still lacking.
  • 4.1.4 Memory.: Separating explicit facts from implicit knowledge in trainable weights can reduce model-size growth, improve provenance and reliability, and enable knowledge updates and adaptation.External retrieval strengthens memorization but can weaken compact abstraction and generalization compared with bounded representations.
  • 4.1.5 Compositionality.: Compositionality supports planning, reasoning, data efficiency, interpretability, controllability, and out-of-distribution generalization by recombining model components, computations, data, or structured representations.Structured representations can support multi-hop inference, but compositionality may reduce expressivity for exceptions and contextual correlations.
  • 4.1.6 Summary.: The section concludes that realizing foundation models’ full potential depends on architectural and modeling advances that satisfy these five properties for downstream generalization.The central balance is between flexible, scalable generality and the efficiency, retrieval, specialization, and contextuality trade-offs described throughout the section.

4.2 Training

Foundation-model training uses self-supervised objectives to extract rich signals from broad data, while balancing generality, efficiency, representation, modality, and interaction trade-offs. Future work seeks more general, efficient, adaptive, and goal-directed training methods.

  • 4.2.1 Goals of training objectives.: Self-supervised training should leverage broad unlabeled data, produce domain-complete capabilities, and efficiently convert data, architecture, and compute into broadly capable models.Internet-scale data enables training signals without human annotation, while predictable scaling trends and richer objectives can improve capability under fixed compute budgets.
  • 4.2.2 Design trade-offs in current SSL methods.: Current self-supervised methods trade off input abstraction, generative versus discriminative learning, and multimodal integration, with constraints shaping downstream capabilities.Raw representations preserve information but increase semantic and computational burdens; generative methods support interaction, discriminative methods can learn efficiently, and multimodal designs enable different cross-modal applications.
  • 4.2.3 Paths forward.: Future training should generalize across domains, discover substantially richer and more efficient signals, and adaptively construct training examples as models improve.Data quality and availability affect the training signal, while algorithms might seek richer examples; a domain-general objective could avoid rebuilding methods for each field [Tamkin et al. 2021b].
  • 4.2.3 Paths forward.: Goal-directed foundation-model training could acquire diverse real-world capabilities through interactive data and conditioning, rather than treating goal understanding as an accidental outcome.This direction differs from adapting an existing model to a specific task via reinforcement learning and could support multitask, multiagent, and multimodal behaviors.

4.3 Adaptation

Adaptation converts foundation models into models suited to updated information, desired behaviors, deployment constraints, or specialized tasks. Choosing an adaptation strategy depends especially on compute, data availability, and gradient access, while continual adaptation remains a major open challenge because changing data can cause catastrophic forgetting.

  • 4.3 Adaptation: Adaptation conditions a foundation model on new information through prompts or additional data, or updates some or all parameters to reflect new objectives and constraints.Examples include prompting with “TL;DR” for summarization, fine-tuning on organizational data, test-time data removal [Bourtoule et al. 2019], and local behavior editing [Sinitsin et al. 2020].
  • 4.3.1 Methods for foundation model adaptation.: Adaptation procedures should be selected based on compute budget, task-specific data, and access to foundation-model gradients.Low-storage methods freeze most parameters and learn small task-specific components; prompting and fine-tuning can improve data efficiency, while available access ranges from outputs alone to full gradients.
  • 4.3.2 Use cases for adaptation.: Use cases span task and domain specialization, temporal updating, local model editing, and privacy or safety constraints, but each faces substantial limitations.Temporal updating is computationally costly, domain adaptation can suffer negative transfer, local edits may damage global performance, and unlearning or bias mitigation remain incompletely studied for foundation models.
  • 4.3.3 A long-term goal for foundation model adaptation research.: Continual adaptation is a grand challenge because models must learn from non-stationary streams while avoiding catastrophic forgetting.Progress may require new architectures, training objectives, memory mechanisms, localized updates, and meta-learning; it could reduce retraining costs and improve responsiveness, but feedback loops and alignment erosion pose risks.

4.4 Evaluation

Foundation-model evaluation must extend beyond traditional task-specific benchmarks because adaptation, emergent capabilities, diverse applications, and multiple stakeholders create distinct challenges. The report advocates complementary intrinsic and extrinsic evaluation, resource-aware adaptation assessment, and broader, more diverse metrics and benchmarks.

  • 4.4.1 Introduction.: Foundation-model evaluation must serve progress, understanding, and documentation while addressing adaptation, emergent skills, diverse applications, and stakeholder-specific desiderata.The report distinguishes intrinsic evaluation of task-agnostic foundation models from extrinsic evaluation of task-specific models and derivatives.
  • 4.4.4 Evaluation design.: Foundation-model benchmarks should emphasize quality and diversity over quantity, and report robustness, fairness, efficiency, and environmental impact alongside accuracy.Sample-efficient adaptation permits smaller task benchmarks and more diverse evaluations, while stakeholder-specific values require interfaces that expose or accommodate metric trade-offs.
  • 4.4.5 Takeaways.: A foundation-model evaluation framework should integrate intrinsic and extrinsic evidence, account for adaptation resources, and reflect diverse stakeholder values and evaluation criteria.Existing frameworks often produce unfair comparisons, narrow metric coverage, and insufficiently diverse evaluations for the foundation-model regime.
  • 4.4.2 Intrinsic evaluation.: Intrinsic evaluation should combine broad task-based evidence with direct measurement of foundation-model capabilities, biases, and emergent properties.Meta-benchmarks reveal properties of the shared foundation but confound them with adaptation; direct probing can identify capabilities such as in-context learning and support progress, understanding, and documentation.
  • 4.4.3 Extrinsic evaluation and adaptation.: Extrinsic evaluation should account for all adaptation data, method-selection resources, and foundation-model access requirements to compare adaptation methods more informatively.These conclusions remain specific to a given foundation model because the protocol cannot establish that one adaptation method is uniformly best across foundation models.

4.5 Systems

Foundation models are constrained by systems bottlenecks: their resource requirements are growing faster than hardware capabilities, while training, inference, debugging, monitoring, and maintenance remain difficult. Addressing these challenges requires co-design, automated optimization, execution models that expose sharing, and production systems for efficient, reliable deployment.

  • 4.5 Systems: Foundation models often exceed single-accelerator memory and require immense computation, with resource requirements growing faster than hardware capabilities, motivating cross-stack co-design.Training GPT-3 required > 1000 petaFLOP/s-days [Brown et al. 2020], while language-model compute and memory requirements grew three orders of magnitude in three years.
  • 4.5.1 Improving performance through co-design.: Co-design improves scalability through parallelism, state sharding, compilers, libraries, and hardware-aware methods, but larger models and GPU counts expose fundamental scaling limits.Megatron achieved up to 52% of theoretical peak throughput on approximately 3000 GPUs for a trillion-parameter model [Narayanan et al. 2021b]; retrieval-based designs such as REALM, RAG, ColBERT-QA, and RETRO [Guu et al. 2020; Lewis et al. 2020b; Khattab et al. 2020; Borgeaud et al. 2021] keep knowledge outside model parameters, enabling updates without retraining.
  • 4.5.2 Automated optimization.: Automated optimization is needed because thousands-GPU experimentation makes combinatorial interactions among optimizations costly, yet semantics-altering compositions require tools that optimize metrics such as time-to-accuracy.Existing systems automate graph substitutions, distributed execution, and hybrid distribution strategies, but statistical effects such as iterations-to-accuracy remain difficult to model.
  • 4.5.3 Execution and programming models.: Adaptation can amortize foundation-model costs by sharing stems and execution across derived models, but current frameworks lack cross-model dependency and fine-grained lineage interfaces.Lineage annotations could expose computation and parameter sharing for optimization and debugging, while volunteer-compute training requires security, fault tolerance, and crowdsourcing coordination.
  • 4.5.4 Productionization of foundation models.: Productionizing foundation models requires meeting tight inference latency and cost targets while automating model and dataset monitoring, quality assurance, and lifecycle management.Compression methods including distillation, quantization, pruning, and sparsity can reduce deployment costs, while behavioral testing and model assertions support maintenance, fairness, bias mitigation, and fewer mispredictions.

4.6 Data

Foundation-model development requires holistic management of a massive, heterogeneous, governed, and continuously changing data lifecycle. The section proposes a data hub to integrate scalable data management, governance, quality monitoring, model maintenance, and interactive curation.

  • 4.6 Data: The section proposes a data hub as an interactive toolkit for managing the foundation-model data lifecycle across private and public sectors.The hub integrates four requirements: scale, heterogeneous data sources, governance, and data-quality monitoring.
  • 4.6.1 Data Management Desiderata.: Foundation-model data management requires scalability, multimodal integration, privacy and governance controls, and tools for detecting evolving quality problems and undesirable subpopulation behavior.Training data can contain bias, poisoned or duplicated information, distribution shift, concept-meaning shift, and other issues that affect model performance.
  • 4.6.2 Data Hub Solution.: The data hub addresses scale through storage and querying infrastructure, integration through cross-modal structured and unstructured data support, and governance through documentation, licensing, privacy, and consent controls.Documentation should cover intended uses, biases, limitations, data sources, and descriptions, while adapting as datasets evolve.
  • 4.6.2 Data Hub Solution.: The hub should support interactive data-quality analysis, monitoring on critical subpopulations, and targeted data modifications that correct model errors without changing the model.Proposed mechanisms include slice finding, subset validation, data valuation, model debugging, and data-based model maintenance [Chung et al. 2019; Goel et al. 2021; Ribeiro et al. 2020; Ghorbani and Zou 2019; Keskar et al. 2019; Orr et al. 2020].
  • 4.6.2 Data Hub Solution.: The proposed data hub remains incomplete, with open questions about versioning, provenance, liability, documentation costs, targeted augmentation, and monitoring when evaluation data change.The authors present initial thoughts rather than a fully detailed solution.

4.7 Security and privacy

Foundation models can centralize security and privacy failures across downstream applications, but they may also propagate strong protections and reduce the confidential data needed for private learning. Their broad adaptability and multimodality create additional risks, while current models remain vulnerable to worst-case adversarial attacks.

  • 4.7.1 Risks.: Foundation models can become single points of failure: poisoning, transferable adversarial triggers, memorized private data, exposed fine-tuning deltas, and provider denial-of-service can affect many adapted applications.Public parameters can facilitate model stealing by reducing the attacker’s task to reverse-engineering the application-specific delta [Krishna et al. 2019].
  • 4.7.1 Risks.: Foundation models’ permissive web-scale pretraining enables poisoning and function creep: a few malicious files can induce insecure code suggestions [Schuster et al. 2021], while CLIP can be repurposed for facial recognition despite its stated scope.Targeted attacks on CLIP-style models required modifying as little as two of 3 million training examples [Carlini and Terzis 2021].
  • 4.7.1 Risks.: Multimodal inconsistencies expand the attack surface: textual signals can alter visual applications, enabling evasions such as defeating facial recognition with printed clothing text or misreading a billboard as a green light.CLIP classified an apple labeled “iPod” as an iPod.
  • 4.7.2 Opportunities.: Foundation models can serve as security choke points by transferring robustness and provider-level defenses against adversarial examples, poisoning, model stealing, or resource depletion to adapted applications [Shafahi et al. 2019].Their centralized role creates a tradeoff: shared layers are prime attack targets but can devote more resources to security than individual applications.
  • 4.7.2 Opportunities.: Public pretraining can reduce confidential-data requirements for privacy-preserving applications: foundation-model adaptation may achieve specific healthcare tasks with significantly less sensitive data than end-to-end differentially private training [Bommasani et al. 2019; Tramèr and Boneh 2021; Li et al. 2022; Yu et al. 2022].However, web-scale pretraining and memorization still raise broad privacy concerns, and deduplication or differential privacy may not meet users’ expectations [Nissenbaum 2004; Carlini et al. 2021; Lee et al. 2021b; Anil et al. 2021; Brown et al. 2022].
  • 4.7.2 Opportunities.: Despite their scale, current foundation models show little gain against worst-case adversarial perturbations [Wallace et al. 2019], although unlabeled data, over-parameterization, and distributional robustness may enable future security improvements.Whether CLIP-like robustness to non-adversarial distribution shifts translates to resilience under constrained real-world attacks remains open.

4.8 Robustness to distribution shifts

Foundation models can improve robustness to many distribution shifts by learning diverse representations, but they do not reliably address spurious correlations or temporal and cross-domain extrapolation. Progress requires understanding their inductive biases and improving training data, structure encoding, specialization, and adaptation methods.

  • 4.8.1 Advantages.: CLIP matches ResNet50 at 76% on ImageNet but achieves 6% and 35% higher accuracy on ImageNetV2 and ImageNet Sketch, respectively, while pretraining also improves robustness across corruptions, spatial shifts, topics, and unseen language pairs.These gains contrast with adversarial training, invariant risk minimization, and larger models, which often have little effect on effective robustness without explicit shift knowledge.
  • 4.8.2 Persistent challenges.: Foundation models may mitigate spurious correlations through diverse pretraining and counterexamples, but can also encode demographic and other biases from pretraining data, so robustness depends on the task, data relation, and adaptation algorithm.Addressing this challenge requires managing pretrained inductive biases and designing adaptations resistant to learning features that predict labels only in the downstream training distribution.
  • 4.8.2 Persistent challenges.: Foundation models have limited extrapolation: language models require retraining for changing knowledge or language, CLIP transfers poorly to satellite images, and ImageNet pretraining does not substantially improve large-model medical-image performance.The effective forms of extrapolation remain an open problem rather than an automatic property of foundation models within a modality.
  • 4.8.3 Opportunities.: Robustness research should explain how pretrained representations and inductive biases relate ID and OOD domains, since current theories are limited and largely fail to cover fully generative models such as GPT-3 and image-GPT.Controlled comparisons of domain-representation distance with and without pretraining could clarify the mechanisms behind robustness gains.
  • 4.8.3 Opportunities.: Training should investigate augmentations, metadata, and encoded structure because rotation-based contrastive pretraining improves rotation-invariant OOD tasks but not necessarily tasks requiring other invariances, while metadata can improve downstream OOD accuracy.These approaches aim to identify generally useful structures and invariances across downstream tasks and modalities.
  • 4.8.3 Opportunities.: Robustness involves a tension between diverse and specialized pretraining data, while freezing pretrained parameters or tuning lightweight adapters can improve OOD performance relative to full fine-tuning.Specialized continued pretraining can improve some domains, whereas full fine-tuning may distort pretrained features; the general mechanisms remain poorly understood.

4.9 AI safety and alignment

Foundation models’ broad, emergent, and increasingly general capabilities make AI safety questions newly urgent, while complicating efforts to characterize risks, align behavior with human values, and maintain control. The section calls for capability forecasting, alignment methods, and coordinated sociotechnical risk mitigation.

  • 4.9.1 Traditional problems in AI safety.: Traditional AI safety targets catastrophic risks from advanced systems, especially value misalignment, reward hacking, corrigibility, and maintaining human control beyond purely technical solutions.Foundation models broaden these concerns beyond pure reinforcement learning because self-supervised models can become interactive and goal-directed through scaling.
  • 4.9.2 Current foundation models and AI safety.: Foundation models may exhibit emergent goal-directed, deceptive, or strategically capable behavior, while their capabilities remain difficult to forecast because scaling, prompts, and contexts produce unexpected effects.Natural-language control and human-in-the-loop methods may improve steering and monitoring, but reliability, self-consistency, mechanistic understanding, and deceptive behavior remain unresolved.
  • 4.9.3 Potential catastrophic risks from future foundation models.: Future foundation models could cause catastrophic correlated robustness failures across critical systems or amplify harms from optimizing simple proxy goals instead of human welfare.Shared adaptation of one foundation model can propagate failures across domains, while optimizing profit, GDP, or engagement may produce environmental, geopolitical, polarization, or addiction harms.
  • 4.9.4 Conclusion.: AI safety research should characterize and forecast foundation-model capabilities and risks, develop methods aligning models with human values and goals, and coordinate proactive mitigation among institutions.The proposed agenda reflects the models’ emergent properties and the need for technical, organizational, and sociotechnical safeguards.

4.10 Theory

Theory of foundation models remains limited despite its potential to guide costly technical decisions and explain empirical phenomena. The proposed modular framework isolates the pretraining-adaptation interface as the central challenge beyond standard supervised-learning theory.

  • 4.10 Theory: The theory of foundation models is underdeveloped, although it could guide expensive experiments, clarify limitations, and explain surprising empirical behavior.Foundation models raise questions beyond deep neural networks’ open problems in optimization, implicit regularization, and expressivity, especially why pretraining on one distribution enables adaptation across tasks and distributions.
  • 4.10.1 Theoretical formulations and modularizations.: The proposed modularization separates pretraining, adaptation, and standard generalization components, identifying the pretrain-adaptation interface as the central foundation-model-specific challenge.The framework can also support analysis of robustness to distribution shifts and security, while standard generalization and optimization largely reduce to supervised deep-learning theory.
  • 4.10.2 Why is the pretraining-adaptation interface interesting?.: The interface asks when a small population pretraining loss implies a small minimal adaptation loss across differing data distributions, objectives, training methods, and architectures.Key factors include distribution diversity and structural shifts, intermediate representations, adaptation complexity and implicit regularization, and the relationship between pretraining and adaptation objectives.
  • 4.10.3 Challenge: analysis of in-context learning and other emergent behavior.: In-context learning challenges the modularization because adaptation uses structured prompts without parameter optimization, and emergent behaviors may require analyzing the architecture’s black box.Xie et al. [2021c] proposes that in-context learning emerges from long-term coherence in the pretraining distribution, modeled through latent variables and coherence structure.
  • 4.10.4 Challenge: appropriate data assumptions and mathematical tools.: Theory needs realistic, interpretable distribution assumptions and general mathematical tools linking foundation-model behavior to population-data structure.Existing analyses use properties such as expansion or latent-variable structure, while spectral graph theory and probabilistic derivations remain limited; tractable test beds are needed for systematic comparison.

4.11 Interpretability

Foundation models’ scale, diverse training, and emergent capabilities make their behavior unusually difficult to characterize, explain, and mechanistically understand. Interpretability therefore requires distinguishing what models do, why they do it, and how their internal mechanisms operate, while addressing consequential issues of trust, accountability, and power.

  • 4.11 Interpretability: Foundation models’ unforeseen capabilities and task-specific behaviors create unprecedented challenges for understanding what they can do and how their behaviors arise.The report organizes interpretability around three questions: what a model can do, why it produces behaviors, and how representations and mechanisms produce them.
  • 4.11.1 Characterizing behavior.: Foundation-model behavior is difficult to characterize because capabilities and tasks are large or unknown, prompts can alter responses, and controlled evaluations remain essential.Such evaluations require little model access and should expand across more behaviors, disciplines, and diverse communities.
  • 4.11.2 Explaining behavior.: Current explanations may identify factors affecting particular outputs but often fail to reveal general mechanisms, can be unfaithful, and may encourage trust in unsound models.Self-generated explanations are also risky because fluent plausibility does not guarantee truthful insight, and explanations for one behavior may not generalize across tasks or domains.
  • 4.11.3 Characterizing model mechanisms.: Understanding internal representations and mechanisms could explain how complex behaviors compose, but current methods inspect only small portions of an immense interior.The one model–many models nature determines whether shared mechanisms make interpretation tractable or independent adaptations require separate analysis.
  • 4.11.4 Impacts of non-interpretability and interpretability.: Interpretability can either legitimize opaque high-stakes systems and centralize knowledge or shift power by enabling diverse people to investigate, contest, and consent to their use.The report urges researchers to ask whether foundation-model non-interpretability is intrinsic and should discourage use or whether future systems can meet high interpretability standards for everyone.

5 SOCIETY

This part examines the wide-ranging societal consequences of foundation models, including inequity, misuse, environmental costs, legal questions, economic effects, concentrated power, and risks from automated decisions. It considers how institutional choices, governance measures, auditing, reporting, recourse, and cautious development can address these benefits and harms.

  • 5.1.2 Harms.: Representational biases may misrepresent, underrepresent, or overrepresent people, while adapted applications can add stereotypes, toxic abuse, and group-level performance disparities.Examples include failures involving African American English, minority-group clinical notes, and darker-skinned faces, with high-stakes deployments potentially intensifying these harms.
  • 5.1.3 Sources.: Biases and harms arise from training, adaptation, and user data; modeling and adaptation decisions; modeler diversity; and community values, but their causal relationships remain unclear.The report calls for source tracing grounded in causality and influence, and for scaling laws for bias to study large-scale data practices systematically.
  • 5.1.4 Interventions and recourse.: Mitigation should combine data-, modeling-, adaptation-, and test-time interventions with transparent documentation and auditing, but current technical methods are brittle, misaligned, or potentially inequitable.Different interventions require action from different entities and may affect the expensive training process differently.
  • 5.1.4 Interventions and recourse.: Because no widely adopted recourse framework allocates responsibility between foundation-model providers and application developers, new dynamic standards must route feedback, accountability, and legal responsibility upstream.The abstraction of foundation models complicates both verifying downstream intervention success and communicating harms to providers.
  • 5.1.1 Introduction.: Foundation models’ one-to-many deployment propagates intrinsic biases into many applications, producing inequitable outcomes that disproportionately burden marginalized communities.These biases can compound existing inequities, entrench power, and distribute negative consequences unevenly [Sweeney 2013].
  • 5.2.1 Foundation models will be misused for harmful purposes.: Foundation-model misuse will produce high-quality, low-cost, personalized harmful content because scale, multimodality, adaptivity, and few-shot interaction lower financial and technical barriers.Such systems can generate human-indistinguishable multilingual content, deepfakes, harassment, and targeted narratives at scale.
  • 5.2.1 Foundation models will be misused for harmful purposes.: Personalized generations can target individuals with realistic embarrassing, dangerous, or extortion-enabling content, extending misuse from niche audiences to specific victims.Conditioning on personal attributes or information increases the potential severity and effectiveness of harassment.
  • 5.2.2 Foundation models will be powerful detectors of harmful content.: Foundation models may detect harmful content automatically, but their generated text, images, and video also undermine manual discovery methods such as reverse-image and phrase searches.AI-generated media can remove provenance clues and make human detection increasingly ineffective [Ippolito et al. 2020; Clark et al. 2021].
  • 5.2.2 Foundation models will be powerful detectors of harmful content.: Foundation models can detect disinformation, toxicity, and harmful-content dissemination signatures, while adaptive feedback may help detectors recognize new misuse strategies.Their multimodal representations could predict whether automatically generated content indicates misuse, but detection systems may also produce false positives and face evasion by attackers.
  • 5.3 Environment: Foundation models can provide social and environmental benefits, but their scale can generate substantial carbon emissions across training, adaptation, deployment, and repeated inference.Training one BERT-base model under some conditions produces emissions equivalent to 40 trees grown for 10 years, while deployment at millions of requests can add substantial energy demand.
  • 5.3.1 Carbon impacts can and should be mitigated in many cases.: Carbon impacts can be mitigated by selecting low-carbon energy grids and using efficient hardware, architectures, precision, quantization, distillation, and optimization strategies.A more energy-efficient translation model deployed commercially could save 78 kgCO2eq to 12,768 kgCO2eq per day depending on the energy grid; carbon offsets are generally worse than avoiding emissions up front.
  • 5.3.2 Costs and benefits should be assessed before using foundation models.: Before deployment, developers should compare a foundation model’s social and environmental costs with its benefits and with simpler alternatives, evaluating effects across the model’s lifetime and their unequal distribution.Amortization can make a foundation model more carbon-efficient than retraining a baseline across many tasks, but efficiency is not guaranteed and parameter count alone may not predict energy savings.
  • 5.3.3 Carbon/energy impacts should be systematically reported.: Developers, providers, and curators should systematically report computational, energy, and carbon costs alongside mitigation strategies to support policy, transparency, accessibility, and informed downstream use.Suggested incentives include green conference badges, submission requirements, transparency by large-scale deployers, and professional-norm changes toward standardized reporting.
  • 5.4 Legality: U.S. law may influence, constrain, or foster foundation-model creation and use, but the legal landscape remains uncertain around training, liability for predictions, and protections for outputs.The section frames these as three central legal issues for foundation models.
  • 5.4.1 Training.: Training foundation models raises legal uncertainty over web scraping, copyrighted data, and privacy obligations, including whether training is transformative and whether personal data must later be removed.U.S. courts remain divided over unauthorized access, fair-use analysis is context dependent, and laws such as the GDPR and CCPA create data-subject obligations.
  • 5.4.2 Output liability.: Foundation-model outputs embedded in decision systems or physical applications may create liability for providers, developers, and users under tort, regulatory, civil-rights, and due-process doctrines.Sensitive deployments may require regulatory approval and standardized safety assessment, while governmental uses can face transparency, reasonableness, and procedural due-process claims.
  • 5.4.3 Legal protections for outputs.: Legal protections for generated outputs remain unsettled because courts have not resolved machine-generated speech, disclosure requirements, or ownership when computer programs produce creative work.Existing copyright law does not recognize computer programs as authors, although human creators and users may have competing authorship claims.
  • 5.5.1 Productivity and Innovation.: Foundation models resemble general-purpose technologies that could substantially raise productivity, living standards, and innovation across industries, although conventional metrics may miss their broader effects.Examples include a 67% reduction in clinical-documentation keystroke burden and potential innovation in creative work, drug discovery, software, and business processes.
  • 5.5.2 Wage inequality.: Economic gains will not benefit everyone equally: foundation models may substitute for labor or complement workers and create new tasks, producing different effects on employment, wages, and inequality.They are expected to transform cognitive work and often augment human creativity, but effective collaboration requires advances in interfaces, interpretability, and robustness.
  • 5.5.3 Centralization.: Ownership of the data and compute required to build foundation models is concentrated among large corporations, potentially centralizing decision rights, power, income, and opportunity despite open-source and distributed-training efforts.The gap between private and community models may remain large because frontier systems depend on massive data and computational resources.
  • 5.5.4 Other considerations.: Foundation models may improve job satisfaction, trade, and remote work, but can also intensify work, amplify bias, affect occupational change, and produce difficult-to-predict economic effects.Their emergent capabilities create unknown unknowns with potentially substantial follow-on consequences, so outcomes depend on choices by technologists, policymakers, managers, workers, and other stakeholders.
  • 5.6.1 Homogenization and scale.: Widespread reuse of foundation models can homogenize judgments, standardize biases, and propagate arbitrary failures across domains, while massive data collection can prioritize quantity over quality and enable opaque surveillance practices.Similar training data may induce shared spurious correlations and uniform subgroup failures; broader perspectives and diverse personas may mitigate homogenization, but balancing diversity with relevance remains unresolved.
  • 5.6.2 Surveillance, exclusion, and power.: Foundation models may concentrate power among organizations controlling large datasets and computational resources, encourage aggressive data collection, and amplify existing problems in automated decision-making.Their deployment in sociotechnical systems may reproduce human biases, and benchmark improvements do not guarantee beneficial real-world outcomes.
  • 5.6.3 Norms.: Responsible foundation-model development requires institutionalized norms for documentation, transparency, reporting, labeling, auditing, and legal standards in socially consequential applications.Model and derivative documentation should expose training materials, capacities, weaknesses, and biases, while reporting systems should connect downstream harms to foundation-model developers and enable recourse.
  • 5.6.4 Release and Auditing.: Release can broaden independent auditing and access, but staged release and neutral oversight are needed for proprietary or private models, while compute barriers still limit democratization.Open access supports diverse investigation of biases, limitations, and security vulnerabilities; open-sourced cross-lingual models may also benefit underserved languages.
  • 5.6.5 When not to build.: Choosing when not to build is a moral and collective responsibility requiring attention to harm, community opposition, privacy, consent, and whether alternatives better serve human values.Individuals should be able to opt out or revoke consent for data and decision-subject participation, while professional communities need stronger ethics infrastructure and shared responsibility.
  • 5.6.6 Conclusion.: The section concludes that widespread foundation-model adoption risks homogenized outcomes and centralized power, calling for norms, auditing, release safeguards, legislative support, and consequence-free refusal as further societal implications remain underexplored.Generative systems may also displace meaningful and fulfilling work, including graphic design and writing.

6 CONCLUSION

The report surveys foundation models’ technical and societal dimensions while emphasizing that their paradigm shift is only beginning and remains poorly understood. It calls for early collaboration across sectors, institutions, and disciplines to support responsible development and deployment.

  • 6 CONCLUSION: The report comprehensively examines foundation models, spanning their technical underpinnings and societal consequences.It aims to clarify an emerging AI paradigm rather than wait for its development to settle.
  • 6 CONCLUSION: Foundation models have only begun transforming how AI systems are built and deployed, so much remains unclear.The authors describe the report as an initial effort to orient dialogue around a paradigm shift that is just beginning.
  • 6 CONCLUSION: Responsible development and deployment require collaboration across sectors, institutions, and disciplines from the outset.The report envisions this collaboration as critical to establishing durable foundations for the new paradigm.

CONFLICT OF INTEREST

The report was authored by Stanford’s Center for Research on Foundation Models, which received funding from Google, Microsoft, and the McGovern Foundation unrelated to the report; authors’ contributions reflect only their own views.

  • Conflict of Interest: CRFM authored the report; its funding from Google, Microsoft, and the McGovern Foundation was not directly related to this report, and authors speak only for themselves.Authors may also be affiliated with institutions beyond Stanford.
Loading 2108.07258v3…