On the Opportunities and Risks of Foundation Models

1.1 Emergence and homogenization

• GPT-3 [Brown et al. 2020] exhibits emergent in-context learning, while homogenization spreads improvements broadly but also propagates shared flaws, making opaque, uncertain models risky to standardize.Because capabilities emerge rather than being explicitly constructed, the authors identify reducing risk as a central challenge.Source: §1.1 · pp. 5–6

1.1.1 Naming.

• The term “foundation models” names an incomplete model class that serves as a common basis for many adapted task-specific models and highlights the paradigm’s broad research and deployment impact.The authors stress that the trustworthiness of this foundation is not yet understood and must be addressed by researchers, providers, developers, policymakers, and society.Source: §1.1.1 · pp. 6–7

1.2 Social impact and the foundation models ecosystem

• Social impact arises across the ecosystem from data creation through deployment, so assess unfinished models with representative downstream surrogate metrics and document them.The report calls for rigorous testing and auditing of deployed systems; application-specific safeguards and gradual releases can help mitigate harms.Source: §1.2 · pp. 8–9

1.3 The future of foundation models

• Commercial incentives can neglect marginalized users, social externalities, and open participation, while academia’s public-good mission and disciplinary breadth position it to shape socially beneficial development.The authors argue that social and ethical considerations should be integrated from the start, rather than left to post-hoc audits.Source: §1.3 · p. 10
• Rising resource and access barriers centralize foundation-model development, leaving academia and startups struggling to match industry and big-tech scale.Some models and datasets are unavailable, and training costs and engineering requirements limit who can build them. Public computing infrastructure could help close the gap, while volunteer computing remains technically challenging.Source: §1.3 · pp. 11–12
• Because emergent capabilities make widespread deployment uncertain, the authors urge caution and professional norms built through collaboration between industry and academia.The report argues that industry makes deployment decisions, while academia can offer technically and ethically grounded guidance shaped by disciplinary diversity and non-commercial incentives.Source: §1.3 · p. 12

1.4 Overview of this report

• The report aims to connect technical, application, and societal perspectives, while noting that its coverage is incomplete in a rapidly evolving field.It was developed to address gaps in mutual understanding; some affected application areas and research connections are not covered.Source: §1.4 · p. 12

1.4.1 Overview of capabilities.

• The capabilities overview spans language, vision, robotics, reasoning, interaction, and questions about understanding, while identifying domain-specific challenges and opportunities.It highlights language variation and grounding, data and safety challenges in robotics, and the potential for multimodal models to understand natural language.Source: §1.4.1 · pp. 14–15

1.4.2 Overview of applications.

• Healthcare, law, and education offer potential uses for foundation models, but each requires capabilities and technical advances attentive to domain-specific sociotechnical risks.Healthcare and law may benefit from sample-efficient adaptation, while education could support personalized learning; privacy, bias, factuality, access, and plagiarism are among the concerns raised.Source: §1.4.2 · pp. 15–16

1.4.3 Overview of technology.

• The technology overview calls for advances in modeling, training, adaptation, and evaluation, supported by systems co-design to scale capabilities efficiently.It identifies multimodality, memory, and compositionality as modeling desiderata; broader training objectives and adaptation methods also require suitable evaluation protocols.Source: §1.4.3 · pp. 16–17
• The report proposes better data management and highlights security, privacy, robustness, safety, alignment, and theory as unresolved technology concerns.It proposes a data hub amid limited transparency, notes that broader pretraining can improve robustness without resolving temporal extrapolation or spurious correlations, and says existing learning theory does not fully explain foundation models.Source: §1.4.3 · pp. 18–19
• Interpretability research must address foundation models’ many-task scope and emergent properties, including whether adapted models share decision-making components with the original model.The discussion also considers the validity of model-generated post hoc explanations and the societal impact of interpretability or its absence.Source: §1.4.3 · p. 19

1.4.4 Overview of society.

• The report distinguishes intrinsic biases in foundation models from extrinsic harms in applications, and considers source tracing, proactive interventions, and reactive recourse to address unfair outcomes.Potential sources include training data, limited developer diversity, and the broader sociotechnical context; the report does not treat unfairness as inevitable.Source: §1.4.4 · p. 19
• Foundation models may enable higher-quality, easier-to-personalize harmful content, while also offering potential as automated misuse detectors.Misuse means using models as intended, such as generating language or video, with the goal of causing societal harm.Source: §1.4.4 · pp. 19–20
• Foundation models’ energy-intensive training can increase carbon emissions, so the report calls for efficiency measures, environmental evaluation, and broader documentation and measurement.The environmental calculus depends on assumptions about large one-time training costs and amortizing them across repeated use.Source: §1.4.4 · p. 20
• Foundation models have uncertain legal footing, and the report highlights unresolved liability and protections questions for models’ development and use, especially given their intermediary status.The discussion centers on the United States and the need to advance legal standards for these issues.Source: §1.4.4 · p. 20
• The report considers economic effects on productivity, wage inequality, and ownership concentration, alongside ethical concerns about homogenization, concentrated power, and release norms.These issues are considered in light of foundation models’ broad potential applications across industries and occupations.Source: §1.4.4 · p. 20

2.1.1 The nature of human language.

• Language is central to human thought, social relations, identity, and shared knowledge, and its diversity and complexity make understanding and generation important AI challenges.NLP, together with speech recognition and text-to-speech, aims to give computers the ability to understand and generate human language.Source: §2.1.1 · p. 22

2.1.2 Impact of foundation models on NLP.

• A single foundation model’s generality and adaptability—not just its language generation—has made it central to NLP systems and research.The field has shifted from task-specific architectures toward adapting and analyzing foundation models.Source: §2.1.2 · pp. 22, 24
• 91.6% on the NY Regents 8th grade science exam (2019 adapted foundation model) versus 73.1% (2018 best pre-foundation-model system); adapted models outperform prior systems on most NLP tasks.Clark et al. [2019] report the exam comparison; the broader pattern comes from the section’s account of task adaptation.Source: §2.1.2 · p. 23
• Language-generation objectives now underpin language analysis as well as generation, while the foundation-model paradigm is extending to speech through self-supervised ASR pretraining and adaptation.This shift helped spur research on summarization and dialogue generation; wav2vec 2.0 is trained on speech audio and adapted using audio with transcriptions.Source: §2.1.2 · pp. 23–24

2.1.3 Language variation and multilinguality.

• Multilingual training enables transfer to low-resource languages, but uneven data, language similarity, and competition for model capacity leave robust and equitable representation unresolved.Models discussed are trained on around 100 languages; multilingual models show transfer, yet perform better on languages similar to high-resource training languages.Source: §2.1.3 · pp. 24–25

2.1.4 Inspiration from human language acquisition.

• Unlike human learners, foundation models use far more language data, mostly learn from ungrounded text, may lack systematic generalizable abstractions, and have relatively static linguistic systems.The section identifies grounding, systematicity, and flexible accommodation to linguistic change as research directions.Source: §2.1.4 · pp. 25–27

2.2.1 Key capabilities and approaches.

• Self-supervision lets vision foundation models learn from large quantities of raw, diverse data; early systems show progress on traditional tasks, but remain nascent and focused on RGB inputs and core tasks.Reported progress includes competitive image-classification and object-detection performance without explicit training annotations, plus visual synthesis such as DALL-E and CLIP-guided generation.Source: §2.2 · p. 29, §2.2.1 · p. 30
• Reducing dependence on annotations may advance visual commonsense and temporal reasoning, but embodied and interactive perception remain challenging and the path to these capabilities is open.The section points to physical-scene understanding, social affordances, and reasoning over visual events as higher-order goals.Source: §2.2.1 · pp. 30–31

2.2.2 Central research challenges.

• Vision foundation models may improve healthcare and home assistance, mobile interaction, and content editing, but their growth also heightens concerns about bias, surveillance, and deepfake misinformation.The ubiquity of cameras makes careful consideration of these risks essential as visual and generative capabilities advance.Source: §2.2.2 · pp. 31, 33
• Vision models still struggle with novel compositions and lack robust generalization to natural scenes, objects, and physical or geometric properties at human levels.Although models show early signs of geometry understanding, methods to extend those capabilities robustly remain an open challenge.Source: §2.2.2 · pp. 31–32

2.3 Robotics

• Robotic foundation models could help build generalist robots by learning across tasks, environments, and embodiments in a high-dimensional, closed-loop setting.The section focuses primarily on household mobile manipulation while noting potential uses from manufacturing and construction to driving and personal assistance.Source: §2.3 · pp. 34–35

2.3.1 Opportunities.

• Robotic task-specification models could translate varied human descriptions and feedback into reward signals that guide task progress, behavior optimization, and failure diagnosis.Inputs may include language, goal states, human videos, comparisons, interactive corrections, and physical feedback; generalizing these signals across tasks and environments remains open.Source: §2.3.1 · p. 36
• Robot foundation models could combine action, observation, and reward prediction with multimodal self-supervision to learn representations and support few-shot adaptation across tasks, environments, and embodiments.Potential objectives include predicting future observations from actions and past observations, actions from observations and goals, or rewards from observations and actions.Source: §2.3.1 · pp. 36–37

2.3.2 Challenges and risks.

• Robotic foundation models need large, diverse data, but collecting it at scale and bridging robot experience with web videos, language, or simulation remain open challenges.Simulated data can be rich and interactive, but transferring manipulation and locomotion skills to real robots remains difficult.Source: §2.3.2 · pp. 37–38
• Safe robotic deployment is difficult because new environments can expose unforeseen hazards, motivating safety constraints, formal evaluation and guarantees, and interpretable boundaries for learning.Constraints set before data collection may prove inadequate when unsafe behaviors emerge; possible approaches include limiting environmental complexity or enabling autonomous resets.Source: §2.3.2 · pp. 38–39

2.4 Reasoning and search

• Multimodal reasoning can use visual properties such as symmetry and geometry alongside symbolic language to prune search and find promising solution constructions.Source: Figure 9 · p. 40

2.4.1 What are the current tasks?

• Reasoning tasks span unbounded search spaces across mathematics and real-world domains, where shared solution-tree structures and simulator feedback guide search; language models can generate arbitrary actions.Scaling models and applying language-model pretraining have also improved performance on reasoning tasks.Source: §2.4.1 · pp. 40–42

2.4.2 What’s the role of foundation models?

• Foundation models may support reasoning through generative candidate selection, transfer of heuristics and meta-knowledge across tasks, and grounding symbols in multimodal or shared representations.Aligned multimodal reasoning data is scarce, and whether models can discover cross-modal connections without supervision remains open.Source: §2.4.2 · p. 42

2.4.3 Future challenges in reasoning.

• Reasoning remains constrained by scarce annotated data and models’ focus on low-level steps rather than high-level plans, while principled self-supervision, reliable architectures, interpretability, and out-of-domain robustness remain open challenges.Interactive human guidance may help alleviate data scarcity, but scaling approaches for abstract planning to realistic settings remains unresolved.Source: §2.4.3 · p. 43

2.5.1 Impact on AI-infused application developers’ development process.

• Foundation models could lower application-development barriers for non-ML experts and raise the range and quality of possible interactions, but their complexity and unpredictability make them difficult to manage reliably.Fine-tuning, prompt engineering, calibration, or task-specific endpoints are suggested as ways to improve predictable behavior.Source: §2.5 · p. 44, §2.5.1 · p. 45

2.5.2 Impact on end-user interaction with AI-infused applications.

• Generative and multimodal models may enable diverse creative tools and interactions, while their potential to surprise users or reflect community-incongruent biases raises monitoring, agency, authorship, and societal concerns.Because models learn from observed data rather than necessarily representing causality, their use may reproduce the past rather than advance a desired future.Source: §2.5.2 · pp. 45–46

2.5.3 Blurring the line between developers and end-users.

• Foundation models may let users create bespoke applications through natural-language prompts and in-context adaptation, but bias and unpredictable behavior remain barriers to robust participation by non-experts.For example, a community could define a content classifier using a task description, though the capability could also be misused.Source: §2.5.3 · p. 47

2.6 Philosophy of understanding

• Current models produce fluent language but also incoherence, and it remains premature to conclude either that future foundation models can understand language or that they cannot..Source: §2.6 · p. 48

2.6.1 What is a foundation model?

• Foundation models are characterized here as self-supervised learners of symbol co-occurrence, which enables plausible generation but does not directly establish meaning, though multimodal associations may reflect aspects of the world.The account focuses on self-supervision as the model’s only formal objective.Source: §2.6.1 · pp. 48–49

2.6.2 What is at stake?

• Understanding may matter for trusting language behavior, improving interpretability and control, and assigning accountability to artificial agents [The HAI Adaptive Agents Group 2021].Its possible role in these goals motivates developing a framework for theorizing about understanding.Source: §2.6.2 · p. 49

2.6.3 What is understanding?

• Understanding theories differ: internalism requires internal representations, referentialism links language to truth or referents, and pragmatism treats appropriate linguistic dispositions as sufficient.These views imply different relationships between understanding, trust, and interpretability.Source: §2.6.3 · p. 50
• Multimodal traces such as images, audio, and sensor readings may help models learn meaning mappings that linguistic inputs alone could leave fundamentally limited.The authors tentatively see no a priori reason that any of the three views of understanding could not be learned.Source: §2.6.3 · pp. 50–51
• Behavioral tests and benchmarks may not establish understanding, while internalist or referentialist targets call for structural evaluation of representations and dynamics.Tests can have gaps that unsophisticated models exploit, or fail to reveal a mapping a system has achieved; proposed structural methods include probing, studying dynamics, and causal interventions.Source: §2.6.3 · pp. 51–52

2.6.4 Moving the discussion forward.

• Multimodal training may be the most viable strategy for language understanding, but whether self-supervision suffices remains completely open.The authors see no easy answers to whether foundation models will ever understand language.Source: §2.6.4 · p. 52

3.1 Healthcare and biomedicine

• Foundation models could serve as multimodal medical knowledge hubs, queried and updated by professionals and adapted to healthcare and biomedical tasks.The framework spans data from providers, payers, institutions, pharma, wearables, and publications, and modalities including images, video, chemical graphs, EHR tables, text, time series, and genetic data.Source: §3.1 · pp. 54–55, Figure 12 · p. 54

3.1.1 Opportunities in healthcare.

• In healthcare, foundation models could support providers through EHR interfaces and patients through medical information and question-answering, but advice must be factually accurate.Potential uses include patient-record summaries, relevant-case retrieval, test and treatment suggestions, surgical-robot assistance, appointment information, preventive-care answers, and public-health guidance.Source: §3.1.1 · pp. 55–56

3.1.2 Opportunities in biomedicine.

• In biomedicine, foundation models could accelerate drug discovery, integrate data across modalities, personalize treatment, and improve clinical-trial design and patient matching.Their generative capacity could help design molecules and experimental protocols; language-model transfer to protein-fold prediction illustrates cross-modal adaptation, while trial failure is often linked to efficacy, safety, or patient-matching problems.Source: §3.1.2 · pp. 56–57

3.1.3 Challenges and future research in foundation models.

• Medical foundation models must address multimodal learning, explainability, legal and ethical safeguards, and extrapolation to new technologies and disease settings.Current models generally learn modalities separately; healthcare decisions require evidence and logical steps, while deployments must ensure accuracy, uncertainty handling, privacy, and fairness. Research is also needed to adapt reliably to new assays, imaging techniques, and diseases.Source: §3.1.3 · pp. 57–58

3.2 Law

• In the U.S. legal context, foundation models may expand access to justice, but specialized language, ambiguous standards, scarce labels, and high-stakes use pose substantial challenges.The discussion focuses mainly on text-based tasks and warns that legal deployment demands scrutiny of fairness, accountability, transparency, and explainability; current models may not be ready for pressing legal problems.Source: §3.2 · pp. 59–60

3.2.1 Opportunities in law.

• In civil law, foundation models could reduce service costs and help clients and attorneys with issue identification, tailored guidance, document review, translation, legal research, and discovery.Few-shot or zero-shot adaptation may help in low-resource legal translation and document retrieval, where traditional supervised approaches face costly data collection or labeling.Source: §3.2.1 · pp. 60–61
• Models could also assist with trial preparation, courtroom-bias analysis, judicial review, and criminal-justice tasks, but risk scoring requires particular caution about bias.Possible uses include predicting judicial focus, assessing courtroom audio and video for bias, flagging errors in opinions, and helping overburdened defense attorneys; models are not a solution on their own.Source: §3.2.1 · pp. 61–62
• Legal applications include detecting anomalous parole outcomes and disrespectful police communications, removing racial cues from reports, and supporting government services.Potential public-law uses include analyzing public comments, assisting patent examination, retrieving documents for Freedom of Information Act requests, and aiding mass adjudication; government use must meet constitutional, statutory, and administrative obligations.Source: §3.2.1 · pp. 62–63

3.2.2 How can foundation models uniquely help?

• Legal work’s costly annotation and sensitive data make few-shot learning promising, while foundation models could represent historical and current legal contexts alongside individual case nuances.Legal labels may require expensive attorney expertise, and sensitive data may be unsuitable for pooling; legal decisions also require context spanning historical decisions, currently relevant case law, and the individual case.Source: §3.2.2 · p. 63

3.2.3 What are foundation models lacking that requires more research?

• Legal model progress is constrained by immature few-shot methods, scarce and difficult-to-create benchmarks, potentially unrepresentative data, and serious deployment risks.Larger models have improved some challenging tasks, but legal datasets are often small, unavailable, or simple; public opinions may not represent typical disputes, and data-rich large firms could gain an advantage. Failures can harm clients and attorneys.Source: §3.2.3 · pp. 65–66
• Current foundation models struggle with legal briefing and reasoning, which the paper describes as beyond their present capabilities but potentially attainable in the future.Legal briefs require long contexts and narratives, retrieval of relevant and still-valid law, persuasive argumentation, rule application, logical reasoning, and precision; GPT-3 failed to apply a liquidated-damages rule to a concrete case, and BERT did not deduce tax-law outcomes.Source: §3.2.3 · pp. 63–66

3.3 Education

• Education AI may support feedback, teacher improvement, and personalized learning, but task-specific data collection limits scalable solutions and motivates reusable models.Foundation models already support some educational tasks, including knowledge tracing with MathBERT [Shen et al. 2021b] and interpreting answers in the feedback challenge [Wu et al. 2021e].Source: §3.3 · pp. 67–68

3.3.1 Important concerns for centering foundation models in education research.

• Education research must consider bias, feedback loops, teacher replacement, student authorship and plagiarism, and student-data privacy and security.Removing teachers could affect socioemotional development; generative tools may complicate judging student contributions and undermine novice learning. FERPA and COPPA constrain data use, and model weights might leak private training data.Source: §3.3.1 · pp. 68–69

3.3.2 Foundation models of student thought.

• A foundation model adapted to subject matter and question-specific data graded a Stanford introductory computer science midterm as effectively as human teaching assistants.Models can use examples to understand fluent code and can combine prompts, diagrams, and rubrics; the reported grading system adapted a foundation model on GitHub code and a small dataset for each question’s subject matter [Wu et al. 2021e].Source: §3.3.2 · p. 70
• Understanding subject matter is more directly supported than diagnosing why students make mistakes, a less-explored capability central to useful feedback.Effective feedback requires both subject-matter understanding and the ability to infer the reasons behind errors; typical classroom data may be insufficient to learn both from scratch.Source: §3.3.2 · p. 70
• Foundation models could be adapted to notice student mistakes using instructor feedback, public interaction data, or generated examples based on instructors’ descriptions of errors.Instructor feedback may be privately held in disparate datasets, while public interactions and hand-written generative descriptions of mistakes offer other possible adaptation sources.Source: §3.3.2 · p. 71

3.3.3 Foundation models for instruction.

• Instructional models would need pedagogical understanding, including Socratic prompts, supportive language, and examples tailored to students’ difficulty level, interests, and backgrounds.Computational instruction tasks include content personalization, question generation, adaptive curriculum design, and predicting instructor intervention.Source: §3.3.3 · p. 71
• Public forums, encyclopedias, textbooks, videos, lesson plans, and graded feedback could provide pedagogical behaviors for adapting instructional models.The passage gives question-answering forums and encyclopedias as examples of sources for common Socratic questions and often factually correct answers.Source: §3.3.3 · p. 71
• Instructional models need teacher-like, respectful language, yet suitable tone varies by age, subject, and context, requiring adaptation beyond factual domain shift.The Tay chatbot’s rapid generation of hate speech illustrates the importance of explicitly accounting for language in education; lecture or office-hour recordings are proposed as adaptation sources.Source: §3.3.3 · p. 71
• Multimodal foundation models may support rich analogies and contrasts grounded in learners’ real-world experiences and prior knowledge.Examples include linking a sign-language hand shape to sunrise and comparing Swahili nane with English nine as a false friend.Source: §3.3.3 · p. 72

• Foundation-model technology gives rise to the capabilities that determine the models’ potential.Source: §4 · p. 73
• Model development and adaptation involve data, architectures, training systems, and theory.Source: §4 · p. 73
• Evaluation and interpretation, alongside robustness, security, privacy, and long-term AI safety, support model reliability when deployed in society.Source: §4 · p. 73

4.1 Modeling

• The five proposed foundation-model properties are expressivity, scalability, multimodality, memory capacity, and compositionality, enabling information capture, accumulation, representation, and generalization.Together, they describe models intended to distill information from varied sources and domains, organize it in scalable representations, and adapt it to novel contexts.Source: §4.1 · p. 74

4.1.1 Expressivity.

• Generative networks produce high-fidelity, diverse, realistic samples across text, audio, and vision, while attention and transformers support broad applicability across tasks.Modeling extremely long-range dependencies remains challenging because contemporary transformer approaches incur quadratic computation; linear-complexity attention variants seek to balance efficiency and expressivity.Source: §4.1.1 · pp. 75–76

4.1.2 Scalability.

• Foundation models need scalability across depth, width, training time, parameter count, and data volume, alongside trainability, adaptability, and compatibility with evolving hardware.Parallel computation helped transformers outperform recurrent approaches, while distributed training and sparse computation are proposed as ways to use future hardware effectively.Source: §4.1.2 · pp. 76–77

4.1.3 Multimodality.

• Multimodal models should connect modalities in shared representations, and broad, general-purpose approaches have succeeded across language and vision.The optimal degree of parameter sharing and the stage and form of cross-modal fusion remain open questions; models beyond shallow vision-language alignment and grounded language learning in embodied environments remain underexplored.Source: §4.1.3 · pp. 77–78

4.1.4 Memory.

• Separating explicit facts into external memory from implicit knowledge in model weights can reduce model size, improve provenance, and enable knowledge updates and adaptation.Retrieval can access particular facts, but over-reliance on it may weaken compact abstraction and generalization compared with bounded representations.Source: §4.1.4 · pp. 78–79

4.1.5 Compositionality.

• Compositionality can support efficient learning, reasoning, and out-of-distribution generalization through modular computation and structured representations, but may miss contextual exceptions.The section points to balancing compositional structure with contextuality; graph-based representations may organize entities and relations to support multi-hop reasoning and recombination.Source: §4.1.5 · pp. 79–80

4.1.6 Summary.

• Realizing foundation models’ potential for downstream tasks depends on architectural and modeling advances that fulfill all five properties.These properties respectively target capturing real-world information, handling high-volume data, processing multiple domains, storing and retrieving knowledge, and generalizing to novel settings.Source: §4.1.6 · p. 80

4.2.1 Goals of training objectives.

• Self-supervised objectives exploit unlabeled, internet-scale data by designing training signals around the structure of each modality, while domain-complete tasks aim to elicit broadly useful capabilities.The breadth of capabilities required by an objective is difficult to predict or evaluate in advance.Source: §4.2.1 · p. 81
• Training methods should make efficient use of compute: objectives with richer signals can help models learn faster, and predictable capability scaling can guide choices among architectures, data, and compute.Models are increasingly evaluated by the amount and kind of compute needed to reach a capability level.Source: §4.2.1 · pp. 81–82

4.2.2 Design trade-offs in current SSL methods.

• Raw-byte inputs preserve detail but can emphasize less semantic features and raise transformer costs, while tokenization or patch embeddings reduce input size but may discard useful information.The choice between continuous and discrete inputs also affects adaptation.Source: §4.2.2 · p. 82
• Generative objectives learn input distributions and support generation-based interaction, whereas discriminative objectives may learn more efficiently for classification or regression in high-dimensional continuous domains.Generative approaches include autoregressive and denoising models; discriminative methods often compare representations across views or distinguish real from fake input parts.Source: §4.2.2 · pp. 82–83
• Multimodal training can encode modalities separately for cross-modal retrieval or jointly for tasks such as visual question answering, but the capabilities enabled by these designs remain underexplored.The appropriate relationship between modalities depends on a model’s context and goals.Source: §4.2.2 · p. 83

4.2.3 Paths forward.

• Future work seeks a general self-supervised objective that works across domains and more efficient training signals, potentially by having algorithms construct richer examples as models improve.Current objectives are domain-specific, and data quality and availability also shape the training signal.Source: §4.2.3 · pp. 83–84
• Goal-directed training could teach foundation models to carry out diverse real-world tasks through online or offline interactions, without human annotations or task construction.The proposed direction targets general capabilities rather than adapting an existing model to one task, and may use goal-conditioned sequence training.Source: §4.2.3 · p. 84

4.3 Adaptation

• Adaptation conditions a foundation model on new information or objectives through prompts, added data, or parameter updates.It can specialize a model for a task or domain, or address settings such as test-time data removal and editing behavior on particular inputs.Source: §4.3 · p. 85, Figure 18 · p. 85

4.3.1 Methods for foundation model adaptation.

• Adaptation choices depend on compute budget, task-specific data, and gradient access, which ranges from output-only prompting to full fine-tuning.With output-only access, in-context learning freezes parameters and uses designed prompts; input-gradient access can support continuous prompts, while parameter-gradient access enables full fine-tuning.Source: §4.3.1 · pp. 85–87
• Lightweight methods freeze most pretrained parameters and can match full fine-tuning while updating 1000× fewer parameters, though their scaling remains uncertain.Methods include tuning selected parameters, adding modules, and optimizing soft prompts; low-memory techniques may trade memory for computation and energy use.Source: §4.3.1 · p. 86

4.3.2 Use cases for adaptation.

• Temporal shifts make frequent retraining costly, while reweighting, dynamic evaluation, time conditioning, and retrieval can help keep models current but do not fully solve adaptation.Retrieval-based approaches update database information without retraining, but still face challenges in accurate retrieval and conditioning.Source: §4.3.2 · p. 88
• Domain specialization can preserve broad task capability, but general pretraining may cause negative transfer, and continued domain training does not always outperform training from scratch.Intermediate training on unlabeled domain data has improved performance in satellite images and specialized text topics, while outcomes vary by domain.Source: §4.3.2 · p. 88
• Local model editing aims to change predictions for a single input or its neighborhood without affecting unrelated inputs, but methods vary in reliability and may be costly to scale.Approaches include gradient-based objectives, higher-order networks, and constrained fine-tuning.Source: §4.3.2 · pp. 88–89
• Adaptation can reduce memorized personal data and harmful outputs, but machine unlearning remains understudied for foundation models and safe, equitable training needs broader research.Further fine-tuning on curated data and controllable generation have shown some success in mitigating harmful behavior.Source: §4.3.2 · p. 89

4.3.3 A long-term goal for foundation model adaptation research.

• Continual adaptation is a long-term research challenge: it must approach the performance of training on an aggregate i.i.d. mixture while learning from non-stationary streams without catastrophic forgetting.Potential benefits include faster updating, reduced retraining burdens, and less relearning, but training on deployed-model experiences risks feedback loops and eroded alignment.Source: §4.3.3 · pp. 89–90

4.4.1 Introduction.

• Evaluation must track progress, explain model behavior, and document results, but adaptation, emergent skills, and myriad applications complicate comparison, anticipation, and comprehensive reporting.The framework distinguishes intrinsic evaluation of foundation models from extrinsic evaluation of task-specific derivatives, with different stakeholder needs and desiderata.Source: §4.4.1 · p. 91

4.4.2 Intrinsic evaluation.

• Complement broad task benchmarks with direct intrinsic evaluation, since adaptation can obscure foundation-model properties and training-objective metrics may not generalize across objectives or capabilities.Direct measures of capabilities, skills, and biases can help surface emergent phenomena such as in-context learning; human interaction may aid discovery, while downstream outcomes can help validate intrinsic measures.Source: §4.4.2 · pp. 92–94

4.4.3 Extrinsic evaluation and adaptation.

• Extrinsic evaluations should account for adaptation resources and access requirements to compare adaptation methods more informatively and guide method selection for a given foundation model.Count all data used to adapt a model or choose an adaptation method, and distinguish access needs such as gradients for fine-tuning versus black-box access for prompting; conclusions remain specific to the foundation model evaluated.Source: §4.4.3 · pp. 94–95

4.4.4 Evaluation design.

• Foundation-model benchmarks can be smaller and more diverse, while evaluation should report multiple desiderata and let stakeholders account for trade-offs in rankings.Sample-efficient adaptation can shift benchmark priorities from quantity toward quality and diversity; criteria may include robustness, fairness, efficiency, and environmental impact alongside accuracy.Source: §4.4.4 · pp. 95–96

4.4.5 Takeaways.

• Evaluation tailored to foundation models should combine task-derivative performance with direct measurement, account for adaptation resources, and reflect diverse stakeholder values.The section presents these changes as ways to make evaluation more informative for how adaptation is conducted and to use sample efficiency to support more diverse evaluations.Source: §4.4.5 · p. 96

4.5 Systems

• Foundation-model compute and memory demands are a major systems bottleneck, growing faster than hardware and making large-scale training and inference difficult.Model requirements have grown by three orders of magnitude in three years; the figure reports language-model growth of roughly 10× a year versus hardware capacity growth of roughly 10× in four years. GPT-3 training can require > 1000 petaFLOP/s-days.Source: Figure 19 · p. 97, §4.5 · p. 97

4.5.1 Improving performance through co-design.

• Co-design can improve performance, but scaling and resource-saving techniques introduce constraints or alter training semantics.Parallelism, state-sharding, compilers, and optimized libraries enable efficient training, yet data, pipeline, and tensor parallelism each have scaling limits; retrieval-based models keep knowledge in updateable text passages while requiring retriever and fine-grained interaction functions not readily supported by common frameworks and indexes [Guu et al. 2020; Lewis et al. 2020b; Khattab et al. 2020; Borgeaud et al. 2021].Source: §4.5.1 · pp. 98–99

4.5.2 Automated optimization.

• Automating optimization is difficult because combinations interact combinatorially, and semantics-altering techniques require estimating their joint effects on time-to-accuracy.Existing systems automate graph substitutions, distributed execution, and hybrid distribution strategies, but broader automated composition requires collaboration between systems and machine-learning experts.Source: §4.5.2 · p. 99

4.5.3 Execution and programming models.

• Shared pretrained-model components can amortize storage and execution, but systems lack interfaces for specifying cross-model dependencies and adaptation lineage.Lineage can aid debugging by distinguishing errors inherited from pretraining from those introduced during adaptation; volunteer-compute approaches can lower one entity’s training cost but require security, fault tolerance, and crowdsourcing collaboration [Ryabinin and Gusev 2020; Diskin et al. 2021].Source: §4.5.3 · pp. 99–100

4.5.4 Productionization of foundation models.

• Production deployment requires meeting inference latency and cost constraints while automating model and dataset quality assurance.Compression and parallelization can help meet deployment constraints; behavioral tests and model assertions support runtime monitoring, continuous improvement, and reducing mispredictions and fairness or bias issues.Source: §4.5.4 · p. 100

4.6.2 Data Hub Solution.

• The proposed data hub is a holistic toolkit for interactive, end-to-end management of the foundation model data lifecycle, addressing gaps in existing systems.Existing systems may lack first-class data integration, lifecycle support through model predictions, or interaction-driven curation and refinement.Source: §4.6 · p. 101, §4.6.2 · pp. 102–103

4.6.1 Data Management Desiderata.

• Foundation models’ growing data scale, including multimodal datasets, requires highly scalable data-management techniques.Public data is only a small fraction of the petabytes of business and personal data collected daily and used in industrial pipelines.Source: §4.6.1 · pp. 101–102
• Integrating structured and unstructured data can improve generalization and factual recall, but scalable integration across modalities and domains remains a challenge.Relevant data may span text, video, eye-tracking, and robotic simulations, as well as government, business, and science.Source: §4.6.1 · p. 102
• Data quality tools are immature despite data containing biases, poisoned or false information, duplicates, and shifts that can harm model behavior on critical subpopulations.The section calls for interactive, iterative tools that detect and potentially mitigate undesirable data while adapting to continuously changing training data.Source: §4.6.1 · p. 102
• Training-data use raises privacy and consent risks, while unresolved governance and copyright questions around web-scraped data create a need for tools that adapt to emerging rules.Downstream uses may not be anticipated, and data may be used outside the context for which consent was given.Source: §4.6.1 · p. 102

4.6.2 Data Hub Solution.

• The hub should provide scalable storage and querying alongside first-class integration of heterogeneous structured and unstructured data across modalities and domains.It should support changing datasets, scalable selection and filtering, heterogeneous compute, and cloud infrastructure.Source: §4.6.2 · p. 103
• The hub should enforce access controls through dataset documentation, license and privacy safeguards, and tools to identify licensing or governance violations.Documentation should explain data sources, intended uses, biases, and limitations, and may need updating as datasets evolve.Source: §4.6.2 · p. 103
• Open questions include reproducible versioning, privacy-preserving provenance, public-sector responsibility, affordable documentation, targeted data modifications, and monitoring for update needs.The authors present the hub as initial thoughts rather than a complete or fully detailed solution.Source: §4.6.2 · p. 104

4.7.1 Risks.

• A foundation model can become a single point of failure: poisoning or adversarial attacks may affect adapted applications, while memorized private data or centralized services can expose sensitive information.Applications may also need their own security guarantees, even when the underlying model is secure.Source: §4.7.1 · pp. 105–106, §4.7 · p. 105
• Public foundation-model parameters can facilitate model stealing by reducing the task to reverse-engineering an adaptation, while high-cost queries may worsen denial-of-service attacks on providers.The model-stealing example is a linear model trained on features from a public, frozen model.Source: §4.7.1 · p. 106
• Web-scraped, weakly supervised training data can be poisoned; GPT-2 code completion was induced to suggest insecure snippets with a few malicious files [Schuster et al. 2021].Targeted attacks on CLIP-style models required modifying as few as two of three million training examples [Carlini and Terzis 2021].Source: §4.7.1 · p. 106
• General-purpose features create function-creep and dual-use risks: despite facial recognition being out of scope, CLIP can be repurposed for surveillance [Radiya-Dixit and Tramèr 2021].CLIP learned rich facial features while trained to predict image-text pairs, illustrating how difficult it can be to foresee or constrain later uses.Source: §4.7.1 · p. 106
• Multimodal inconsistencies can be exploited, from CLIP mistaking an apple labeled “iPod” for an iPod to text cues misleading vision-based facial recognition or driving systems.Adapted tasks that rely on one modality may remain sensitive to signals in another, such as imprinted text or a “green” billboard.Source: §4.7.1 · pp. 106–107

4.7.2 Opportunities.

• Foundation models can act as security choke points: robustness or defenses against attacks may transfer to adapted applications, though their ubiquity also makes them attractive targets.A model robust to adversarial examples can retain robustness after adaptation [Shafahi et al. 2019]; providers could also defend against poisoning, model stealing, or resource depletion.Source: §4.7.2 · p. 107
• Public-data pretraining may reduce the confidential data needed for sensitive tasks such as healthcare, although current privacy mitigations may not meet users’ broad expectations.End-to-end differentially private training for healthcare can require vast amounts of sensitive data, whereas adapting a publicly pretrained model may require significantly less confidential data.Source: §4.7.2 · p. 107
• Foundation models’ scale may support adversarial robustness research, but current models show little improvement against worst-case perturbations; whether distributional robustness helps against constrained real-world attacks remains open.Research points to the potential value of unlabeled data and over-parameterization, while CLIP-like models’ distributional robustness has not yet established resilience to attacks.Source: §4.7.2 · pp. 107–108

4.8.1 Advantages.

• On ImageNet, CLIP and ResNet50 each achieve 76% accuracy; on related but different ImageNetV2 and ImageNet Sketch distributions, CLIP achieves 6% and 35% higher accuracy, respectively [Radford et al. 2021].Source: §4.8.1 · p. 109
• Large-scale pretraining improves robustness across common corruptions, label shifts, satellite imagery, and language tasks, while multilingual BERT improves unseen language-pair performance [Hendrycks et al. 2019a,b; Liu et al. 2020b].Foundation-model representations improve robustness for many shifts, but not all, including spurious correlations and changes over time.Source: §4.8 · p. 109, §4.8.1 · p. 110, Figure 21 · p. 110

4.8.2 Persistent challenges.

• Foundation models may mitigate or amplify spurious correlations depending on the task and training data; scale alone does not solve reliance on non-generalizing features.Diverse pretraining can help models learn from counterexamples, but pretrained demographic biases can also be inherited; managing inductive bias and adaptation is needed.Source: §4.8.2 · pp. 110–111
• Foundation models have limits in extrapolation: language models struggle with changing world knowledge, CLIP transfers poorly to satellite imagery, and ImageNet pretraining does not substantially improve medical-image performance.Researchers need to distinguish forms of extrapolation enabled by foundation models from those that remain out of reach.Source: §4.8.2 · p. 111

4.8.3 Opportunities.

• Robustness research should explain how pretrained representations and inductive biases help across distribution shifts, since current empirical gains lack a complete theoretical account.Proposed work includes measuring whether pretraining brings in-distribution and out-of-distribution representations closer; existing theories do not cover effective generative language models such as GPT-3 [Brown et al. 2020].Source: §4.8.3 · pp. 111–112
• Robustness-oriented training should align augmentations with needed invariances and use metadata or other structure as pretraining targets to improve downstream OOD accuracy.Rotation augmentations may help tasks requiring rotation invariance but not other shifts; metadata has improved OOD accuracy in satellite imagery [Xie et al. 2021a].Source: §4.8.3 · p. 112
• More diverse training data is not always better than specialization; continued pretraining on satellite imagery or specialized text can improve downstream performance [Reed et al. 2021; Gururangan et al. 2020].Research should clarify how specialization affects both in-distribution and out-of-distribution performance.Source: §4.8.3 · p. 112
• Freezing pretrained parameters can improve OOD performance over full fine-tuning in vision, but the mechanism remains unclear and shift-aware adaptation is an open direction [Wortsman et al. 2021; Kumar et al. 2022].Full fine-tuning can distort pretrained features, while lightweight language-model tuning also appears to offer OOD benefits [Houlsby et al. 2019; Li and Liang 2021; Lester et al. 2021].Source: §4.8.3 · pp. 112–113

4.9.1 Traditional problems in AI safety.

• Traditional AI safety seeks to characterize and reduce catastrophic risks through technical control and sociotechnical safeguards, while RL alignment must address hard-to-specify human values, reward hacking, and corrigibility.Agents may resist attempts to change their goals; because foundation models can become interactive and goal-directed through self-supervised training, alignment concepts developed for pure RL need careful reconsideration.Source: §4.9.1 · pp. 114–115

4.9.2 Current foundation models and AI safety.

• Foundation models can exhibit goal-directed behavior without explicit goal optimization, making alignment and control central safety problems.Natural-language steering may help specify and interpret tasks, but reliability and mechanistic understanding remain unresolved, and models may acquire deceptive or otherwise undesirable behavior from human data.Source: §4.9.2 · pp. 115–116
• Forecasting foundation-model capabilities is difficult because applications are open-ended, abilities emerge with scale, and prompt variation can change performance.Future training in rich environments could produce highly capable agents with deception or strategic planning skills, raising unresolved challenges for scalable oversight and control.Source: §4.9.2 · p. 116

4.9.3 Potential catastrophic risks from future foundation models.

• Potential catastrophic risks include correlated robustness failures across critical uses and the large-scale pursuit of misaligned proxy goals.Optimizing engagement, profit, or GDP instead of broader human welfare could contribute to societal, environmental, or geopolitical harms [Kenton et al. 2021; Gabriel 2020].Source: §4.9.3 · pp. 116–117

4.9.4 Conclusion.

• The authors urge research to forecast foundation-model capabilities and risks, develop alignment methods, and coordinate proactive mitigation across states, research labs, and businesses.Current and potential future emergent properties make foundation models important subjects for AI safety research.Source: §4.9.4 · p. 117

4.10.1 Theoretical formulations and modularizations.

• The modular framework separates pretraining and adaptation, reducing generalization and optimization analyses largely to supervised learning while isolating the pretraining–adaptation interface as the core challenge.It decomposes downstream loss into generalization error and minimal adaptation loss; the unresolved question is why low population pretraining loss can yield low minimal adaptation loss.Source: Figure 22 · p. 118, §4.10.1 · pp. 119–120

4.10.2 Why is the pretraining-adaptation interface interesting?.

• The key interface question is when and why small population pretraining loss implies small minimal adaptation loss, beyond what standard generalization theory explains.The framework aims to clarify interactions among pretraining and adaptation quantities while setting aside generalization concerns.Source: §4.10.2 · p. 121
• Successful adaptation may depend on distribution relations, architecture and intermediate representations, adaptation complexity, and how pretraining and adaptation objectives are combined.Understanding these factors could help predict robustness to distribution shifts, improve few-shot sample efficiency, and identify surrogate pretraining objectives that support broad downstream tasks.Source: §4.10.2 · p. 121

4.10.3 Challenge: analysis of in-context learning and other emergent behavior.

• In-context learning lacks parameter updates, so the proposed optimization-based modularization does not directly explain its mechanism or other emergent behaviors.Under assumptions of infinite pretraining data and sufficient expressivity, Xie et al. [2021c] propose that in-context learning arises from long-term coherence in the pretraining distribution, modeled with latent variables.Source: §4.10.3 · pp. 121–122

4.10.4 Challenge: appropriate data assumptions and mathematical tools.

• Theory needs realistic, interpretable data assumptions and broader mathematical tools to connect foundation-model properties with diverse population distributions.Existing analyses use properties such as expansion or structured latent-variable models; spectral graph theory and tractable models offer approaches, but current latent-variable analyses remain limited to relatively simple cases.Source: §4.10.4 · p. 122

4.11 Interpretability

• Understanding interpretability requires determining whether a foundation model uses shared, general mechanisms or task-specific independent ones, and asking what it does, why, and how.Shared mechanisms could support explanations across tasks; independent ones would require studying behaviors separately.Source: §4.11 · p. 123, Figure 23 · p. 124

4.11.1 Characterizing behavior.

• Unknown task spaces, vast input and output domains, and prompt-sensitive adaptations make foundation-model capabilities difficult to characterize; controlled evaluations can probe them without implementation access.The authors call for broader evaluations by diverse communities and experts across disciplines.Source: §4.11.1 · pp. 124–125

4.11.2 Explaining behavior.

• Post-hoc explanations may be unfaithful and task-local, while fluent self-explanations may sound plausible without providing truth or insight into model behavior.Existing approaches include local explanations of particular inputs and global explanations of data qualities affecting behavior.Source: §4.11.2 · p. 126

4.11.3 Characterizing model mechanisms.

• Internal probes and activation-based methods inspect representations and mechanisms, but cover only a small slice; whether adaptations share mechanisms determines tractability or demands separate study.Understanding mechanisms can help assess how complex behaviors are assembled, while identifying mechanisms such as race estimation may support responsibility to restrict high-stakes uses.Source: §4.11.3 · pp. 127–128

4.11.4 Impacts of non-interpretability and interpretability.

• Interpretability work can be used to ‘ethics-wash’ opaque models and centralize knowledge, yet accessible methods can empower affected communities to investigate, consent to, improve, or contest their use.The authors urge researchers to examine whether interpretability is achievable for all or whether its absence is intrinsic and should discourage use or increase regulation.Source: §4.11.4 · p. 128

5.1.2 Harms

• Intrinsic representational biases can remain latent until adaptation and use, while downstream applications can cause individual harms and group-level performance disparities with more severe consequences in high-stakes settings.The chapter distinguishes biases within foundation models from harms experienced in specific applications, including representational harms, abuse, and disparities affecting marginalized groups.Source: §5.1.2 · pp. 130–131

5.1.3 Sources

• Biases can arise from training and adaptation data, modeling choices, and developer representation, but how data practices relate to model biases remains unclear.The authors call for new protocols, including studying bias scaling laws, and for user feedback to inform model design.Source: §5.1.3 · pp. 132–133

5.1.4 Interventions and recourse

• Technical mitigation methods may be brittle, misaligned with stakeholder goals, or worsen inequity; recourse is also difficult because responsibility and feedback paths between providers and application developers are unclear.The authors call for upstream feedback and accountability norms and standards that can adapt as models and applications change.Source: §5.1.4 · pp. 134–135

5.1.5 Takeaways

• The one-to-many use of foundation models propagates intrinsic biases across applications, and addressing resulting inequities requires source attribution, proactive intervention, and reactive recourse.Reliable attribution requires further technical research; proposed responses include changes to data and modeling as well as feedback and accountability mechanisms.Source: §5.1.5 · p. 135

5.2.1 Foundation models will be misused for harmful purposes

• Foundation models may enable high-quality, low-cost, personalized harmful content at scale, including human-like content in many languages and material tailored to audiences or individuals.The section defines misuse as intentionally harmful use of models’ intended capabilities, distinguishing it from inequity without harmful intent and security exploits of unintended capabilities.Source: §5.2 · p. 136, §5.2.1 · p. 137

5.2.2 Foundation models will be powerful detectors of harmful content

• Foundation-model-generated content can frustrate human discovery methods, while models’ interactive and multimodal capabilities may aid automatic detection, which carries risks when deployed online.The text notes that synthetic photos already undermine reverse-image searches and says similar challenges extend to text and video.Source: §5.2.2 · p. 138

5.2.2 Foundation models will be powerful detectors of harmful content.

• Foundation models could detect generated disinformation and toxicity [Zellers et al. 2019b; Schick et al. 2021], adapt to new misuse strategies from feedback [Lee et al. 2021a], and model dissemination signatures.Prior work found that misinformation spreads faster than authentic content; joint representations of harmful content and dissemination patterns could help predict misuse.Source: §5.2.2 · p. 138
• False positives can flag fair human content and harm users, while widespread detection may trigger an arms race with attackers who adapt content to evade detectors.Although models may be locally edited to address systemic false positives, corner cases may be deprioritized and recourse difficult.Source: §5.2.2 · p. 139

5.3 Environment

• 78–12,768 kgCO2eq per day could be saved by a slightly more efficient translation model deployed commercially, while training and inference both consume substantial energy.The report notes that training one BERT-base model can, under some conditions, produce emissions equivalent to those offset by 40 trees grown for 10 years.Source: §5.3 · pp. 140–141

5.3.1 Carbon impacts can and should be mitigated in many cases.

• Training emissions can be reduced by choosing low-carbon energy grids; when that is not possible, developers should reduce unnecessary energy use and prioritize emitting less over relying on offsets.Options include efficient hardware and architectures, mixed-precision training, quantization, and distillation; offsets can have harmful side effects such as reduced biodiversity [Holl and Brancalion 2020; Heilmayr et al. 2020; Hong et al. 2020b].Source: §5.3.1 · pp. 141–142

5.3.2 Costs and benefits should be assessed before using foundation models.

• Before deployment, compare a model’s social and environmental costs with its benefits, consider simpler alternatives, and examine how benefits and harms are distributed across communities.The framework is intended to compare effects and their relative importance despite uncertainty in monetary estimates, not to provide definitive valuations.Source: §5.3.2 · pp. 142–144
• ∼80 tasks are needed in a hypothetical BERT Base comparison to recover its upfront energy costs; foundation models are not guaranteed to outperform every baseline in efficiency.Lifetime costs may be amortized across tasks, but smaller parameter counts alone do not guarantee energy efficiency, so developers should assess models and adaptation methods before large-scale training.Source: Figure 27 · p. 144, §5.3.2 · pp. 144–145

5.3.3 Carbon/energy impacts should be systematically reported.

• Developers, providers, and curators should report computational, energy, and carbon costs and mitigation strategies; standardized reporting can support policy and help users identify carbon-friendlier usage.Suggested incentives include conference green badges, requiring relevant metrics in submissions, and encouraging transparency from large-scale deployers.Source: §5.3.3 · p. 145

5.4 Legality

• The legal landscape for algorithmic tools remains uncertain, and the section considers how US law may affect foundation-model training, liability for predictions, and protections for outputs.Source: §5.4 · p. 146

5.4.1 Training.

• Foundation-model training raises legal questions about scraping, copyright, and privacy, but ethical frameworks and studying misuse and security are also critical to preventing harms ex ante, unlike law’s often ex post treatment.Source: §5.4.1 · p. 146, §5.4 · p. 146

5.4.2 Output liability.

• Foundation models used in decisions or physical systems may expose creators and operators to liability, but courts’ standards and allocation of responsibility remain unclear.Physical deployments raise tort questions, and sensitive domains may require regulatory approval and standardized safety assessment [Wu et al. 2021g]. Classification linked to protected attributes may prompt civil-rights claims in hiring, housing, or credit, while traditional discrimination doctrine may constrain fairness methods [Ho and Xiang 2020]. Government use may also invite procedural due-process, reasonableness, and transparency challenges.Source: §5.4.2 · p. 147

5.4.3 Legal protections for outputs.

• Legal protections for generated content remain uncertain, including free-speech coverage, disclosure duties, and who may claim ownership of model outputs.Courts have not settled whether AI-generated content is protected speech or whose speech it is, and disclosure requirements could affect users and creators. Copyright law does not treat computer programs as authors, though human creators and users may have authorship claims; disputes are likely to grow as models are used in creative and routine production.Source: §5.4.3 · pp. 147–148

5.5.1 Productivity and Innovation.

• 13% of occupations have a primary writing task, and their wage bill exceeds 675 billion dollars, illustrating foundation models’ broad productivity potential.An autocomplete system for clinical documentation reduced keystroke burden by 67% [Gopinath et al. 2020]. Foundation models may also boost innovation by enabling new creative work, products, and processes [Ramesh et al. 2021], though traditional productivity and GDP measures may miss benefits and costs that emerge over years or decades.Source: §5.5.1 · pp. 149–150

5.5.2 Wage inequality.

• Whether productivity gains raise or depress employment and wages depends on whether models substitute for workers, complement their labor, or create new tasks.Foundation models may augment humans in creative work rather than replace them, but effective collaboration requires better interfaces, interpretability, and robustness [See et al. 2019]. Effects on income inequality therefore depend on how the models are used.Source: §5.5.2 · p. 150

5.5.3 Centralization.

• Concentrated ownership of data and models may centralize decision-making power, while foundation models’ economic outcomes depend on choices by societal stakeholders.Open-source and distributed-training efforts counter concentration, but resource demands may keep a large gap between private and community models. Models may also change job quality, trade, and occupational shifts, with emergent capabilities creating difficult-to-predict effects; the chapter frames them as a general-purpose technology with potential to raise living standards or inequality.Source: §5.5 · p. 149, §5.5.3 · p. 151, §5.5.4 · p. 151

5.6.1 Homogenization and scale.

• Reusing foundation models across domains can spread common errors and biases, producing homogenized judgments and culturally dominant perspectives.Shared training data can yield similar spurious correlations and uniform failures across adapted models; broadening dataset perspectives and using diverse personas may help, but balancing diversity with relevance and utility remains unresolved [Creel and Hellman 2021; Gandy 2021; Zhou et al. 2021a; Sheng et al. 2021]. Because weaknesses are hard to predict and remedy, widespread adoption increases developers’ responsibilities of care.Source: §5.6.1 · pp. 152–153

5.6.2 Surveillance, exclusion, and power.

• The scale of foundation-model development can encourage quantity over data quality and shift social power toward model creators.Mass data collection relies on automated preprocessing that may be blunt or poorly documented; despite evolving data-protection laws, questionable acquisition practices include opaque policies and manipulative interfaces.Source: §5.6 · p. 152, §5.6.2 · p. 153
• Data and compute advantages may concentrate foundation-model development among well-resourced organizations while encouraging aggressive data collection and control over data produced by derivatives.The section warns that this can reinforce market concentration and limit marginalized communities’ participation.Source: §5.6.2 · p. 154
• Automated decisions can reproduce existing problems: benchmark gains may not translate to real-world outcomes, and human decision-makers may reintroduce bias.The section calls for ecologically valid ongoing evaluation, clear communication about capabilities and limitations, and dialogue about power and adoption.Source: §5.6.2 · p. 155

5.6.3 Norms.

• Responsible-model norms need institutional support, including documentation, public reporting channels, and labels that let affected people trace problems through adapted derivatives.Universities can model norms through audits and ethics review boards; feedback from derivatives can help reveal problems spanning uses, while technical and social barriers can impede tracing.Source: §5.6.3 · p. 156
• Socially consequential uses should have legal standards requiring adapted derivatives to demonstrate specified properties, alongside frameworks for attributing harm and providing recourse.Reporting the full pipeline can help trace harms, but tracing alone may not lead to changes without responsibility and recourse mechanisms.Source: §5.6.3 · pp. 156–157

5.6.4 Release and Auditing.

• A neutral staged-release board or federal auditors could broaden access across disciplines and sectors while reducing favoritism and pressure on auditors to withhold unflattering findings.Open-sourced models can also support adaptation to languages underserved by commercial providers, though compute barriers still limit access.Source: §5.6.4 · pp. 157–158
• The section argues that release can enable independent audits and broader discovery of uses and limitations, and that these benefits outweigh potential misuse by less-resourced actors.For proprietary or private-data models, it recommends staged audits; it argues that powerful institutional actors capable of severe harms could likely create similar models without release.Source: §5.6.4 · pp. 157–158

5.6.5 When not to build.

• Choosing not to build can be a deliberate, context-sensitive response when a model may cause more harm than good or worsen systemic problems better addressed otherwise.The decision is also collective: ethically dubious work can make it competitively harder for others to abstain, while the section calls for stronger professional ethics and respect for consent, including ways to opt out or revoke it.Source: §5.6.5 · pp. 159–160

5.6.6 Conclusion.

• The section concludes that developers should adopt norms for development, auditing, and release, supported by legal requirements, while people should be able to refuse data or decision-subject participation without repercussion.It also flags a largely unsurveyed concern: generative models may replace work many people find meaningful, such as graphic design and writing.Source: §5.6.6 · p. 160

• Foundation models have only begun transforming how AI systems are built and deployed, and much about this emerging paradigm remains unclear.Source: §6 · p. 161
• The report surveys critical aspects of foundation models, from technical underpinnings to societal consequences, to orient and frame dialogue about this new AI paradigm.Source: §6 · p. 161
• Collaboration across sectors, institutions, and disciplines from the outset is critical to foundation models’ responsible development and deployment on durable foundations.Source: §6 · p. 161

• The report was authored by CRFM, a Stanford University center that emerged from the Stanford Institute for Human-Centered Artificial Intelligence.Source: CONFLICT OF INTEREST · p. 161
• As of July 2022, CRFM received funding from Google, Microsoft, and the McGovern Foundation, though that funding was not directly related to this report.Source: CONFLICT OF INTEREST · p. 161
• Authors may have affiliations beyond Stanford, and their contributions reflect only their views, not those of those institutions.Source: CONFLICT OF INTEREST · p. 161