PaperlayerView exact revision on arXiv

Source-linked AI summary

The Foundation Model Transparency Index

Rishi Bommasani, Kevin Klyman, Shayne Longpre, Sayash Kapoor, Nestor Maslej, Betty Xiong, Daniel Zhang, Percy Liang

arXiv:2310.12941v1cs.LGcs.AI

TL;DR

Transparency evidence about foundation model developers remains limited, despite the technology’s growing societal impact and acknowledged risks. The paper introduces an index with 100 indicators and scores 10 developers’ flagship models. Its findings characterize the status quo as widespread but uneven lack of transparency, while warning that disclosure scores do not measure responsible business practices.

  • Problem

    Evidence about which foundation model developers disclose which information, and where industry blind spots lie, remains limited despite concerns about opacity and its risks.

    Source: §1 · p. 4

  • Method

    The Foundation Model Transparency Index scores 10 developers’ flagship models using 100 indicators across upstream resources, model-level information, and downstream practices.

    Source: §1 · p. 4

  • Results

    The ecosystem’s status quo features widespread lack of transparency across developers, with substantial unevenness in performance and areas for improvement.

    Source: §10 · p. 63

  • Takeaways & Limitations

    The index establishes a transparency baseline and identifies gaps that can inform efforts to improve transparency over time.

    Source: §1 · p. 4, §10 · p. 63

    The index measures whether information is disclosed, not whether developers follow responsible business practices, and may reward performative disclosure.

    Source: §9.2 · pp. 60–61

Abstract

from arXiv · show

Foundation models have rapidly permeated society, catalyzing a wave of generative AI applications spanning enterprise and consumer-facing contexts. While the societal impact of foundation models is growing, transparency is on the decline, mirroring the opacity that has plagued past digital technologies (e.g. social media). Reversing this trend is essential: transparency is a vital precondition for public accountability, scientific innovation, and effective governance. To assess the transparency of the foundation model ecosystem and help improve transparency over time, we introduce the Foundation Model Transparency Index. The Foundation Model Transparency Index specifies 100 fine-grained indicators that comprehensively codify transparency for foundation models, spanning the upstream resources used to build a foundation model (e.g data, labor, compute), details about the model itself (e.g. size, capabilities, risks), and the downstream use (e.g. distribution channels, usage policies, affected geographies). We score 10 major foundation model developers (e.g. OpenAI, Google, Meta) against the 100 indicators to assess their transparency. To facilitate and standardize assessment, we score developers in relation to their practices for their flagship foundation model (e.g. GPT-4 for OpenAI, PaLM 2 for Google, Llama 2 for Meta). We present 10 top-level findings about the foundation model ecosystem: for example, no developer currently discloses significant information about the downstream impact of its flagship model, such as the number of users, affected market sectors, or how users can seek redress for harm. Overall, the Foundation Model Transparency Index establishes the level of transparency today to drive progress on foundation model governance via industry standards and regulatory intervention.

1 INTRODUCTION

The paper introduces an index to assess transparency across foundation model development and use, against a backdrop of growing societal impact and limited transparency evidence. Its findings show uneven disclosure across developers and domains, and it recommends practical steps for improving transparency.

1 INTRODUCTION

  • The index scores 10 developers against 100 indicators across upstream resources, model characteristics, and downstream practices, using each developer’s flagship model.The indicators organize transparency into three domains and cover matters such as data, labor, compute, capabilities, risks, distribution, and affected geographies.Source: §1 · p. 4

1.1 Findings

  • Meta scores 54/100 versus Amazon’s 12/100, a 42-point spread around a 37-point mean; 82 indicators are already met by at least one developer.The developers fall into three clusters: four above the mean, three around it, and three below it.Source: §1.1 · p. 5
  • Upstream transparency scores worst, and no company scores on data creators, copyright and license status, or copyright mitigations.AI21 Labs, Inflection, and Amazon receive zero points across the 32 upstream indicators.Source: §1.1 · p. 5
  • Developers disclose capabilities more often than limitations, risks, and mitigations; none provides externally reproducible or third-party assessments of mitigation efficacy.Just two developers demonstrate limitations, and none evaluates multiple intentional harms its models could facilitate.Source: §1.1 · p. 6
  • No developer reports affected sectors, individuals, geographies, or usage, and none offers redress; the Impact subdomain averages 11%.Only three developers minimally characterize the number of downstream applications.Source: §1.1 · p. 6

1.2 Recommendations

  • Developers are urged to learn from competitors’ practices, deployers to press for transparency, and policymakers to target requirements using evidence about gaps.Data Labor and Impact are identified as especially urgent areas for policy attention.Source: §1.2 · p. 7

1.3 Contributions

  • The paper contributes a taxonomy, standardized scores for 10 developers, 35 empirical findings, public materials, and plans for future index iterations.The taxonomy spans three domains, 23 subdomains, and 100 indicators; future versions are intended to track change over time.Source: §1.3 · p. 8

2 BACKGROUND

The background situates foundation models within a broad supply chain and explains why transparency matters across its stages. It also introduces composite indexes as a way to make complex constructs measurable, while noting that transparency and indexes have important limits.

2.1 Foundation models

  • Foundation models require substantial upstream resources but can be adapted for many downstream uses, linking a small set of developers to a much larger user ecosystem.The supply chain runs from resources such as data, hardware, energy, labor, and code through models and distribution channels to applications across sectors and geographies.Source: §2.1 · p. 9

2.2 Transparency

  • Transparency can support accountability and public-interest interventions, but disclosure alone does not ensure responsible practices or desired change.Transparency may also create tensions with privacy and competitive advantage, and can be performative or gamified.Source: §2.2 · pp. 11–12
  • The index takes an ecosystem-level approach, spanning upstream resources, models, and downstream use and impact rather than focusing only on data or models.Prior documentation and evaluation efforts often focus on data and models, while downstream impact has been comparatively understudied.Source: §2.2 · p. 13

2.3 Indexes

  • Composite indexes aggregate low-level indicators into scores that can aid public understanding and decision-making, but their choices can oversimplify complex constructs.Index design requires judgments about indicator selection, weighting, and grading.Source: §2.3 · pp. 13–14

3 THE FOUNDATION MODEL TRANSPARENCY INDEX

The Foundation Model Transparency Index provides a framework for assessing and tracking foundation-model transparency across the supply chain. Its 100 indicators cover upstream resources, model properties, and downstream use.

3 THE FOUNDATION MODEL TRANSPARENCY INDEX

  • The index aims to clarify transparency debates with empirical comparisons and a frame for tracking whether developers become more or less transparent over time.The authors plan to update indicators as the ecosystem and AI policy evolve.Source: §3 · p. 15

4 INDICATORS

  • The 100 indicators were derived from scientific literature and public concerns, refined with external researchers, and organized across upstream, model, and downstream domains.The authors describe the index as covering the foundation-model supply chain from resources through deployment.Source: §4 · p. 16, Figure 2 · p. 17

4.1 Upstream indicators

  • The 32 upstream indicators assess data, data labor, data access, compute, training methods, and data mitigations.They cover data size and composition, worker conditions, external access, hardware and energy use, training specifications, and privacy or copyright mitigations.Source: §4.1 · p. 16, Figure 3 · p. 18

4.2 Model indicators

  • The 33 model indicators cover basic properties, access, capabilities, limitations, risks, mitigations, trustworthiness, and inference.Evaluation indicators emphasize reproducibility and independent assessment, while risk indicators distinguish unintentional harms from intentional harms.Source: §4.2 · pp. 17, 19, Figure 4 · p. 20

4.3 Downstream indicators

  • The 35 downstream indicators assess distribution, policies, interfaces, data protection, updates, feedback, impact, and downstream documentation.Impact includes affected people, sectors, and geographies, usage reporting, monitoring, and redress; the authors include downstream use because it is a direct way AI affects society.Source: §4.3 · pp. 21, 23, Figure 5 · p. 22
  • Scoring requires developers to state or explicitly link indicator-related information in documentation; explicit statements that an action is not performed can also earn a point.The authors say such disclosures are exceedingly rare and developers generally share little about what they do or do not do.Source: §4.3 · p. 23

5 FOUNDATION MODEL DEVELOPERS

The authors assess ten influential foundation-model developers to compare their transparency practices. They select developers for impact, ecosystem diversity, and longitudinal relevance, and assess each in relation to a flagship model.

5 FOUNDATION MODEL DEVELOPERS

  • The authors assess developers directly with common indicators, enabling more consistent scoring and comparison than frameworks that rely on developers’ own uptake.They argue that comparing scores can provide context for improving transparency across the ecosystem.Source: §5 · p. 24

5.1 Selecting developers

  • The ten selected companies were chosen for model influence, diversity in release strategy, modality and market position, and their suitability as enduring targets for longitudinal assessment.The selected developers are AI21 Labs, Amazon, Anthropic, Cohere, Google, Hugging Face, Inflection, Meta, OpenAI, and Stability AI.Source: §5 · p. 24, Table 1 · p. 24, §5.1 · p. 25
  • Release ranges from closed access through staged, interface, and API access to downloadable weights, sometimes with training data; the authors often group releases as open or closed by weight availability.For the ten models considered, six were available via API, one was downloadable, two included downloadable training data, and one was fully closed at the time of writing.Source: §5.1 · pp. 25–26, Figure 6 · p. 26

5.2 Selecting flagship models

  • Each developer is assessed against its most salient and/or capable foundation model, judged using the company’s public description, because transparency may differ across a developer’s models.The assessment uses BLOOMZ for Hugging Face and GPT-4, rather than GPT-4V, for OpenAI.Source: §5.2 · p. 27

6 SCORING

The Index assesses transparency across 100 indicators for 10 foundation-model developers. Scores show uneven disclosure, with upstream resources especially opaque and substantial room for improvement across the ecosystem.

6 SCORING

  • Scoring uses publicly available information, binary indicator judgments, and independent ratings by two researchers followed by discussion to resolve disagreements.Across 1,000 indicator-developer pairs, agreement was 85.2% (Cohen’s κ=0.67); binary scoring reduced subjectivity but can leave out information captured by more complex schemes.Source: §6 · p. 28
  • All 10 companies responded to the score review, and feedback led to some score increases and more robust indicator definitions and justifications.Eight companies contested scores; most scores stayed unchanged, while rebuttals produced an average increase of 1.25 points among the eight.Source: §6 · p. 29

7 ANALYSIS

  • The analysis examines results by developer, indicator, domain, and broader dimensions of transparency.The finalized scores cover 100 indicators across 10 companies.Source: §7 · p. 30, §7.1 · p. 30, Figure 9 · p. 31

7.1 Overarching results

  • Meta scores 54 of 100 indicators, versus Amazon's 12; 82 indicators are achieved by at least one developer and 71 by multiple developers.The mean score is 37, with developers clustering well above, around, or well below the mean.Source: §7.1 · pp. 30–31, Figure 7 · p. 30
  • Upstream averages 7.2/32 indicators, below model at 14.1/33 and downstream at 15.7/35, while several model and user-facing subdomains score comparatively well.OpenAI's upstream score is 7, compared with 14 for Meta, 21 for Hugging Face, and 16 for Stability AI; model and downstream scores help explain differences among similarly ranked developers.Source: Figure 8 · p. 31, §7.1 · pp. 32–33

7.2 Upstream results

  • Upstream scores span 21 points: Hugging Face earns 21/32 indicators, while AI21 Labs, Inflection, and Amazon each earn 0.Upstream indicators concern resources such as data, labor, compute, methods, and code, which relate to model capabilities, risks, and development externalities.Source: §7.2 · p. 33, Figure 10 · p. 35
  • No developer scores on six upstream indicators, including data creators, licensing, copyrighted data, copyright mitigations, compute use, and broader environmental impact.Methods has the most upstream coverage, while data labor is least transparent; most developers also disclose little about compute, energy, or environmental impacts.Source: §7.2 · pp. 33, 34, 37

7.3 Model results

  • Model indicators average 14.1/33, with Model Basics, Capabilities, Limitations, and Model Access among the stronger-scoring subdomains.Some comparatively high subdomain scores reflect generous indicators, such as disclosing input and output modalities or describing capabilities and limitations.Source: Figure 11 · p. 36, §7.3 · p. 37

Transparency on capabilities does not translate to transparency on limitations, risks,

This section examines developer transparency about model capabilities, limitations, risks, mitigations, trustworthiness, and downstream impact, as well as company-level indicator correlations across upstream, model, and downstream domains. It also considers how downstream indicator-level heterogeneity is less apparent in aggregate scores.

Transparency on capabilities does not translate to transparency on limitations, risks,

  • Capabilities average 62%, versus 60% for Limitations, 24% for Risks, and 26% for Model Mitigations; eight developers demonstrate capabilities, but only Cohere demonstrates limitations.Capabilities are more transparent than other model subdomains, with 9 of 10 developers describing multiple capabilities and 3 of 10 providing third-party evaluations.Source: Transparency on capabilities does not… · pp. 37–38
  • Only OpenAI, Cohere, and AI21 Labs report rigorous trustworthiness evaluations, and only Cohere and AI21 Labs provide sufficient detail for external reproducibility; mitigation-risk links are often vague or absent.No developer scores on intentional-harm evaluation, external reproducibility of mitigation evaluations, third-party mitigation evaluations, or inference-compute evaluation. Model Access scores total 17 of 30 points.Source: Transparency on capabilities does not… · pp. 38, 40

7.4 Downstream results

  • Impact averages 11%: no developer reports affected sectors, individuals, usage reports, geographic statistics, or redress, and only five disclose potential monitoring mechanisms.Eight developers score points on just one of seven Impact indicators; the other two score none. Downstream scores otherwise cluster between 14 and 21 points for nine developers, while Amazon scores 6 of 35.Source: §7.4 · p. 40
  • Across downstream transparency dimensions, Distribution averaged 59%, while Usage Policy indicators averaged 44% and Model Behavior Policy indicators averaged 23% (five and three indicators, respectively).Every developer disclosed distribution channels and terms of service; only OpenAI and Inflection scored on usage-policy violation appeals.Source: §7.4 · p. 41
  • User-data protection scores are identical across developers, while feedback transparency is 30%; five developers disclose model-update practices, and user-interface scores average 85%.No developer discloses a usage-data access protocol or feedback summaries; every developer but Amazon has a public feedback mechanism. The interface indicators assess AI disclosure and disclaimers.Source: §7.4 · p. 42

7.5 Results for open and closed developers

  • Open developers score higher on average in every domain and on 15 of 23 subdomains, although closed developers lead on seven subdomains, including Risks and Model Mitigations.Mean scores for open versus closed developers are 53% versus 9% upstream, 51% versus 39% on model indicators, and 49% versus 43% downstream; the trends also hold for medians.Source: §7.5 · p. 44

7.6 Correlations between companies

  • Cohere and AI21 Labs are the most-correlated pair (SMC 0.87); Meta is the only developer without another developer correlated at SMC ≥0.80, and its most similar match is Google (0.78).The three most-correlated pairs are all among the four lowest-scoring companies; the least-correlated pairings are between open and closed developers.Source: §7.6 · pp. 45–46
  • Upstream correlations range from a minimum SMC of 0.34 to near 1, with Meta and Hugging Face at 0.53 despite their high scores.The four lowest-scoring companies overall score zero or nearly zero upstream, making them highly correlated; Hugging Face’s distinct points on data sources and data labor coincide with its differences from Meta and Stability AI.Source: §7.6 · pp. 47–48
  • Model-domain correlations vary: Cohere–AI21 Labs and Inflection–Amazon each score 0.85, while Google–Anthropic scores 0.88.OpenAI–Anthropic scores 0.79 and OpenAI–Google 0.67; Hugging Face–Stability AI scores 0.85, while Meta’s highest correlation is with Hugging Face at 0.76.Source: Figure 16 · p. 48, §7.6 · p. 49
  • Downstream correlations reveal shared patterns among open developers and substantial heterogeneity despite tightly clustered aggregate scores.Meta, Hugging Face, and Stability AI correlate strongly with one another (SMCs 0.89, 0.83, and 0.89, respectively); Meta also correlates strongly with Google, and OpenAI with Anthropic. Inflection is especially dissimilar from open developers, while OpenAI and Amazon are more dissimilar than in other correlation analyses. Shared downstream challenges such as monitoring and policy enforcement may contribute to the open-developer pattern.Source: §7.6 · p. 49, Figure 17 · p. 50

8 RECOMMENDATIONS

The section turns the assessment into improvement guidance for foundation model developers and deployers, and policymakers, while inviting researchers to scrutinize the index approach. Direct scoring gives assessed developers feedback on flagship-model transparency and guidance relevant to other models and future releases.

8 RECOMMENDATIONS

  • The authors offer concrete recommendations for developers and deployers, and policymakers, based on the index design, assessment, and analysis.They also encourage researchers to scrutinize the overall approach to clarify how foundation model transparency can be better understood.Source: §8 · p. 51
  • The authors invite researchers to scrutinize the index approach to clarify how foundation model transparency can be better understood.Source: §8 · p. 51

8.1 Recommendations for foundation model developers

  • Direct scoring gives the 10 assessed companies feedback on transparency in their current flagship models and offers guidance relevant to other models and future releases.The index may also inform transparency practices among developers the paper did not assess.Source: §8.1 · p. 51

1. Increase transparency for existing foundation models.

Developers should improve transparency for existing flagship models now, beginning with straightforward disclosures and clearer documentation. They should also state which information they withhold and why.

1. Increase transparency for existing foundation models.

  • Developers should release more information about their current flagship models, since transparency about future releases alone will not address opacity in systems already widely built upon.The passage notes that hundreds of products, services, and other models are built on today’s flagship models.Source: §1 · p. 51
  • Clarify ambiguous documentation, centralize existing information, and share information that poses minimal market-competition or legal risks as low-hanging improvements.Source: §1 · p. 51
  • Explicitly identify the subdomains where information is not released and explain the reasons for withholding it.Source: §1 · p. 51

2. Increase transparency for future foundation model releases.

The paper recommends improving transparency across future releases and the wider foundation-model ecosystem. Its proposals include learning from industry practices, working with other ecosystem actors and adapting disclosure as risks and technologies change.

2. Increase transparency for future foundation model releases.

  • Developers should publicly disclose information covered by the 100 indicators in future releases and consider transparency throughout model development, while weighing disclosure risks indicator by indicator.The authors suggest beginning with maximum transparency and removing items when risks outweigh benefits; independent third-party audits can address some competition concerns but are not always a substitute for public transparency.Source: §2 · pp. 51–52
  • Existing voluntary commitments target specified new, sufficiently powerful releases, while the authors argue developers should assess competitive-advantage claims carefully and consider independent audits.The cited commitments define scope by power thresholds, not all current flagship models.Source: §2 · pp. 51–52

3. Follow industry best practices with respect to transparency.

  • Every developer could improve transparency by drawing on peers’ practices; at least one developer scores on 82 of the 100 indicators.The paper also points to existing developer recommendations on usage guidelines, known weaknesses, vulnerabilities, and safety practices.Source: §3 · p. 53

4. Work with deployers to increase transparency.

  • Developers should work with deployers to obtain usage statistics and disclaimers, and give downstream developers documentation and encouragement to improve transparency for fine-tuned models.If leverage over large deployers is limited, developers may partner with similarly situated developers; otherwise assessing downstream impact may be difficult.Source: §4 · p. 53, §5 · p. 53

6. Work with regulators to increase transparency.

  • When public disclosure is too risky, developers should still share relevant information with regulators so governments can scrutinize them in the public interest.Source: §6 · p. 53

7. Use transparency to improve trust, safety and reliability.

  • Sharing internal practices, documentation, and risk details may bring short-term criticism but can foster more community trust over time; the authors argue transparency is needed for safety and reliability.Source: §7 · p. 53

8. Dedicate resources to continue improving transparency over time.

  • As risks and baselines shift, developers should adapt documentation, use semantic versioning, give practicable deprecation notice, and document model dependencies to support supply-chain monitoring.The paper suggests well-resourced developers dedicate personnel to adapting releases and low-resourced developers seek funding; three months is given as an example of advance notice.Source: §8 · p. 54, §9 · p. 54

10. Use the Foundation Model Transparency Index to increase transparency.

  • Developers should use the Index’s detailed indicators to assess non-flagship models, while deployers should consider the implications of relying on systems with little public information.The Index specifies units and precision for quantitative metrics and often gives instructions for sharing qualitative information.Source: §10 · p. 54, §8.2 · p. 55

1. Assess the risks of deploying a foundation model without adequate transparency.

The authors recommend improving transparency across developer–deployer relationships and policy, while cautioning that disclosure alone does not ensure responsible development. They present the index as a tool for assessing the status quo and informing change, but identify limits in what its scores and coverage establish.

1. Assess the risks of deploying a foundation model without adequate transparency.

  • Deployers should assess opacity-related risks, seek greater developer disclosure, and negotiate public reporting, pooling leverage when needed.Risks include legal liability, reduced user trust, and lower product performance; deployers can also share usage statistics with developers.Source: §1 · p. 55, §2 · p. 55

3. Do not put undue trust in opaque foundation models.

  • Developers’ public-facing opacity should not be taken as evidence that deployers or regulators have all relevant information or that requirements are met.Assuming compliance may create legal exposure for deployers, and developer–deployer relationships do not guarantee information across the identified subdomains.Source: §3 · p. 55

1. Transparency should be a top priority for AI legislation.

  • Policy mechanisms can make disclosure less of a competitive disadvantage, while added information can support accountability, science, innovation, and regulation.The authors also urge policymakers to ensure the public has adequate information about powerful AI systems.Source: §1 · p. 56

3. Policymakers should be realistic about the limits of transparency.

  • Transparency requirements alone are unlikely to reduce harm; policymakers should pair them with substantive measures while weighing risks to open developers.The authors warn that disclosure about worker and environmental impacts may not improve conditions or reduce emissions, and that requirements making open models harder to develop could undermine transparency.Source: §3 · p. 57
  • Open models are often more transparent, and a diverse ecosystem may make it harder for incumbents to coordinate around weak disclosure or capture regulation.More transparent open developers may also prompt closed developers to disclose more to build trust.Source: §3 · pp. 57–58

9.1 Theory of change

  • The index is intended to spur improvement through score comparisons, give company teams actionable indicators, and be updated over time.The authors say future versions are necessary for sustained impact, although they have not set a cadence or strategy.Source: §9.1 · pp. 59–60
  • The index can help policymakers explore disclosure requirements, clarify definitions and edge cases, and target enforcement using evidence about current practice.The authors suggest it could highlight gaps in policy proposals, including upstream labor and downstream impact.Source: §9.1 · p. 60

9.2 Limitations and risks

  • The index measures disclosure rather than responsible conduct, and its binary, sometimes generous scoring can be gamed or encourage performative transparency.The assessment also omits important regions and developers, including East Asian developers and Falcon-180B; assessed models were predominantly text-to-text language models.Source: §9.2 · pp. 60–62

10 CONCLUSION

The conclusion finds widespread but uneven transparency across foundation model developers. The authors argue that meaningful change is needed for foundation models to advance the public interest.

10 CONCLUSION

  • The ecosystem shows a widespread lack of transparency, with substantial differences among developers and areas for improvement.The authors describe this finding as a serious indictment of the overall ecosystem.Source: §10 · p. 63
  • Transparency is a broadly necessary condition for other substantive societal progress, and opaque foundation models are likely to contribute to harm.Source: §10 · p. 63
  • The authors call for real change to address the ecosystem’s fundamental lack of transparency as foundation models are rapidly developed, deployed, and adopted.Source: §10 · p. 63

A AUTHOR CONTRIBUTIONS

This section credits the team’s project roles and sets out transparency criteria across the foundation-model lifecycle, from upstream resources and model properties to access, use, monitoring, impact, and responsible deployment. The indicators define what disclosures and evaluations should be available and how they can be assessed.

A AUTHOR CONTRIBUTIONS

  • Rishi Bommasani led and coordinated the project, Kevin Klyman co-led it, and Percy Liang shaped its initial vision and advised across stages.Betty Xiong, Daniel Zhang, Nestor Maslej, Sayash Kapoor, and Shayne Longpre contributed to design, execution, analysis, and/or presentation.Source: A AUTHOR CONTRIBUTIONS · p. 77

B INDICATORS

  • Data indicators cover dataset size, sources, creators, source-selection protocols, and curation practices.Data size should use appropriate units and be broken down by modality; source disclosures should identify meaningful sources rather than refer only to the Internet or licensed sources.Source: B INDICATORS · pp. 78–79
  • Additional data indicators address augmentation, harmful-content filtering, copyright and license status, personal information, and privacy and copyright mitigations.The criteria allow disclosure that no augmentation or harmful-data filtering is performed, and require meaningful data decomposition for copyright, license, or personal-information disclosures.Source: B INDICATORS · pp. 79, 80, 85
  • Data-labor indicators cover labor phases, employing organizations, worker geography, wages, instructions, and protections.They apply to data created by or on behalf of the developer, including labor involved in annotation or red-teaming.Source: B INDICATORS · pp. 80–81
  • Data-access indicators assess third-party involvement and whether external entities can query or directly access training data.Queryable access may use mechanisms such as an interface or developer-mediated program; direct access means access without developer gating, subject to stated exceptions.Source: B INDICATORS · p. 82
  • Compute indicators cover model-development compute, duration, hardware amount and type, hardware ownership, carbon emissions, and broader environmental effects.Reported quantities should use appropriate units and one-significant-figure precision; disclosures should clarify which development runs or phases they cover.Source: B INDICATORS · pp. 82–84
  • Methods and mitigation indicators assess development stages and objectives, core frameworks, additional dependencies, and disclosed steps addressing privacy and copyright in data.Descriptions of stages may characterize their intent, mechanism, or empirical effects; developers may also state that no additional dependencies or mitigation steps are used.Source: B INDICATORS · pp. 84–85
  • Model-basics indicators address input and output modalities, components, size, architecture, and centralized documentation such as a model card or technical report.Model size is measured in appropriate units, generally parameters, and broken down by named component.Source: B INDICATORS · pp. 86–87
  • Capability and limitation indicators assess whether evaluations are externally reproducible, third parties evaluate capabilities, and multiple model limitations are described or demonstrated.Capability evaluations count as reproducible when data and protocols are public or sufficiently specified; third-party evaluation results must be public and disclosed or referenced by the developer.Source: B INDICATORS · pp. 89–90
  • Risk indicators cover disclosure and demonstration of multiple risks from intentional and malicious use, alongside rigorous evaluation of unintentional and intentional harms.Risk evaluations should be reported before or with initial release; unintentional harms include bias, toxicity, and fairness, while intentional harms include fraud, disinformation, cyberattacks, and illegal uses.Source: B INDICATORS · pp. 90–91
  • Risk-evaluation indicators further assess external reproducibility and whether independent third-party risk evaluations are publicly available and disclosed by developers.Reproducibility requires accessible or reconstructible data and sufficiently described protocols; a justification may suffice when an evaluation cannot be made reproducible.Source: B INDICATORS · pp. 91–92
  • Mitigation indicators assess disclosure and public demonstration of mitigations, rigorous evaluation of multiple mitigations, and whether independent third parties can evaluate them.Mitigation evaluation reproducibility usually requires information about comparison with a baseline, which may be an unmitigated model; reporting no mitigations can satisfy disclosure and demonstration indicators.Source: B INDICATORS · pp. 92–93
  • Trustworthiness and inference indicators assess disclosed evaluations of qualities such as robustness, reliability, uncertainty, calibration, interpretability, and explainability, plus inference duration and compute for specified tasks and hardware.Trustworthiness evaluation reproducibility requires at least one reproducible evaluation; inference duration is reported in seconds and compute in FLOPS, each to one significant figure.Source: B INDICATORS · pp. 93–94
  • Distribution indicators cover release decision protocols and processes, known distribution channels, model-dependent products and services, content-detection mechanisms, and model licenses.Release descriptions may address staged access, and distribution-channel disclosure covers all channels known to the developer.Source: B INDICATORS · pp. 95–96
  • Usage-policy indicators address terms of service, permitted and prohibited users and uses, policy enforcement, justifications for enforcement actions, and appeal mechanisms.Use disclosures meet the relevant indicator when at least two categories—permitted, restricted, or prohibited uses—are described; enforcement disclosure requires more than a general account-termination reservation.Source: B INDICATORS · pp. 96–98
  • Downstream indicators also address permitted, restricted, and prohibited model behaviors, user-facing notices and usage disclaimers, and disclosure of user-data protection and use protocols.Behavior disclosures qualify when at least two behavior categories are described or the developer reports imposing no restrictions; user-data use disclosures cover permitted and prohibited uses across relevant channels.Source: B INDICATORS · pp. 98–99
  • Developers should disclose usage-data access protocols, model versioning, change logs, and deprecation policies.The criteria seek clear descriptions of access steps, model instances and changes, and how users should respond to deprecation.Source: B INDICATORS · p. 100
  • Feedback criteria ask for a mechanism to report issues, a summary of feedback or the developer’s response, and summaries of government inquiries.A meaningful, potentially incomplete summary can qualify for the feedback and government-inquiry criteria.Source: B INDICATORS · p. 101
  • For each distribution channel, developers should disclose a specific model-use monitoring mechanism or state that the channel is not monitored.Merely acknowledging that usage data is collected does not meet the criterion.Source: B INDICATORS · p. 101
  • Developers should disclose downstream application counts, affected market sectors, and the number of individuals affected by the model.The criteria call for meaningful estimates or summaries and clear descriptions of what counts as an application or affected individual.Source: B INDICATORS · p. 102
  • Transparency criteria cover usage reports, geographic usage statistics, and mechanisms for user redress after harm.Reports and geographic statistics may be incomplete or vague; developers also receive credit for stating that no redress mechanism exists.Source: B INDICATORS · pp. 102–103
  • Deployers should have centralized documentation for downstream use and guidance on responsible use, such as mitigation instructions or recommendations.The centralized artifact should contain most substantive information relevant to using or repurposing the model.Source: B INDICATORS · p. 103

C SEARCH PROTOCOL

The section outlines the process used to search for evidence that a developer meets an indicator’s requirements.

C SEARCH PROTOCOL

  • The authors search for evidence that a foundation model developer satisfies each indicator’s requirements.The passage introduces the search process but gives no further procedural details.Source: C SEARCH PROTOCOL · p. 104

C.1.1 Keyword Definitions.

Each reviewed item has associated search keywords recorded in the project’s GitHub repository.

C.1.1 Keyword Definitions.

  • Search keywords for each reviewed item are listed in the project’s GitHub repository.The passage links to the repository but does not describe how keywords are defined.Source: §C.1.1 · p. 104

C.1.2 Model-Item Pair Searches.

The assessment searches developer resources for evidence tied to model-item pairs, records its justification, and uses additional checks when needed.

C.1.2 Model-Item Pair Searches.

  • For each model-item pair, the authors search the centralized resources associated with that model.The passage refers to a list of model-associated resources.Source: §C.1.2 · p. 104

C.1.3 Search Methodology.

  • For each model-item-keyword tuple, the researchers used a Google search format consisting of a site, model name, and keyword, then read through the first 10 search results.Source: §C.1.3 · p. 104

C.1.4 Justification.

  • For each confirmed item, reviewers record the source and justification and preserve an archive.org link to the supporting evidence.Archiving is used to maintain records rather than linking directly to developer pages.Source: §C.1.4 · p. 104

C.1.5 Avoid Search Personalization.

  • All searches are conducted in a private or incognito browser tab to reduce the influence of personalized results.Source: §C.1.5 · p. 104

C.1.6 Determination Criteria.

  • Reviewers stop searching when one piece of evidence fully justifies either 1 point or, more rarely, 0 points for an item.Source: §C.1.6 · p. 104

C.1.7 Distribution Channels.

  • When ordinary searches find no information about distribution-channel indicators, reviewers may inspect the disclosed API or related documentation, using proxies if access requires an enterprise account.Source: §C.1.7 · p. 104

C.2 Developer website

  • The assessment covers 10 developers and their flagship models, including OpenAI’s GPT-4, Google’s PaLM 2, and Meta’s Llama 2.The listed developers also include AI21 Labs, Amazon, Anthropic, Cohere, Hugging Face, Inflection, and StabilityAI.Source: §C.2 · p. 105

D CALLS FOR TRANSPARENCY

Governments, international organizations, developers, researchers, workers, and civil society groups call for greater transparency in foundation model development and use. The paper frames transparency as a necessary first step toward accountability for AI harms.

D CALLS FOR TRANSPARENCY

  • Transparency has become a concern across governments, companies, researchers, workers, and civil society, with less powerful actors also pressing for changes to business practices.Source: D CALLS FOR TRANSPARENCY · pp. 108, 113

Calls for transparency from governments

  • U.S. initiatives propose transparency reporting and disclosure of training data, limitations, accuracy, safety, and user interaction; EU drafts and Chinese rules also set disclosure or complaint-handling requirements. 69, 71, 72, 73, 74, 75, 76The cited examples include U.S. voluntary commitments and draft legislation, EU obligations for foundation models, and Chinese requirements for complaint portals.Source: D CALLS FOR TRANSPARENCY · pp. 108–109
  • Government guidance and laws address system capabilities, training data, risk mitigation, consumer information, and transparency in data privacy and AI use. 77, 79, 80, 82, 83The passages describe examples from Canada, India, the UAE, and the UK and U.S. consumer-protection authorities.Source: D CALLS FOR TRANSPARENCY · pp. 109–110

Calls for transparency from international organizations

  • International bodies call for human-rights impact communication and grievance transparency, AI risk reporting, and transparency as a principle of trustworthy AI. 89, 90, 91, 92, 93, 94The examples include the UN, G7, and OECD; the G20 adopted the OECD transparency principle. 95Source: D CALLS FOR TRANSPARENCY · pp. 110–111

Calls for transparency from foundation model developers

  • Developers have advocated publishing usage guidelines and documenting model weaknesses, safety practices, risks, and development choices. (Bai et al., 2022) 97, 99, 100, 101, 102, 103Stability AI links open models to researcher scrutiny and risk management. 103Source: D CALLS FOR TRANSPARENCY · pp. 111–112
  • Developer transparency efforts also include reporting on data labor, privacy, datasets, models, and system-level choices. 105, 106, 107, 108; Phang et al. (2022)The examples include Meta’s Llama 2 guide, Amazon’s privacy reporting, Google’s data and model cards, and Microsoft’s Transparency Notes.Source: D CALLS FOR TRANSPARENCY · pp. 112–113

Calls for transparency from researchers, civil society, and labor

  • Researchers, data workers, creators, and civil society seek transparency for auditing, worker conditions, impact assessments, environmental effects, and identifying censorship pressures. (Mitchell et al., 2018; Gebru et al., 2018) 110, 111, 112, 113, 114, 115The African Content Moderators Union’s petition alleges inadequate information and psychosocial support; Freedom House points to transparency as a way to identify censorship pressures.Source: D CALLS FOR TRANSPARENCY · pp. 113–115
  • The paper describes transparency, despite its limits, as a necessary and broadly popular first step toward accountability for harm caused by AI systems.Source: D CALLS FOR TRANSPARENCY · p. 115

Related papers

Loading 2310.12941v1…