Source-linked AI summary
Towards Publicly Accountable Frontier LLMs: Building an External Scrutiny Ecosystem under the ASPIRE Framework
Markus Anderljung, Everett Thornton Smith, Joe O'Brien, Lisa Soder, Benjamin Bucknall, Emma Bluemke, Jonas Schuett, Robert Trager, Lacey Strahm, Rumman Chowdhury
TL;DR
Frontier LLM decisions carry significant risks, yet stakeholders outside developers need reliable information to guide them. The paper surveys external scrutiny and organizes six requirements for it in the ASPIRE framework. It concludes that scrutiny’s effectiveness is not assured and depends on how it is designed and implemented.
Problem
Method
Results
Takeaways & Limitations
External scrutiny can help verify developer claims and uncover issues by bringing wider expertise, perspectives, and motivations to risk identification.
Source: §3 · p. 2Wider access can increase the risk of unintentional, irreversible model proliferation, so access must be balanced with information security.
Source: §5.1 · p. 3
Abstract
from arXiv · showhide
With the increasing integration of frontier large language models (LLMs) into society and the economy, decisions related to their training, deployment, and use have far-reaching implications. These decisions should not be left solely in the hands of frontier LLM developers. LLM users, civil society and policymakers need trustworthy sources of information to steer such decisions for the better. Involving outside actors in the evaluation of these systems - what we term 'external scrutiny' - via red-teaming, auditing, and external researcher access, offers a solution. Though there are encouraging signs of increasing external scrutiny of frontier LLMs, its success is not assured. In this paper, we survey six requirements for effective external scrutiny of frontier AI systems and organize them under the ASPIRE framework: Access, Searching attitude, Proportionality to the risks, Independence, Resources, and Expertise. We then illustrate how external scrutiny might function throughout the AI lifecycle and offer recommendations to policymakers.
1 Risks from Frontier LLMs
Frontier LLMs pose significant present and future risks, but those harms are not inevitable: they depend on decisions about model development, deployment, and use.
1 Risks from Frontier LLMs
- Frontier LLMs pose significant risks of harm both now and in the future.The risks include discrimination, disinformation and election interference, surveillance, cyberattacks, and proliferation of weapons of mass destruction, especially biological weapons.Source: §1 · p. 1
- Potential harms include discrimination, disinformation and election interference, surveillance, cyberattacks, and weapons proliferation.The passage particularly highlights biological weapons.Source: §1 · p. 1
- These risks are not inevitable; they depend on decisions about model development, deployment, and use.Source: §1 · p. 1
2 AI Governance as an Information Problem
Good AI governance is partly an information problem: stakeholders outside developers need reliable information about frontier LLMs to make informed decisions and design well-targeted policies.
2 AI Governance as an Information Problem
- Reliable information about frontier LLMs is needed by stakeholders outside AI developers to make informed decisions and design well-targeted policies.Source: §2 · p. 1
- Users can use better understanding to choose systems and press developers to act responsibly, while civil society can research and advocate for harm-reducing policies and standards.Source: §2 · p. 2
- Policymakers can use better understanding of AI’s present and future risks and impacts to legislate and regulate more effectively.Source: §2 · p. 2
- Reliable information is close to a necessary condition for good governance, but is far from sufficient on its own.Source: §2 · p. 2
3 Sourcing Reliable Information
Because developer-provided information cannot be the sole basis for policy decisions, the paper calls for external actors to help generate and distribute information about frontier LLMs.
3 Sourcing Reliable Information
- AI developers are primarily responsible for system safety, but policy decisions must not rest solely on information they provide.Source: §3 · p. 2
- External scrutiny involves actors outside AI developers conducting audits, evaluations, red-teaming, or other research on AI systems.The paper describes this involvement as a way to democratize control over frontier LLM impacts.Source: §3 · p. 2
- Good governance needs information about model capabilities, controllability, and impacts, including what tasks models perform, whether they follow intentions, and their effects on the world.Source: §3 · p. 2
- External scrutiny can check developer claims and uncover issues developers miss by bringing wider expertise, perspectives, and motivations to risk identification.The passage links verification to reducing information asymmetries and discovery to the broad space of model behaviors and uses and the difficulty of understanding LLM internals.Source: §3 · p. 2
4 Calls for External Scrutiny
Governments, companies, and policymakers have begun supporting external scrutiny, but the ecosystem remains nascent and its effectiveness depends on specific choices.
4 Calls for External Scrutiny
- The United Kingdom established a Frontier AI Task Force with £100 million to ensure AI risk assessments are conducted by neutral third parties.Source: §4 · p. 2
- External scrutiny efforts include U.S. voluntary commitments for red-teaming, congressional proposals, and an EU parliamentary proposal for independent experts in foundation-model design and testing.AI developers have also begun voluntarily inviting external parties to evaluate their systems.Source: §4 · p. 3
- The external scrutiny ecosystem is nascent, and its effectiveness depends on specific decisions.Source: §4 · p. 3
5 Requirements for Effective External Scrutiny
The paper proposes ASPIRE as six requirements for effective external scrutiny, addressing failure modes and the practical conditions needed for scrutiny across frontier LLMs.
5 Requirements for Effective External Scrutiny
- Enron’s failed audit illustrates how dependence, incentives, formulaic standards, and weak expertise or resources can undermine scrutiny; ASPIRE specifies six requirements.Poorly designed scrutiny can also fail to provide risk-reducing information or create false security.Source: §5 · p. 3
5.1 Access
- Scrutinizers need access to models, development and use information, third-party impact data, and fine-tuning, balanced against proliferation and information-security risks [36].Structured access uses controlled, arm’s-length cloud interactions.Source: §5.1 · p. 3
5.2 Searching Attitude
- Because formulaic tests can miss risks, become obsolete, or be gamed, scrutinizers need incentives, protections, and exploratory tests for unknown capabilities [38] [39] [41].The paper frames scrutiny more as scientific discovery than checking systems against best practices [8].Source: §5.2 · pp. 3–4
5.3 Proportionality to risk
- Scrutiny should scale with risk proxies spanning capabilities, novelty, accident and misuse potential, application domain, and the number of people likely to be affected.Because true capabilities are unknown before evaluation, regulators can begin with predictions from prior models and adjust scrutiny as capabilities emerge.Source: §5.3 · p. 4
5.4 Independence
- Developers should relinquish control over scrutinizer selection and compensation, scope and methods, access, and post-scrutiny reporting to protect independence.Developer control over access can amount to a de-facto veto over scrutiny.Source: §5.4 · p. 4
5.5 Resources
- Effective scrutiny needs adequate time, competitive compensation, and computing resources such as API credits.Evaluation time should increase as model risks or evaluation complexity increase.Source: §5.5 · p. 5
5.6 Expertise
- No single team can cover frontier LLMs’ wide-ranging expertise needs, so scrutiny should incorporate diverse perspectives and specialized knowledge.Biosecurity assessment may require expertise in countering biological weapons proliferation and understanding pathogen harms, expertise that can be difficult to hire [6].Source: §5.6 · p. 5
6 External Scrutiny in the AI Lifecycle
External scrutiny should span development, pre-deployment, and post-deployment, with its findings informing decisions at each stage.
6.1 Development
- Scrutiny findings should inform whether to train a model and what information-security measures are warranted; earlier identification of safety issues may make them less costly to fix.Models with significant misuse potential may need high information-security standards because they could be stolen before deployment.Source: §6.1 · p. 5
- During training, scrutinizers can assess capabilities and controllability through similar smaller models or training data, and evaluate whether safety measures can keep pace [20].These evaluations can inform whether to continue or adjust development.Source: §6.1 · pp. 5–6
6.2 Pre-deployment
- Before deployment, scrutiny can simulate malicious use, stress-test safeguards, and help developers report information downstream users need to judge suitability and prepare for harms.Source: §6.2 · p. 6
6.3 Post-deployment
- After deployment, scrutiny can track changing capabilities, controllability, and impacts, informing safeguard updates or access restrictions such as recalling an API model.Users have historically discovered capabilities after deployment, including through prompt engineering or tool access [20].Source: §6.3 · p. 6
Appendix I: Policy Recommendations
Building an external scrutiny ecosystem for meaningful public accountability will be difficult and take time, but policymakers can consider concrete actions now.
Appendix I: Policy Recommendations
- Building an external scrutiny ecosystem that contributes meaningfully to public accountability will be difficult and lengthy.Source: Appendix I · p. 7
- Policymakers can take concrete actions across the ASPIRE framework now.Source: Appendix I · p. 7
- Those actions can inform the design of new standards, regulations, or legislation.Source: Appendix I · p. 7
Access
The recommendations call for qualified, secure access and infrastructure that supports outside research and model scrutiny.
Access
- Mandate sufficient, secure model access for qualified external scrutinizers, using robust criteria such as accreditation or vetting.Source: Access · p. 7
- Develop a research API and provide access to third-party data, potentially through companies, government, or a public-private partnership.Source: Access · p. 7
- Support structured-access tools and privacy-enhancing technologies.Source: Access · p. 7
Searching attitude
Effective external scrutiny requires conditions that let scrutinizers explore model behavior, expose flaws, and adapt evaluations to changing practices.
Searching attitude
- Protect scrutinizers from undue liability or retaliation, such as through safe harbors, to avoid discouraging exploratory scrutiny.Fear of repercussions for honest mistakes may bias scrutiny toward check-list, compliance-focused approaches.Source: Searching attitude · p. 7
- Engage multiple independent teams in adversarial audits and ideally reward teams that successfully expose flaws.Source: Searching attitude · p. 7
- Set standards around desirable and undesirable model features rather than prescribed methods, allowing scrutiny to adapt while remaining targeted.Source: Searching attitude · p. 7
Proportionality
The intensity of mandated external scrutiny should reflect the risks posed by a model, with risk-based classifications as one possible guide.
Proportionality
- Scale mandated external scrutiny in proportion to the risks a model poses.Source: Proportionality · p. 7
- A classification scheme analogous to biosafety levels could vary scrutiny and regulatory oversight according to risk.Source: Proportionality · p. 7
- The proposed scheme would link higher levels of model risk to varying degrees of scrutiny and regulatory oversight.Source: Proportionality · p. 7
Independence
The paper calls for independent, adequately resourced and expert scrutiny, supported by access suited to evaluation needs. It presents external scrutiny as a route to broader stakeholder involvement while noting that its recommendations remain high-level and that scrutiny cannot replace developer responsibility.
Independence
- Set independence standards covering selection, compensation, scope, methods, access, and post-scrutiny actions, while reducing administrative burdens and establishing oversight.Suggested measures include standardized contracts, limits on unreasonable NDA requirements, and conflict-of-interest standards.Source: Independence · p. 7
Resources
- Provide scrutinizers enough financial and computational resources and time; current frontier LLMs should receive at least six months of scrutiny before deployment.Scrutiny time should increase as potential risks increase.Source: Resources · p. 7
Expertise
- Fund research programs to build evaluation capacity outside developers and provide government expertise where scrutiny is particularly necessary, including national-security assessments.Source: Expertise · p. 7
Appendix II: Access Requirements
- Access requirements include systematic model sampling, fine-tuning, base models, model families and previous versions, and potentially model internals.The paper notes current internals-based evaluation approaches are in their infancy; base-model access can be provided through an API without model weights.Source: Appendix II: Access Requirements · p. 8
- Scrutiny may also require training-data sources and metadata, deployed-system components, third-party impact data, and model information, while minimizing sensitive-information or model leakage.Training-data access itself may not be necessary if sources, cleaning decisions, and composition metadata are available.Source: Appendix II: Access Requirements · p. 8
Appendix III: Social Impacts Statement
- The paper aims to inform policy discussions and provide a framework for design choices, with external scrutiny potentially paving the way for broader stakeholder involvement in frontier-LLM governance.Source: Appendix III: Social Impacts Statement · p. 9
- The framework is a high-level design tool; granular implementation choices depend on the situation and political context, making the recommendations less concrete.Source: Appendix III: Social Impacts Statement · p. 9
- External scrutiny is unlikely to identify every important AI risk, may create false security, and matters only insofar as it informs and changes decisions; developers remain ultimately responsible for system safety.It is one tool in the AI governance toolbox, and effective influence may require regulators and the public to have more levers over developers.Source: Appendix III: Social Impacts Statement · p. 9