Source-linked AI summary
Practical Principles for AI Cost and Compute Accounting
Stephen Casper, Luke Bailey, Tim Schreier
TL;DR
Policymakers use development cost and compute as proxies for AI capabilities and risks, but ambiguous accounting can undermine threshold-based oversight. The paper proposes seven principles covering what to count, what to exclude, and how to estimate costs and compute, with the aim of reducing gaming, preserving risk mitigation, and supporting consistent implementation. The framework does not specify regulatory thresholds or how laws should apply them.
Problem
Threshold-based AI governance relies on development cost and compute, but unresolved accounting ambiguities can undermine practical and consistent implementation.
Source: §1 · p. 1Method
The paper proposes seven principles for counting upstream technical costs and compute, excluding qualifying activities or open resources, and permitting reasonable estimates.
Source: §1 · p. 1, §3.1 · p. 2, §3.2 · p. 3Results
Takeaways & Limitations
Cost and compute accounting standards should be practical, clear, consistent, and aligned with the public interest.
Source: §4 · p. 5The paper does not recommend specific thresholds, regulatory triggers, or how accounting standards should be incorporated into legal frameworks.
Source: §4 · p. 5
Abstract
from arXiv · showhide
Policymakers increasingly use development cost and compute as proxies for AI capabilities and risks. Recent laws have introduced regulatory requirements for models or developers that are contingent on specific thresholds. However, technical ambiguities in how to perform this accounting create loopholes that can undermine regulatory effectiveness. We propose seven principles for designing AI cost and compute accounting standards that (1) reduce opportunities for strategic gaming, (2) avoid disincentivizing responsible risk mitigation, and (3) enable consistent implementation across companies and jurisdictions.
1. Introduction
Policymakers are increasingly using development cost and compute thresholds to identify AI systems warranting heightened oversight, but these metrics require clear accounting standards to be practical. The paper proposes seven principles to make accounting less gameable, compatible with responsible risk management, and consistent across jurisdictions.
- Cost and compute thresholds are increasingly used in AI governance to identify systems warranting heightened oversight.Recent laws and governance frameworks make requirements contingent on development costs and computational resources.Source: §1 · p. 1
- Their regulatory practicality depends on resolving ambiguities about what development activities and resources should be counted.Without clear accounting standards, thresholds may be difficult to implement consistently.Source: §1 · p. 1
- The paper asks how development cost and compute can be counted practically while limiting gaming and avoiding disincentives for responsible risk management.The challenges include deciding which activities to count, establishing reporting requirements, and adapting standards as technology evolves.Source: §1 · p. 1
- The authors propose seven principles intended to resolve accounting ambiguities, align standards with the public interest, and support consistent implementation across companies and jurisdictions.The framework does not take positions on specific regulatory thresholds or requirements.Source: §1 · p. 1
2. Background
Prior work establishes the relevance of cost and compute to AI governance but leaves important accounting questions unresolved. The background also defines the entities and processes covered, while noting that closely related models may create practical regulatory complications.
Related work
- Scaling-law research supports the theoretical relevance of computational resources to AI regulation, but standardized accounting methods remain unresolved.Open questions include which activities to count and how developers should report them.Source: §2 · p. 1
- Existing guidance supports excluding freely obtained open resources, counting data curation and teacher models, and using reasonable estimates, but the paper identifies gaming risks in narrower practices.The authors criticize context-dependent calculations, end-to-end-only counts, and exclusions for recomputations or discarded branches.Source: §2 · pp. 1–2
Key terms
- Developer means the entity undertaking AI-model creation, including multiple legal entities working under a formal partnership.This definition is intended to prevent loopholes involving formally collaborating entities.Source: §2 · p. 2
- Development covers data curation, model training, scaffolding, and system testing, but excludes human labor, operations, and hardware procurement.Source: §2 · p. 2
- An AI model is a machine-learning-trained neural information-processing structure, whereas an AI system combines one or more models with other software components.GPT-4o is given as an AI-model example, while ChatGPT-GPT-4o is given as an AI-system example.Source: §2 · p. 2
Related problem
- Closely related models may have similar but not always identical behavior, making redundant requirements and the choice between regulating models or model families difficult.Practical recommendations for handling these cases are outside the paper’s scope.Source: §2 · p. 2
3. Principles
The paper proposes seven principles for accounting AI development costs and compute, aiming to close loopholes, preserve incentives for risk reduction, and support practical oversight. The principles cover upstream development activities, exemptions, estimation, reporting, thresholds, and updates.
3.1. Count all of a project’s expended costs and compute upstream of the final system
- Count all technical costs and compute expended upstream of the final system, including integral but non-proximal activities such as data curation and teacher-model training.This approach targets narrow-accounting loopholes, including distillation, while recognizing that attributing upstream research can involve genuine grey areas.Source: §3.1 · pp. 2–3
3.2. Exclude costs and compute behind pre-existing, openly-available resources
- Exclude costs and compute behind pre-existing, openly available resources unless the developer recently released partially developed components that could otherwise evade accounting.The paper leaves handling further development of already-threshold-crossing open models to regulators, outside its scope.Source: §3.2 · pp. 3–4
3.3. Exclude activities undertaken only to reduce societal risks
- Exempt activities undertaken only to reduce societal risks when they do not enhance capabilities, supported by rigorous, auditable justifications to address safetywashing concerns.Examples include filtering CSAM, refusal fine-tuning, and national-security-risk testing.Source: §3.3 · p. 4
3.4. Allow for reasonable estimates
- Permit reasonable estimates when precise cost or compute information is impractical to obtain, with documented methods, justifications, and regulator guidance on methodologies and error margins.The paper gives externally queried closed-source systems as an example where market value and contextual knowledge can support estimation.Source: §3.4 · p. 4
3.5. Require itemized accounting reports
- Require auditable, itemized accounting reports that explain activities, calculations, final amounts, open-resource reliance, risk-reduction exemptions, and estimates.Such reports support oversight, due diligence, and review of technical exemptions.Source: §3.5 · pp. 4–5
3.6. Use independent thresholds for costs and compute
- Trigger regulatory requirements independently through separate cost and compute thresholds because developers can decouple these measures and each can provide a check when the other is uncertain.Machine-generated data can be cheap but compute-intensive, while human-generated data can be expensive but computationally free.Source: §3.6 · p. 5
3.7. Require regular updates to thresholds and standards
- Regularly update thresholds and accounting standards because rapid advances can change scaling trends, efficiency gains, and developers’ cost and compute requirements.The paper suggests quarterly or semiannual review by government offices or standards bodies.Source: §3.7 · p. 5
4. Discussion
The paper presents seven principles for clear, consistent cost and compute accounting standards that reduce gameability and avoid discouraging societal risk mitigation. It also identifies regulatory burden, information-security risks, unresolved grey areas, and the need for future concrete standards.
4. Discussion
- Seven principles are proposed to make AI cost and compute accounting clear, consistent, publicly aligned, and less vulnerable to strategic gaming.The framework also aims to avoid disincentivizing societal risk mitigation and support implementation across companies and jurisdictions.Source: §4 · p. 5
- Cost and compute thresholds are practical but technically challenging tools for regulating frontier AI models, making accounting standards central to their viability.Source: §4 · p. 5
- Rigorous accounting can impose regulatory burdens and create information-security risks when sensitive information is shared with regulators.Regulators should weigh these drawbacks against the benefits of accounting standards.Source: §4 · p. 5
- The principles do not specify regulatory thresholds, triggering requirements, or legal integration, and grey areas will remain inevitable.The paper recommends itemized reporting, continuous awareness, and evolving oversight rather than a single fixed regime.Source: §4 · p. 5
- Future work must translate the principles into concrete standards tailored to each law's purpose and scope, with possible logistical or political compromises.Source: §4 · p. 5