PaperlayerView exact revision on arXiv

Source-linked AI summary

The Open-Weight Paradox: Why Restricting Access to AI Models May Undermine the Safety It Seeks to Protect

Vinicius Santana Gomes

arXiv:2604.17413v1cs.CYcs.AI

TL;DR

Open-weight AI governance is often framed as a choice between openness and safety, but restrictions without governed alternatives may displace risks. This paper examines that tension through infrastructure and risk evidence and argues for a layered governance approach. It concludes that restriction alone can shift activity toward less visible and less governable settings.

Abstract

from arXiv · show

The governance of open-weight artificial intelligence (AI) models has been framed as a binary choice: openness as risk, restriction as safety. This paper challenges that framing, arguing that access restrictions, without governed alternatives, may displace risks rather than reduce them. The global concentration of compute infrastructure makes open-weight models one of the most viable pathways to sovereign AI capacity in the Global South; restricting such access deepens asymmetries while driving proliferation into unsupervised settings. This analysis proposes that hardware-layer governance, including chip-level attestation mechanisms such as FlexHEG, trusted execution environments, confidential computing, and complementary software-layer safeguards, offers a defense-in-depth alternative to the current binary. A threat model taxonomy mapping misuse vectors to hardware, software, institutional, and liability layers illustrates why no single governance mechanism suffices. To operationalize this approach, the paper argues that effective AI governance as a dual-use technology will likely require a multilateral institutional architecture functionally analogous, though not identical, to the role performed by the IAEA in the nuclear domain, with explicit safeguards against the co-option of hardware controls for domestic repression. The relevant policy question is how to make openness safer through technical and institutional design while addressing the transition realities of legacy hardware, attestation at scale, and civil liberties protection.

I. Introduction

The paper challenges the binary framing of open-weight models as dangerous and restricted access as safe. It argues that restricting access without governed alternatives may deepen dependency and opacity, and that the policy task is to build verifiable safeguards for models that can already diffuse across borders.

I. Introduction

  • The debate’s open-equals-dangerous, closed-equals-safe binary does not adequately describe the risks of open-weight AI.The paper recognizes merit in both restriction and openness arguments but finds neither sufficient to explain the risks.Source: §I · p. 1
  • Open-weight AI refers to trained weights available for third-party download, execution, and adaptation, enabling local use without ongoing dependence on a centralized provider.The paper distinguishes this from fully open-source software and remote API access.Source: §I · p. 1
  • Restrictions without governed alternatives may amplify structural dependency and opacity without guaranteeing greater safety.The paper focuses on risks arising when advanced AI pathways run through a small number of cloud providers controlled by a few nations and corporations.Source: §I · p. 1
  • DeepSeek R1’s release under severe hardware export restrictions illustrates that access restrictions alone may not prevent highly capable models from emerging.The model claimed to rival leading proprietary systems at a fraction of the cost.Source: §I · p. 1
  • Open-weight models can diffuse AI capability to actors excluded from the frontier compute ecosystem.Quansah (2025) observed that DeepSeek R1 weakened the assumption that advanced AI would remain accessible only to wealthy nations and large multinational corporations.Source: §I · p. 2
  • Because weights are already partly available and digitally copiable across borders, the paper frames the policy question as building technical and institutional safeguards rather than whether to permit them.The analysis examines infrastructure, fine-tuning vulnerabilities, and institutional governance.Source: §I · p. 2

II. Infrastructure, Dependency, and Risk Displacement

Compute infrastructure is concentrated, leaving many countries dependent on foreign providers or limited local hardware. The paper argues that restricting open-weight access without governed alternatives can displace activity into less visible settings, even as open weights offer one route to locally adapted AI capacity.

II. Infrastructure, Dependency, and Risk Displacement

  • Without governed alternatives, restrictions can drive Shadow AI proliferation, deepen dependence on foreign cloud infrastructure, and shift demand to providers outside Western regulatory frameworks.The paper says activity may then move from partially auditable environments to settings with minimal oversight.Source: §II · p. 2
  • Evidence of displacement includes chip-smuggling and cloud-compute circumvention, a Hugging Face ecosystem exceeding two million public models, and 69% of surveyed organizations suspecting or finding prohibited public generative AI use.The Gartner survey covered 302 cybersecurity leaders in March–May 2025; its projection of incidents in more than 40% of enterprises by 2030 is a forecast, not an observed outcome.Source: §II · pp. 2–3
  • The United States hosts roughly three-quarters of global GPU cluster performance, while China ranks second at approximately 15%.Only 39 countries host public cloud regions, of which 30 offer GPUs; China and the United States together host 49 GPU-enabled regions, compared with 52 across the rest of the world combined (Lehdonvirta, Wú, and Hawkins, 2024; Pilz et al., 2025).Source: §II · p. 3
  • Open-weight models let organizations without GPUs download smaller models and adapt them locally, partially bypassing an otherwise substantial hardware barrier.The paper identifies this as a primary mechanism for expanding AI capacity accessible to the Global South.Source: §II · pp. 3–4
  • Falling inference costs do not establish sovereign training capacity: inference on trained models has become cheaper, while advanced training infrastructure remains concentrated.The paper notes that cheap API inference can shift dependency from hardware access to foreign platforms routing national data.Source: §II · pp. 4–5
  • Inference-cost declines vary by benchmark and performance milestone, but the paper says they concern already-trained models rather than the training capacity needed for sovereignty.Epoch AI reports declines ranging from 9-fold to 900-fold per year, with mid-range benchmarks declining approximately 40-fold per year.Source: §II · pp. 4–5
  • Smaller locally executable models may fall below EU AI Act thresholds, while the safety-relevant modifications can require negligible computing resources.The paper says training-scale criteria leave less visible adaptation, redistribution, and decentralized deployment below those thresholds.Source: §II · p. 5
  • Compute-anchored regulation has fewer levers over downloadable models adapted and deployed on consumer hardware, though some uses may still be covered by rules for the final system and its context.The paper also notes that in most nations the choice is typically limited models on conventional servers or routing data to external providers via API, with data-sovereignty implications.Source: §II · pp. 2, 6

III. The Fine-Tuning Vulnerability: Why Restriction Misses the Target

Fine-tuning can remove safety alignment with little effort, and the vulnerability applies to open-weight and fine-tunable closed API models alike. The paper argues that restricting weight distribution therefore misses the underlying mechanism and proposes coordinated hardware, software, institutional, and liability safeguards.

III. The Fine-Tuning Vulnerability: Why Restriction Misses the Target

  • Safety alignment can be compromised with as few as ten adversarial examples for less than $0.20, and safeguards can be removed with a few dozen examples in minutes.The findings are attributed to Qi et al. (2024) and the UK AI Security Institute (2025), respectively.Source: §III · p. 6
  • Fine-tuning attacks affect both open-weight models and closed API models when fine-tuning access is provided, while guardrail-removing modifications may fall below significant-modification thresholds.Qi et al. (2024) demonstrated the attack on Llama-2 and GPT-3.5 Turbo; the paper describes this as a regulatory blind spot.Source: §III · p. 6
  • Restricting open-weight distribution addresses one vector but leaves the fine-tuning mechanism itself, and its technical vulnerability, intact.The paper locates the governance mismatch between the object being controlled and the technical layer where control would need to operate.Source: §III · p. 6
  • The paper proposes shifting intervention toward compute infrastructure when reconfiguring and running models outside verifiable oversight is the decisive vulnerability.This reframes the intervention target from software circulation to the infrastructure underpinning training and execution.Source: §III · p. 6
  • Table 1 maps five principal open-weight threats to governance layers, and the mapping shows that no single layer addresses every threat.Hardware mechanisms address execution-level risks but do not create institutional accountability; liability creates incentives for commercial actors but leaves noncommercial misuse unaddressed.Source: §III · pp. 6–7, Table 1 · p. 7
  • Software safeguards—including pre-release evaluations, red-teaming, staged deployment, watermarking, and provenance tracking—complement the architecture but do not suffice alone.Model-level guardrails can be removed through low-resource fine-tuning, and sophisticated attackers can circumvent software-level defenses (Bengio et al., 2025b; UK AISI, 2025).Source: §III · pp. 7–8

A. The Case for Hardware-Layer Governance

Hardware-layer governance shifts oversight from controlling model distribution to verifying the conditions of compute execution. FlexHEG could provide privacy-preserving, programmable chip safeguards, but it is one layer in a broader architecture and faces transition and industry-dependence challenges.

A. The Case for Hardware-Layer Governance

  • FlexHEG combines a tamper-proof chip enclosure and guarantee processor to enable privacy-preserving verification and enforcement of chip-use compliance.Its design supports programmable, updatable rules and aggregated verification that can report abstract claims, such as whether training used less than a specified FLOP threshold, without revealing technical details.Source: A. The Case for Hardware-Layer… · p. 8
  • Full FlexHEG mechanisms primarily target new chips, leaving legacy hardware with limited or no guarantees during a transition period.The frontier chip supply chain is concentrated in Nvidia’s design and TSMC’s fabrication, and early versions may trade security against rule sophistication.Source: A. The Case for Hardware-Layer… · p. 8
  • The paper frames hardware governance as applying certification, monitoring, investigation, and standards updates to compute rather than relying on prohibition or unrestricted release.It uses aviation as a functional analogy for embedding safety in hardware.Source: A. The Case for Hardware-Layer… · p. 8
  • Safety-critical governance layers safeguards so failure of any single mechanism does not produce catastrophic outcomes.The paper cites lifecycle risk management in FDA software-as-a-medical-device regulation as an example.Source: A. The Case for Hardware-Layer… · p. 8
  • FlexHEG is one layer in a defense-in-depth architecture: hardware verification addresses compute, software safeguards address model behavior, and institutional oversight addresses accountability and legitimacy.The paper argues no single layer suffices, and that other layers lack a verifiable foundation without hardware-level verification.Source: A. The Case for Hardware-Layer… · p. 9

B. The Broader Landscape: TEEs, Confidential Computing, and Verifiable Training

The paper situates FlexHEG among existing hardware, enclave, and cryptographic mechanisms rather than treating it as a standalone solution. These approaches support different governance functions, but CPU-bound confidential computing limits current reach to smaller models and distilled workloads.

B. The Broader Landscape: TEEs, Confidential Computing, and Verifiable Training

  • Hardware governance spans enforceable mechanisms including trusted execution environments, confidential computing, and cryptographic verification; FlexHEG is one point on that spectrum.The paper maps hardware attestation and TEE controls to infrastructure, coordination standards to the protocol layer, and safety evaluations and release norms to the application layer.Source: B. The Broader Landscape: TEEs,… · pp. 9, 11
  • TALOS monitors enclave control-flow during migration and enforces one-to-one mappings between application instances and system identifiers to prevent replication and fork-based cloning.It has been prototyped on Intel SGX and RISC-V Keystone and is designed to be TEE-architecture agnostic.Source: B. The Broader Landscape: TEEs,… · p. 9
  • Keyfort adds secure timekeeping, state continuity, updates, and migration to RISC-V Keystone TEEs, with 10 ms update downtime and 0.7 s migration time for enclaves with up to 16 KB of state.The added mechanisms include trusted time and anti-rollback protection.Source: B. The Broader Landscape: TEEs,… · p. 9
  • DistilLock protects model weights and user data during on-device distillation, while TBNet uses one-way REE-to-TEE connections and reduces TEE memory use by up to 2.45× versus full in-enclave execution.Both are presented as weight-protection approaches available on current hardware without purpose-built governance chips.Source: B. The Broader Landscape: TEEs,… · pp. 9–10
  • DeepSeek-R1-1.5B achieved 25.67 tokens/s under TDX versus 10.25 tokens/s on CPU alone, but the average GPU-to-CPU performance ratio is 12 across model sizes.Current CPU-based TEEs limit support for large LLMs, and CPU-host/GPU communication currently occurs in plaintext; the paper characterizes TEE governance as viable now for smaller and distilled models.Source: B. The Broader Landscape: TEEs,… · p. 10
  • Zero-knowledge proofs can verify computational claims without revealing model internals, while Laminator proposes hardware-assisted attestations of ML properties such as accuracy and fairness.The paper also identifies compute reporting, monitoring, subsidized access, and signed external certificates as institutional or cryptographic complements to hardware attestation.Source: B. The Broader Landscape: TEEs,… · p. 10

C. Implementation Challenges and Transition Realities

Implementation must address legacy hardware, offline deployments, and the choice of initial rules. The proposed starting rules prioritize verifiable records and instance integrity while allowing capability restrictions, rather than immediate shutdown, when compliance check-ins lapse.

C. Implementation Challenges and Transition Realities

  • The paper identifies three implementation problems: transition from ungoverned legacy hardware, offline or air-gapped deployments, and defining a minimal near-term rule set.Source: C. Implementation Challenges and… · p. 11
  • Legacy chips will remain operational during a substantial transition, leaving some compute outside hardware governance, while new governed accelerators diffuse through the installed base.The paper argues that universal coverage is not required for effectiveness, though smuggling and diversion remain residual risks that may be mitigated through enforcement, tracking, and inspections.Source: C. Implementation Challenges and… · p. 11
  • Offline deployments can use locally verified operating licenses, periodic renewal, secure logging where implemented, and capability throttling or shut-off after license expiration.Periodic reauthorization and inspections trade some real-time enforcement capacity for sovereignty and operational flexibility.Source: C. Implementation Challenges and… · p. 11
  • Overly specific rules risk false positives and compliance costs that push users toward ungoverned alternatives, while underspecified rules risk irrelevance.The paper proposes three minimal rules for FlexHEG-capable and TEE-attested hardware.Source: C. Implementation Challenges and… · pp. 11–12
  • The proposed rules require signed training-run records above a compute threshold, bind higher-compute models to attested deployment instances, and require periodic compliance check-ins.Missed check-ins trigger capability restrictions rather than immediate shutdown; the rules establish visibility and single-instance integrity without restricting what can be trained or how models are deployed.Source: C. Implementation Challenges and… · p. 12
  • Hardware governance is not invulnerable; the paper frames its goal as raising evasion costs and complexity compared with having no hardware-layer governance.Source: C. Implementation Challenges and… · p. 12

A. The Adoption Pathway Problem

The paper identifies adoption—not technical feasibility—as the primary constraint on hardware-layer governance and argues that fragmented regulation may give manufacturers and governments incentives to coordinate. It proposes safety-conditioned access and institutional support, including regional verification and funded pilots, as parts of a deployment pathway.

A. The Adoption Pathway Problem

  • The primary limiting factor for hardware-layer governance is adoption: securing chip-maker participation before mechanisms become regulatory mandates remains unresolved.The paper notes that whether industry cooperates voluntarily or awaits legislation remains to be seen.Source: A. The Adoption Pathway Problem · p. 12
  • Divergent AI rules across U.S. states and jurisdictions create a coordination problem, while manufacturers may prefer multilateral standards to multiple compliance lines.The paper compares this harmonization incentive to the adoption of International Civil Aviation Organization standards.Source: A. The Adoption Pathway Problem · pp. 12–13
  • The paper cites prior conditional-access arrangements, including the VEU program and the withdrawn AI Diffusion Rule, as examples of market access tied to compliance structures.It also notes the August 2025 U.S. arrangement allowing H20 and MI308 chip sales to China in exchange for 15% of sale revenues, describing it as legally contested.Source: A. The Adoption Pathway Problem · p. 13
  • Auditable, tamper-resistant governance modules could make chip compliance verifiable regardless of destination, aligning manufacturers’ expanded market access with governments’ oversight goals.The paper presents safety-conditioned exports as a possible alternative to blunt embargoes.Source: A. The Adoption Pathway Problem · p. 13
  • A Global South deployment blueprint includes funded public-sector pilots, regional attestation authorities, compliance consortia, and sovereign evaluation and monitoring capacity.Regional bodies would verify compliance, handle disputes, and adapt standards to local contexts; consortia could help smaller nations pool expertise and infrastructure.Source: A. The Adoption Pathway Problem · pp. 13–14
  • Without technical and financial capacity-building, hardware governance risks reproducing asymmetry by leaving the Global South able to use, but not govern, verification infrastructure.The cited proposal includes multistate collaboration to build large-scale AI compute for Global South use.Source: A. The Adoption Pathway Problem · p. 14

B. The IAEA Model: Multilateral Oversight for Dual-Use Technology

The paper proposes a multilateral AI oversight architecture that combines hardware verification with institutional oversight, drawing functionally—but not literally—on the IAEA. It sketches what the regime would verify, how scrutiny could escalate, and what safeguards and limitations it would need.

B. The IAEA Model: Multilateral Oversight for Dual-Use Technology

  • The IAEA precedent offers international inspections and multilateral standard-setting, but the paper treats the analogy as functional rather than literal given AI’s replicability and faster innovation cycles.The paper notes that enforcement remains difficult during geopolitical tensions and that AI governance would also face speed and regulatory-capture concerns (Wasil et al., 2024).Source: B. The IAEA Model: Multilateral… · p. 14
  • A multilateral body could combine FlexHEG-style hardware verification with IAEA-style oversight to set standards, certify compliance, and audit contested cases.Privacy-preserving local verification is proposed to address sovereignty concerns.Source: B. The IAEA Model: Multilateral… · p. 14
  • The proposed regime would routinely attest to compute use, model properties, safety evaluations, and declared deployment constraints, with enhanced scrutiny triggered by anomalies or non-compliance.Triggers include anomalous compute patterns, credible reports, missing attestation reports, or runs above an agreed threshold.Source: B. The IAEA Model: Multilateral… · p. 15
  • The verification proposal adapts routine nuclear material accountancy and inspections into compute attestation and scheduled reviews, with special scrutiny for specified triggers.The IAEA system also uses containment and surveillance alongside routine and special inspections.Source: B. The IAEA Model: Multilateral… · pp. 14–15
  • Responses would escalate from remediation and monitoring to suspended certification, then referral to a multilateral body and coordinated state action for serious violations.Export controls would remain coordinated state measures, not direct enforcement tools of the oversight body.Source: B. The IAEA Model: Multilateral… · p. 15
  • Privacy-preserving verification is technically plausible, but verifying aggregate training compute and architectural properties at scale remains an active research area.Source: B. The IAEA Model: Multilateral… · p. 15

VI. Reframing the Debate: Open Does Not Mean Ungoverned

The paper argues that openness is a distribution property, not a final measure of whether a model is governed. It presents hardware verification, software safeguards, institutional oversight, and liability as complementary layers rather than relying on access restrictions alone.

VI. Reframing the Debate: Open Does Not Mean Ungoverned

  • Openness is a distribution property; governance also depends on modifiability, execution environment, audit mechanisms, and oversight authority.The paper argues that physical hardware can offer enforcement mechanisms unavailable to software distribution controls.Source: §VI · p. 15
  • More than doubled was the number of companies publishing Frontier AI Safety Frameworks during 2025, while the report also found sophisticated attackers frequently circumvent current defenses.The paper presents hardware governance as one necessary part of a defense-in-depth architecture (Bengio et al., 2025b).Source: §VI · pp. 15–16
  • OECD principles call for mechanisms to override, repair, or safely decommission harmful AI systems, while the EU GPAI Code of Practice leaves structural gaps through voluntary compliance and compute-based thresholds.The OECD recommendation does not prescribe a specific technical mechanism.Source: §VI · p. 16
  • Commercial distributors and other commercial actors may face liability for foreseeable harms from defective safety features or inadequate safeguards, while noncommercial free and open-source software is exempt under the cited EU directive.Liability complements technical verification and institutional oversight at the point of distribution as well as execution.Source: §VI · p. 16

VII. Limits and Objections

The paper qualifies its case for hardware governance: restrictions can reduce some risks, but without alternatives their benefits may be offset by displacement and dependency. Hardware controls also risk concentrating power and being repurposed, so their design requires institutional safeguards and interoperable standards.

VII. Limits and Objections

  • Restrictions can reduce real risks in some contexts, but their containment gains are conditional and may be offset without alternative governance infrastructure.The paper does not claim open-weight models are intrinsically safe or that restrictions are always counterproductive.Source: §VII · p. 16
  • Hardware governance repositions rather than eliminates chip-supply concentration, making multilateral and auditable arrangements a condition of its potential advantage.Source: §VII · p. 16
  • Chip controls could enable surveillance, competitor suppression, or politically motivated denial of access, raising civil-liberties and adoption concerns (Sastry et al., 2024).Source: §VII · p. 17
  • The proposed safeguards include multilateral authorization for enforcement, tamper-evident logs, periodic reauthorization, and civil-society participation with audit and challenge rights.The paper also proposes multisignature protocols so no single government, vendor, or regulator controls enforcement activation (Sastry et al., 2024).Source: §VII · p. 17
  • Open attestation standards and interoperable verification interfaces are needed to avoid vendor or jurisdictional lock-in and preserve distributed power.The paper distinguishes open interfaces from implementation internals, which may remain protected under its proposed intellectual-property approach.Source: §VII · p. 17
  • An IAEA-inspired institution would be a functional analogy, not a direct transplant, because AI software is replicable and innovation cycles are faster.Source: §VII · p. 17
  • Material export controls can be effective for traceable hardware, but software restrictions have struggled against copying, as the Crypto Wars illustrate (Kehl et al., 2015).The paper compares open-weight models to encryption rather than centrifuges, motivating governance at the hardware layer.Source: §VII · p. 18

VIII. Conclusion

The conclusion argues that closing access alone displaces risks into less visible spaces, while deepening dependency and Shadow AI proliferation. It calls for layered technical safeguards and multilateral governance, with explicit measures to protect civil liberties and broaden access to governed compute.

VIII. Conclusion

  • Restrictions without alternatives can displace risk into Shadow AI, deepen Global South infrastructure dependency, and shift demand to less transparent ecosystems.The paper’s threat taxonomy distributes these risks across hardware, software, institutional, and liability layers.Source: §VIII · p. 18
  • There is less time to build governance arrangements as models become easier to run, adapt, and redistribute beyond traditional tracking mechanisms.The paper warns that regulatory inertia favors more opaque and unequal proliferation.Source: §VIII · p. 18
  • Governments and multilateral bodies should pilot hardware-enabled governance, from chip-level attestation and trusted execution environments to weight protection and confidential computing.The proposal begins with voluntary adoption on next-generation chips and public audits; funded programs should support governed-chip deployment in the Global South through regional attestation and capacity-building consortia.Source: §VIII · pp. 18–19
  • The paper calls for multilateral negotiations on compute oversight using graduated verification, routine attestation logs, enhanced scrutiny, and measures adapted to digital technologies.Source: §VIII · p. 18
  • Governance should layer software safeguards, hardware verification, and institutional oversight rather than rely on any single mechanism (Bengio, 2024).Source: §VIII · p. 18
  • Liability regimes should assign commercial distributors proportionate responsibility for foreseeable harms, drawing on the EU’s revised Product Liability Directive.The directive classifies software, including AI systems, as a product subject to no-fault liability when supplied commercially.Source: §VIII · pp. 18–19
  • Hardware governance should include multilateral quorum requirements, transparency logs, independent civil-society audits, and sunset clauses to guard against censorship or political repression.Source: §VIII · p. 19
Loading 2604.17413v1…