PaperlayerView exact revision on arXiv

Source-linked AI summary

Regulating ChatGPT and other Large Generative AI Models

Philipp Hacker, Andreas Engel, Marco Mauer

arXiv:2302.02337v8cs.CYcs.AI

TL;DR

LGAIMs are transforming communication, creation, and work across society, while existing regulation may leave them insufficiently addressed and expose risks including discrimination, privacy, and harmful content. The paper proposes differentiated actors and layered rules, concluding that technology-neutral laws can sometimes fare better than technology-specific regulation and adding policy proposals for LGAIM governance.

  • Problem

    LGAIMs are transforming communication, creation, and work across society, while existing regulation may not apply to them and risks including discrimination, privacy, and harmful content require attention.

  • Method

    The paper develops differentiated terminology for actors in the LGAIM value chain and proposes three layers of rules applicable to LGAIMs.

  • Results

    The study concludes that technology-neutral laws sometimes fare better than technology-specific regulation and adds policy proposals to the regulatory landscape surrounding LGAIMs.

  • Takeaways & Limitations

    Regulatory approaches for LGAIMs should account for distinct actors in the AI value chain and apply rules directly to all LGAIMs through existing technology-neutral regulation.

  • Takeaways & Limitations

    Final court decisions may take substantial time, so compliance directions from lower-court judgments or agency decisions may need revision if higher courts reverse them.

Abstract

from arXiv · show

Large generative AI models (LGAIMs), such as ChatGPT, GPT-4 or Stable Diffusion, are rapidly transforming the way we communicate, illustrate, and create. However, AI regulation, in the EU and beyond, has primarily focused on conventional AI models, not LGAIMs. This paper will situate these new generative models in the current debate on trustworthy AI regulation, and ask how the law can be tailored to their capabilities. After laying technical foundations, the legal part of the paper proceeds in four steps, covering (1) direct regulation, (2) data protection, (3) content moderation, and (4) policy proposals. It suggests a novel terminology to capture the AI value chain in LGAIM settings by differentiating between LGAIM developers, deployers, professional and non-professional users, as well as recipients of LGAIM output. We tailor regulatory duties to these different actors along the value chain and suggest strategies to ensure that LGAIMs are trustworthy and deployed for the benefit of society at large. Rules in the AI Act and other direct regulation must match the specificities of pre-trained models. The paper argues for three layers of obligations concerning LGAIMs (minimum standards for all LGAIMs; high-risk obligations for high-risk use cases; collaborations along the AI value chain). In general, regulation should focus on concrete high-risk applications, and not the pre-trained model itself, and should include (i) obligations regarding transparency and (ii) risk management. Non-discrimination provisions (iii) may, however, apply to LGAIM developers. Lastly, (iv) the core of the DSA content moderation rules should be expanded to cover LGAIMs. This includes notice and action mechanisms, and trusted flaggers. In all areas, regulators and lawmakers need to act fast to keep track with the dynamics of ChatGPT et al.

1 INTRODUCTION

LGAIMs are transforming communication and creative work while introducing significant risks that existing AI regulation has not adequately addressed. The paper proposes differentiated duties across the AI value chain, application-focused risk regulation, and expanded content-moderation rules.

  • Motivation: LGAIMs are rapidly transforming communication, creation, and work across sectors while creating risks involving discrimination, privacy, errors, manipulation, fake news, and harmful speech.Their applications range from business and medicine to education, research, coding, entertainment, and the arts.
  • Regulatory gap: Existing EU and international AI regulation has primarily focused on conventional AI models rather than LGAIMs.The paper asks how current and future law should be tailored to LGAIM capabilities.
  • Approach: The paper examines LGAIM regulation through technical foundations, direct AI regulation, data protection, content moderation, and policy proposals.It critiques the EU AI Act and analyzes communication-related issues through the DSA and non-discrimination law.
  • Direct regulation: Because LGAIMs can serve many purposes, requiring one model to manage every imaginable high-risk use is considered prohibitive and unnecessary.The paper therefore generally places risk regulation on deployed applications rather than the pre-trained model, while allowing non-discrimination duties to reach developers.
  • Content moderation: The paper identifies a regulatory loophole because the DSA does not apply to LGAIMs, despite risks from scalable hate speech and polished fake news.It proposes extending specific content-moderation mechanisms, including notice-and-action procedures and trusted flaggers, to LGAIMs.
  • Policy proposals: The proposed framework has three layers: minimum standards for all LGAIMs, high-risk obligations for high-risk uses, and collaboration across the AI value chain.Specific proposals concern transparency, risk management, non-discrimination, and LGAIM-specific content moderation.

2 TECHNICAL FOUNDATIONS OF LARGE GENERATIVE AI MODELS AND EXEMPLARY USAGE SCENARIOS

LGAIMs are large-scale models that generate new text, images, audio, or video by learning patterns from extensive data. Their broad utility creates challenges involving data quality, harmful or misleading outputs, moderation, and the allocation of responsibilities among actors using the models.

  • Technical foundations: LGAIM training commonly requires billions of parameters, large datasets, substantial computing power, and significant energy.Examples include CLIP trained on 400 million image-text pairs and BASIC using 6.6 billion such pairs.
  • Technical foundations: LGAIMs are advanced machine-learning models trained to generate new data, including text, images, audio, and video.Unlike models designed only for prediction or classification, they can produce outputs across multiple modalities.
  • Technical foundations: Models learn patterns and relationships from training data, then generate new examples by sampling and mixing learned probability distributions.The resulting content can extend beyond the original training dataset while remaining similar to it.
  • Risks: Internet-sourced training data may be imperfect, allowing generated content to become biased, prejudiced, or harmful.The paper identifies data curation as a mitigation need for model developers.
  • Content moderation: LGAIMs can produce offensive material, hard-to-distinguish fake information, and volumes of output that are difficult to review manually.Automated detection addresses abusive content but does not eliminate the challenge of misleading synthetic content.
  • Usage scenarios: The paper illustrates the value chain through a sportswear company using a developer’s pre-trained model and a deployer’s fine-tuning, alongside private chatbot use.The examples distinguish organizational deployment from non-professional individual use.

3 DIRECT REGULATION OF THE AI VALUE CHAIN: THE EUROPEAN AI ACT

The EU AI Act developed provisions for general-purpose AI systems and sought to distribute responsibilities along the AI value chain. The paper describes these rules, including high-risk treatment, territorial reach, exemptions, notice-and-action duties, and production monitoring.

  • General-purpose AI systems: The proposed AI Act provisions for general-purpose AI systems became a central and contested element of direct LGAIM regulation.The Council defined such systems as usable across multiple contexts and potentially integrable into other AI systems.
  • High-risk obligations: General-purpose AI systems may receive high-risk obligations when they can be used as high-risk systems or as components of them.The Commission would specify through implementing acts how those high-risk rules should be adapted.
  • Territorial scope: The AI Act’s territorial scope can cover providers outside the EU when systems are placed on the EU market or their output is used in the Union.The rules may therefore apply even when the provider is entirely based outside the EU.
  • Provider duties: Providers detecting or receiving information about misuse must take proportionate measures to stop it and avoid harm.This notice-and-action mechanism complements an active production-monitoring obligation.

3.1 Critique of the GPAIS AI Act Rules

The paper argues that the AI Act’s GPAIS rules inadequately fit LGAIMs because their broad capabilities make model-level high-risk compliance over-inclusive, costly, and difficult to operationalize. It therefore favors targeted obligations for concrete use cases while retaining selected duties for models and developers.

  • Critique of the GPAIS AI Act Rules: The proposed GPAIS rules fail to accommodate LGAIMs’ versatility and broad applicability, especially the requirement for comprehensive model-level risk management.Providers may need to identify and analyze an impractically broad set of possible uses.
  • Toward a Definition of GPAIS: The paper defines LGAIM generality through breadth of abilities, tasks, or outputs, rather than merely potential integration into different use cases.This approach distinguishes genuinely general systems from simple recognition tools that the AI Act’s broad definition may capture.
  • Economic Effects: Uniform high-risk obligations may make compliance prohibitively costly for open-source developers and SMEs, potentially accelerating concentration among large firms.The paper identifies this as an unintended anti-competitive consequence of the proposed regime.
  • Proposed Direction: The paper recommends focusing most high-risk obligations on concrete applications, while applying minimum data-governance and selected non-discrimination duties directly to foundation models or developers.It also supports transparency requirements and more specific notice-and-action mechanisms for generative AI.
  • Critique of the GPAIS AI Act Rules: Treating foundation models as high-risk applications would require performance, robustness, cybersecurity, and risk-management assessments across all possible high-risk uses.The paper considers this nearly infeasible because many potential uses will never be realized.

3.2 Proposal: Focus on Deployers and Users

The paper proposes reallocating regulatory responsibility across the LGAIM value chain, concentrating high-risk duties on deployers and professional users while retaining selected obligations for developers. Because no actor has complete knowledge or control, effective compliance requires structured collaboration and documented responsibility-sharing.

  • Regulatory Focus: High-risk AI Act obligations should primarily target deployers and professional users who adapt or apply LGAIMs in concrete high-risk use cases.General rules on data governance, non-discrimination, and cybersecurity may still apply to foundation models.
  • AI Value Chain: The proposed terminology distinguishes developers, deployers, professional users, and non-professional users to allocate duties more precisely along the AI value chain.Developers create and pre-train models, while deployers fine-tune them for specific use cases.
  • Allocation of Duties: Developers should retain duties concerning non-discrimination and data governance, whereas deployers and users should handle use-case risk management and performance or robustness requirements.This allocation reflects the actors’ differing influence over the deployed system.
  • Collaboration: Because developers may lack the knowledge and control needed to satisfy all downstream duties, the paper calls for regulator-shaped collaboration among providers, deployers, and users.The proposed framework includes compelled cooperation where developers can make necessary technological adjustments or absorb compliance costs.
  • Accountability and Liability: Written internal compliance agreements, disclosure of their core terms, and joint and several liability would support accountability, evidence access, and injured persons’ compensation.Trade secrets should be protected while sufficient allocation information remains available to potential claimants.
  • Conclusion: The paper concludes that AI Act responsibility and liability should follow actors’ influence over concrete deployments rather than regulate models per se.The EP version’s value-chain rules therefore require greater specificity to function effectively.

4 NON-DISCRIMINATION LAW

The paper argues that non-discrimination law can apply to LGAIM developers when model preparation is concretely linked to protected activities, while generic models ordinarily fall outside that reach. It also proposes output screening duties for all users, including non-professional users, limited to evident significant harms.

  • Developer Duties: Technology-neutral non-discrimination law may apply directly to LGAIM developers in appropriate circumstances.Significant underperformance for legally protected groups may indicate indirect or direct discrimination and establish a prima facie case.
  • Scope of Application: The paper distinguishes preparatory model development from concrete deployments, noting that anti-discrimination rules clearly apply to relevant deployment scenarios.EU case law supports applying such provisions to preparatory activities when their relationship to covered activity is not merely hypothetical.
  • Scope of Application: The required link exists when model pre-training is specifically prepared for discrimination-relevant scenarios such as employment, education, or public goods and services.A generic model without a specific connection to those scenarios generally is not covered at the development stage.
  • User Duties: Professional users must monitor high-risk systems for discriminatory bias, but the paper argues that non-professional users should also screen outputs for evident significant harm.The proposed duty would be limited to best efforts and flagrant cases, such as overtly discriminatory statements.
  • User Duties: Users who detect flagrant discriminatory output should notify developers, integrating lay users into enforcement without overburdening standard LGAIM use.The proposal preserves a limited screening obligation for private users.

5 DATA PROTECTION UNDER THE GDPR AND THE ITALY BAN

The paper identifies GDPR challenges involving training data, model inversion, hallucinated personal data, discriminatory output, and chat-interface transparency. It treats the Italian GPDP’s action against OpenAI as a call for stronger disclosure and accountability, while noting unresolved questions about the adequacy of remedial measures.

  • Training Data: GDPR compliance for LGAIM training depends on factors including model purpose, personal-data type, inversion likelihood, and re-identification probability.The paper notes scholarly disagreement over whether personal-data processing for machine learning satisfies the balancing test.
  • Model Inversion: Model inversion creates a particularly serious GDPR risk for sensitive data because legitimate developer interests may not outweigh the possibility of reproduction.The paper identifies truly critical purposes, potentially medical or emergency uses, as a possible exception.
  • Italy Ban: The Italian GPDP temporarily limited processing of Italian users’ data by OpenAI, after which OpenAI geoblocked ChatGPT in Italy for several weeks.The episode concerned transparency about training data and the processing of user inputs.
  • Transparency: Training-data scraping may constitute a borderline Article 14 GDPR case, requiring controllers to document their balancing exercise and publicly disclose how training data were collected.Individual notice may involve disproportionate effort when data come from many people with unknown contact details.
  • Generated Output: LGAIM hallucinations can produce inaccurate personal data contrary to GDPR accuracy requirements, while discriminatory outputs may violate the fairness principle.These risks arise at the level of generated output rather than only during training.
  • Open Questions: The adequacy of the Italian response remains uncertain, including whether the age gate and information supplied satisfy GDPR duties and whether the objection mechanism is effective.The paper also questions whether the temporary ban was proportionate given less drastic alternatives and ongoing inquiries.
  • Italy Ban: The paper views the Italian decisions as a wake-up call for developers to disclose information about training, personal data, and pertinent risks.OpenAI subsequently described data processing and user rights, offered access, deletion, correction, and objection channels, and introduced age-related measures.

6 GENERATIVE MODEL CONTENT MODERATION: THE EUROPEAN DIGITAL SERVICES ACT

LGAIMs create content-moderation challenges that the DSA was not designed to address, leaving important regulatory gaps despite its mechanisms for online platforms.

  • Risks: LGAIMs can mass-produce sophisticated, seemingly factual misinformation and harmful speech, while prompt engineering and filter circumvention facilitate misuse.Their confident hallucinations can generate text disconnected from reality and support harmful campaigns.
  • Regulatory gap: The DSA was drafted for intermediary services and traditional user-generated content, not systems that generate content themselves.LGAIMs do not readily fit the DSA’s access, caching, or hosting-service categories.
  • Regulatory gap: Consequently, standalone LGAIM content falls outside the full range of DSA notice, action, trusted-flagger, dispute-resolution, and risk-management mechanisms.The full DSA framework continues to apply most clearly when users post LGAIM-generated content on traditional social networks.
  • Conclusion: Current EU law therefore does not adequately address LGAIMs’ contribution to fake news, hate speech, and other harmful content.Direct speech regulation remains largely dependent on divergent Member State law and often lacks the DSA’s procedural safeguards.
  • Regulatory gap: A second gap arises when LGAIM-generated content circulates through interpersonal channels or private groups that are not distributed to the general public.Malicious actors may thereby evade both the DSA’s scope and its enforcement tools.

7 POLICY PROPOSALS

The paper proposes risk-tailored LGAIM regulation organized around minimum standards, high-risk use cases, and collaboration across the AI value chain. Its proposals emphasize transparency, proportionate risk management, data governance, non-discrimination, and expanded content moderation.

  • Regulatory framework: LGAIM regulation should shift from wholesale model regulation toward concrete risks and use cases, while keeping compliance feasible for developers of different sizes.The authors link feasibility to avoiding market concentration and supporting innovation, consumer welfare, and sustainability.
  • Regulatory framework: The proposed framework has three layers: minimum standards for all LGAIMs, high-risk rules for concrete high-risk uses, and collaboration obligations across the AI value chain.Minimum standards include existing EU law, transparency, selected data-governance duties, cybersecurity, sustainability, and content moderation.
  • Transparency: LGAIMs should face two transparency duties regardless of high-risk categorization: developers and deployers report model information, while professional users disclose generated content.Relevant disclosures include training-data provenance, performance metrics, harmful-content incidents, mitigation strategies, and potentially greenhouse-gas emissions.
  • Transparency: Professional-user disclosure may be tailored by context, with stronger justification in journalism, academic research, and education than in some sales, production, or B2B settings.The authors nevertheless advocate a general professional-user disclosure obligation for the time being.
  • Transparency: Non-professional users should generally not be required to disclose AI use, partly because malicious actors would likely disregard disclosure rules for harmful social-media content.Technical measures such as model-imprinted watermarks and improved detection could support enforcement, but further research is needed.

7.2 Risk Management and Staged Release

The paper argues that comprehensive high-risk obligations should attach when an LGAIM is used in a high-risk application, while powerful models may warrant staged release and regulated self-regulation.

  • Staged release: Powerful models may be released in limited stages, initially restricting access to security researchers and selected stakeholders.The proposal trades broader public scrutiny against the risk of misuse and considers the balance of power among developers.
  • Regulated self-regulation: Codes of conduct and regulated self-regulation could add community-based oversight with potentially binding effect, rather than relying solely on voluntary commitments.The authors prefer an approach modeled on Article 40 GDPR to the purely voluntary strategy envisioned in Article 69 AI Act.
  • Application-based obligations: Comprehensive AI Act risk-management duties should apply only when an LGAIM is used for a concrete high-risk purpose.This follows a product-safety approach in which stringent requirements depend on the application rather than every generic component.

7.3 Non-Discrimination and training data

The paper assigns selected data-curation and non-discrimination duties to LGAIM developers, and proposes extending DSA-style content moderation to models and their deployers through coordinated monitoring.

  • Non-discrimination: LGAIM developers should address representativeness and balance between protected groups in training data rather than delegating discrimination risks entirely to users.The proposed burden should remain proportionate to company size, abstract risk, and the type of training material.
  • Training data: Developers should audit training data for protected-group misrepresentation and implement feasible mitigation measures, including synthetic data where appropriate.Synthetic augmentation is proposed to counter historical and societal biases in online sources.
  • Content moderation: DSA rules should be selectively expanded to LGAIM developers and deployers, including notice-and-action mechanisms, trusted flaggers, and audits for models with particularly many users.Standalone LGAIMs present the sharpest loophole, while platform integration would still require explicit coverage of generated content.
  • Content moderation: LGAIM-generated content should be flagged when technically feasible, because its susceptibility to misinformation makes provenance relevant to moderation and enforcement.The authors identify amendments to the DSA or Article 29 AI Act as possible legal routes.
  • Content moderation: The proposed system combines decentralized user reporting and trusted-flagger testing with centralized engineering responses that modify models or block problematic outputs.Very large systems would additionally establish comprehensive compliance systems, while further research should integrate factuality and moderation into models.
  • Conclusion: Technical advances alone will not resolve LGAIM risks; they must be embedded in societal discourse and regulation.The paper presents transparent models connected to knowledge bases as one promising direction.

7.5 Outlook: Technology-specific vs. technology-neutral regulation

The paper favors technology-neutral regulation tailored through agencies and courts over technology-specific rules that may become outdated before or upon enactment. This approach can more flexibly establish compliance guidance, safe harbors, and protections for affected persons.

  • Technology-specific regulation may be outdated before or when enacted, whereas technology-neutral rules can better track LGAIM development dynamics.
  • Agencies and courts can tailor technology-neutral laws to specific technologies through guidelines, decisions, and judgments.
  • Extending the DSA to LGAIMs through legislation would require concurring decisions by the European Parliament and Council.
  • Agency and court interpretations can develop flexible compliance tools, preliminary safe harbors, and red lines protecting affected persons.

8 CONCLUSION

The conclusion proposes actor-specific, layered regulation for LGAIMs, generally directing high-risk duties toward concrete applications rather than pre-trained models. It also advocates technology-neutral rules, value-chain collaboration, transparency, and expanded content moderation obligations.

  • Technology-neutral laws may better capture LGAIM dynamics than currently enacted or proposed technology-specific AI and platform regulation.
  • The paper distinguishes developers, deployers, professional and non-professional users, and recipients to allocate duties across the LGAIM value chain.
  • Three regulatory layers combine minimum standards for all LGAIMs, high-risk obligations for concrete applications, and collaboration across the value chain.
  • High-risk obligations should generally target concrete uses rather than pre-trained models, because comprehensive model-level risk management is practically infeasible.
  • Providers and deployers must cooperate on compliance while balancing access to information against trade-secret protection.
  • Transparency duties should cover developer and deployer performance and pre-training risks, while users should disclose LGAIM-generated content.
  • DSA content-moderation rules should extend to LGAIMs through notice-and-action mechanisms, trusted flaggers, and risk management for very large developers.
  • Regulators and lawmakers should act quickly because LGAIM dynamics are changing rapidly and regulation must support online civility and a level playing field.

APPENDIX H1: PROMPTS

The appendix lists prompts addressing what LGAIMs are, how they differ from other AI systems, their technical foundations, and content moderation objectives and practices.

  • Prompt 1 asks what large generative AI models are.
  • Prompt 2 asks how large generative AI models differ from other AI systems.
  • Prompt 3 requests a simple explanation of the technical foundations of large generative models for inexperienced readers.
  • Prompts 4 and 5 address content-moderation objectives, obstacles, and ChatGPT’s moderation practices.
Loading 2302.02337v8…