PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative AI in EU Law: Liability, Privacy, Intellectual Property, and Cybersecurity

Claudio Novelli, Federico Casolari, Philipp Hacker, Giorgio Spedicato, Luciano Floridi

arXiv:2401.07348v4cs.CYcs.AI

TL;DR

Unpredictable generative AI outputs raise questions about the adequacy of EU law across liability, privacy, intellectual property, and cybersecurity. The paper analyzes these legal issues and the EU framework, identifying shortcomings and proposing regulatory changes; it concludes that the EU does not seem adequately prepared for these developments.

  • Problem

    Generative AI’s unpredictable outputs raise concerns about the lawfulness and accuracy of generated content, while the adequacy of EU law for these systems remains in question.

    Source: §1 · p. 2, §6 · p. 27

  • Method

    The paper analyzes EU legal and regulatory challenges involving generative AI across liability, privacy, intellectual property, and cybersecurity.

    Source: §6 · p. 27

  • Results

    The paper concludes that the EU does not seem adequately prepared to address generative AI’s unpredictable outputs and their legal concerns.

    Source: §6 · p. 27

  • Takeaways & Limitations

    The paper proposes updating current and forthcoming regulations and enacting specific regulations for generative AI.

    Source: §6 · p. 27

    The paper identifies reliance on FLOPs alone as an inadequate indicator of systemic risk, since risk also depends on application context, architecture, training quality, and interacting non-computational factors.

    Source: §2 · p. 4

Abstract

from arXiv · show

The advent of Generative AI, particularly through Large Language Models (LLMs) like ChatGPT and its successors, marks a paradigm shift in the AI landscape. Advanced LLMs exhibit multimodality, handling diverse data formats, thereby broadening their application scope. However, the complexity and emergent autonomy of these models introduce challenges in predictability and legal compliance. This paper delves into the legal and regulatory implications of Generative AI and LLMs in the European Union context, analyzing aspects of liability, privacy, intellectual property, and cybersecurity. It critically examines the adequacy of the existing and proposed EU legislation, including the Artificial Intelligence Act (AIA) draft, in addressing the unique challenges posed by Generative AI in general and LLMs in particular. The paper identifies potential gaps and shortcomings in the legislative framework and proposes recommendations to ensure the safe and compliant deployment of generative models, ensuring they align with the EU's evolving digital landscape and legal standards.

1. Overview

Generative AI and LLMs broaden AI’s capabilities through scalable language generation and multimodal processing, but their outputs can be unpredictable. The article examines the EU legal and regulatory response to these systems, with a focus on LLMs and on liability, privacy, intellectual property, and cybersecurity.

1. Overview

  • LLMs generate language from vast text training data, unlike traditional AI models that output scores or labels.Generative AI covers more systems than LLMs, but the paper focuses on LLMs because their legal concerns overlap.Source: §1 · p. 1
  • Multimodal LLMs can process and produce different data formats, while their broad capabilities are accompanied by unpredictable outputs that raise concerns about lawfulness and accuracy.The passage gives GPT-4 as an example of handling text, image, and audio inputs.Source: §1 · p. 2
  • The article assesses EU legislation, including the AI Act, against concerns about liability, privacy, intellectual property, and cybersecurity, and recommends changes where it identifies gaps or flaws.Source: §1 · p. 2

2. Liability and AI Act

The paper argues that EU liability rules for AI need to account for generative models’ technical characteristics and deployment contexts. It reviews the proposed liability directives and AI Act framework, identifies shortcomings, and recommends adjustments to risk assessment, liability criteria, and evidence disclosure.

2. Liability and AI Act

  • The AILD and PLD depend on AI Act requirements that the paper considers ill-suited to govern LLMs effectively, making improvement before enforcement important.The paper notes that GPAI obligations apply on a shorter transition schedule than other AI Act rules.Source: §2 · p. 3
  • The AILD’s disclosure mechanism and rebuttable presumption apply only to high-risk AI systems, raising the question of when generative AI models fall within its scope.The AI Act instead gives GPAI models a distinct tiered classification with different provider obligations.Source: §2 · pp. 3–4
  • The paper questions classifying systemic risk mainly by a 10^25-FLOP threshold, since application context, architecture, training quality, and interacting non-computational risks also matter.It notes that models below the threshold, including GPT-3 and Bard at 10^24 or 10^23 FLOPs, may be equally risky.Source: §2 · p. 4
  • The paper proposes tailoring GPAI liability exemptions to deployment contexts and potential harms, rather than relying only on computational potency.It contrasts resume-screening optimization with automated virtual interviews and suggests lower-risk uses such as video games could qualify for exemptions.Source: §2 · p. 5
  • Because LLMs may lack a single specific purpose before adaptation, AI Act risk-management and transparency requirements can be difficult to apply during development.The PLD and AILD assess defectiveness and fault through compliance with AI Act requirements.Source: §2 · p. 5
  • The authors recommend evaluating generative AI fault and defectiveness through methods that steer nondeterministic outputs and monitor uncertainty and divergence from planned operation.Examples include temperature scaling, top-k sampling, prompt engineering, adversarial training, and regularization; uncertainty information could help evaluate designers’ and deployers’ fault or defectiveness.Source: §2 · pp. 5–6
  • The PLD and AIA disclosure rules are misaligned, and neither liability proposal specifies what information defendants must disclose.The AIA lacks blanket lifecycle logging for GPAI providers, while ongoing incident reporting is limited to systemically risky models.Source: §2 · p. 6
  • The paper recommends extending evidence disclosure beyond high-risk systems and including incident details, interaction logs, and documentation of training data and data selection.It also recommends encouraging producers to notify users of potential harm, since disclosure depends on users having a reasonable basis to suspect harm and support a claim.Source: §2 · pp. 6–7

3. Privacy and Data Protection

Generative AI creates privacy and data-protection challenges across training, user interactions, and model outputs. The section examines the resulting legal questions and proposes GDPR and AI Act adaptations.

Overview

  • LLMs can expose personal data through memorization, user interactions, and inversion attacks, while existing privacy-preserving strategies offer limited protection.The section identifies seven issues, including legal bases, information duties, erasure, automated decision-making, minors, and purpose limitation.Source: §3 · p. 8

Legal basis for AI training on personal data

  • Because consent is generally impractical for large web-scraped datasets, AI training typically relies on Article 6(1)(f), whose balancing test requires case-by-case assessment.Reasonable expectations of training use will rarely be met; safeguards and the type, scope, and transparency of processing affect the balance.Source: §3 · p. 8, 1.a) Consent and the balancing test · p. 9

Sensitive data

  • The ability to infer sensitive traits from available data may bring AI training under Article 9 GDPR, whose research exemption does not cover commercial exploitation.The authors call for a new Article 9 exemption with strong safeguards to balance socially beneficial AI training and individual rights.Source: 1.a) Consent and the balancing test · p. 10

Prompts and information requirements

  • Users cannot ordinarily consent for third parties whose personal data they enter in prompts, and the LLM operator may control later storage or leakage.Article 13 information duties apply to personal data users submit about themselves; the disproportionate-effort exemption may apply to some scraped training data instead.Source: 1.a) Consent and the balancing test · pp. 10–11

Model inversion, data leakage, and the right to erasure

  • Model inversion can reconstruct training data, while memorization can cause personal information to appear in outputs; treating an LLM as personal data could raise demands to erase the model itself.Removing training-set data may not prevent later retrieval, and whole-model deletion can be technically difficult and have sustainability costs; the authors point to machine unlearning as a needed strategy.Source: 1.a) Consent and the balancing test · pp. 12–13

Automated decision-making and protection of minors

  • LLM evaluations may fall under Article 22 when they are paramount to a decision, even with later human sign-off; the section also flags minors’ protections and age-verification measures.Consent may be difficult amid power imbalances, and the effectiveness of age-verification tools remains under evaluation.Source: 1.a) Consent and the balancing test · pp. 13–14

Purpose limitation and data minimization

  • Restricting data for open-ended LLM purposes may be counterproductive, so the authors suggest proportional privacy safeguards, including differential privacy for large datasets.They also identify smaller datasets with few- or zero-shot learning as an alternative approach.Source: 1.a) Consent and the balancing test · p. 14

Ways forward

  • The authors propose tailored GPAI data-governance measures, training on publicly available data where possible, and privacy measures such as opt-outs and machine unlearning.They argue the AI Act’s Article 10 may be too generic for GPAI complexities; proposed Article 28(b) data-governance language did not enter the final Act.Source: 1.a) Consent and the balancing test · pp. 14–15

4. Intellectual Property

The section examines copyright issues in LLM training and output generation, arguing that existing EU rules leave difficult questions about lawful dataset use and protection of AI-generated works. It distinguishes outputs reflecting human creative choices from substantially autonomous outputs and discusses possible liability and rights-allocation approaches.

Training

  • LLM training on protected material may qualify as text and data mining, but commercial use is permitted only if right holders have not expressly reserved it.The authors also urge interpreting the exception to cover validation and testing; a training-only reading would require deleting training materials after use.Source: §4 · p. 17
  • A training-only reading of the TDM exception would bar retaining protected material for validation or testing, so the authors urge a broader interpretation.The exception also covers reproductions and extractions, not modifications needed to prepare materials for AI training; its reach is further constrained by the three-step test, whose application in individual cases remains unclear.Source: §4 · pp. 17–18

Output generation

  • Whether an output infringes or derives from pre-existing works requires case-specific comparison; training on protected works alone does not establish infringement.Substantial direct similarities to protected elements may infringe, while unauthorized indirect adaptations may qualify as derivative works; authorial style alone is not protected in most European legal systems.Source: §4 · pp. 18–19
  • A person prompting an infringing output would be primarily liable, while LLM developers might also be liable if platform duties were adapted to them.The authors say that such a transposition may not be straightforward.Source: §4 · p. 19
  • Substantially autonomous outputs face a barrier to copyright or patent protection under IP law's anthropocentric premises, although protection of investment is raised as an alternative rationale.The paper notes unresolved questions about who would hold rights if protection were granted; in civil-law systems, it suggests an employer-like approach could vest rights in the LLM user during entrepreneurial activity.Source: §4 · pp. 20–22
  • Predominantly human creative choices in conception, execution, or redaction can support protection, whereas a prompt or minor edits alone are likely insufficient.The paper suggests significant modifications may warrant some protection.Source: §4 · pp. 20–21

5. Cybersecurity

The section argues that EU cybersecurity rules do not adequately address several risks posed by generative AI, including adversarial attacks and misinformation. It proposes broader cybersecurity obligations and complementary measures under EU instruments, while describing NIS2's relevance to AI services.

The Cyber Resilience Act and the AI Act

  • The CRA does not explicitly address generative AI or LLMs, and the authors recommend adapting it to encompass these systems.They say the AIA's GPAI framework could provide a pathway for integration, potentially through Commission delegated acts.Source: §5 · p. 23
  • The authors propose mandatory stringent cybersecurity measures for all GPAI, not only systemic-risk models, because upstream vulnerabilities could expose many downstream products.They recommend including GPAI in CRA Annex III and linking systemic-risk GPAI requirements under the AIA and CRA.Source: §5 · pp. 23–24

Adversarial attacks

  • The AIA and CRA risk categories do not reliably predict vulnerability to adversarial attacks, so safeguards should instead reflect each model's technical attack triggers.The authors propose proportionate technical safeguards through supplementary implementation acts; possible measures include limiting access to trusted users and restricting query quantity or type.Source: §5 · pp. 24–25

Misinformation

  • LLMs can spread misinformation cheaply and widely; hallucinations—plausible text that diverges from input or facts—can contribute to erroneous decisions and further dissemination.The section notes that question-and-answer hallucinations are frequent in recent LLMs and that a study found hallucinations particularly common on a wide range of legal tasks.Source: §5 · p. 25
  • EU legislation lacks specific rules for generative-AI misinformation; the authors propose updating the AIA and DSA and broadening platform duties beyond illegal content.They also identify design and development measures, including trusted-user access limits, text fingerprinting, traceable training data, and reinforcement learning for fact sensitivity.Source: §5 · pp. 25–26

Ways forward: NIS2

  • NIS2 expands cybersecurity coverage to additional sectors and entities and requires designated entities to report certain incidents to authorities within 24 hours.The authors identify this as relevant to AI entities in covered digital infrastructure and services sectors.Source: §5 · p. 27

The Cyber Resilience Act and the AI Act

  • The CRA covers digital products but does not fully extend cybersecurity standards to services, while NIS2 can cover qualifying cloud-service providers and impose incident-notification duties.NIS2's broader entity-based approach may complement CRA coverage where generative AI is integrated into services beyond remote data processing.Source: §5 · p. 27

6. Conclusion

Generative AI and LLMs combine broad task performance with unpredictable outputs that raise concerns about lawfulness and accuracy; the EU appears inadequately prepared. The paper calls for regulatory updates and specific rules while leaving the design of a coherent EU digital-law architecture for further work.

6. Conclusion

  • The EU does not seem adequately prepared for high-performing Generative AI and LLMs whose unpredictable outputs raise concerns about lawfulness and accuracy.The paper proposes updating current and forthcoming regulations and enacting rules specific to Generative AI.Source: §6 · p. 27
  • The paper leaves the broader question of how to develop a coherent EU architecture of digital laws unresolved.It suggests considering how AI might improve the relevance, coherence, and timeliness of law, but identifies that topic as beyond the article’s scope.Source: §6 · pp. 27–28
Loading 2401.07348v4…