PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on ChatGPT: AI-Generated Contents, Challenges, and Solutions

Yuntao Wang, Yanghe Pan, Miao Yan, Zhou Su, Tom H. Luan

arXiv:2305.18339v2cs.CYcs.AIcs.CL

TL;DR

AIGC’s rapid growth creates unresolved security, privacy, ethical, and legal challenges alongside its benefits for automated content creation. This survey synthesizes AIGC technologies and architecture, threat and defense research, watermark-based regulation, and future directions, concluding that integrated protection and continued work on trustworthy systems are needed.

  • Problem

    AIGC enables fast, low-cost content creation but raises unresolved security, privacy, ethical, legal, and trust challenges.

  • Method

    The paper surveys AIGC principles, threats, defenses, regulatory and watermark-based IP-protection solutions, and open research directions.

  • Results

    The survey synthesizes state-of-the-art approaches and identifies challenges spanning trustworthy, explainable, efficient, accountable, and secure AIGC systems.

  • Takeaways & Limitations

    Effective AIGC protection requires combining complementary techniques while improving watermark efficiency, robustness, interpretability, and security.

Abstract

from arXiv · show

With the widespread use of large artificial intelligence (AI) models such as ChatGPT, AI-generated content (AIGC) has garnered increasing attention and is leading a paradigm shift in content creation and knowledge representation. AIGC uses generative large AI algorithms to assist or replace humans in creating massive, high-quality, and human-like content at a faster pace and lower cost, based on user-provided prompts. Despite the recent significant progress in AIGC, security, privacy, ethical, and legal challenges still need to be addressed. This paper presents an in-depth survey of working principles, security and privacy threats, state-of-the-art solutions, and future challenges of the AIGC paradigm. Specifically, we first explore the enabling technologies, general architecture of AIGC, and discuss its working modes and key characteristics. Then, we investigate the taxonomy of security and privacy threats to AIGC and highlight the ethical and societal implications of GPT and AIGC technologies. Furthermore, we review the state-of-the-art AIGC watermarking approaches for regulatable AIGC paradigms regarding the AIGC model and its produced content. Finally, we identify future challenges and open research directions related to AIGC.

I. INTRODUCTION

AIGC uses generative AI to assist or replace humans in producing personalized content efficiently, while ChatGPT exemplifies increasingly capable language-based generation. Its adoption also introduces security, privacy, trust, and ethical risks across the service lifecycle.

  • AIGC and ChatGPT: AIGC assists or replaces humans in creating personalized, high-quality content across text, images, audio, video, and interactive 3D media.It supports automated, low-cost production for applications such as metaverse environments and digital twins.
  • AIGC and ChatGPT: AIGC typically operates in two phases: extracting user intent and producing content based on the extracted intentions.
  • AIGC and ChatGPT: ChatGPT is a transformer-based large language model that generates human-like text and supports tasks including code generation, storytelling, translation, and semantic analysis.GPT-4 extends these capabilities to multimodal tasks.
  • AIGC and ChatGPT: AIGC depends on extensive data, large models, and computing power, with ChatGPT integrating self-supervised learning, reinforcement learning, prompt learning, and chain-of-thought prompting.
  • Risks and Challenges: AIGC faces security threats including poisoned or adversarial training samples, backdoors, model theft, jailbreaks, and prompt injection attacks.These threats can occur during training and after deployment.
  • Risks and Challenges: AIGC services raise privacy, trust, and ethical concerns because models may memorize sensitive information, generate misinformation, enable fraud, and facilitate phishing, plagiarism, and academic cheating.

B. Comparison with Existing Surveys and Contributions of Our Work

The survey distinguishes itself from earlier AIGC reviews by combining foundational analysis with comprehensive treatment of security, privacy, ethics, regulation, watermark-based IP protection, and future research directions. It organizes these contributions across the paper’s architecture, threat, protection, and outlook sections.

  • Scope and Distinction: Unlike earlier surveys, this work covers AIGC fundamentals together with security, privacy, ethical issues, regulations, IP protection, and open research directions.
  • AIGC Foundations: The survey investigates AIGC’s three-layer architecture, enabling technologies, working modes, characteristics, applications, prototypes, and effects on content creation and knowledge representation.
  • Threats and Defenses: It surveys security and privacy threats from security, privacy, trust, and ethics perspectives, while examining defense mechanisms and their feasibility.
  • IP Protection: The paper systematically reviews watermark-based IP protection for AIGC models and content, including threats to watermarking and corresponding countermeasures.
  • Future Directions: The survey identifies future directions involving green, explainable, distributed and scalable, trustworthy and regulatable, and secure-by-design AIGC systems.
  • Paper Organization: The paper is organized into sections on AIGC principles, security and privacy, IP protection and regulation, future research directions, and conclusions.

A. Development Roadmap of AIGC •

AIGC evolved from early computer-synthesized content to practical large-scale generation through advances in generative algorithms, pretrained large models, multimodality, and computing infrastructure.

  • Development Roadmap of AIGC: AIGC progressed from early computer-synthesized music and novels to neural speech translation before the rapid development of generative AI algorithms and large models.The development stage included GANs, transformers, and continuous-video generation, while the flourishing stage began with practical pretrained large models.
  • Development Roadmap of AIGC: The convergence of generative AI algorithms, pretrained large models, and multimodal technologies drives the flourishing of the AIGC paradigm.These technologies support generation of realistic images, videos, and text from learned patterns and data.
  • Development Roadmap of AIGC: Pretrained large models such as ChatGPT enable flexible AIGC services by processing vast text data and producing coherent, naturalistic outputs.ChatGPT also integrates chain-of-thought prompting and reinforcement learning from human feedback for more interactive generation.
  • Development Roadmap of AIGC: Multimodal technology fuses text, images, languages, and other modalities, enriching the diversity of content produced or processed by AI.CLIP is presented as an example that performs text-image association through language and vision analysis.
  • Development Roadmap of AIGC: AIGC shifts content creation beyond professional and user-generated paradigms by enabling large-scale, real-time, low-cost production, including immersive 3D content.The paper describes AIGC as addressing limitations of traditional paradigms in scale, quality, and cost.

D. Paradigm Shift of Knowledge Representation and Usage

Large AI models shift knowledge representation from explicit databases and keyword retrieval toward knowledge encoded in model parameters and accessed through natural language.

  • D. Paradigm Shift of Knowledge Representation and Usage: Knowledge representation developed from relational databases requiring SQL, through Internet search engines using keywords, to large AI models using natural language.The paper presents these as three successive stages of knowledge representation and usage.
  • D. Paradigm Shift of Knowledge Representation and Usage: ChatGPT and GPT-4 represent knowledge as parameters inside large AI models rather than as explicitly stored records.The models incorporate knowledge from the Internet, user input, and other sources.
  • D. Paradigm Shift of Knowledge Representation and Usage: The AIGC-as-a-Service architecture comprises infrastructure, AIGC engine, and AIGC service layers.The basic layer supplies computing power, algorithms, training data, and storage; the engine layer contains multimodal foundation models; the service layer delivers generated content.
  • D. Paradigm Shift of Knowledge Representation and Usage: Foundation models provide general capabilities, while ToB services can fine-tune them on smaller annotated business datasets for specific tasks.The paper notes that one-size-fits-all foundation models may not perform as well on specialized tasks as dedicated models.

F. Working Modes of AIGC

AIGC supports human-assisted and autonomous content creation, combining rapid, low-cost production with diverse multimodal outputs across many content types and applications.

  • F. Working Modes of AIGC: AIGC has two content-generation modes: assisted generation with human intervention and autonomous generation without human intervention.Assisted creation allows humans to edit AI suggestions, whereas autonomous creation produces content entirely by AI.
  • F. Working Modes of AIGC: Human-assisted creation can improve AI-produced content through editing but tends to be slower and more expensive.Autonomous creation is described as faster and cheaper, while output quality depends on the generative AI models.
  • F. Working Modes of AIGC: AIGC offers fast, intensive, and low-cost generation, with algorithms producing dozens or thousands of content pieces simultaneously.The paper attributes these capabilities to rapid information processing, pattern finding, and continuous operation.
  • F. Working Modes of AIGC: AIGC prototypes and applications illustrate the paradigm’s practical use across diverse generated content and services.The paper refers to representative large AIGC models and applications in Table IV.
  • F. Working Modes of AIGC: Multimodal AIGC processes and generates text, images, videos, and audio to create diverse content and more immersive human-machine interactions.Combining modalities can improve scene understanding and support comprehensive experiences.
  • F. Working Modes of AIGC: Applications span text, image, audio, video, 3D, digital-human, and cross-modal generation.Cross-modal examples include text-to-image, image-to-text, text-to-code, text-to-video, and text-to-audio.

III. SECURITY AND PRIVACY THREATS TO AIGC SERVICES

AIGC services face security, privacy, trust, ethical, and legal threats throughout their lifecycle, spanning training data, model behavior, deployment, and generated content.

  • III. SECURITY AND PRIVACY THREATS TO AIGC SERVICES: The survey organizes AIGC risks and countermeasures into security threats, privacy threats, trust issues, and ethical issues.It frames these concerns across the AIGC service lifecycle.
  • Security threats: Training-stage attacks include data poisoning, model poisoning, and adversarial examples that can degrade performance, inject backdoors, or manipulate outputs.Distributed training can be compromised through malicious updates, while crafted input perturbations can produce targeted outputs.
  • Security threats: Sponge examples can increase model latency and energy consumption, compromising AIGC service availability.The paper compares this attack type with denial-of-service attacks in traditional networks.
  • Ethical issues: Biased data and annotations can cause generative models to learn biases affecting fairness across race, gender, age, language, occupation, and geography.Users may experience offense or harm when interacting with biased AIGC models.
  • Trust, privacy, and legal issues: AIGC raises accountability and intellectual-property concerns through personal-data misuse, unauthorized training data, model resale or theft, and unclear ownership of produced content.The paper distinguishes IP issues in data preparation, generative models, and AI-produced contents.
  • Security threats: Deployment-stage threats include model functionality theft, jailbreaks, and prompt injection that can expose model capabilities, bypass safeguards, or trigger unauthorized outputs and actions.Prompt injection can manipulate outputs through malicious or biased prompts, while jailbreaks can elicit illegal or controversial responses.

B. Privacy Threats to AIGC •

AIGC creates privacy risks across data collection, user interaction, model memory, collaborative training, and regulatory compliance. These risks include exposure or reconstruction of sensitive information and difficult-to-reverse retention of personal data.

  • Data Collection: AIGC services collect private user data, multimodal inputs, Internet content, and third-party datasets, creating privacy risks during training and deployment.Sensitive inputs can include private figures and PDF files, while publicly available personal data may also create privacy concerns.
  • Interaction Privacy: ChatGPT may retain conversation histories and multimodal intentions to infer user preferences and profiles, potentially compromising privacy during interaction.The paper cites concerns arising when conversational data are used for further training and profiling.
  • Model Memory: AIGC models’ memorization capabilities expose sensitive training information, including contact and residential details, through data theft from model memory.The paper notes that large models are trained on vast corpora, increasing the significance of memorization-related privacy risks.
  • Collaborative Training: Federated AIGC training can expose participants’ local datasets when an aggregation server analyzes uploaded gradients, discouraging data contribution.Differential privacy and secure aggregation are discussed as countermeasures, but their trade-offs remain important.
  • Trade-offs: Privacy-preserving schemes can degrade model performance or increase resource consumption, leaving utility–privacy trade-offs unresolved.The paper identifies this balance as a crucial problem for further investigation.
  • Compliance and Control: AIGC raises compliance challenges involving data-protection laws, membership inference, prompt theft, and data owners’ right to revoke training authorization.Prompt theft may occur through interception or model inversion, while enforcing the right to be forgotten can require downstream retraining.

C. Trust Issues in AIGC Services •

AIGC services face trust issues because they can produce inaccurate, fabricated, or deceptive content while obscuring how outputs are generated. Their scale and low cost also intensify phishing, malicious-code generation, fraud, dependence, regulatory, and monopoly risks.

  • Content Reliability: ChatGPT can generate plausible-sounding but incorrect, nonsensical, or factually false responses, undermining trust in AIGC services.The paper cites inaccurate Stack Overflow answers and a CBT summary containing overstated effectiveness and false statements.
  • Deception and Misuse: AIGC can cheaply generate large volumes of fake news, impersonations, fabricated identities, phishing emails, and malicious code, making authenticity and safety harder to verify.These capabilities can support fraud, identity theft, malware distribution, and evasion of existing restrictions.
  • Identity Threats: Synthetic images, videos, audio, and text can enable convincing impersonation and fabricated identities that increase online fraud risks.The paper emphasizes the need to verify the authenticity of received media and the identity of interlocutors.
  • Explainability: The black-box operation of commercial AIGC services limits understanding of input interpretation and output generation, complicating assessment of fairness, transparency, and bias.Explainable-AI approaches are presented as a possible direction for inspecting AIGC models.
  • Harm and Fraud: AIGC can generate harmful or violent content and support scalable cyber fraud, including forged voices and automated deceptive conversations.The paper links these risks to harmful training data and increasingly capable low-cost models.
  • Societal and Governance Risks: Overreliance, cross-border supervision, and technology concentration create additional societal and governance concerns for AIGC services.The paper connects overreliance with reduced personal responsibility and identifies jurisdictional coordination and barriers to entry as concerns.

E. Summary and Lessons Learned

AIGC inherits many existing AI security and privacy threats while introducing or intensifying risks linked to its scale, adaptability, memory, and low-cost content generation. The survey therefore frames regulation and protection as requiring broad, integrated responses.

  • Threat Landscape: Most existing AI security and privacy threats persist in AIGC and can become more severe because models can generate enormous poisoned or adversarial samples.The survey presents Fig. 5 as a taxonomy of these threats in the AIGC era.
  • Privacy Risks: Dynamic learning from user inputs and possible memorization of training information create privacy threats such as model-memory theft and interaction leakage.These risks arise alongside broader security concerns across the AIGC service lifecycle.
  • Emerging Risks: AIGC’s fast, intensive, low-cost, and diverse generation introduces newer concerns including intellectual-property issues, jailbreaks, and prompt injection.The survey places these risks alongside ongoing concerns about the authenticity of generated content.
  • Protection and Regulation: The survey reviews IP protection and regulation methods before examining watermark-based solutions, threats to watermarking, and countermeasures.Its structure connects the threat taxonomy to regulatable AIGC and future research directions.

A. IP Protection and Regulation Methods in AIGC

The survey reviews watermarking, cryptography, hardware, blockchain, and legal regulation for protecting AIGC models and content. It emphasizes combined protections, while identifying adaptive, robust, efficient watermarking as an open requirement under continual retraining and evolving attacks.

  • Watermarking: Watermarks act as digital fingerprints for AIGC models and produced content, supporting ownership verification, accountability, and deterrence of unauthorized use.Model watermarks can provide provable ownership evidence during suspected duplication, imitation, or counterfeiting.
  • Complementary Protections: Cryptography, trusted hardware, blockchain, and laws provide complementary mechanisms for access control, confidentiality, integrity, traceability, copyright, and authorized use.Examples include DRM, trusted execution environments, trusted platform modules, distributed copyright systems, and binding legal instruments.
  • Lessons Learned: The survey concludes that protection techniques have distinct advantages and limitations and should be integrated to mitigate IP infringement and unauthorized use.Its recommended direction is an integrated approach that incorporates new and innovative protection and regulation techniques.
  • Model Watermarks: Watermarking AIGC models embeds a vector or activation pattern during or after training and verifies ownership after release against a suspected model.Model watermarking may use white-box parameter access, while data-based methods use trigger sets for black-box verification.
  • Watermarking Taxonomy: Centralized-AI watermarking includes model-based, data-based, and feature-based approaches, while federated-AI research extends watermark verification to black-box and white-box settings.Feature-based methods rely on owner-known features, whereas data-based methods use specific trigger inputs.
  • Model and Content Coverage: Comprehensive IP protection generally requires watermarks on both AIGC models and AI-produced content, potentially combining techniques for stronger protection.The survey identifies model and content watermarking as the two main AIGC watermarking aspects.
  • Open Challenges: Frequent retraining and adaptive attacks require lower-cost embedding and verification, greater watermark robustness and security, and mechanisms that scale to large-scale AIGC production.Continually updated training data make adaptive model watermark embedding especially important.

C. Threats and Countermeasures to AIGC Watermarking

AIGC watermarking faces attacks that distort, remove, overwrite, forge, or extract watermarks from models and generated content. The survey reviews these threats and identifies robustness limitations, including unresolved collusion attacks.

  • Watermark distortion modifies model parameters to increase the distance between embedded vectors and corresponding parameters.
  • Permutation and scaling attacks target model parameters, but permutation can reduce performance while scaling preserves outputs and often leaves parameter signs unchanged.
  • Fine-tuning, pruning, retraining, and data enhancement can remove embedded or content-enabled watermarks, although most surveyed methods resist fine-tuning and pruning.
  • Reverse engineering, model distillation, and model extraction threaten model-based watermarks by removing backdoors or replicating model functionality.
  • Ambiguity attacks forge additional watermarks to cast doubt on ownership, while watermark overwriting replaces an original watermark with a new one.
  • Watermark collusion remains an open problem because federated clients may cooperate to remove, alter, replace, steal, or counterfeit watermarks.

V. FUTURE RESEARCH DIRECTIONS

The paper identifies several future research directions for AIGC. These directions target environmental efficiency, interpretability, scalability, trustworthiness, personalization, and security.

  • Future AIGC research is organized around multiple aspects requiring further investigation.

A. Green AIGC Architectures

Future AIGC architectures must address high energy consumption, limited interpretability, and scalability constraints. Proposed directions include greener computing, explainability, and distributed cloud-edge systems, each with unresolved trade-offs or deployment challenges.

  • A. Green AIGC Architectures: ChatGPT’s energy consumption for answering one question for 590 million users equals the monthly electricity usage of 175,000 Danes.
  • A. Green AIGC Architectures: Green AIGC architectures can combine cloud-edge computing, specialized hardware, energy-efficient algorithms, and model compression to reduce lifecycle energy demands.
  • B. Explainable AIGC Models: Black-box AIGC models lack transparency and interpretability, raising concerns about biased outcomes, security threats, and ethical issues.
  • B. Explainable AIGC Models: Explainable AIGC can use interpretable algorithms, visualization techniques, and human-in-the-loop approaches to provide explanations for model outputs.
  • B. Explainable AIGC Models: Explainable AIGC must balance accuracy, interpretability, and security while establishing frameworks to evaluate interpretability.
  • C. Distributed & Scalable AIGC Algorithms: Centralized AIGC inference may create high latency and low scalability as workloads rise, motivating distributed architectures that parallelize tasks.
  • C. Distributed & Scalable AIGC Algorithms: Cloud-edge cooperation assigns complex general-purpose tasks to cloud foundation models and real-time processing to dedicated edge models.
  • C. Distributed & Scalable AIGC Algorithms: Distributed cloud-edge AIGC still faces insufficient edge security resources, difficult resource allocation, heterogeneous quality of service, and device compatibility problems.

D. Trustworthy & Regulatable AIGC Services

Trustworthy and regulatable AIGC services must address misinformation, authenticity assessment, legal compliance, personalization, and security. The survey discusses auditing, blockchain, watermarking, personalized modeling, and security-by-design, while noting data and feedback challenges.

  • AIGC can produce disinformation, misinformation, deepfakes, and propaganda, making authenticity evaluation difficult and increasing the need for legal compliance.
  • Explainable AI, auditing, blockchain, watermarking, and regulatory compliance frameworks are proposed to improve trustworthiness and regulatability.
  • Personalized AIGC services adapt responses to users’ profiles, behavior, location, mood, preferences, and needs.
  • Intention modeling dynamically updates user profiles, goals, intents, and preferences from multimodal prompts and user feedback to select suitable responses.
  • Personalization requires substantial high-quality data, which can be expensive and privacy-sensitive, while noisy or adversarial feedback can degrade performance and robustness.
  • Security-by-design integrates security into AIGC architecture, with adversarial robustness supported by robust optimization and adversarial training.
  • The paper highlights jailbreaks, data theft, deepfakes, and biased or harmful content as security-related consequences of AIGC expansion.
  • The survey reviews AIGC principles, threats, trust and ethical issues, regulatory solutions, solution limitations, and future directions.
Loading 2305.18339v2…