PaperlayerView exact revision on arXiv

Source-linked AI summary

ChatGPT: More than a Weapon of Mass Deception, Ethical challenges and responses from the Human-Centered Artificial Intelligence (HCAI) perspective

Alejo Jose G. Sison, Marco Tulio Daza, Roberto Gozalo-Brizuela, Eduardo C. Garrido-Merchán

arXiv:2304.11215v1cs.CYcs.AI

TL;DR

The paper examines ethical risks from ChatGPT, especially its use as a weapon of mass deception and as an enabler of deceptive criminal activity. Using the Human-Centered Artificial Intelligence framework, it assesses the system’s capabilities and limitations and proposes technical and non-technical responses aimed at reducing misuse while supporting human wellbeing.

  • Problem

    ChatGPT’s rapid adoption creates ethical risks because its outputs can support deception, academic misconduct, misinformation, and criminal activity.

  • Method

    The paper uses the Human-Centered Artificial Intelligence framework to assess ChatGPT’s technical potentials, limitations, ethical challenges, and mitigation measures.

  • Results

    The paper identifies ChatGPT’s use as a weapon of mass deception as its principal danger and recommends technical and non-technical measures to reduce misuse.

  • Takeaways & Limitations

    Proper use of ChatGPT requires human responsibility, oversight, and attention to individual and social wellbeing.

  • Takeaways & Limitations

    Watermarking can be removed from open-source models, erased through paraphrasing or altered metadata, and may compromise algorithmic performance.

Abstract

from arXiv · show

This article explores the ethical problems arising from the use of ChatGPT as a kind of generative AI and suggests responses based on the Human-Centered Artificial Intelligence (HCAI) framework. The HCAI framework is appropriate because it understands technology above all as a tool to empower, augment, and enhance human agency while referring to human wellbeing as a grand challenge, thus perfectly aligning itself with ethics, the science of human flourishing. Further, HCAI provides objectives, principles, procedures, and structures for reliable, safe, and trustworthy AI which we apply to our ChatGPT assessments. The main danger ChatGPT presents is the propensity to be used as a weapon of mass deception (WMD) and an enabler of criminal activities involving deceit. We review technical specifications to better comprehend its potentials and limitations. We then suggest both technical (watermarking, styleme, detectors, and fact-checkers) and non-technical measures (terms of use, transparency, educator considerations, HITL) to mitigate ChatGPT misuse or abuse and recommend best uses (creative writing, non-creative writing, teaching and learning). We conclude with considerations regarding the role of humans in ensuring the proper use of ChatGPT for individual and social wellbeing.

1. Introduction

The paper frames ChatGPT’s rapid adoption and ethical controversy through Human-Centered Artificial Intelligence (HCAI), focusing on human wellbeing, agency, and responsible governance. It examines technical capabilities and limitations alongside technical and non-technical responses to misuse.

  • 1. Introduction: ChatGPT’s rapid adoption and financial enthusiasm have developed alongside ethical concerns including bias, privacy, misinformation, and job displacement.The paper notes that ChatGPT reached 100 million users within two months and that generative-AI investment expanded rapidly.
  • 1. Introduction: HCAI serves as the paper’s framework because it treats technology as a means to empower human agency and places human wellbeing among its central challenges.The framework also emphasizes responsible design, privacy, governance and oversight, and respect for human cognitive capacities.
  • 1. Introduction: The paper investigates ChatGPT’s technical specifications to assess its potentials, limitations, and safeguards against misuse or abuse.Its stated focus is on understanding the technology in order to support human flourishing and wellbeing.
  • 1. Introduction: The article focuses on ChatGPT’s ethical challenges as a potential weapon of mass deception and proposes technical and non-technical mitigation measures.The planned discussion covers watermarking, detectors, fact-checking, terms of use, transparency, educator considerations, and human oversight.
  • 1. Introduction: The authors limit their evidence base because ChatGPT’s rapid, unsettled development has left relatively few peer-reviewed studies, requiring reliance on specialized news and technical blogs.They also narrow the analysis to ChatGPT while integrating technical and non-technical ethical concerns around human and social wellbeing.

2. The ethical challenges posed by ChatGPT as a kind of generative AI

The paper presents generative AI as a source of ethical risks spanning privacy, bias, employment, public discourse, manipulation, and deception. It emphasizes that ChatGPT can scale academic misconduct, disinformation, and criminal activity because humans can deploy its fluent outputs for deceit.

  • 2. The ethical challenges posed by ChatGPT as a kind of generative AI: Generative AI raises ethical concerns involving foundational questions, privacy and copyright, bias, employment and automation, and manipulation of social media and public discourse.The paper describes outputs across text, code, images, music, video, and human-like voices, broadening the contexts in which these concerns arise.
  • 2. The ethical challenges posed by ChatGPT as a kind of generative AI: ChatGPT’s low-cost production of human-sounding text and user-friendly natural-language interface make large-scale deceptive communication easier to produce and distribute.The paper connects these characteristics to disinformation, fake reviews, impersonation, propaganda, lobbying, and trolling.
  • 2. The ethical challenges posed by ChatGPT as a kind of generative AI: ChatGPT can be used as a weapon of mass deception through academic misconduct, disinformation and impersonation, and criminal activities involving malware.The paper distinguishes human misuse from intentional deception by the model itself.
  • 2. The ethical challenges posed by ChatGPT as a kind of generative AI: Overdependence on ChatGPT can deskill users, potentially stymieing the capacity for innovation.The paper presents this concern alongside the broader importance of preserving human competence and cognitive capacities.
  • 2. The ethical challenges posed by ChatGPT as a kind of generative AI: Academic deception involves misrepresenting ChatGPT-generated work as one’s own while failing to develop or demonstrate the competence that academic work is meant to assess.The paper separates this ethical problem from whether the output is original or whether machine output can be plagiarized.
  • 2. The ethical challenges posed by ChatGPT as a kind of generative AI: Passing tests does not establish practical competence, as illustrated by reported strengths in some examinations alongside failures in elementary mathematics.The paper notes that ChatGPT performed near passing across the three steps of the U.S. Medical Licensing Exam without implying that it should receive a medical license.
  • 2. The ethical challenges posed by ChatGPT as a kind of generative AI: ChatGPT can assist cybercriminal activity by generating malware for phishing, unauthorized access, ransom encryption or decryption, and dark-web marketplaces.The paper presents these uses as forms of criminal activity involving deceit.
  • 2. The ethical challenges posed by ChatGPT as a kind of generative AI: The paper criticizes deployment decisions that prioritize business opportunities over safe, reliable, and secure AI design, highlighting manipulation risks and proposed high-risk regulatory treatment.It discusses criticism of Microsoft’s Bing deployment and European lawmakers’ proposal concerning text-generation systems without human oversight.

3. Analytical description of generative AI models

The paper analytically describes neural networks, VAEs, GANs, GPT, and BERT to clarify how generative AI produces content and why its outputs raise ethical concerns.

  • 3.1. Artificial neural networks and deep learning: Deep neural networks transform inputs through layered parameterized weights and nonlinear activations, with parameters optimized by backpropagation and stochastic optimization.Their hyperparameters regulate data handling, model capacity, and learning behavior.
  • 3.2.1. Variational Auto Encoder (VAE):: VAEs compress observable inputs into lower-dimensional latent representations and reconstruct approximations, enabling similar nondeterministic outputs from one prompt.The paper contrasts this with standard autoencoders, where small mapping changes can strongly alter reconstructions.
  • 3.2.2. Generative Adversarial Network (GAN):: GANs generate apparently similar data through a minimax game between a generator that seeks discriminator errors and a discriminator that seeks accurate classification.The adversarial optimization estimates dataset distribution parameters and produces new points such as images.
  • 3.3. Large language models: GPT models generate text autoregressively by predicting the next word from preceding context, whereas BERT learns bidirectional representations from left and right contexts.GPT’s unidirectional design differs fundamentally from BERT’s joint conditioning on both sides of a word.
  • 3.3.1. Generative Pretrained Transformers (GPT):: Because GPT output depends on statistical training data and prompts, biased data can be reproduced and prompts can deliberately generate misinformation.The paper therefore links technical behavior to the need for fair training and content verification.

4. Engaging with the ethical challenges of ChatGPT

The paper frames ChatGPT’s ethical response as a two-pronged effort combining technical and non-technical measures. Because no combination can completely prevent deception, the aim is to reduce harm across the system’s operational stages.

  • 4. Engaging with the ethical challenges of ChatGPT: Technical and non-technical measures should target model construction, model access, content dissemination, and belief formation.Examples include watermarking during construction and detectors or fact-checking during dissemination.

4.1. Technical reasons

The paper surveys technical tools for mitigating ChatGPT-enabled deception, including provenance signals, AI-style identification, detectors, and fact-checking. It emphasizes that these tools are contextual aids rather than definitive safeguards.

  • 4.1.1. Statistical watermarking: Statistical watermarking can embed an unnoticeable secret signal in generated text to support provenance identification.The watermark is intended to remain robust when words are inserted, removed, or rearranged.
  • 4.1.1. Statistical watermarking: Watermarking is limited because open-source models, AI paraphrasing, altered metadata, and performance compromises can remove or weaken its value.The paper also notes that ChatGPT’s proprietary status affects applicability.
  • 4.1.2. Identifying AI style: An AI styleme could provide a unique, indelible linguistic fingerprint distinguishing generated text from human writing, but other AI systems could defeat it.The proposed fingerprint would itself most likely depend on AI-based detection.
  • 4.1.3. ChatGPT detectors: ChatGPT detectors analyze textual properties or model probabilities, but their variable reliability means they should not be sole decision-making tools.GPTZeroX uses perplexity and burstiness, while DetectGPT identifies generated text through negative-curvature regions in log probability.
  • 4.1.3. ChatGPT detectors: OpenAI’s classifier correctly identified 26 percent of machine-generated English texts and falsely labeled 9 percent of human texts, with poorer reliability for short, non-English, predictable, or out-of-distribution texts.Its design prioritizes a low false-positive rate by marking text as machine-written only when highly confident.
  • 4.1.4. Verification and fact-checking: Fact-checking websites, search engines, and expert sources can support verification, with the paper recommending comparison across at least two independent sources.Named resources include Factmata, Snopes, Fact Check Explorer, and PolitiFact.

4.2. Non-technical reasons

The paper proposes non-technical measures spanning policies, transparency, education, and human oversight to reduce ChatGPT-enabled deception and misuse.

  • 4.2. Non-technical reasons: These measures draw on law, ethics, psychology, and human-AI interaction principles to target actors, behaviors, content, and belief formation.The paper summarizes the non-technical resource set as terms and moderation, transparency, supervised educator use, and human oversight.
  • 4.2.1. Enforce terms of use, content moderation, safety & overall best practices: Terms of use, content moderation, safety practices, and usage policies are presented as primary measures against deceptive and malicious ChatGPT applications.They address academic abuse, disinformation, impersonation, rights violations, and some criminal uses, although implementation remains difficult.
  • 4.2.2. Transparency: Transparency requires explaining ChatGPT’s capabilities and limitations because human-like responses may contain fabricated or inaccurate information.The paper emphasizes that users should verify outputs rather than assume that fluent responses are reliable.
  • 4.2.3. Educator considerations: Educators should supervise age-, educational-level-, and domain-appropriate ChatGPT use while teaching accuracy checks, AI literacy, and responsible disclosure.The paper supports limited uses such as initiating research or improving essay language, provided students disclose the assistance.
  • 4.2.4. Humans in the loop: Human-in-the-loop arrangements provide accountability, oversight, and supervision across model construction, access, and content dissemination.Developers should test responses, restrict users or functions when necessary, and establish procedures for handling misuse.

4.3. ChatGPT fails in responsible design, privacy, human-centered design principles, and appropriate governance and oversight

The paper argues that ChatGPT falls short across several HCAI grand challenges, including responsible design, privacy, human-centered design, and governance.

  • Responsible design: ChatGPT remains largely opaque and requires stronger responsibility allocation, fairness, accuracy, robustness, and liability arrangements.The paper specifically identifies jailbreak susceptibility and unresolved gaps in liability and retribution for private and criminal harms.
  • Privacy: ChatGPT insufficiently respects privacy, with conversations potentially infringing personal rights and causing psychological unease or harm.The cited rights include control over personal information, secrecy, personhood, and intimacy.
  • Human-centered design principles: The system does not adequately follow human-centered design because it poorly calibrates risks and prioritizes AI objectives over human and societal wellbeing.The paper calls for greater attention to preserving users’ dignity and agency while augmenting their experience.
  • Appropriate governance and oversight: ChatGPT has not been adequately subjected to governance and oversight aligned with regulatory standards and certifications.The paper highlights insufficient attention to fairness, integrity, resilience, and explainability.

4.4. Best uses for ChatGPT

ChatGPT is presented as useful for creative writing, routine writing assistance, and teaching and learning, provided humans critically assess its outputs. Its strengths include stylistic flexibility, broad text-generation capabilities, and support for educational activities, while its lack of grounding makes verification necessary.

  • 4.4.1. Tool for creative writing: ChatGPT supports creative writing through brainstorming, counterfactual exploration, style transformation, and generation of texts in an author’s style.Its combinatory power enables imaginative outputs, but it can mislead users who cannot assess whether the generated content is accurate or appropriate.
  • 4.4.2. Tool for non-creative writing: ChatGPT assists with routine writing tasks, including grammar correction, summarization, copy editing, advertising, social media, and coding support.These uses are most appropriate when users have enough expertise to evaluate the outputs.
  • 4.4.3. Tool for teaching and learning: ChatGPT can complement human creativity and critical thinking when used as a companion rather than a replacement for human partners.The paper frames its educational and writing applications as assistance that should preserve human judgment and cognitive participation.
  • 4.4.3. Tool for teaching and learning: In education, ChatGPT can support lesson planning, scripts, lectures, student questions, critical-thinking exercises, and writing-quality benchmarks under human supervision.Educational materials may be passable but require verification because information can be incorrect, uninsightful, or insufficiently analytical.
  • 4.4. Best uses for ChatGPT: The paper groups best uses into creative writing, non-creative writing, and teaching and learning.Examples include brainstorming and style transformation; spelling, summarization, copy editing, and coding; and lesson preparation, critical-thinking assessment, language tutoring, and writing benchmarks.

4.5. ChatGPT and human cognitive abilities

The paper evaluates ChatGPT through a human-centered view of human-AI interaction that distinguishes competition, supplementation, interdependence, and collaboration. It favors uses that facilitate work while respecting human cognitive capacities rather than displacing them.

  • 4.5. ChatGPT and human cognitive abilities: HCAI frames desirable human-AI interaction as facilitating work while respecting human cognitive capacities.The paper distinguishes four interaction modes: competing, supplementing, interdependence, and full collaboration.

5. ChatGPT and human wellbeing

The paper argues that ChatGPT’s central ethical danger is believable but potentially false communication, which humans may exploit for deception and criminal activity. HCAI therefore places responsibility on humans to verify outputs and use the system in ways that support wellbeing.

  • 5. ChatGPT and human wellbeing: HCAI is used to assess ChatGPT as a technology intended to empower and augment human agency while supporting reliable, safe, trustworthy AI and human wellbeing.The paper applies this framework to identify ethical problems and propose responses to ChatGPT’s use.
  • 5. ChatGPT and human wellbeing: ChatGPT’s greatest danger is human use of the system as a weapon of mass deception, not intentional deception by the model itself.The model can produce erroneous data because of its training and algorithmic limitations, while users remain responsible for caution about outcomes.
  • 5. ChatGPT and human wellbeing: Because ChatGPT communicates without being aligned with truth or veracity, convincing falsehoods can undermine trust and cause widespread harm.The paper contrasts human truth-oriented communication with ChatGPT’s inability to distinguish fiction from non-fiction or truth from falsehood.
  • 5. ChatGPT and human wellbeing: Humans must verify and fact-check ChatGPT outputs, especially in scientific research, because the system is not grounded in reality and cannot perform this verification itself.The responsibility for distinguishing appearance from reality therefore remains with users.
  • 5. ChatGPT and human wellbeing: The paper reports that GPT-4’s scaling produced apparently incremental improvements while hallucinations and reasoning errors persisted.The authors state that their major findings therefore remain valid, although future work must verify them.
Loading 2304.11215v1…