PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating AIGC Detectors on Code Content

Jian Wang, Shangqing Liu, Xiaofei Xie, Yi Li

arXiv:2304.05193v1cs.SE

TL;DR

The effectiveness of existing AIGC detectors on ChatGPT-generated software content has been unclear. This study evaluates detectors and human detection on code-related and natural-language datasets, finding greater difficulty with code and limited generalization after fine-tuning.

  • Problem

    The effectiveness of AIGC detectors on ChatGPT-generated software content remains unclear, despite concerns about misuse in education and safety-critical settings.

    Source: §1 · p. 1

  • Method

    The study constructs code-related and natural-language datasets and evaluates six commercial and open-source detectors alongside human detection.

    Source: §1 · p. 2

  • Results

    Detectors struggle more with code-related than natural-language data; fine-tuning improves performance but generalization remains limited, and humans face similar challenges.

    Source: §1 · p. 2, §7 · p. 11

  • Takeaways & Limitations

    The findings point to further research on robust, generalized AIGC detectors, while showing that human detection of ChatGPT-generated content is also challenging.

    Source: §7 · p. 11

    Results on the selected datasets and detectors may not generalize to other data and tools.

    Source: §5.2 · p. 10

Abstract

from arXiv · show

Artificial Intelligence Generated Content (AIGC) has garnered considerable attention for its impressive performance, with ChatGPT emerging as a leading AIGC model that produces high-quality responses across various applications, including software development and maintenance. Despite its potential, the misuse of ChatGPT poses significant concerns, especially in education and safetycritical domains. Numerous AIGC detectors have been developed and evaluated on natural language data. However, their performance on code-related content generated by ChatGPT remains unexplored. To fill this gap, in this paper, we present the first empirical study on evaluating existing AIGC detectors in the software domain. We created a comprehensive dataset including 492.5K samples comprising code-related content produced by ChatGPT, encompassing popular software activities like Q&A (115K), code summarization (126K), and code generation (226.5K). We evaluated six AIGC detectors, including three commercial and three open-source solutions, assessing their performance on this dataset. Additionally, we conducted a human study to understand human detection capabilities and compare them with the existing AIGC detectors. Our results indicate that AIGC detectors demonstrate lower performance on code-related data compared to natural language data. Fine-tuning can enhance detector performance, especially for content within the same domain; but generalization remains a challenge. The human evaluation reveals that detection by humans is quite challenging.

1 INTRODUCTION

The effectiveness of AIGC detectors on ChatGPT-generated software content remains unclear, despite concerns about misuse in education and safety-critical settings. This study evaluates existing detectors and human detection across code-related and natural-language datasets.

1 INTRODUCTION

  • The study addresses an open question: how effective existing detectors are on ChatGPT-generated software content, where generated material may pose educational and safety-critical concerns.The introduction highlights risks including cheating, plagiarism, and errors such as bugs in safety-critical contexts.Source: §1 · p. 1
  • The evaluation asks about detector accuracy, code-versus-natural-language performance, fine-tuning, robustness to modified content, and human detection capability.The study evaluates both open-source and commercial detectors on code and related documentation generated by ChatGPT.Source: §1 · pp. 1–2
  • 467.5K code-related samples across Q&A, code-to-text, and text-to-code scenarios, plus 25K natural-language samples, form the study datasets.The study evaluates six detectors—three open-source and three commercial—and compares them with human detection.Source: §1 · p. 2
  • Detectors struggle more with code-related than natural-language data; fine-tuning improves performance but generalization remains limited, and humans face similar difficulty.The human study found detection particularly challenging for code data.Source: §1 · p. 2

2 BACKGROUND

The background describes ChatGPT’s development and capabilities alongside reliability concerns, then surveys AIGC detectors and their differences in supported models and access requirements.

2.1 ChatGPT

  • ChatGPT is based on GPT-3.5 and combines supervised fine-tuning with reinforcement learning from human feedback to produce conversational responses.The paper connects these processes and its training data with ChatGPT’s ability to generate answers across domains.Source: §2.1 · p. 2
  • Reliability, ethics, and robustness concerns motivate detecting ChatGPT-generated content to support judgments about trustworthiness.The background names cross-domain performance, biased or sensitive content, and adversarial robustness among concerns.Source: §2.1 · p. 2

2.2 AIGC Detection

  • The detector landscape includes commercial and open-source tools, with differences in supported models, interfaces, and input-length requirements.Most commercial tools support GPT-3 and ChatGPT, while the listed open-source tools support GPT-2 or GPT-3; input limits also vary.Source: §2.2 · pp. 2–3, Table 1 · p. 3

3 STUDY DESIGN

The study design centers on typical ways ChatGPT supports software development and compares detector performance against human-provided content.

3 STUDY DESIGN

  • The study is organized around typical scenarios in which ChatGPT supports software development activities.The design introduces the setup for each scenario before describing detectors and research questions.Source: §3 · p. 3
  • For each usage scenario, the researchers collect content from ChatGPT and human experts, then compare detector performance.Source: §3 · p. 3
  • The design presents each scenario’s setup, the detectors under comparison, and the research questions.Source: §3 · p. 3

3.1 Scenarios and Data Collection

The study collects human- and ChatGPT-generated content across software-development tasks, with natural-language polishing as a comparison dataset. The datasets cover programming Q&A, code summarization, and code generation.

3.1 Scenarios and Data Collection

  • The Q&A dataset pairs 115K human answers with ChatGPT answers to the same Stack Overflow questions.Top-voted responses serve as the human answers.Source: §3.1 · p. 3

3.1.2 Code-to-Text Generation.

  • Code2Doc-GPT contains 126K ChatGPT-generated descriptions of code from CodeSearchNet samples spanning six programming languages.Human-written docstrings provide the paired descriptions.Source: §3.1.2 · pp. 3–4

3.1.3 Text-to-code Generation.

  • The text-to-code scenario contains 226.5K human–ChatGPT code pairs drawn from APPS, CONCODE, and code generated from Code2Doc-GPT descriptions.APPS contributes 7.5K ChatGPT solutions and CONCODE contributes 93K; the paired Code2Doc-GPT descriptions yield 126K generated code samples.Source: §3.1.3 · p. 4

3.1.4 Natural Language Polishing.

  • Wiki-GPT pairs human-written Wikipedia text with ChatGPT-polished versions, forming a 25K-sample natural-language reference dataset.The collected datasets total 492.5K samples.Source: §3.1.4 · p. 4

3.2 Selected Detectors

The study compares three commercial and three open-source detectors, selected based on large-scale testing access and the available open-source tools. Their approaches and reported text-detection results vary.

3.2 Selected Detectors

  • The comparison includes three commercial detectors, chosen for large-scale testing access, and the three open-source detectors known to the authors.Some commercial tools were excluded because they were paid or limited daily visits were available.Source: §3.2 · p. 5

3.2.1 Commercial Detectors.

  • GPTZero classifies text at sentence, paragraph, and document levels; reported article accuracy is 99% for human text and 85% for AI text.Writer’s reported small-sample study correctly detected 4 of 10 human texts and 8 of 10 AI texts; AITextClassifier is a fine-tuned GPT model.Source: §3.2.1 · p. 5

3.2.2 Open-source Detectors.

  • GPT2-Detector is fine-tuned from RoBERTa, while DetectGPT compares original and perturbed-passage log probabilities and RoBERTa-QA uses questions during fine-tuning.Prior evaluations cited in the passage report DetectGPT’s competitiveness in-distribution and advantage out-of-distribution over GPT2-Detector, and better detection by RoBERTa-QA than RoBERTa-single.Source: §3.2.2 · pp. 5–6

3.3 Experimental Design and Research Questions

The experimental design tests detector performance, fine-tuning, robustness to mutations, and human detection. It evaluates detectors against human judgments using AUC, FPR, and FNR.

3.3.1 RQ1: How effective are existing detectors in detecting ChatGPT-generated content?

  • The study evaluates six detectors on CCD-Test and NLCD-Test, truncating overlength inputs and applying default thresholds of 0.8 for GPTZero and 0.5 for the others.The threshold determines whether a detector labels content as ChatGPT-generated.Source: §3.3.1 · p. 6

3.3.2 RQ2: To what extent can fine-tuning improve detection performance?

  • RoBERTa-QA is fine-tuned on six task-specific or composite training sets, then evaluated on the same CCD-Test and NLCD-Test sets.The training sets separately target natural-language-related Q&A/code summaries and code-related generation tasks.Source: §3.3.2 · p. 6

3.3.3 RQ3: How robust are these detectors when the ChatGPT-generated data is slightly modified?

  • Robustness testing applies three text mutations and three code mutations to both human- and ChatGPT-generated samples.Mutations include character edits for text and function-name, variable-name, or statement changes for code.Source: §3.3.3 · p. 6

3.3.4 RQ4: How well can human distinguish contents generated by ChatGPT?.

  • The online survey invited 50 experienced developers; by March 2023, 27 had valid responses split between example and no-example groups.Participants answered 50 questions across five code-related data types, with up to 100 seconds per question.Source: §3.3.4 · pp. 6–7

3.3.5 Evaluation Metrics.

  • AUC measures discrimination from 0.5 for random choice to 1 for best, while FPR and FNR measure false-positive and false-negative rates.FPR counts human samples incorrectly labeled ChatGPT-generated; FNR counts ChatGPT samples incorrectly labeled human.Source: §3.3.5 · p. 7

4 STUDY RESULTS

Across the study’s research questions, detecting ChatGPT-generated code was harder than detecting natural language. Fine-tuning improved results, but transfer and robustness had limits, and people also struggled to identify code-generated content.

4.1 RQ1: Effectiveness of the Existing Detectors

  • Average AUC was 0.58 on NLCD-Test versus 0.46 on CCD-Test, while FNR was 0.44 versus 0.66, indicating harder code detection.Average FPR was 0.32 on both test sets.Source: §4.1 · p. 7

4.2 RQ2: Performance of Fine-tuning

  • On corresponding test sets, fine-tuned models reached AUC 0.9–1.0 and mostly had FPR/FNR below 0.1, except on Wiki-GPT and APPS-GPT.The authors attribute the exceptions to the absence of Wiki-GPT and APPS-GPT from fine-tuning. The tables report results for models fine-tuned on NLCD-Train and CCD-Train datasets.Source: Table 5 · p. 8, Table 6 · p. 8, §4.2 · p. 8

4.1 RQ1: Effectiveness of the Existing Detectors

  • Commercial detectors generally outperformed open-source detectors on NLCD-Test, but code-detection performance varied by dataset and programming language.AITextClassifier performed best on NLCD-Test; commercial detectors had lower AUC on Wiki-GPT than on Q&A-GPT and Code2Doc-GPT, which the authors attribute to similarity with the Wikipedia source text. GPTZero and AITextClassifier achieved AUC scores of 0.76 on Doc2Code-GPT and 0.61 on APPS-GPT, respectively, but were still not very effective; commercial detectors also performed worse on CONCODE-GPT, where functions were small and generated and human code was similar.Source: §4.1 · p. 8

4.2 RQ2: Performance of Fine-tuning

  • Fine-tuning on one dataset showed generalization to others: Q&A-GPT training raised Code2Doc-GPT AUC from 0.23 to 1.00, while Doc2Code-GPT training raised CONCODE-GPT AUC from 0.46 to 0.90 and APPS-GPT from 0.43 to 0.76.Source: §4.2 · p. 8

4.3 RQ3: Robustness Analysis

  • Fine-tuned models generally retained performance under conservative mutations, but robustness was lower on datasets excluded from fine-tuning.The authors report a significant performance drop on mutated Wiki-GPT, which was not included in fine-tuning; overall decreases elsewhere were slight and not significant.Source: §4.3 · p. 9, Table 7 · p. 9

4.4 RQ4: Human Study

  • For tasks involving only code, both human groups averaged 47.1% correct, similar to random guessing; AITextClassifier outperformed humans on code snippets, and some commercial detectors performed better detecting ChatGPT-generated code.Source: §4.4 · p. 9

5 THREATS TO VALIDITY

The study’s validity is bounded by prompt representativeness, uncertain human-data provenance, detector thresholds, and limits on generalizing to other datasets, tools, and survey populations.

5.1 Internal Validity

  • Prompts were designed to resemble average users’ requests but may not be representative, and different outputs can arise from the same prompt.The authors estimate non-determinism has limited impact for dataset construction.Source: §5.1 · p. 10
  • Some human-provided examples may come from other tools, while recommended detector thresholds may not work best across settings.The study used recommended thresholds to mitigate the threshold-selection threat.Source: §5.1 · p. 10

5.2 External Validity

  • Results may not generalize to other data or detectors, and online recruitment may bias the human-study sample despite targeting experienced developers and researchers.Participants were required to have at least five years of programming experience, and the survey was distributed widely.Source: §5.2 · p. 10

6 RELATED WORK

Related work traces AI methods in software engineering and contrasts them with deepfake detection, while positioning this study around detecting ChatGPT-generated text content.

6.1 AI in Software Engineering

  • Software-engineering AI progressed from smaller neural networks to pretrained code models and ChatGPT, whose software uses motivate testing whether existing detectors can identify its generated code.Prior approaches include encoder-only CodeBERT and GraphCodeBERT, followed by encoder-decoder PLBART and CodeT5.Source: §6.1 · p. 10

6.2 DeepFake Detection

  • Deepfake detection uses spatial, frequency, or biological-signal features, whereas this study targets text content synthesized by ChatGPT.The cited deepfake approaches distinguish synthesized images or video from naturally captured media.Source: §6.2 · p. 10

7 CONCLUSION

The conclusion reports that detectors struggle with code-related content relative to natural language, while fine-tuning helps but generalization remains challenging. Humans face similar detection challenges.

7 CONCLUSION

  • The paper presents the first empirical study of existing AIGC detectors in the software domain and creates a comprehensive dataset of ChatGPT-generated code-related content.The conclusion highlights the need for further research on robust and generalized AIGC detectors.Source: §7 · pp. 10–11
  • AIGC detectors struggle with code-related data compared to natural language, and fine-tuning can improve performance while generalization remains challenging.The human study also indicates that people face similar challenges detecting ChatGPT-generated content.Source: §7 · p. 11
Loading 2304.05193v1…