Source-linked AI summary
ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text
Sandra Mitrović, Davide Andreoletti, Omran Ayoub
TL;DR
The paper asks whether short human and ChatGPT-generated reviews can be distinguished and what linguistic features drive the decisions. It combines Transformer-based classification with SHAP explanations across custom-query and rephrased-review experiments, finding that rephrased text is harder to detect while the ML approach reaches 0.79 accuracy in Experiment 2.
Problem
The study addresses the need to distinguish human from ChatGPT-generated text, especially short reviews whose misuse can affect information trustworthiness and businesses.
Method
The paper trains a Transformer-based classifier, compares it with perplexity-based classification, and uses SHAP to explain individual model decisions.
Results
Rephrased reviews are harder to distinguish: ML accuracy falls from 0.98 in Experiment 1 to 0.79 in Experiment 2, while remaining higher than perplexity-based accuracy in both experiments.
Takeaways & Limitations
SHAP explanations indicate that ChatGPT reviews tend to be formal, polite, impersonal, general, and less expressive of feelings than human reviews.
Takeaways & Limitations
The explanations are data dependent, and broader analysis is needed before generalizing ChatGPT-versus-human writing-style distinctions in every detail.
Abstract
from arXiv · showhide
ChatGPT has the ability to generate grammatically flawless and seemingly-human replies to different types of questions from various domains. The number of its users and of its applications is growing at an unprecedented rate. Unfortunately, use and abuse come hand in hand. In this paper, we study whether a machine learning model can be effectively trained to accurately distinguish between original human and seemingly human (that is, ChatGPT-generated) text, especially when this text is short. Furthermore, we employ an explainable artificial intelligence framework to gain insight into the reasoning behind the model trained to differentiate between ChatGPT-generated and human-generated text. The goal is to analyze model's decisions and determine if any specific patterns or characteristics can be identified. Our study focuses on short online reviews, conducting two experiments comparing human-generated and ChatGPT-generated text. The first experiment involves ChatGPT text generated from custom queries, while the second experiment involves text generated by rephrasing original human-generated reviews. We fine-tune a Transformer-based model and use it to make predictions, which are then explained using SHAP. We compare our model with a perplexity score-based approach and find that disambiguation between human and ChatGPT-generated reviews is more challenging for the ML model when using rephrased text. However, our proposed approach still achieves an accuracy of 79%. Using explainability, we observe that ChatGPT's writing is polite, without specific details, using fancy and atypical vocabulary, impersonal, and typically it does not express feelings.
1 Introduction
The paper examines whether short restaurant reviews can be distinguished as human- or ChatGPT-generated and why this detection matters. It targets potential harms from undetected AI-generated reviews in sensitive and commercial contexts.
- ChatGPT capabilities: ChatGPT combines broad query handling with highly realistic, human-like text generation across diverse domains.Its training incorporated human-and-chatbot conversations to learn patterns and nuances of human language.
- Motivation: Detecting AI-generated text supports information trustworthiness and can help address misinformation, disinformation, and social-engineering attacks.The paper also identifies political campaigns, financial reports, legal documents, and customer reviews as sensitive applications.
- Study focus: The study focuses on restaurant reviews of no more than two sentences, where limited verification and short length make detection especially relevant and challenging.False reviews could damage high-quality businesses or improve low-quality businesses, potentially harming clients.
- Research aims: The authors aim to evaluate ML-based discrimination between human and ChatGPT text and identify text characteristics driving model decisions.The study uses publicly available human reviews and manually collected ChatGPT-generated reviews.
2 Detecting ChatGPT-generated Text
The paper formulates detection as supervised binary classification and pairs a Transformer-based classifier with explainable AI analysis. It asks how well ML and perplexity approaches detect short AI-generated text, including rephrased human text, and which features influence decisions.
- 2.1 Problem Statement and Research Questions: The task learns a function mapping text features to a binary label: 1 for ChatGPT-generated text and 0 otherwise.The model also supports analysis of which features contribute positively to detecting ChatGPT-generated text.
- 2.2 Methodology: A benchmark uses perplexity, motivated by the idea that lower scores indicate text more predictable by language models.The benchmark estimates a classification threshold from GPT-2 perplexity scores.
- 2.1 Problem Statement and Research Questions: The study evaluates detection performance for short AI-generated text and for text produced by rephrasing original human-generated text.A second research question examines whether model explanations reveal characteristics associated with AI-generated text.
- 2.2 Methodology: The approach combines a machine learning discriminator with an explainable AI framework for interpreting its outcomes and comparing human and ChatGPT writing styles.These are the study’s two main building blocks.
- 2.2 Methodology: The classifier uses a pre-trained Transformer-based sequence-classification model fine-tuned on collected data to detect ChatGPT-generated samples.The implementation uses uncased DistilBERT with its corresponding tokenizer.
- 2.2 Methodology: SHAP assigns feature-importance values to input words or phrases, providing local explanations for individual classification decisions.These explanations identify which text parts most influenced a model output.
3 Datasets and Experimental Setup
The experiments compare human restaurant reviews with two ChatGPT datasets: one generated from custom queries and one created by rephrasing human reviews. They evaluate both ML and perplexity-based classification on these two settings.
- 3.1 Datasets: The study uses one human-generated dataset and two ChatGPT-generated restaurant-review datasets.The ChatGPT datasets are intended to be made publicly available.
- 3.1 Datasets: The human dataset contains 1000 restaurant reviews, with reviews shorter than 11 words removed.Original sentiment labels are discarded because they are unnecessary for this detection task.
- 3.1 Datasets: The ChatGPTquery dataset contains 395 reviews produced from diverse custom queries targeting aspects such as food cost, quality, and review length.The varied queries were used to diversify ChatGPT outputs.
- 3.1 Datasets: The ChatGPTrephrase dataset contains 1000 reviews, each produced by asking ChatGPT to rephrase a human-generated review.This construction preserves a direct relationship with the original human reviews.
- 3.1 Datasets: The reviews are mostly short texts across the three datasets, aside from a minor number of longer ChatGPTquery cases.Review length is measured by number of words.
- 3.2 Experimental Setup: Experiment 1 compares human reviews with custom-query ChatGPT reviews, while Experiment 2 compares human reviews with rephrased ChatGPT reviews.The experiments are designed to test the discriminating power of the proposed ML model.
- 3.3 Benchmark Approach: Perplexity-based Classification: The benchmark classifies texts using a GPT-2 perplexity threshold estimated from training-set scores.Samples above the threshold receive the human label, while lower-perplexity samples are treated as more likely language-model output.
4 Quantitative Results and Explanations
The Transformer-based classifier substantially outperformed perplexity-based detection, but rephrased reviews remained harder to classify. SHAP explanations linked predictions to stylistic cues including pronouns, feelings, unusual vocabulary, politeness, and impolite expressions.
- Quantitative results: 0.98 and 0.79 accuracy were achieved by the ML approach in Experiments 1 and 2, respectively, versus 0.84 and 0.69 for perplexity-based classification.Experiment 2 used ChatGPT-rephrased human reviews and was harder than Experiment 1, which used reviews generated from custom queries.
- Quantitative results: 84 versus 34 average perplexity separated human and ChatGPT-generated texts in Experiment 1, while Experiment 2 averages were 84 versus 47.The rephrased ChatGPT texts also had a lower maximum perplexity than the human texts, reducing detectability.
- SHAP explanations: Personal pronouns and feeling-related words contributed toward human-generated classifications in Experiment 1.Examples included “We,” “me,” “our,” “feels like,” “felt sick,” and “hate.”
- SHAP explanations: Words such as “stand out feature,” “incredibly polite,” “waitstaff,” and “knowledgeable” contributed positively toward ChatGPT-generated classifications.The study reports these words as uncommon in human restaurant reviews.
- SHAP explanations: Aggressive and impolite language, including “sucks” and “shoe leather,” contributed toward human-generated classifications.The model associated rude vocabulary with human writing rather than ChatGPT-generated text.
- Experiment 2: In Experiment 2, similar stylistic patterns appeared less often because rephrasing constrained ChatGPT’s word choice and writing style.Misclassifications included ChatGPT text containing personal or food-related words that contributed toward human classifications.
5 Discussion
Rephrased reviews are harder for the classifier to distinguish from human writing, although accuracy remains around 79%. SHAP analysis identifies recurring stylistic patterns but requires broader validation.
- Rephrased reviews are more challenging for the ML model to distinguish than reviews generated from custom queries.
- 79% accuracy was achieved despite the greater difficulty of detecting rephrased reviews.
- Rephrasing existing reviews can help avoid detection in short texts and scale the production of potentially misleading reviews.
- ChatGPT writing was characterized as impersonal, repetitive, and generally lacking personal feelings, while often describing experiences.
- The authors caution that SHAP explanations are data dependent and require more extensive analysis before generalizing across writing styles.
6 Conclusion
The paper uses a Transformer-based model to detect ChatGPT-generated short text and SHAP explanations to examine its decisions. The results support ML-based detection while revealing stylistic differences between ChatGPT and human writing.
- The study builds a Transformer-based model to distinguish human-written from ChatGPT-generated short texts.
- Traditional perplexity-based methods do not give good results for this detection task.
- The model discriminates better when ChatGPT text is generated from customer queries rather than rephrased from human texts.
- SHAP explanations suggest that ChatGPT writing is extremely polite but lacks some profoundness of human language, including irony and metaphors.
- Future work will consider alternative ML models, different domains, and different query types.