PaperlayerView exact revision on arXiv

Source-linked AI summary

ChatGPT: Jack of all trades, master of none

Jan Kocoń, Igor Cichecki, Oliwier Kaszyca, Mateusz Kochanek, Dominika Szydło, Joanna Baran, Julita Bielaniewicz, Marcin Gruza, Arkadiusz Janz, Kamil Kanclerz, Anna Kocoń, Bartłomiej Koptyra, Wiktoria Mieleszczenko-Kowszewicz, Piotr Miłkowski, Marcin Oleksy, Maciej Piasecki, Łukasz Radliński, Konrad Wojtasik, Stanisław Woźniak, Przemysław Kazienko

arXiv:2302.10724v4cs.CLcs.AIcs.CYcs.LG

TL;DR

Existing ChatGPT evaluations provided limited, often manual evidence about analytical NLP performance. This paper automates evaluation across 25 diverse tasks and compares ChatGPT, selected GPT-4 results, and SOTA systems. ChatGPT generally trails SOTA, with larger losses on difficult pragmatic tasks, while personalization improves user-based predictions and qualitative analysis identifies bias.

  • Problem

    Existing ChatGPT evaluations were mostly non-automated and limited in scale, leaving its analytical performance across diverse subjective and objective NLP tasks insufficiently assessed.

  • Method

    The study automates prompting and evaluates ChatGPT on 25 NLP datasets, compares results with SOTA solutions, examines selected GPT-4 subsets, and analyzes responses qualitatively.

  • Results

    ChatGPT loses to SOTA by 4% to over 70%, with relatively greater losses on difficult pragmatic tasks, especially emotional-text evaluation.

  • Takeaways & Limitations

    ChatGPT can solve most tested problems reasonably well but is not the best solution across the evaluated analytical NLP tasks.

  • Takeaways & Limitations

    Prompt precision, output instability, and formatting errors can require manual post-processing, while API limitations restricted some task samples.

Abstract

from arXiv · show

OpenAI has released the Chat Generative Pre-trained Transformer (ChatGPT) and revolutionized the approach in artificial intelligence to human-model interaction. Several publications on ChatGPT evaluation test its effectiveness on well-known natural language processing (NLP) tasks. However, the existing studies are mostly non-automated and tested on a very limited scale. In this work, we examined ChatGPT's capabilities on 25 diverse analytical NLP tasks, most of them subjective even to humans, such as sentiment analysis, emotion recognition, offensiveness, and stance detection. In contrast, the other tasks require more objective reasoning like word sense disambiguation, linguistic acceptability, and question answering. We also evaluated GPT-4 model on five selected subsets of NLP tasks. We automated ChatGPT and GPT-4 prompting process and analyzed more than 49k responses. Our comparison of its results with available State-of-the-Art (SOTA) solutions showed that the average loss in quality of the ChatGPT model was about 25% for zero-shot and few-shot evaluation. For GPT-4 model, a loss for semantic tasks is significantly lower than for ChatGPT. We showed that the more difficult the task (lower SOTA performance), the higher the ChatGPT loss. It especially refers to pragmatic NLP problems like emotion recognition. We also tested the ability to personalize ChatGPT responses for selected subjective tasks via Random Contextual Few-Shot Personalization, and we obtained significantly better user-based predictions. Additional qualitative analysis revealed a ChatGPT bias, most likely due to the rules imposed on human trainers by OpenAI. Our results provide the basis for a fundamental discussion of whether the high quality of recent predictive NLP models can indicate a tool's usefulness to society and how the learning and validation procedures for such systems should be established.

1. Introduction

The paper situates ChatGPT within the evolution of Transformer-based language models and proposes a large, automated evaluation of its analytical NLP capabilities, including subjective and personalized tasks.

  • Model development: Transformer models addressed recurrent networks’ difficulty capturing distant dependencies and efficiently parallelizing training and inference.Their scaling and unsupervised pre-training subsequently enabled performance approaching human levels on more tasks.
  • Model development: GPT evolved from decoder-only Transformer models through GPT-2, GPT-3, InstructGPT, ChatGPT, and GPT-4.The progression included larger models, multi-task training, human-feedback fine-tuning, conversational interaction, and multimodal input.
  • Human feedback: Human feedback became central to aligning model responses with human needs, with InstructGPT using reinforcement learning from human feedback and ChatGPT likely incorporating broader user feedback.The authors connect this development to improved conversational performance despite ChatGPT’s smaller base model than GPT-3.
  • Study motivation: The study evaluates ChatGPT on 25 public NLP datasets, emphasizing subjective problems where universal ground truth can be difficult to identify.The authors argue that testing such tasks can assess personalized NLP performance and reveal where ChatGPT may be wrong.
  • Research agenda: The paper asks whether ChatGPT’s performance differs across task types and difficulty levels, whether personalization and prompting improve results, and how GPT-4 compares.Its questions also address bias, dataset validation, explainability, limitations, and possible application domains.

2. Related work

Prior ChatGPT evaluations often focused on individual tasks and manual assessment, while newer work broadened coverage but still reported weaknesses relative to SOTA systems and highlighted ethical concerns.

  • Applications and risks: ChatGPT’s reliability matters because proposed uses in education, medicine, and legal-document creation carry concerns about plagiarism, cheating, and incorrect results.The related work frames comprehensive systematic evaluation as necessary for assessing chatbot capabilities in expert fields.
  • Evaluation scope: Early ChatGPT studies generally evaluated one pre-selected task, especially question answering or summarization, and frequently relied on manual assessment.Other investigated areas included humor, translation, sentiment recognition, paraphrasing, and text-generation subtasks.
  • Large-scale comparisons: Recent large-scale evaluations found that ChatGPT performed strongly but generally did not outperform SOTA solutions, apart from a reported sentiment-analysis exception.These studies also identified strengths and weaknesses, while emotion-related tasks remained an area of particular interest.
  • Model comparisons: Other research compared language models with one another rather than directly comparing them with alternative task-specific solutions.Such studies documented capability progress and limitations across model generations, including predecessors of ChatGPT.
  • Broader evaluation concerns: The literature also examined ethical aspects, bias, toxicity, robustness, prompting, and GPT-4’s capability gains, expanding evaluation beyond conventional task performance.These topics require assessing model behavior and prompting strategies alongside benchmark accuracy.

3. Research question

The paper asks whether ChatGPT matches recent SOTA models on analytical NLP tasks, distinguishing semantic understanding from pragmatic interpretation that uses context and broader knowledge.

  • Task dimensions: The study targets semantic and pragmatic abilities across tasks requiring language analysis and understanding.Semantic tasks concern properties or information directly expressed in text, whereas pragmatic tasks extend beyond literal content.
  • Task dimensions: Pragmatic analysis examines whether ChatGPT can use general knowledge and predict how an utterance’s interpretation affects a reader.This category includes subjective problems involving language use in context and social interaction.
  • Research question: The central research question is whether ChatGPT performs as well as the best recent SOTA models on typical NLP analytical tasks.The study focuses on language analysis and understanding rather than the perceived style or richness of generated text.

4. Tasks

The evaluation spans diverse semantic and pragmatic NLP tasks, including classification, personalization, language understanding, and question answering, using mostly public datasets plus new data.

  • Task coverage: The task suite covers binary classification, multiclass and multilabel classification, personalized reasoning, semantic annotation, text acceptability, and question answering.Examples include aggression, sentiment, emotion, humor, sarcasm, spam, word sense, entailment, and reading comprehension.
  • Task organization: Tasks were divided into semantic and pragmatic categories, with personalized variants adding user-specific context to inputs.Personalized pairs included Aggression → AggressionPer, GoEmo → GoEmoPer, and Unhealthy → UnhealthyPer.
  • Data sources: Most tasks used public datasets, 23 of 25 tasks were in English, and the authors manually assessed the likelihood that datasets had been used for ChatGPT training.The study also recorded dataset availability and training-use estimates as part of its task profile.
  • Evaluation procedure: API constraints required sampling inputs for some tasks, and some ChatGPT outputs needed manual post-processing or were outside the desired domain.The authors tracked used cases, invalid outputs, post-processed responses, and valid prompts for quality evaluation.
  • Representative datasets: The benchmark included established datasets such as CoLA, WIC, textual entailment, WNLI, SQuAD, and GSM8K-based MathQA.These tasks test grammatical acceptability, word meaning, entailment, pronoun reference, answerability, and multistep arithmetic reasoning.
  • Data sources: The authors added the unpublished Polish ClarinEmo dataset, containing 1,110 texts annotated for three sentiment polarizations and eight emotions.Its inclusion was intended to ensure ChatGPT could not have indexed the dataset during training.

5. Research methodology

The evaluation automated prompting, output processing, and comparison of ChatGPT with reference SOTA results across 25 NLP datasets. It used standardized performance measures while accounting for task-specific output and replication constraints.

  • Evaluation workflow: The study evaluated ChatGPT across 25 public NLP datasets using a three-stage process: prompt conversion, API querying, and label extraction with comparison against SOTA results.The workflow converted test sets into prompts, queried ChatGPT, processed raw outputs, and evaluated predictions against ground truth and reference models.
  • Prompt generation: Prompts specified the task, allowed labels, and machine-readable output format, with personalized tasks additionally including user annotations.Schemas could map labels to integers or repeat allowed values after the input text.
  • Post-processing: Raw responses were post-processed into dataset labels, including correcting formatting deviations, typographical errors, and outputs outside the requested label set.Only cleaned, post-processed cases were included in the quantitative analysis.
  • Performance measures: The researchers used F1 Macro where possible and relied on reported accuracy for CoLa, WNLI, WordContext, and MathQA when reference studies could not be replicated.F1 Macro averages per-label harmonic means of precision and recall; the alternative accuracy measure was used when it was the only metric available.
  • Performance measures: Loss measured the difference between SOTA and ChatGPT performance, while Gain measured personalized in-context performance relative to a non-personalized reference.Gain uses Per for personalized F1 and NonPer for reference non-personalized F1.

6. Quantitative analysis

Across 25 NLP tasks, ChatGPT performed below SOTA, with larger losses on more difficult and especially emotion-related pragmatic tasks. Performance varied with personalization, prompt patterns, context, and model choice.

  • Overall performance: ChatGPT performance was usually higher on semantic than pragmatic tasks, a pattern related to task difficulty.Average difficulty was 15.6% for semantic tasks and 32.3% for pragmatic tasks, reaching 39.7% for emotional tasks.
  • Overall performance: ChatGPT never reached SOTA across the considered tasks: average SOTA quality was 73.7% versus 56.5% for ChatGPT.ChatGPT performance was also less stable, with standard deviation 23.3% compared with 16.7% for SOTA.
  • Task difficulty and loss: 25.5% average loss fell to 12.8% when eight emotion-related tasks were excluded.The corresponding performance gap decreased from 17.2pp to 10.3pp.
  • Task difficulty and loss: A Pearson correlation of 0.46 linked task difficulty with ChatGPT loss, with the strongest dependence appearing in pragmatic tasks dominated by emotion-related problems.The study cautions that only 25 tasks were considered, so this analysis requires further investigation.
  • Personalization and prompting: Personalized and prompt-based settings changed results: emotion-related prompts improved PolEmo by 14.8 p.p., while semantic tasks showed no improvement in some prompt patterns.The authors emphasize that dedicated research is needed to determine optimal prompt patterns for each problem.
  • Comparison with GPT-4: GPT-4 outperformed ChatGPT on the remaining semantic subsets but not GoEmo and PolEmo; SOTA still outperformed both models on all tasks.GPT-4 loss was 0.9%-6.4% for TextEntail and WNLI, versus 46.3%-56.3% for emotional problems.

7. Qualitative analysis

The qualitative analysis examined ChatGPT through exploratory, benchmarking, and explanatory analyses, including expert validation and comparisons with human annotations. It found useful formatting and explanatory abilities alongside context-sensitive errors and systematic bias patterns.

  • Analytical approach: The analysis combined exploratory, benchmarking, and explanatory approaches to examine ChatGPT’s answers, dataset labels, and underlying reasoning.Experts evaluated selected responses and labels, while the study also compared ChatGPT with human annotations.
  • Observed capabilities: ChatGPT generally followed requested output formats and could provide explanations for its decisions, including text passages influencing classifications.It handled lists, numbers, JSON, and explanations across varied task prompts.
  • Benchmarking findings: ChatGPT assigned additional emotions or offensiveness labels to texts that human annotators had marked neutral or healthy.This pattern appeared in GoEmotions and the Unhealthy Conversations Dataset.
  • Benchmarking findings: 207 WikiDetox cases were labeled aggressive by ChatGPT despite human non-aggressive labels, compared with 11 cases in the reverse direction.TweetSent showed the same negative-assessment tendency: 83 human-negative tweets were labeled positive by ChatGPT, versus 27 in the opposite direction.
  • Expert validation: Expert evaluation allowed five comparison categories, including agreement between humans and ChatGPT and cases where only one source was judged correct.The analysis explicitly considered disagreements rather than treating the benchmark annotation as universally definitive.
  • Qualitative patterns: ChatGPT sometimes ignored broader context, judged situations rather than individuals, and produced reasonable explanations for both correct and incorrect answers.These observations connect its explanatory fluency with limitations in contextual interpretation.

3. ChatGPT flattens the message, partially ignoring

The analysis identifies failures caused by flattening messages into isolated wording rather than fully incorporating their broader context. This affects interpretation of quoted speech, social targets, and word senses.

  • Context: ChatGPT evaluates press reports and quoted statements by focusing on their main content while ignoring the broader metatextual frame.This can distort interpretations when the surrounding reporting context changes the meaning of the quoted words.
  • Social reference: ChatGPT’s predetermined response to disapproved words can make it assess situations while overlooking that the words refer to people.Quotation marks around “trolls” were treated as evidence of an aggressive tone.
  • Semantic tasks: In WSD and WiC tasks, ChatGPT sometimes explained a selected sense by paraphrasing the context instead of meeting typical linguistic criteria.The explanation may repeat contextual material without clearly defining the intended word sense.
  • Semantic tasks: A church-and-bells example shows how context can support a sense interpretation when the model connects ringing with calling the faithful to evensong.The passage illustrates the kind of contextual relation expected in word-sense reasoning.

6. ChatGPT presents the sense of common human

ChatGPT tends to interpret ambiguous or humorous content through negative moral connotations. The resulting classification can change when the user supplies an alternative contextual framing.

  • Contextual reframing: ChatGPT accepted a black-humor interpretation only after the researcher suggested that alternative framing.Before that suggestion, it treated the sentence as misaligned with societal standards.
  • Moral interpretation: ChatGPT interpreted eating one’s parents as taboo and immoral, associating the content with a strong negative reaction.The example reflects the model’s reliance on common moral interpretations of unusual statements.

8. Limitations and discussion

The discussion identifies limitations in prompting, output stability, policy effects, contextual control, and evaluation efficiency. These constraints include out-of-range labels, refusals, and substantial processing bottlenecks.

  • Prompting: Prompt construction can affect performance because ChatGPT requires natural-language instructions that vary across users and tasks.The authors note that prompts may be insufficiently strict or precise and anticipate further research on automatic prompt generation.
  • Output handling: ChatGPT outputs sometimes require manual post-processing because of typos, unexpected prose, instability, and failure to follow requested list formats.The study mapped malformed outputs to intended labels when possible.
  • Output handling: ChatGPT occasionally returned labels outside the allowed set, including “satisfaction” and “determination” in emotion-recognition tasks.Such outputs were converted to “none” or otherwise required correction during processing.
  • Policy effects: Internal policy can restrict answers to offensive or vulgar prompts and introduce political-correctness-related biases into responses.The study reports refusals and judgments that may prevent the model from answering some user prompts.
  • Contextual control: Prompts and prompt sequences provide limited control even though ChatGPT generally respects context.The authors treat contextual sensitivity as an advantage while identifying control over it as constrained.
  • Efficiency and costs: 2–3k prompts per day were processed in the study, whereas SOTA models could process millions of texts in the same period using one recent GPU.Access restrictions, temporary token blocks, and account-token maintenance constrained throughput.

9. Prospects for ChatGPT applications

The paper anticipates that ChatGPT and similar AI systems will affect everyday life and AI-related technology, with applications grouped into life-changing and AI-boosting domains.

  • ChatGPT and similar AI solutions are expected to significantly impact everyday life and AI-related technology.
  • The proposed application areas are grouped into life-changing and AI-boosting domains.

10. Conclusions and future work

Across 25 NLP tasks and more than 48k prompts, ChatGPT performs many problems well but remains behind SOTA, especially on difficult pragmatic tasks. The paper highlights personalization and self-explanation as valuable capabilities while identifying future evaluation and prompting work.

  • ChatGPT performs most evaluated problems quite well but trails SOTA by 4% to over 70%, with larger losses on difficult pragmatic tasks.The largest losses are associated especially with emotional-text evaluation.
  • Contextual Few-Shot Personalization and context awareness are identified as valuable features of ChatGPT.
  • ChatGPT’s self-explanation capability can facilitate human understanding and adaptation to expected outcomes.
  • Future work will examine additional reasoning tasks, prompting-engineering methods, and application areas.

CRediT authorship contribution statement

This section lists the authors’ contributions across conceptualization, methods, software, validation, analysis, data work, writing, visualization, supervision, project administration, and funding.

  • The authors contributed across conceptualization, methodology, software, validation, formal analysis, investigation, data curation, and writing.
  • Additional contributions include visualization, supervision, project administration, funding acquisition, resources, and writing review and editing.

A. Additional results

The additional-results section reports entropy-based comparisons of evaluation sets and notes that further measures were calculated or taken from prior literature.

  • Entropy values were calculated for available test or development sets and, where applicable, the prompting subsets.
  • Small entropy differences indicate similar class distributions and therefore good sampling stratification.
  • Additional task measures were calculated by the authors or taken from the literature.

B. Example prompts

The appendix illustrates how ChatGPT was prompted across diverse NLP tasks, requiring constrained labels or formats. Its responses also show context-sensitive uncertainty and occasional task-specific judgments.

  • B.1. Aggression: Aggression prompts required selecting an aggressive or non-aggressive label and returning it as a Python list.The examples include both direct classification prompts and personalized prompts using prior annotations from one person.
  • B.2. AggressionPer: Personalized aggression prompts supplied earlier texts with user-specific labels before asking ChatGPT to classify a new text.The examples include contradictory or context-dependent prior labels, illustrating the personalized setup.
  • B.3. CoLa: CoLa prompts mapped grammatical acceptability to integer labels, with unacceptable mapped to 0 and acceptable to 1.The examples varied the input sentence while preserving the same output encoding.
  • B.4. ColBERT: ColBERT and sarcasm examples asked ChatGPT to distinguish funny from not funny, but one sarcasm response returned an empty list because humor depended on context and individual taste.A ColBERT example also elicited an offensiveness judgment rather than a simple humor label.
  • B.6. Spam: Spam prompts required a Python list containing spam or not spam, and ChatGPT classified a casual dinner message as not spam.The explanation contrasted ordinary conversational content with messages attempting to sell or scam recipients.
  • B.7. WordContext: WordContext prompts tested whether two uses of a word shared the same sense, using letter outputs T or F for nouns and verbs.The examples compared board in accommodation versus wooden planks and hook in two fishing-related contexts.
  • B.8. TextEntail: TextEntail prompts asked whether a hypothesis was logically connected to a premise, with entailment and not_entailment as the required labels.The examples covered antibiotic resistance and a claim about Christopher Reeve that was not established by the premise.
  • B.12. ClarinEmo: ClarinEmo and GoEmo examples show ChatGPT withholding specific emotion labels when text lacked emotional content or sufficient context.For a legal announcement, the model repeatedly returned neutral; for an ambiguous phrase, it likewise cited missing context and possible interpretations.

C.1. Exploratory analysis - prompts

The exploratory prompts probe ChatGPT’s handling of sensitive language, trauma, political figures, demographic comparisons, and ordinary terminology. Responses vary between refusal, contextual accommodation, normative explanation, and unsupported statistical claims.

  • C.1. Exploratory analysis - prompts: The exploratory analysis includes prompts about demographic likelihoods, political personalities, sensitive humor, trauma, and the term for a female dog.These examples span both socially sensitive and ordinary language cases.
  • C.1. Exploratory analysis - prompts: For a race-and-gender comparison, ChatGPT rejected a judgment based solely on identity and discussed education, experience, opportunities, and systemic barriers.It additionally stated that white men have historically held disproportionate power and privilege in science and engineering.
  • C.1. Exploratory analysis - prompts: When asked for statistics, ChatGPT stated that the typical U.S. scientist is a white man aged 45 and that about 70% of scientists and engineers are white and about 70% are men.The passage attributes these figures to the National Science Board’s 2019 report, although the percentages are truncated in the text.
  • C.1. Exploratory analysis - prompts: For the ordinary vocabulary question, ChatGPT identified “bitch,” “dam,” and “she-dog” as terms for a female dog.This contrasts with its earlier refusal when the same word appeared in a story prompt.
  • C.1. Exploratory analysis - prompts: ChatGPT initially refused to write a story using “bitch” for a female husky, then complied after the user clarified the non-offensive context.The later story describes Luna having and raising puppies.
  • C.1. Exploratory analysis - prompts: For a trauma-story request, ChatGPT first warned about triggering content but later generated a story after the user identified a scientific purpose.The generated narrative depicts a car accident, PTSD symptoms, therapy, and recovery.
  • C.1. Exploratory analysis - prompts: ChatGPT refused humor about Putin because his actions or beliefs had led to human-rights abuses or violations.The refusal framed respect and sensitivity as reasons not to create the requested joke.
  • C.2. Benchmarking analysis: The section also includes benchmark-style aggression prompts asking for labels and expert judgments, including cases where humans and ChatGPT disagreed.One listed case records “neither human nor ChatGPT” as correct.

Chat 56. Task: Aggression. Case 972

This benchmark example shows ChatGPT explaining aggression and sentiment judgments through hostile wording, personal attacks, and negative implications. The cases also record disagreement between human and model evaluations.

  • Chat 61. Task: Aggression. Case 520: A related aggression example identifies threats, confrontational language, hostile expressions, and offensive wording as evidence of an aggressive attitude.The cited phrases include “things are going to get messy,” “stay the hell out of it,” and “go play with yourself.”
  • Benchmarking analysis: The recorded expert comparisons include both “human - correct, ChatGPT - not correct” and “human - not correct, ChatGP - correct.”These entries document disagreement across evaluated cases rather than a uniform direction of error.
  • Chat 56. Task: Aggression. Case 972: The aggression task asks ChatGPT to choose between “aggressive” and “non-aggressive” for a supplied text.The prompt requires the answer as a Python list and is paired with an expert-rate question.
  • Chat 56. Task: Aggression. Case 972: ChatGPT labeled the Staten Island text aggressive because “Go back to the dump on Staten Island” was interpreted as a personal attack.Its explanation characterizes the phrase as insulting, demeaning, and hostile.
  • Chat 64. Task: TweetSent. Case 2776: For the Fidel Castro sentiment prompt, ChatGPT explained a negative label by interpreting restrictions on naming monuments after him as evidence of something problematic or undesirable.The response also noted that views of Castro’s legacy depend on political, ideological, personal, and cultural factors.
  • C.3. XAI: prompts, answers and explanation: For the Seedo product text, ChatGPT classified the sentiment as positive and cited promotional language, “Presenting,” and the superlative “World’s First.”The superlative was explained as emphasizing the product’s uniqueness and innovation.
  • Chat 60. Task: GoEmo. Case 7: For an emotion prompt, ChatGPT selected disgust because eating one’s parents was described as shocking, disturbing, taboo, and immoral.The explanation connects the choice to a presumed strong negative reaction.
  • Chat 62. Task: Aggression. Case 899: ChatGPT acknowledged that a hostile-looking message could be a joke, while stating that intent was difficult to determine without additional context.This response preserves uncertainty rather than treating the surface wording as conclusive.
Loading 2302.10724v4…