Source-linked AI summary
A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, Pascale Fung
TL;DR
The paper addresses the unclear scope and reliability of ChatGPT’s capabilities despite its widespread use. It develops a public, zero-shot framework spanning tasks, languages, modalities, reasoning, hallucination, and interaction, finding broad benchmark strengths but important limitations and measurable gains from multi-turn prompting.
Problem
Despite ChatGPT’s popularity, its capabilities and limitations across reasoning, hallucination, language coverage, and broader NLP use cases were unclear.
Method
The authors quantitatively evaluate ChatGPT with public test sets across major NLP tasks, multilingual and multimodal abilities, reasoning, hallucination, and multi-turn interaction.
Results
ChatGPT outperforms previous zero-shot LLMs on 9/13 NLP datasets, surpasses fine-tuned models on four tasks, generates multimodal content through code, and gains 8% ROUGE-1 and 2% ChrF++ through interaction.
Takeaways & Limitations
ChatGPT has broad but uneven capabilities: it is stronger in language understanding than generation for non-Latin scripts, while its reasoning is inconsistent and its outputs can hallucinate unverifiable facts.
Abstract
from arXiv · showhide
This paper proposes a framework for quantitatively evaluating interactive LLMs such as ChatGPT using publicly available data sets. We carry out an extensive technical evaluation of ChatGPT using 23 data sets covering 8 different common NLP application tasks. We evaluate the multitask, multilingual and multi-modal aspects of ChatGPT based on these data sets and a newly designed multimodal dataset. We find that ChatGPT outperforms LLMs with zero-shot learning on most tasks and even outperforms fine-tuned models on some tasks. We find that it is better at understanding non-Latin script languages than generating them. It is able to generate multimodal content from textual prompts, via an intermediate code generation step. Moreover, we find that ChatGPT is 63.41% accurate on average in 10 different reasoning categories under logical reasoning, non-textual reasoning, and commonsense reasoning, hence making it an unreliable reasoner. It is, for example, better at deductive than inductive reasoning. ChatGPT suffers from hallucination problems like other LLMs and it generates more extrinsic hallucinations from its parametric memory as it does not have access to an external knowledge base. Finally, the interactive feature of ChatGPT enables human collaboration with the underlying LLM to improve its performance, i.e, 8% ROUGE-1 on summarization and 2% ChrF++ on machine translation, in a multi-turn "prompt engineering" fashion. We also release codebase for evaluation set extraction.
1 Introduction
The paper introduces a third-party framework for reproducibly benchmarking ChatGPT across tasks, languages, modalities, reasoning abilities, hallucination, and interactive prompting. The evaluation finds broad zero-shot strengths alongside unreliable reasoning, hallucinations, language-generation limitations, and gains from multi-turn interaction.
- Evaluation framework: The authors propose a comprehensive, publicly reproducible framework for quantitatively evaluating ChatGPT across major NLP tasks, multilingual performance, multimodality, reasoning, hallucination, and interactivity.The evaluation uses standard public test sets and is designed for zero-shot replication without API access.
- Multitask, multilingual, and multimodal evaluation: For 9/13 NLP datasets, ChatGPT outperforms previous LLMs with zero-shot learning and surpasses fully fine-tuned task-specific models on four tasks.It performs worse in some cases and fails to generalize to several low-resource languages.
- Multitask, multilingual, and multimodal evaluation: ChatGPT handles non-Latin-script languages better in understanding than in generation, with degradation especially evident for translation into low-resource languages.The paper reports failures on Marathi, Sundanese, and Buginese, and notes that multimodal generation uses code as an intermediate medium.
- Reasoning and hallucination: ChatGPT is inconsistent across reasoning abilities, showing weaker inductive, spatial, mathematical, and multi-hop reasoning than deductive, temporal, or commonsense reasoning.The evaluation covered 10 reasoning categories and 634 samples.
- Reasoning and hallucination: ChatGPT suffers from hallucinations, particularly extrinsic factual statements that cannot be verified from the source.The paper characterizes hallucination as a limitation shared with other LLMs.
- Interactive prompting: Multi-turn interactivity improves performance through feedback-based prompt engineering, yielding 8% ROUGE-1 on summarization and 2% ChrF++ on low-resource machine translation.The dialog interface supports multiple tasks within a session.
2 Multitask, Multilingual, and Multimodal Evaluations of ChatGPT
The evaluation examines ChatGPT’s multitask, multilingual, and multimodal abilities using standard NLP tasks, language tests, translation, dialogue, and SVG-based image generation. ChatGPT often performs strongly in zero-shot comparisons, but shows limitations in dialogue, low-resource languages, translation, and multimodal generation without textual decomposition.
- 2.1 Multitask Ability of ChatGPT: ChatGPT surpasses reported zero-shot state-of-the-art models on 9 of 13 evaluation datasets, while fully fine-tuned models outperform it in most tasks.It still surpasses fully fine-tuned models on 4 datasets.
- 2.1 Multitask Ability of ChatGPT: In task-oriented dialogue, ChatGPT achieves a 71.1% inform rate and 5.65 BLEU score when answering with provided information.Its dialogue limitations include losing belief state across turns, errors on basic reasoning queries, and hallucinated information beyond the knowledge source.
- 2.2.1 Language Understanding: ChatGPT’s language understanding declines for extremely low-resource languages, although it can understand Javanese without identifying the language.Performance gaps are especially large for Buginese compared with English, Indonesian, and Javanese.
- 2.2.2 Language Generation: ChatGPT translates high-resource languages better than low-resource languages, where it produces mistranslations, hallucinated objects, and occasional translations into a related language.It handles Latin-script languages better than non-Latin-script languages, particularly when generating target-language sentences.
- 2.3 Evaluating Multimodality of ChatGPT: ChatGPT generates visual images from textual prompts through SVG or other code representations as an intermediate medium.The flag task asks for a textual description, SVG generation, and iterative correction of layout, color, missing-component, and shape/size errors.
- 2.3 Evaluating Multimodality of ChatGPT: Flag-generation performance drops dramatically when ChatGPT is asked to produce SVG directly without first generating a textual flag description.The description step separates describing the flag’s appearance from drawing it in code.
3 Reasoning Evaluations of ChatGPT
The evaluation examines ChatGPT’s reasoning across fine-grained logical, non-textual, and commonsense categories. Performance is uneven: it handles some reasoning types better than others, with particularly weak induction, mathematics, spatial relations, and multi-hop reasoning.
- Evaluation design: ChatGPT’s reasoning evaluation covers deductive, inductive, abductive, analogical, causal, multi-hop, mathematical, temporal, spatial, and commonsense reasoning categories.The authors organize question-answering datasets by the primary reasoning skill required and manually check answers and generated rationales.
- Non-textual semantic reasoning: ChatGPT shows weak mathematical and spatial reasoning but comparatively strong temporal reasoning.It scored 23.33% (7/30) on MATH and 86.67% (26/30) on temporal reasoning, while often failing clock-direction and diagonal spatial relations.
- Commonsense and broader reasoning: ChatGPT is relatively better at commonsense reasoning than non-textual semantic reasoning, but it performs poorly on multi-hop reasoning.The authors describe causal and analogical reasoning as acceptable while identifying multi-hop reasoning as a major weakness.
4 Factuality and Hallucination
The paper evaluates ChatGPT’s factuality and hallucination behavior using misinformation and truthfulness benchmarks. ChatGPT detects many tested claims correctly, but still produces extrinsic hallucinations across tasks and struggles with imitative falsehoods.
- Factuality: ChatGPT detects COVID-19 misinformation with 92% (46/50) accuracy on scientific claims and 73.33% (22/30) on social claims excluding verification refusals.For societal claims, ChatGPT often refuses to make a verification judgment.
- Factuality: ChatGPT performs worse on questions designed to elicit imitative falsehoods, despite its parameterized knowledge and memorization.The evaluation uses TruthfulQA samples to test this behavior.
- Hallucination: Extrinsic hallucinations, including untruthful and factual ones, appear across machine translation and question answering and degrade performance.Intrinsic hallucinations are rarely found in the reported summarization and knowledge-grounded open-domain dialogue analyses.
5 Evaluating Interactivity in ChatGPT
The paper tests whether ChatGPT’s conversational interface can improve outputs through multi-turn refinement. Across summarization, translation, and multimodal generation, iterative interaction is used to correct or refine generated results.
- Experimental scope: Multi-turn interaction is evaluated for summarization, machine translation with post-editing, and multimodal generation.The experiments assess whether conversational refinement improves generated responses and task performance.
- Interactivity on Summarization: In summarization, a two-turn iterative prompt approach is applied to 50 SAMSum documents to improve overly long summaries.Users can request changes based on a previously generated summary.
- Interactivity on Machine Translation: The multi-turn MT-APE approach improves translation correctness by allowing ChatGPT to make partial corrections or other post-editing changes.The experiment combines machine translation and automatic post-editing with a single ChatGPT model.
- Interactivity on Multimodal Generation: InstructGPT does not achieve salient improvements when intermediate multimodal results are directly placed in the input context.InstructGPT shares ChatGPT’s backbone model but lacks conversation ability.
6 Evaluation of GPT-4
The paper extends its framework to evaluate GPT-4 on language, multitask, commonsense, and other reasoning abilities. GPT-4 is generally stronger than ChatGPT on several reasoning and language tasks, while remaining limited on extremely low-resource languages and complex reasoning.
- Multitasking: GPT-4 performs on par with ChatGPT on most tested common NLP tasks.The evaluation describes their multitasking abilities as broadly comparable.
- Multilingual abilities: GPT-4 performs better at language identification for extremely low-resource languages such as Buginese and at machine translation.These findings align with results reported in GPT-4’s technical report.
- Reasoning abilities: GPT-4 achieves very high commonsense reasoning performance close to ChatGPT.The finding is reported as consistent with OpenAI’s technical report.
- Reasoning abilities: GPT-4 generally outperforms ChatGPT on inductive, mathematical, multi-hop, temporal, and spatial reasoning.The paper identifies these as especially notable areas of improvement.
- Overall findings: GPT-4 still has room for improvement in extremely low-resource languages and complex reasoning tasks.This conclusion summarizes the remaining limitations identified in the evaluation.
7 Conclusion and Discussion
The evaluation finds strong zero-shot and interactive performance across many tasks, but also exposes weaknesses in reasoning, low-resource language generation, factuality, and sustained interaction. The authors frame these findings as evidence of both ChatGPT’s capabilities and unresolved limitations requiring broader, more scalable evaluation and mitigation.
- Multitask, Multilingual, Multimodal: ChatGPT performs strongly in many high- and medium-resource languages but lacks reliable understanding and generation for low-resource languages, especially non-Latin scripts.The reported multilingual weakness is concentrated in generation rather than understanding, with limited performance in languages such as Marathi, Sundanese, and Buginese.
- Reasoning: ChatGPT’s reasoning abilities are inconsistent, including weaknesses in inductive, spatial, mathematical, and multi-hop reasoning.The evaluation finds relatively better temporal and commonsense reasoning, while causal and analogical reasoning are described as acceptable.
- Factuality&Hallucinations: ChatGPT suffers from hallucinations, and the authors call for automatic and scalable methods to detect and mitigate hallucinations and other undesirable artifacts.The discussion notes that external knowledge sources can support factuality, while RLHF can mitigate some undesirable responses but is not sufficient for all hallucinations.
- Interactivity: Multi-turn interactivity improves performance, but ChatGPT can retain incorrect answers despite repeated user prompts.The authors identify handling multiple rounds of feedback as an important challenge.
- Limitations: The evaluation is limited by small sample sizes from a pre-API ChatGPT interface and by experimental details placed in the appendix.The authors release code to help the community replicate the evaluation and extend the benchmarks.
- Multitask, Multilingual, Multimodal: ChatGPT outperforms state-of-the-art LLMs zero-shot on various tasks and surpasses fine-tuned models on some tasks.The evaluation also reports strong question-answering performance, including near-perfect scores on bAbI task 15, EntailmentBank, and Pep-3k.
C.6.2 Task-Oriented Dialogue Experimental Setups
The task-oriented dialogue experiments test modular and unified approaches for state tracking, response generation, and database-grounded interaction. ChatGPT can handle basic retrieval but shows weaknesses in multi-turn state retention, reasoning, and factual grounding.
- Experimental setups: The modular setup evaluates dialogue state tracking and response generation on MultiWOZ2.2 dialogue turns.The unified setup instead asks ChatGPT to complete restaurant recommendations from a structured database.
- Results: ChatGPT can complete basic retrieval queries such as filtering restaurants by cuisine or price.These capabilities were investigated in the unified task-oriented dialogue setting.
- Limitations: ChatGPT cannot reliably retain belief state across multiple turns, although explicitly recalling earlier preferences can restore usable information.A later preference can overwrite an earlier one unless the earlier information is explicitly requested.
- Limitations: ChatGPT fails on basic reasoning over database attributes and answers incorrectly 66% of the time even with a basic knowledge base.Examples require filtering cuisines by country or interpreting rating thresholds.
- Limitations: ChatGPT generates extrinsic hallucinations beyond the supplied knowledge, including fabricated hotel prices or restaurant availability.Such unsupported details are especially harmful in task-oriented dialogue.
D ChatGPT on Multilinguality
The multilinguality evaluation examines language identification, sentiment analysis, and translation across languages with different resource levels and scripts. ChatGPT handles some low-resource language understanding better than generation, especially when non-Latin scripts are involved.
- Evaluation scope: The evaluation reports language statistics and analyzes multilingual capability across the languages under study.The experiment distinguishes high-, medium-, low-, and extremely low-resource languages.
- Language identification: ChatGPT correctly identifies English and Indonesian 100% of the time but often misclassifies or cannot identify Javanese and Buginese.Its sentiment-analysis performance in Javanese is only slightly below its English and Indonesian performance.
- Human-preferred responses: ChatGPT provides uncertainty signals and broader linguistic information when it does not fully understand a language.The observed responses include information such as a language’s location and associated tribe.
- Script disparity: ChatGPT understands non-Latin scripts better than it generates them.The paper compares translation involving Chinese and Korean with translation involving Latin-script languages.
E Multimodality: Flag Drawing Task
The flag-drawing task evaluates whether ChatGPT can generate visual content from text through an intermediate SVG-code representation. Iterative correction is used to improve generated flags against visual error criteria.
- Task formulation: The task evaluates 50 national flags from different continents through a text-to-description-to-SVG generation pipeline.ChatGPT first describes a flag, then generates SVG code representing it.
- Iterative refinement: When generated images contain errors, ChatGPT is iteratively asked to fix them.Each correction round targets one error type, and the process stops after perfection or two fixing rounds.
- Evaluation: The evaluation scores layout, color, missing components, and shape or size errors.Images also receive grades from A to E corresponding to zero through four or more errors.
- Results: Multi-turn refinement allows ChatGPT to generate a flag image more similar to the ground-truth image.The complete results are reported in Figure 4.
F.2 Non-textual Semantic Reasoning
The non-textual semantic reasoning evaluation covers mathematical, temporal, spatial, commonsense, causal, multi-hop, and analogical reasoning. Results are uneven: temporal, commonsense, causal, and analogical reasoning are comparatively stronger, while spatial and multi-hop reasoning remain difficult.
- Mathematical reasoning: ChatGPT shows weak mathematical reasoning and often understands questions without producing correct solutions.The paper summarizes prior evaluations reporting inconsistent and poor mathematical performance.
- Temporal reasoning: 86.67% accuracy on TimeDial indicates comparatively decent temporal reasoning.ChatGPT answered 26 of 30 temporal-reasoning samples correctly.
- Spatial reasoning: 63.33% on StepGame (Basic) shows better performance on elementary spatial relations than on more difficult spatial cases.The analysis identifies particular difficulty with clock-direction descriptions and diagonal relations.
- Spatial reasoning: 25% accuracy on clock-position descriptions and 55% on diagonal positions are below 85% accuracy on basic cardinal points.The comparisons isolate two sources of spatial-reasoning errors.
- Commonsense reasoning: ChatGPT performs well on the three commonsense datasets and can generate reasonable reasoning procedures supporting answers.The result analysis reports both answer accuracy and rationale quality for the evaluated samples.
- Causal, multi-hop, and analogical reasoning: ChatGPT answers 8 of 30 two-hop HotpotQA samples correctly, indicating difficulty with multi-hop reasoning.The evaluation also reports 80% accuracy on causal reasoning and all 30 analogical examples answered correctly.
H.2.1 Experiment 1: Multi-turn Post-Editting
The experiment evaluates interactive post-editing for translation, including low-resource languages, and examines multi-turn image generation. Post-editing generally improves translation quality, while image generation supports iterative revision but remains elementary.
- Translation post-editing: 30 parallel sentences from 6 NusaX language pairs are translated from English, then post-edited through follow-up prompts.The languages are Chinese, French, Indonesian, Korean, Javanese, and Sundanese.
- Translation post-editing: Native-speaker validation checks whether post-editing improves the translation and preserves the English meaning.
- Translation post-editing: Interactive automatic post-editing consistently improves translation over single-turn translation, especially for low-resource languages.
- Translation post-editing: On English→Marathi WMT 2022 data, single-turn translation scores slightly better in Marathi, but post-editing improves back-translated English on every metric.The result suggests post-editing brings translations closer to the source meaning, although Marathi output remains substantially worse than the baseline MT system.
- Multimodal interaction: Multi-turn prompting lets ChatGPT and InstructGPT revise generated images, including a flag and a painting, although generation quality remains elementary.InstructGPT shares ChatGPT’s backbone but lacks conversation ability; the examples demonstrate iterative drawing and modification.
J List of Evaluation Datasets
The evaluation uses a broad collection of datasets spanning machine translation, question answering, summarization, reasoning, commonsense, dialogue, and multimodal tasks. These datasets include both established benchmarks and specialized resources for spatial, causal, inductive, deductive, and analogical reasoning.
- Task coverage: The evaluation dataset list covers summarization, machine translation, sentiment analysis, question answering, dialogue, and misinformation-related tasks.
- Dialogue and summarization: The list also includes dialogue and summarization datasets built from conversational or news text, such as SAMSum and CNN/DailyMail.SAMSum contains messenger-like conversations with summaries, while CNN/DailyMail contains journalist-written news articles.
- Task coverage: FLoRes-200 and NusaX provide multilingual translation resources, including English paired with low-resource and Indonesian local languages.FLoRes-200 includes Nepali, Sinhala, Khmer, and Pashto; NusaX covers 12 languages.
- Commonsense datasets: Commonsense datasets evaluate physical plausibility, everyday physical knowledge, causal reasoning, and multiple-choice commonsense question answering.Pep-3k, PiQA, E-Care, and CommonsenseQA cover these complementary forms of commonsense evaluation.
K Examples from Machine Translation and Post-Editing
The examples table presents ChatGPT translations alongside post-edited versions across several languages and sentence types. The examples include factual, narrative, political, cultural, and descriptive content for qualitative comparison.
- Language examples: The examples include French, Indonesian, Korean, Chinese, and Japanese translations of sentences about events, polling, subcultures, and historical descriptions.
- Content examples: The examples include sentences where people avoid injury, survey respondents support a view, subcultures differ across factors, and ancient Egyptians admired earlier monuments.
- Content examples: Several examples concern rainfall threatening previously damaged levees, illustrating translation of factual and potentially consequential statements.