Source-linked AI summary
Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness
Bo Li, Gexiang Fang, Yang Yang, Quansen Wang, Wei Ye, Wen Zhao, Shikun Zhang
TL;DR
The paper addresses limited evidence about ChatGPT’s overall reliability on information extraction, beyond raw task performance. It systematically evaluates seven fine-grained IE tasks across four dimensions and finds contrasting Standard-IE and OpenIE performance, trustworthy explanations, overconfidence, and mostly faithful outputs.
Problem
Because ChatGPT is closed and its responses can affect users, the paper asks how reliable it is across information extraction tasks, not only how well it performs.
Method
The study evaluates ChatGPT on 7 fine-grained IE tasks across 14 datasets and measures performance, explainability, calibration, and faithfulness using 15 keys.
Results
ChatGPT performs poorly in Standard-IE but excellently in human-evaluated OpenIE, gives high-quality explanations, is overconfident, and is largely faithful to the original text.
Takeaways & Limitations
The findings provide a systematic assessment of ChatGPT for information extraction and motivate further research using the released datasets and code.
Takeaways & Limitations
Standard-IE results may differ from prior studies because this work evaluates entire test sets with concise, unified prompts rather than small samples or domain-specific prompts.
Abstract
from arXiv · showhide
The capability of Large Language Models (LLMs) like ChatGPT to comprehend user intent and provide reasonable responses has made them extremely popular lately. In this paper, we focus on assessing the overall ability of ChatGPT using 7 fine-grained information extraction (IE) tasks. Specially, we present the systematically analysis by measuring ChatGPT's performance, explainability, calibration, and faithfulness, and resulting in 15 keys from either the ChatGPT or domain experts. Our findings reveal that ChatGPT's performance in Standard-IE setting is poor, but it surprisingly exhibits excellent performance in the OpenIE setting, as evidenced by human evaluation. In addition, our research indicates that ChatGPT provides high-quality and trustworthy explanations for its decisions. However, there is an issue of ChatGPT being overconfident in its predictions, which resulting in low calibration. Furthermore, ChatGPT demonstrates a high level of faithfulness to the original text in the majority of cases. We manually annotate and release the test sets of 7 fine-grained IE tasks contains 14 datasets to further promote the research. The datasets and code are available at https://github.com/pkuserc/ChatGPT_for_IE.
1 Introduction
The paper evaluates ChatGPT across seven fine-grained information extraction tasks using performance, explainability, calibration, and faithfulness dimensions. ChatGPT performs poorly in Standard-IE but strongly in OpenIE, while offering trustworthy explanations, showing overconfidence, and remaining largely faithful to source text.
- Study scope: The study evaluates ChatGPT on 7 fine-grained IE tasks across performance, explainability, calibration, and faithfulness.It collects 15 keys from ChatGPT or domain experts.
- Performance: ChatGPT performs poorly in Standard-IE but achieves surprisingly good OpenIE results confirmed by human evaluation.The study compares ChatGPT with other popular models across both settings.
- Explainability: ChatGPT provides high-quality and trustworthy explanations for its decisions.Explainability is assessed as one of the study’s four evaluation dimensions.
- Calibration: ChatGPT is overconfident in its predictions, resulting in low calibration.Calibration measures predictive uncertainty.
- Faithfulness: ChatGPT is largely faithful to the original text in most cases.The paper evaluates whether its responses remain grounded in the input text.
2 Related Work
Related work positions ChatGPT within research on large language models, ChatGPT’s broader impacts and NLP abilities, and information extraction. IE is framed as structured factual information extraction from unstructured text across diverse entity, relation, and event tasks.
- Large language models: Large language models use very large parameter counts to support few-shot and zero-shot learning and emergent abilities.Examples include GPT-3, Gopher, LaMDA, Megatron-turing-NLG, and PaLM.
- ChatGPT research: Research on ChatGPT examines ethical, educational, medical, and broader natural language processing applications.The cited work covers potential impacts on human life and evaluations of mathematical and reasoning capabilities.
- Information extraction: Information extraction aims to extract structured factual information from unstructured texts.Its outputs concern entities, relations, and events.
- IE tasks: IE includes entity typing, named entity recognition, relation classification, relation extraction, event detection, event argument extraction, and event extraction.These tasks differ in whether they identify entities, classify relations or types, detect events, extract arguments, or combine stages.
3 ChatGPT for Information Extraction
The paper defines seven fine-grained IE tasks and evaluates ChatGPT in Standard-IE and OpenIE settings. It measures four capability dimensions through 15 keys, combining automatic outputs with human annotation.
- Information Extraction: The analysis covers 7 fine-grained IE tasks, including entity, relation, and event extraction tasks.The tasks are entity typing, named entity recognition, relation classification, relation extraction, event detection, event argument extraction, and event extraction.
- Standard-IE and OpenIE: Standard-IE supplies task descriptions, input text, prompts, and candidate labels, while OpenIE omits the predefined label set.OpenIE assesses whether ChatGPT can generate reasonable predictions from the task description, prompt, and input text.
- Evaluation dimensions: The study gathers 15 keys across performance, explainability, calibration, and faithfulness.These keys assess ChatGPT’s capabilities from four complementary dimensions.
- Explainability: Explainability combines ChatGPT-generated reasons, self-checks, and manual human judgments of reason acceptability.The keys include Reason_Open, Reason_Standard, self-approval measures, and manual evaluations.
- Calibration: Calibration measures predictive uncertainty and examines whether ChatGPT is overconfident in its predictions.A properly calibrated classifier’s predictive scores should reflect the probability of correctness.
- Faithfulness: Faithfulness uses manual expert annotation to assess whether ChatGPT’s reasons are faithful to the original input.The relevant keys are FicR_Open(Manual) and FicR_Standard(Manual).
4 Performance
The evaluation compares ChatGPT with supervised baselines across 14 IE datasets, finding weak Standard-IE performance but stronger results when outputs are generated without fixed labels. Performance varies with task complexity, while top-k predictions can improve recall.
- Standard-IE: ChatGPT’s Standard-IE performance is not comparable to baseline and state-of-the-art models in most cases.The comparison uses the whole test set, whereas ChatGPT operates in a zero-shot scenario and other models are task-supervised.
- Standard-IE: ChatGPT’s Standard-IE performance may be reduced when it must choose among predefined labels that are difficult to understand.The authors identify this as one possible explanation for the observed performance gap.
- Standard-IE: ChatGPT performs well on relatively simple IE tasks but struggles with complex tasks requiring deeper contextual analysis and reasoning.Entity typing, NER, and RC are more acceptable, whereas RE and EE remain challenging.
- OpenIE: OpenIE removes the predefined label set and evaluates whether domain experts judge predictions reasonable in light of the input and background knowledge.Unlike Standard-IE, predictions do not need to match ground-truth labels.
- OpenIE: More than 84% of ChatGPT’s OpenIE predictions are considered reasonable by domain experts in ET, NER, and RC.Performance remains relatively poorer on more challenging tasks such as RE and EE.
- Top-k Recall Analysis: Top-3 recall improves by 19.6% on SemEval2010, while top-5 recall reaches 94.9% on BBN and 76.0% on SemEval2010.These results support using ChatGPT as a candidate generator under Standard-IE.
5 Explainability, Calibration and Faithfulness
ChatGPT offers strong explanations and faithfulness to source text, but its confidence is often poorly calibrated. Its explanations are generally judged reasonable, while predicted probabilities frequently fail to reflect accuracy.
- Explainability: Explainability was evaluated by asking ChatGPT to provide reasons for predictions in both Standard-IE and OpenIE settings.For Standard-IE, the explainability analysis considered only correctly predicted samples.
- Explainability: Over 90% of datasets achieved reasonable explanation scores in both Standard-IE and OpenIE under most evaluations.ChatGPT and domain experts largely agreed on the quality of these explanations.
- Calibration: ChatGPT was overconfident in most predictions, motivating evaluation of whether its stated confidence could reduce misinterpretation risks.The study elicited prediction probabilities for both correct and incorrect samples.
- Calibration: ChatGPT showed much poorer calibration than BERT-based methods, indicating that its confidence scores often do not represent true probabilities.Miscalibration dominated most tasks, despite low ECE in some tasks such as ET and NER.
- Faithfulness: ChatGPT’s explanations were highly faithful to the original text, with over 95% of samples judged faithful in nearly all datasets and settings.False explanations were rare when predictions were correct.
6 Conclusion
The paper systematically evaluates ChatGPT across information extraction tasks and four dimensions, finding a contrast between weak Standard-IE performance and strong OpenIE performance. It also reports strong explanations and faithfulness, alongside overconfidence and low calibration.
- Conclusion: The study evaluates ChatGPT on 7 fine-grained IE tasks across 14 datasets using performance, explainability, calibration, and faithfulness.The evaluation collects 15 keys identified by ChatGPT or domain experts.
- Conclusion: ChatGPT performs poorly relative to BERT-based models in most Standard-IE cases but achieves excellent OpenIE accuracy according to human annotators.The conclusion contrasts the two IE settings rather than reporting a single overall score.
- Conclusion: ChatGPT provides high-quality, trustworthy explanations and remains highly faithful to the original text, indicating that predictions are grounded in the input.These findings are reported as broad conclusions across the evaluation.
- Conclusion: ChatGPT tends toward overconfidence, resulting in low calibration.The conclusion identifies calibration as a central issue despite the model’s strengths in explanations and faithfulness.
A.1 Dataset
The dataset appendix describes the evaluation resources for the study’s seven information extraction tasks. Each task uses two commonly used datasets, with statistical characteristics and annotation counts reported.
- A.1 Dataset: Each task is evaluated using two commonly used datasets.The appendix reports detailed dataset statistics in Table 10.
- A.1 Dataset: The appendix reports the number of manually annotated samples for each dataset as #Ann.These annotation counts are shown in Table 9.
A.2 The State-of-the-Art Methods on Single Dataset
The appendix surveys state-of-the-art methods used as reference points across the single-dataset IE tasks. These methods include task-specific architectures and graph or generation-based approaches.
- Entity Typing (ET): PICOT achieves state-of-the-art results on the BBN and OntoNotes 5.0 entity typing datasets.It models hierarchical type differences and incorporates type information into entity contexts.
- Named Entity Recognition (NER): ACE is presented as a method for automatically finding better embeddings for structured prediction in named entity recognition.The passage introduces ACE as the referenced NER method.
- Relation Classification (RC): KLG achieves state-of-the-art results on TACRED by using a label graph and a learned top-k prediction set.Its dynamic k-selection mechanism supports discriminative relation representations.
- Relation Extraction (RE): PL-Marker represents span pairs with packed markers and models neighborhood and subject-oriented span interactions for relation extraction.The method is designed to incorporate entity-boundary and span-pair information.
- Event Detection (ED): SaliencyED distinguishes trigger-dependent from context-dependent event types, while ONEIE globally optimizes extraction as a graph.The methods are associated with ACE05-E and ACE05-E+ respectively.
- Event Argument Extraction (EAE): DEGREE formulates event extraction as conditional generation and obtains state-of-the-art results on ACE05-E and ACE05-E+.Predictions are extracted from generated sentences using a deterministic algorithm.
A.3 Exemplar of the Input
This section presents an event detection input example from ACE05-E and explains that its three parts are jointly imported into ChatGPT.
- Table 11 presents the input example to clarify the implementation of the event detection task.
- The event detection example is extracted from ACE05-E.
- All three parts of the example are jointly imported into ChatGPT.