Source-linked AI summary
Let's have a chat! A Conversation with ChatGPT: Technology, Applications, and Limitations
Sakib Shahriar, Kadhim Hayawi
TL;DR
Against the backdrop of ChatGPT’s public attention, this paper reviews chatbot history, ChatGPT’s technology, and potential applications while presenting responses to selected questions. It reports coherent, engaging, and grammatically precise responses, while noting that ChatGPT can hallucinate and has other limitations.
Problem
The paper addresses how ChatGPT’s capabilities emerged and where it may be applied, following the public attention its human-like text generation attracted.
Source: §1 · pp. 1–2Method
The paper reviews chatbot history, ChatGPT’s technological development and potential applications, and presents ChatGPT’s responses to selected questions.
Source: §1 · p. 2, §6 · p. 10Results
The authors report coherent, engaging, grammatically precise responses and strong factual accuracy for names and dates, with knowledge bounded through September 2021.
Source: §1 · p. 2Takeaways & Limitations
The paper describes potential uses in healthcare, education, and research, including medical support, tutoring, and help with writing, summarizing, and translation.
Source: §3 · p. 5, §6 · p. 10ChatGPT may hallucinate, so the authors recommend verifying and fact-checking its responses.
Source: §5 · p. 9
Abstract
from arXiv · showhide
The emergence of an AI-powered chatbot that can generate human-like sentences and write coherent essays has caught the world's attention. This paper discusses the historical overview of chatbots and the technology behind Chat Generative Pre-trained Transformer, better known as ChatGPT. Moreover, potential applications of ChatGPT in various domains, including healthcare, education, and research, are highlighted. Despite promising results, there are several privacy and ethical concerns surrounding ChatGPT. In addition, we highlight some of the important limitations of the current version of ChatGPT. We also ask ChatGPT to provide its point of view and present its responses to several questions we attempt to answer.
1. Introduction
The introduction traces chatbots’ development and introduces ChatGPT as a widely noticed system whose capabilities and applications the paper reviews. It reports strong coherence and factual detail in ChatGPT’s responses, alongside bounded knowledge.
1. Introduction
- The section traces chatbot origins from Turing’s 1950 question and test to rule-based ELIZA, which simulated conversation with scripted responses.The introduction notes that some early programs arguably relied on tricks to pass the Turing Test rather than demonstrating significant intelligence.Source: §1 · pp. 1–2
- Despite broad adoption, most chatbots lack personalization and user satisfaction remains questionable, prompting focus on conversational engagement.The introduction says these concerns motivated researchers and developers to focus on engagement.Source: §1 · p. 1
- Released in November 2022, ChatGPT drew substantial public attention; the paper reviews its technological evolution and applications and presents its responses to selected questions.The paper describes ChatGPT as generating logically consistent responses to a broad array of inquiries.Source: §1 · pp. 1–2
- NLP and later AI and machine-learning advances accompanied chatbots’ shift toward more natural language interaction and broader uses.The passages mention early customer-service uses and later applications including personal assistants and entertainment.Source: §1 · p. 2
- The authors report that ChatGPT’s responses were coherent, engaging, grammatical, and accurate on factual names and dates, with knowledge extending only through September 2021.They also observe that it refrained from providing information about itself and often ended with a summary.Source: §1 · p. 2
2. ChatGPT Technology: From Language Models to Reinforcement Learning
The section explains how language models generate text and describes reinforcement learning from human feedback as a way to steer ChatGPT toward preferred responses. The authors also note that ChatGPT’s own development account omits details about that final training phase.
2. ChatGPT Technology: From Language Models to Reinforcement Learning
- BERT is described as primarily supporting language-context recognition and classification, whereas GPT is more suited to language generation tasks such as translation and question-answering.Both are presented as prominent language models within the transformer and large-language-model family.Source: §2 · p. 2
- The paper characterizes large language models as neural-network systems trained on extensive datasets to model word-sequence probabilities and generate or interpret language.It says model inputs and outputs are represented as tokens, numerical representations of words.Source: §2 · pp. 2–3
- GPT-style models predict the next token in a sequence, while reinforcement learning from human feedback uses evaluator preferences to guide responses toward more useful and ethical outputs.The paper describes ChatGPT as built on GPT 3.5 and explains that human evaluators ranked its responses.Source: §2 · pp. 2–3
- ChatGPT’s account identifies unsupervised learning for pattern recognition but calls its final phase testing and refinement without specifying reinforcement learning or other details.The authors say this account aligns with descriptions of unsupervised learning in the GPT paper, while omitting specifics of the final phase.Source: §2 · p. 3
3. Applications of ChatGPT
This section surveys emerging ChatGPT applications across domains through a scoping review, alongside studies evaluating capabilities and concerns. The evidence includes proposed uses and uneven task-specific findings.
3. Applications of ChatGPT
- The authors use a scoping review to capture emerging applications and evidence, including some papers that were not peer-reviewed when the review was written.Google Scholar was the primary search engine; the authors included non-peer-reviewed work to provide a current overview in a rapidly advancing field.Source: §3 · p. 3
- Medical applications include clinical decision support and education, but reported exam performance varies: ChatGPT scored 60% on US medical licensing tests and 45.8% on a Chinese licensing exam.The cited studies also report 55.8% and 42.7% accuracy on two ophthalmology exams; image-based questions were removed from the US licensing tests.Source: §3 · pp. 3–4
- Education studies report potential peer-tutoring benefits, while also finding limits in mathematics and essay writing and raising concerns about bias, cheating, privacy, and transparency.Seventy percent of ChatGPT’s Algebra hints could result in positive learning gains; a study with nine students per group found no evidence of improved essay quality.Source: §3 · pp. 3–4
- Reported performance is uneven in software testing and translation: ChatGPT answered 55.6% of software-testing questions correctly, and translation lagged for low-resource or distant languages.For high-resource European languages, it performed on par with commercial translation applications; a separate vaccine-response study found it dismissed conspiracy statements but gave pros and cons on mandates.Source: §3 · pp. 4, 7
- ChatGPT’s suggested uses span healthcare, education, and research, including patient interaction, customized learning materials, data analysis, text mining, and research collaboration.These are examples in ChatGPT’s response, not evidence that the applications have been validated in practice.Source: §3 · p. 5
- Research-writing uses include editing, generating headlines and ideas, and drafting literature reviews, but authorship policies vary and the paper calls for human oversight and fact-checking.One study found significant plagiarism-checker matches when ChatGPT paraphrased sentences; domain expertise was associated with improved research quality in another.Source: §3 · p. 6
4. Analysis of ChatGPT Performance on Exams
ChatGPT’s exam accuracy varies substantially by exam and subject, despite an overall average accuracy of 59.53% across the reported exams. The section highlights strong USMLE results alongside weaker performance in ophthalmology, Chinese medical licensing, and math-heavy taxation questions.
4. Analysis of ChatGPT Performance on Exams
- 59.53% was ChatGPT’s average accuracy across all exams summarized in Table 2.Some studies reported qualitative rather than quantitative results.Source: §4 · p. 8
- USMLE accuracy was reported as 89.5% by Fijačko et al. (2023) and 94.6% by Kung et al. (2023), while two ophthalmology exams yielded 55.8% and 42.7%.The Chinese National Medical Licensing Examination also showed low performance, though no value is given here.Source: §4 · p. 8
- Exam results span a broad range, indicating that ChatGPT’s performance depends on the specific exam or domain.The authors suggest training and fine-tuning may need further refinement to improve proficiency across fields.Source: §4 · p. 8
- Within law exams, Constitutional Law reached 84.0% accuracy, while Taxation, particularly mathematical questions, had the table’s lowest accuracy at 27.6%.The authors note that qualitative-only studies cannot be directly compared with numerical results and call for more uniform evaluation metrics.Source: §4 · p. 8
5. Limitations of ChatGPT
The paper cautions that ChatGPT’s fluent responses can be unreliable and bounded by technical and human-interaction limitations. It recommends verifying responses and notes constraints in reasoning, input length, modalities, timeliness, and suitability for counseling.
5. Limitations of ChatGPT
- ChatGPT may hallucinate despite producing meaningful, conversational sentences, so the authors recommend verifying and fact-checking its responses.The passage attributes the hallucination concern to Alkaissi & McFarlane (2023).Source: §5 · p. 9
- ChatGPT makes errors in simple reasoning, logic, mathematics, and factual information (Borji, 2023).The passage contrasts GPT-3’s 175 billion parameters with a then-expected GPT-4 estimate, but does not establish that the estimate was realized.Source: §5 · p. 9
- The current model is limited to 5000 input tokens, which can challenge tasks such as text summarization.The current interface also does not accept image or audio uploads, and its image-drawing abilities are described as limited.Source: §5 · p. 9
- ChatGPT tends to be wordy unless instructed otherwise and expresses fewer emotions than an average human.The authors say it cannot replace human connection or serve for personal therapy or counseling that requires intimate human connection.Source: §5 · p. 9
- Its knowledge cutoff of September 2021 means users should not expect it to provide the latest news.The passage illustrates this boundary with ChatGPT’s response about a 2023 earthquake in Turkey.Source: §5 · p. 9
- ChatGPT’s stated limitations include dependence on provided context and training-data quality, inability to interpret nonverbal cues, and inability to perform physical tasks.The paper presents these as ChatGPT’s own responses to a question about its limitations.Source: §5 · pp. 9–10
6. Concluding Remarks: Consideration for Ethical and Privacy Concerns
The paper reviews ChatGPT’s technological background and potential applications across healthcare, education, and research, while emphasizing ethical and privacy concerns. It concludes that broad utility coexists with risks involving bias, misuse, copyright, plagiarism, and personal information.
6. Concluding Remarks: Consideration for Ethical and Privacy Concerns
- The paper identifies potential uses in healthcare for medical screening, general questions, and exam preparation; in education for tutoring and plagiarism detection; and in research for writing, summarizing, and translation.Source: §6 · p. 10
- The authors describe human-like text generation and general-purpose versatility as advantages that can support natural-language tasks, accessibility, learning, brainstorming, and writing assistance.Source: §6 · p. 10
- The paper notes ethical concerns including reported race and gender bias, cheating in education, and copyright or plagiarism risks in research (Zhuo et al., 2023).It says generated content may resemble or draw heavily on training texts, potentially raising copyright and attribution concerns.Source: §6 · p. 10
- Privacy risks may arise because training data can contain personal information and user prompts may include sensitive details that the model could learn and later reproduce.Source: §6 · p. 10
Conflicts of Interest
The authors declare no conflicts of interest in this work.
Conflicts of Interest
- The authors declare that they have no conflicts of interest in this work.Source: Conflicts of Interest · p. 11