Source-linked AI summary
"I think this is the most disruptive technology": Exploring Sentiments of ChatGPT Early Adopters using Twitter Data
Mubin Ul Haque, Isuru Dharmadasa, Zarrin Tasnim Sworna, Roshan Namal Rajapakse, Hussain Ahmad
TL;DR
The paper asks what early ChatGPT adopters discuss and feel, because their reactions may reveal the technology’s prospects and problems. It analyzes 10,732 Twitter tweets through topic modelling and qualitative sentiment analysis, finding overwhelming excitement alongside limited concerns, particularly about education and misuse. The findings provide an early snapshot of public responses and identify ethical implications for ChatGPT’s use and development.
Problem
Early-adopter sentiments matter because they can indicate ChatGPT’s potential success or failure and reveal issues, strengths, and weaknesses.
Method
The study analyzes 10,732 early-adopter tweets using topic modelling to identify topics and qualitative analysis to examine sentiments within them.
Results
Early adopters expressed overwhelming excitement about ChatGPT’s assistance across domains, alongside limited concerns about ethical effects including education-related activities.
Takeaways & Limitations
The study offers an early snapshot of ChatGPT responses and highlights ethical implications requiring consideration in its use and further development.
Takeaways & Limitations
The study uses Twitter as a representative source, so future research using Stack Overflow and blogs could further generalize the findings.
Abstract
from arXiv · showhide
Large language models have recently attracted significant attention due to their impressive performance on a variety of tasks. ChatGPT developed by OpenAI is one such implementation of a large, pre-trained language model that has gained immense popularity among early adopters, where certain users go to the extent of characterizing it as a disruptive technology in many domains. Understanding such early adopters' sentiments is important because it can provide insights into the potential success or failure of the technology, as well as its strengths and weaknesses. In this paper, we conduct a mixed-method study using 10,732 tweets from early ChatGPT users. We first use topic modelling to identify the main topics and then perform an in-depth qualitative sentiment analysis of each topic. Our results show that the majority of the early adopters have expressed overwhelmingly positive sentiments related to topics such as Disruptions to software development, Entertainment and exercising creativity. Only a limited percentage of users expressed concerns about issues such as the potential for misuse of ChatGPT, especially regarding topics such as Impact on educational aspects. We discuss these findings by providing specific examples for each topic and then detail implications related to addressing these concerns for both researchers and users.
I. INTRODUCTION
The paper studies early ChatGPT adopters because their sentiments can indicate the technology’s prospects, problems, strengths, and weaknesses. It uses Twitter topic modelling and qualitative sentiment analysis to characterize discussions and perceptions.
- Early adopters can shape broader perceptions of new technology and provide insights into potential product success or failure.
- Their feedback may identify problems early, before issues become widespread, potentially improving ChatGPT’s market prospects.
- Twitter provides accessible, large-scale user-generated data for investigating public perceptions and sentiments about emerging technologies.
- The study analyzes 10,732 early-adopter tweets using topic modelling followed by manual qualitative sentiment analysis.
- The paper contributes topic-level analysis, sentiment categorization, ethical and societal implications, and specific tweet examples.
- It presents an early qualitative account of ChatGPT adopters’ sentiments and feedback, addressing a gap in directly related research.
III. RESEARCH METHODOLOGY
The methodology combines research questions, Twitter data collection, topic identification, and sentiment analysis to investigate ChatGPT early adopters and their discussions.
- The research methodology covers research questions, dataset construction, preprocessing, topic identification, and sentiment analysis.
- The study asks about early adopters’ characteristics, the main Twitter discussion topics, and sentiments expressed about those topics.
- Tweets were collected from December 5–7, 2022 using the keyword “ChatGPT” and English-language filtering through Python and the Twitter API.
C. Data Pre-processing
The preprocessing pipeline cleans the Twitter corpus before topic modelling by removing duplicates, textual noise, stop-words, domain terms, and inflectional variation.
- Removing retweets and duplicates reduced the ChatGPTTweet dataset to 10,732 analyzed tweets.
- Preprocessing removed punctuation, URLs, emojis, and Twitter handles while retaining alphanumeric data.
- Stop-word removal used NLTK’s English list and excluded frequently appearing domain-specific terms such as ChatGPT, OpenAI, and AI.
- NLTK WordNet lemmatization mapped inflected word forms to dictionary root forms.
- LDA grouped tweets by word co-occurrence and frequency, assigning each tweet probabilities of belonging to specific topics.
- The researchers selected N = 9 topics after coherence experiments and manual examination of tweets for 9 ≤ N ≤ 12.
E. Sentiment Analysis on ChatGPT Topics
The study examines ChatGPT topics and early-adopter characteristics before manually analyzing sentiments within topic-specific samples. It finds geographically dispersed and professionally diverse adoption, with limited verification among users.
- Sentiment analysis: Manual sentiment analysis labelled 9 topic datasets, each containing 100 randomly selected tweets, as negative, neutral, or positive.
- Sentiment analysis: Open coding identified recurring discussion patterns by recording summarized key points and short codes for each tweet.
- Sentiment analysis: Two authors labelled each topic dataset, and disagreements were resolved through discussion among all five authors.
- Early-adopter characteristics: The early adopters were geographically dispersed, with most tweets originating from North American and Asian regions.
- Early-adopter characteristics: USA, India, UK, Canada, and Germany were the top five countries by expressed opinions during ChatGPT adoption.
- Early-adopter characteristics: Only 2% of early adopters were verified Twitter users, indicating adoption beyond verified accounts.
- Early-adopter characteristics: Software practitioners, academics, and students were the three largest occupation groups among a broad range of early adopters.
being discussed about ChatGPT in Twitter?
ChatGPT attracted diverse communities, and the study identified nine topics discussed by early adopters on Twitter. Topic names were developed from LDA keywords and sampled tweets through author consensus.
- ChatGPT attracted researchers, managers, practitioners, entertainers, business analysts, and educationists.
- ChatGPT reached 1 million users within five days of its beta release.Facebook, Netflix, and Instagram took approximately 300, 1200, and 75 days, respectively, to reach 1 million users.
- Authors named topics by reviewing top keywords and 30 randomly selected tweets per topic, then reaching consensus.
- Nine topics were identified among early adopters’ Twitter discussions.
1) Disruptions to software development:
Early adopters discussed ChatGPT across software development, entertainment, natural-language processing, education, intelligence, business analysis, search, and question-and-answer testing. These discussions included coding assistance, creative generation, information retrieval, and broader effects on careers and institutions.
- 1) Disruptions to software development:: Users discussed ChatGPT generating code, assisting debugging, and summarizing or translating code.
- 2) Entertainment and exercising creativity:: Entertainment uses included generating poems, jokes, humorous writing, and combinations of characters or concepts.
- Impact on educational aspects: Early adopters discussed ChatGPT for childhood learning, syllabus development, literature review, and crisis-management learning.
- Intelligence of ChatGPT: Users highlighted ChatGPT’s ability to understand queries, write and debug complex code, solve optimization problems, and answer challenging questions.
- Business analysis: Business discussions covered startup pitches, business use cases, business plans, and financial advice.
7) Implications for Search Engines:
ChatGPT was discussed as a way to retrieve and explain information beyond conventional search-engine functionality, while Twitter sentiment analysis summarized positive, negative, and neutral reactions by topic. The supplied passages describe the analysis approach and report strong positivity for software-development discussions.
- 7) Implications for Search Engines:: ChatGPT presented information conveniently by selecting appropriate material and explaining it simply, offering a novel search experience.
- ChatGPT Sentiments: Sentiment results were summarized as positive, negative, and neutral percentages for each topic.
- 1) Disruptions to software development:: 81% positive sentiment was reported for software-development discussions, with users especially impressed by coding assistance.
- 1) Disruptions to software development:: Users praised ChatGPT for coding, debugging, error handling, and troubleshooting, while some cautioned that its development assistance requires mindful use.
- 1) Disruptions to software development:: A neutral tweet noted that ChatGPT can produce plausible-sounding but wrong answers, particularly for code using changing external libraries.
2) Entertainment and exercising creativity:
Entertainment and creative experimentation were prominent positive-use cases among early ChatGPT adopters. Users generated humorous and literary outputs, and the resulting screenshots received high engagement on Twitter.
- 2) Entertainment and exercising creativity:: 92% positive sentiment was reported for entertainment and creativity discussions.
- 2) Entertainment and exercising creativity:: Users generated poems, jokes, humorous writing, and amusing combinations of characters, personalities, and concepts.
- 2) Entertainment and exercising creativity:: Creative prompts produced imagined scenes and stories, including a Shakespearean-style Friends scene and a detective story about a dog.
- 2) Entertainment and exercising creativity:: ChatGPT-generated outputs were positively received and showed high engagement when users shared screenshots on Twitter.
- Across the dataset, 83% of tweets expressed satisfaction, 14% expressed concerns, and 3% were neutral.
- 3) Natural Language Processing.: Early adopters praised natural-language processing and realistic human-like text generation, while some questioned generated-text quality and misinformation.
4) Chatbot Intelligence.:
Early adopters viewed ChatGPT’s intelligence and business potential positively, while also expressing concerns about harmful societal effects and educational misuse. Educational adoption was notably more mixed than business adoption.
- Chatbot Intelligence.: 78% of tweets expressed favourable sentiment toward ChatGPT’s intelligence, compared with 20% harmful-impact and 2% neutral views.The findings characterize most early adopters as supportive of ChatGPT’s intelligence capability.
- Chatbot Intelligence.: Users associated ChatGPT’s intelligence with practical benefits such as creating applications and supporting software development.Examples include generating readable instructions for creating a React application and potentially saving developers substantial time and effort.
- Chatbot Intelligence.: Concerns about ChatGPT’s intelligence included possible increases in terrorism, hacking, and unemployment, prompting a recommendation for usage policies and regulation.The proposed policy response is intended to mitigate negative impacts and improve future user acceptability.
- Impact on Educational Aspects: Educational adoption was more divided, with 52% positive, 32% negative, and 16% neutral views.Users supported grading, learning assessment, syllabus preparation, and personalized teaching, but also worried that student use for assignments could hinder learning and create plagiarism concerns.
- Business Applications: 75% of business-related tweets were positive, while 5% were negative and 20% neutral, reflecting broad optimism about ChatGPT’s business applications.Users discussed applications ranging from business-plan development and investor pitches to decision-making, technical support, and financial advice.
7) Implications for Search Engines:
Early adopters saw ChatGPT as a potentially faster or more accurate alternative to search and knowledge-sharing services, but questioned the reliability of its answers. Q&A testing revealed positive interaction qualities alongside substantial concern about confidently incorrect responses.
- Implications for Search Engines: 54% of search-related tweets were positive, 31% neutral, and 15% negative, with some users describing ChatGPT as a potential replacement for search engines.The #googlekiller label reflected perceptions that ChatGPT could threaten existing search services.
- Implications for Search Engines: Users reported that ChatGPT could outperform current search engines in speed and knowledge-sharing platforms in accuracy, posing potential risks to services such as Stack Overflow, Quora, and Wikipedia.These observations concern possible future effects on search and knowledge-sharing platforms.
- Implications for Search Engines: Users cautioned that ChatGPT’s internet-based information can be inaccurate and should be verified against other sources.Some users had not fully understood these limitations, while browser extensions were described as emerging responses.
- Q&A Testing: Q&A testing produced 38% positive, 40% neutral, and 22% negative sentiment, with praise for quality, speed, human-friendly interaction, and answer explanations.Adopters also valued precise questioning and interactive responses.
- Q&A Testing: Negative Q&A sentiment stemmed from wrong or invalid answers, including incorrect chemistry responses and a high confidence level that could impede wider adoption.Examples included incorrect units and plausible-sounding but incorrect coding answers.
9) Future Careers & Opportunities:
Early adopters expressed strong optimism about ChatGPT’s role in future careers and opportunities, especially for efficient work, innovation, and customized customer support. At the same time, some feared job displacement, particularly in software development.
- Future Careers & Opportunities: 75% of tweets about future careers and opportunities were positive, compared with 9% neutral and 16% negative sentiment.Positive sentiment was linked to ChatGPT’s fast and effective solutions as a factor that could support successful careers.
- Future Careers & Opportunities: Users associated ChatGPT with innovation, learning, adaptation, and easier job tasks, while imagining AI coding and decision-support systems enabling very small teams to create valuable products and services.These views emphasized convenience and productivity in work contexts.
- Future Careers & Opportunities: Adopters viewed ChatGPT’s customization as useful for personalized messages, services, and answers that could increase future customer engagement.Technical customer support was offered as an example of personalized service.
- Future Careers & Opportunities: Some users feared losing jobs to ChatGPT, including software-programmer roles, because it could perform many development activities rapidly.The concern extended to possible replacement of product managers and broader disruption in the IT industry.
V. IMPLICATIONS
The study translates its findings into practical guidance for ChatGPT users and research directions concerning benefits, reliability, ethics, and validity boundaries. It emphasizes responsible use while noting that the findings are limited by Twitter-based sampling, human judgment, and topic-model selection.
- Implications for users: ChatGPT may change traditional software development and support developers in creating software and maintaining effective development processes.
- Implications for users: Because ChatGPT outputs are not verified or fact-checked by established authorities, users should critically evaluate them and consult verified data sources for critical tasks.
- Implications for researchers: Educational misuse concerns include using ChatGPT to write essays or complete assignments, potentially hampering students’ learning.
- Implications for researchers: Future research should examine ethical use and practical implications across resources such as Stack Overflow and blogs to further generalize the findings.
- Threats to validity: Manual topic-wise sentiment analysis may involve human judgment bias, although two researchers labelled each topic sample and resolved disagreements through discussion.
- Threats to validity: Topic modelling introduces threats because selecting the optimal number of topics N is difficult, so the study experimented across a broad range of N values.
- Conclusion: Overall, early adopters expressed overwhelming excitement and limited concerns, while the study highlights ethical implications and the need for continued research and dialogue on responsible use.