Source-linked AI summary
ChatGPT in education: A discourse analysis of worries and concerns on social media
Lingyao Li, Zihui Ma, Lizhou Fan, Sanggyu Lee, Huizi Yu, Libby Hemphill
TL;DR
Concerns about ChatGPT’s educational use may affect academic integrity, learning and skill development, AI capability expectations, policy, society, and work. The study analyzes Twitter discourse with BERT-based topic modeling and social-network analysis, finding five concern categories and influential stakeholder groups.
Problem
Existing concerns about ChatGPT in education require broader public evidence because expert and literature-based accounts may miss diverse perspectives and current issues.
Method
The study analyzes Twitter data using BERT-based topic modeling of negative-sentiment tweets and social-network analysis to identify concerns and influential users.
Results
Twitter users generally expressed positive attitudes, while negative discussions converged on five concerns: academic integrity; learning outcomes and skills; capability limitations; policy and social concerns; and workforce challenges.
Takeaways & Limitations
Responsible and ethical AI use in education requires collaboration among policymakers, technology stakeholders, educators, and media agencies to establish guidelines.
Takeaways & Limitations
The data covered only four months after ChatGPT’s release, so attitudes and emerging risks may change as usage, models, and education policies evolve.
Abstract
from arXiv · showhide
The rapid advancements in generative AI models present new opportunities in the education sector. However, it is imperative to acknowledge and address the potential risks and concerns that may arise with their use. We analyzed Twitter data to identify key concerns related to the use of ChatGPT in education. We employed BERT-based topic modeling to conduct a discourse analysis and social network analysis to identify influential users in the conversation. While Twitter users generally ex-pressed a positive attitude towards the use of ChatGPT, their concerns converged to five specific categories: academic integrity, impact on learning outcomes and skill development, limitation of capabilities, policy and social concerns, and workforce challenges. We also found that users from the tech, education, and media fields were often implicated in the conversation, while education and tech individual users led the discussion of concerns. Based on these findings, the study provides several implications for policymakers, tech companies and individuals, educators, and media agencies. In summary, our study underscores the importance of responsible and ethical use of AI in education and highlights the need for collaboration among stakeholders to regulate AI policy.
1. Introduction
Generative AI and ChatGPT create new educational opportunities while raising ethical, academic, learning, and information-quality concerns. This study uses Twitter discourse and social-network analysis to identify public concerns and influential stakeholders.
- ChatGPT offers personalized learning, educational-content creation, language support, and feedback that may affect teaching and learning outcomes.It can help teachers generate questions, quizzes, assignments, games, and simulations tailored to students’ learning styles.
- ChatGPT’s ability to write essays and support outsourced student writing raises concerns about research originality, academic integrity, plagiarism detection, and writing-skill development.Its outputs may also be biased or nonsensical, potentially disseminating incorrect information.
- The rapid spread of ChatGPT in education has generated extensive social-media discussion where critical opinions can inform decision-makers.Social platforms also enable knowledge sharing and collaboration among policymakers, technology companies, educators, and students.
- Crowdsourced social-media opinions provide more diverse public perspectives than expert opinions and reveal users frequently implicated in the conversation.Social-network analysis can identify important users involved in discussing concerns.
- The study analyzes Twitter data through topic modeling of negative-sentiment tweets and social-network analysis of opinion leaders and frequently implicated users.The analysis aims to inform stakeholders about prevailing concerns and their responsibilities in mitigating risks.
2. Literature review
Generative AI models are promising educational tools, but their human-like content raises concerns about learning, academic integrity, and broader educational impacts. This literature review motivates a social-media analysis that mines diverse Twitter opinions and identifies influential stakeholders.
- 2.1. The use of Generative AI models in education: Generative AI models can support personalized and innovative teaching through educational content generation, student-data analysis, and learning-pattern identification.Prior work describes applications including quizzes, tests, worksheets, and insights for refining teaching methods.
- 2.1. The use of Generative AI models in education: Human-like AI-generated content raises concerns that student reliance may hinder critical thinking and problem-solving skills and compromise academic integrity.The authenticity and originality of such content remain contested because models can mimic human writing.
- 2.2. Opportunities and risks of using generative AI models in education: Existing concerns regarding ChatGPT’s educational implementation were compiled from prior research to organize ethical and practical risks.Table 1 presents the review’s concern inventory, although the supplied caption does not provide individual entries.
- 2.3. Social media discussion of ChatGPT: Prior studies using social media found generally favorable perceptions of ChatGPT alongside concerns about education, cheating, honesty, privacy, and manipulation.These studies examined Twitter and TikTok discussions, including stakeholders’ perceptions, user experiences, and student usage.
- 2.3. Social media discussion of ChatGPT: The study addresses gaps in expert- and literature-based reviews by using NLP to mine a broad Twitter corpus and social-network analysis to identify influential users and profile information.The approach is intended to capture more diverse, current public perspectives and provide practical implications.
3. Data and methods
The study collected English-language Twitter data about ChatGPT in education, classified tweet sentiment, modeled negative tweets into topics, and analyzed user networks to identify concern themes and influential accounts.
- 3.1. Data collection: 247,484 tweets were collected through the academic Twitter Search API using education-related terms and “ChatGPT” between December 1, 2022, and March 31, 2023.The dataset included original tweets, mentions, replies, and retweets, including 84,828 original tweets.
- 3.2. Sentiment analysis: The “twitter-roberta-base-sentiment-latest” model classified tweet sentiment, and the analysis retained 70,318 negative tweets.Sentiment labels followed the highest softmax score returned by the RoBERTa model.
- 3.3. Topic modeling: BERTopic clustered BERT-based tweet embeddings using UMAP, K-Means, and c-TF-IDF to identify interpretable topics and thematic categories.The researchers tested 50, 100, and 200 clusters and used representative tweets and topical keywords to interpret themes.
- 3.4. Social network analysis: Social network analysis focused on mentions and retweets to identify frequently implicated users and opinion leaders communicating concerns to broader audiences.Mentions identified users receiving attention, while retweets identified users whose messages spread risks and concerns.
- 3.4. Social network analysis: In-degree centrality measured users’ incoming connections to identify influential accounts receiving attention from other Twitter users.NodeXL was used with the CNM algorithm for community structure and the Fruchterman-Reingold layout for network visualization.
4. Results
Twitter users were generally positive about ChatGPT in education, but negative discussions clustered around five recurring concern categories and shifted with notable events and policies.
- Sentiment trends and analysis: Twitter users expressed a generally positive attitude toward ChatGPT in education, with 70,318 positive tweets versus 49,528 negative tweets.
- Sentiment trends and analysis: Negative sentiment followed events including ChatGPT passing professional exams, institutional bans, and an open letter calling for a pause on giant AI experiments.
- Discourse analysis of concerns: BERTopic clustered 200 topics from 16,011 original negative tweets, which were manually classified into concern categories.
- Discourse analysis of concerns: The five most discussed concerns were academic integrity, learning outcomes and skill development, capability limitations, policy and social issues, and workforce challenges.
- Temporal patterns of concerns: Concern priorities shifted over time, with disruption and learning concerns appearing early and policy concerns emerging after New York City schools blocked ChatGPT.
- Discourse analysis of concerns: Users reported cheating and unclear acceptable use, feared weakened critical thinking and problem-solving, and questioned ChatGPT’s false or nonexistent outputs.
- Discourse analysis of concerns: Policy discussions included institutional bans and ethical questions, while workforce concerns focused on educator displacement and the devaluation of professional training.
5. Discussion
The study found generally positive attitudes toward ChatGPT in education alongside five recurring concern categories and identifiable groups shaping the discussion. It translates these findings into stakeholder responsibilities for responsible AI integration.
- Findings: Twitter users generally viewed ChatGPT’s educational applications positively, although sentiment varied around major events and model-related developments.Examples included ChatGPT passing professional exams, updates from model creators, and public comments by technology professionals.
- Findings: Five concerns dominated the discussion: academic integrity, learning outcomes and skill development, capability limitations, policy and social issues, and workforce challenges.Users also reported misinformation, false references, incorrect answers, service breakdowns, and concerns about bans in schools and universities.
- Findings: Tech, education, and media accounts were frequently implicated, but politicians and government agents were barely present in the conversation.The discussion primarily involved technology companies, professors, and news agencies rather than active government oversight.
- Findings: Individual users from education and technology sectors, especially professors, helped drive the wider diffusion of concerns.Thirty users were identified as having concern-related tweets that were retweeted most often.
- Implications: The findings support broader discussion and coordinated responsibilities among policymakers, technology stakeholders, educators, and media organizations.The proposed focus is preparing people to work effectively alongside AI technologies.
- Implications: Policymakers should develop clearer educational AI guidance, while technology companies should improve capabilities and collaborate with educators and policymakers.The recommendations address academic integrity, social impacts, operational problems, access limitations, and responsible use.
- Implications: Educators are encouraged to voice concerns publicly and use ChatGPT as a classroom learning tool.Professors’ views were widely retweeted, indicating that their perspectives were considered influential in the conversation.
6. Conclusions
The study combines sentiment, topic, and social network analyses of Twitter discourse about ChatGPT in education. It finds overall positive attitudes alongside five concern areas and emphasizes collaborative, ethical governance.
- Conclusion: BERT-based sentiment and topic modeling identified concerns, while social network analysis identified frequently implicated accounts in the discussion.The study used Twitter discourse to examine both public concerns and influential participants.
- Conclusion: Twitter users were broadly positive toward ChatGPT, but negative tweets clustered around five concerns: integrity, learning and skills, capabilities, policy and society, and workforce challenges.The five areas capture concerns about academic integrity, learning outcomes, limitations, social policy, and employment.
- Conclusion: Tech, education, and media users were highly implicated, while individual education and technology users helped diffuse concerns to broader audiences.The conclusion distinguishes sectoral involvement from the leadership role of individual users.
- Implications: The study calls for collaboration among policymakers, technology stakeholders, educators, and media agencies to establish responsible AI-in-education guidelines.The stated goal is to address identified risks while preserving the educational opportunities of generative AI.
viewpoint. American Journal of Medicine Open, 100036. https://doi.org/10.1016/j.ajmo.2023.100036
This passage set consists of bibliographic entries for prior work on generative AI, education, social media, machine learning, and related methods. It does not provide substantive findings for an independent section summary.
- Methods: The references also cover machine-learning and network-analysis methods used in computational text and social-media research.Entries include BERT, TweetEval, BERTopic, community detection, graph drawing, and code-evaluation research.
- Related work: The references include studies on ChatGPT, generative AI, and educational applications.Several entries concern opportunities, challenges, student productivity, teaching, learning, and responsible implementation.
- Related work: The bibliography includes prior analyses of social media data and public sentiment about ChatGPT and related technologies.Referenced work includes Twitter, TikTok, and other social-media-oriented studies.