PaperlayerView exact revision on arXiv

Source-linked AI summary

Using ChatGPT for Science Learning: A Study on Pre-service Teachers' Lesson Planning

Gyeong-Geon Lee, Xiaoming Zhai

arXiv:2402.01674v1cs.CYphysics.ed-ph

TL;DR

Empirical evidence about ChatGPT’s practical use in science teaching and learning remains limited. This study analyzed pre-service teachers’ science lesson plans and related perceptions; their plans received relatively high TIAR scores, though classroom use remains to be examined.

Abstract

from arXiv · show

Despite the buzz around ChatGPT's potential, empirical studies exploring its actual utility in the classroom for learning remain scarce. This study aims to fill this gap by analyzing the lesson plans developed by 29 pre-service elementary teachers from a Korean university and assessing how they integrated ChatGPT into science learning activities. We first examined how the subject domains and teaching and learning methods/strategies were integrated with ChatGPT in the lesson plans. We then evaluated the lesson plans using a modified TPACK-based rubric. We further examined pre-service teachers' perceptions and concerns about integrating ChatGPT into science learning. Results show diverse applications of ChatGPT in different science domains. Fourteen types of teaching and learning methods/strategies were identified in the lesson plans. On average, the pre-service teachers' lesson plans scored high on the modified TPACK-based rubric, indicating a reasonable envisage of integrating ChatGPT into science learning, particularly in 'instructional strategies & ChatGPT'. However, they scored relatively lower on exploiting ChatGPT's functions toward its full potential compared to other aspects. The study also identifies both appropriate and inappropriate use cases of ChatGPT in lesson planning. Pre-service teachers anticipated ChatGPT to afford high-quality questioning, self-directed learning, individualized learning support, and formative assessment. Meanwhile, they also expressed concerns about its accuracy and the risks that students may be overly dependent on ChatGPT. They further suggested solutions to systemizing classroom dynamics between teachers and students. The study underscores the need for more research on the roles of generative AI in actual classroom settings and provides insights for future AI-integrated science learning.

I. INTRODUCTION

Research on ChatGPT in education has emphasized its anticipated potential, while empirical evidence of its classroom use in science learning remains limited. This study examines pre-service teachers’ plans for integrating ChatGPT and situates that work within broader discussions of technology integration.

I. INTRODUCTION

  • Prior discussion of ChatGPT’s educational potential was largely visionary, prompting questions about how broadly it could be used across science domains and teaching strategies.The authors also note limited empirical evidence on teachers’ and students’ views of its utility in science learning.Source: §I · p. 1
  • The study examines how pre-service science teachers incorporated ChatGPT into lesson plans, addressing limited empirical research on its practical use in science education.The authors frame lesson-plan analysis as a way to gain insight into pre-service teachers’ preparedness and inclination to embed technology in science curricula.Source: §I · p. 2
  • Three questions address ChatGPT’s range of uses in science lesson plans, teachers’ planning proficiency, and their perceived benefits and concerns.Source: §I · p. 2

A. Technology Integration in Science Education

  • Technology integration in science education has expanded alongside ICT and AI, with the pandemic accelerating the shift toward technology when face-to-face instruction became impractical.Source: A. Technology Integration in Science… · p. 2
  • Common teacher-reported barriers to integrating technology include inadequate resources, limited time and funding, technical issues, and resistance to change.The passage also identifies willingness to use technology, supportive resources, and a conducive environment as important to successful integration.Source: A. Technology Integration in Science… · p. 2
  • Constructivist perspectives position technology as a mediating tool and emphasize students taking greater responsibility for learning, making teachers’ technology-integration perspectives a relevant research focus.Source: A. Technology Integration in Science… · p. 2

B. Large-language Models and Education

The paper situates ChatGPT within proposed educational applications of language models and the limited empirical study of their use in science pedagogy. It also uses lesson plans as a lens on teacher knowledge and describes the participating pre-service teachers.

B. Large-language Models and Education

  • ChatGPT has been proposed for science-learning activities including questioning, content creation, formative assessment, and other instructional tasks, alongside concerns about dependence undermining authentic learning.The authors note that much LLM assessment research has focused on automated assessment, while interest in tangible educational settings is growing.Source: B. Large-language Models and Education · p. 3
  • The authors identify a lack of empirical studies examining ChatGPT or similar LLMs integrated into science pedagogy.They argue that investigating potential classroom roles can inform discussion of LLMs in future science education.Source: B. Large-language Models and Education · p. 3

C. Lesson Plan Analysis and Science Teachers' TPACK

  • Lesson plans reflect teachers’ content, pedagogical, and technological knowledge, and TPACK provides a framework for considering their integration in science instruction.The paper notes a lack of research on teachers’ TPACK specifically in relation to LLMs or ChatGPT.Source: C. Lesson Plan Analysis and Science… · p. 3

A. Participants

  • The study included 29 pre-service elementary science teachers, mostly sophomores at a Korean teachers’ university, who had completed school-based teacher training.Source: A. Participants · p. 3

B. Context

Participants received four weekly sessions introducing ChatGPT, science-teaching strategies, and lesson planning before drafting a lesson around a ChatGPT interaction. The study evaluated plans with an adapted TPACK rubric and analyzed survey responses and simulated chat data.

B. Context

  • Four weekly two-hour sessions introduced ChatGPT, its educational possibilities, science teaching methods, and lesson planning, with instructor feedback during plan development.Examples included using ChatGPT to make and revise science analogies, which could improve elaboration or reduce scientific rigor and mapping quality.Source: B. Context · pp. 3–4

C. Data Collection

  • Each participant drafted a 45-minute elementary science lesson using at least one studied teaching method and a ChatGPT query, and attached a simulated conversation aligned with the plan.Participants had access to the Korean national curriculum and elementary science textbooks.Source: C. Data Collection · p. 4

D. Rubric

  • The study adapted the TIAR rubric by replacing technology with ChatGPT and assessing curriculum goals, instructional strategies, function selection, and overall fit.Function selection measured how well the plan used ChatGPT’s affordances to support student learning; the four-item rubric’s reported reliability was α = .69 across 29 plans.Source: D. Rubric · p. 4

E. Survey

  • An open-ended survey asked participants why they selected their lesson content and methods, why they considered ChatGPT beneficial, and what outcomes they anticipated.Source: E. Survey · p. 4

F. Analysis

  • Two researchers coded lesson plans and survey responses, resolving discrepancies through discussion; survey categories were refined iteratively, and attached ChatGPT conversations supported triangulation.Survey analysis used the constant comparative method.Source: F. Analysis · p. 4

Ⅳ. RESULTS

Across 29 plans, pre-service teachers showed relatively high TIAR scores, with a significant advantage for integrating ChatGPT into instructional strategies over selecting its functions. Plans covered multiple science domains and strategies, while examples included both suitable and unsuitable applications.

A. How ChatGPT was Integrated in Science Lesson Plans by Pre-service Teachers

  • Plans addressed biology (9/29), chemistry and earth science (7/29 each), and physics and environmental science (4/29 each), with cross-cutting plans allowed to select multiple domains.Source: A. How ChatGPT was Integrated in… · p. 5
  • Cooperative learning (7/29), POE and the learning cycle (6/29 each), analogy generation (4/29), experimentation (3/29), and concept mapping (3/29) appeared among the strategies used.Source: A. How ChatGPT was Integrated in… · p. 5
  • Participant_11’s ecology role-play plan proposed ChatGPT for extending textbook learning, preparing roles and scripts, adapting support, and providing feedback and reflection.The plan was designed for fifth-grade students and also included classroom phases, materials, and evaluation criteria.Source: A. How ChatGPT was Integrated in… · p. 5

B. Strengths and Weaknesses of ChatGPT-integrated Lesson Plan by Pre-service Teachers

  • M = 3.29 (SD = .91) on the 4-point TIAR, and instructional strategies & ChatGPT scored significantly higher than ChatGPT function selection(s).The respective item means were M = 3.48 (SD = .99) and M = 3 (SD = .93); the paired comparison was t(28) = 2.3173, p = .0280.Source: B. Strengths and Weaknesses of… · p. 5
  • Plans aligned ChatGPT with curriculum goals including conceptual knowledge, scientific inquiry and thinking, creative thinking, motivation, interest, and attitude.The reported concepts included the solar system and stars, acids and bases, life and environment, and animal life.Source: B. Strengths and Weaknesses of… · pp. 5–6
  • Teachers proposed ChatGPT for cooperative learning, POE prediction or explanation, learning-cycle exploration, and language-based analogy generation.Cooperative learning was the most common strategy (7/29), followed by POE and the learning cycle (6/29 each).Source: B. Strengths and Weaknesses of… · p. 6
  • Appropriate uses included supplementary information, role-play organization, scenario summarization, and extended questioning, while unsuitable examples included visual constellation searches and nonexistent video links.One proposed use—having student groups design acid-and-base experiments—was flagged as potentially undermining practical inquiry skills over time.Source: B. Strengths and Weaknesses of… · p. 6
  • The authors attributed variation in ChatGPT function-selection scores to a mixture of suitable and unsuitable applications in the lesson plans.Source: B. Strengths and Weaknesses of… · p. 6

C. Pre-service Teachers' Perceived Usefulness and Concerns of Integrating ChatGPT in Lesson Planning

Pre-service teachers anticipated ChatGPT supporting questioning, individualized and self-directed learning, formative evaluation, and digital competency, while raising concerns about accuracy and over-dependence. They also proposed diversifying information sources and structuring classroom interactions to address dependence.

Perceived usefulness

  • 16/29 pre-service teachers anticipated that ChatGPT could support knowledge construction through questioning and self-paced learning.They linked consecutive questioning to active participation and managing one's own learning process.Source: C. Pre-service Teachers' Perceived… · p. 6
  • Pre-service teachers valued multi-turn dialogue for consecutive questions and saw it as a possible way to bolster critical thinking.Three participants thought it could outperform standard internet searches for information retrieval; five viewed consecutive questioning as a critical-thinking opportunity.Source: C. Pre-service Teachers' Perceived… · p. 6
  • 9/29 anticipated individualized learning support, while 8/29 saw potential for formative evaluation through feedback or evaluation in classroom activities.Some also imagined comparing students' answers with ChatGPT's responses to deepen answers.Source: C. Pre-service Teachers' Perceived… · pp. 6–7
  • 4/29 argued that ChatGPT integration could foster students' digital competency or literacy for an increasingly AI-centric society.Three participants specifically connected this benefit to AI-centric industries.Source: C. Pre-service Teachers' Perceived… · p. 7

Concerns and responses

  • 11/29 were concerned about ChatGPT's accuracy and reliability, including possible misconceptions and incorrect lab-safety information; 13/29 worried about over-dependence.Almost all of the 11 teachers raising dependence concerns proposed remedies; one was diversifying information sources, including teachers, students, and peers.Source: C. Pre-service Teachers' Perceived… · p. 7
  • 11/29 proposed structuring classroom interactions before, during, and after ChatGPT use to reduce over-dependence.Suggestions included preparation in digital literacy and critical thinking, group discussions, and experiments to verify predictions made with ChatGPT.Source: C. Pre-service Teachers' Perceived… · p. 7

V. DISCUSSION

The study examined 29 pre-service teachers after professional learning about ChatGPT, finding extensive envisioned applications, moderate lesson-planning competence, and balanced views of its merits for science education.

V. DISCUSSION

  • The study found extensive envisioned applications of ChatGPT in teaching and learning, moderate competence in integrating it into lesson plans, and balanced perspectives on its merits for science education.The study provided professional learning about ChatGPT to 29 pre-service teachers and examined their lesson plans and perspectives.Source: §V · p. 7

A. Pre-service teachers envisioned the extensive applicability of ChatGPT in science teaching and learning (RQ1)

Pre-service teachers envisioned ChatGPT across science domains and teaching strategies, while the discussion calls for further exploration of learning settings and potential implementation barriers.

A. Pre-service teachers envisioned the extensive applicability of ChatGPT in science teaching and learning (RQ1)

  • Lesson plans integrated ChatGPT across biology, chemistry, earth science, physics, and environmental science, using 14 teaching and learning methods or strategies.Cooperative learning, POE, and the learning cycle each appeared in 6/29 plans; analogy generation appeared in 4/29.Source: A. Pre-service teachers envisioned the… · p. 7
  • Participants primarily envisioned face-to-face use, and the discussion calls for exploring remote learning and developing innovative science teaching strategies that leverage ChatGPT.The authors also urge further exploration of ChatGPT's applications in science teaching and learning.Source: A. Pre-service teachers envisioned the… · p. 8
  • The discussion cautions that technical problems, poor administrative support, and poor training may remain barriers to integrating ChatGPT across science teaching formats.The authors recommend identifying potential classroom integration barriers even if individual teachers' behavior, investments, and commitment pose minimal barriers.Source: A. Pre-service teachers envisioned the… · p. 8

integrating ChatGPT as technology for teaching and learning (RQ2)

Lesson plans showed an average TPACK-based score above the midpoint, with stronger scores for curriculum goals and instructional strategies than for ChatGPT function selection and fit. The discussion calls for professional development focused on teachers' use of ChatGPT functions.

integrating ChatGPT as technology for teaching and learning (RQ2)

  • 3.29/4 was the average lesson-plan score, above the 2.5 midpoint; instructional strategies & ChatGPT scored 3.48/4, while ChatGPT function selection(s) scored 3/4.Curriculum goals & ChatGPT scored 3.45/4, and ChatGPT fit scored 3.24/4.Source: integrating ChatGPT as technology for… · p. 8
  • The authors call for professional development in AI integration, particularly to address pre-service teachers' limited understanding of ChatGPT functionalities.They also call for qualitative investigation of how teachers choose and arrange content, instructional strategies, and ChatGPT functions.Source: integrating ChatGPT as technology for… · p. 8

merit of integrating ChatGPT for teaching and learning (RQ3)

Pre-service teachers anticipated that ChatGPT could support student questioning, self-directed and personalized learning, and formative assessment, while identifying misinformation and student overdependence as concerns requiring structured classroom interactions.

merit of integrating ChatGPT for teaching and learning (RQ3)

  • ChatGPT was anticipated to support richer student questioning and, in turn, self-directed and personalized learning beyond constraints imposed by teacher-student ratios.Some pre-service teachers also proposed formative assessment through real-time feedback.Source: merit of integrating ChatGPT for… · p. 8
  • Some pre-service teachers anticipated ChatGPT could contribute to formative assessment in science classrooms through automated, real-time feedback.Source: merit of integrating ChatGPT for… · p. 8
  • Pre-service teachers identified misinformation as a risk and proposed checking ChatGPT’s information against teachers, peers, and other sources.They also suggested structured discussions, POE, student-led presentations, and teacher feedback to counter possible overdependence.Source: merit of integrating ChatGPT for… · p. 8

D. Limitations and Future Directions

The authors identify limits in relying on lesson plans, affordability, the study’s data-collection period, and uncertainty about classroom impacts, and outline directions for research and technical support.

D. Limitations and Future Directions

  • Because the study relies on lesson plans, planned ChatGPT use may differ from enacted lessons, leaving actual classroom utilization and perceived usability for empirical study.Source: D. Limitations and Future Directions · p. 8
  • ChatGPT’s subscription fee may make it unaffordable for underresourced teachers and students; the authors suggest exploring open-source LLMs for equitable and transparent practice.Source: D. Limitations and Future Directions · p. 8
  • Data were collected in May–June 2023, when few prompt-engineering techniques had reached broader users; the authors call for technical support for teachers integrating LLMs.By January 2024, tools tailored to specific scientific content and instructional methods were deemed possible.Source: D. Limitations and Future Directions · p. 8
  • The authors call for strategies against teachers’ excessive reliance on GenAI, which may present incorrect or nonexistent information that teachers accept without checking.They identify critical thinking and digital literacy about ChatGPT’s output as important expertise for future science teachers.Source: D. Limitations and Future Directions · p. 8
  • The study does not establish student-level effects; future work should examine science learning, inquiry, and attitudes, including through quasi-experimental comparisons of experimental and control groups.Source: D. Limitations and Future Directions · pp. 8–9

E. Implications

The authors present the lesson-plan findings as insights for future AI-integrated science education, emphasizing digital literacy, broad potential for ChatGPT integration, and teachers’ continuing role in managing classroom use.

E. Implications

  • Teachers and students need digital or AI literacy to discern which GenAI-generated content is suitable for learning.Source: E. Implications · p. 9
  • The lesson plans suggest ChatGPT could support personalized learning, classroom management in large classes, and interactive activities across many combinations of science domains and instructional methods.The authors describe this apparent breadth as scalability and call for development of ChatGPT-integrated science instructional programs.Source: E. Implications · p. 9
  • Pre-service teachers voluntarily proposed ways to address ChatGPT’s pitfalls, including cross-checking information and structuring classroom interactions to manage student use.The authors interpret this as responsive TPACK that preserves teachers’ directing role in classrooms.Source: E. Implications · p. 9

lesson plans (revised from Harris et al. [22])

The excerpt illustrates ChatGPT integration through an ecological-balance lesson that combines individual exploration with group role play. Its plan also specifies teacher guidance intended to support student thinking and limit over-reliance.

lesson plans (revised from Harris et al. [22])

  • The rubric’s fit descriptions range from a mismatch among content, strategy, and ChatGPT function to a strong fit among all three.The excerpt also gives intermediate descriptions of partial and somewhat aligned fit.Source: lesson plans (revised from Harris et… · p. 11
  • The example lesson uses ChatGPT to explore ecological-balance disruption and, if needed, draft a role-play script within a 40-minute lesson.The plan names role play as its teaching strategy and ecological balance as its topic.Source: lesson plans (revised from Harris et… · p. 18
  • The teacher is to teach ChatGPT use and prompting, elicit students’ own ideas first, distinguish prior knowledge from chatbot output, and check accuracy against science.The plan also calls for ongoing feedback and thinking-stimulating questions.Source: lesson plans (revised from Harris et… · p. 19
  • Students first record prior knowledge, use ChatGPT to investigate causes and solutions, then choose a cause to develop into a group role play.The plan separates individual exploration from collaborative script development.Source: lesson plans (revised from Harris et… · p. 19
  • Group guidance emphasizes discussion and creative ideas over dependence on ChatGPT, while allowing it to suggest or refine ideas and summarize lengthy scripts.The role play is limited to three minutes.Source: lesson plans (revised from Harris et… · p. 19
Loading 2402.01674v1…