Source-linked AI summary
Can AI-Assisted Inquiry Enhance Students' Decision-Making Skills in Socio-Scientific Issues? A Three-Group Experimental Study on Climate Change
Dimitrios Gousopoulos
TL;DR
The study asks whether conversational AI deepens students’ climate-change decision-making or does the thinking for them. In a three-group pretest-posttest experiment, AI-assisted inquiry produced the largest gains, exceeding inquiry-only and traditional instruction.
Problem
Students must make defensible climate-change decisions by weighing evidence, competing values, interests, trade-offs, and uncertainty, but whether AI deepens this reasoning remains unclear.
Method
A quasi-experimental pretest-posttest study compared AI-assisted inquiry, inquiry-only, and traditional instruction among 270 secondary students.
Results
AI-assisted inquiry produced the largest gains across all seven decision-making steps, with 68% of student-by-step ratings rising at least one rubric level.
Takeaways & Limitations
AI appears most useful when designed to question rather than answer and embedded in inquiry, which accounts for much of the observed benefit.
Takeaways & Limitations
Because the intervention was short and post-testing followed soon after, the durability and transfer of the gains remain unresolved.
Abstract
from arXiv · showhide
Climate change is a socio-scientific issue: it rests on science but cannot be settled by science, because any serious response forces people to weigh costs, values, and competing interests under uncertainty. Helping students make such decisions well is a central aim of science education, and the arrival of generative artificial intelligence raises a sharp question: does a conversational AI partner deepen students' reasoning, or simply do the thinking for them? This study tested whether AI-assisted inquiry improves secondary students' decision-making about climate change. Using a pretest-posttest design with three groups (AI-assisted inquiry, inquiry without AI, and traditional instruction; 270 students, 90 per group), reasoning was assessed across seven decision-making steps, from defining the problem to monitoring with adaptive management, using a four-level analytic rubric scored through content analysis with high inter-coder agreement. All three groups began at comparable, mostly low levels and all improved, but the gains differed sharply. The AI-assisted group improved most, ahead of inquiry-only and of traditional instruction. Between-group effect sizes on gains were large for AI-assisted versus traditional instruction and moderate-to-large for AI-assisted versus inquiry-only, with the clearest advantages on stakeholder engagement, alternatives, implementation, and monitoring. Within the AI group, the number of times students checked the AI's claims against the sources predicted their gains, and no student was flagged for over-reliance. The findings suggest that AI helps most when it is designed to question rather than to answer, and that the inquiry it is embedded in carries much of the benefit.
Abstract · Introduction
Socio-scientific issues require students to use evidence while balancing trade-offs, competing values, interests, and uncertainty. The study frames AI-assisted inquiry as a potentially distinct scaffold for developing and assessing decision-making across seven stages.
- Abstract: The study concerns socio-scientific issues, decision-making, generative artificial intelligence, inquiry-based learning, climate change education, and secondary education.These terms define the paper’s central subject areas.
- Introduction: Socio-scientific issues combine scientific and social dimensions, so evidence alone cannot determine how societies should respond.Examples include climate change, pandemics, emerging technologies, and limited natural resources.
- Introduction: Responsible classroom decision-making requires students to balance trade-offs, competing values and interests, and uncertainty.The goal extends beyond explaining science to deliberating and reaching defensible decisions.
- Introduction: Decision-making is structured as seven stages spanning evidence, stakeholder engagement, alternatives, comparison, action planning, implementation, and monitoring with adjustment.This structure supports both instruction and assessment of reasoning at each stage.
- Introduction: Inquiry-based learning develops higher-order thinking by having students investigate real questions, use evidence, and construct and test explanations.Its effectiveness depends on adequate support during inquiry.
- Introduction: Generative AI introduces a conversational scaffold that can ask probing questions, while the study design separates AI effects from inquiry effects.The comparison uses three groups and assesses reasoning across seven decision-making steps with a graded rubric.
- Introduction: The research examines students’ initial decision-making levels, whether AI-assisted inquiry outperforms inquiry without AI and traditional instruction, and which steps improve most or least.It also considers within-level and between-level shifts to examine how AI shapes reasoning.
- Introduction: The paper situates socio-scientific issue teaching within Vision II and Vision III approaches to science education.These approaches connect science learning with situations citizens encounter and the ethical and social dimensions of navigating them.
Decision-making and socio-scientific reasoning
Socio-scientific decision-making involves generating and weighing positions on messy, open problems by integrating evidence with emotions and values. Explicit strategies and structured phases can support students’ competence in developing, comparing, implementing, and monitoring solutions.
- Informal reasoning: SSI decisions require generating and weighing positions on messy, open problems through informal reasoning.This reasoning addresses problems that cannot be settled by a single straightforward calculation.
- Informal reasoning: Students’ SSI reasoning combines rationalistic, emotive, and intuitive patterns rather than relying only on evidence or calculation.A good decision blends evidence with feeling and value.
- Decision-making phases: Structured socio-environmental problem-solving frameworks organize decision-making into defining problems, gathering evidence, engaging stakeholders, comparing options, implementing, and monitoring.These interlocking phases provide a common structure for reasoning about socio-environmental solutions.
- Decision-making phases: Explicitly teaching decision-making strategies raises secondary students’ competence on socio-scientific issues.Instruction can support students as they examine values and norms while developing and judging possible solutions.
Inquiry-based learning and scaffolding
Inquiry-based learning supports higher-order thinking most strongly when students receive appropriate guidance, while AI appears most beneficial when embedded as a scaffolded dialogue partner within supervised pedagogy. However, unstructured AI use can encourage cognitive offloading, automation bias, and reduced epistemic agency, making climate change a demanding but valuable socio-scientific context.
- Inquiry-based learning: Inquiry-based learning positions students as investigators who question, analyse data, and construct and critique explanations.Its intellectual roots include Dewey’s structured experience and Kolb’s emphasis on reflection.
- Inquiry-based learning: Meta-analyses indicate that inquiry improves conceptual understanding and higher-order thinking, with the largest gains occurring when students are properly guided.
- AI scaffolding: AI supports science learning most effectively as a dialogue partner that scaffolds metacognition, under teacher supervision and task designs that determine how it is used.Evidence for conceptual gains is promising but mixed.
- Risks and epistemic agency: Unstructured or heavy AI use can promote cognitive offloading, weaker critical thinking, and metacognitive laziness, especially among younger users.Offloading may reduce mental demands while also limiting effortful processing needed for learning and memory.
- Risks and epistemic agency: Automation bias can lead students to accept incorrect AI suggestions, so deliberate friction and reflective prompts are needed to preserve critical reliance and epistemic agency.Fluent falsehoods from large language models make inappropriate deference a persistent risk.
- Climate change as a socio-scientific issue: Climate change is a challenging socio-scientific issue because responding to it requires integrating scientific evidence with social, economic, and ethical judgments across decades and continents.Effective classroom approaches are personally relevant, deliberative, and action-oriented, while misconceptions remain a recurring obstacle.
Participants · The socio-scientific issue and learning context
The study involved 270 secondary-school students investigating an ill-structured, value-laden climate-change socio-scientific issue with no single right answer. In the AI-assisted condition, a generative-AI assistant was designed to scaffold inquiry through questioning, evidence demands, and stakeholder perspectives rather than provide answers.
- Participants: 270 secondary-school students participated, assigned equally across three conditions with 90 students per group.Students came from public schools and were assigned at the level of intact classes.
- Participants: Grades 8 to 11 were represented, with a mean age of 14.6 years and a standard deviation of 1.1.The sample was drawn from lower and upper secondary students.
- Participants: The sample was close to balanced by gender, including 133 male, 129 female, and 8 students reporting other or no response.Whole classes rather than individuals were assigned to conditions, creating class- and school-level clustering.
- The socio-scientific issue and learning context: The climate-change scenario asked adolescents how a community should respond to worsening impacts through mitigation and adaptation while balancing science, cost, fairness, and obligations to future generations.The issue combined evidential and moral reasoning.
- The socio-scientific issue and learning context: The scenario was deliberately ill-structured and value-laden, with no single right answer, to exercise the full set of decision-making steps and range of reasoning.The task required students to weigh scientific evidence alongside values and competing considerations.
- The socio-scientific issue and learning context: The AI-assisted inquiry group used a generative-AI assistant built on a large language model and guided to scaffold rather than answer.Its system instructions and prompting protocol elicited Socratic questions, evidence and justification, stakeholder perspectives, and counter-arguments.
Inquiry-only group (IN-G) · Traditional-instruction control group (C-G) · Procedure
The inquiry-only group completed the same climate-change investigation and decision-making sequence as the AI group, using conventional scaffolds instead of generative AI. The traditional-instruction group received teacher-led content instruction without student-driven investigation or AI, while all groups followed equalized pre-, intervention, and post-research phases.
- Inquiry-only group (IN-G): Students in IN-G investigated the same socio-scientific issue through the same sequence of decision-making steps as AI-G.The condition was designed to isolate the AI contribution by replacing conversational AI with non-generative supports.
- Inquiry-only group (IN-G): IN-G used printed guiding questions matched to the AI protocol, curated resources and data, and teacher facilitation.These supports constituted guided inquiry without generative AI.
- Inquiry-only group (IN-G): The inquiry-only condition enabled a direct comparison between AI-G and IN-G to identify AI’s specific contribution.Both groups otherwise worked through the same investigation and decision-making sequence.
- Traditional-instruction control group (C-G): C-G covered the same climate-change content through explanation, guided discussion, and small-group work led by the teacher.Instruction followed a constructivist practice without student-driven investigation or generative AI.
- Traditional-instruction control group (C-G): C-G mirrored ordinary classroom practice and provided the baseline against which both inquiry conditions were measured.Teaching time was equalized across all three conditions.
- Procedure: The procedure comprised pre-research baseline worksheets, equal-duration interventions, and post-research follow-up worksheets after a set interval.Where feasible, a delayed post-test examined whether skills lasted and transferred rather than merely appearing immediately after instruction.
Instruments · Data analysis
Students completed individual pretest and posttest worksheets covering seven climate-change decision-making steps, assessed through mixed-methods content analysis. Baseline reasoning was mostly low, with particular difficulty in forward-looking implementation and monitoring tasks.
- Instruments: Students individually completed structured worksheets before and after the intervention, organized around seven climate-change decision-making steps.The steps ran from defining the problem and causes through implementation, monitoring, and adaptation.
- Instruments: The authentic scenarios carried ethical and political weight, requiring informal, moral, and socioscientific reasoning rather than recall.
- Instruments: Science-education experts reviewed the instruments for content validity, and the worksheets were piloted with a comparable group.
- Data analysis: Worksheets were coded through systematic qualitative content analysis within a wider mixed-methods framework, guided by the rubric.
- Data analysis: Cohen's kappa showed substantial to near-perfect agreement for each rubric dimension after calibration and proportional independent re-coding across the three conditions.
- Data analysis: About 85% of responses were at Emerging or Developing levels, while monitoring and adaptive management averaged 1.57, decision implementation 1.73, and problem identification 1.99.The weakest performance was on forward-looking steps, whereas problem identification was relatively strongest.
Did the groups improve differently? (RQ2)
All three groups improved significantly, but AI-assisted inquiry produced the largest gains, followed by inquiry-only and traditional instruction. Its advantage was clearest on later, more demanding decision-making steps and in upward rubric-level movement.
- All three groups improved significantly from pre-test to post-test, with the AI-assisted group ahead of inquiry-only and traditional instruction on all seven steps.Every within-group change was statistically significant (all p < 0.001).
- AI-assisted inquiry increased total scores from 12.3 to 18.3 (paired d = 2.94), versus 12.7 to 17.3 (d = 2.43) for inquiry-only and 12.3 to 14.9 (d = 1.68) for traditional instruction.
- 77.9: gain scores differed across the three conditions for the total score, F(2, 267) = 77.9, p < 0.001.The Kruskal-Wallis test agreed throughout, with differences also found for every decision-making step.
- AI-assisted inquiry beat traditional instruction by d = 1.88 and inquiry-only by d = 0.69 on total gains, while inquiry-only beat traditional instruction by d = 1.18.
- The AI advantage over inquiry-only was widest on monitoring (d = 0.52), implementation (d = 0.44), and stakeholder engagement (d = 0.35).Post-test differences were also clearest on stakeholder engagement, decision implementation, and monitoring and adaptive management after correction (alpha = 0.05/7, about 0.007).
- 68% of AI-assisted student-by-step ratings moved up at least one rubric level, compared with 55% for inquiry-only and 33% for traditional instruction.Almost everyone improved on at least one step: AI-G 100%, IN-G 99%, and C-G 91%; no student in any group regressed.
How the AI was used
AI-assisted inquiry involved active interaction and source checking rather than passive consumption. Students who checked AI claims against provided sources more often achieved larger overall gains.
- How the AI was used: Students sent about 20 prompts during the intervention (mean 19.8, standard deviation 4.1).This process indicator did not point to passive consumption.
- How the AI was used: Students checked an AI claim against the provided sources roughly eight times (mean 8.2, standard deviation 2.2).Source checking was a central indicator of how students used the AI.
- How the AI was used: r = 0.52, p < 0.001: more evidence checks correlated positively with total gains.The figure likewise reports that students who checked the AI's claims against sources more often gained more overall.
Discussion
AI-assisted climate-change inquiry produced the strongest improvement in students’ decision-making, but the advantage was conditional: inquiry itself accounted for much of the benefit. The AI-assisted group improved more than inquiry-only and far more than traditional instruction, despite comparable starting levels.
- Students using scaffolding AI improved more than those completing the same inquiry without AI and far more than students taught traditionally.The evidence supports an encouraging but conditional effect of AI assistance rather than an unconditional replacement of student thinking.
- The AI-assisted group made the largest and most numerous upward moves in decision-making, ending clearly ahead of the other two groups.All three groups started at comparable levels, but their outcomes separated by the end of the study.
- Inquiry itself contributed substantially to the gains because the inquiry-only group also outperformed traditional instruction by a wide margin.The three-group design makes it possible to distinguish the contribution of inquiry from the additional contribution of AI assistance.
Limitations
The conclusions are qualified by class-level assignment, imperfect measurement through written worksheets, and a short intervention with an immediate post-test. Clustering, writing fluency, and the durability and transfer of gains therefore remain concerns for future research.
- Design: Intact classes, rather than individual students, were assigned to conditions, so within-class or within-school responses may not be fully independent and precision could be inflated.Baseline equivalence was confirmed and the effects were large, but future work should model clustering explicitly or randomise at the class level.
- Measurement: Written worksheets scored with a rubric may capture reasoning imperfectly and favour students who write fluently, despite strong inter-coder agreement.
- Timing: The intervention was relatively short and the post-test followed soon after, leaving the durability and transfer of gains unresolved.A delayed post-test would test whether the AI-assisted advantage lasts.
Conclusion
AI-assisted inquiry improved secondary students’ climate-change decision-making more than inquiry without AI and substantially more than traditional teaching. Much of the benefit belonged to inquiry, while a carefully designed AI added a further increment, especially for stakeholder engagement and planning and monitoring action.
- Conclusion: AI-assisted inquiry improved students’ decision-making more than the same inquiry without AI and substantially more than traditional teaching.The finding comes from the study’s three-group design in a climate-change context.
- Conclusion: The clearest advantages involved engaging stakeholders and planning and monitoring action.
- Conclusion: Much of the benefit belonged to inquiry, while carefully designed AI added a real increment on top.