Source-linked AI summary
Analysis of Types of Inquiries in Student-AI Interaction: A case study of two CS2 tasks
Matin Amoozadeh, Amin Alipour
TL;DR
Students increasingly use generative AI for programming help, but little is known about how their inquiries evolve during problem solving. This study classifies CS2 students’ AI interactions using an adapted Graesser taxonomy and finds that questioning shifts substantially as tasks progress, from assertion and verification toward more procedural and causal reasoning.
Problem
Little research adapts educational question taxonomies to generative-AI programming interactions or examines how students’ inquiry forms transition during problem solving.
Method
The study uses a few-shot approach grounded in the Graesser taxonomy to classify and analyze inquiry patterns across two CS2 programming tasks.
Results
Students’ questioning shifted substantially across tasks, with early assertion and verification prompts giving way to more procedural and causal reasoning questions.
Takeaways & Limitations
The findings provide insights for designing AI-supported instructional systems that account for evolving inquiry behavior in computing education.
Takeaways & Limitations
The study was conducted during graded laboratory activities in a single CS2 course at a large U.S. public university, with mostly freshmen and sophomores.
Abstract
from arXiv · showhide
Background and Context: Question and inquiry are integral parts of knowledge seeking and learning. Despite their importance, students tend not to ask enough questions in the classroom. However, studies have shown that students interact extensively with generative AI systems for learning and problem solving. Objective: In this paper, we seek to better understand the types of questions that students ask AI systems, and how those questions evolve during problem solving and across tasks. Method: We use the Graesser et al. taxonomy to classify students' inquiries into 18 types. We develop a few-shot learning approach to automatically classify students' interactions with AI into these categories. We use this system to analyze 830 interactions of CS2 students across two programming tasks. Findings: Our results suggest that a small subset of question types accounts for the majority of student inquiries, and that the types of questions students ask change substantially as the task progresses.
1 Introduction
This study examines how CS2 students’ questions to AI systems are structured and evolve during programming problem solving. It applies the Graesser et al. taxonomy and a few-shot classifier to 830 interactions across two C++ programming tasks.
- Motivation: Students increasingly use AI alongside or instead of peers, teaching assistants, documentation, and online forums during programming activities.This makes understanding student-AI interaction an important challenge for computing education research.
- Research gap: Prior research has focused mainly on performance, perceptions, or usage frequency, leaving the structure and evolution of students’ AI-directed questions insufficiently examined.The study addresses how students transition between different inquiry forms during AI-supported programming problem solving.
- Method: The study adapts the Graesser et al. taxonomy to classify inquiries by cognitive and functional roles, including verification, procedural guidance, causal reasoning, interpretation, and evaluation.The framework supports fine-grained analysis of question asking in AI-supported programming tasks.
- Method: 830 prompts from CS2 students were analyzed using a few-shot learning approach across two laboratory sessions on object-oriented programming in C++.The analysis examines question-type distributions, transitions between inquiry states, and changes across sessions.
- Findings: Students frequently used assertion and verification prompts early, whereas later sessions showed greater use of procedural and causal reasoning questions.The results indicate that questioning behavior changes as tasks progress.
- Research questions: The paper investigates inquiry types, transitions between question types, and changes in questioning behavior across programming tasks.It reports shifts across conceptual areas and varying degrees of cognitive engagement, including increased reliance on AI for design decisions about polymorphism.
2 Related Work
Generative AI enables continuous conversational help-seeking during programming, while established question taxonomies support analysis of learners’ cognitive and functional inquiry roles. However, relatively little research adapts these taxonomies to programming-related student-AI interactions or examines how inquiry forms evolve across conversations.
- Help-Seeking in Computing Education: Generative AI creates a continuous conversational form of help-seeking for students programming with AI assistants.This differs from traditional help-seeking interactions by enabling ongoing exchanges during programming.
- Question Asking and Educational Dialogue: Graesser and Person’s taxonomy classifies questions by cognitive and functional roles, including verification, causal reasoning, procedural inquiry, interpretation, and evaluation.The framework has been applied in educational dialogue research and intelligent tutoring systems to study information seeking and understanding during problem solving.
- Question Asking and Educational Dialogue: Question types can reflect students’ cognitive processes, misconceptions, engagement levels, and productive or unproductive learning strategies.Educational dialogue and intelligent tutoring research uses inquiry behavior to investigate these learning-related characteristics.
- Research Gap: Relatively little work adapts educational question taxonomies to conversational student-AI interactions in programming education.Existing research often emphasizes performance outcomes, perceptions, or isolated prompt characteristics rather than inquiry evolution across interactions.
3 Methodology
The study examined CS2 students’ AI-supported programming work across two laboratory sessions using an integrated programming environment and a few-shot classification of prompts into 18 Graesser question categories. Researchers analyzed logged inquiries descriptively, compared sessions and participant groups, and modeled transitions among question types.
- Study setting and participants: The study involved CS2 students learning C++ object-oriented programming during supervised laboratory assignments at a large public U.S. university.Data collection occurred during the Fall 2025 semester in the regular classroom environment with teaching assistants present.
- Study setting and participants: Two approximately 60-minute sessions used programming tasks differing in topic and conceptual complexity, enabling comparisons of students’ AI interactions across sessions.Session 1 covered constructors, encapsulation, and method implementation; Session 2 covered inheritance and polymorphism using Pet and Dog classes.
- Study setting and participants: 72 unique students participated: 60 in Session 1 and 37 in Session 2, with 25 participating in both sessions.The repeated participants enabled examination of how inquiry patterns evolved across different points in the semester.
- Data collection and platform: Students worked individually in a web-based environment integrating assignment instructions, code editing, program execution, and an AI chat assistant.The system automatically logged prompts, responses, timestamps, user identifiers, and interaction metadata; the assistant was powered by GPT-4 through the OpenAI API.
- Prompt classification: Student prompts were classified into 18 Graesser question categories using few-shot learning with large language models and labeled examples.GPT-5.2 and Claude classifications were independently reviewed by two authors, with disagreements manually examined and resolved through discussion.
- Quantitative analysis: The analysis examined category distributions, session differences, changes among common participants, and within-session transitions represented as a state machine.The descriptive analysis included 830 prompts: 432 from Session 1 and 398 from Session 2.
4 Results
Across 830 prompts from two sessions, students’ inquiry patterns shifted from confusion-reporting toward more procedural and task-oriented interaction, while question usage differed by generation status. First-generation students asked fewer questions overall, and session-specific shifts were descriptive rather than statistically significant after correction.
- Dataset summary: 830 prompts were recorded from 97 session participations: Session 1 produced 432 inquiries from 60 students, and Session 2 produced 398 from 37 students.The corresponding averages were 7.20 prompts per student in Session 1 and 10.76 in Session 2.
- Question-type distributions: Assertion was most frequent in both sessions, with S1 = 86 and S2 = 82, while Instrumental/procedural prompts became dominant in Session 2.Assertion represented confusion, lack of understanding, or problem reports without a specific question.
- Question-type distributions: Verification decreased from S1 = 88 to S2 = 66, whereas Request/Directive increased from S1 = 41 to S2 = 52.Interpretation and Judgment remained secondary, while several higher-order categories had lower presence.
- Statistical comparison: The session difference approached but did not reach conventional significance, χ2(17) = 27.03, p = .0576.The result indicated a trend from confusion-reporting in Session 1 toward more procedural and task-oriented inquiry in Session 2.
- Generation-status patterns: First-generation students asked fewer questions overall: 161 versus 271 in Session 1 and 159 versus 239 in Session 2 for first-generation and continuing-generation students, respectively.Between sessions, first-generation students increased Instrumental/Procedural questions from ≈12.5% to ≈27%, while continuing-generation students shifted toward Verification from ≈8.5% to ≈18%; these shifts were not statistically significant after Bonferroni correction.
- Question-type transitions: Students most often began with Assertion in Session 1 (prob=0.48) and Instrumental/Procedural questions in Session 2 (prob=0.43), tending to remain longer in Assertion, Request/Directive, and Instrumental/Procedural states.The paper associates the second session’s broader variety of question types with greater maturity in question-asking skills.
5 Discussion and Concluding Remarks
The discussion highlights persistent differences in AI-supported help-seeking, with first-generation students asking fewer questions, especially during the more complex second task. It also emphasizes that students’ inquiry behaviors evolved as programming tasks became more conceptually demanding, shifting toward assertion and verification in Session 1.
- First-Generation Students and AI-Supported Help-Seeking: First-generation students asked fewer questions than continuing-generation students, consistent with prior evidence of lower help-seeking and questioning behavior.This difference persisted despite conversational AI potentially reducing interpersonal barriers associated with classroom help-seeking.
- First-Generation Students and AI-Supported Help-Seeking: During the more complex second task involving object-oriented programming concepts, first-generation students asked substantially fewer questions.The finding suggests AI-supported tutoring systems may need scaffolding that encourages question formulation and active help-seeking.
- Dynamics of Inquiry in AI-Supported Programming: As programming tasks became more conceptually demanding, students’ questioning behaviors evolved across sessions.In Session 1, students relied more heavily on assertion and verification prompts to confirm their understanding of foundational concepts, reflecting novice help-seeking patterns.