Characterizing Questioning Patterns and Student Engagement Through Contextual Analysis of Real-Time Classroom Interactions

1 Introduction

• Correctness and participation rates reveal little about a poll’s instructional purpose or the thinking it requires.The same True/False format can test recall, check comprehension, reinforce an idea, or direct attention to a deliberate error.Source: §1 · p. 2
• A poll’s meaning depends on the lecture material and timing surrounding it, not wording alone.A factual explanation may make a True/False item a recall check, while a conceptual explanation may make it a comprehension check.Source: §1 · p. 2
• The study examines polls as interactions among lecture context, questions, and responses using 604 polls from 47 live sessions.It uses time-aligned poll records, lecture transcripts, and attendance data.Source: §1 · p. 2

2 Background Study

• Polling reviews report increased participation, attention, and timely feedback, with benefits generally modest and dependent on how polling is used,,,.Outcomes appear to depend more on surrounding pedagogy than on the technology itself.Source: §2 · p. 3
• Peer Instruction and formative-assessment research emphasize question purpose and quality, including reasoning, misconception detection, and distinct feedback functions,,,,.Peer discussion between successive votes can improve student responses,.Source: §2 · p. 3
• Cognitive-psychology studies report that answering questions can strengthen retention and that brief in-lecture questions can reduce mind-wandering and improve subsequent retention.Source: §2 · p. 3
• Bloom’s taxonomy and teacher-questioning research describe cognitive demands, while learning analytics finds uneven participation,,,,,,.These areas generally examine question cognition, participation, and poll context separately; evidence connecting contextual interpretations to students’ reported experience is limited.Source: §2 · pp. 3–4

3.1 Research Questions

• The study asks whether lecture context can determine a poll’s answer and function, what cognitive levels and functions occur, and how engagement relates to correctness.Source: §3.1 · p. 4

3.2 Unit of Analysis and Coding

• Each poll is analysed with surrounding discourse, its question, and response distribution, allowing identical True/False items to be distinguished by function.Cognitive level reflects the process required in context, while function describes the poll’s purpose; objectively answerable polls receive an answer key, and opinion or logistical items are excluded from accuracy analysis.Source: §3.2 · pp. 4–5
• A language model coded intent and cognitive process from transcript context before assigning Bloom levels and functions, which were treated as descriptive categories.The response distributions were withheld from coding; authors reviewed outputs, manually verified majority-disagreement answer keys, and a second automated coder agreed with rule-based functions at 65% (κ = 0.48) on 40 polls.Source: §3.2 · p. 5

3.3 Survey and Triangulation

• A separate six-question post-program survey was matched to measured records and compared at class and individual levels.Its questions corresponded to measured findings and included open-ended questions about poll purpose and perceived difficulty.Source: §3.3 · p. 5, Figure 1 · p. 6

4.1 Study Context

• The corpus covers an online summer internship orientation program whose focus on motivation and exposure to concepts shapes its lower-order questioning profile.The study distinguishes findings likely specific to this orientation genre from those expected to generalise.Source: §4.1 · p. 6

4.2 Data Streams

• The full corpus contains 604 poll questions, 340,668 responses, and 2,807 distinct learners; most questions are True/False, with time-aligned poll, transcript, and attendance records for each session.Attendance records include per-attendee summaries and join and leave intervals, allowing the researchers to determine who was present during each poll.Source: §4.2 · p. 6

4.3 Alignment

• Poll timestamps were aligned to the transcript timeline, and a window from roughly six minutes before to three minutes after launch supplied lecture context for interpreting function.Source: §4.3 · pp. 6–7, Figure 2 · p. 7

4.4 Data Handling and Ethics

• Participation partly reflects a requirement to answer polls, so the study accounts for that constraint when interpreting participation results.Student data were used with informed consent and aggregate data or salted-hash pseudonyms; no individual student is identified in tables or figures.Source: §4.4 · p. 7

4.5 Supplementary Material

• The online appendix provides detailed dataset summaries, taxonomies, answer-key and correctness coverage, engagement measures, per-session tables, and full survey methods and results.Source: §4.5 · p. 7

5.1 Results

• 89% of polls (539 of 604) had answers objectively locatable in the surrounding lecture, which also revealed Attention/Distortion-Checks that wording alone could mislead readers about.These polls restate a lecture point with a deliberate error, such as changing a number, inserting a negation, or misattributing a claim.Source: §5.1 · p. 8
• 96.8% of the 512 cognitive polls were at Bloom’s Knowledge or Comprehension levels; the corpus also contained seven recurring instructional functions.Only sixteen polls were Application, Analysis, or Evaluation, Synthesis was absent, and the cognitive profile showed no clear shift across 39 days.Source: §5.1 · p. 8, Figure 3 · p. 9, Table 1 · p. 9
• With True/False format held constant, response distributions varied by instructional function (H = 17.8, p = 0.007), while participation rates did not (H = 4.7, p = 0.58).Recall checks and comprehension verification accounted for roughly three-quarters of cognitive polls; attention-checking polls were 10%.Source: §5.1 · pp. 8–9
• About 87% of attendees answered at least one poll, but the most active fifth supplied 74% of responses; class majorities were correct on 88.5% of polls.Consensus and correctness were closely related (r ≈0.80), yet confident misconceptions—large majorities choosing incorrectly—could not be identified from agreement alone, and individual participation was weakly related to accuracy.Source: §5.1 · pp. 9–10, Figure 4 · p. 10
• Survey respondents’ reports aligned with measured participation but not self-assessed correctness; polls they found hardest were not those they answered least accurately.The survey compared 579 respondents with measured records; attention-checking polls accounted for 46% of difficulty responses.Source: §5.1 · pp. 10–11, Figure 5 · p. 11

5.2 Discussion

• The authors interpret polling in this orientation program primarily as sustaining participation rather than assessing mastery, while noting that its corpus-specific patterns may differ elsewhere.The corpus came from one orientation program; survey respondents were self-selected and more engaged, and answering polls partly determined continued participation.Source: §5.2 · p. 12, §5.3 · pp. 12–13

• Contextual reading located an objective answer for 89% of polls and revealed an attention-checking function not recognizable from wording alone.Source: §6 · p. 13
• The polls were primarily lower-order and spanned seven instructional functions; response distributions varied while participation rates remained stable.Source: §6 · p. 13
• Engagement was broad but concentrated, individual participation related only weakly to correctness, and the class majority answered roughly nine out of ten polls correctly.Source: §6 · p. 13
• Comparing agreement with correctness exposed a small set of confident misconceptions, while students’ self-assessed correctness did not fully match measured performance.Source: §6 · p. 13
• Future work could follow the same cohort, vary poll function and timing under controlled conditions, and compare questioning profiles across instructors, subjects, and graded settings.Source: §6 · p. 13
• The contextual approach could also be applied during live sessions to track poll functions and response patterns, including confident misconceptions, as sessions progress.Source: §6 · p. 13