PaperlayerView exact revision on arXiv

Source-linked AI summary

WIP: DBWorkout: A Gamified SQL Practice Platform to Support Formative Learning in Database Courses

Sehrish Basir Nizamani, Deepika Devaraj, Tien Nguyen, Khyati Goyal, Saad Nizamani, Sally Hamouda, Jaren Goldberg

arXiv:2610.01174v1cs.CYcs.DBcs.HC

TL;DR

Undergraduate SQL practice is constrained by limited opportunities for iteration and immediate, meaningful feedback. DBWorkout combines sandbox practice, result-based feedback, session-bounded gamification, and instructor-reviewed LLM authoring; students reported high perceived value and enjoyment, but feedback sufficiency remained a gap.

  • Problem

    Undergraduates have limited opportunities for iterative SQL practice and immediate, meaningful feedback, making correct query formulation challenging.

    Source: §I · p. 1

  • Method

    DBWorkout combines isolated sandbox execution, multidimensional result-based feedback, session-bounded gamification, and instructor-reviewed LLM-assisted content generation.

    Source: §I · p. 1, A. Design Goals · p. 2

  • Results

    90–92% agreement on perceived-value items and 87% enjoyment accompanied only 42% finding feedback adequate; 40% of open-ended responses raised feedback-depth concerns.

    Source: A. Quantitative Findings · p. 4, B. Qualitative Findings · p. 4

  • Takeaways & Limitations

    The deployment indicates early pedagogical potential, while identifying feedback actionability as the clearest improvement target.

    Source: §X · p. 5

    The single-institution study measured perceived learning value rather than objective gains, so effects on SQL proficiency remain unestablished.

    Source: §VIII · p. 5

Abstract

from arXiv · show

This research WIP paper presents DBWorkout, a web-based platform that supports formative SQL learning through sandbox-based execution, automated result-based feedback, and session-based gamification. Learning Structured Query Language (SQL) remains challenging for undergraduate students due to limited opportunities for interactive practice and immediate feedback. Students iteratively practice SQL on live database instances while receiving multi-dimensional feedback on query correctness, including row values, column structure, and ordering. To reduce instructor workload, DBWorkout incorporates large language model (LLM)-assisted tools for schema and task generation within a human-in-the-loop workflow. A pilot study with teaching assistants and a classroom deployment involving 170 undergraduate students across two in-class sessions show strong perceived learning value (90% agreement) and engagement (87% enjoyment), alongside low reported pressure (22%). However, only 42% of students found the automated feedback sufficiently actionable, a finding independently corroborated by 40% of open-ended responses raising feedback quality concerns, providing cross-method triangulation of this gap. These findings demonstrate the technical feasibility and early pedagogical potential of DBWorkout while identifying directions for enhancing feedback quality and supporting sustained SQL learning.

I. INTRODUCTION

DBWorkout addresses limited opportunities for iterative SQL practice and meaningful immediate feedback with a gamified sandbox platform. The paper examines usability, perceived learning value, engagement, and feedback sufficiency during classroom practice.

I. INTRODUCTION

  • Limited iterative practice and immediate, meaningful feedback make SQL learning challenging for undergraduates.Students may struggle to translate conceptual understanding of JOIN and GROUP BY into correct queries.Source: §I · p. 1
  • DBWorkout combines sandbox-based iterative SQL practice, automated result-based feedback, session leaderboards, and LLM-assisted instructor content generation.The LLM tools assist with generating database schemas, practice questions, and reference SQL solutions.Source: §I · p. 1
  • The study asks how students perceive usability and learning value, whether leaderboards engage without anxiety, and what feedback improvements students identify.An internal teaching-assistant pilot preceded deployment to approximately 170 students across two in-class sessions, followed by a post-session survey.Source: §I · p. 1

II. RELATED WORK AND INSTRUCTIONAL CHALLENGES

Prior SQL tools address automated grading or tutoring, but iterative practice, immediate feedback, adaptable authoring, and sustained engagement remain challenges. DBWorkout combines sandbox execution, feedback, gamification, and instructor-supported authoring in response to these gaps.

II. RELATED WORK AND INSTRUCTIONAL CHALLENGES

  • Browser SQL platforms offer automated grading, while constraint-based tutoring can require substantial upfront effort to encode a domain.Wen et al. describe a browser-based platform; Mitrovic and Ohlsson report the authoring burden of constraint-based tutors.Source: §II · p. 1
  • Gamification effects vary by context; DBWorkout uses a transparent, session-resetting absolute leaderboard to avoid semester-long ranking history.Prior work reports less consistent gains for large groups and identifies longitudinal and novelty effects as underexamined.Source: §II · p. 2
  • SQL learners face schema interpretation, ambiguous prompts, and multiple valid solutions, with JOIN and GROUP BY problems linked to elevated errors.Alkhabaz et al. analyzed over 357,000 submissions from 462 students; DBWorkout responds with operation-specific feedback and task tagging.Source: §II · p. 2
  • DBWorkout combines result-based feedback with human-in-the-loop LLM content creation to address limits in guidance and instructor adaptability.A review found 96% of automated programming feedback tools identify mistakes, while fewer than 20% provide explicit task-processing-step guidance; prior work also notes risks of unguided LLM use.Source: §II · p. 2

III. SYSTEM DESIGN

DBWorkout combines isolated live-database practice, multidimensional result comparison, and session-based competition. Its design targets authentic practice, iterative refinement, instructor workload, and participation without persistent ranking pressure.

A. Design Goals

  • Each SQL submission runs in a cloned, restricted PostgreSQL schema that is dropped afterward, providing isolated sandbox execution within one database instance.The three-tier system connects a React frontend and Flask API to PostgreSQL; sandbox creation takes under a second.Source: A. Design Goals · p. 2

B. Automated Feedback Mechanism

  • Automated feedback compares reference and student results across seven weighted dimensions, with descriptive messages to support iterative refinement.Checks include execution, statement count, column names and types, row count and values, and ordering when applicable; DML and DDL tasks receive specialized comparisons.Source: B. Automated Feedback Mechanism · p. 2

C. Gamification and Leaderboard Integration

  • The live leaderboard ranks students by completed tasks and completion time, with session-only standings and no persistent points, badges, or streaks.The design resets rankings each session so students begin on equal footing, addressing concerns about discouragement from accumulated low standing.Source: C. Gamification and Leaderboard… · pp. 2–3

D. LLM-Supported Instructor Authoring Tools

DBWorkout uses instructor-authorized LLM workflows to generate practice tasks and database schemas, with instructor review and validation before student use.

D. LLM-Supported Instructor Authoring Tools

IV. PILOT STUDY

A teaching-assistant pilot identified and resolved three defects before DBWorkout was deployed in two in-class sessions with 170 undergraduates. The pilot also exercised instructor and student workflows, including LLM-assisted task curation.

IV. PILOT STUDY

  • Seven teaching assistants tested course management and student tasks under a scripted protocol, then submitted defect and usability reports.Two TAs acted as instructors and five as students.Source: §IV · p. 3
  • Three defects—reference-solution validation, feedback persistence on navigation, and enrollment timeouts—were resolved before the sandbox, leaderboard, and LLM tools performed as intended.Source: §IV · p. 3

A. Study Context

  • DBWorkout was deployed in two in-class sessions totaling 170 students, with instructor-authored tasks optionally assisted by the LLM authoring tool and faculty-reviewed before use.The morning session had 104 students and the evening session had 66.Source: A. Study Context · p. 3

B. Survey Instruments

The evaluation combined validated survey subscales with system-specific questions and open-ended responses. Researchers analyzed substantive comments through reflexive thematic analysis with independent coding review.

B. Survey Instruments

  • The post-activity survey used UEQ subscales and IMI measures of enjoyment, competence, effort, value, and pressure/tension, alongside system-specific items on a 1–5 Likert scale.An open-ended item invited students to describe their experience and suggest improvements.Source: B. Survey Instruments · p. 3
  • Table I summarizes descriptive statistics for survey items from 170 students.Source: Table I · p. 4

C. Qualitative Analysis

  • Of 170 open-ended responses, 98 substantive responses were analyzed using reflexive thematic analysis, with a second researcher reviewing and coding data and disagreements resolved through discussion.Source: C. Qualitative Analysis · p. 4

VI. PRELIMINARY FINDINGS

Students reported strong perceived value and enjoyment with low pressure, but feedback sufficiency was the weakest result. Open-ended responses also surfaced feedback, interface, task-clarity, attempt-limit, and competition concerns.

A. Quantitative Findings

  • Usability averaged M = 4.16, though “Easy over Complicated” was M = 3.69 (60% positive), and novelty scores were M = 3.71–3.78.Source: A. Quantitative Findings · p. 4
  • 90–92% agreed on perceived-value items (M = 4.21–4.32), 87% reported enjoyment, and 22% reported pressure (M = 2.75).83% said they would use DBWorkout again.Source: A. Quantitative Findings · p. 4
  • Only 42% found automated feedback adequate (M = 3.18, SD = 1.09); frustration was 35% (M = 2.98), while schema quality was 76% (M = 3.85).Source: A. Quantitative Findings · p. 4

B. Qualitative Findings

  • Among six themes, 40% raised feedback depth and specificity; others concerned positive reception (17%), UX (16%), task clarity (12%), attempt limits (6%), and mixed leaderboard reactions (9%).UX concerns included missing auto-save, schema navigation, and occasional lag.Source: B. Qualitative Findings · p. 4

VII. DISCUSSION

The discussion treats perceived value as initial support for DBWorkout’s formative design while identifying feedback specificity as the clearest next step. It also motivates investigating alternatives to competitive engagement mechanics.

A. RQ1: Usability and Perceived Learning Value

  • The feedback gap aligned across a structured survey item and unprompted comments, and the authors identify more specific output or directional hints as the clearest next step.Examples include showing query output or hints for missing JOIN conditions; the discussion relates the gap to Keuning et al..Source: A. RQ1: Usability and Perceived… · p. 4

B. RQ2: Gamification and Engagement

  • Although the authors report high enjoyment and low pressure with the session-bounded leaderboard, some students preferred collaborative or non-ranked practice, motivating study of team-based challenges.The discussion connects its interpretation to Li et al..Source: B. RQ2: Gamification and Engagement · p. 4

C. RQ3: Feedback Actionability and Design Implications

The deployment’s most significant finding was a feedback-actionability gap, alongside stable performance with 170 concurrent students. Findings are bounded by a single course and first exposure, and learning gains were perceived rather than objectively assessed.

C. RQ3: Feedback Actionability and Design Implications

  • DBWorkout identified an actionability gap: tools often identify mistakes but fall short of next-step guidance, as described by Keuning et al..The paper identifies this as the deployment’s most significant finding.Source: C. RQ3: Feedback Actionability and… · p. 5

D. Scalability

  • The sandbox architecture handled 170 concurrent students with no infrastructure failures, demonstrating scalability under authentic classroom conditions.Source: D. Scalability · p. 5

VIII. LIMITATIONS

  • Evidence is bounded to one course and first exposure, measures perceived rather than objective learning gains, and includes researcher-coded qualitative analysis despite second-coder review.The authors note that findings may not generalize and engagement may change with sustained use.Source: §VIII · p. 5

IX. FUTURE WORK

DBWorkout’s deployment exposed feedback actionability as its primary gap, while its session-bounded leaderboard balanced motivational benefits against persistent-ranking risks. Future work targets feedback improvements and more inclusive competition, with controlled evaluation planned.

IX. FUTURE WORK

  • Planned work includes cell-level diffs, directional error hints, jargon tooltips, and score-based closeness messages, evaluated with pre- and post-assessments and feedback measures.The team will also investigate pair competition and disability-aware time normalization.Source: §IX · p. 5

X. CONCLUSION

  • 42% of students found automated feedback sufficient, while independent qualitative evidence corroborated feedback actionability as the primary gap.Source: §X · p. 5
  • The session-bounded leaderboard balanced motivational benefits against persistent-ranking demotivation risks, alongside strong perceived learning value, high engagement, and low performance anxiety.Source: §X · p. 5
  • DBWorkout is presented as a step toward integrating authentic practice, scalable feedback, and principled engagement mechanics in SQL education.Source: §X · p. 5
Loading 2610.01174v1…