Source-linked AI summary
WIP: LLM Odyssey: A Game-Based Platform for Teaching LLM Engineering Concepts
Priyamvada Tripathi
TL;DR
LLM engineering education lacks comprehensive, structured resources connecting foundational concepts to production practice. This paper presents LLM Odyssey, an open-source browser-based platform of 13 games organized into theory-informed learning tiers and mechanics. Initial review established technical feasibility and face validity, while identifying adaptive difficulty as a priority; learning effectiveness remains unestablished pending formal evaluation.
Problem
Existing instructional resources lack structured, progressive coverage of LLM engineering, while available tools address individual concepts without pedagogical scaffolding or learning pathways.
Method
The paper presents LLM Odyssey, an open-source platform of 13 interactive games using tiered progression and five learning-science-informed mechanics.
Results
Initial expert review provided evidence of technical feasibility and face validity, while reviewers identified adaptive difficulty as the primary enhancement priority.
Takeaways & Limitations
The publicly available platform and documented N=50 mixed methods protocol support future studies of knowledge gains, engagement, and pedagogical elements.
Takeaways & Limitations
The single-institution sample without a comparison group restricts causal inference about effectiveness.
Abstract
from arXiv · showhide
This work-in-progress (WIP) innovative practice category paper presents LLM Odyssey, an open source, browser-based serious gaming platform comprising 13 interactive games for teaching Large Language Model (LLM) engineering concepts. Topics such as tokenization, transformer architecture, prompt engineering, retrieval augmented generation (RAG), and production deployment are underrepresented in computer science curricula. Existing interactive tools address individual concepts but lack pedagogical scaffolding or structured learning pathways. LLM Odyssey addresses this gap through three learning tiers aligned with Bloom's revised taxonomy: Cognitive Core (7 foundational games), Systems Forge (5 production engineering games), and Foundry Arena (capstone challenges). Each game incorporates five pedagogical strategies drawn from the literature: immediate formative feedback, scaffolded hints grounded in the Zone of Proximal Development, progressive difficulty informed by flow theory, worked examples to manage cognitive load, and authentic scenarios drawn from production practice. The platform was deployed in Winter 2026 semester at a Canadian college for an initial review. Feedback confirmed functional requirements and identified adaptive difficulty as a priority for future development. A formal mixed methods evaluation protocol (N=50) has been designed, comprising pre and post knowledge tests, validated surveys, engagement analytics, and interviews, and is documented here to enable future evaluation studies with the publicly available platform.
I. INTRODUCTION
LLM engineering curricula often leave graduates without practical experience in prompt engineering, RAG, fine-tuning, and production deployment. Existing tools address isolated concepts but generally lack comprehensive coverage, scaffolding, and structured progression.
- Industry identifies prompt engineering, retrieval augmented generation, fine tuning, and production deployment as necessary competencies that CS programs do not consistently develop.
- Students may complete degrees with theoretical machine learning foundations but limited hands on exposure to operational challenges specific to LLM systems.
- Prompt engineering involves iterative experimentation, while LLM systems also present context window, latency, cost, safety, and probabilistic-output challenges.
- Existing interactive tools such as OpenAI Playground and Hugging Face Spaces address individual concepts but lack progressive difficulty, pedagogical scaffolding, and structured learning pathways.
- Game-based learning research reports generally positive, though context-dependent, engagement effects, while immediate feedback and authentic contexts support learning and knowledge transfer.
II. BACKGROUND AND RELATED WORK
LLM engineering poses instructional challenges because key mechanisms are difficult to observe and production constraints are often absent from coursework. LLM Odyssey is positioned as a structured, progressive game-based response grounded in established learning theories.
- Attention mechanisms, embedding spaces, and gradient flows are not directly observable, while parameter interactions can be nonlinear and counterintuitive.
- Production concerns including API cost, latency budgets, and failure modes are largely absent from academic coursework.
- Game-based learning is supported by generally positive but context-dependent engagement evidence, immediate formative feedback, authentic contexts, and interactive creation and reflection.
- The platform design applies constructivism, scaffolding, mastery learning, and Self Determination Theory through experimentation, graduated support, unlimited retries, progress visualization, and tier unlocks.
- No existing platform provides structured, progressive coverage from foundational LLM engineering concepts through production practice, motivating LLM Odyssey.
III. PLATFORM DESIGN AND ARCHITECTURE
LLM Odyssey organizes 13 interactive games into three tiers that progress from foundational understanding to applied synthesis and align with Bloom’s revised taxonomy.
- LLM Odyssey comprises 13 interactive games organized across three tiers progressing from foundational understanding to applied synthesis.
- Tier Progression: Tier I, Cognitive Core, contains Games 1 to 7 and targets Remember, Understand, and Apply through parameter manipulation and real-time observation.
- Tier Progression: Tier II, Systems Forge, contains Games 8 to 12 and targets Apply, Analyze, and Evaluate through scenario-based decisions under realistic production constraints.
- Tier Progression: The tier structure establishes foundational competency before production concepts are introduced.
A. Game Mechanics and Pedagogical Design
Each game combines five mechanics—feedback, hints, progressive difficulty, worked examples, and authentic contexts—to support practice, scaffolding, cognitive-load management, and transfer in LLM engineering scenarios.
- Each game implements five pedagogical strategies grounded in learning science and expressed through specific game mechanics.
- Immediate Formative Feedback: Immediate formative feedback updates token counts, attention patterns, loss curves, and cost metrics after each student action.
- Scaffolded Hints: Scaffolded hints progress from conceptual cues to partial solutions, with a one point reduction per hint encouraging independent attempts first.
- Progressive Difficulty: Progressive difficulty uses five rounds with increasing complexity, reflecting flow theory’s proposed zone between boredom and anxiety.
- Worked Examples: Worked examples provide annotated code, visual diagrams, and step-by-step explanations intended to reduce extraneous cognitive load for novices.
- Authentic Context: Authentic production scenarios connect game challenges to real-world tasks, including reducing a $50,000 monthly inference cost by 40% while maintaining P95 latency below 200ms.
B. Sample Game: Token Forge
Token Forge teaches tokenization through interactive choices, immediate feedback, and progressively harder production-oriented text challenges. Its design connects algorithmic comparisons to practical cost implications.
- Token Forge uses interactive tokenizer selection to introduce foundational concepts through mechanics, immediate feedback, and progressive challenges.Students compare tokenization approaches across increasingly difficult tasks.
- Students choose among BPE, WordPiece, SentencePiece, and Unigram for multilingual content, Python code, and legal documents.
- Immediate outputs include color-coded segmentation, token count, vocabulary efficiency, and estimated API cost after each tokenizer selection.
- Five rounds increase difficulty from simple English text to multilingual legal documents drawn from realistic production scenarios.
- An integrated concept guide links tokenization algorithm theory with practical cost implications.
IV. IMPLEMENTATION AND EVALUATION
LLM Odyssey uses a client-side architecture for responsive browser gameplay while a serverless backend stores anonymous analytics and persistence data. The platform records engagement, performance, assessment, and survey measures for future evaluation.
- Client-side game logic eliminates server round trips during gameplay, maintaining interaction response times below 16ms in testing.The frontend uses React 18.2 and Tailwind CSS.
- A serverless backend handles persistence and analytics using anonymous session identifiers, with no personally identifiable information collected.
- The platform logs time on task, completion rates, game sequences, session duration, scores, accuracy trends, hints, errors, retries, and pre/post knowledge tests.
- Validated Likert-scale survey data captures perceived learning and difficulty alongside interaction and assessment records.
A. Design Challenges
Development involved balancing educational fidelity with browser feasibility, calibrating difficulty for varied expertise, and simulating production constraints without live infrastructure.
- Precomputed token breakdowns preserve educational accuracy while maintaining browser performance for computationally intensive BPE processing.The trade-off is presented to students as an example of a real engineering constraint.
- Expert review and pilot observation indicated that a uniform difficulty curve insufficiently serves novice and experienced students.
- Adaptive preassessment routing was identified as the priority for a subsequent iteration after difficulty adjustments based on observation.
- Parameterized scenarios use explicit numerical constraints to simulate production engineering decisions within browser-based execution limits.An example specifies 10,000 requests per day, a 200ms P95 latency budget, and an $8,000 monthly cost ceiling.
B. Preliminary Findings and Evaluation Protocol
An initial two-faculty review supported the platform’s usability and technical feasibility but was not evidence of learning effectiveness. The paper therefore documents a future mixed-methods protocol to evaluate knowledge, engagement, pedagogy, and expertise-related trajectories.
- N=2 faculty reviewers identified game-based learning and immediate quantitative feedback as strengths, while viewing Systems Forge scenarios as industry-relevant.
- The preliminary review was not a formal evaluation and does not constitute evidence of learning effectiveness.
- Adaptive difficulty was the primary enhancement priority because expertise variation within cohorts may leave some students underchallenged and others blocked.
- Technical targets were met over four weeks, with page load below 100ms and no availability incidents.The findings provide initial evidence of technical feasibility and face validity, while learning outcomes require a controlled study.
- A single-institution sample without a comparison group restricts causal inference in the planned evaluation.
V. CONCLUSION
LLM Odyssey is an open-source, browser-based platform of 13 interactive games designed to teach LLM engineering through learning-science-grounded mechanics. The paper establishes technical feasibility and proposes an N=50 evaluation protocol, while noting that learning effectiveness remains unestablished.
- LLM Odyssey comprises 13 interactive games for LLM engineering education.The platform addresses topics including tokenization, transformer architectures, prompt engineering, RAG, and production deployment.
- The platform grounds its game mechanics in constructivism, scaffolding theory, mastery learning, and cognitive load theory.
- Initial expert review supports technical feasibility and face validity, but learning effectiveness has not yet been established.
- N=50 evaluation protocol measures pre-to-post knowledge gains, engagement patterns, and associations between pedagogical elements and learning outcomes.The protocol is designed to collect initial data for future evaluation studies.
- Causal effectiveness claims require studies across multiple institutions and cohorts.The platform is publicly available, and the paper presents tiered progression mapped to Bloom’s revised taxonomy as a transferable design pattern.