An Exploratory Case Study of LLM-Assisted Refactoring and Gameplay Feature Generation in an Endless Runner Game

1 Introduction

• Empirical evidence remains limited on LLM reliability for game-development tasks requiring integration into existing software rather than isolated benchmark-style code generation [3] [11].This gap persists despite growing use of LLMs for software development [16] and increasing attention to their role in games [3].Source: §1 · p. 1
• Generated game code must interact with gameplay logic, visual systems, event-driven behavior, and other interdependent runtime components.Structural complexity and recurring patterns make games a useful applied setting for LLM-supported programming. Interactive playtesting can reveal behavioral integration problems missed by code-level evaluation [3].Source: §1 · p. 1
• The research questions ask whether GPT-4o can preserve functionality during refactoring and effectively generate gameplay features within an existing architecture.The task classes contrast localized transformations with integration-intensive additions; they are not treated as equivalent in difficulty or scope.Source: §1 · p. 1
• 3 use cases per task class form two structured experiments, with implementations evaluated using software metrics, unit tests, and manual gameplay assessments.The task classes are refactoring and gameplay feature generation.Source: §1 · p. 1
• The paper contributes an exploratory case study of GPT-4o [14], documenting development support and remaining integration challenges within one existing Python/Pygame endless runner.Both refactoring and gameplay feature generation are examined within the same project context.Source: §1 · p. 1

2 Related Work

• LLM effectiveness in programming depends strongly on task complexity, prompting strategy, and evaluation context [9] [10].Research examines code generation, transformation, and interactive assistance [16], but plausible-looking outputs can contain defects, maintainability problems, or inconsistent behavior [5] [9] [10].Source: §2 · p. 2
• Game-related LLM research spans development, AI, narrative, and research applications [3] [22], but empirical evidence for specific engineering tasks within existing games remains limited.Early empirical work suggests support for failure analysis and debugging workflows [11]. Integration requires generated or modified code to interact with existing gameplay mechanics, asset handling, input processing, and state management.Source: §2 · p. 2
• Shirafuji et al. [19] demonstrated simpler Python refactorings, while B. Liu et al. [7] found useful refactoring recommendations alongside unsafe or functionality-changing suggestions.Generated code can exhibit correctness, maintainability, and security problems [9] [10]. Practical studies suggest greater usefulness for exploration, examples, or early-stage assistance than immediately production-ready output [5] [15].Source: §2 · p. 2
• This study compares localized refactoring with integration-intensive gameplay extensions within one existing game and shared validation setup, connecting research strands often studied separately.The exploratory comparison does not treat the task classes as equivalent in difficulty or scope.Source: §2 · p. 2

3 Methodology

• Three refactoring tasks targeted central state management, redundant asset loading, and separate movement-control implementations.R1 replaced nested state/event decisions with modular event handling, while R2 sought structured, configurable asset loading instead of repetitive statements. R3 sought generic control logic unifying movement, jumping, and sliding.Source: §3 · p. 3
• Three gameplay-feature tasks targeted pixel-perfect collision detection, a temporary shrink power-up, and moving cars usable as platforms.F1 replaced bounding-box collision logic with visible-pixel masks; F2 required shrinking to preserve interactions with jumping, sliding, and weapons. F3 required specialized car collision handling and stable movement and dismounting within the existing game's collision and player-state systems.Source: §3 · pp. 2–3
• Six tasks in one Python/Pygame endless runner supported an exploratory, qualitative comparison rather than a controlled benchmark of refactoring versus gameplay-feature generation.Tasks varied in technical scope and integration depth without being calibrated for equivalent difficulty. Findings concern the evaluated GPT-4o snapshot and development setting, not newer models, specialized code models, or agentic tools.Source: §3 · pp. 2–3
• 31,947 tokens comprising the complete codebase were supplied without truncation, while each task used a separate chat to prevent context carryover.The publicly available project used dedicated task branches originating from the same main-branch version, enabling before-and-after inspection. An initial codebase-summary check assessed GPT-4o's understanding of project structure and dependencies.Source: §3 · p. 3
• Task-dependent prompting used self-consistency, few-shot, and test-guided strategies [18], with each iteration comprising a prompt, generated implementation, and validation.Follow-up prompts addressed unmet structural or functional requirements. Gameplay-feature evaluation additionally used feature-specific tests and iterative revisions when implementations failed expected functionality.Source: §3 · pp. 3–4
• Evaluation combined software metrics, unit tests, and manual gameplay assessments, with success requiring specified functionality, passing relevant tests, and correct gameplay behavior.SonarQube-based analysis supplied lines of code, cyclomatic complexity, cognitive complexity, and code smells; a consistently applied custom Python script calculated maintainability index. Failure included stalled refinement, increasing specification deviations, or gameplay failure despite passing code-level tests.Source: §3 · pp. 3–4

• Iteration counts represent task-specific refinement cycles within a single run, not independent model runs; refactoring revisions addressed missing functionality, compatibility issues, or test-detected regressions.Failed gameplay features instead exhibited persistent integration problems even after code-level tests passed, revealing mismatches between generated code and runtime gameplay behavior.Source: §4 · p. 4
• 3, 7, and 5 iterations produced functionally stable refactoring outcomes, but structural metric changes ranged from clear improvements to marginal or unfavorable changes.The evaluated metrics included lines of code, cyclomatic complexity, cognitive complexity, code smells, and maintainability index.Source: §4 · p. 4
• Redundant asset-loading refactoring delivered the strongest structural improvement: the affected file became substantially shorter and its maintainability index increased, although cyclomatic and cognitive complexity increased.Useful refactoring support therefore did not imply equally pronounced quality gains across tasks.Source: §4 · p. 4
• 4 iterations successfully integrated pixel-perfect collision detection with moderate code changes, whereas failed gameplay features involved broader cross-file changes or larger code-size and complexity increases.For multi-file tasks, LOC, CC, CGC, and CS were aggregated across affected files; MI was summarized qualitatively because it is file-specific.Source: §4 · p. 4
• 8 refinement attempts failed to complete the shrink power-up; the car platform mechanic failed gameplay testing despite passing all unit tests after 6 iterations.Character scaling disrupted jumping, sliding, and weapon behavior. The car mechanic left the character dying or moving incorrectly when landing on and moving with cars.Source: §4 · pp. 4–5
• 3 refactoring tasks succeeded functionally, compared with only one correctly integrated gameplay feature in this case study.This observed difference is exploratory rather than generalizable.Source: §4 · pp. 4–5, Table 2 · p. 5

5 Discussion

• GPT-4o handled localized refactoring more reliably than gameplay-feature tasks requiring integration across interdependent systems in this case study.The selected refactoring tasks produced functionally stable outcomes and, in several cases, structural improvements.Source: §5 · p. 5
• LLMs may support bounded code improvement, exploratory restructuring, and alternative implementations, but complex gameplay features require careful integration, testing, and developer supervision.Successful use often required prompt refinement, selective code integration, and corrective feedback; generated gameplay functionality should not be treated as immediately production-ready.Source: §5 · pp. 5–6
• Passing unit tests did not ensure correct gameplay: the car platform mechanic passed tests but exhibited incorrect runtime behavior during manual assessment.Gameplay correctness can depend on timing, movement, collisions, animation state, and other interactions that unit tests alone may not fully capture.Source: §5 · p. 6
• The single-project, author-developed design and limited model and task coverage restrict generalizability and may introduce selection or design bias.Only GPT-4o was evaluated, across three refactoring tasks and three gameplay-feature tasks, precluding broader statistical conclusions or assumed transfer to other models.Source: §5 · p. 6
• Unrepeated runs and unmatched task difficulty prevent assessing run-to-run variation or treating task-class differences as a controlled benchmark comparison.Evaluation used software metrics, unit tests, and manual gameplay assessments, but did not fully capture long-term code quality, maintainability, or developer effort.Source: §5 · p. 6
• Responsible use requires balancing iterative prompting’s computational costs against practical benefits while retaining expert review, transparent reporting, and systematic testing.Plausible generated code may contain subtle defects, reinforcing the need for both code-level and gameplay-level validation.Source: §5 · p. 6

6 Conclusion & Future Work

• GPT-4o handled selected refactoring tasks more reliably than gameplay feature generation tasks, although the exploratory single-case design limits interpretation.The observed pattern favored localized code transformations over integration-intensive gameplay extensions.Source: §6 · p. 6
• Generated code should be evaluated for functional behavior within the running game, not only for structural properties.Source: §6 · p. 6
• Future studies should test additional LLMs, larger and more diverse task sets, repeated independent model runs, and different game projects.Task difficulty should be calibrated before comparing development-task classes.Source: §6 · pp. 6–7
• Future work should examine whether emerging agentic coding systems can reduce human guidance in iterative development workflows.Source: §6 · p. 7