Synergizing Code Coverage and Gameplay Intent: Coverage-Aware Game Playtesting with LLM-Guided Reinforcement Learning

Motivation

• Frequent Games as a Service updates increase defect risk and demand automated testing that responds efficiently to rapid iteration.Updates support retention and monetization but can introduce unforeseen defects into stable codebases.Source: §1 · p. 2

Existing approaches

• Code-centric testing prioritizes structural coverage, whereas player-centric testing validates high-level gameplay behavior such as quest completion.These paradigms emphasize different aspects of game correctness.Source: §1 · p. 2

Problem

• Single-dimensional testing can execute modified code without validating gameplay intent, or validate quest flows while missing newly added code.Game updates combine macro-level functional changes with micro-level structural changes.Source: §1 · pp. 2–3

SMART

• SMART uses LLMs to decompose code updates into sequential functional subgoals aligned with structural anchors.This mapping supplies context-aware guidance for update testing.Source: §1 · p. 3
• SMART’s hybrid reward guides reinforcement-learning agents toward goal completion while encouraging exploration of edge-case interactions in modified branches.The framework therefore combines functional validation with structural verification.Source: §1 · p. 3

2.1 Abstract Syntax Tree (AST)

• An AST represents source-code syntax as hierarchical nodes and relationships, omitting surface details such as formatting and punctuation.Its structure captures syntactic and semantic content rather than the concrete textual representation.Source: §2.1 · pp. 3–4
• AST operators and variables are composed inductively into typed tree structures representing constructs such as expressions and statements.The formal construction uses sorts, operators, arities, and argument ASTs.Source: §2.1 · pp. 3–4
• Hierarchical AST relationships represent control-flow constructs such as if-else statements and loops, making ASTs useful intermediate representations for software analysis.Branches and loop components appear as related child nodes.Source: §2.1 · p. 4

2.2 Reinforcement Learning

• Reinforcement learning trains an agent to choose actions through environment interaction while maximizing cumulative reward.This formulation suits sequential decision-making problems such as video games.Source: §2.2 · p. 4
• PPO uses a clipped surrogate objective to constrain policy-probability changes between updates and improve training stability.The clipping range is controlled by ϵ, typically 0.2.Source: §2.2 · p. 5

3.1 Overview

• SMART’s five-stage pipeline parses AST differences, generates semantic subgoals and rewards, maps structural anchors, and guides testing with an adaptive hybrid reward.The semantic component targets functional correctness, while the structural component targets comprehensive exploration.Source: §3.1 · pp. 5–6

3.2 Motivating Example

• The Onion Pizza example decomposes a quest into ordered actions from preparing ingredients through baking, plating, topping, and delivery.The update also adds code for item states, ingredient combinations, baking, toppings, and delivery validation.Source: §3.2 · p. 6

3.3 AST Difference Parser

• The AST Difference Parser identifies added or modified lines and altered control-flow branches as structural anchors for testing.Modified branches include the true and false paths of changed if-else conditions.Source: §3.3 · pp. 6–7

3.4 Stage 2: Subgoal Generation

• The LLM Subgoal Generator analyzes code dependencies and quest logic to produce an ordered sequence S of verifiable natural-language gameplay steps.The sequence decomposes complex quests into intermediate objectives such as obtaining ingredients, assembling, and baking.Source: §3.4 · p. 7

3.5 Stage 3: Semantic Reward Generation

• Semantic Reward Generation converts each subgoal into a binary state predicate, producing ordered checkpoints that must be achieved sequentially.Rules use observable variables and can reward intermediate progress as well as final state transitions.Source: §3.5 · pp. 7–8

3.6 Stage 4: Structural Anchor Mapping

• Structural Anchor Mapping assigns each modified line or branch to one or more relevant subgoal stages using static reachability followed by LLM semantic filtering.This prevents agents from repeatedly exploring code that is irrelevant to the current gameplay stage.Source: §3.6 · pp. 8–9

3.7 Stage 5: Adaptive Hybrid Reward Function

• The adaptive hybrid reward trains agents to complete the subgoal sequence while discovering modified anchors across episodes through instrumentation and persistent coverage tracking.An in-episode subgoal tracker and global coverage map balance semantic progress with novel structural discovery.Source: §3.7 · pp. 9–11
• Structural rewards encourage productive deviations from a learned happy path, while semantic rewards provide a stable objective for functional competence.The design faces learning instability because structural rewards are inherently unstable, so reward magnitudes must be balanced.Source: §3.7 · pp. 9–10

4.1 Experiment Game Environment

• SMART is evaluated in Overcooked and Minecraft, representing discrete recipe-based tasks and a more complex open-world environment.Overcooked uses an open-source grid-world toolkit, while Minecraft provides parameterized high-level commands over a 3D sandbox state.Source: §4.1 · pp. 11–12, §4.2.3 · p. 13

4.2.1 Comparison Baseline Methods

• The evaluation compares SMART with random exploration, PPO using only quest completion, PPO+ICM curiosity-driven exploration, and SMART ablations.The ablations isolate semantic rewards, structural rewards, global hybrid rewards, and the full context-aware hybrid design.Source: §4.2.1 · pp. 12–13

4.2.2 Settings in Overcooked

• Overcooked contains 21 update-derived tasks from dough and oven changes, with approximately 200 changed lines and around 100 changed branches.Each task is trained for up to 100,000 interactions with a 300-step episode horizon and shaped semantic and terminal rewards.Source: §4.2.2 · p. 13

4.2.3 Settings in Minecraft

• Minecraft contains 20 tasks derived from a gold update, with approximately 180 changed lines and around 100 changed branches, using up to 1,000,000 interactions per task.The environment uses the same reward levels and reinforcement-learning library while retaining the policy and hyperparameters used in Overcooked.Source: §4.2.3 · pp. 13–14

4.3 Experiment Results and Discussion

• SMART-Full reaches 0.9 line coverage and 0.95 branch coverage, while PPO and PPO+ICM achieve only 0.46–0.55 line coverage despite approximately 0.7 task success.The comparison indicates that task-oriented agents tend to follow shortest paths and miss update-related edge cases, whereas structural rewards direct exploration into modified code.Source: §4.3 · p. 14
• SMART-Semantic achieves about 97% success and 52% line coverage versus 93% line coverage for the full method in Overcooked, revealing a high-success but low-coverage happy path.SMART-Structural performs near random levels, while SMART-Global-Hybrid has low success and coverage, supporting the need for semantic subgoals and context-aware anchor mapping.Source: §4.3 · p. 15
• SMART’s cumulative anchor coverage grows fastest as task complexity increases, while baselines plateau early; however, some defensive and exception-handling branches remain uncovered.Uncovered branches reflect both logically avoided fallback behavior and a trade-off between exhaustive exploration and training efficiency.Source: §4.3 · pp. 16–17

4.4 Limitations

• SMART’s effectiveness is bounded by LLM semantic accuracy, AST visibility, update scope, reward-weight tuning, and its intended role as incremental rather than exhaustive testing.The paper focuses on core gameplay logic and element-addition updates, while recommending complementary techniques for non-AST assets, dynamic behavior, and residual edge cases.Source: §4.4 · pp. 18–19

5.1 Structural Game Testing

• Traditional scripts and recorded traces reduce repetitive labor but require substantial manual maintenance when game interfaces or logic change.Model-based and behavior-tree methods add flexibility, yet still depend on upfront design and modeling effort.Source: §5.1 · p. 20, §5.2 · p. 20
• Structural regression methods select or execute tests around changed code, but cannot determine whether modified interactions achieve their intended gameplay goals.GameRTS uses static analysis and code differences to select relevant tests, while the Overcooked example shows why structural execution alone is insufficient.Source: §5.1 · p. 20

5.2 Intent-Driven Game Testing

• Intent-driven agents support exploratory gameplay testing, but their high-level rewards usually lack direct connections to underlying structural changes.RL, curiosity, evolutionary, and persona-based approaches improve exploration or behavioral realism without ensuring coverage of specific modified code.Source: §5.2 · pp. 20–21
• Intent-driven RL agents may converge on a “Happy Path,” avoiding erroneous or suboptimal interactions that exercise defensive and exception-handling branches.Consequently, task completion can coexist with insufficient coverage of alternative states introduced by structural updates.Source: §5.2 · p. 21

5.3 Position of This Paper

• SMART targets the gap between structural verification and functional validation by connecting source-code differences to gameplay intent.It uses AST diffs and LLM semantic interpretation to ground testing in both modified code and higher-level goals.Source: §5.3 · pp. 21–22
• SMART integrates LLM-inferred semantic intent with structural coverage rewards so agents pursue gameplay goals while exploring modified code paths.Functional intent is decomposed into ordered semantic subgoals aligned with structural anchors.Source: §5.3 · p. 22

6 Conclusion and Future Work

• SMART combines code coverage with gameplay intent by using LLM-interpreted AST differences to guide RL agents through semantic subgoals and modified code.Its context-aware hybrid reward system targets both functional requirements and structural exploration.Source: §6 · p. 22
• SMART outperforms traditional RL and curiosity-driven baselines on Overcooked and Minecraft while maintaining high task success and modified-branch coverage.The supplied conclusion states the overall comparison and scope but does not provide the abstract’s exact percentages.Source: §6 · p. 22
• Future work will add closed-loop LLM reward refinement, multimodal analysis for non-code updates, and symbolic-execution or fuzz-testing hybrids.These directions address hallucinated subgoals, asset and UI changes absent from ASTs, and deeper defensive-logic exploration.Source: §6 · p. 22

• SMART’s subgoal-generation prompt extracts new or expanded gameplay tasks from AST differences, decomposes them into ordered atomic subgoals, and annotates each with structural code anchors.Anchors use file paths and unified-diff line numbers, prioritizing added lines and core gameplay logic.Source: Appendix · pp. 22–23
• The semantic-reward prompt generates JSON reward definitions from task subgoals and game information, requiring full-process coverage, fine-grained progress rewards, one-time triggers, and a 200-point main reward.Item rewards require simultaneous inventory conditions, subtask rewards activate upon completion, and each step incurs a 0.1 penalty.Source: Appendix · pp. 24–25
• The structural-anchor prompt maps Minecraft subgoals to verified modified-source locations using project-inspection tools, forbidding invented paths or spans and preferring new-version line numbers.Subgoals must be copied verbatim, and locations may be empty only when no relevant code change exists.Source: Appendix · pp. 25–26
• The pizza_onion task specifies subgoals for picking up a clean plate, preparing dough as a pizza base, chopping a tomato, and placing tomato and cheese onto dough.Its locations data links to overcooked/overcookedPlus/module/item_manager.py, span 55–118.Source: Appendix · p. 26

Example Subgoal Sequence

• SMART decomposes a pizza-with-onion task into an ordered subgoal sequence spanning ingredient retrieval, preparation, baking, plating, topping, and delivery.The sequence includes optional onion preparation and dependencies between successive subgoals.Source: Appendix · pp. 27–29

Semantic Reward Rules

• Semantic reward rules combine shaping, event, and terminal rewards with state predicates that detect progress toward each pizza-making subgoal.Examples include rewards for approaching or picking up ingredients, completing chopping and baking, assembling the pizza, adding onion, and successful delivery.Source: Appendix · pp. 28–29

Mapped Structural Anchors

• The framework maps gameplay subgoals to modified and added code symbols, including pizza synthesis, baking, onion handling, chopping, and plate containment.Mapped anchors identify relevant files, spans, change types, functions, classes, and variables.Source: Appendix · pp. 29–30

Evaluation Tasks

• The evaluation uses newly added task sets covering gold-related Minecraft crafting and combat objectives and diverse Overcooked recipes.The listed Overcooked tasks include pizza, roastfish, dough, and ingredient combinations.Source: Appendix · p. 30