Source-linked AI summary
AutoHarness: improving LLM agents by automatically synthesizing a code harness
Xinghua Lou, Miguel Lázaro-Gredilla, Antoine Dedieu, Carter Wendelken, Wolfgang Lehrach, Kevin P. Murphy
TL;DR
LLM agents can produce strictly illegal actions despite strong capabilities, while fine-tuning and hand-designed harnesses are costly or brittle. AutoHarness has the LLM synthesize a task-specific code harness from environment feedback, achieving fully legal actions and outperforming larger models on several evaluations; it also generates pure-code policies with higher average reward and nearly zero test-time cost.
Problem
LLM agents can perform strictly illegal actions, while fine-tuning is costly and hand-designed harnesses are brittle and labor-intensive across games.
Method
AutoHarness uses the LLM to synthesize a task-specific harness through iterative code refinement, maintaining code hypotheses and using environment feedback to improve action verification.
Results
The synthesized harness achieves 100% legal-action success across evaluated games, outperforms Gemini-2.5-Pro in two-player win rate and one-player reward, and reaches 0.870 average reward as a pure-code policy.
Takeaways & Limitations
A smaller model can synthesize custom harnesses or entire code policies that outperform larger models across these TextArena evaluations while reducing decision-time cost.
Takeaways & Limitations
The method currently generates a separate harness for each environment, and generating strategic policies for two-player games would require learning a code world model for search.
Abstract
from arXiv · showhide
Despite significant strides in language models in the last few years, when used as agents, such models often try to perform actions that are not just suboptimal for a given state, but are strictly prohibited by the external environment. For example, in the recent Kaggle GameArena chess competition, 78% of Gemini-2.5-Flash losses were attributed to illegal moves. Often people manually write "harnesses" around LLMs to prevent such failures. In this paper, we demonstrate that Gemini-2.5-Flash can automatically synthesize such a code harness, using a small number of rounds of iterative code refinement given feedback from the (game) environment. The resulting harness prevents all illegal moves in 145 different TextArena games (both 1-player and 2-player), enabling the smaller Gemini-2.5-Flash model to outperform larger models, such as Gemini-2.5-Pro. Pushing our technique to the limit, we can get Gemini-2.5-Flash to generate the entire policy in code, thus eliminating the need to use the LLM at decision making time. The resulting code-policy receives a higher average reward than Gemini-2.5-Pro and GPT-5.2-High on 16 TextArena 1-player games. Our results show that using a smaller model to synthesize a custom code harness (or entire policy) can outperform a much larger model, while also being more cost effective.
1 Introduction
LLM agents can understand tasks yet still make prohibited actions, motivating code harnesses that automatically learn to enforce environment-specific validity. AutoHarness frames harness generation as program search refined through environment feedback.
- 78% of Gemini-2.5-Flash losses in Kaggle GameArena chess were attributed to illegal moves rather than strategic blunders.
- Hand-coded harnesses verify move validity but are brittle and labor-intensive because each new game requires additional work.Fine-tuning is described as costly and potentially harmful to performance on other tasks.
- Code as harness lets an LLM complete an agent by coding a task-specific harness that calls the model and rejects unacceptable answers.The framework learns what counts as acceptable, functioning as a task-conditioned rejection sampler.
- Harness generation is formulated as program-space search, with the LLM proposing code refinements from execution feedback.The search balances exploring distinct logic structures with refining partially working harnesses.
2 Related work
Prior work applies LLMs to game agents, code-based policies, and iterative program refinement. AutoHarness combines these directions through environment-guided code refinement and structured search.
- Prior game-agent research spans text adventures, Minecraft, and chess, while benchmarks expose persistent state-tracking and action-validity difficulties.
- Earlier code-generation approaches stored executable skills, evolved reward functions, or represented robot control directly as code.AutoHarness is related but emphasizes iterative code refinement using tree search and environment feedback.
- Refinement methods range from verbal reflection and large-scale sampling to evolutionary mutation of codebases.AutoHarness integrates these ideas into structured tree search with Thompson sampling.
3 Method
AutoHarness learns environment-specific code harnesses by refining multiple code hypotheses with Thompson-sampling tree search and execution feedback. The framework supports action filtering, action verification, and fully coded policies.
- Thompson sampling selects which code hypothesis to refine, using average legal-move accuracy as each node’s heuristic value.The base LLM mutates code in response to critic feedback about legality and reward.
- The action-filter harness generates legal moves with propose_action() and uses the LLM to rank them.
- The action-verifier harness asks the LLM for an action, checks it with is_legal_action(), and retries invalid proposals with an illegal-action warning.
- Harness-as-policy uses code to choose actions and can run without an LLM at inference time.In the reported setting, it uses primitive Python functions and standard libraries such as numpy.
4 Experimental results
The experiments evaluate automatically synthesized harnesses across TextArena games, measuring legality, gameplay performance, and full policy generation. The harness prevents illegal actions and improves performance against larger or comparable agents, while code-only policies reduce test-time cost.
- Experimental setup: 145 TextArena games remain after excluding 9 free-form text or dialogue games, spanning 1-player and 2-player settings.The benchmark includes Chess, Checkers, Blackjack, Sudoku, and novel variants.
- Experimental setup: The evaluation removes Available Moves hints from some games, requiring agents to infer legal actions from environmental feedback.Without this modification, the harness could copy legal actions directly from the prompt.
- 4.1 Training: 100% legal action success was achieved for all tested games, with training ending after 14.5 tree-search iterations on average.The legality test uses novel 1000-step rollouts across 10 random seeds per game; 19/32 games required fewer than 10 iterations.
- 4.2 Evaluation: 56.3% overall win rate let the harnessed Gemini-2.5-Flash beat Gemini-2.5-Pro in 9/16 2P games.Against vanilla Gemini-2.5-Flash, the method won 12/16 games with a 64.8% overall win rate.
- 4.2 Evaluation: 0.745 average reward exceeded Gemini-2.5-Pro’s 0.707 and Gemini-2.5-Flash’s 0.673 across 16 1P games.The method achieved higher reward than Gemini-2.5-Pro in 8/16 games and tied in 5/16.
- 4.3 Harness-as-Policy: 0.870 average reward made Harness-as-Policy the highest-scoring agent across 16 1P games, above GPT-5.2-High’s 0.844.The pure Python policy has nearly zero test-time cost, whereas the GPT-5.2 experiments cost approximately $640.
- 4.3 Harness-as-Policy: Two-player policy learning remains harder because it requires reasoning about an opponent’s policy, often with runtime MCTS-like methods.A fully coded policy would also need a code world model for search, which is challenging for text games.
5 Conclusion and Future Work
The paper presents automatic code-harness synthesis as a way to improve LLM agents. Future work targets distillation, reusable harness libraries, and more challenging multimodal games.
- Conclusion and Future Work: The approach automatically synthesizes a separate code harness for each game to improve LLM-agent performance.The authors propose distilling these domain-specific agents into the base LLM and building reusable harness libraries.
- Conclusion and Future Work: Future applications include more challenging multimodal games such as Craftax and Terra Nova.
A.1 List of all 145 games
Table 1 lists all 145 TextArena games and records each learned harness’s accuracy and required LLM calls. The 32 games used for end-to-end evaluation are marked.
- A.1 List of all 145 games: Table 1 covers all 145 TextArena games with learned-harness accuracy and the number of LLM calls needed to achieve it.Games used for end-to-end agent evaluation are marked with an asterisk.
A.2 Per-game reward
Figure 6 presents TextArena 1P per-game reward results.
- Figure 6 reports reward separately for each TextArena 1P game.
- The figure covers the TextArena 1P evaluation setting.
- Per-game reward is the metric displayed in Figure 6.
A.3 Per-game Legal Action Rate
The legal-action harness is refined through environment feedback and code-focused prompting, with the generated functions designed to propose and verify valid actions. The evaluation measures legal-action success across TextArena games.
- Code refinement: The harness is refined from game boards, error feedback, existing code, and required function signatures.
- Refinement objectives: Refinement targets observed failures, game progress, loop avoidance, legal actions, and compatibility with all observed boards.
- Implementation constraints: The prompt allows random sampling among the best legal actions when necessary and requires safe, concise Python code.
- Action verification: The generated code implements propose_action and is_legal_action functions for text-game boards and actions.
- Action objective: The action verifier is intended to return one of the best legal actions while maximizing final reward.
D.1 Minesweeper-v0
The Minesweeper-v0 harness proposes legal actions by combining first-move handling, logical deduction, and probabilistic risk estimates. It repeatedly propagates deductions, selects guaranteed-safe cells when available, and otherwise chooses a minimum-risk guess.
- Strategy overview: The propose_action() harness handles first moves, guaranteed-safe-cell deduction, and probabilistic guessing.The snippet is only a partial view of the full harness.
- First move: On an entirely unrevealed board, the harness selects a central starting cell.For an even-sized board, the implementation adjusts the central index before returning the move.
- Logic deduction: Simple deduction marks unknown neighbors as mines when the remaining mine count equals their number, or as safe when no mines remain.The harness tracks revealed cells as non-mines and unknown cells separately from known mines.
- Action selection: The harness repeats deduction until no new deductions appear, then randomly selects a guaranteed-safe cell if one exists.Propagation is capped at num_rows * num_cols iterations, and the safe-cell choice is sampled from the accumulated safe list.
- Logic deduction: Subset reasoning compares clue constraints and identifies difference cells as safe when the mine-count difference is zero, or as mines when it equals their count.These advanced deductions are applied alongside the simple rules during repeated propagation.
- Action selection: When no guaranteed-safe cell is found, the harness scores unknown cells using clue-derived and global mine probabilities, then selects a minimum-risk move.For cells influenced by revealed clues, it averages the probabilities derived from those clues; isolated cells use the global probability.