PaperlayerView exact revision on arXiv

Source-linked AI summary

Test-Driven AI Agent Definition (TDAD): Compiling Tool-Using Agents from Behavioral Specifications

Tzafrir Rehan

arXiv:2603.08806v1cs.SEcs.AI

TL;DR

Tool-using agents lack systematic behavioral verification because manual prompt editing and spot-checking can miss edge cases and regressions. TDAD compiles specifications into tested prompts, adds hidden-test, mutation, and evolution checks, and reports strong v1 performance alongside weaker evolved-spec compilation and bounded test-strength guarantees.

  • Problem

    Manual prompt editing and spot-checking do not verify agent behavior across all specified scenarios and can miss edge cases, data leakage, and tool-order errors.

  • Method

    TDAD converts behavioral specifications into executable tests, iteratively refines prompts using visible tests, and evaluates hidden tests, semantic mutations, and specification evolution.

  • Results

    Across 24 trials, TDAD achieved 92% v1 compilation success with 97% HPR, while v2 achieved 58% compilation success, 78% HPR, 86–100% mutation scores, and 97% regression safety.

  • Takeaways & Limitations

    TDAD provides a test-driven workflow for compiling agent behavior while exposing specific blind spots through mutation testing and measuring backward compatibility during specification evolution.

  • Takeaways & Limitations

    Evaluation covers four specifications with three trials per version, uses Claude Sonnet 4.5 for all roles, and does not test larger agents, other model families, or individual anti-gaming mechanisms.

Abstract

from arXiv · show

We present Test-Driven AI Agent Definition (TDAD), a methodology that treats agent prompts as compiled artifacts: engineers provide behavioral specifications, a coding agent converts them into executable tests, and a second coding agent iteratively refines the prompt until tests pass. Deploying tool-using LLM agents in production requires measurable behavioral compliance that current development practices cannot provide. Small prompt changes cause silent regressions, tool misuse goes undetected, and policy violations emerge only after deployment. To mitigate specification gaming, TDAD introduces three mechanisms: (1) visible/hidden test splits that withhold evaluation tests during compilation, (2) semantic mutation testing via a post-compilation agent that generates plausible faulty prompt variants, with the harness measuring whether the test suite detects them, and (3) spec evolution scenarios that quantify regression safety when requirements change. We evaluate TDAD on SpecSuite-Core, a benchmark of four deeply-specified agents spanning policy compliance, grounded analytics, runbook adherence, and deterministic enforcement. Across 24 independent trials, TDAD achieves 92% v1 compilation success with 97% mean hidden pass rate; evolved specifications compile at 58%, with most failed runs passing all visible tests except 1-2, and show 86-100% mutation scores, 78% v2 hidden pass rate, and 97% regression safety scores. The implementation is available as an open benchmark at https://github.com/f-labs-io/tdad-paper-code.

1 Introduction

TDAD applies test-driven development to tool-using agents by compiling behavioral specifications into prompts and tool configurations, then measuring generalization, mutation resistance, and regression safety. It addresses manual verification, silent prompt regressions, tool misuse, and disconnected evaluation workflows.

  • Motivation: Current workflows rely on manual prompt editing, spot-checking, and hoping changes preserve prior behavior.This leaves teams without systematic verification across specified scenarios.
  • Motivation: Agents may pass happy-path cases while failing edge cases, leaking sensitive data, or calling tools in the wrong order.
  • Motivation: Prompt changes can silently break existing behavior, with failures discovered only after deployment through complaints or compliance violations.
  • Motivation: TDAD treats agent development as test-driven compilation from product requirements to executable behavioral tests and refined prompts.The methodology separates test generation, prompt compilation, mutation evaluation, and runtime execution.
  • Method: TDAD uses visible and hidden tests, semantic mutation testing, and specification evolution to reduce specification gaming and assess robustness.Only visible tests drive compilation; hidden tests and mutation scores are measurement-only.
  • Evaluation infrastructure: SpecSuite-Core evaluates four deeply specified agents using distinct roles, with a reference implementation integrated into standard testing and agent-execution tooling.The roles include TestSmith, PromptSmith, MutationSmith, and the Built Agent.

2 Related Work

TDAD extends prompt optimization and behavioral testing by compiling natural-language specifications into decision-tree tests while explicitly evaluating anti-gaming robustness. It targets the specification-to-tests-to-compilation workflow rather than only pre-built agent task performance.

  • Prompt Optimization: TDAD differs from prompt optimizers by targeting behavioral decision trees, anti-gaming mechanisms, and natural-language specifications instead of task accuracy alone.Direct empirical comparison with DSPy and other optimizers is difficult because their input formats and evaluation setups differ.
  • Behavioral Testing: TDAD adopts MFT, INV, and DIR behavioral test taxonomies while deriving tests proactively from specifications rather than prompt edit histories.
  • Agent Benchmarks: SpecSuite-Core evaluates the PRD-to-tests-to-compilation-to-regression workflow, whereas common agent benchmarks evaluate pre-built agents on diverse tasks.
  • Specification Gaming: Specification-gaming research motivates hidden tests and executable evaluations that resist optimizing for a reward or test while missing intended goals.

3 The TDAD Methodology

TDAD represents specifications as executable behavioral contracts and iteratively compiles them into prompts and tool descriptions. Separate coding-agent roles generate tests, refine the artifact, and evaluate mutation resistance, while deterministic fixtures and trace assertions support measurement.

  • Compilation model: TDAD uses “compile” as shorthand for iterative prompt refinement until an executable behavioral contract is satisfied.The specification is the source, behavioral tests are the intermediate representation, and prompt/configuration is the compiled artifact.
  • Specification Format: A TDAD specification encodes tools, prioritized policies, a decision tree, a response contract, test guidance, and mutation intents in YAML.Tests implement the specification rather than defining it.
  • Test Generation: TestSmith traverses decision-tree leaves, generates MFT, INV, and DIR tests, and creates deterministic fixtures with canary values for leakage detection.Every expectation must be derivable from a specific specification clause.
  • Prompt Compilation: PromptSmith runs visible tests, clusters failures by root cause, applies minimal edits, and repeats until success or the iteration budget is exhausted.When few tests fail, a focused inner loop runs only failing tests to reduce last-mile iteration time.
  • Mutation Evaluation: MutationSmith applies activated semantic mutations to the compiled prompt and checks whether the visible suite detects the resulting faulty behavior.Mutation evaluation occurs after compilation rather than driving prompt refinement.
  • Compiled Agent: The compiled agent includes system-prompt and tool-description overrides, with structured respond-tool outputs enabling deterministic assertions over tool-call traces.The respond tool is called exactly once per turn with schema-validated fields.
  • Compilation Results: Convergence typically occurs in 2–5 iterations across SpecSuite-Core specifications.

4 Preventing Specification Gaming

TDAD addresses specification gaming by separating compilation from evaluation, testing semantic failure modes, and measuring whether evolved prompts preserve prior behavior. Its pipeline combines hidden tests, mutation testing, and held-out regression evaluation while accounting for stochastic agents.

  • Hidden test splits: Hidden tests comprise 30–60% of the suite and are withheld during compilation to measure generalization rather than guide prompt optimization.Visible tests drive PromptSmith iteration; hidden tests are used only for reporting, with the split varying by specification complexity.
  • Semantic mutation testing: Semantic mutation testing generates plausible faulty prompt variants after compilation and checks whether visible tests detect them.MutationSmith uses intent-based mutations because dynamically synthesized prompts lack a fixed source artifact for syntactic patching.
  • Semantic mutation testing: 87% of mutation intents activated successfully, while non-activating mutants were excluded from the mutation score after failing activation attempts.Activation probes validate that a mutated prompt actually differs behaviorally from the compiled prompt.
  • Mutation score: A mutation is killed when at least one visible test fails on the mutated prompt, making mutation score a signal of whether the suite covers specified failure modes.Trace, text, and JSON predicates assess observable behavior rather than prompt structure.
  • Mutation score: Complex semantic checks such as whether a SQL query contains LIMIT are unsupported directly, so mutations are evaluated through observable outcomes.This constraint favors behavioral contracts over implementation-detail checks.
  • Spec evolution: Spec evolution starts v2 compilation from the v1 prompt while withholding v1 invariant tests, allowing SURS to measure backward compatibility under changed requirements.Each v2 introduces 2–3 coordinated changes, such as a new tool and branch, stricter policy, or schema change.
  • Stochasticity and limitations: RPR is recommended for production but was not evaluated, with standard scenarios requiring N = 10 and high-risk scenarios N = 50 under the stated thresholds.The authors note that reruns would multiply the approximately 30–60-minute cost of a single spec version.

5 SpecSuite-Core Benchmark

SpecSuite-Core is a small but deeply specified benchmark for evaluating agent compilation workflows across four multi-turn, tool-using specifications. Its design emphasizes auditability through deterministic testing, realistic behavioral branches, hidden tests, and versioned requirement changes.

  • Benchmark Design: SpecSuite-Core contains four deeply specified agent specifications with multi-turn flows, tool contracts, decision trees exceeding 10 branches, hidden tests, mutation catalogs, and v1-to-v2 evolution scenarios.Each mutation intent represents a realistic domain-specific regression mode and applies to both v1 and v2 compiled prompts.
  • Design Principle: The benchmark prioritizes depth over breadth: four auditably rigorous specifications replace thousands of shallow single-turn tasks.The benchmark is intended to make agent-compilation results easier to trust and inspect.
  • Spec Summaries: SupportOps tests priority-ordered policies and adds abuse detection in v2 while preserving v1 behaviors.Its policy order places PII protection above identity verification, plan eligibility, confirmation, and fraud escalation.
  • Spec Summaries: DataInsights requires SQL execution before grounded answers, ambiguity detection, and no fabrication; v2 adds cost estimation before expensive queries.HALLUCINATE_NUMBERS survived in v1, exposing a test gap that v2 closed.
  • Spec Summaries: IncidentRunbook enforces evidence-first ordering, severity-based routing, and runbook compliance, with v2 addressing a surviving runbook-lookup mutation.All seven mutations activated in both versions, while v1 had one surviving mutation.
  • Spec Summaries: ExpenseGuard enforces spending caps, receipt requirements, and disallowed-item rejection; v2 adds manager approval above cap and requires more compiler iterations.V2 required five iterations versus two for v1 because approval thresholds needed careful test guidance.

6 Experimental Results

Across 24 independent trials, TDAD compiled v1 specifications reliably with strong hidden-test generalization, while v2 evolution reduced compilation success and increased variance. Successful v2 runs nevertheless achieved complete mutation detection and high regression safety, with failures largely associated with conflicting tests or exhausted iteration budgets.

  • Experimental Setup: 24 independent trials evaluated four specifications across v1 and v2, using three independent runs per specification version.Each trial regenerated tests, compiled independently, and ran the full evaluation.
  • Main Results: 92% v1 compilation success and 97.3% hidden pass rate were achieved across successful runs, while v2 compilation success fell to 58%.V2 also showed higher variance than v1.
  • Spec Evolution: 97.2% average SURS indicated that adding new capabilities rarely broke existing behaviors.SURS measures the fraction of held-out v1 invariant tests that remained passing after v2 compilation.
  • Compilation Failure Analysis: V2 failures arose from conflicting tests in two runs or exhausted PromptSmith iteration budgets in three runs.Failed runs with recoverable logs still exceeded 95% VPR before budget exhaustion, and oscillation between tests suggested conflicting expectations.
  • Mutation Testing: 100% v2 mutation scores were achieved across successful runs, closing v1 gaps including the surviving HALLUCINATE_NUMBERS mutation.V1 mutation scores ranged from 86–100%, with two surviving mutants across DataInsights and IncidentRunbook.
  • Cost and Iterations: The full pipeline typically cost $2–3 per specification version, most successful compilations converged in 2–4 iterations, and 18 successful runs cost $45.15 total.IncidentRunbook v2 averaged $4.23, showing that some runs cost more as iteration counts increased.
  • Discussion: Compilation improved hidden-test performance over seed prompts, while mutation testing surfaced systematic blind spots that ordinary evaluation could miss.The lowest observed HPR was 51.1%, and SURS averaged 97.2% across successful runs.

7 Reference Implementation

The reference implementation organizes TDAD as an isolated pytest-based pipeline with separate containers and artifact access for compilation and evaluation. Its harness checks tool-call traces, structured outputs, PII leakage, and numeric grounding.

  • Repository: The repository uses a standard pytest-based layout for specifications, tests, compiled artifacts, and harness code.Claude Code serves the TestSmith, PromptSmith, and MutationSmith roles in Docker containers.
  • Isolation: The compiler mounts only visible tests, while hidden tests remain in a separate volume and execute in a separate evaluation container.Visible tests are read-only during compilation, and PromptSmith can write only prompt artifacts.
  • Harness: The harness provides assertions for tool-call presence and order, structured outputs, PII canaries, and numeric grounding.These checks target observable behavioral contracts rather than natural-language parsing alone.

8 Limitations

TDAD’s limitations concern specification completeness, stochastic and cost overhead, adversarial-test generation, and evaluation scope. These constraints mean its safeguards reduce but do not eliminate gaming risk, and broader applicability remains untested.

  • Specification completeness: Requirements such as “be empathetic” may resist encoding as precise behavioral tests.
  • Specification completeness: Mutation testing measures test strength but cannot guarantee completeness, while excluding non-activating mutants may overstate suite quality.
  • Stochastic variation and overhead: V1 HPR varies by ±2–4% and V2 by ±4–16%, while each spec version takes 30–60 minutes and costs $2–3.
  • Adversarial test generation: TestSmith may avoid genuinely hostile or profane inputs, so human-curated test corpora remain necessary for robust abuse-detection coverage.
  • Evaluation scope: Evaluation covers four specifications with three trials per version and does not test other model families, cross-model configurations, larger agents, or authoring effort.

9 Conclusion

TDAD treats agent prompts as compiled artifacts and adds hidden tests, mutation testing, and spec evolution to support regression safety. Across SpecSuite-Core trials, it reports strong v1 results but lower success for evolved specifications, while providing an open benchmark implementation.

  • TDAD treats prompts as compiled artifacts by specifying behavior as tests, iterating until tests pass, and maintaining the suite as a regression safety net.
  • 92% v1 compilation success and 97% HPR were achieved across 24 trials, compared with 58% v2 success and 78% HPR for evolved specifications.
  • The methodology is presented as applying software-engineering rigor to agent development as agents take on higher-stakes decisions.
  • The specifications, test harness, mutation packs, and Docker infrastructure are available as an open benchmark.

Appendices

The appendices define the reporting metrics used to evaluate visible and hidden test performance, valid-mutant detection, specification-update regression, invariant preservation, and reliability across runs.

  • Metric definitions: Visible Pass Rate (VPR) and Hidden Pass Rate (HPR) quantify performance on visible and withheld tests, respectively.
  • Metric definitions: HPR measures the proportion of hidden tests that pass after compilation, while only visible tests drive compilation.
  • Metric definitions: Mutation scoring uses valid activating mutants and excludes non-activating mutants that fail activation probes after k attempts.
  • Metric definitions: Spec Update Regression Score (SURS) measures preservation of v1 invariant tests after the specification update.
  • Metric definitions: Reliability Pass Rate (RPR) summarizes whether tests pass across repeated runs.

B Mutation Testing Results

Table 6 reports selected mutation outcomes, showing which representative failure intents were killed or survived across specifications. V1 mutation scores range from 86–100%, while all v2 runs reach 100%.

  • Table 6 shows selected mutation results using Killed and Survived outcomes, with four representative intents displayed from each spec’s 5–7 total intents.
  • The surviving mutants occurred in 1/3 DataInsights runs and 2/2 IncidentRunbook runs.
Loading 2603.08806v1…