PaperlayerView exact revision on arXiv

Source-linked AI summary

WitnessGym: Benchmarking Coding Agents on the Construction of Bug Witnesses

Haomin Qi, Xiangzhe Xu, Yiming Huang, Jingbo Shang, Chengpeng Wang

arXiv:2609.36635v1cs.SEcs.AI

TL;DR

Bug-validation benchmarks are constrained by manual construction and reuse of public historical bugs and witnesses, despite the need for executable evidence developers can act on. WITNESSGYM automatically injects bugs into test-reached paths of real projects and retains cases exposed by construction-time witnesses. Across evaluated agents, witness construction remains difficult, with the best configuration achieving 68.9% average validation success.

  • Problem

    Existing bug-validation benchmarks rely on manually curated tasks or historical bugs and witnesses, limiting the supply of new evaluation targets and creating opportunities for prior exposure to solutions.

    Source: §1 · pp. 1–2

  • Method

    WITNESSGYM injects bugs into test-reached paths of real projects, validates them with construction-time witnesses, and varies surrounding structure with bug-preserving transformations.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    68.9% average validation success is achieved by Codex with GPT-5.4, while success generally falls with longer execution contexts and deeper transformations.

    Source: §1 · p. 2

  • Takeaways & Limitations

    WITNESSGYM provides a way to generate fresh validation tasks as agents evolve and study progress toward more reliable AI-assisted code auditing.

    Source: §6 · p. 10

    The current implementation covers ten observable bug types in Maven-based Java projects with replayable tests; broader language, model, and developer coverage remains future work.

    Source: A LIMITATIONS · p. 14

Abstract

from arXiv · show

Bug validation asks a coding agent to produce an executable witness for a reported bug. The witness combines a concrete input with a testing harness and exposes faulty behavior during execution. Such evidence makes audit findings actionable, yet benchmark evaluation is difficult when cases reuse public historical bugs and witnesses or require manual construction. We present WitnessGym, an automated framework for constructing bug-validation benchmarks through bug injection. It injects bugs into test-reached paths of real projects, rebuilds each project, and retains cases exposed by a construction-time witness. Bug specifications and execution adapters allow extension to additional bug types and languages. Bug-preserving transformations vary the surrounding structure while preserving the witness behavior. Based on real-world Java projects with test suites, WitnessGym automatically constructs 1,300 benchmark cases. The injected patches resemble historical bug patches and are difficult for the two evaluated models to distinguish in blinded comparisons. We evaluate four coding agent frameworks in six framework/model pairings across bug types, execution contexts, and transformation depths. Witness construction remains difficult even when the bug pattern is known. Our framework, benchmark cases, and evaluation scripts are available.

1 INTRODUCTION

Executable witnesses turn bug reports into inspectable evidence, but existing benchmarks depend on manual curation or public historical bugs and witnesses. WITNESSGYM automates fresh case construction in real projects, and evaluation shows that witness construction remains difficult.

Motivation

  • Existing benchmarks rely on curated challenges or historical vulnerabilities, limiting new evaluation targets and creating opportunities for prior exposure to solutions.Executable witnesses provide concrete evidence developers can inspect and use to distinguish real bugs from false alarms.Source: §1 · pp. 1–2

Approach

  • WITNESSGYM injects bugs into test-exercised production paths, validates them with construction-time witnesses, then removes those witnesses before agent evaluation.Bug-preserving transformations vary surrounding call, data, and control structure while retaining the witness behavior.Source: §1 · p. 2

Evaluation

  • The best configuration, Codex with GPT-5.4, achieves 68.9% average validation success, leaving substantial headroom on witness construction.Across agents and models, average success differs by 50 percentage points; Value Flow and Logic bugs are generally harder than API Contract bugs, and success declines with longer execution contexts or deeper transformations.Source: §1 · p. 2

Contributions

  • The benchmark contains 1,300 execution-validated cases spanning ten bug types and six real-world Java projects with test suites.The framework evaluates agents and analyzes factors affecting bug-validation performance.Source: §1 · p. 2

2 BACKGROUND AND MOTIVATION

Bug validation requires constructing an executable witness from a reported bug, not merely locating suspicious code. Existing benchmark approaches trade off curation, historical case supply, real-project context, and control over structural variation; WITNESSGYM combines automated fresh cases with real projects and adjustable context.

Bug Validation

  • A bug-validation witness combines a concrete input and testing harness to expose a bug through an observable failure in the required environment.The agent must recover relevant state, call sequence, and path conditions; evaluation therefore needs real program contexts and a known failure oracle.Source: §2 · p. 3

Existing Benchmarks

  • Curated CTF benchmarks require new challenges and environments, historical benchmarks depend on available artifacts, and prior injection methods offer limited control over surrounding structure.The compared approaches include NYU CTF Bench and Cybench for curated challenges, LIBRO, Issue2Test, and CyberGym for historical bugs, and LAVA, EvilCoder, and FixReverter for injection.Source: §2 · p. 3

Construction Goals

  • WITNESSGYM injects fresh bugs into test-reached code and applies witness-preserving transformations to produce real-project cases with adjustable surrounding structure.The stated construction goals include multiple bug families and an executable oracle for every accepted case, without per-case manual bug design or witness authoring.Source: §2 · p. 3

3 METHODOLOGY

WITNESSGYM builds bug-validation cases from real projects and existing tests, retaining cases whose construction-time witness exposes the injected bug. Its stages collect execution contexts, inject bugs, and apply transformations that preserve the observed failure.

3 METHODOLOGY

  • WITNESSGYM uses an existing test as the harness and concrete input for each injected bug, then removes that construction-time witness before agent evaluation.Each injection and transformation is checked by rebuilding the project and rerunning the witness.Source: §3 · p. 4

3.1 EXECUTION CONTEXT COLLECTION

  • For each test, instrumentation records an ordered trace of function locations and forms an execution context from the unique locations in that trace.The execution context map links each test case to its context; restricting injection to functions exercised by a test improves reachability likelihood and reduces subsequent execution-guided injection cost.Source: §3.1 · p. 4

3.2 BUGGY CODE INJECTION

  • WITNESSGYM groups CodeQL-derived patterns into API Contract, Value Flow, and Logic categories, randomly selecting ten eligible types across the categories.A coding agent injects a bug into a test-reached context; the project is rebuilt and the test rerun, with execution feedback used for refinement until the bug is observed or the attempt budget is exhausted. Accepted bugs must produce observable runtime failures.Source: §3.2 · pp. 4–5

3.3 BUG-PRESERVING CODE TRANSFORMATION

  • Bug-preserving transformations alter surface representation and contextual structure while preserving bug behavior under the same construction-time witness.Operators manipulate call-stack, data-flow, and control-flow structure; failed transformations are reverted, yielding variants that share an underlying bug and witness.Source: §3.3 · p. 5

4 EVALUATION

The evaluation tests six coding-agent framework/model pairings on a 900-case sample and examines bug type, execution-context length, transformation depth, and patch realism. Agents achieve varied success, with longer contexts and deeper transformations associated with lower success; blinded judgments find injected patches difficult to distinguish from historical bugs.

4.1 EVALUATION SETUP

  • The benchmark covers 1,300 cases from six Java projects, with ten bug types, four execution-context-length categories, and four transformation-depth levels; 900 cases were evaluated across six pairings in NoEC and WithEC settings.NoEC omits execution context; WithEC supplies it alongside the bug description and location.Source: §4 · p. 5, §4.1 · pp. 5–6, §4.2 · p. 6

4.2 RQ1: HOW WELL DO CODING AGENTS VALIDATE BUGS THROUGH EXECUTION?

  • 68.9% (Codex) and 59.4% (Claude Code) average validation success; OpenHands with DeepSeek-V3.2 exceeds OpenCode with Qwen3-Coder-480B-A35B-Instruct by 29.7 percentage points.The best-performing agent still fails on 31.1% of cases.Source: Table 3 · p. 6, §4.2 · p. 6
  • Execution context improves success by 2.2–4.7 percentage points across agents, but does not change their ranking; the WithEC leader succeeds on 70.7% of cases.Supplied context guides code inspection but does not provide a complete witness.Source: §4.2 · p. 6

4.3 RQ2: HOW DO BUG TYPES AFFECT EXECUTABLE BUG VALIDATION?

  • Logic bugs challenge A1 and A2; A3 matches A2 on Guard Predicate Inversion and slightly exceeds it on In-Bounds Collection Misrouting, but trails on both Value Flow types.The 500-case subset uses 25 short- and 25 long-context cases per bug type, with depth-3 transformations.Source: §4.3 · pp. 6–7, Table 4 · p. 7
  • WithEC gains vary across agents and bug types; knowing involved functions still leaves agents to choose inputs, establish program state, and write checks that expose faulty behavior.The comparison holds the depth-3 transformation configuration fixed across bug types.Source: Figure 3 · p. 7, §4.3 · p. 7

4.4 RQ3: HOW DOES BUG-VALIDATION PERFORMANCE VARY WITH EXECUTION-CONTEXT LENGTH?

  • WithEC success falls for every evaluated agent as execution contexts grow; Codex with GPT-5.4 drops from 80% on short contexts to 52% on long contexts.Longer contexts require reconstructing more intermediate states, setup choices, and dependency chains.Source: Figure 4 · p. 8, §4.4 · p. 8

4.5 RQ4: HOW DOES BUG-VALIDATION PERFORMANCE VARY WITH TRANSFORMATION DEPTH?

  • WithEC success generally declines with transformation depth: Codex with GPT-5.4 falls from 80% at depth 1 to 59% at depth 10.These comparisons characterize the evaluated transformation sequences, not the independent contribution of each operator.Source: §4.5 · p. 8

4.6 RQ5: HOW REALISTIC ARE THE INJECTED BUG PATCHES?

  • 51.64% and 52.14% pairwise accuracy meet the two judges’ primary and auxiliary equivalence criteria; positive-control accuracies of 92.00% and 84.50% show sensitivity to conspicuous artifacts.Claude Sonnet 4.6’s supplementary pairwise AUC interval slightly exceeds the equivalence bound.Source: Figure 5 · p. 9, §4.6 · p. 9

5 RELATED WORK

Related work spans input-generation methods, bug-reproduction benchmarks, and bug-injection approaches. WitnessGym is positioned as constructing multi-category repository targets with construction-time witnesses and controlled structural variants.

Input Generation

  • Input-generation tools use execution feedback, coverage objectives, path constraints, or target-directed signals to produce tests and reach bug-revealing conditions.WITNESSGYM instead evaluates repository cases from supplied bug descriptions and locations, requiring agents to build both input and execution harness.Source: §5 · p. 9

Benchmarks

  • Bug-reproduction benchmarks use reports, vulnerabilities, program pairs, or curated fault–test pairs, while bug-injection methods also construct bugs.WITNESSGYM creates multi-category targets with construction-time witnesses and controlled structural variants.Source: §5 · p. 9

Task and Data Construction

  • Change2Task derives verified agent tasks from repository history across five maintenance families (Qi et al., 2026); WITNESSGYM complements it with fresh, execution-validated bug-witness targets.Related data-governance work studies selecting and combining heterogeneous sources during multilingual model adaptation (Qi et al., 2025).Source: §5 · pp. 9–10

6 CONCLUSION

WITNESSGYM generates executable bug-validation tasks through injection and witness-preserving transformations, and the evaluation finds that agents struggle as contexts grow and transformations deepen. Its present implementation is limited to Maven-based Java projects, while broader context and patch-realism assessments remain open.

6 CONCLUSION

  • Coding agents struggle to construct witnesses as execution contexts grow and transformations deepen; blinded comparisons found injected patches difficult to distinguish from historical bugs under the evaluated protocol.The conclusion presents WITNESSGYM as a way to generate fresh validation tasks as agents evolve and study progress toward more reliable AI-assisted code auditing.Source: §6 · p. 10

A LIMITATIONS

  • Context-length and transformation-depth comparisons fix the bug type, so broader combinations are needed to test how these factors interact across types.The blinded patch study covers two judges and one presentation protocol; broader model coverage and developer assessments could examine other aspects of realism.Source: A LIMITATIONS · p. 14

B BROADER IMPACT

  • The implementation focuses on Maven-based Java projects and CodeQL Java-derived bug types; applying the workflow elsewhere requires runnable tests, observable validation signals, and language-specific adapters.The framework can support further study of bug validation for AI-assisted software engineering and security analysis.Source: B BROADER IMPACT · p. 14

C IMPLEMENTATION DETAILS

The implementation uses replayable Maven tests in six Java projects, injects observable bugs into test-reached code, and retains cases only after validation. It then applies sequential structural transformations while requiring each case to remain valid and expose the same bug.

Project and Bug Type Preparation

  • Six real-world Java projects use Maven, enabling individual test cases to be replayed with localized commands; tests involving external services, network access, nondeterministic timing, or excessive runtime are excluded.Adapting beyond Maven-based Java requires isolated execution of a designated test plus tracing, transformation, and validation adapters.Source: C IMPLEMENTATION DETAILS · p. 14
  • The benchmark selects ten observable bug types across three categories; pure performance regressions are out of scope because their effects are difficult to observe deterministically.Types were randomly selected from eligible patterns surveyed from CodeQL Java query examples to bound construction and evaluation cost.Source: C IMPLEMENTATION DETAILS · p. 14

Execution Context Collection

  • Execution context length counts distinct production-side functions reached during a test; contexts are bucketed as short [1, 40], mid-short [41, 70], mid-long [71, 120], and long [121, 150].Longer contexts can span multiple production functions, classes, or modules; shorter contexts correspond to more constrained paths.Source: C IMPLEMENTATION DETAILS · pp. 14–15

Buggy Code Injection

  • Injection candidates must make a genuine production-code change consistent with the intended bug type, and replay must pass on the clean baseline while exposing the bug in the modified repository.Each candidate gets up to three attempts from a clean workspace; compilation failures, infrastructure failures, and cases that do not expose the intended bug are discarded.Source: C IMPLEMENTATION DETAILS · p. 15

Bug-Preserving Transformation

  • Ten transformation operators across three families rewrite the bug region, with 1, 3, 5, or 10 operators applied sequentially; each step rebuilds and reruns replay, retaining only cases that preserve the same witness.The transformations vary program structure without repairing the bug; Table 6 identifies the ten operators.Source: C IMPLEMENTATION DETAILS · p. 15, Table 6 · p. 16

D DETAILED BENCHMARK STATISTICS

The benchmark contains 1,300 archived buggy cases from six Java projects, spanning 201 execution contexts, ten bug types, and ten transformation operators. Construction costs approximately $16,500, with about 63% of attempted candidates passing the pipeline.

D DETAILED BENCHMARK STATISTICS

  • 1,300 archived cases span 201 execution contexts: 56 short, 45 mid-short, 45 mid-long, and 55 long contexts.Each case stores a buggy repository snapshot, execution context, bug type, transformation sequence, and construction-time witness; Figure 6 summarizes benchmark composition.Source: D DETAILED BENCHMARK STATISTICS · p. 16, Figure 6 · p. 17
  • 688 cases use Guard Predicate Inversion and 68 each use the other nine bug types; category totals are 824 Logic, 136 Value Flow, and 340 API Contract cases.The anchor bug type supports systematic variation of transformation depth and execution context length.Source: D DETAILED BENCHMARK STATISTICS · p. 16
  • The ten transformation operators occur 4,406 times; Call-Site Rerouting appears 1,028 times, Helper Call Insertion 1,026, Local State Bundling 801, and Rare Input Path Conditioning 609.The remaining operators each occur fewer than 219 times.Source: D DETAILED BENCHMARK STATISTICS · p. 16
  • Approximately 63% of attempted candidates pass construction, and the estimated end-to-end construction cost is approximately $16,500.The estimate includes failed attempts as well as successful injection, transformation, context collection, replay validation, and construction-time witness validation.Source: D DETAILED BENCHMARK STATISTICS · p. 16

E EXTENDED ANALYSIS OF EXECUTION CONTEXT EFFECTS

In a controlled subset, validation success decreases with longer execution contexts for every agent under both NoEC and WithEC. Longer contexts expand the behavior a witness must reconstruct before the bug becomes observable.

E EXTENDED ANALYSIS OF EXECUTION CONTEXT EFFECTS

  • The extended analysis uses the same 400-case controlled subset of Guard Predicate Inversion as the main text, with four context-length buckets and four transformation-depth buckets.Table 7 aggregates cases by context length over transformation depths.Source: E EXTENDED ANALYSIS OF EXECUTION… · p. 17
  • Under WithEC, success falls from 80% to 52% for A1, 76% to 40% for A2, 64% to 28% for A3, and 60% to 28% for A4 between short and long contexts.The table groups 400 Guard Predicate Inversion cases across four context-length and four transformation-depth buckets, with 25 validated cases per combination.Source: E EXTENDED ANALYSIS OF EXECUTION… · p. 17
  • Execution-context guidance improves several buckets, but WithEC success still drops 28 percentage points for A1 and 36 percentage points for A2 from short to long contexts.Longer contexts require selecting relevant setup operations, preserving intermediate states, and constructing an oracle that exposes the bug; they expand the behavior a witness must reconstruct.Source: E EXTENDED ANALYSIS OF EXECUTION… · p. 17

F EXTENDED ANALYSIS OF TRANSFORMATION DEPTH EFFECTS

Under WithEC, validation success generally declines as transformation depth increases, though individual agent results show plateaus and reversals. Depth is a coarse difficulty control that complements execution-context length.

F EXTENDED ANALYSIS OF TRANSFORMATION DEPTH EFFECTS

  • WithEC success decreases from depth 1 to depth 10: A1, 80% to 59%; A2, 72% to 51%; A3, 60% to 40%; and A4, 55% to 40%.Source: F EXTENDED ANALYSIS OF TRANSFORMATION… · p. 17
  • Execution-context length increases the triggering behavior to reconstruct, whereas transformation depth changes how the bug is structurally embedded in code.Transformations can add code fragments, alter control structure, or change data dependencies, making depth complementary to execution-context length as a construction-side complexity factor.Source: F EXTENDED ANALYSIS OF TRANSFORMATION… · p. 18
  • Depth-related success declines include plateaus and reversals: A5 stays at 32% from depth 3 to 5, while A6 rises from 16% at depth 5 to 17% at depth 10 under WithEC.The authors characterize transformation count as a coarse difficulty control; finer measures would account for individual operators’ structural and semantic effects.Source: F EXTENDED ANALYSIS OF TRANSFORMATION… · p. 18

G ADDITIONAL ANALYSIS OF TRANSFORMATION DIMENSION EFFECTS

Transformation operators alter code surrounding injected bugs, and deeper sequences produce larger patches while reshaping earlier edits. The analysis also specifies how historical and injected patches were compared under blinded judgments.

G ADDITIONAL ANALYSIS OF TRANSFORMATION DIMENSION EFFECTS

  • Call-stack, data-flow, and control-flow operators alter boundaries, value movement, and buggy-path conditions, changing what agents must reconstruct to expose a bug.Depth comparisons concern evaluated operator sequences because later steps can reshape earlier edits, so they do not isolate each operator’s contribution.Source: G ADDITIONAL ANALYSIS OF… · p. 18
  • From depth 1 to 10, mean final patch size grows from 65.0 to 243.7 non-empty diff lines and mean hunk count from 2.33 to 6.09, while changed files remain nearly flat at 1.22 to 1.44.Figure 7 and Table 9 report patch evolution; the additional edits remain concentrated within a small number of files.Source: G ADDITIONAL ANALYSIS OF… · pp. 18–19, Figure 7 · p. 19, Table 9 · p. 20
  • At depth 10, early transformation steps retain 77.5% of added lines but have 30.0% sequence similarity to the final patch, versus 94.9% and 80.7% for late steps.First-step retention drops from 100.0% at depth 1 to 74.9% at depth 10, while sequence similarity falls to 25.9%; cumulative step-patch lines reach 6.63× final-patch lines at depth 10.Source: G ADDITIONAL ANALYSIS OF… · p. 19, Figure 8 · p. 19
  • Patch metrics describe evaluated sequences for one fixed bug type; they do not establish which structural changes cause validation failures or isolate an individual operator’s effect.The analyses use the same 400-case transformation-depth subset and examine 1,900 step-level patches alongside each case’s final buggy patch.Source: G ADDITIONAL ANALYSIS OF… · pp. 18–19

H.1 SAMPLES AND JUDGMENT PROTOCOL

  • The source-discrimination study sampled 700 injected cases, stratified by repository and bug type, and matched them to 700 historical bugs by repository and edit size.The protocol standardized patch direction, formatting, and context length, and withheld commit metadata and source labels.Source: §H.1 · p. 20
  • Two judges made repeated single-patch and pairwise judgments in fresh contexts; artifact-bearing positive controls tested whether conspicuous artificial edits could be recognized.Each model made 4,200 single-patch, 1,400 main pairwise, and 200 positive-control judgments.Source: §H.1 · p. 20

H.2 STATISTICAL ANALYSIS

Both judges met the study’s primary and auxiliary equivalence criteria, though one supplementary AUC interval exceeded the bound. Repeated judgments varied across patches, and position preferences qualify interpretation of aggregate results.

H.2 STATISTICAL ANALYSIS

  • Pairwise accuracy averages each pair’s two orientation-specific correctness indicators; equivalence requires the complete 90% confidence interval to lie strictly within 0.45–0.55.The study uses 10,000 whole-cluster bootstrap resamples to retain related judgments together.Source: §H.2 · p. 20
  • Both judges met the primary and auxiliary equivalence criteria and passed the sensitivity check, but Judge B’s supplementary pairwise AUC interval reached 0.55252, above the equivalence bound.The authors caution that these findings concern source discrimination by two judges under this protocol, not equivalence between broader bug distributions.Source: §H.2 · p. 21, Table 10 · p. 21, Table 11 · p. 21

H.3 PREFERENCE AND STABILITY DIAGNOSTICS

  • Figure 9’s split category means different underlying patch choices after positions were swapped, not a tie; correct judgments totaled 723/730 for the main sample and 184/169 for controls for judges A/B.Main groups contain 700 pairs and control groups 100.Source: §H.3 · p. 21, Figure 9 · p. 22
  • More than half the patches received non-unanimous single-patch labels, and judge B showed a left-choice tendency; balancing historical-patch position does not establish position invariance.Aggregate equivalence should not be read as stable decisions on every individual patch.Source: §H.3 · p. 21, Table 12 · p. 21

I PROMPT DESIGN

The prompt design combines layered behavioral and repository constraints with case-specific instructions to guide bug injection, bug-preserving transformations, and executable witness validation, including structured reports of transformations and trigger attempts.

I PROMPT DESIGN

  • A reusable system prompt sets global behavioral constraints, while dynamically assembled payloads supply case-specific bug, transformation, execution, replay, anchor, and file-scope information.The appendix describes the layered prompt structure used during benchmark construction and executable bug validation.Source: I PROMPT DESIGN · p. 22

I.1 BUG INJECTION SYSTEM PROMPT

  • Bug injection must create a localized production-code diff that makes the provided replay command fail for the intended bug, with at most three construction-side replay runs.The system prompt forbids unauthorized test, build, configuration, and API-signature changes, and requires a single-line INJECT_REPORT_JSON with operational evidence.Source: Bug Injection System Prompt · p. 23

I.2 BUG INJECTION TASK PAYLOAD

  • The injection payload specifies bug-type requirements, forbidden patch elements, failure and trigger hints, execution context, candidate anchors, and permitted production-file scope.It requires the patch to match the bug-type contract and report site-selection rationale and expected trigger path.Source: Bug Injection Task Payload · p. 23

I.3 BUG-PRESERVING TRANSFORMATION TASK PAYLOAD

  • Transformation payloads specify operator intent and constraints, and require localized production-code changes that compile while preserving the injected bug’s replay-triggering behavior.The transformations need not preserve the full behavior of the original program; examples include rare-input conditioning, helper insertion, and private-adapter rerouting.Source: Bug-Preserving Transformation Task… · pp. 23–24

I.4 BUG VALIDATION SYSTEM PROMPT

  • Bug validation asks for a Java test that exposes the injected bug under the provided Maven command, allowing at most three attempts and prohibiting production or configuration changes.The test must be materialized under src/test/java before verification, and success requires compilation and exposure of the injected bug.Source: Bug Validation System Prompt · p. 24

I.5 BUG VALIDATION TASK PAYLOAD

  • Validation payloads provide case metadata and bug summaries; WithEC includes construction-collected execution context, while NoEC omits it, and neither supplies the construction-time witness or oracle.The payload requests a TRIGGER_REPORT_JSON describing the suspected mechanism, test selection, trigger strategy, and expected failure signal.Source: Bug Validation Task Payload · p. 25

I.6 END-TO-END CASE PROMPT EXAMPLE

  • The end-to-end example uses Dubbo’s Guard Predicate Inversion bug with a depth-3 transformation chain and WithEC validation, illustrating an inverted guard that can return null instead of the expected parameter.The example identifies Rare Input Path Conditioning, Helper Call Insertion, and Call-Site Rerouting / Private Adapter Layer as its transformations.Source: §I.6 · pp. 25, 27
  • The transformation report schema records changed files and edit sites, additions or replacements, compatibility, and expected effects on stealth and trigger depth.Source: §I.6 · p. 28
  • Three applied transformations add private helpers and adapter methods around URLParam lookup while preserving public APIs, compilation, replay triggerability, and the injected guard inversion.The report describes the resulting structure as resembling localized helper extraction and profile-aware lookup, with extra calls before the wrong value is observed.Source: §I.6 · p. 28
  • The agent must create a minimal Java test that exposes the injected bug, without fixing code or changing production sources, build files, or unrelated files.The test must be materialized under src/test/java and follow the specified package and class conventions.Source: §I.6 · p. 29
  • The example allows at most three validation attempts, each using the provided replay command and the same test-only constraints.Source: §I.6 · p. 29
  • Success requires a compiling test whose verification fails from a semantic bug-triggering assertion, not from compilation or harness errors.Source: §I.6 · p. 29
  • The case targets a guard-predicate inversion in Dubbo URLParam method-specific lookup, with the agent instructed to exercise that path and assert a valid expected result.The supplied context identifies the production and official test paths, relevant methods, and a replay command; it withholds the reference trigger implementation and validation oracle.Source: §I.6 · pp. 29–30
  • The trigger report schema captures the suspected mechanism, inspected and selected test files, trigger strategy, expected failure signal, and materialized files.The example strategy constructs URLParam with multiple parameters, exercises method-specific lookup, and asserts the expected result.Source: §I.6 · p. 30
Loading 2609.36635v1…