Source-linked AI summary
Skill Issue: Lessons from Optimizing Repository SKILLs for Coding Agents
Mykhailo Kozyrev, Andrei Kozyrev, Anton Podkopaev
TL;DR
Repository SKILL optimization lacks ready-made benchmarks, and synthetic tasks can leave capable agents with little room to improve. The paper mines reverted merged pull requests and compares each candidate with a seed rollout on the same task. GEPA raises the mean score by 4.9pp, while SkillOpt raises it by 0.1pp; these gains cannot be separated from agent variance at the available split size.
Problem
Repository-specific SKILL optimization lacks a ready-made task distribution and metric, while prior synthetic tasks can leave capable agents with little headroom.
Source: §1 · p. 2Method
The authors mine merged pull requests reverted at one frozen base and compare each SKILL candidate with the seed on the same task using the same agent.
Source: §1 · p. 2, §3.1 · p. 4Results
Takeaways & Limitations
A koog maintainer found project-specific knowledge in both SKILLs, alongside general advice the agent did not need.
Source: §1 · p. 3, §4.3 · p. 7The requirement for at least 100 graded tasks yields held-out splits of only 20 to 26 tasks, too small to distinguish few-point gains from agent variance.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Coding agents increasingly read repository knowledge from SKILLs --- plain \texttt{.md} files versioned alongside the code. Recent work synthesizes these files automatically, by optimizing the document against a benchmark. A bare repository comes with no benchmark, and the synthetic tasks prior work builds are small enough that a capable agent saturates them with no document at all. We mine harder tasks --- merged pull requests of the repository, reverted at a single frozen base commit; and score a candidate document by whether the same agent does better with it than without it. On three Kotlin repositories, the documents GEPA finds raise this score by $4.9$pp on average, and the ones SkillOpt finds leave it where it started, $0.1$pp above the seed. The GEPA gain matches what prior work reports with the same optimizer, and at the dataset size a single repository supplies it cannot be separated from the agent's run-to-run variance; settling that would take more tasks than one repository's history yields. The documents themselves read better than the score: a maintainer of one repository found in them knowledge one only gets by working in the project.
1 Introduction
Repository SKILL optimization lacks repository-native benchmarks, while synthetic tasks can leave capable agents with little room to improve. This paper mines merged pull requests and compares optimized SKILLs with a seed, finding a 4.9pp average gain for GEPA and 0.1pp for SkillOpt, with gains difficult to separate from agent variance at this dataset size.
- Repository-specific SKILL optimization lacks a ready-made task distribution and metric, while synthetic tasks can saturate a capable agent.Prior work’s strongest configuration reached 100% on one repository without a skill and 94.8% on another.Source: §1 · p. 2
- The study optimizes a single repository-specific .md document against a mined metric, comparing GEPA with SkillOpt.The same agent is evaluated while only the SKILL varies.Source: §1 · p. 2
- The verifier reverses merged pull requests at one shared frozen base, then compares a candidate’s rollout with the seed’s on the same task.The tests that fail after reversion specify the task, and the agent’s model, tools, and limits remain fixed.Source: §1 · p. 2
- Across three Kotlin repositories, GEPA raised the mean score by 4.9pp and SkillOpt by 0.1pp, neither statistically significant on splits of 20 to 26 held-out tasks.GEPA’s gain matches prior work using the same optimizer, but the split size cannot separate it from run-to-run variance.Source: §1 · pp. 2–3
- The score’s uncertainty reflects both agent run-to-run variance and limited repository task supply, while each candidate-task score costs a full agent rollout.For koog, mining and validation leave 119 tasks from 660 merged pull requests; a rollout averaged $0.84.Source: §1 · p. 3
- A koog maintainer found project-specific knowledge in both optimized SKILLs, alongside general advice the agent did not need.On two live issues, either SKILL was associated with under half the wall-clock time and lower cost than no SKILL.Source: §1 · p. 3
2 Preliminaries
The paper compares two reflection-driven optimizers that edit natural-language artifacts using agent rollouts. GEPA rewrites candidates freely from a Pareto frontier, whereas SkillOpt applies bounded edits and accepts them only after a held-out improvement.
- Both optimizers were chosen to edit natural-language artifacts by reflecting on rollouts, without extra inference-time infrastructure.Source: §2 · p. 3
GEPA
- GEPA selects candidates from a Pareto frontier, gathers rollout scores and feedback on minibatches, and uses a reflection model to propose rewrites.Improved candidates enter a pool and are evaluated on a larger selection set used to maintain the frontier and choose the final candidate.Source: §2 · p. 3
SkillOpt
- SkillOpt reflects on successes and failures to propose bounded edits, accepting a candidate only when it strictly improves a held-out selection score.Its edit budget decays over time, while an epoch-wise update folds durable lessons into a protected document field.Source: §2 · p. 4
- GEPA freely rewrites candidates sampled from a Pareto frontier, while SkillOpt makes budgeted edits behind a strict improvement gate.Both are instrumented against the same validation score to separate metric effects from optimizer effects.Source: §2 · p. 4
3 Methodology
The methodology mines repository-specific tasks and evaluates SKILL documents with a shared procedure that keeps the coding agent fixed. It defines task collection, validation, and a paired score comparing a candidate with the empty seed SKILL.
3 Methodology
- Evaluation runs the same agent in an isolated repository copy with history hidden and returns its attempt, edits, and resulting test behavior.The first candidate has no advice; later candidates edit that initial document.Source: §3 · p. 4
3.1 Task collection
- PR mirroring adapts merged Kotlin changes into tasks, but forward tasks can fail on dependent changes and teach obsolete repository layouts.Forward tasks apply a change’s test patch at its historical parent commit and ask the agent to implement the change.Source: §3.1 · p. 4
- Reverse mining reverts implementations at one frozen base commit, using reverse-apply, structural undo, then LLM reconstruction for eligible modified files.This keeps tasks anchored to the current repository; changes whose code is no longer present at the base are unavailable.Source: §3.1 · p. 4
- LLM reconstruction supplies most graded tasks in the reported repositories, but can introduce defects the validation gate misses.The paper reports a static check for catching these defects.Source: §3.1 · p. 5
- When no issue is linked, an LLM rewrites the pull-request description into issue-style text, checks it for solution leakage, and repository links are scrubbed.Containers also strip .git and are reinitialized with a single commit before the agent starts.Source: §3.1 · p. 5
- The validation gate admits tasks only when reverting the implementation switches tests from passing to failing, and adds up to 30 regression guards.It rejects tasks with no switching test, pre-existing failures, an inapplicable patch, or a reversion that disrupts too much of the suite.Source: §3.1 · p. 5
3.2 The objective
- Each candidate is scored against the empty seed SKILL by comparing same-task rollouts from the same agent under the same conditions.The paired score asks whether the agent did better with the document than without it.Source: §3.2 · p. 5
- Rollouts are compared first for test-file edits or false success claims, then for hidden-test completion; remaining ties favor bounded measures of test fraction, honesty, diff size, and tool calls.The hierarchy prevents a cheaper or tidier rollout from outranking a correct one; absolute scores were rejected because solved-task counts left almost all rollouts at zero, while document grades could improve without changing agent behavior.Source: §3.2 · p. 5
4 Experiments
The experiments test whether repository-mined tasks reveal SKILL gains and whether maintainers value the resulting documents. GEPA scores exceed the seed on all three repositories, but the available splits cannot establish significance.
4.1 Setup
- Claude Code with Sonnet 4.6 was frozen across fresh Docker attempts at each repository’s frozen base; only SKILL.md changed.Tools and limits were held constant, with tasks split once into train, validation, and test.Source: §4.1 · p. 6
- The mined pools retained 119/660 koog PRs, 131/452 ktor PRs, and 100 kotest tasks; pooled no-SKILL resolution was 53%.Median patch sizes were 54 lines across 3 files on koog, 27 across 2 on ktor, and 16 within one file on kotest; the repositories differ, so the contrast is between task-building methods, not projects.Source: §4.1 · p. 6
4.2 The measured gain
- GEPA scored 0.554/0.567/0.525 on kotest/ktor/koog versus seed 0.5; SkillOpt scored 0.546/0.437/0.519, respectively.On the same tasks and seed, GEPA won on every repository, while SkillOpt's loss on ktor canceled gains elsewhere.Source: §4.2 · pp. 6–7
- No proposer run cleared p = 0.05; with 20–26 tasks, the sign test rejects only if the document wins four of five disagreements.Pooled over 69 tasks, the test rejects only when the document wins two of three disagreements.Source: §4.2 · p. 7
4.3 A maintainer’s reading
- The koog maintainer valued GEPA’s multiplatform boundaries and module-dependency guidance, but flagged omissions around @Tool and expect/actual patterns and over-specific unstable APIs.The maintainer wanted more developer-spelled-out detail for multiplatform work.Source: §4.3 · p. 7
- SkillOpt’s standout koog-specific item was the undocumented three-file module-registration change; the maintainer preferred GEPA’s document as a draft to polish.They described much of SkillOpt’s remaining content as general best practices, while noting this was one maintainer’s opinion of two documents.Source: §4.3 · p. 7
- On two koog issues, either SKILL cut end-to-end time from 15 and 20 minutes without one to 6–8 minutes, and cost from $6.06 and $4.85 to $2.44–$4.20.All three patches for #1275 lacked an integration test; the maintainer judged the #1354 solutions overall okay.Source: §4.3 · p. 7
5 Related Work
Prior work supplies optimization machinery and repository context-file formats, but a bare repository lacks the task distribution and metric those methods assume. Existing synthesis approaches vary in their artifacts, evaluation settings, and the repository tasks they use.
Prompt and context optimization
- Textual-context optimizers typically assume a task distribution and scalar metric are given, whereas a bare repository supplies neither, making metric construction part of the problem.The setting inherits optimization machinery but not that premise.Source: §5 · p. 8
Repository-level context files
- Repository-level .md files can be reviewed and maintained with code, but evaluations found generated files did not significantly improve resolution and could restate README content.When other repository documentation was stripped, generated files were slightly ahead (+2.7%).Source: §5 · p. 8
Automatic skill synthesis
- Prior skill-synthesis work ranges from executable skills for reward-rich environments to natural-language experience libraries, while code-repository methods must first define a task metric.Skill-optimizer constructs isolated test environments, while gskill generates task datasets over a target repository.Source: §5 · p. 8
- Gskill’s target agent template has a 34-character instance prompt, versus 2,369 characters in the stock mini-swe-agent template, so its skill can recover missing scaffolding.The reviewed excerpt offers generic agent hygiene, and the authors caution that skills may help SWE-smith-style issue-fixing tasks more than general coding practices.Source: §5 · p. 8
- Gskill’s released Go instances are PR-mirrored, and only 1 of 957 Python instances is; 37.6% of Go instances lack problem statements used in evaluation.SWE-smith’s own ablations rank PR mirroring as its most valuable strategy, yet it is nearly absent from the released data; the described SWE-smith instances are comparatively simple.Source: §5 · pp. 8–9
6 Limitations
The mining pipeline requires enough graded tasks to support separate training, validation, and test splits, but that threshold does not ensure a sensitive evaluation. Some repositories fail the pipeline’s reverse-application or test-validation checks.
6 Limitations
- The pipeline runs only when mining and validation leave at least 100 graded tasks, a floor needed to make three disjoint splits.The resulting held-out splits contain only 20 to 26 tasks.Source: §6 · p. 9
- At 20–26 held-out tasks, pass-rate comparisons cannot separate a few-point gain from the agent’s own variance.The threshold enabling three splits is not evidence that the dataset is large enough for sensitive comparisons.Source: §6 · p. 9
- Two attempted repositories failed different checks: tracy retained 5 tasks from 203 PRs after reverse-application, while fewer than 100 of http4k’s 150 reverse-applicable PRs survived validation.Reverse-applicability alone does not establish that reverting a change makes a test fail in isolation.Source: §6 · p. 9
7 Conclusion
Merged-pull-request tasks leave room for SKILLs to matter, but the measured gains are difficult to separate from agent variance at the scale one repository provides. A maintainer found repository-history knowledge in the documents, while broader validation remains open.
7 Conclusion
- Merged-pull-request tasks are harder than prior synthetic tasks, where capable agents already solve nearly all small, mostly single-file tasks.The baseline resolve rate on the mined tasks leaves room for a SKILL to matter.Source: §7 · p. 9
- 4.9pp for GEPA and 0.1pp for SkillOpt were the measured gains, which are not separable from agent variance at this dataset size.A repository yields few merged-pull-request tasks, and each task costs a full agent run.Source: §7 · p. 9
- A maintainer of one studied library found repository knowledge in the documents that is available only by working in the project.Whether this holds more broadly remains an open question.Source: §7 · p. 9
Responsible use statement
Synthesized SKILLs can present stale or over-general claims as authoritative repository knowledge. They should be reviewed before merging, because the paired score measures one agent's behavior on one mined dataset rather than document accuracy.
Responsible use statement
- A synthesized SKILL can present stale or over-general claims as authoritative repository knowledge.Examples include a mistaken distinction between repository variants and general advice presented as repository knowledge.Source: Responsible use statement · p. 9
- Review any synthesized SKILL before merging it, as with any other pull request.Source: Responsible use statement · p. 9
- The paired score measures one agent's behavior on one mined dataset and does not certify that a SKILL is accurate.Source: Responsible use statement · p. 9
A Defects the LLM reverter introduces
Reverting old pull requests against current code can leave misleading defects: a missing import may break compilation, or a stranded function may leave the solution in place. A static check caught both shapes in an intermediate task pool, including cases that passed dynamic validation.
A Defects the LLM reverter introduces
- Reverting a pull request can leave an unimported symbol that makes the task repository fail to compile.Source: A Defects the LLM reverter introduces · p. 12
- A stranded fix can leave the function itself in place after deleting its only call, with the answer still written out.Source: A Defects the LLM reverter introduces · p. 12
- A static check rejected 56 of 280 tasks in an intermediate koog pool; 36 had already passed dynamic validation.The 56 comprised 36 unimported-symbol cases, 26 stranded-fix cases, and 6 with both.Source: A Defects the LLM reverter introduces · p. 12
B Cost and time of the two open koog issues
Table 2 summarizes the cost and time of six runs on two open koog issues, solved under three SKILL configurations. It distinguishes time spent in model calls from end-to-end time.
B Cost and time of the two open koog issues
- Table 2 covers six runs: two open issues, each solved with no SKILL, the GEPA document, and the SkillOpt document.Source: B Cost and time of the two open koog… · p. 12, Table 2 · p. 12
- The patches from the six runs are printed in § J.Source: B Cost and time of the two open koog… · p. 12
- The table distinguishes API time spent in model calls from end-to-end wall time.Source: Table 2 · p. 12
C Extended limitations
Reverse-applying a pull request only establishes that its diff fits the frozen tree; validation must also show that its reversion isolates failing tests. Thus, candidate counts do not predict how many graded tasks a new repository will supply.
C Extended limitations
- A task survives validation only when reverting it makes previously passing tests fail and those failures isolate that change; unrelated failures, an already-red base, or no failing tests disqualify it.There is no numeric cutoff for extra failures: once failures spill beyond the change, the task no longer grades that change.Source: C Extended limitations · p. 12
- Reverse-application can leave hundreds of candidates, yet fewer than 100 graded tasks, because only validation determines whether reversion isolates tests.Candidates without their own test hunk may still qualify if existing tests fail on revert.Source: C Extended limitations · p. 12
- From 700 listed http4k pull requests, 150 changes reverse-applied, but validation left fewer than 100 graded tasks.The 150 included 35 changes applied with git, 82 with the LLM reverter, and 33 without their own test hunk.Source: C Extended limitations · p. 13
- With only four hand-collected http4k training tasks, SkillOpt rejected every rewrite because the empty SKILL already scored 1.0 on their mean.Each task scored 1 only if the agent passed every hidden test, and 0 otherwise.Source: C Extended limitations · p. 13
- For a new repository, touched files must remain in the frozen tree, the base test suite must pass, and each reversion must cause isolated test failures.A linked issue is not required; when absent, a leak-checked statement is synthesized from the pull request.Source: C Extended limitations · p. 13
- After both checks, ktor retained 131 graded tasks from 452 recent pull requests and koog retained 119 from 660; these yields are not expected rates for new repositories.Collection stopped once 100 graded tasks remained, so the reported counts reflect what those repositories yielded.Source: C Extended limitations · p. 13
D Objectives we rejected
The rejected objectives reward different properties of a document or the agent, but do not consistently measure whether the agent improves with the document. Pairing instead asks whether the same agent does better on the same task than with the seed.
D Objectives we rejected
- About 92% of binary-resolve rollouts scored zero, so the optimizer rarely saw a gradient; when solved-task scoring produced a document, it wrote training pull requests back into prose.The resulting recipes were useful for the instances seen, not a reusable policy, and the score could rank split difficulty or overfit the harness.Source: D Objectives we rejected · p. 13
- On the gold-match protocol, held-out score fell, and ten runs of the same SKILL on the same solved task spanned CodeBLEU 0 to 1.The objective rewards matching the maintainer’s patch, so a correct fix written differently is punished; the computable Kotlin CodeBLEU is mostly n-gram overlap.Source: D Objectives we rejected · p. 13
- Cost-based objectives either favor an empty SKILL with no added prompt tokens or a long repository map that consumes turns; under a 14-turn cap, the long document emitted no patch.Pricing tool calls selected a full Tracy map, while a work-priced mix selected a leaner guide.Source: D Objectives we rejected · p. 14
- A judge that reads only the patch and problem statement can score the agent’s writing while leaving no document to ship; one that reads the SKILL selected an onboarding note instead of measuring agent improvement.The latter described repository identity, the top-level tree, and TracingManager.Source: D Objectives we rejected · p. 14
- File-quality scores can rise on held-out tasks without agent improvement: the ideal document was worst on every behavior metric in the shared-pool comparison.The gmu document emitted no patch on that pool.Source: D Objectives we rejected · pp. 14–15, Table 4 · p. 14
- Pairing asks whether a document changes the agent’s behavior on the same task relative to the seed; on the preliminary shared pool, it shipped the seed.The other objective families can remain useful when a project wants a different factor, but they select a different kind of file.Source: D Objectives we rejected · pp. 13, 15
D.1 Generic seed
- The preliminary Tracy sweep used a short, repository-agnostic checklist as its baseline, while paired runs started from an empty file.The checklist advised reading project documentation, locating code before editing, testing early, and making minimal changes.Source: §D.1 · p. 15
D.2 Solved-task skill
The partial-credit test-resolve objective selected a SKILL containing concrete Tracy task recipes, including a test-fixtures conversion, version bumps, and LangfuseMediaSpanProcessor guidance.
D.2 Solved-task skill
- Partial-credit test resolve selected task-specific Tracy recipes for a java-test-fixtures conversion, version bumps, and LangfuseMediaSpanProcessor.The selected candidate’s reported mean was 0.833 across three tasks.Source: §D.2 · p. 15
- The document’s recipes specify updating all Gradle consumers for a converted module, locating Kotlin and library version bumps, and handling Langfuse media uploads.It also gives repository-specific test commands and tracing-domain guidance.Source: §D.2 · p. 16
D.3 Gold-match skill
The gold-match skill aligned its repository modules and APIs with the maintainer’s patch. The token-price skill instead selected compact Gradle test-task conventions.
D.3 Gold-match skill
- The gold-match skill listed modules and APIs aligned with the maintainer’s patch, including Tracy module structure and tracing-core APIs.The document described the modules, source layouts, and APIs represented in the selected skill.Source: §D.3 · pp. 16–17
D.4 Token-price skill
- The token-price objective selected compact Gradle test-task conventions, distinguishing `test` for JVM modules from `jvmTest` for multiplatform modules.The skill advises checking the module’s applied plugin before choosing a task.Source: §D.4 · pp. 17–18
D.5 Tool-call-effort skill
The repository skills map Tracy’s modules, source sets, tracing APIs, and build commands. They also give agents concrete guidance for testing and making changes across the codebase.
D.5 Tool-call-effort skill
- The D.5 guide maps tracing modules and recommends provider-specific client instrumentation, while documenting shared test fixtures and module test commands.Source: §D.5 · pp. 18–20
- Tracing core defines span and attribute handling; content-capture policies redact inputs or outputs when their respective capture flags are false.Source: §D.5 · pp. 19–20
- The D.5 guide lists project build and test commands, notes local-test opt-ins and live-endpoint requirements, and flags Gradle naming and compiler pitfalls.Source: §D.5 · pp. 19–21
D.6 Work-mix skill
- The D.6 guide describes Tracy’s modules and Kotlin Multiplatform source sets, including JVM implementations and JavaScript stubs in tracing:core.Source: §D.6 · p. 21
- The eval module supplies generator, evaluator, test-case, and result interfaces, with a JUnit base for multi-run experiments and a console test subclass.Source: §D.6 · pp. 21–22
- Before editing, inspect repository guidance and locate code paths; run a small test subset early, diagnose failures from output and tests, and favor root-cause fixes.Source: §D.6 · p. 22
D.7 Skill-reading ensemble skill
This section presents an ideal file-policy skill and repository SKILLs for Koog, Kotest, and Ktor, spanning project architecture and maintenance notes, module and build-layout facts, and general editing and testing policies. Their guidance covers implementation tracing and investigation, source-aware edits, Gradle task selection, and build, compilation, test, and API checks.
D.7 Skill-reading ensemble skill
- The Tracy skill centers tracing on TracingManager and documents Kotlin Multiplatform expect/actual conventions, including JVM implementations and JS stubs.For new cross-platform abstractions, it calls for declarations in commonMain and actual implementations in jvmMain and jsMain.Source: §D.7 · pp. 23–24
- When upgrading OpenAI or Anthropic SDKs, verify that reflection-accessed client field names remain valid and keep both adapters in sync.The skill identifies okHttpClient, originalHttpClient, and httpClient as fields accessed by reflection.Source: §D.7 · p. 24
D.8 File-facts skill (gmu)
- The file-facts skill distinguishes the Kotlin Multiplatform tracing module’s jvmTest task from the standard test task used by other modules.It warns that using jvmTest on ai-dev-kit-test-base or tracking-provider modules fails because the task is not found.Source: §D.8 · p. 25
- The file-facts skill records tracing architecture, including processor setup, trace publishing, and the requirement to implement expect declarations for both JVM and JS.It also describes provider modules binding publishers and configurators through Kodein dependency injection.Source: §D.8 · p. 26
D.9 File-policy skill (ideal)
- The file-policy skill advises verifying repository facts, making minimal changes, running the relevant tests, and limiting exploration to the module implicated by the task.For multi-module projects, it specifies jvmTest for Kotlin Multiplatform modules and test for plain JVM modules.Source: §D.9 · p. 27
- The ideal file-policy skill directs agents to trace symbols, update all call sites and implementors, respect visibility and expect/actual rules, and verify compilation and tests.It also recommends checking test imports against declared dependencies and diagnosing failures from compiler errors or test assertions before editing.Source: §D.9 · pp. 27–28
E GEPA generated SKILL for koog
- The Koog skill grounds API changes in module API files, preserves source-set and dependency boundaries, and checks streaming and serialization contracts.It recommends scoped compilation for cross-source-set changes and validating stream termination and generic type handling.Source: E GEPA generated SKILL for koog · pp. 30–32
F GEPA generated SKILL for kotest
- The Kotest skill maps its module and source-set layout and highlights JVM-only checks, expect/actual conventions, and API-dump requirements for public changes.It notes that module inclusion can depend on OS, CI, or branch configuration.Source: F GEPA generated SKILL for kotest · p. 34
- Kotest guidance distinguishes execution-changing extensions from observational listeners, favors existing extension points and composition, and documents core spec, matcher, property, tag, and project-configuration models.It also treats public API signatures as contracts that must be reflected in module API dumps.Source: F GEPA generated SKILL for kotest · p. 35
G GEPA generated SKILL for ktor
- Ktor uses a flattened Gradle layout: nested directory paths do not define project names, so tasks should target the leaf module name.For example, the directory ktor-client/ktor-client-curl maps to :ktor-client-curl.Source: G GEPA generated SKILL for ktor · p. 37
H SkillOpt generated SKILL for koog
- For Koog bug investigation, use a working sibling implementation as an oracle, inspect ignored lambda parameters and lossy raw-class overloads, and check changelogs and related implementations.The guidance also recommends looking for orphaned KDoc, stale imports, and non-exhaustive when branches as clues to what changed.Source: H SkillOpt generated SKILL for koog · p. 40
- For package moves, update every affected sibling file, old-package reference, and consumer import, then fix compile fallout across modules.The guidance calls out tests, examples, API descriptors, and related DTO field changes as possible parts of the same missing diff.Source: H SkillOpt generated SKILL for koog · pp. 40–41
- Use API declarations and every symbol referenced by the failing test as a checklist, then compile the whole module and run the named test.Check that declared members exist in source, including expect/actual counterparts and required dependencies; unresolved sibling-module errors also need fixing before the test can run.Source: H SkillOpt generated SKILL for koog · p. 41
- If a reported runtime bug has no source implementation, use the failing test as its specification and mirror a peer implementation.Common fixes include restoring a missing sealed subtype, rename alias, visibility, or DTO field, and correcting iteration direction or capability-aware defaults.Source: H SkillOpt generated SKILL for koog · pp. 41–42
- Fix related fallout in the same pass: reconcile public API call sites, inspect corrupted files end-to-end, and run tests that can fail at runtime.In explicit API mode, public overrides need declared return types.Source: H SkillOpt generated SKILL for koog · p. 42
- For module or dependency failures, restore missing build scripts and registrations, add required test dependencies, and classify the first test error before summarizing.New modules require updates to their build script, settings include, and umbrella included/excluded set.Source: H SkillOpt generated SKILL for koog · pp. 42–43
I A SKILL optimized over per-PR bases
The per-PR-base SKILL describes two repository variants whose layouts and APIs do not coexist at one commit. It warns agents to identify the active variant before applying repository-specific guidance.
I A SKILL optimized over per-PR bases
- The per-PR-base document combines repository details that hold at no single commit, presenting two variants for agents to distinguish.The repository drift over the mined window surfaces in both its opening premise and its detailed layouts and APIs.Source: I A SKILL optimized over per-PR bases · p. 44
- The document distinguishes a flat single-module demo from a multi-module Kotlin Multiplatform library, and advises identifying which repository shape is active.The demo uses root-level Gradle files and src/main/kotlin and src/test/kotlin; the library separates tracing, evaluation, plugin, tracking, and example modules.Source: I A SKILL optimized over per-PR bases · pp. 44–45
- For the demo variant, the guidance flags mismatched type arity, absent client constants, artifact argument order, and suspend calls from non-suspend contexts.It also records package-name variation, integration-test dependence on a local MLflow server, and the ByteBuddy tracing approach's inability to handle suspend functions.Source: I A SKILL optimized over per-PR bases · pp. 44–46
- For the library variant, the tracing module has common, JVM, and JS source sets, while a compiler plugin wraps annotated functions with tracing calls.The plugin supports suspending functions through withTraceSuspended; JS implementations are stubs, and JVM-only tracing relies on OpenTelemetry.Source: I A SKILL optimized over per-PR bases · pp. 46–47
- The guidance recommends compiling before tests, checking package declarations, and reporting integration checks that external services prevent verifying.It notes that the ByteBuddy agent requires -java-parameters, only targets the ai and org.example packages, and that abstract test bases need concrete runnable subclasses in src/test/kotlin.Source: I A SKILL optimized over per-PR bases · pp. 45, 47
J Agent patches for the two open koog issues
- Six patches associated with table 2 were reviewed anonymously by the maintainer.The passage identifies the patches and review but supplies no outcomes or comparison.Source: J Agent patches for the two open koog… · p. 48
Issue #1275, no SKILL
This section describes repository changes to expose cache and reasoning token usage in response metadata and response headers through HTTP client exceptions. It also covers provider-specific metadata conventions and test support for configuring and checking exception headers.
Issue #1275, no SKILL
- Metadata is null when no cache usage is reported, and `cacheCreationInputTokens` is Anthropic- and Bedrock-specific.Source: Issue #1275, no SKILL · pp. 48, 53
- Anthropic response metadata exposes cache-read and cache-creation token counts, including in streaming responses.The streaming test checks counts of 120 cache-read tokens and 30 cache-creation tokens.Source: Issue #1275, no SKILL · pp. 48–49
- Google and OpenAI clients map cached tokens to `cacheReadInputTokens` in response metadata across their response paths.Source: Issue #1275, no SKILL · pp. 50–52
- The no-SKILL passage documents cache-usage metadata keys for clients to report tokens read from and written to cache.The named keys are cacheReadInputTokens and cacheCreationInputTokens.Source: Issue #1275, no SKILL · p. 54
Issue #1275, GEPA SKILL
- GEPA’s changes map Anthropic, Google, and OpenAI cache and reasoning usage into response metadata.Anthropic uses cacheCreationInputTokens and cacheReadInputTokens; Google and OpenAI use cachedInputTokens and reasoningTokens. An OpenAI test checks values of 4 and 3 for the latter keys.Source: Issue #1275, GEPA SKILL · pp. 55–57
Issue #1275, SkillOpt SKILL
- SkillOpt’s changes map Google and OpenAI cache and reasoning token counts into response metadata.For Google and OpenAI Responses API usage, the metadata keys are cachedInputTokens and reasoningTokens.Source: Issue #1275, SkillOpt SKILL · pp. 58–59
- The SkillOpt-SKILL patch tests that OpenAI response metadata exposes cached input and reasoning token counts.The test checks cachedInputTokens as 4 and reasoningTokens as 3.Source: Issue #1275, SkillOpt SKILL · p. 60
Issue #1354, no SKILL
- The no-SKILL patch adds response headers to HTTP client exceptions and exposes case-insensitive header lookup.Exception messages include rate-limit headers only, avoiding disclosure of other headers such as cookies.Source: Issue #1354, no SKILL · pp. 61–62
Issue #1354, GEPA SKILL
- The GEPA-SKILL variant changes the HTTP client exception constructor to accept a headers map.Its API signature specifies Map<String, String>.Source: Issue #1354, GEPA SKILL · p. 65
- GEPA added a lower-cased Map<String, String> of response headers to KoogHttpClientException, defaulting to empty when none were supplied.The documentation notes that lower-cased names support case-insensitive HTTP headers and rate-limit hints.Source: Issue #1354, GEPA SKILL · p. 66
- GEPA populated exception headers from Java, Ktor, OkHttp, and Spring WebClient responses, joining each header’s values into one string.A shared test checks that the content-type header is exposed with a lower-cased name.Source: Issue #1354, GEPA SKILL · pp. 66–68
Issue #1354, SkillOpt SKILL
- SkillOpt stored response headers as lower-cased Map<String, List<String>> values and added header(name) to return the first value case-insensitively.The method returns null when the header is absent.Source: Issue #1354, SkillOpt SKILL · p. 69
- SkillOpt passed response-header maps into exceptions and preserved value lists in the Java and Ktor response-header adapters.The Java adapter uses the response headers map, while the Ktor adapter associates header names with their value lists.Source: Issue #1354, SkillOpt SKILL · p. 70
- Response headers are added to error details for Ktor, OkHttp, and Spring WebClient clients.OkHttp also captures headers before closing an unsuccessful response.Source: Issue #1354, SkillOpt SKILL · pp. 70–71
- Tests verify that error headers are exposed in lowercase and can be retrieved case-insensitively.The failure fixture supplies an X-RateLimit-Reset-Requests header with value 6m0s.Source: Issue #1354, SkillOpt SKILL · p. 71
- The mock server accepts configurable response headers and adds them to its response.The test configuration adds a responseHeaders map, which the server iterates to set headers.Source: Issue #1354, SkillOpt SKILL · p. 72