Beyond Instruction Following: Learning Grounded Skill-Following with Skill Contracts

• Grounded Skill-Following fixes a visible expert-authored skill and requires completing every prescribed phase, since a correct final answer can bypass the required procedure.The framework treats procedural compliance as a task requirement distinct from outcome correctness.Source: §1 · pp. 2–3
• A skill contract pairs visible instructions with a runtime that tracks phases, admissible actions, transitions, and accepted termination against environment observations.The runtime checks admissibility, blocks protocol violations, and guides valid next actions.Source: §1 · p. 2
• Verified Progress Credit rewards initially completed milestones, while Contract-State Feedback guides valid next actions and the runtime evaluates accepted procedural completion.The framework uses GRPO for reinforcement-learning policy optimization and reports Protocol Completion Rate separately from Task Outcome.Source: §1 · pp. 2–3
• Across two domains and five executors, contract training achieves high Protocol Completion Rates, but higher PCR does not uniformly yield superior Task Outcomes.The paper reports task-dependent effects of instructions and observations.Source: §1 · pp. 2–3
• Controlled studies and observation-withholding experiments examine the effects of skill instructions, training signals, observation content, and Contract-State Feedback.The reported findings indicate task-dependent impacts of instructions and observations.Source: §1 · pp. 2–3

2.1 FROM INSTRUCTION FOLLOWING TO AGENT SKILLS

• Research on agent skills emphasizes discovery and retrieval, evolution and composition, or extraction and internalization, rather than verifying execution of a fixed skill (Wang et al., 2026b; Su et al., 2026; Xia et al., 2026; Shi et al., 2026a; Zhang et al., 2026; Dang et al., 2026; Lu et al., 2026b; Wang et al., 2026a; Lin et al., 2026).This literature follows work on response-level instruction constraints and interactive tool-using architectures (Wei et al., 2021; Sanh et al., 2021; Nakano et al., 2021; Yao et al., 2022).Source: §2.1 · p. 3
• SkillCoach and SKT evaluate or adapt provided skills to maximize task outcomes, but allow modification or soft prompts, whereas grounded skill-following enforces a fixed expert skill as a phase-by-phase contract (Zhu et al., 2026; Tan et al., 2026).The contract runtime evaluates protocol compliance during agent-environment interaction rather than permitting outcome-driven shortcuts.Source: §2.1 · p. 3

2.2 VERIFIABLE CONTROL AND INTERMEDIATE SUPERVISION

• Prior work uses finite-state reward machines, process or turn-level supervision, constrained decoding, execution-rule interventions, and procedural auditing (Icarte et al., 2018; Lightman et al., 2024; Zeng et al., 2025; Jin et al., 2025b; Feng et al., 2026; Lu et al., 2026a; Ugare et al., 2024; Liu et al., 2026; Ma et al., 2025; Huang et al., 2026; Zhou et al., 2026).These approaches address long-horizon control, credit assignment, inference-time output constraints, skill execution rules, or procedural evaluation.Source: §2.2 · p. 3
• Unlike rigid-state reward machines and inference-only constrained decoding, skill contracts pair visible instructions with a runtime for dynamic phase transitions, contract-state feedback, and verified progress credit (Icarte et al., 2018; Ugare et al., 2024).The framework also separates Protocol Completion Rate from Task Outcome to measure procedural compliance independently.Source: §2.2 · p. 3

3.1 PROBLEM AND LEARNING REQUIREMENT

• A skill contract tracks required phases, completed milestones, and recovery state; its verifier checks actions against observations and enforces valid transitions and accepted termination.The current phase determines admissible next actions and valid termination.Source: §3.1 · p. 4
• When binary Task Outcome gives sampled trajectories identical returns, Verified Progress Credit can distinguish them through different recovery-adjusted milestone totals.Task Outcome alone can yield zero group-normalized advantages and no signal to distinguish trajectories; milestone credits can break such ties, though different milestone sets do not guarantee different totals.Source: §3.1 · p. 5, §3.2 · p. 5

3.2 CONTRACT-STATE FEEDBACK AND VERIFIED PROGRESS CREDIT

• Contract-State Feedback reports action admissibility, the current phase, and an admissible next action; Verified Progress Credit rewards each milestone’s first completion, discounted after rejected actions.Rejected actions leave the phase and milestones unchanged and set a recovery flag; accepted actions update state, and accepted environment actions return observations.Source: §3.2 · p. 5

3.3 OPTIMIZATION AND TASK INSTANTIATION

• The clipped GRPO objective uses one trajectory-level advantage for all generated tokens, while skill instructions, observations, and feedback receive no loss.Neither Verified Progress Credit nor feedback guarantees a nonzero expected parameter gradient, convergence, or improved Task Outcome.Source: §3.3 · p. 6
• Math contracts require reasoning over Python execution results, while Search contracts map retrieval actions to observations and may include an optional evidence-summary phase.Both tasks share contract fields and optimizer specifications; contracts are manually authored, and automatic conversion of arbitrary skill text is outside the paper’s scope.Source: §3.3 · p. 6

4.1 EXPERIMENTAL SETUP

• Evaluation covers training and held-out Math and Search datasets; Math uses pass@1 after numerical normalization, Search uses normalized exact match, and PCR measures accepted completion separately from answer correctness.Task Outcome measures answer correctness; PCR requires completion of all required phases and accepted termination.Source: §4.1 · pp. 6–7

4.2 RQ1: GROUNDED SKILL-FOLLOWING RESULTS

• Qwen3.5-4B joint training achieves Math/Search Task Outcomes of 82.95%/46.61%, PCR of 99.27%/99.96%, and RF-PCR of 92.90%/95.40%, while Task Outcome gains over Recipe vary.Across separate and joint policies, Math PCR ranges from 94.46% to 99.27% and Search PCR from 97.00% to 99.99%; joint training does not consistently outperform separate training.Source: §4.2 · p. 7, Table 1 · p. 8

4.3 RQ2: TRAINING BEHAVIOR

• Credit-only reaches Math/Search Task Outcomes of 59.76%/29.28% and PCR of 93.17%/98.69%, while Feedback + Credit reaches 64.61%/30.11% and 97.29%/98.90%, respectively.Outcome-only and Feedback-only remain at zero; under the same contract runtime with feedback disabled, Credit-only develops nonzero Task Outcome and PCR while Outcome-only remains at zero.Source: §4.3 · p. 7, Figure 3 · p. 8, Figure 4 · p. 9
• With ρ = 0.90, Math/Search RF-PCR is 93.78%/98.18%, compared with 0.00%/88.33% for ρ = 1.00; ρ = 1.00 has higher PCR and Task Outcome.The ρ = 1.00 condition's Math/Search Task Outcome is higher by 2.49/0.70 points, but these independently trained conditions establish no consistent advantage across metrics or seeds.Source: §4.3 · p. 8, Table 2 · p. 9

4.4 RQ3: INFERENCE BEHAVIOR

• Removing visible skill instructions lowers Math/Search Task Outcome by 5.89/9.57 points, while paraphrasing preserves most performance.These interventions test instruction dependence with the rest of the contract fixed; wrong-skill substitution also changes action grammar, limiting its interpretation to compatibility.Source: §4.4 · p. 9, Figure 4 · p. 9
• Withholding observations lowers Task Outcome by 13.62, 11.22, 4.88, and 13.20 points on MATH, GSM8K, 2Wiki, and HotpotQA; removing feedback lowers Math/Search Task Outcome by 6.36/16.96 points.Removing feedback also reduces PCR from 97.29% to 86.93% on Math and from 98.90% to 43.05% on Search; these interventions show behavioral dependence, not token-level evidence use or feedback's training effect.Source: §4.4 · p. 9, Figure 4 · p. 9, Table 2 · p. 9

4.5 EMPIRICAL INSIGHTS

• High PCR does not imply improved Task Outcome over matched Recipe: Math is more sensitive to cross-task skill substitution, while Search is more sensitive to feedback removal, though substitution also changes action grammar.Small training-time Task Outcome gains on Search do not imply feedback independence: removing feedback at inference substantially lowers Task Outcome and PCR.Source: §4.3 · p. 7, §4.5 · pp. 9–10, §4.4 · p. 9

5 CONCLUSION

• Controlled learning curves show nonzero Protocol Completion and Task Outcome with verified progress credit, while outcome-only training remains at zero.The experiments separately report PCR, RF-PCR, and Task Outcome for Math and Search.Source: §5 · p. 10
• Grounded skill-following means executing a fixed expert-authored skill through its required phases while using environment observations to complete the task.Source: §5 · p. 10
• Skill contracts make procedural progress verifiable during post-training and evaluation.Source: §5 · p. 10
• Reliance on visible skill instructions and contract-state feedback varies by task, and findings are limited to manually specified contracts and tested executors.The stated scope limitation is documented in Appendix F.Source: §5 · p. 10

AI USE STATEMENT

• Generative AI tools assisted with drafting and revising sections of the manuscript.Source: AI USE STATEMENT · p. 10
• AI tools supported literature retrieval and discovery, including identifying relevant prior work.Source: AI USE STATEMENT · p. 10
• The authors used AI tools for research ideation.Source: AI USE STATEMENT · p. 10
• AI tools also supported experimental execution.Source: AI USE STATEMENT · p. 10
• The authors used AI assistance in developing proofs of mathematical claims.Source: AI USE STATEMENT · p. 10
• The authors take full responsibility for the final references, mathematical claims and proofs, experimental results, analyses, and text.Source: AI USE STATEMENT · p. 10

A.1 CANONICAL CONTRACT RUNTIME

• The runtime tracks phase, completed milestones, observations, and contract-state feedback; it also records recovery status and task-specific bookkeeping for validity, tool-call budgets, retries, and evidence organization.After an accepted action, newly verified milestones use the pre-action recovery flag, which is then reset; rejected-action feedback sets the flag, while disabling feedback keeps it zero.Source: §A.1 · p. 15
• Rejected actions remain trainable but do not advance the phase or milestones, end the rollout, or incur a negative penalty; execution can continue until accepted termination or budget depletion.Each model turn ends after its first complete leading action, while environment responses, skill instructions, and feedback are loss-masked; evaluation does not isolate rejected-action token gradients.Source: §A.1 · pp. 15–16
• By induction, every accepted transition preserves contract validity, so PCR measures structural adherence when accepted termination requires all phases.Rejected actions leave the current phase and milestones unchanged; the guarantee depends on valid verifier logic and does not establish that the executor reaches termination.Source: §A.1 · p. 16
• Structural adherence does not guarantee termination, answer correctness, or meaningful semantic processing of observations, and removing rejected actions changes feedback and recovery bookkeeping.Rejections are not themselves validated raw actions; replaying only the remaining actions cannot reproduce the original trajectory.Source: §A.1 · p. 16

A.2 FEEDBACK, MILESTONES, AND TASK ADAPTERS

• Feedback reports action acceptance, current phase, and an admissible next action without revealing the task answer; adapters require valid tool observations before continuing or answering.Math observations need successful Python execution with non-empty stdout, while Search observations need at least one non-empty retrieved document; these checks assess form, not semantic relevance or sufficiency.Source: §A.2 · pp. 16–17
• Search permits an optional <summary> action that preserves the current phase and earns credit only on first acceptance, while rejected actions discount later milestone credit by ρ = 0.90 in main runs.The summary must contain at least 24 normalized characters and four word tokens without embedded XML tags; O-Search assigns separate 0.10 credits to evidence reasoning and optional evidence organization.Source: §A.2 · p. 17, Table 5 · p. 17

A.3 MOTIVATION FOR GROUP RETURN VARIANCE UNDER SPARSE REWARDS

• Differing recovery-adjusted milestone credits can create within-group return variance before task success is common, while identical progress credits leave normalized GRPO advantages unchanged.In the two-return model, nonzero variance requires both return values in a sampled group; its probability is 1 − p^G − (1 − p)^G and is highest at p = 0.5.Source: §A.3 · pp. 18–19
• Return variance alone does not guarantee a policy gradient: returns must covary with trajectory scores, and the derivation is an idealized baseline rather than the practical GRPO implementation.The practical method also uses group normalization, token-level averaging, PPO clipping, and KL regularization.Source: §A.3 · p. 19
• A unified contract verifier derives both feedback and recovery-adjusted progress credit, aligning requested next actions with rewarded progress without asserting that feedback and credit are equally necessary.Rejected actions trigger recovery feedback, and milestones first completed on the next accepted transition are discounted by ρ.Source: §A.3 · p. 19

B.1 EXACT VISIBLE SKILL INTERFACE

• The shared scaffold offers code-interpreter and QA-search skills, instructs the agent to load a skill before reasoning, and returns its visible instructions after an accepted skill call.Single-task conditions expose one catalog entry and Joint exposes both; feedback is appended with loss mask zero.Source: §B.1 · p. 20
• The Math skill requires planning, executable Python that prints a result, environment-owned observations, result interpretation, and a boxed answer only after a valid result.Source: §B.1 · p. 20
• The Search skill requires query planning, retrieved-evidence assessment, optional evidence refinement or summary, and an answer only after valid evidence analysis.The summary organizes evidence but does not replace reasoning.Source: §B.1 · p. 21

C.1 TRAINING CONFIGURATION

• Online runs use slime with Megatron and SGLang; Math and Search use 200 rollout rounds, while Joint conditions use 400.Training draws on MATH and GSM8K for Math, Natural Questions and HotpotQA for Search, and combined Math data with sampled Search data for Joint.Source: §C.1 · p. 21
• Training uses Adam at learning rate 10−6, with specified clipping and regularization settings; Table 7 gives task-specific batch and update budgets.Trajectories allow at most 20 executor turns, 8 environment calls, and 1,024 tokens per turn within a 16,384-token context; budget exhaustion without accepted termination is protocol failure.Source: §C.1 · p. 21, Table 7 · p. 21
• The KL reference policy πref is frozen at the executor’s initial checkpoint and is distinct from the sampling policy πθold.The passage identifies Equation 8’s regularizer as using a sampled approximation but does not provide its expression.Source: §C.1 · p. 21
• Figure 3 monitors matched Outcome-only and Credit-only conditions with feedback disabled, using separate 500-example evaluation plans every 25 rounds.All-zero groups remain in training with zero normalized advantages after group centering, and each plotted point is one monitor evaluation without a repeated-run interval.Source: §C.1 · p. 21
• Search uses an E5-base-v2 retriever over frozen 2018 Wikipedia with top-3 FAISS retrieval, while Math uses a restricted Python sandbox.The Math sandbox requires nonempty output and imposes fixed resource limits.Source: §C.1 · p. 21

C.2 EVALUATION AND BASELINES

• Full-suite evaluation covers 6,319 Math and 51,713 Search examples, while Direct omits the runtime and Prompt preloads the skill and runtime without training.Direct and Prompt score answers from the whole response even when termination is not accepted, so these runs compare evaluation conditions rather than knowledge loss.Source: §C.2 · p. 22
• PCR and RF-PCR measure accepted termination, with RF-PCR additionally requiring no rejected-action feedback; Task Outcome scores accepted answers and Hint Rate tracks feedback exposure.The paper reports Protocol Completion separately from Task Outcome and does not pool Math and Search.Source: §C.2 · p. 22
• The matched ReTool-style Math and Search-R1 Recipe rows share task environments and evaluators with O-Math and O-Search, providing local context rather than official reproductions.The ReTool-style adaptation caps incorrect-trajectory scores at −0.6, unlike the referenced recipe’s 0.Source: §C.2 · p. 22

D COMPLETE MODEL-VISIBLE EXECUTION TRACES

• The execution-trace section presents correct Ours-S policy examples, preserving message wording and environment observations while replacing chat-template formatting with role labels and LaTeX-safe punctuation.The examples illustrate complete model-visible execution rather than only final answers.Source: D COMPLETE MODEL-VISIBLE EXECUTION… · p. 22

D.1 MATH

• The Math trace reports 81 lattice points within or on the circle; the policy requires executable Python, an environment-owned result, and subsequent reasoning to interpret it.Source: §D.1 · p. 23

D.2 SEARCH

• The Search trace loads a search skill, plans a concise query, receives retrieved information, and analyzes the evidence before answering the harmonica question.The protocol permits refined research and optional evidence summaries, but requires analysis of valid evidence before the answer.Source: §D.2 · pp. 24–25
• The trace identifies Sonny Boy Williamson as the harmonica player on “Bring It On Home,” based on the retrieved document about his recording.The model states this conclusion in its reasoning, summary, and final answer.Source: §D.2 · pp. 25–26

E CONTROLLED INTERVENTION RESULTS

• RQ2 independently trains Qwen2.5-3B conditions, while RQ3 tests inference-time interventions on frozen checkpoints; PCR is specific to each runtime and may include recovery after rejection.The training controls use MATH and GSM8K for Math and 2WikiMultiHopQA and HotpotQA for Search; their PCR values are not pooled as one evaluation pass.Source: E CONTROLLED INTERVENTION RESULTS · p. 26

E.1.1 FEEDBACK-CREDIT FACTORIAL

• Outcome-only, Feedback-only, Credit-only, and Feedback + Credit vary contract-state feedback and verified progress credit while holding the task reward and contract runtime fixed.In Outcome-only, rejected actions receive no penalty or termination, and only an answer accepted after all required phases can receive task reward; the factorial tests learning under one runtime.Source: Table 8 · p. 27

E.1.2 RECOVERY COEFFICIENT

• ρ = 0.90 is the main recovery setting and ρ = 1.00 the independently trained no-discount condition; Math RF-PCR is zero despite high PCR at ρ = 1.00.This indicates every trajectory reaching accepted termination receives rejected-action feedback; RF-PCR can still include feedback after accepted actions and does not measure execution without feedback.Source: §E.1.2 · p. 27

E.2.1 VISIBLE SKILL INTERVENTIONS

• S0–S3 vary visible skill instructions while keeping the contract runtime and task verifier fixed; S3 is a compatibility stress test because it changes the action grammar.S0 uses original instructions, S1 removes them, S2 paraphrases them, and S3 injects the other task’s instructions.Source: §E.2.1 · p. 27, Table 9 · p. 28
• S2 paraphrases preserve action inventory and order, environment-owned observations, repeated-action permission, and final-answer format while revising plan, relevance, and query wording.The Math rewrite describes identifying the needed calculation and alternating code with reasoning; the Search rewrite emphasizes retrieval, relevance evaluation, and revising queries when evidence is insufficient.Source: §E.2.1 · pp. 27–28

E.2.2 OBSERVATION AND FEEDBACK INTERVENTIONS

• Observation withholding replaces only the model-visible payload with a non-evidential placeholder after action execution and validation, preserving the recorded validity and contract transition.The intervention removes observation content available to the executor without changing whether the original action and observation passed runtime validation.Source: §E.2.2 · p. 28
• Feedback removal suppresses the complete inference-time feedback channel while retaining validation.Its result is reported in main-text Table 2, but the supplied passage does not provide the result values.Source: §E.2.2 · p. 28

F DISCUSSION

• Under the tested runtime, Credit-only achieved nonzero PCR and Task Outcome, while Outcome-only remained at zero; PCR and Task Outcome measure protocol completion and answer correctness separately.RF-PCR identifies completion without rejected-action feedback.Source: F DISCUSSION · p. 28
• Joint and separate training comparisons suggest high Protocol Completion can coexist with task-dependent reliance on execution signals and differing Task Outcome gains, but do not identify shared or task-specific representations.The proposed shared and task-specific mechanisms remain hypotheses; differing task exposure limits interpretation, and testing them requires targeted interventions under controlled exposure. The hypothesis is motivated by prior studies of skill neurons and layer-aware task arithmetic.Source: F DISCUSSION · p. 28
• Contracts are manually specified, and the interventions establish dependence on visible instructions, observations, and contract-state feedback only within this implementation—not token-level causal evidence use or scaffold-free internalization.Results establish the proposed problem and method in tested Math, Search, and five Qwen executor configurations, but not automatic contract construction, universal transfer, guaranteed optimization improvement, or failure of terminal-only RL under a permissive runtime.Source: F DISCUSSION · p. 28