PaperlayerView exact revision on arXiv

Source-linked AI summary

SkillEffect: Checked Lowering for Memory-Bounded Agent Tools

Yinuo Wang, Yiyu Shi

arXiv:2608.17007v1cs.AI

TL;DR

Model-generated tool plans can preserve task semantics yet exceed hard memory caps because physical execution choices differ. SkillEffect checks and lowers such plans into bounded implementations, with experiments showing bounded access expands completion under external caps across registered relations.

  • Problem

    Semantically valid model-generated tool plans can diverge in memory demand and hard-cap feasibility across heterogeneous data-processing tools.

  • Method

    SkillEffect uses a common checked-lowering trust boundary that reconstructs registered relation instances, validates bounded targets, and executes them under capacity control.

  • Results

    Across six operator families and six plugins, bounded access preserves verified outputs, expands completion under external memory caps, and rejects semantic and resource violations.

  • Takeaways & Limitations

    One checked dispatch architecture can enforce heterogeneous registered memory relations while reusing shared checking, execution, resource-control, and publication mechanisms.

  • Takeaways & Limitations

    SkillEffect does not automatically verify arbitrary Python programs; each supported computation requires an audited relation over a closed source grammar.

Abstract

from arXiv · show

Agent Skills can specify procedural and resource obligations for tool use, and language models instantiate them as concrete programs. However, when models turn this guidance into code for existing tool interfaces, even a semantically correct program may load an entire input and exceed the memory available to one tool call. We present SkillEffect, a checked-lowering runtime for computations with a recoverable source relation, an audited bounded implementation, and a registered output postcondition. Before granting execution authority, an independent checker rebuilds each proposed lowering from the submitted program and immutable input. Every relation plugin supplies a source recognizer, input-fact extractor, bounded-IR constructor, arena-bound function, and postcondition; one common runtime provides checked selection, bounded-VM execution, atomic capacity leasing, and staged publication. Generality in SkillEffect is architectural rather than automatic: each supported computation requires an audited relation plugin, while the dispatch, resource-control, execution, and publication mechanisms are shared across plugins. Across six operator families, bounded access substantially reduces peak memory and improves completion under externally fixed caps. Six plugins instantiate the same contract across five execution patterns, from streaming reduction to bounded-heap Top-k. The XLSX onboarding study and Top-k extension show that a new relation and a new retained-state pattern reuse the same trust boundary, while the checker accepts all evaluated legal configurations and rejects all adversarial proposals. Together, these results show that one checked-lowering architecture can enforce heterogeneous registered memory relations at Agent tool dispatch.

1 Introduction

SkillEffect addresses the gap between procedural tool guidance and physically bounded execution by independently checking relation-preserving lowerings before dispatch. Its shared runtime supports heterogeneous registered computations while relation-specific plugins define semantics, input facts, bounded implementations, resource bounds, and result validation.

  • Motivation: Under a 100 MiB tool-stage cap, eager materialization can determine whether an otherwise valid tool call completes.In an external smolagents harness, all 16 otherwise valid calls chose eager access without resource feedback.
  • Problem: Existing skill compression, cost profiles, and process gates do not construct a relation-preserving bounded program or recover its intended result.These layers preserve procedural obligations, prefer cheaper tools, or stop unsafe processes, but leave physical live-set assignment and recovery unaddressed.
  • Scope: SkillEffect targets resource-sensitive declarative computations with recoverable source semantics, bounded implementations, and registered result postconditions across heterogeneous workflows.Examples include CSV aggregation, sequence reduction, scientific metadata, chunked arrays, workbook streaming, and risk-ranked audit-log selection.
  • Architecture: Each checked relation comprises source recognition, immutable-input inspection, canonical bounded-IR construction, arena derivation, and postcondition checking before dispatch.Skill routing establishes task and tool context, while an independent checker reconstructs and validates the concrete relation instance.
  • Evaluation and contributions: The evaluation measures physical opportunity across six operator families and instantiates the common relation contract across six plugins without changing declared task results.The introduction frames the central question as converting over-budget model-generated plans into checked bounded implementations before dispatch.

2 System Model and Guarantees

SkillEffect is a checked-lowering boundary that reconstructs submitted programs and immutable input facts to select audited bounded implementations under a hard tool-stage memory cap. Admission requires a unique accepted relation, bounded execution, atomic capacity accounting, and verified staged publication; otherwise execution abstains before allocation.

  • Admission policy: Admission certifies feasibility under the current registry and cap, while unknown effects, unjustified plans, and ambiguous matches return control before capacity is consumed.Abstention occurs before bounded-VM execution or publication, allowing the scheduler or caller to choose another tool, budget, or execution site.
  • Checked relations and rule selection: The registry matches submitted programs to audited relations that bind recognized computations and immutable input facts to bounded target IR, memory bounds, and publication checks.The checker reconstructs the complete allowed source AST and source-derived semantic parameters; proposal parameters such as windows and staged-output capacities remain checked.
  • Checked relations and rule selection: Selection admits exactly one accepted rule; zero accepted rules abstain, while multiple accepted rules are rejected as ambiguous.The selected bounded IR is supplied to the VM, and the certifier receives neither a task nor a family label.
  • Capacity and publication: The VM result and serialized staged file must satisfy the registered online postcondition before no-overwrite atomic publication.The experimental verifier V_q is applied only by the host after container exit, supplying evaluation rather than execution authority.
  • Threat model and limitations: The current checker lacks independently enforced bounds on malicious source size, AST nodes, depth, and certification time, leaving parser denial-of-service as a deployment limitation.Adversarial mutations test semantic and binding confusion, not parser denial of service; production deployment must isolate and bound preflight before parsing.
  • Capacity and publication: Atomic leasing preserves Σ_j∈A_t U_j ≤ B across serialized lifecycle transitions, so admitted execution transactions remain within the configured tool-stage capacity.If each checked bound covers its cgroup’s physical live set with swap disabled, the ledger invariant implies physical occupancy at most B, while the cgroup independently enforces the cap.

3 Design

SkillEffect establishes a three-stage proposal–check–execute trust boundary in which an independent checker reconstructs the source relation, bounded target, and resource bound before execution and publication. Its reusable architecture combines audited relation plugins with common dispatch, capacity control, bounded execution, and postcondition-gated publication.

  • Trust boundary: The trusted path reparses complete source, reconstructs input facts and bounded IR, derives the checked peak bound, and gates publication on a registered postcondition.The stages are proposal, independent checking, and execution under acquired capacity; only checker-rebuilt IR reaches the bounded VM.
  • Admission: Admission depends on structurally equal checker-derived records, while dynamic calls, unresolved names, ambiguous parameters, unsupported expressions, and missing output keys are rejected as unproven.The checker derives right-hand-side values independently of the builder, and unknown calls remain unproven.
  • Relation contract: Each registered relation implements five obligations: source recognition, immutable input-fact extraction, bounded-IR construction, arena computation, and publication checking.These obligations are represented as S_r, I_r, L_r, A_r, and G_r in the common relation contract.
  • Audit boundary: Generality is architectural rather than automatic: relation authors audit each supported source language, recovered semantic facts, target semantics, arena arithmetic, and publication predicate.The common contribution is composability of audited relations with one dispatch and resource-control path, not synthesis of arbitrary program equivalences.
  • Plugin architecture: Six executable plugins span five bounded-execution patterns while sharing the same five-obligation contract and common checker mechanisms.The runtime separates relation-specific retained state and obligations from shared validation of source, input, platform, target-IR, and bound bindings.

4 Evaluation

Across fixed-cap evaluations, bounded lowering substantially improves completion, memory safety, and throughput while preserving verified semantics. The checker and shared runtime generalize across relation plugins, although model semantic capability and final-answer fidelity remain separate limitations.

  • Fixed-cap completion: At 64 MiB, bounded lowering completes 12/24 tasks versus 0/24 for direct execution, and completes all 24 from 128 MiB onward.Its normalized log-cap AUC is 0.950 versus 0.467, with stable C50/C90 of ≤64/128 MiB versus 512/2048 MiB.
  • Lowering behavior: The bounded lowerer preserves five already-safe plans unchanged and converts five unsafe plans from 0/5 to 5/5 completion, achieving 30/30 runs without cap failures.Across safe no-op tasks, direct and bounded-lowering median peaks differ by at most 0.21 MiB.
  • Model and agent limitations: Mistral produces only four verifier-valid attempts across two tasks versus 25 attempts across 10 tasks for Qwen2.5-14B, revealing a semantic capability floor before memory control.In native integration, Mistral generates matching bounded actions for all 16 tasks, but 15/16 final answers match exactly.
  • Checking and extensibility: The checker accepts all 60 legal configurations and rejects all 500 mutations by independently reconstructing semantics, input facts, target IR, and live-set bounds.The XLSX extension likewise accepts 12/12 legal configurations and rejects all 64 mutations.
  • Runtime validation: Across five relations, all 15 checked executions pass postconditions and resource gates, commit, match the oracle 15/15 times, and record zero swap, OOM, or memory-limit events.Median physical peaks range from 14.67 MiB for FCS to 71.84 MiB for XLSX, while checked bounds are 80.14–80.27 MiB.
  • Throughput and concurrency: At concurrency 16, bounded lowering completes 48/48 submissions at a 629.53 MiB median peak, versus 36/48 for direct execution at the cap.Verified goodput is 0.462 results/s versus 0.258 results/s, a 1.79× aggregate improvement; an uncapped run reports 0.543/s versus 0.461/s, or 17.9%.

5 Related Work

SkillEffect relates prior work on executable Skills, resource-aware program transformation, and Agent-runtime budget enforcement. Its contribution is connecting these lines at dispatch through independent checking, bounded execution, capacity leasing, and postcondition-gated publication.

  • Research landscape: SkillEffect sits at the intersection of executable Skill artifacts, resource-constrained program transformation, and Agent-runtime budget enforcement.These lines determine what can run, how implementations can improve, and where resource use is controlled.
  • Skill compilation and executable artifacts: Skill compilation work preserves, compresses, decomposes, profiles, and recompiles procedural requirements into executable artifacts.Examples include SkillZip, Control Under Compression, and SkVM.
  • Checked transformation and resource reasoning: Checked-transformation work applies precondition-checked rewrites, verifiable query-plan rules, reusable Pandas optimizations, and profile-guided resource repairs.The cited systems span concrete-program rewrites, compiler-rule extraction, static checkers, and codebase-scale patches.
  • Budgeted Agent execution: Budgeted-execution work profiles tool costs, formalizes multidimensional budgets, integrates capability enforcement, and controls tool-call memory spikes with cgroups.The passage names CATP-LLM, Agent Contracts, Agent libOS, AgentCgroup, and Linux cgroup v2.
  • SkillEffect’s connection: SkillEffect connects these lines at dispatch by rebuilding a bounded target and live-set bound from the submitted program and immutable input, then coupling execution to leasing and publication checks.Execution uses an atomic capacity lease, while publication is gated by a registered postcondition.

6 Limitations and Future Work

SkillEffect currently covers deterministic, local, read-only programs with checked results through six audited relation plugins spanning five bounded-execution patterns. Future work must broaden postconditions, relation and source coverage, platform calibration, external-effect protocols, and evaluation breadth.

  • Current scope: The current implementation supports deterministic, local, read-only tool programs whose results can be checked before publication.Its six relation plugins span five bounded-execution patterns.
  • Postconditions: Richer Agent tasks will require postconditions based on schemas, invariants, relational constraints, or delayed validation beyond current deterministic or input-derived checks.Each registered relation currently requires a publication check for its declared result.
  • Relation and source-form coverage: SkillEffect does not automatically verify arbitrary Python; every supported computation requires an audited relation over a closed source grammar.The current implementation includes six relations spanning five bounded-execution patterns, while broader corpora and coding-Agent traces are needed to measure coverage.
  • Platform calibration: Porting across software or hardware stacks requires fresh platform calibration and capped validation because memory bounds depend on runtime, allocator, page size, and fixed reserves.More granular reserve profiles could reduce the conservatism of the unified manifest.
  • Stateful and remote effects: Email, payment, and mutable service calls require idempotency, authorization, and transaction or compensation protocols coupling external effects to result publication.The current commit protocol stages local outputs only.
  • Evaluation breadth: Future evaluation should integrate released optimization systems, additional model families, and larger multistep workloads to test interoperability and comparative performance.The present executable baselines share a common harness.

7 Conclusion · Appendix

SkillEffect establishes a checked trust boundary at agent tool dispatch, where registered relation plugins reconstruct source computations, validate bounded targets, and execute them under capacity control. Across six operator families, this architecture preserves verified outputs and improves completion under external memory caps.

  • 7 Conclusion: Tool dispatch is the boundary where semantically valid Skill plans can diverge in memory demand and hard-cap feasibility.Resource-sensitive Agent Skills become concrete programs over concrete inputs, exposing these physical consequences at dispatch.
  • 7 Conclusion: SkillEffect uses a checked trust boundary to reconstruct source computations, extract immutable input facts, validate bounded targets, and execute checker-rebuilt programs.The common relation contract supplies five obligations, while the bounded VM executes under capacity control.
  • 7 Conclusion: Staged publication links the physical execution decision to the declared task result.Publication is staged after the checker-rebuilt target executes under the runtime’s capacity controls.
  • 7 Conclusion: Bounded access modes preserve verified outputs and expand completion under external memory caps across six operator families.The result concerns operator characterization across heterogeneous data-processing Skills.
  • 7 Conclusion: Registered plugins reuse one five-obligation contract across streaming and row-window reductions, metadata projection, chunked reduction, and bounded-state Top-k.These instances cover multiple execution patterns while retaining the same contract.
  • 7 Conclusion: The checker and bounded VM reject semantic and resource violations while completing isolated capped transactions.This provides the runtime behavior reported across the registered plugin instances.

A Detailed Experimental Protocols

The experiments use controlled protocols for cap-feedback generation, natural-plan evaluation, harness comparison, physical admission, and matched-latency measurement. These protocols isolate verifier behavior, resource effects, and runtime overhead under fixed execution conditions.

  • Cap-feedback protocol: Cap-feedback generation gives Qwen2.5-14B the family cap and input size initially, then permits at most two feedback-driven retries after failure.Feedback includes prior code, process status, cgroup peak, OOM events, streams, parse status, and one verifier bit; expected results, lowerings, and API hints remain hidden.
  • Natural plans: Natural-plan evaluation selects the first verifier-valid plan across three predeclared seeds per task without exposing verifier, memory, or lowering feedback during generation.Qwen2.5-14B-Instruct-AWQ evaluates 24 tasks at temperature 0.2, while Mistral-7B uses the same tasks, compressed Skills, seeds, and feedback policy.
  • Harness protocols: Native and smolagents harnesses reuse 16 tasks across four registered relations, while planners receive neither caps nor verifier or lowering feedback.The smolagents harness preserves its tool schemas, memory, call processing, and final-answer loop; a transport-only adapter converts vLLM responses into framework tool-call objects.
  • Five-relation admission and oracle isolation: The physical-admission study freezes the code closure, container image, five relation schemas, and one source/input bundle per relation under swap-disabled 128 MiB cgroups.CSV, FASTA, FCS, Zarr, and XLSX each run three times in fresh cgroups, with expected output withheld from the container until exit.
  • Matched latency: Matched-latency tests run all 24 tasks in fresh cgroups using six measured adjacent AB/BA pairs at an external 2 GiB cap.Warm-cache launch-to-exit time includes bounded lowering, envelope checking, execution, verification, and staged commit, but excludes Docker create/start.

B Operator Characterization

All 36 family–scale–mode cells pass verification in every repeat. At the largest scale, bounded implementations reduce fresh-cgroup memory peaks by 3.75–24.25×, with a median reduction of 8.45×, while preserving verified results.

  • Verification: All 36 family–scale–mode cells pass their verifier in every repeat.The result holds across every evaluated family, scale, and access mode.
  • Memory peaks: 3.75–24.25×: bounded implementations reduce largest-scale fresh-cgroup peaks, with a median reduction of 8.45×.The reduction range spans evaluated operator families and access modes.
  • Resource contract: Access mode belongs in the resource contract because input-size-based reserves waste memory for metadata and streaming tasks but can be unsafe for eager arrays and workbooks.The observed spread in peak reductions motivates mode-specific resource treatment.
  • Result equivalence: Paired eager and bounded executions produce the same verified result.Figure B.1 compares largest-input physical peaks while annotating eager-to-bounded ratios.

C Development-Cap Prompting and Repair

Under per-family development caps, prompting rarely constructs correct bounded programs, while reject-only gates safely abstain without failures. Cap feedback improves completion for a minority of calls but does not replace deterministic lowering.

  • Construction and Repair: Pinning the resource clause yields correct bounded programs for 9/24 tasks, while ordinary retry succeeds for 4/24.These caps test mechanism behavior rather than the externally fixed-cap completion result.
  • Construction and Repair: 5/24 tasks complete after 67 total attempts with cap feedback, but AnnData, Polars, XLSX, and Zarr remain 0/4.The five successes comprise three Biopython and two FlowIO tasks, and all selected programs pass three fresh repeats.

D Frontend, Scale Transfer, and Latency

The frontend recovered dispatchable contracts and effects while rejecting negative controls, and the calibrated envelope respected bounds across evaluated scales. Bounded execution reduced latency relative to direct execution, with low preflight and checker overhead, although prompting alone was unreliable.

  • 13/20 held-out positive Skills yielded dispatchable contracts, effect-kind and evidence were recovered for 14/20, and all five negative controls abstained.The static frontend accepted 13 dispatchable contracts among 20 held-out positive Skills.
  • None of six admission-eligible bounded cells exceeded its bound across four calibration scales and a fifth evaluation scale.One eager estimate was ineligible for small-cap dispatch.
  • 44.7% and 99.3% were the median and maximum reserve overheads among admitted cells.Across 10,000 preflight iterations, AST capture plus envelope instantiation took 0.804 ms median and 1.260 ms P95.
  • 0.811 was the median bounded/direct latency ratio across 24 task-level medians, with every task below one and a 0.590–0.958 range.All 144 measured latency pairs were valid.
  • 2.744 ms and 0.839 ms were the median bounded preflight and checker phases, while the operator dominated the measured pipeline.The always-pin baseline produced a verifier-passing bounded program for only 9/24 tasks, so faster bounded operators did not make prompting reliable enforcement.

E Five-Relation Physical Admission · F Relation-Extension Protocols

The five-relation physical-admission study validates a frozen configuration under a shared 128 MiB cap, while separate XLSX and Top-k protocols define how relation extensions are evaluated. Across these protocols, execution uses isolated configurations, bounded resources, and host-side evaluation after container completion.

  • E Five-Relation Physical Admission: Each relation ran in three fresh 128 MiB cgroups under the frozen five-relation admission configuration.Every run passed its online postcondition and committed before host-side exact evaluation.
  • E Five-Relation Physical Admission: All physical-validation runs recorded zero swap, OOM, and memory-limit events.They also satisfied P_phys ≤ U ≤ B.
  • E Five-Relation Physical Admission: Table E.1 reports per-relation memory values over three fresh 128 MiB cgroups and median capped-phase wall times.The table provides the physical values underlying the unified 15-transaction study.
  • F Relation-Extension Protocols: The XLSX onboarding study used a separately frozen code closure, mutation suite, workbook, and cgroup protocol.Its 64 onboarding-time mutation canaries were distinct from the five-relation matrix’s 500 mutations.
  • F Relation-Extension Protocols: The XLSX physical repeats were separate from the uniform five-relation admission matrix.The two frozen configurations yielded a 72.32–72.43 MiB onboarding range and a reported 71.84 MiB value.
  • F Relation-Extension Protocols: The Top-k semantic-breadth protocol fixed a 750,000-record, 124,455,560-byte strict JSONL ledger with k = 64.It also fixed risk ordering as score DESC followed by event id ASC.
  • F Relation-Extension Protocols: Top-k used a 128 MiB primary cap and a 2 GiB eager-validation cap, with separate checked and eager repeat schedules.The primary schedule used three balanced eager/checked repeats, while the 2 GiB control used three eager repeats; evaluation remained host-side after container exit.
Loading 2608.17007v1…