PaperlayerView exact revision on arXiv

Source-linked AI summary

PREPING: Building Agent Memory without Tasks

Yumin Choi, Sangwoo Park, Minki Kang, Jinheon Baek, Sung Ju Hwang

arXiv:2605.13880v1cs.AIcs.CL

TL;DR

Agents face a cold-start gap because existing memory construction depends on target-environment task experience. PREPING builds procedural memory beforehand by controlling synthetic practice and selectively storing validated trajectories, improving performance across three benchmarks while lowering deployment cost; its applicability assumes sufficiently detailed tool documentation.

  • Problem

    Existing memory construction relies on offline demonstrations or online interactions, leaving newly connected environments without task-specific experience for building procedural memory.

  • Method

    PREPING uses proposer memory to guide synthetic task generation, a Solver to execute tasks, and a Validator to admit only suitable trajectories into solver memory.

  • Results

    PREPING improves over no-memory baselines by 17.1 points on AppWorld, 19.3 points on BFCL v3, and 5.4 points on MCP-Universe, while remaining competitive with target-task-based methods.

  • Takeaways & Limitations

    Procedural agent memory can be constructed before deployment, with proposer-side control and selective updates contributing beyond synthetic task volume alone.

  • Takeaways & Limitations

    PREPING assumes sufficiently detailed API or tool documentation, which may limit applicability when tool semantics, preconditions, or state constraints are poorly documented.

Abstract

from arXiv · show

Agent memory is typically constructed either offline from curated demonstrations or online from post-deployment interactions. However, regardless of how it is built, an agent faces a cold-start gap when first introduced to a new environment without any task-specific experience available. In this paper, we study pre-task memory construction: whether an agent can build procedural memory before observing any target-environment tasks, using only self-generated synthetic practice. Yet, synthetic interaction alone is insufficient, as without controlling what to practice and what to store, synthetic tasks become redundant, infeasible, and ultimately uninformative, and memory further degrades quickly due to unfiltered trajectories. To overcome this, we present Preping, a proposer-guided memory construction framework. At its core is proposer memory, a structured control state that shapes future practice. A Proposer generates synthetic tasks conditioned on this state, a Solver executes them, and a Validator determines which trajectories are eligible for memory insertion while also providing feedback to guide future proposals. Experiments on AppWorld, BFCL v3, and MCP-Universe show that Preping substantially improves over a no-memory baseline and achieves performance competitive with strong playbook-based methods built from offline or online experience, with deployment cost $2.99\times$ lower on AppWorld and $2.23\times$ lower on BFCL v3 than online memory construction. Further analyses reveal that the main benefit does not come from synthetic volume alone, but from proposer-side control over feasibility, redundancy, and coverage, combined with selective memory updates.

1 Introduction

PREPING addresses the cold-start gap by constructing procedural memory before deployment, without target-environment task experience. It controls both which synthetic tasks to practice and which resulting trajectories to store.

  • Existing offline and online memory construction methods rely on target-environment task experience, creating a gap for newly connected environments.Offline methods require costly human-designed or solved tasks, while online methods begin with empty memory and expose users to early failures and update costs.
  • Pre-task memory construction builds reusable procedural memory before any target-environment tasks, using documentation, tool execution, and observed feedback instead.The setting excludes human-provided tasks, demonstrations, solved trajectories, and deployment-time user interactions.
  • Naive synthetic practice can produce redundant, infeasible, or poorly grounded tasks whose trajectories contaminate memory.The challenge is jointly shaping what to practice and what to store.
  • PREPING uses proposer-guided synthetic practice and validation-gated memory admission to build procedural memory before deployment.A Proposer generates tasks from proposer memory, a Solver executes them, and a Validator filters trajectories before solver-memory insertion.
  • PREPING improves over no-memory baselines by 17.1 points on AppWorld, 19.3 points on BFCL v3, and 5.4 points on MCP-Universe.It remains competitive with methods based on human-defined or deployment-time target tasks, despite requiring no target-task experience.

2 Related Work

Related work studies reusable agent memory and self-generated practice as separate directions. PREPING combines controlled self-generated practice with procedural memory construction for tool-using agents.

  • Reusable agent memory helps fixed LLM agents adapt across tasks through persistent memory, workflow knowledge, playbooks, and long-term context.These forms can be inspected, revised, and transferred across models or modules.
  • Self-generated tasks, self-play, and automatic curricula have been used to improve agent policies or model behavior without human annotations.Tool-use examples include challengers that generate executable tasks and executors optimized with evaluation feedback.

3 Method

PREPING constructs solver memory through controlled synthetic practice in a documented, executable environment. Separate proposer and solver memories guide future proposals and selectively admit validated trajectories as deployment-facing procedural guidance.

  • 3.1 Pre-Task Memory Construction: PREPING turns environment access before target-task experience into procedural memory through controlled synthetic practice.The construction procedure can inspect documentation, call tools, and observe feedback, but outputs solver memory for deployment.
  • 3.1 Pre-Task Memory Construction: The pre-task setting excludes access to the target task distribution, so the agent must create and execute its own task-level objectives.Documentation provides callable interfaces but rarely reveals useful compositions, preconditions, or failure modes.
  • 3.2 PREPING: Controlled Synthetic Practice for Pre-Task Memory Construction: PREPING separates construction-time proposer memory from deployment-facing solver memory.Proposer memory records practice history, coverage gaps, and failures, while solver memory contains reusable procedural guidance.
  • 3.2 PREPING: Controlled Synthetic Practice for Pre-Task Memory Construction: The construction loop generates tasks, executes them, validates feasibility and completeness, and updates memory.The displayed workflow includes task generation, solver execution, validation, and asymmetric memory updates.
  • 3.2 PREPING: Controlled Synthetic Practice for Pre-Task Memory Construction: All experience updates proposer memory, but only feasible task-trajectory pairs can update solver memory.Rejected tasks remain useful for shaping future practice, while infeasible trajectories are excluded from deployment memory.
  • 3.3 Proposer Memory Controls What to Practice: Proposer memory makes task generation history-aware, coverage-seeking, and grounded in the executable environment.It records prior tasks, invoked tools, validation outcomes, failure reasons, usage summaries, entities, states, preconditions, and constraints.
  • 3.2 PREPING: Controlled Synthetic Practice for Pre-Task Memory Construction: Validator gating prevents unreliable synthetic trajectories from becoming misleading procedural guidance.Validator outputs gate solver-memory insertion, inform future proposals through proposer memory, and support distillation into compact procedural bullets.

4 Experiments

PREPING is evaluated across three complementary agent benchmarks using pre-task memory construction without target-environment task data. It achieves the strongest pre-task results, improves over no memory, remains competitive with task-informed methods, reduces deployment-time cost, and mitigates early online cold start.

  • Experimental setup: PREPING is evaluated on AppWorld, BFCL v3, and MCP-Universe, spanning stateful application workflows, structured function calling, and MCP-server tool use.
  • Pre-task methods: PREPING constructs memory from proposer-guided synthetic tasks and admits only validator-approved task-trajectory pairs into solver memory.
  • Main results: PREPING achieves the strongest pre-task performance across all three benchmarks, improving average score over Base by 17.1 points on AppWorld, 19.3 points on BFCL v3, and 5.4 points on MCP-Universe.
  • Main results: PREPING remains competitive with task-informed methods despite using no human-defined or deployment-time target tasks for memory construction.
  • Ablation analysis: Validation-gated admission improves AppWorld performance from 47.8/26.8 to 78.2/60.7 and BFCL v3 average performance from 59.5 to 62.0.
  • Ablation analysis: Practice history expands tool support, while environmental information anchors proposals to observed entities, states, and constraints; combining both yields the best downstream performance.
  • Construction budget: With 30 synthetic tasks, PREPING reaches 76.6; with 50, it reaches 80.0, approaching the ACE-Online reference at 80.6.
  • Deployment cost: ACE-Online costs about 2.99× more on AppWorld and 2.23× more on BFCL v3 because PREPING performs memory construction before deployment.

5 Conclusion

PREPING constructs deployment-ready procedural memory through controlled synthetic practice before target-task experience is available. Its appendix documents the algorithm, prompts, baselines, and implementation details used to reproduce the framework and comparisons.

  • Conclusion: PREPING converts environment access into solver memory through proposer-guided task generation, solver execution, and validator-gated memory updates.The algorithm initializes proposer and solver memory, iterates over generated task batches, executes tasks, updates task history and grounded environment information, and returns solver memory.
  • Conclusion: The implementation includes separate prompts for task generation, validation, environment summarization, trajectory reflection, curation, and downstream task solving.These prompts support both PREPING’s construction process and the solver’s later use of memory across benchmark environments.
  • Conclusion: The appendix records benchmark-specific task-solving prompts and shared execution rules for AppWorld, BFCL, and MCP-Universe.The task-solving prompts use task-specific fields, tool descriptions, and solver memory; AppWorld prompts also include shared execution rules.
  • Conclusion: Baseline comparisons include documentation-only direct memory, random exploration, and guided exploration under matched construction budgets.The exploration baselines differ in how trajectories are generated while using the same downstream reflector-curator memory-induction pipeline.

A.6 Compute Resources

Experiments use API-hosted LLM inference with CPU workers and parallel API calls rather than local GPU training. The exploration baselines use distinct random or coverage-oriented instructions for AppWorld.

  • Compute Resources: All experiments use API-hosted LLM inference, with agent execution and memory construction running on CPU workers through parallel API calls.Compute cost is reported in token cost rather than GPU-hours, while wall-clock time depends on provider latency and worker parallelism.
  • Compute Resources: AppWorld Random Exploration instructs agents to test APIs across apps with varied parameters and observe outputs, errors, and edge cases.The prompt uses a Python REPL and asks the agent to complete the task after exploratory interaction.
  • Compute Resources: AppWorld Guided Exploration prioritizes maximizing coverage by visiting new, unexplored APIs.Its execution rules recommend inspecting available apps and API documentation, using small code chunks, verifying behavior, and handling pagination.

B.1 Full Main Results with Standard Deviations

The appendix reports full main benchmark results with standard deviations for AppWorld, BFCL v3, and MCP-Universe. These results provide uncertainty estimates over three independent runs.

  • B.1 Full Main Results with Standard Deviations: Tables 6 to 8 report the full main results with standard deviations over three independent runs.The tables correspond to AppWorld, BFCL v3, and MCP-Universe, respectively.

B.2 Iteration Dynamics of Component Ablations

The appendix analyzes how component-ablation variants evolve during ten synthetic-task construction iterations. It also examines how validator signals affect solver-memory and proposer-memory updates.

  • B.2 Iteration Dynamics of Component Ablations: Figs. 16 and 17 show component-ablation trajectories across ten synthetic-task construction iterations.Curves average three independent runs, shaded bands show standard deviation, and Naive is omitted from invalid-task panels because it lacks validator labels.
  • B.2 Iteration Dynamics of Component Ablations: Validator signals label task completion for solver-memory updates, helping distinguish successful trajectories from incomplete or failed executions.This signal is passed into the reflector-curator pipeline when extracting reusable procedural insights.
  • B.2 Iteration Dynamics of Component Ablations: Validator-derived signals also guide proposer-memory updates by informing future practice about prior task outcomes and environment coverage.The supplied passage indicates that proposer-side signals include success, failure, and related practice-control information.

B.4 Qualitative Example of AppWorld Proposer Memory

The qualitative example shows proposer memory combining environment information with prior task history to steer the next synthetic task toward underexplored Gmail practice, which the Validator accepts as successful.

  • Proposer memory combines environment information with prior task history before generating the next synthetic task.The stored state includes recently overused apps and APIs alongside solved, failed, and infeasible task records.
  • The example redirects practice toward Gmail, whose available APIs support authenticated inbox inspection and attachment-based filtering.The environment description includes Gmail login requirements, stored credentials, and paginated inbox-thread filtering by attachment presence.
  • The proposer-memory record also distinguishes successful, failed, and infeasible practice, including an alarm task with an unverified date and a nonexistent-song playlist task.These examples illustrate how task outcomes and failure reasons become part of the construction history.
  • A Gmail task asking for the number of inbox threads with attachments is judged fully feasible because the required app, login, and filtering API exist.The feasibility assessment assigns feasibility_score 5 and explains that all required entities and capabilities are available.
  • The task is completed correctly with a single-number answer of 70 email threads containing attachments.The completion reason states that the agent counted the matching threads and satisfied the requested response format.

B.5 Qualitative Example of Infeasible Tasks and Memory Contamination

The example demonstrates how an expired named payment card can produce either a constrained, identity-preserving failure or contamination when memory permits arbitrary substitution of another card.

  • The contaminated ablation deletes the invalid card, renames a valid MasterCard as Wells Fargo, and completes the transfer.The sequence updates the alternative card and reports successful withdrawal of $100.
  • This arbitrary substitution lets memory override the task’s fixed resource identity instead of treating the task constraint as infeasible.The distilled rule recommends replacing the named resource after the original becomes invalid, which is identified as the contamination failure.
  • The requested Wells Fargo card is expired, causing the direct $100 withdrawal attempt to fail.The payment-card listing identifies card 252 as Wells Fargo, and the withdrawal returns a 422 expired-card error.
  • The successful withdrawal uses payment card 253 after it has been relabeled Wells Fargo, despite the original request referring to a different card.The execution output confirms a $100 withdrawal and transfer after the substitution.

B.6 Trajectory Steps in Synthetic Practice and Online Evaluation

PREPING’s broad practice coverage is attributed to targeted proposer-memory control rather than collecting more interaction steps, while reusable memory lowers construction-inclusive cost relative to online construction.

  • PREPING’s synthetic-task trajectories require substantially fewer steps than benchmark-task trajectories across AppWorld and BFCL v3.The comparison uses the average trajectory-step counts reported for the two task distributions.
  • Targeted practice shaped by proposer memory, rather than greater interaction volume, accounts for PREPING’s broad coverage.The paper explicitly rejects collecting more interaction steps as the explanation for the coverage shown in its analysis.
  • A single AppWorld memory is reused across 585 evaluation tasks, illustrating the reuse advantage of pre-deployment construction.The reported total includes 168 Test-Normal and 417 Test-Challenge tasks.
  • Including construction cost, PREPING is 2.83× cheaper than ACE-Online on AppWorld and 1.97× cheaper on BFCL v3.The comparison includes generation, validation, synthetic and evaluation solving, and memory-update costs.

C Limitations and Broader Impact

PREPING depends on sufficiently detailed tool documentation, and deployments should add safeguards because the mechanism can increase pre-deployment capability in sensitive workflows.

  • Limitations: PREPING assumes sufficiently detailed API or tool documentation to ground synthetic tasks before user tasks are observed.The assumption concerns tool semantics, preconditions, and state constraints needed by the proposer.
  • Limitations: Poorly documented environments may limit PREPING’s applicability, although structured tool descriptions make the assumption realistic for many MCP-based settings.The paper identifies noisier documentation as a direction for future work.
  • Broader Impact: Because PREPING can increase agent capability before deployment, the paper recommends sandboxing, least-privilege permissions, audit logs, and task-specific safety checks.The broader-impact guidance is especially relevant to workflows involving external services or sensitive data.
  • Broader Impact: The paper advises against constructing memory directly from private or high-stakes environments.
Loading 2605.13880v1…