Source-linked AI summary
From Memory to Guide: Spatio-Temporal Composer for Procedural Coding Memory
Zhixuan Tan, Pengjie Gu, Zhao Li, Yihan Hu, Xu He, Dong Li, Jianye Hao
TL;DR
Memory-augmented agents use retrieved skills as text, but that leaves a gap between stored procedures and adapting them to changing execution contexts. From Memory to Guide compiles retrieved knowledge into localized, bounded Runtime Guides; on EngramBench, Composer outperformed the static-Skills baseline across evaluated tasks, with a 7.2-point pass-rate gain and lower token use on the most complex subset.
Problem
Static prompt injection requires the base model to interpret and adapt retrieved knowledge at each step, frequently leading to context mismatches and negative transfer in long-horizon tasks.
Source: §1 · p. 1, §3.1 · p. 5Method
Composer adapts retrieved skills to environmental constraints and execution stages, compiling them into bounded Runtime Guides without updating model parameters.
Source: §1 · p. 3, §7 · p. 12Results
7.2 percentage points higher aggregate confirmed-pass rate and 32.2% lower main-agent token usage on the most complex subset; Composer outperformed the native Codex + Skills baseline across evaluated tasks.
Source: §1 · p. 3, §7 · p. 12Takeaways & Limitations
Memory preserves reusable policy knowledge, while Guide gives it an actionable form and stage-specific scope.
Source: §7 · p. 12
Abstract
from arXiv · showhide
Memory-augmented agents typically integrate procedural knowledge by injecting retrieved skills directly into text prompts. This approach dangerously equates readable text with reliable execution. To bridge this gap, we introduce From Memory to Guide, a novel paradigm that transitions procedural memory from passive text delivery to active, inference-time policy adaptation. We instantiate this paradigm through the Spatio-Temporal Composer, an active policy compiler that explicitly manages exactly how and when retrieved knowledge should be applied. Rather than treating skills as plug-and-play modules, Composer dynamically aligns historical knowledge with current environmental constraints (spatial adaptation) and precisely dictates its applicable lifecycle (temporal orchestration). It actively transforms static memories into strictly bounded Runtime Guides---equipping the agent with localized objectives and behavioral guardrails without requiring a single parameter update. Extensive evaluations on 13 demanding, long-horizon software engineering tasks in EngramBench demonstrate the clear advantages of this architecture. Composer not only robustly prevents context mismatch but drives an absolute pass-rate increase of 7.2 percentage points on the most complex tasks, while simultaneously slashing the main agent's token usage by 32.2%.
1 Introduction
Procedural memory lets agents reuse accumulated skills, but static prompt injection leaves them to adapt knowledge from source contexts to changing execution conditions. From Memory to Guide treats memory as an adaptable policy resource, compiling it into bounded, stage-specific guidance; Composer improves results on demanding EngramBench tasks while using fewer tokens.
1 Introduction
- Procedural memory supports agent learning by accumulating reusable step-by-step procedures, decision rules, and debugging experience without retraining.The paper positions reusable skills as infrastructure for applying hands-on experience to lengthy tasks such as software engineering.Source: §1 · p. 1
- Static prompt injection treats retrieved skills as plug-and-play, although source-context assumptions can clash with the target workspace or execution phase.The introduction illustrates spatial mismatch with npm versus yarn and temporal mismatch with an architectural skill applied during a bug fix.Source: §1 · p. 2
- 7.2 percentage points higher aggregate confirmed-pass rate and 32.2% lower main-agent token usage on the most complex tasks; Composer also outperformed native Codex with static Skills across all tasks.Ablations report pass-rate gains of 7.03 points from spatial execution evidence and 10.09 points from temporal lifecycle boundaries.Source: §1 · p. 3
- From Memory to Guide separates persistent procedural knowledge from stage-specific action guidance, treating stored skills as adaptable policies rather than static text.This reframes experience reuse as adapting policies to project configurations and a changing execution lifecycle.Source: §1 · pp. 2–3
- Composer adapts retrieved skills to workspace constraints and execution phases, then compiles them into Runtime Guides with local goals and behavioral boundaries.The guidance is bounded by when it applies and when it expires, enabling inference-time policy adaptation without parameter updates.Source: §1 · p. 3
2 Related Work
Related work develops guidance for frozen executors, experience selection and organization, and runtime skill interfaces. Composer’s distinct focus is explicitly transforming selected procedural knowledge into local action guidance with a defined lifecycle.
Advising frozen executors
- Unlike trainable advisors optimized from executor outcomes, Composer organizes persistent procedural knowledge into stage-specific guidance for frozen executors.How to Train Your Advisor uses GRPO and supports executor-initiated advice requests through tool calls.Source: §2 · pp. 3–4
Utility-optimized experience generation
- Composer retains an explicit Skill Bank and instantiates selected knowledge as a locally scoped Runtime Guide, unlike SEAM, which internalizes experience in generator parameters.SEAM generates structured experience and optimizes guidance through executor rollouts and GRPO, with optional deployment-time supervised fine-tuning.Source: §2 · p. 4
Procedural memory and runtime policy adaptation
- Composer makes explicit how selected policy knowledge becomes state-conditioned guidance and when its local guiding role ends, extending experience abstraction and reuse.ExpeL extracts reusable insights, while Agent Workflow Memory induces workflows for subsequent interactions.Source: §2 · p. 4
Runtime skill interfaces and stage-aware execution
- Composer focuses on state-conditioned natural-language policies with local goals, behavioral boundaries, and guidance lifecycles, alongside work on runtime skill interfaces and stage-aware execution.The passage contrasts Composer with SkVM, SkillSmith, and Hierarchical Memory Tree.Source: §2 · p. 4
Learning experience utility and evolving contexts
- Where memory-utility methods select or evolve experience, Composer makes the subsequent transformation into local, time-bounded action guidance explicit.The passage names Memento, MemRL, and Agentic Context Engineering as approaches to experience selection, utility, or organization.Source: §2 · p. 4
3 From Memory to Guide: The Spatio-Temporal Composer
The Spatio-Temporal Composer reframes procedural memory as an adaptable policy resource, separating stored experience from stage-specific guidance. It selects and composes skills using current requirements and execution context, then bounds each Runtime Guide to a local goal and lifecycle.
3.1 Problem Formulation: Rethinking Procedural Memory
- Native Skill retrieval leaves the base policy to interpret and adapt retrieved knowledge at every step, risking context mismatches and negative transfer in long-horizon tasks.Composer instead separates runtime adaptation from execution, matching experience to project constraints and managing its lifecycle over the task timeline.Source: §3.1 · p. 5
3.2 Spatio-Temporal Composer Architecture
- Composer exposes reusable skills through a Catalog and synthesizes task requirements, execution evidence, and the previous Guide into runtime guidance.The Coding Agent requests guidance based on its local goal, progress, and blockers.Source: §3.2 · p. 5, Figure 1 · p. 6
3.3 Context-Conditioned Skill Selection
- The Selector decides whether to generate a Guide, identifies relevant Skills, and records why they are needed now relative to the previous Guide.Separating skill references from the rationale preserves the distinction between which experience to use and why it applies to the current request.Source: §3.3 · p. 6
3.4 Spatial Composition and Temporal Adaptation
- Spatial composition turns general rules into situated recommendations around a local goal while enforcing behavioral boundaries from the task profile.In DockChain, a Port Call Guide applies consistency and idempotency skills to a specific PostgreSQL transaction and interface semantics.Source: §3.4 · p. 7
- Temporal adaptation gives each Guide applicability prerequisites and termination conditions, prompting requirement re-entry and a new guidance request when local work is complete or circumstances change.In DockChain, completing and validating the Port Call implementation ends its Guide before the agent identifies the next local goal.Source: §3.4 · p. 7, Figure 2 · p. 7
4 Experimental Setup
The evaluation tests runtime policy adaptation and its components across EngramBench transfer tasks, using a fixed Skill Bank and controlled comparisons. It reports task quality, execution-side token use, and development time.
4 Experimental Setup
- The evaluation tests whether runtime policy adaptation improves fixed-Skill-Bank use in long-horizon software engineering and examines contextualized composition, dynamic evidence, and guidance lifecycles.It uses all 13 EngramBench transfer tasks, three runs per task and condition, the same execution model, and 18 reusable Skills held fixed during execution.Source: §4 · p. 7, §4.1 · p. 8
4.2 Compared Methods
- The main comparison pairs native Codex with static Skills against Composer’s runtime selection and composition of state-conditioned Runtime Guides.Selector-only and Generic Advisor isolate contextual composition and Skill-grounded guidance; design ablations test dynamic evidence and explicit Guide lifecycles on four tasks.Source: §4.2 · pp. 8–9
4.3 Evaluation Metrics
- Task quality is the mean confirmed-pass rate across three runs, with excluded and diagnostic executions retained in the scheduled-test denominator.Aggregate rates pool counts across the stated tasks and runs; relative count improvements are distinguished from percentage-point changes in rate.Source: §4.3 · p. 9
- Execution-side token efficiency measures the main Coding Agent’s cumulative input and output tokens, while development time includes tool execution and Guide waiting but excludes final hidden-test evaluation.Cumulative development time sums individual run durations and characterizes the execution horizon independently of experiment parallelism.Source: §4.3 · p. 9
5 Results and Analysis
Composer improves confirmed-pass rates while reducing execution-side token use on long-horizon EngramBench tasks. Ablations compare its components, and an execution example shows how a Guide applies to a local objective and stage transition.
5.1 Task Quality and Execution-Side Efficiency
- Composer consistently outperforms native Codex + Skills across all 13 tasks; the main comparison covers 78 development runs and more than 335 run-hours.Runs averaged over 4.3 hours, with the longest exceeding 12 hours.Source: §5.1 · p. 9
5.2 Beyond Skill Selection and Generic Advice
- Using the same Skill Bank, Composer records 229 cumulative confirmed passes versus 173 for native Codex + Skills, 149 for Selector-only, and 157 for Generic Advisor.Composer outperforms all three component-comparison baselines on each of the four tasks.Source: §5.2 · pp. 9–10
5.1 Task Quality and Execution-Side Efficiency
- On the four-task subset averaging 8.48 hours per run, Composer raises confirmed-pass rate from 29.9% to 37.1% (+7.2 percentage points) and cuts main-agent tokens by 32.2%.The nine remaining tasks show 458 to 563 cumulative confirmed passes (+22.9%) and a 6.86% decrease in main-agent token usage.Source: §5.1 · p. 9, Figure 3 · p. 10
5.3 Contributions of Execution Evidence and Lifecycle
- Dynamic execution evidence raises cumulative confirmed passes from 190 to 229, while explicit lifecycle specifications raise them from 173 to 229.The execution-evidence gain occurs on three of four tasks; the lifecycle gain has higher counts on all four and adds value beyond state-conditioned advice.Source: §5.3 · pp. 10–11, Table 1 · p. 11
5.4 Guides in Action
- In the DockChain example, the Port Call Guide binds transaction consistency, idempotency, and interface compatibility to the local objective before implementation and validation.After reporting local results, the agent rereads requirements and requests guidance for clearance and lease handling; reusable requirements persist across stages.Source: §5.4 · p. 11
6 Toward Self-Evolving Procedural Policies
The paper proposes using task outcomes and text feedback to improve Composer over time. This would support a recursive loop in which the system generates guides, tests them, and updates Composer.
Toward Self-Evolving Procedural Policies
- Runtime Guides’ explicit goals, boundaries, and stopping points could support side-by-side tests of different ways to apply the same skill.The authors identify this as a future use of the Guides’ well-defined steps.Source: §6 · p. 11
- Task outcomes and text feedback could be converted into training data to fine-tune Composer while the Skill Bank and main agent remain fixed.The proposal draws on memory credit assignment and text-based feedback.Source: §6 · p. 11
- A proposed recursive self-improvement loop would generate a Guide, test it in the environment, and use feedback to update Composer.The system could then adjust guide strictness or detail as the main agent’s abilities improve.Source: §6 · p. 11
7 Conclusion
The conclusion presents Composer as a way to separate persistent procedural knowledge from context-specific guidance. Its evaluation reports gains across EngramBench tasks, with ablations supporting the roles of spatial grounding and temporal lifecycles.
Conclusion
- Composer aligns past experience with environmental constraints and specifies when guidance applies, compiling skills into bounded Runtime Guides without updating execution-model parameters.The paper frames this as inference-time policy adaptation that separates persistent procedural knowledge from context-specific action guidance.Source: §7 · p. 12
- On the most complex EngramBench subset, Composer increases aggregate confirmed-pass rate by 7.2 percentage points and cuts main-agent token usage by 32.2%.It consistently outperforms native Codex + Skills across all evaluated tasks; ablations support the importance of spatial grounding and temporal lifecycles.Source: §7 · p. 12
- Memory retains reusable policy knowledge, while Guide gives it an actionable form and stage-specific scope.The conclusion characterizes this shift as moving from passive experience delivery to active runtime policy adaptation.Source: §7 · p. 12